CN110333824A - A kind of expansion method and device of storage system - Google Patents
A kind of expansion method and device of storage system Download PDFInfo
- Publication number
- CN110333824A CN110333824A CN201910494118.7A CN201910494118A CN110333824A CN 110333824 A CN110333824 A CN 110333824A CN 201910494118 A CN201910494118 A CN 201910494118A CN 110333824 A CN110333824 A CN 110333824A
- Authority
- CN
- China
- Prior art keywords
- node
- data
- expanded
- list
- storage system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
- G06F3/0623—Securing storage systems in relation to content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of expansion method of storage system and devices, which comprises obtains present node list, the present node list is the list of the current node for carrying out reading and writing data；Node to be expanded is obtained, the node to be expanded is the mobile transient node of newly-increased carry out data；The data distribution of the present node list and the node to be expanded is calculated, to obtain the prediction result of the data storage of each node；According to the prediction result, the difference data in the present node list is obtained；Difference data in the present node list is moved in the node to be expanded；The data of the node to be expanded and the data of the present node list are synchronized, to obtain data synchronization result；According to the data synchronization result, increase the node to be expanded within the storage system, to carry out dilatation to storage system.The data balancing speed of the method is fast, data reliability it is high and on external service feature without influence.
Description
Technical field
The present invention relates to storage system dilatation field more particularly to the expansion methods and device of a kind of storage system.
Background technique
When heritage storage system dilatation, new storage service node is added to storage system, according in storage system one
Determine algorithm and carry out data balancing, Xiang Xinzeng node moves data.Storage system is had been added to since this increases memory node newly, in number
Data storage service is provided according in moving process while to external client traffic.
Therefore during carrying out data balancing, data are moved to newly-increased memory node inside storage system, this is newly-increased
Memory node as data receiver, occupy a large amount of hard-disc storages, network bandwidth and the computing resource of the memory node
Deng especially hard disk write-in data are maximum bottlenecks, and eventually leading to the node reduces client traffic service ability, and dilatation
The journey duration is long, becomes the traffic bottleneck node in cluster.
In order to keep service feature in dilation process not reduce, limitation is generallyd use to dilatation node and moves data speed
The influence to business is reduced, but will lead to the growth of dilatation time in this way, hardware event occurs for storage system during data balancing
The probability of barrier increases, to increase data reliability risk.
Summary of the invention
In order to which service feature when solving the problems, such as storage system dilatation or dilatation speed are affected, storage system expansion is obtained
Outward service normally comes and goes when appearance and dilatation speed is without reduced technical effect, and the present invention provides a kind of expansions of storage system
Hold method and apparatus.
On the one hand, the present invention provides a kind of expansion methods of storage system, which comprises
Present node list is obtained, the present node list is the list of the current node for carrying out reading and writing data；
Node to be expanded is obtained, the node to be expanded is the mobile transient node of newly-increased carry out data；
The data distribution of the present node list and the node to be expanded is calculated, is deposited with obtaining the data of each node
The prediction result of storage；
According to the prediction result, the difference data in the present node list is obtained；
Difference data in the present node list is moved in the node to be expanded；
The data of the node to be expanded and the data of the present node list are synchronized, to obtain data synchronization result；
According to the data synchronization result, increase the node to be expanded within the storage system, to carry out to storage system
Dilatation.
On the other hand a kind of flash chamber of storage system is provided, described device includes: that present node list obtains mould
Block, node to be expanded obtain module, distribution results computing module, difference data mobile module, data simultaneous module and system and expand
Molar block；
It is when advance that the present node list, which obtains module for obtaining present node list, the present node list,
The node listing of row reading and writing data；
The node to be expanded obtains module for obtaining node to be expanded, the node to be expanded be increase newly count
According to mobile transient node；
The distribution results computing module is used to calculate the data point of the present node list and the node to be expanded
Cloth, to obtain the prediction result of the data distribution of each node；
The difference data mobile module is used for according to the prediction result, the difference data in mobile present node list
To node to be expanded；
The data simultaneous module is used to synchronize the data of the node to be expanded and the data of the present node list,
To obtain data synchronization result；
The System Expansion module is used for according to the data synchronization result, and the node to be expanded is added to storage system
In system, to carry out dilatation to storage system.
On the other hand a kind of computer readable storage medium is provided, for storing program, described program is performed reality
A kind of expansion method of existing storage system.
On the other hand a kind of terminal device is provided, the terminal device includes a kind of dilatation dress of above-mentioned storage system
It sets.
The expansion method and device of a kind of storage system provided by the invention, the method are carrying out dilatation to storage system
When, it needs temporarily to increase node to be expanded, after stored data are redistributed, movable part data to section to be expanded
Point finally does increment synchronization between the memory node having had in node to be expanded and storage system, and difference after synchronization
When heteromerism is according to preset threshold is less than, the node to be expanded is become a full member of in storage system.The data balancing of the method
Speed is fast, data reliability it is high and on external service feature without influence.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is a kind of application scenarios schematic diagram of the expansion method of storage system provided in an embodiment of the present invention；
Fig. 2 is a kind of flow chart of the expansion method of storage system provided in an embodiment of the present invention；
Fig. 3 is the data storage that each node is obtained in a kind of expansion method of storage system provided in an embodiment of the present invention
Prediction result method flow chart；
Fig. 4 is in a kind of expansion method of storage system provided in an embodiment of the present invention from the method for noticing that layer model executes
Flow chart；
Fig. 5 is that difference data is moved in a kind of expansion method of storage system provided in an embodiment of the present invention to described wait expand
Hold the flow chart of the method in node；
Fig. 6 is that obtain data by increment synchronization in a kind of expansion method of storage system provided in an embodiment of the present invention same
Walk the flow chart of the method for result；
Fig. 7 is stream when a kind of expansion method of storage system provided in an embodiment of the present invention is applied to storage system dilatation
Cheng Tu；
Fig. 8 is a kind of structural schematic diagram of the flash chamber of storage system provided in an embodiment of the present invention；
Fig. 9 is the knot of the distribution results computing module in a kind of flash chamber of storage system provided in an embodiment of the present invention
Structure schematic diagram；
Figure 10 is the difference data mobile module in a kind of flash chamber of storage system provided in an embodiment of the present invention
Structural schematic diagram；
Figure 11 is the structure of the data simultaneous module in a kind of flash chamber of storage system provided in an embodiment of the present invention
Schematic diagram；
Figure 12 is a kind of equipment for realizing method provided by the embodiment of the present invention provided in an embodiment of the present invention
Hardware structural diagram.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, the present invention is made into one below in conjunction with attached drawing
Step ground detailed description.Obviously, described embodiment is only a part of the embodiments of the present invention, rather than whole implementation
Example.Based on the embodiments of the present invention, those of ordinary skill in the art are obtained without making creative work
Every other embodiment, shall fall within the protection scope of the present invention.
In the description of the present invention, it is to be understood that, term " first ", " second " are used for description purposes only, and cannot
It is interpreted as indication or suggestion relative importance or implicitly indicates the quantity of indicated technical characteristic.Define as a result, " the
One ", the feature of " second " can explicitly or implicitly include one or more of the features.Moreover, term " first ",
" second " etc. is suitable for distinguishing similar object, without being used to describe a particular order or precedence order.It should be understood that in this way
The data used are interchangeable under appropriate circumstances, so that the embodiment of the present invention described herein can be in addition to scheming herein
Sequence other than those of showing or describe is implemented.
Referring to Figure 1, which show a kind of application scenarios of the expansion method of storage system provided in an embodiment of the present invention
Schematic diagram, the application scenarios include user terminal 110, server 120 and node to be expanded 130, and the user terminal includes
But be not limited to smart phone, tablet computer, laptop, desktop computer etc..The server is written from user terminal
The data information of server, data information required for the user terminal is read by server.When the server carries out
When dilatation operates, temporarily increase node to be expanded, it is described data portion stored in server is transferred to it is described wait expand
Hold in node, while the node to be expanded is not open to user terminal, user terminal, which only passes through in access server, has section
Point is to carry out reading and writing for data.After the completion of the data are mobile, the node to be expanded is become a full member of into service
In device, to carry out dilatation to the server.
Fig. 2 is referred to, which show a kind of expansion methods of storage system, can be applied to server side, the method packet
It includes:
S210. present node list is obtained, the present node list is the list of the current node for carrying out reading and writing data.
S220. node to be expanded is obtained, the node to be expanded is the mobile transient node of newly-increased carry out data.
Specifically, the node to be expanded is not involved in the service for externally carrying out reading data or write-in, described to be expanded
Node only carries out the mobile operation of the data inside storage system, and after the completion of data are mobile, the node to be expanded formally adds
Enter into storage system.
The node to be expanded is deposited during subsequent progress data are mobile since node to be expanded is not added really
Storage system, so hardware fault etc. is abnormal, and the data that will not reduce former storage system can if the node to be expanded occurs
By property.
S230. the data distribution of the present node list and the node to be expanded is calculated, to obtain each node
The prediction result of data storage.
Further, the data distribution for calculating the present node list and the node to be expanded, it is every to obtain
A node data storage prediction result include:
S310. the node to be expanded is increased in the present node list, to obtain newly-increased node listing.
S320. according to preset algorithm, data distribution calculating is carried out to the newly-increased node listing, to obtain the newly-increased section
The prediction result of each node data storage in point list.
It specifically, can be by original present node list and wait expand after node to be expanded being temporarily added in storage system
Hold node one newly-increased node listing of composition to be calculated according to the number of members in newly-increased node listing by preset data distribution
Method carries out data distribution calculating, to obtain the prediction result of each node storing data in newly-increased node listing, i.e., is in storage
After node to be expanded is temporarily added in system, data stored in storage system are allocated again.For example, current storage system
In there are tri- memory nodes of A, B and C, 4 data are stored in each memory node, when carrying out the dilatation of storage system, are faced
When node D to be expanded is added, then 12 data in former storage system are re-assigned in tetra- nodes of A, B, C and D,
The prediction result of obtained data storage is that each node stores 3 data.
By carrying out data distribution prediction, the data for needing to store in each node can be obtained, convenient for again to each
The memory space of node is allocated.
S240. according to the prediction result, the difference data in the present node list is obtained；
Specifically, described according to the prediction result, the difference data obtained in the present node list includes:
The prediction result is compared with the storing data in the present node list, obtains present node column
Difference data in table, the difference data store for the storing data of each node in present node list with data pre-
Survey the difference data between result.Such as there are tri- memory nodes of A, B and C in current storage system, in each memory node
4 data are stored, node D to be expanded, after data distribution algorithms are again distributed data, each memory node is added
The data number of middle storage is 3 data, in 4 data and prediction result that tri- memory nodes of A, B and C store originally at this time
It there is a difference data between 3 data of storage.The difference data generated in the present node list of storage system,
It videlicet will be moved into the data in node to be expanded.
S250. the difference data in the present node list is moved in the node to be expanded.
Further, the difference data by the present node list, which is moved in the node to be expanded, wraps
It includes:
S410. according to the prediction result and the difference data, the task list of the node to be expanded is generated, it is described
Task list is task list of the present node list distribution data to node to be expanded.
S420. according to the task list, the difference data in the present node list is concurrently moved to described wait expand
Hold in node.
Specifically, task list is to need to shift which data in present node list into node to be expanded for listing
Task list.Such as there are tri- memory nodes of A, B and C in current storage system, 4 numbers are stored in each memory node
According to storing this four data of x1, x2, x3 and x4 in the A memory node, store x5, x6, x7 in the B memory node
With this four data of x8, this four data of x9, x10, x11 and x12 are stored in the C memory node.It is calculated by data distribution
After method is again distributed data, the difference data x8 and C in difference data x4, the B memory node in A memory node are deposited
Difference data x12 in storage node is moved in node D to be expanded, then task list just includes that x4 is moved to section to be expanded
X8 is moved to node D to be expanded and x12 is moved to these three tasks of node D to be expanded by point D.
According to being shown in task list for task, difference data can be moved in node to be expanded.Carrying out data
It, can utmostly simultaneously due to the business that node to be expanded carries out other than mobile data without other needs when mobile
Hair executes the task of difference data duplication, it is possible to reduce data replicate duration, you can faster complete data balancing.
S260. the data of the node to be expanded and the data of the present node list are synchronized, it is synchronous to obtain data
As a result；
Further, the data of the data for synchronizing the node to be expanded and the present node list, to obtain
Data synchronization result includes:
S510. the variance data between the node to be expanded and the present node list is obtained.
S520. increment synchronization is carried out to the variance data between the node to be expanded and the present node list, with
Obtain data synchronization result.
Specifically, the memory node when node to be expanded carries out data movement, in the present node list of storage system
The contact of business can externally be carried out, it is thus possible to which there are reading data or the operations of write-in, so that in present node list
Memory node in data change.Such as 6:00 starts to node motion data to be expanded in the morning, to 9:00 in morning
It is mobile to terminate data, during which storage system externally may exist business contact, the method that can use increment synchronization at this time, selection
Data move the time point started, i.e. the data come and gone in storage system after 6:00 in morning, carry out with node to be expanded same
Step, to reduce the variance data between node and present node list to be expanded.
Node to be expanded complete data movement after, it is also necessary to by node to be expanded with it is each in present node list
Node carries out increment synchronization, so that the data balancing in each node.
Carrying out increment synchronization makes the variance data of final node to be expanded and each node in present node list smaller,
Therefore data can be completed in the extremely short time to synchronize, to reduce data reliability risk.
S270. according to the data synchronization result, increase the node to be expanded within the storage system, to storage system
Carry out dilatation.
Further, according to the data synchronization result, increase the node to be expanded within the storage system, to storage
System carries out dilatation
S610. the data synchronization result and preset discrepancy threshold are compared, to obtain comparing result；
If S620. the data synchronization result is less than preset discrepancy threshold, increase within the storage system described wait expand
Hold node, to carry out dilatation to storage system.
If S630. the data synchronization result is greater than preset discrepancy threshold, repeat to the node to be expanded and institute
The variance data stated between present node list carries out increment synchronization.
Specifically, when progress data increment is synchronous, data synchronization result after the completion of increment synchronization each time can be compared
With the size between preset discrepancy threshold, with obtain this carry out increment synchronization data synchronization result.The data are synchronous
As a result the data difference between node and present node list to be expanded after the completion of increment synchronization each time, if the data are same
It walks result and is greater than preset discrepancy threshold, then the process of above-mentioned increment synchronization data can be repeated, until increment variance data is small
Until preset discrepancy threshold, the preset discrepancy threshold is the 5% of usual disk size.Formally add in node to be expanded
When entering storage system, since variance data is less than preset discrepancy threshold, data can be completed in the extremely short time and synchronized, thus
Reduce data reliability risk.
Fig. 7 is referred to, Fig. 7, which is shown, utilizes general flow when the progress storage system progress dilatation of method described in the present embodiment
Journey, the process are as follows:
S710. current storage system members list is obtained；
S720. list is added in node to be expanded；
S730. newcomer's list；
S740. according to the data distribution algorithms in storage system, data distribution is calculated；
S750. the task list with data balancing for newly-increased node is generated；
S760. difference data replication task is concurrently executed；
S770. it is synchronous to carry out incremental data；
S780. judge whether variance data is less than preset threshold, if variance data is greater than preset threshold, repeat to increase
It is synchronous to measure data.
If S790. variance data is less than preset threshold, storage system formally is added in node to be expanded.
Specifically, present node list in storage system is obtained, the present node list is to be counted in storage system
According to service node members list, when storage system needs to carry out dilatation, it is added node to be expanded, and by the node to be expanded
Newly-increased node listing is merged into present node list.The node to be expanded can be multiple newly-increased nodes.It is according to storage
Data distribution algorithms in system carry out the calculating of data distribution referring to node listing is increased newly, and prediction is deposited in node to be expanded addition
After storage system, each node needs the data bulk stored.The data bulk stored is needed according to each node of prediction, is obtained
Each node needs to be moved to the difference data in node to be expanded in the present node list of storage system, generates for newly-increased
The task list of the pending data equilibrium of node.Starting difference data replication task concurrently executes, since node to be expanded is not formal
Storage system is added, the redundant data replication task during data balancing can be utmostly parallel, even if the node is complete
Portion's resource, such as hard-disc storage, network bandwidth and computing resource etc. can move task by internal data and occupy, at this time
It will not influence the performance of external business.Node to be expanded carries out data movement with all strength, can shorten the lasting duration of dilatation.It will
It is synchronous with the node progress incremental data in present node list to node to be expanded after data are moved to node to be expanded.If
After progress data increment synchronizes, the variance data in node to be expanded and present node list is less than preset discrepancy threshold, then
Node is become a full member of into storage system；
Otherwise, it is synchronous that incremental data is carried out again, until variance data is less than preset discrepancy threshold, then is formally deposited this
It stores up node and storage system is added.
When carrying out data movement due to the node to be expanded, the shifting of data can be carried out by the way of replicate data
It is dynamic, thus before node to be expanded becomes a full member of storage system, dilatation operation can be cancelled at any time, it will not be to original storage system
System has any influence.
A kind of expansion method of storage system provided in an embodiment of the present invention, the method by increasing node to be expanded newly,
Partial data is moved in node to be expanded, and temporarily storage system is not added in node to be expanded, so that node to be expanded
External business service is not undertaken, even if whole resources of the node are all occupied by internal data resettlement task, will not influence to deposit
Storage system handles the performance of external business, and it is multiple can to reduce data utmostly concurrently to execute redundant data replication task
Duration processed faster completes data balancing process.
The embodiment of the invention also provides a kind of flash chambers of storage system, refer to Fig. 8, and described device includes: to work as
Prosthomere point list obtains module 810, node to be expanded obtains module 820, distribution results computing module 830, difference data movement
Module 840, data simultaneous module 850 and System Expansion module 860；
It is to work as that the present node list, which obtains module 810 for obtaining present node list, the present node list,
The preceding node listing for carrying out reading and writing data；
The node to be expanded obtains module 820 for obtaining node to be expanded, the node to be expanded be increase newly into
The mobile transient node of row data；
The distribution results computing module 830 is used to calculate the data of the present node list and the node to be expanded
Distribution, to obtain the prediction result of the data distribution of each node；
The difference data mobile module 840 is used for according to the prediction result, the difference in mobile present node list
Data are to node to be expanded；
The data simultaneous module 850 is used to synchronize the data of the node to be expanded and the number of the present node list
According to obtain data synchronization result；
The System Expansion module 860 is used to be added to the node to be expanded according to the data synchronization result and deposit
In storage system, to carry out dilatation to storage system.
Further, Fig. 9 is referred to, the distribution results computing module 830 includes newly-increased node listing obtaining unit 910
With data distribution computing unit 920；
The newly-increased node listing obtaining unit 910 is used to the node to be expanded increasing to the present node list
In, to obtain newly-increased node listing；
The data distribution computing unit 920 is used for according to preset algorithm, carries out data point to the newly-increased node listing
Cloth calculates, to obtain the prediction result of each node data storage in the newly-increased node listing.
Further, referring to Figure 10, the difference data mobile module 840 include difference data obtaining unit 1010,
Task list generation unit 1020 and Data Movement Unit 1030；
The difference data obtaining unit 1010 is used for the prediction result and having deposited in the present node list
Storage data are compared, and obtain the difference data in present node list；
The task list generation unit 1020 is used for according to the prediction result and the difference data, generate it is described to
The task list of dilatation node, the task list are task list of the present node list distribution data to node to be expanded；
The Data Movement Unit 1030 is used to concurrently be moved in the present node list according to the task list
Difference data is into the node to be expanded.
Further, referring to Figure 11, the data simultaneous module 850 includes variance data acquiring unit 1110, increment
Synchronization unit 1120 and comparison unit 1130；
The variance data acquiring unit 1110 is for obtaining between the node to be expanded and the present node list
Variance data；
The increment synchronization unit 1120 is used for the difference between the node to be expanded and the present node list
Data carry out increment synchronization, to obtain data synchronization result；
The comparison unit 1130 is for comparing the data synchronization result and preset discrepancy threshold, to obtain
Comparing result.
Any embodiment of that present invention institute providing method can be performed in the device provided in above-described embodiment, has execution this method
Corresponding functional module and beneficial effect.The not technical detail of detailed description in the above-described embodiments, reference can be made to the present invention is any
A kind of expansion method of storage system provided by embodiment.
The present embodiment additionally provides a kind of computer readable storage medium, and computer is stored in the storage medium to be held
Row instruction, the computer executable instructions are loaded by processor and execute a kind of dilatation of the above-mentioned storage system of the present embodiment
Method.
The present embodiment additionally provides a kind of equipment, and the equipment includes processor and memory, wherein the memory is deposited
Computer program is contained, the computer program is suitable for being loaded by the processor and executing a kind of above-mentioned storage of the present embodiment
The expansion method of system.
The equipment can be terminal, mobile terminal or server, and the equipment, which may also participate in, constitutes this hair
Device or system provided by bright embodiment.As shown in figure 12, mobile terminal 12 (or terminal 12 or server 12) can
To include that one or more (using 1202a, 1202b ... ... in figures, 1202n to show) (processor 1202 can for processor 1202
To include but is not limited to the processing unit of Micro-processor MCV or programmable logic device FPGA etc.), storage for storing data
Device 1204 and transmitting device 1206 for communication function.It in addition to this, can also include: that display, input/output connect
Mouth (I/O interface), network interface, power supply and/or camera.It will appreciated by the skilled person that structure shown in Figure 12
Only illustrate, the structure of above-mentioned electronic device is not caused to limit.For example, mobile device 12 may also include than institute in Figure 12
Show more perhaps less component or with the configuration different from shown in Figure 12.
It is to be noted that said one or multiple processors 1202 and/or other data processing circuits lead to herein
Can often " data processing circuit " be referred to as.The data processing circuit all or part of can be presented as software, hardware, firmware
Or any other combination.In addition, data processing circuit for single independent processing module or all or part of can be integrated to shifting
In any one in other elements in dynamic equipment 12 (or terminal).As involved in the embodiment of the present application,
The data processing circuit controls (such as the selection for the variable resistance end path connecting with interface) as a kind of processor.
Memory 1204 can be used for storing the software program and module of application software, as described in the embodiment of the present invention
Corresponding program instruction/the data storage device of method, the software journey that processor 1202 is stored in memory 1204 by operation
Sequence and module realize that above-mentioned one kind is based on from attention network thereby executing various function application and data processing
Timing behavior capture frame generation method.Memory 1204 may include high speed random access memory, may also include non-volatile memories
Device, such as one or more magnetic storage device, flash memory or other non-volatile solid state memories.In some instances, it deposits
Reservoir 1204 can further comprise the memory remotely located relative to processor 1202, these remote memories can pass through net
Network is connected to mobile device 12.The example of above-mentioned network includes but is not limited to internet, intranet, local area network, moves and lead to
Letter net and combinations thereof.
Transmitting device 1206 is used to that data to be received or sent via a network.Above-mentioned network specific example may include
The wireless network that the communication providers of mobile terminal 12 provide.In an example, transmitting device 1206 includes that a network is suitable
Orchestration (Network Interface Controller, NIC), can be connected by base station with other network equipments so as to
Internet is communicated.In an example, transmitting device 1206 can be radio frequency (Radio Frequency, RF) module,
For wirelessly being communicated with internet.
Display can such as touch-screen type liquid crystal display (LCD), the liquid crystal display aloow user with
The user interface of mobile device 12 (or terminal) interacts.
Present description provides the method operating procedures as described in embodiment or flow chart, but based on routine or without creation
The labour of property may include more or less operating procedure.The step of enumerating in embodiment and sequence are only numerous steps
One of execution sequence mode, does not represent and unique executes sequence.System in practice or when interrupting product and executing, can be with
It is executed according to embodiment or method shown in the drawings sequence or parallel executes (such as parallel processor or multiple threads
Environment).
Structure shown in the present embodiment, only part-structure relevant to application scheme, is not constituted to this
The restriction for the equipment that application scheme is applied thereon, specific equipment may include more or fewer components than showing,
Perhaps certain components or the arrangement with different components are combined.It is to be understood that method disclosed in the present embodiment,
Device etc., may be implemented in other ways.For example, the apparatus embodiments described above are merely exemplary, for example,
The division of the module is only a kind of division of logic function, and there may be another division manner in actual implementation, such as more
A unit or assembly can be combined or can be integrated into another system, or some features can be ignored or not executed.It is another
Point, shown or discussed mutual coupling, direct-coupling or communication connection can be through some interfaces, device or
The indirect coupling or communication connection of unit module.
Based on this understanding, technical solution of the present invention substantially in other words the part that contributes to existing technology or
The all or part of person's technical solution can be embodied in the form of software products, which is stored in one
In a storage medium, including some instructions are used so that computer equipment (it can be personal computer, server, or
Network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention.And storage medium above-mentioned includes:
USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random
Access Memory), the various media that can store program code such as magnetic or disk.
Those skilled in the art further appreciate that, respectively show in conjunction with what embodiment disclosed in this specification described
Example unit and algorithm steps, being implemented in combination with electronic hardware, computer software or the two, in order to clearly demonstrate
The interchangeability of hardware and software generally describes each exemplary composition and step according to function in the above description
Suddenly.These functions are implemented in hardware or software actually, the specific application and design constraint item depending on technical solution
Part.Professional technician can use different methods to achieve the described function each specific application, but this reality
Now it should not be considered as beyond the scope of the present invention.
The above, the above embodiments are merely illustrative of the technical solutions of the present invention, rather than its limitations；Although referring to before
Stating embodiment, invention is explained in detail, those skilled in the art should understand that: it still can be to preceding
Technical solution documented by each embodiment is stated to modify or equivalent replacement of some of the technical features；And these
It modifies or replaces, the spirit and scope for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution.
Claims (10)
1. a kind of expansion method of storage system, which is characterized in that the described method includes:
Present node list is obtained, the present node list is the list of the current node for carrying out reading and writing data；
Node to be expanded is obtained, the node to be expanded is the mobile transient node of newly-increased carry out data；
Calculate the data distribution of the present node list and the node to be expanded, with obtain each node data storage
Prediction result；
According to the prediction result, the difference data in the present node list is obtained；
Difference data in the present node list is moved in the node to be expanded；
The data of the node to be expanded and the data of the present node list are synchronized, to obtain data synchronization result；
According to the data synchronization result, increase the node to be expanded within the storage system, to carry out dilatation to storage system.
2. a kind of expansion method of storage system according to claim 1, which is characterized in that work as prosthomere described in the calculating
The data distribution of point list and the node to be expanded, with obtain each node data storage prediction result include:
The node to be expanded is increased in the present node list, to obtain newly-increased node listing；According to preset algorithm,
Data distribution calculating is carried out to the newly-increased node listing, to obtain each node data storage in the newly-increased node listing
Prediction result.
3. a kind of expansion method of storage system according to claim 1, which is characterized in that described to be tied according to the prediction
Fruit, the difference data obtained in the present node list include:
The prediction result is compared with the storing data in the present node list, is obtained in present node list
Difference data, the difference data be present node list in each node storing data and data storage prediction knot
Difference data between fruit.
4. a kind of expansion method of storage system according to claim 1, which is characterized in that described by the present node
Difference data in list is moved in the node to be expanded
According to the prediction result and the difference data, the task list of the node to be expanded, the task list are generated
For the task list of present node list distribution data to node to be expanded；
According to the task list, the difference data in the present node list is concurrently moved into the node to be expanded.
5. a kind of expansion method of storage system according to claim 1, which is characterized in that it is described synchronize it is described to be expanded
The data of the data of node and the present node list include: to obtain data synchronization result
Obtain the variance data between the node to be expanded and the present node list；
Increment synchronization is carried out to the variance data between the node to be expanded and the present node list, it is same to obtain data
Walk result.
6. a kind of expansion method of storage system according to claim 1, which is characterized in that tied according to the data are synchronous
Fruit increases the node to be expanded within the storage system, includes: to carry out dilatation to storage system
The data synchronization result and preset discrepancy threshold are compared, to obtain comparing result；
If the data synchronization result is less than preset discrepancy threshold, increase the node to be expanded within the storage system, with
Dilatation is carried out to storage system.
7. a kind of expansion method of storage system according to claim 6, which is characterized in that described same according to the data
Step is as a result, increase the node to be expanded, within the storage system to carry out dilatation to storage system further include:
If the data synchronization result is greater than preset discrepancy threshold, repeat to the node to be expanded and the present node
Variance data between list carries out increment synchronization.
8. a kind of flash chamber of storage system, which is characterized in that described device include: present node list obtain module, to
Dilatation node obtains module, distribution results computing module, difference data mobile module, data simultaneous module and System Expansion mould
Block；
It is currently to be counted that the present node list, which obtains module for obtaining present node list, the present node list,
According to the node listing of read-write；
It is newly-increased carry out data shifting that the node to be expanded, which obtains module for obtaining node to be expanded, the node to be expanded,
Dynamic transient node；
The distribution results computing module is used to calculate the data distribution of the present node list and the node to be expanded, with
Obtain the prediction result of the data distribution of each node；
The difference data mobile module is used for according to the prediction result, the difference data in mobile present node list to
Dilatation node；
The data simultaneous module is for synchronizing the data of the node to be expanded and the data of the present node list, to obtain
To data synchronization result；
The System Expansion module is used to that the node to be expanded to be added to storage system according to the data synchronization result
In, to carry out dilatation to storage system.
9. a kind of flash chamber of storage system according to claim 8, which is characterized in that the mobile mould of the difference data
Block includes difference data obtaining unit, task list generation unit and Data Movement Unit；
The difference data obtaining unit be used for by the storing data in the prediction result and the present node list into
Row compares, and obtains the difference data in present node list；
The task list generation unit is used to generate the node to be expanded according to the prediction result and the difference data
Task list, the task list be present node list distribute data to node to be expanded task list；
The Data Movement Unit is used to concurrently move the difference data in the present node list according to the task list
Into the node to be expanded.
10. a kind of flash chamber of storage system according to claim 8, which is characterized in that the data simultaneous module
Including variance data acquiring unit, increment synchronization unit and comparison unit；
The variance data acquiring unit is used to obtain the difference number between the node to be expanded and the present node list
According to；
The increment synchronization unit is used to carry out the variance data between the node to be expanded and the present node list
Increment synchronization, to obtain data synchronization result；
The comparison unit is for comparing the data synchronization result and preset discrepancy threshold, to obtain comparison knot
Fruit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910494118.7A CN110333824B (en) | 2019-06-05 | 2019-06-05 | Capacity expansion method and device for storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910494118.7A CN110333824B (en) | 2019-06-05 | 2019-06-05 | Capacity expansion method and device for storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110333824A true CN110333824A (en) | 2019-10-15 |
| CN110333824B CN110333824B (en) | 2022-10-25 |
Family
ID=68140769
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910494118.7A Active CN110333824B (en) | 2019-06-05 | 2019-06-05 | Capacity expansion method and device for storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110333824B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111208952A (en) * | 2020-02-13 | 2020-05-29 | 河南文正电子数据处理有限公司 | Storage system capacity expansion method, readable storage medium and computing device |
| CN111723148A (en) * | 2020-04-10 | 2020-09-29 | 浙江大华技术股份有限公司 | Data storage method and device, storage medium and electronic device |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103377073A (en) * | 2012-04-26 | 2013-10-30 | 中国银联股份有限公司 | Data information processing device and method |
| US20140279954A1 (en) * | 2013-03-15 | 2014-09-18 | International Business Machines Corporation | Reducing digest storage consumption by tracking similarity elements in a data deduplication system |
| CN106843745A (en) * | 2015-12-03 | 2017-06-13 | 南京中兴新软件有限责任公司 | Capacity expansion method and device |
| CN106843755A (en) * | 2017-01-04 | 2017-06-13 | 北京百度网讯科技有限公司 | For the data balancing method and device of server cluster |
| WO2018076973A1 (en) * | 2016-10-28 | 2018-05-03 | 华为技术有限公司 | Load adjustment method, apparatus, and system |
| CN108132949A (en) * | 2016-12-01 | 2018-06-08 | 腾讯科技（深圳）有限公司 | The method and device of Data Migration in data-base cluster |
| CN109379158A (en) * | 2018-12-10 | 2019-02-22 | 浪潮电子信息产业股份有限公司 | Increase method for synchronizing time, device and the equipment of node in a kind of storage cluster newly |
| CN109426439A (en) * | 2017-09-04 | 2019-03-05 | 阿里巴巴集团控股有限公司 | The method and device of dilatation is carried out to distributed memory system |
| CN109683826A (en) * | 2018-12-26 | 2019-04-26 | 北京百度网讯科技有限公司 | Expansion method and device for distributed memory system |
-
2019
- 2019-06-05 CN CN201910494118.7A patent/CN110333824B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103377073A (en) * | 2012-04-26 | 2013-10-30 | 中国银联股份有限公司 | Data information processing device and method |
| US20140279954A1 (en) * | 2013-03-15 | 2014-09-18 | International Business Machines Corporation | Reducing digest storage consumption by tracking similarity elements in a data deduplication system |
| CN106843745A (en) * | 2015-12-03 | 2017-06-13 | 南京中兴新软件有限责任公司 | Capacity expansion method and device |
| WO2018076973A1 (en) * | 2016-10-28 | 2018-05-03 | 华为技术有限公司 | Load adjustment method, apparatus, and system |
| CN108132949A (en) * | 2016-12-01 | 2018-06-08 | 腾讯科技（深圳）有限公司 | The method and device of Data Migration in data-base cluster |
| CN106843755A (en) * | 2017-01-04 | 2017-06-13 | 北京百度网讯科技有限公司 | For the data balancing method and device of server cluster |
| CN109426439A (en) * | 2017-09-04 | 2019-03-05 | 阿里巴巴集团控股有限公司 | The method and device of dilatation is carried out to distributed memory system |
| CN109379158A (en) * | 2018-12-10 | 2019-02-22 | 浪潮电子信息产业股份有限公司 | Increase method for synchronizing time, device and the equipment of node in a kind of storage cluster newly |
| CN109683826A (en) * | 2018-12-26 | 2019-04-26 | 北京百度网讯科技有限公司 | Expansion method and device for distributed memory system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111208952A (en) * | 2020-02-13 | 2020-05-29 | 河南文正电子数据处理有限公司 | Storage system capacity expansion method, readable storage medium and computing device |
| CN111208952B (en) * | 2020-02-13 | 2022-07-12 | 河南文正电子数据处理有限公司 | Storage system capacity expansion method, readable storage medium and computing device |
| CN111723148A (en) * | 2020-04-10 | 2020-09-29 | 浙江大华技术股份有限公司 | Data storage method and device, storage medium and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110333824B (en) | 2022-10-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105426439B (en) | Metadata processing method and device | |
| CN109949111A (en) | Electronic bill mark distributing method, electronic bill generation method, apparatus and system | |
| CN106056529B (en) | Method and equipment for training convolutional neural network for picture recognition | |
| CN110502340A (en) | A kind of resource dynamic regulation method, device, equipment and storage medium | |
| KR20170120022A (en) | Method and apparatus for scheduling task | |
| WO2018077295A1 (en) | Data processing method and apparatus for convolutional neural network | |
| CA2896973A1 (en) | Methods, devices and systems for dynamically managing memberships in replicated state machines within a distributed computing environment | |
| US10504492B2 (en) | Apparatus and methods for generating dynamic trace data on a GPU | |
| CN110333824A (en) | A kind of expansion method and device of storage system | |
| CN103634375A (en) | Method, device and equipment for cluster node expansion | |
| CN107357688A (en) | Distributed system and its fault recovery method and device | |
| CN110334072A (en) | A kind of distributed file system, file updating method and device | |
| CN104202253A (en) | Network-on-chip congestion control method based on dynamic routing table | |
| CN109388397A (en) | Product page generation method, system, computer equipment and storage medium | |
| CN109375912B (en) | Model serialization method, device and storage medium | |
| CN110334150A (en) | Monitor method, monitor terminal and the system of master-slave database synchronizing relay | |
| CN108390914A (en) | A kind of service update method and device, system | |
| CN115048254A (en) | Simulation test method, system, equipment and readable medium of data distribution strategy | |
| Jayaram et al. | Just-in-Time Aggregation for Federated Learning | |
| CN112463340A (en) | Tensorflow-based multi-task flexible scheduling method and system | |
| CN107800501A (en) | A kind of method for synchronizing time, apparatus and system | |
| CN110162293A (en) | A kind of random number generation method, device and terminal for equally loaded | |
| CN106953933A (en) | A kind of information push method and device, electronic equipment | |
| CN115292394A (en) | Data processing method, data processing device, computer equipment and storage medium | |
| CN109002908A (en) | One kind is arranged an order according to class and grade dispatching method, system and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |