CN105892952A - Hyper-converged system and longitudinal extension method thereof - Google Patents
Hyper-converged system and longitudinal extension method thereof Download PDFInfo
- Publication number
- CN105892952A CN105892952A CN201610259698.8A CN201610259698A CN105892952A CN 105892952 A CN105892952 A CN 105892952A CN 201610259698 A CN201610259698 A CN 201610259698A CN 105892952 A CN105892952 A CN 105892952A
- Authority
- CN
- China
- Prior art keywords
- accumulation layer
- data
- storage resource
- emerging system
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0614—Improving the reliability of storage systems
- G06F3/0617—Improving the reliability of storage systems in relation to availability
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a hyper-converged system. The system comprises a first storage resource, a second storage resource for storage resource longitudinal extension and a data migration module, wherein the first storage resource and the second storage resource are used for storing data in the hyper-converged system; the data migration module is used for migrating related data fulfilling preset conditions in the first storage resource onto the second storage resource for storage, and updating the access address of the migrated related data. The invention further discloses a longitudinal extension method of the hyper-converged system. According to the longitudinal extension method, data migration processing is performed on the hyper-converged system adopting longitudinal extension, the problem that the longitudinally extended storage node of the hyper-converged system is prevented from becoming the bottleneck of storage performance is avoided, and the integral performance of the hyper-converged system can be promoted.
Description
Technical field
The present invention relates to communication technical field, particularly relate to super emerging system and Longitudinal Extension method thereof.
Background technology
Super emerging system is based on distributed structure/architecture, and its extended capability extending transversely is very strong, if past
Cluster adds new physical store node just can realize calculating resource and the extension of storage resource.Due to
The super ability extending transversely merged is very strong, thus relatively conventional super emerging system framework has bigger
Advantage, but much have purchased the client of super emerging system after the operation through half a year or a year, send out
Existing storage resource is the most much of that, but the calculating such as CPU, internal memory resource is the most sufficient, and this is accomplished by
Individually extension storage resource, namely need to carry out the Longitudinal Extension of super emerging system.
The most commercial super emerging system is after the Longitudinal Extension carrying out storage resource, due to super emerging system
In the storage resource of node that is used for doing Longitudinal Extension will be more much more than other non-longitudinal expanding nodes, because of
And make memory capacity that Longitudinal Extension node can be provided by also big than other non-longitudinal expanding nodes
Many, but owing to super emerging system can't preferentially store data into Longitudinal Extension joint based on These characteristics
In storage resource under Dian, thus this will cause Longitudinal Extension node to readily become depositing of super emerging system
Storage performance bottleneck.
Summary of the invention
Present invention is primarily targeted at offer, it is desirable to provide a kind of super emerging system and Longitudinal Extension side thereof
Method, it is intended to solve the technology of storage performance bottleneck how avoiding Longitudinal Extension node to become super emerging system
Problem.
For achieving the above object, the present invention provides a kind of super emerging system, including the first storage money of self
Source and the second storage resource carrying out storage resource Longitudinal Extension, described first storage resource and described second
Storage resource is for storing the data in described super emerging system, and described super emerging system also includes:
Data Migration module, for when receiving Data Migration request, by described first storage resource
Meet pre-conditioned related data to move to store in described second storage resource, and update migration
The reference address of described related data.
Preferably, described first storage resource includes that some memory nodes, described memory node include some
For storing the disk of data；Described second storage resource include some for disk insert to store
The disk extension cabinet of space dilatation, described disk extension cabinet is connected with described memory node.
Preferably, described first storage resource includes some first memory nodes, described first memory node
Including some disks for storing data and calculating that described first memory node is described super emerging system
Node；Described second storage resource includes some second memory nodes, wherein, described second memory node
For x86 server and memory node that described second memory node is described super emerging system.
Preferably, described super emerging system also includes:
Access frequency logging modle, for drawing the data in described first storage resource according to default size
It is divided into data block, and records the access frequency of all data blocks in described first storage resource and according to number
It is ranked up according to the access frequency of block；
Described Data Migration module is additionally operable to: when receiving the request of described Data Migration, by described first
Access frequency described in storage resource is minimum or described access frequency is moved less than the data block presetting access frequency
Move on to store in described second storage resource, and update the reference address of the data block of migration.
Preferably, described super emerging system also includes:
SRM module, for managing the number stored in the hot accumulation layer disk with cold accumulation layer
According to, wherein, carry out storing the configuration of resource Longitudinal Extension according to user, by described super emerging system
Described first storage resource distribution is described hot accumulation layer, is deposited by described second in described super emerging system
Storage resource distribution is described cold accumulation layer；
Hot accumulation layer monitoring module, for monitoring the service condition of data space in described hot accumulation layer,
And when in described hot accumulation layer, idle storage space is less than preset first threshold value, to described Data Migration
Module sends the request of described Data Migration until idle storage space is higher than presetting second in described hot accumulation layer
Stopping during threshold value sending, wherein, described first threshold is less than described Second Threshold.
Further, for achieving the above object, the present invention also provides for the Longitudinal Extension of a kind of super emerging system
Method, the Longitudinal Extension method of described super emerging system includes:
Whether monitoring user's configuration corresponds to the storage resource Longitudinal Extension configuration preset, wherein, described pre-
If storage resource Longitudinal Extension configuration include by self storage resource distribution be hot accumulation layer, by longitudinal expansion
The storage resource distribution of exhibition is cold accumulation layer；
If user's configuration corresponds to described default storage resource Longitudinal Extension configuration, then monitor described heat and deposit
In reservoir, whether idle storage space meets the condition carrying out Data Migration；
If meeting the condition of Data Migration of carrying out, then by described hot accumulation layer storage meet pre-conditioned
Related data move to store in described cold accumulation layer, and update the described related data of migration
Reference address.
Preferably, described hot accumulation layer include some memory nodes, described memory node include some for
The disk of storage data；Described cold accumulation layer include some for disk insert to carry out memory space dilatation
Disk extension cabinet, described disk extension cabinet is connected with described memory node.
Preferably, described hot accumulation layer includes that some first memory nodes, described first memory node include
Some disks for storing data and the calculating joint that described first memory node is described super emerging system
Point；Described cold accumulation layer includes some second memory nodes, and wherein, described second memory node is x86
Server and the memory node that described second memory node is described super emerging system.
Preferably, if described user configuration corresponds to described default storage resource Longitudinal Extension configuration, then
Monitor the most satisfied condition carrying out Data Migration of idle storage space in described hot accumulation layer to include:
If user's configuration corresponds to described default storage resource Longitudinal Extension configuration, then by described heat storage
Data in Ceng are divided into data block according to default size, and record all data blocks in described hot accumulation layer
Access frequency and be ranked up according to the access frequency of data block；
Monitor whether idle storage space in described hot accumulation layer meets the condition carrying out Data Migration；
If the described satisfied condition carrying out Data Migration, then meeting of storage in described hot accumulation layer is preset
The related data of condition moves to store in described cold accumulation layer, and updates the described dependency number of migration
According to reference address include:
If meeting the condition of Data Migration of carrying out, then by minimum for access frequency described in described hot accumulation layer or
Described access frequency stores in described cold accumulation layer less than the data block migration presetting access frequency,
And update the reference address of the data block of migration.
Preferably, in the described hot accumulation layer of described monitoring, idle storage space is the most satisfied carries out Data Migration
Condition include:
Monitor whether idle storage space in described hot accumulation layer is less than preset first threshold value, the fullest
Foot carries out the condition of Data Migration, if during carrying out Data Migration, idle in described hot accumulation layer
Memory space is then unsatisfactory for carrying out the condition of Data Migration, wherein, described first higher than presetting Second Threshold
Threshold value is less than described Second Threshold.
In the present invention, super emerging system includes that the first storage resource of self longitudinally expands with carrying out storage resource
Second storage resource of exhibition, stores resource by second and then has reached longitudinal dilatation extension storage space
Purpose.Additionally, processed by Data Migration, the first storage resource will meet pre-conditioned dependency number
Store in second storage resource according to moving to, and update the reference address of the described related data of migration,
And then avoid the second storage resource of dilatation to become the storage performance bottleneck of super emerging system, so that super
The overall performance of emerging system is improved.
Accompanying drawing explanation
Fig. 1 is the high-level schematic functional block diagram that the present invention surpasses emerging system first embodiment；
Fig. 2 is the high-level schematic functional block diagram that the present invention surpasses emerging system the second embodiment；
Fig. 3 is that the present invention surpasses the configuration diagram storing resource Longitudinal Extension first embodiment in emerging system；
Fig. 4 is the high-level schematic functional block diagram that the present invention surpasses emerging system the 3rd embodiment；
Fig. 5 is that the present invention surpasses the configuration diagram storing resource Longitudinal Extension the second embodiment in emerging system；
Fig. 6 is the high-level schematic functional block diagram that the present invention surpasses emerging system the 4th embodiment；
Fig. 7 is the high-level schematic functional block diagram that the present invention surpasses emerging system the 5th embodiment；
Fig. 8 is the schematic flow sheet that the present invention surpasses Longitudinal Extension method one embodiment of emerging system；
Fig. 9 is the refinement schematic flow sheet of step S20 in Fig. 8.
The realization of the object of the invention, functional characteristics and advantage will in conjunction with the embodiments, do referring to the drawings further
Explanation.
Detailed description of the invention
Should be appreciated that specific embodiment described herein, only in order to explain the present invention, is not used to limit
The present invention.
It is the high-level schematic functional block diagram that the present invention surpasses emerging system first embodiment with reference to Fig. 1, Fig. 1.This reality
Executing in example, super emerging system includes that self first stores resource 10 and carry out storing resource Longitudinal Extension
Second storage resource 20, wherein, the first storage resource 10 is used for storing super melting with the second storage resource 20
Data in assembly system.
In the present embodiment, the first storage resource 10 and the second storage resource 20 specifically refer to storage device,
Such as disk.It addition, the Longitudinal Extension of the super emerging system described in the present embodiment specifically refers to: when super
When the calculating resource of emerging system is unbalanced with the use of storage resource, namely calculating resource enough stores
Inadequate resource or calculate inadequate resource and store resource enough time, need to carry out accordingly to store the vertical of resource
To extension or the Longitudinal Extension of calculating resource.
Additionally, in the present embodiment, super emerging system also includes: Data Migration module 30, for when receiving
To Data Migration ask time, move to the by the first storage resource 10 meets pre-conditioned related data
Store in two storage resources 20, and update the reference address of the related data of migration.Need explanation
It is, in the present embodiment, for needing the screening conditions of the related data migrated not limit, with specific reference to reality
Needs are configured, such as, arrange the screening conditions etc. carrying out Data Migration according to the access frequency of data.
The present embodiment does not limit the second storage resource 20 and carries out the extended mode corresponding to Longitudinal Extension,
It is configured with specific reference to being actually needed.In the present embodiment, which kind of extended mode is either used to surpass
The memory expanding of emerging system, assumes that the disk number ratio first in the second storage resource 20 of extension
Disk number in storage resource 10 is much more, and then to meet user's demand to memory capacity.And work as
Use after Longitudinal Extension, in original super emerging system the disk number in the first storage resource 10 the same or
Much the same feature is broken, and is used for doing the disk number in the second storage resource 20 of Longitudinal Extension
More much more than the disk number in the first storage resource 10, thus what the second storage resource 20 was provided that
Memory capacity is also much bigger than the first storage resource 10, and then causes the second storage resource 20 to be easy to into
Performance bottleneck for super emerging system.Therefore, the present embodiment is processed by Data Migration, deposit first
Storage resource 10 meets pre-conditioned related data move to store in the second storage resource 20,
And update the reference address of the described related data of migration, and then the second storage resource of dilatation is avoided to become
The storage performance bottleneck of super emerging system, so that the overall performance of super emerging system is improved.
Further alternative, as in figure 2 it is shown, in the present invention surpasses emerging system the second embodiment, first
Storage resource 10 includes some memory nodes 101, and memory node 101 includes some for storing data
Disk；Second storage resource 20 include some for disk insert with carry out memory space dilatation disk expand
Showcase 201, disk extension cabinet 201 is connected with memory node 101, as it is shown on figure 3, by merging toward super
Disk extension cabinet 201 is accessed to carry out storing resource Longitudinal Extension on part memory node in system.
Owing to disk extension cabinet 201 itself is without CPU, internal memory, simply by HBA card (Host bus
Adapter, host bus adaptor) it is connected on super emerging system, namely use direct-connected storage DAS
(Direct-Attached Storage).Generally disk extension cabinet 201 is due to simple in construction, thus can do
Become compact disc cabinet, the such as extension cabinet of a 5U at most can insert about 80 pieces 3.5 cun of disks.As
Shown in Fig. 3: a super emerging system comprising 5 memory nodes 101, wherein, memory node 4 and 5
All connect a disk extension cabinet 201, so by using highdensity disk extension cabinet 201, it is possible to
Realize the very big extension of super emerging system memory capacity.The disk extending out by disk extension cabinet 201
For memory node 4 and 5, it is simply that direct-connected storage, namely can regard as and be inserted in memory node 4
With the disk on 5.
Further alternative, as shown in Figure 4, in the present invention surpasses emerging system the 3rd embodiment, first
Storage resource 10 includes some first memory nodes 102, and the first memory node 102 includes some for depositing
The disk of storage data and calculating node that the first memory node 102 is super emerging system；Second storage resource
20 include some second memory nodes 202, and wherein, the second memory node 202 is x86 server and
Two memory nodes 202 are the memory node (can be referred to as storage-type node) of super emerging system.Wherein, meter
Operator node is relative with memory node, also processes the various computings in super emerging system in calculating node, and
The various data of super emerging system are stored, it addition, calculating node can also be to deposit simultaneously in memory node
Storage node, but memory node is then not necessarily calculating node.As it is shown in figure 5, super emerging system uses
It is more weak that CPU, internal memory etc. calculate resource, but the more x86 server in the dish position of disk stores
The Longitudinal Extension of resource, and then save the expense calculating resource, reduce the operating cost of super emerging system.
Memory node that x86 server described in the present embodiment is only used as in super emerging system exist and not
As calculating node.
It is the high-level schematic functional block diagram that the present invention surpasses emerging system the 4th embodiment with reference to Fig. 6, Fig. 6.Based on
Above-described embodiment, in the present embodiment, super emerging system also includes:
Access frequency logging modle 40, for dividing the data in the first storage resource according to default size
Become data block, and record the access frequency of all data blocks in the first storage resource and according to data block
Access frequency is ranked up；
Data Migration module 30 is additionally operable to: when receiving Data Migration request, by the first storage resource
Access frequency is minimum or access frequency stores in resource to second less than the data block migration presetting access frequency
Store, and update the reference address of the data block of migration.
In the present embodiment, super emerging system is using after Longitudinal Extension, in original super emerging system first
Disk number in storage resource 10 is the same or much the same feature is broken, and is used for doing Longitudinal Extension
Second storage resource 20 on disk number but than first storage resource 10 on disk number much more,
Thus the second storage memory capacity of being provided that of resource 20 is also much bigger than the first storage resource 10, and then
The second storage resource 20 is caused to readily become the performance bottleneck of super emerging system.
Therefore, by solving the performance bottleneck problem that super emerging system is brought after using Longitudinal Extension, this
In embodiment, by access frequency logging modle 40 by the data in the first storage resource 10 according to presetting
Size is divided into data block, and records access frequency and the root of all data blocks in the first storage resource 10
It is ranked up according to the access frequency of data block.Wherein, access frequency specifically refers to being accessed for of data block
Frequency, such as, can use recent minimum use algorithm LRU (Least Recently Used) or other classes
As can show that the algorithm of data block access frequency is to determine the access frequency of data block.
Those skilled in the art it is appreciated that it is different that different data blocks is accessed for frequency,
Some data blocks may access very frequent, and some data blocks may be for a long time all without being accessed once, very
To the data block that has may the most several years of some months all without being accessed once, therefore, in the present embodiment,
The feature of access frequency based on above-mentioned data block, arranges the related data (data block) of needs migration
Screening conditions.In the present embodiment, when Data Migration module 30 receives Data Migration request, by first
In storage resource 10, access frequency is minimum or access frequency is less than presetting the data block migration of access frequency to the
Store in two storage resources 20, and update the reference address of the data block of migration.
In the present embodiment, the Longitudinal Extension ability powerful to realize super emerging system, then have to realize
The hardware of super emerging system and common on software coordinating, and first having to can be easily toward in super emerging system
More disk is added in face, and secondly, the super fusion storage software that super emerging system is used is wanted can be seamless
The disk being newly added is included in super fusion storage and non-interrupting service by ground, and also will not deposit because of part
It is inserted with too much disk (or hard disk) above storage node and causes performance bottleneck.Therefore, the present embodiment
In, on Longitudinal Extension, increase the second storage resource 20 is while extension storage capacity, also by super fusion
In first storage resource 10 of system self, the data of accessed frequency low (namely access frequency is the highest) are moved
Move on in the second storage resource 20, thus realize when the memory space inadequate of the first storage resource 10,
Reach the purpose in longitudinal dilatation extension storage space, also achieve higher storage performance simultaneously and avoid the
Two storage resources 20 become storage performance bottleneck.
It is the high-level schematic functional block diagram that the present invention surpasses emerging system the 5th embodiment with reference to Fig. 7, Fig. 7.Based on
Above-described embodiment, in the present embodiment, super emerging system also includes:
SRM module 50, is stored in the disk managing hot accumulation layer and cold accumulation layer
Data, wherein, carry out storing the configuration of resource Longitudinal Extension, by the in super emerging system according to user
One storage resource distribution is hot accumulation layer, is cold storage by the second storage resource distribution in super emerging system
Layer；
In the present embodiment, when super emerging system uses after Longitudinal Extension, in original super emerging system first
Disk number in storage resource 10 is the same or much the same feature is broken, and is used for doing Longitudinal Extension
Second storage resource 20 on disk number but than first storage resource 10 on disk number much more,
Thus the second storage memory capacity of being provided that of resource 20 is also much bigger than the first storage resource 10, and then
The second storage resource 20 is caused to readily become the performance bottleneck of super emerging system.For solving this problem, this
Embodiment proposes method based on the cold and hot layering of data and utilizes the memory space of dilatation, store first
After data in resource 10 are divided into data block according to a certain size, the most different data blocks is accessed for frequency
Rate is different, and some data blocks may access very frequent, and some data blocks may be for a long time all without interviewed
Ask once, the data block that even has may the most several years of some months all without being accessed once.
Therefore, the present embodiment will utilize the features described above of data, by disks different in super emerging system
Being divided into different levels, concrete dividing mode is:
A) disk (namely first storage resource 10) of super emerging system hardware self is divided into heat to deposit
Reservoir；B) by disk extension cabinet or memory node x86 server disk (namely second storage money
Source 20) it is divided into cold accumulation layer.
Hot accumulation layer monitoring module 60, for monitoring the service condition of data space in hot accumulation layer,
And when in hot accumulation layer, idle storage space is less than preset first threshold value, send to Data Migration module
Data Migration request stops sending higher than when presetting Second Threshold until idle storage space in hot accumulation layer,
Wherein, described first threshold is less than described Second Threshold.
In the present embodiment, for the setting of transition condition, especially by hot accumulation layer monitoring module 60, prison
Control the service condition of data space in hot accumulation layer and select to send data to Data Migration module 30 and move
Move request, be specially when hot accumulation layer monitoring module 60 monitor hot accumulation layer (namely first storage resource
10), when idle storage space is less than preset first threshold value in, Data Migration is sent to Data Migration module 30
Request sends Data Migration until idle storage space in hot accumulation layer higher than stopping when presetting Second Threshold and asks
Ask, thus Data Migration module 30 is by filtering out the data block that meets condition, and (such as access frequency is minimum
Data block or access frequency are less than the data block presetting access frequency) move to cold accumulation layer (namely the
Two storage resources 20) on store, and update the reference address of the data block of migration.
It is the flow process signal that the present invention surpasses Longitudinal Extension method one embodiment of emerging system with reference to Fig. 8, Fig. 8
Figure.In the present embodiment, the Longitudinal Extension method of super emerging system includes:
Step S10, the storage resource Longitudinal Extension whether monitoring user's configuration corresponds to preset configures, wherein,
The storage resource Longitudinal Extension configuration preset includes that by self storage resource distribution be hot accumulation layer, will longitudinally
The storage resource distribution of extension is cold accumulation layer；
In the present embodiment, after super emerging system uses Longitudinal Extension, in original super emerging system, heat is deposited
Disk number on reservoir is the same or much the same feature is broken, and is used for doing the cold of Longitudinal Extension and deposits
Disk number on reservoir but wants the disk number in specific heat accumulation layer much more, thus cold accumulation layer is provided that
Memory capacity also specific heat accumulation layer much bigger, and then cause cold accumulation layer to readily become super emerging system
Performance bottleneck.For solving this problem, the present embodiment proposes method based on the cold and hot layering of data and comes sharp
With the memory space of dilatation, after the data in hot accumulation layer are divided into data block according to a certain size, then
It is different that different data blocks is accessed for frequency, and some data blocks may access very frequent, some numbers
According to block may for a long time all without being accessed once, the data block that even has may the most several years of some months the most not
Can be accessed once.
Therefore, the present embodiment will utilize the features described above of data, by disks different in super emerging system
Being divided into different levels, concrete dividing mode is:
A) disk of super emerging system hardware self is divided into hot accumulation layer；B) by disk extension cabinet or
Disk on person's memory node x86 server is divided into cold accumulation layer.
Step S20, if user's configuration corresponds to the storage resource Longitudinal Extension configuration preset, then monitoring is described
In hot accumulation layer, whether idle storage space meets the condition carrying out Data Migration；
In the present embodiment, do not limit for carrying out the set-up mode of the condition of Data Migration, such as heat is stored
In Ceng, the capacity of idle storage space proceeds by Data Migration process when being less than a certain numerical value.
Step S30, if meeting the condition carrying out Data Migration, then presets meeting of storage in hot accumulation layer
The related data of condition moves to store in cold accumulation layer, and updates the access of the related data of migration
Address.
Additionally, in the present embodiment, when meeting the condition carrying out Data Migration, will hot accumulation layer meet
Pre-conditioned related data moves to store in cold accumulation layer, and update the related data of migration
Reference address.It should be noted that in the present embodiment, for needing the screening bar of the related data migrated
Part does not limits, and is configured with specific reference to being actually needed, such as, arrange according to the access frequency of data and carry out
The screening conditions etc. of Data Migration.
The present embodiment does not limit cold accumulation layer and carries out the extended mode corresponding to Longitudinal Extension, concrete root
It is configured according to being actually needed.In the present embodiment, which kind of extended mode is either used to carry out super fusion system
The memory expanding of system, assumes that the disk in the disk number specific heat accumulation layer in the cold accumulation layer of extension
Number is much more, and then to meet user's demand to memory capacity.And after using Longitudinal Extension, former
In the super emerging system having, the disk number in hot accumulation layer is the same or much the same feature is broken, and
It is used for doing the disk number in the cold accumulation layer of Longitudinal Extension and but wants the disk number in specific heat accumulation layer to obtain more
Many, thus the memory capacity also specific heat accumulation layer that cold accumulation layer is provided that is much bigger, and then cause cold storage
Layer readily becomes the performance bottleneck of super emerging system.Therefore, the present embodiment is processed by Data Migration,
Move to store in cold accumulation layer by hot accumulation layer meets pre-conditioned related data, and update
The reference address of the described related data migrated, and then avoid the cold accumulation layer of dilatation to become super emerging system
Storage performance bottleneck so that the overall performance of super emerging system is improved.
Further alternative, described hot accumulation layer includes some memory nodes, if described memory node includes
Do the disk for storing data；Described cold accumulation layer includes some empty to carry out storage for disk insertion
Between the disk extension cabinet of dilatation, described disk extension cabinet is connected with described memory node.As it is shown on figure 3,
By accessing disk extension cabinet on the part memory node in super emerging system to carry out storing resource longitudinally
Extension.
Owing to disk extension cabinet itself is without CPU, internal memory, simply by HBA card (Host bus adapter,
Host bus adaptor) it is connected on super emerging system, namely use direct-connected storage DAS
(Direct-Attached Storage).Generally disk extension cabinet is due to simple in construction, thus can make height
Density dish cabinet, the such as extension cabinet of a 5U at most can insert about 80 pieces 3.5 cun of disks.Such as Fig. 3
Shown in: a super emerging system comprising 5 memory nodes, wherein, memory node 4 is all connected with 5
One disk extension cabinet, so by using highdensity disk extension cabinet, it is possible to realizing super fusion is
The very big extension of system memory capacity.The disk extending out by disk extension cabinet is for memory node 4 and 5
For, it is simply that direct-connected storage, namely the disk being inserted on memory node 4 and 5 can be regarded as.
Further alternative, described hot accumulation layer includes some first memory nodes, described first storage joint
Point includes that some disks by storing data and described first memory node are based on described super emerging system
Operator node；Described cold accumulation layer includes some second memory nodes, and wherein, described second memory node is
X86 server and the memory node that described second memory node is described super emerging system.Wherein, joint is calculated
Point is relative with memory node, also processes the various computings in super emerging system in calculating node, and is depositing
Storage node memory stores up the various data of super emerging system, it addition, calculating node can also be storage joint simultaneously
Point, but memory node is then not necessarily calculating node.As it is shown in figure 5, in super emerging system use CPU,
It is more weak that internal memories etc. calculate resource, but the more x86 server in the dish position of disk carries out storing resource
Longitudinal Extension, and then save the expense calculating resource, reduce the operating cost of super emerging system.This enforcement
Memory node that x86 server described in example is only used as in super emerging system exist and not as meter
Operator node.
With reference to the refinement schematic flow sheet that Fig. 9, Fig. 9 are step S20 in Fig. 8.Based on above-described embodiment,
In the present embodiment, above-mentioned steps S20 includes:
Step S201, if user's configuration corresponds to the storage resource Longitudinal Extension configuration preset, then deposits heat
Data in reservoir are divided into data block according to default size, and record all data blocks in hot accumulation layer
Access frequency and the access frequency according to data block are ranked up；
In the present embodiment, super emerging system is after using Longitudinal Extension, and in original super emerging system, heat is deposited
Disk number on reservoir is the same or much the same feature is broken, and is used for doing the cold of Longitudinal Extension and deposits
Disk number on reservoir but wants the disk number in specific heat accumulation layer much more, thus cold accumulation layer is provided that
Memory capacity also specific heat accumulation layer much bigger, and then cause cold accumulation layer to readily become super emerging system
Performance bottleneck.
Therefore, by solving the performance bottleneck problem that super emerging system is brought after using Longitudinal Extension, this
In embodiment, by the data in hot accumulation layer are divided into data block according to default size, and record heat
The access frequency of all data blocks and being ranked up according to the access frequency of data block in accumulation layer.Wherein,
What access frequency specifically referred to data block is accessed for frequency, such as, can use recent minimum use algorithm
LRU (Least Recently Used) or other similar algorithms that can show that data block access frequency
To determine the access frequency of data block.
Whether step S202, monitor in described hot accumulation layer idle storage space and meet and carry out Data Migration
Condition；
In the present embodiment, do not limit for carrying out the set-up mode of the condition of Data Migration, such as heat is stored
In Ceng, the capacity of idle storage space proceeds by Data Migration process when being less than a certain numerical value.
It addition, in the present embodiment, above-mentioned steps S30 specifically includes: if meeting the bar carrying out Data Migration
Part, then be less than the related data presetting access frequency by minimum for access frequency in hot accumulation layer or access frequency
Move to store in cold accumulation layer, and update the reference address of the related data of migration.
Those skilled in the art it is appreciated that it is different that different data blocks is accessed for frequency,
Some data blocks may access very frequent, and some data blocks may be for a long time all without being accessed once, very
To the data block that has may the most several years of some months all without being accessed once, therefore, in the present embodiment,
The feature of access frequency based on above-mentioned data block, arranges the related data (data block) of needs migration
Screening conditions.In the present embodiment, when meeting the condition carrying out Data Migration, will hot accumulation layer access
Frequency is minimum or access frequency moves to deposit in cold accumulation layer less than the related data presetting access frequency
Storage, and update the reference address of the related data of migration.
In the present embodiment, the Longitudinal Extension ability powerful to realize super emerging system, then have to realize
The hardware of super emerging system and common on software coordinating, and first having to can be easily toward in super emerging system
More disk is added in face, and secondly, the super fusion storage software that super emerging system is used is wanted can be seamless
The disk being newly added is included in super fusion storage and non-interrupting service by ground, and also will not deposit because of part
It is inserted with too much disk (or hard disk) above storage node and causes performance bottleneck.Therefore, the present embodiment
In, Longitudinal Extension increases cold accumulation layer while extension storage capacity, also by super emerging system from
In the hot accumulation layer of body, the Data Migration of accessed frequency low (namely access frequency is the highest) is to cold accumulation layer
On, thus realize when the memory space inadequate of hot accumulation layer, reach longitudinal dilatation extension storage space
Purpose, also achieves higher storage performance simultaneously and avoids cold accumulation layer to become storage performance bottleneck.
Further alternative, in the present invention surpasses another embodiment of Longitudinal Extension method of emerging system, on
State step S202 to specifically include:
Monitor whether idle storage space in described hot accumulation layer is less than preset first threshold value, the fullest
Foot carries out the condition of Data Migration, if during carrying out Data Migration, idle in described hot accumulation layer
Memory space is then unsatisfactory for carrying out the condition of Data Migration, wherein, described first higher than presetting Second Threshold
Threshold value is less than described Second Threshold.
In the present embodiment, for the setting of transition condition, especially by monitoring data storage in hot accumulation layer
The service condition in space, specially ought monitor idle storage space in hot accumulation layer and be less than default first threshold
During value, it is determined that meet the condition carrying out Data Migration, thus filter out the data block (ratio meeting condition
Data block as minimum in access frequency or access frequency are less than the data block presetting access frequency) and migrate
In cold accumulation layer, carry out storing and updating the reference address of the data block of migration, and when carrying out data
During migration, if idle storage space is higher than presetting Second Threshold in described hot accumulation layer, it is unsatisfactory for
Carrying out the condition of Data Migration, wherein, described first threshold is less than described Second Threshold.
These are only the preferred embodiments of the present invention, not thereby limit the scope of the claims of the present invention, every
Utilize equivalent structure or equivalence flow process conversion that description of the invention and accompanying drawing content made, or directly or
Connect and be used in other relevant technical fields, be the most in like manner included in the scope of patent protection of the present invention.
Claims (10)
1. a super emerging system, stores resource including self first and carries out storing resource Longitudinal Extension
The second storage resource, described first storage resource is used for storing with described second storage resource and melts described surpassing
Data in assembly system, it is characterised in that described super emerging system also includes:
Data Migration module, for when receiving Data Migration request, by described first storage resource
Meet pre-conditioned related data to move to store in described second storage resource, and update migration
The reference address of described related data.
2. super emerging system as claimed in claim 1, it is characterised in that described first storage resource bag
Including some memory nodes, described memory node includes some disks for storing data；Described second deposits
Storage resource includes some inserting to carry out the disk extension cabinet of memory space dilatation for disk, described disk
Extension cabinet is connected with described memory node.
3. super emerging system as claimed in claim 1, it is characterised in that described first storage resource bag
Including some first memory nodes, described first memory node includes some disks for storing data and institute
State the calculating node that the first memory node is described super emerging system；Described second storage resource includes some
Second memory node, wherein, described second memory node is x86 server and described second memory node
Memory node for described super emerging system.
4. super emerging system as claimed any one in claims 1 to 3, it is characterised in that described super
Emerging system also includes:
Access frequency logging modle, for drawing the data in described first storage resource according to default size
It is divided into data block, and records the access frequency of all data blocks in described first storage resource and according to number
It is ranked up according to the access frequency of block；
Described Data Migration module is additionally operable to: when receiving the request of described Data Migration, by described first
Access frequency described in storage resource is minimum or described access frequency is moved less than the data block presetting access frequency
Move on to store in described second storage resource, and update the reference address of the data block of migration.
5. super emerging system as claimed in claim 4, it is characterised in that described super emerging system also wraps
Include:
SRM module, for managing the number stored in the hot accumulation layer disk with cold accumulation layer
According to, wherein, carry out storing the configuration of resource Longitudinal Extension according to user, by described super emerging system
Described first storage resource distribution is described hot accumulation layer, is deposited by described second in described super emerging system
Storage resource distribution is described cold accumulation layer；
Hot accumulation layer monitoring module, for monitoring the service condition of data space in described hot accumulation layer,
And when in described hot accumulation layer, idle storage space is less than preset first threshold value, to described Data Migration
Module sends the request of described Data Migration until idle storage space is higher than presetting second in described hot accumulation layer
Stopping during threshold value sending, wherein, described first threshold is less than described Second Threshold.
6. the Longitudinal Extension method of a super emerging system, it is characterised in that indulging of described super emerging system
Include to extended method:
Whether monitoring user's configuration corresponds to the storage resource Longitudinal Extension configuration preset, wherein, described pre-
If storage resource Longitudinal Extension configuration include by self storage resource distribution be hot accumulation layer, by longitudinal expansion
The storage resource distribution of exhibition is cold accumulation layer；
If user's configuration corresponds to described default storage resource Longitudinal Extension configuration, then monitor described heat and deposit
In reservoir, whether idle storage space meets the condition carrying out Data Migration；
If meeting the condition of Data Migration of carrying out, then by described hot accumulation layer storage meet pre-conditioned
Related data move to store in described cold accumulation layer, and update the described related data of migration
Reference address.
7. the Longitudinal Extension method of super emerging system as claimed in claim 6, it is characterised in that described
Hot accumulation layer includes that some memory nodes, described memory node include some disks for storing data；
Described cold accumulation layer includes some inserting to carry out the disk extension cabinet of memory space dilatation for disk, institute
State disk extension cabinet to be connected with described memory node.
8. the Longitudinal Extension method of super emerging system as claimed in claim 6, it is characterised in that described
Hot accumulation layer includes that some first memory nodes, described first memory node include some for storing data
Disk and calculating node that described first memory node is described super emerging system；Described cold accumulation layer bag
Including some second memory nodes, wherein, described second memory node is that x86 server and described second is deposited
Storage node is the memory node of described super emerging system.
9. the Longitudinal Extension method of the super emerging system as according to any one of claim 6 to 8, it is special
Levy and be, if described user configuration corresponds to described default storage resource Longitudinal Extension configuration, then monitor
In described hot accumulation layer, the most satisfied condition carrying out Data Migration of idle storage space includes:
If user's configuration corresponds to described default storage resource Longitudinal Extension configuration, then by described heat storage
Data in Ceng are divided into data block according to default size, and record all data blocks in described hot accumulation layer
Access frequency and be ranked up according to the access frequency of data block；
Monitor whether idle storage space in described hot accumulation layer meets the condition carrying out Data Migration；
If the described satisfied condition carrying out Data Migration, then meeting of storage in described hot accumulation layer is preset
The related data of condition moves to store in described cold accumulation layer, and updates the described dependency number of migration
According to reference address include:
If meeting the condition of Data Migration of carrying out, then by minimum for access frequency described in described hot accumulation layer or
Described access frequency stores in described cold accumulation layer less than the data block migration presetting access frequency,
And update the reference address of the data block of migration.
10. the Longitudinal Extension method of super emerging system as claimed in claim 9, it is characterised in that institute
The most satisfied condition carrying out Data Migration of idle storage space in the described hot accumulation layer of monitoring of stating includes:
Monitor whether idle storage space in described hot accumulation layer is less than preset first threshold value, the fullest
Foot carries out the condition of Data Migration, if during carrying out Data Migration, idle in described hot accumulation layer
Memory space is then unsatisfactory for carrying out the condition of Data Migration, wherein, described first higher than presetting Second Threshold
Threshold value is less than described Second Threshold.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610259698.8A CN105892952A (en) | 2016-04-22 | 2016-04-22 | Hyper-converged system and longitudinal extension method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610259698.8A CN105892952A (en) | 2016-04-22 | 2016-04-22 | Hyper-converged system and longitudinal extension method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105892952A true CN105892952A (en) | 2016-08-24 |
Family
ID=56704605
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610259698.8A Pending CN105892952A (en) | 2016-04-22 | 2016-04-22 | Hyper-converged system and longitudinal extension method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105892952A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708433A (en) * | 2016-12-06 | 2017-05-24 | 深圳市深信服电子科技有限公司 | Read-write method and device for stored data |
| CN106843755A (en) * | 2017-01-04 | 2017-06-13 | 北京百度网讯科技有限公司 | For the data balancing method and device of server cluster |
| CN106991121A (en) * | 2017-02-23 | 2017-07-28 | 深圳市中博睿存信息技术有限公司 | Super fused data storage method and system |
| CN107037985A (en) * | 2017-02-14 | 2017-08-11 | 中山大学 | The super fusion integrated machine system of one kind and its horizontal and vertical expansion method |
| CN107193500A (en) * | 2017-05-26 | 2017-09-22 | 郑州云海信息技术有限公司 | A kind of distributed file system Bedding storage method and system |
| CN107329801A (en) * | 2017-06-29 | 2017-11-07 | 深信服科技股份有限公司 | A kind of node administration method and device, many component servers |
| CN107621927A (en) * | 2017-09-29 | 2018-01-23 | 南京宏海科技有限公司 | A kind of Longitudinal Extension method and its device based on super emerging system |
| CN108228087A (en) * | 2016-12-21 | 2018-06-29 | 伊姆西Ip控股有限责任公司 | For the device of super fusion architecture |
| CN109951531A (en) * | 2019-02-27 | 2019-06-28 | 广东唯一网络科技有限公司 | Super fusion cloud computing system |
| CN109947531A (en) * | 2019-02-27 | 2019-06-28 | 广东唯一网络科技有限公司 | The expanding storage depth method, apparatus and storage medium of super fusion all-in-one machine |
| CN111538520A (en) * | 2020-07-08 | 2020-08-14 | 北京联想超融合科技有限公司 | Updating method and device for super-converged cluster, terminal and storage medium |
| CN114430458A (en) * | 2020-10-29 | 2022-05-03 | 北京小米移动软件有限公司 | Anti-shake data acquisition method, anti-shake data acquisition apparatus, and storage medium |
| WO2022171000A1 (en) * | 2021-02-09 | 2022-08-18 | 山东英信计算机技术有限公司 | Data transfer method and system, and device and medium |
| CN114924880A (en) * | 2022-05-20 | 2022-08-19 | 创新科信息技术(北京)有限公司 | Workload distribution method, device, equipment and readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140281211A1 (en) * | 2013-03-15 | 2014-09-18 | Silicon Graphics International Corp. | Fast mount cache |
| CN104199784A (en) * | 2014-08-20 | 2014-12-10 | 浪潮(北京)电子信息产业有限公司 | Data migration method and device based on hierarchical storage |
| CN104462240A (en) * | 2014-11-18 | 2015-03-25 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing hierarchical storage and management in cloud storage |
| CN104506589A (en) * | 2014-12-10 | 2015-04-08 | 上海爱数软件有限公司 | Resource migration scheduling method based on super fusion storage |
-
2016
- 2016-04-22 CN CN201610259698.8A patent/CN105892952A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140281211A1 (en) * | 2013-03-15 | 2014-09-18 | Silicon Graphics International Corp. | Fast mount cache |
| CN104199784A (en) * | 2014-08-20 | 2014-12-10 | 浪潮(北京)电子信息产业有限公司 | Data migration method and device based on hierarchical storage |
| CN104462240A (en) * | 2014-11-18 | 2015-03-25 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing hierarchical storage and management in cloud storage |
| CN104506589A (en) * | 2014-12-10 | 2015-04-08 | 上海爱数软件有限公司 | Resource migration scheduling method based on super fusion storage |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708433A (en) * | 2016-12-06 | 2017-05-24 | 深圳市深信服电子科技有限公司 | Read-write method and device for stored data |
| CN108228087B (en) * | 2016-12-21 | 2021-08-06 | 伊姆西Ip控股有限责任公司 | Apparatus for hyper-converged infrastructure |
| CN108228087A (en) * | 2016-12-21 | 2018-06-29 | 伊姆西Ip控股有限责任公司 | For the device of super fusion architecture |
| CN106843755B (en) * | 2017-01-04 | 2019-10-11 | 北京百度网讯科技有限公司 | Data balancing method and device for server cluster |
| CN106843755A (en) * | 2017-01-04 | 2017-06-13 | 北京百度网讯科技有限公司 | For the data balancing method and device of server cluster |
| CN107037985A (en) * | 2017-02-14 | 2017-08-11 | 中山大学 | The super fusion integrated machine system of one kind and its horizontal and vertical expansion method |
| CN106991121A (en) * | 2017-02-23 | 2017-07-28 | 深圳市中博睿存信息技术有限公司 | Super fused data storage method and system |
| CN107193500A (en) * | 2017-05-26 | 2017-09-22 | 郑州云海信息技术有限公司 | A kind of distributed file system Bedding storage method and system |
| CN107329801A (en) * | 2017-06-29 | 2017-11-07 | 深信服科技股份有限公司 | A kind of node administration method and device, many component servers |
| CN107329801B (en) * | 2017-06-29 | 2020-12-15 | 深信服科技股份有限公司 | Node management method and device and multi-subsatellite server |
| CN107621927A (en) * | 2017-09-29 | 2018-01-23 | 南京宏海科技有限公司 | A kind of Longitudinal Extension method and its device based on super emerging system |
| CN107621927B (en) * | 2017-09-29 | 2020-08-14 | 南京宏海科技有限公司 | Longitudinal extension method and device based on super-fusion system |
| CN109951531A (en) * | 2019-02-27 | 2019-06-28 | 广东唯一网络科技有限公司 | Super fusion cloud computing system |
| CN109947531A (en) * | 2019-02-27 | 2019-06-28 | 广东唯一网络科技有限公司 | The expanding storage depth method, apparatus and storage medium of super fusion all-in-one machine |
| CN109947531B (en) * | 2019-02-27 | 2021-05-07 | 广东唯一网络科技有限公司 | Storage capacity expansion method and device of super-fusion all-in-one machine and storage medium |
| CN109951531B (en) * | 2019-02-27 | 2021-05-07 | 广东唯一网络科技有限公司 | Super-fusion cloud computing system |
| CN111538520A (en) * | 2020-07-08 | 2020-08-14 | 北京联想超融合科技有限公司 | Updating method and device for super-converged cluster, terminal and storage medium |
| CN114430458A (en) * | 2020-10-29 | 2022-05-03 | 北京小米移动软件有限公司 | Anti-shake data acquisition method, anti-shake data acquisition apparatus, and storage medium |
| CN114430458B (en) * | 2020-10-29 | 2023-12-19 | 北京小米移动软件有限公司 | Anti-shake data acquisition method, anti-shake data acquisition device, and storage medium |
| WO2022171000A1 (en) * | 2021-02-09 | 2022-08-18 | 山东英信计算机技术有限公司 | Data transfer method and system, and device and medium |
| CN114924880A (en) * | 2022-05-20 | 2022-08-19 | 创新科信息技术(北京)有限公司 | Workload distribution method, device, equipment and readable storage medium |
| CN114924880B (en) * | 2022-05-20 | 2023-03-24 | 创新科信息技术(北京)有限公司 | Workload distribution method, device, equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105892952A (en) | Hyper-converged system and longitudinal extension method thereof | |
| US8775765B2 (en) | Computer system and control method therefor | |
| US8122116B2 (en) | Storage management method and management server | |
| JP4813843B2 (en) | Storage device, disk cache control method, and disk cache capacity allocation method | |
| US20100042797A1 (en) | Computer system, data migration monitoring method and data migration monitoring program | |
| US8069191B2 (en) | Method, an apparatus and a system for managing a snapshot storage pool | |
| CN107506314B (en) | Method and apparatus for managing storage system | |
| JP4684864B2 (en) | Storage device system and storage control method | |
| WO2002029573A2 (en) | Instant snapshot | |
| US20050223278A1 (en) | Control system of NAS, backup method, and program therefor | |
| CN109947787A (en) | A kind of storage of data hierarchy, hierarchical query method and device | |
| US7337283B2 (en) | Method and system for managing storage reservation | |
| CN109271376A (en) | Database upgrade method, apparatus, equipment and storage medium | |
| US7849264B2 (en) | Storage area management method for a storage system | |
| CN103294609A (en) | Information processing device, and memory management method | |
| CN107111627A (en) | Online file system check | |
| US8799569B2 (en) | Multiple enhanced catalog sharing (ECS) cache structure for sharing catalogs in a multiprocessor system | |
| CN107526533B (en) | Storage management method and equipment | |
| CN110247801A (en) | A kind of monitoring system and method for pair of cluster host | |
| CN113434470A (en) | Data distribution method and device and electronic equipment | |
| US7454571B1 (en) | Heuristic cache tuning | |
| CN110858122A (en) | Method and device for storing data | |
| CN114816272B (en) | Magnetic disk management system under Kubernetes environment | |
| CN109508140B (en) | Storage resource management method and device, electronic equipment and system | |
| US11556260B2 (en) | Providing coordination between mapper circuitry and redundant array of independent disks (RAID) circuitry to manage RAID storage space |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160824 |