CN103631666B - The fault-tolerant adaptation management equipment of data redundancy, service equipment, system and method - Google Patents
The fault-tolerant adaptation management equipment of data redundancy, service equipment, system and method Download PDFInfo
- Publication number
- CN103631666B CN103631666B CN201210305197.0A CN201210305197A CN103631666B CN 103631666 B CN103631666 B CN 103631666B CN 201210305197 A CN201210305197 A CN 201210305197A CN 103631666 B CN103631666 B CN 103631666B
- Authority
- CN
- China
- Prior art keywords
- data
- tolerant
- data redundancy
- fault
- redundancy fault
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention discloses a kind of fault-tolerant storage of data redundancy to be adapted to management equipment, including：Management services module, for configuring data redundancy fault-tolerant way and its adaptation strategies in cloud storage.The invention also discloses a kind of fault-tolerant storage adaptation services equipment of data redundancy, including：Metadata service module and data service module.The invention also discloses a kind of fault-tolerant storage adaption system of data redundancy and method, method therein to include：Configuration data redundancy fault-tolerant mode and its adaptation strategies, metadata service module and data service module carry out adaptation adjustment to the data redundancy fault-tolerant way of itself.Using the present invention, a variety of data redundancy fault-tolerant ways can be taken into account, adjustment is neatly adapted to data redundancy fault-tolerant mode.
Description
Technical field
The present invention relates to the cloud storage technology of cloud computing application, more particularly to the storage adapter tube that a kind of data redundancy is fault-tolerant
Manage equipment, service equipment, system and method.
Background technology
Global metadata amount surges, and drives the fast development of whole storage market so that cloud storage technology is advised greatly
Mould application.
Cloud storage is stored and is managed the cloud computing system for core with data as one, passes through cluster application, Grid Technology
The function such as art or distributed file system, by a large amount of various types of storage devices in network by application software together
It is common that data storage and Operational Visit function are externally provided to cooperate.Usual cloud storage system equipment uses general clothes
Business device, such as X86-based server.Here, so-called x86 is that the standard number of intel universal computer serieses is abridged, and is specific
Some computer language instruction set that microprocessor performs, define the basic using rule of chip.
Data storage service is the critical services content that cloud storage service business provides a user, therefore the reliability of data is
The performance that user is concerned about very much, in order to ensure the reliability of data in cloud storage, it is necessary to using data redundancy techniques, i.e., according to
There is provided data redundancy fault-tolerant redundant storage mode by cloud storage software, the redundant storage mode of existing cloud storage software level
Including correcting and eleting codes (EC, Erasure Code) mode, copy mode etc..
For extended control mode, so-called extended control mode is that initial data is divided into N+M blocks, and M blocks are verification data, as long as arbitrarily
N block numbers are according to available, you can recover initial data.Configurable to support 9+3, a variety of extended control modes such as 10+2,10+6, meet different reliable
The needs of property and disk utilization, for example, 9+3 refers to N=9, extended control mode during M=3.Extended control mode is different from complete copy backup
Redundant storage mode, is not direct storage initial data itself, but original data division is passed through coding into multiple data blocks
Algorithm first calculates verification data, and initial data and verification data distribution then are stored in storage different in whole wide area network
On server, there is provided access to user (by verification data block to the original data block that is obtained from different storage servers into
Row restructuring verification).Although having saved memory space, which calculates and restructuring verification influences whether access efficiency sum number
According to reliability.
For copy mode, so-called copy mode is for initial data simple copy one or more parts data, and together
When store and provide access.Copy mode is a kind of redundant storage mode of the backup of copy completely, by generating multiple original numbers
According to copy, and the copy of these initial data is respectively stored in storage server different in wide area network, to reach standby
The purpose of part, as long as there is a copy to keep complete, initial data can normally be retrieved and provided to user's access, that is,
Say, as long as there is a copy to access, data would not lose, and data reliability is high, but it is empty to take storage too much
Between.
The different implementations for contrasting above extended control mode and copy mode are understood：Extended control mode obtained with copy mode it is identical
Under conditions of reliability, memory space can be saved, still, coding and restructuring verify meeting more overhead computational disposal ability, lead
Cause access efficiency low；And copy mode access efficiency is high, but memory space utilization rate is not high.
At present, using the prior art, any of the above, single redundant storage mode can only be used, respectively there is quality, cannot
Neatly adaptation adjustment is carried out to storage mode.
The content of the invention
In view of this, it is a primary object of the present invention to provide a kind of data redundancy it is fault-tolerant storage adaptation management equipment,
Service equipment, system and method, can realize that carrying out neatly adaptation in cloud storage to storage mode adjusts.
To reach above-mentioned purpose, the technical proposal of the invention is realized in this way：
A kind of fault-tolerant storage adaptation management equipment of data redundancy, the equipment include：Management services module, for configuring cloud
Data redundancy fault-tolerant way and its adaptation strategies in storage, the data redundancy fault-tolerant way and its adaptation strategies are supplied to
Service side accesses and indicates the data redundancy fault-tolerant way that the selection of service side is adapted.
Wherein, the management services module, be further used for according to storage domain, and/or application, and/or user, and/or
Data object configures the data redundancy fault-tolerant way and its adaptation strategies.
Wherein, the management services module, is further used under the application scenarios of video frequency requested program broadcasting-like service described in configuration
When data redundancy fault-tolerant way and its adaptation strategies, to hot video file, configuration selection copy mode is needed；To non-hot video
File, needs configuration selection correcting and eleting codes extended control mode.
Wherein, the management services module, is further used for configuring institute under the application scenarios of equipment fault maintenance management
When stating data redundancy fault-tolerant way and its adaptation strategies, configure a storage domain and correspond to reserved multiple secure data blocks more than needed, and
Configuration copy mode and extended control mode are all suitable for.
Wherein, the management services module, is further used for causing system alarm in remaining physics insufficient memory
When the data redundancy fault-tolerant way and its adaptation strategies are configured under application scenarios, one storage domain correspondence of configuration is reserved multiple
The quantity of secure data block more than needed needs to reduce, and configures copy mode and extended control mode is all suitable for；When detecting in a manner of copy
When its correspondence temperature of the data block of storage reduces, it is changed to select extended control mode.
A kind of fault-tolerant storage adaptation services equipment of data redundancy, the equipment include：Metadata service module, data service
Module；Wherein,
The metadata service module, for Access Management Access side and read configuration the data redundancy fault-tolerant way and its
Adaptation strategies, select the data redundancy fault-tolerant way that is adapted and carry out adaptation adjustment to the data redundancy fault-tolerant way of itself；
The instruction for being adapted to adjustment its data redundancy fault-tolerant mode is sent to the data service module；
The data service module, for the instruction according to reception, the data redundancy for correspondingly adjusting itself is fault-tolerant
Mode.
Wherein, the data service module, is further used for after adjusting the data redundancy fault-tolerant way of itself, returning response
To the metadata service module；
The metadata service module, is further used for after receiving the response, updates corresponding metadata.
A kind of fault-tolerant storage adaption system of data redundancy, the system include：The management services module of management-side, services side
Metadata service module and data service module；Wherein,
The management services module, for configuring data redundancy fault-tolerant way and its adaptation strategies in cloud storage, by institute
State data redundancy fault-tolerant way and its adaptation strategies are supplied to service side to access and indicate that the data that the selection of service side is adapted are superfluous
Remaining fault-tolerant way；
The metadata service module, for Access Management Access side and read configuration the data redundancy fault-tolerant way and its
Adaptation strategies, select the data redundancy fault-tolerant way that is adapted and carry out adaptation adjustment to the data redundancy fault-tolerant way of itself；
The instruction for being adapted to adjustment its data redundancy fault-tolerant mode is sent to the data service module；
The data service module, for the instruction according to reception, the data redundancy for correspondingly adjusting itself is fault-tolerant
Mode.
Wherein, the management services module, be further used for according to storage domain, and/or application, and/or user, and/or
Data object configures the data redundancy fault-tolerant way and its adaptation strategies.
Wherein, the management services module, is further used under the application scenarios of video frequency requested program broadcasting-like service described in configuration
When data redundancy fault-tolerant way and its adaptation strategies, to hot video file, configuration selection copy mode is needed；To non-hot video
File, needs configuration selection extended control mode.
Wherein, the management services module, is further used for configuring institute under the application scenarios of equipment fault maintenance management
When stating data redundancy fault-tolerant way and its adaptation strategies, configure a storage domain and correspond to reserved multiple secure data blocks more than needed, and
Configuration copy mode and extended control mode are all suitable for.
Wherein, the management services module, is further used for causing system alarm in remaining physics insufficient memory
When the data redundancy fault-tolerant way and its adaptation strategies are configured under application scenarios, one storage domain correspondence of configuration is reserved multiple
The quantity of secure data block more than needed needs to reduce, and configures copy mode and extended control mode is all suitable for；When detecting in a manner of copy
When its correspondence temperature of the data block of storage reduces, it is changed to select extended control mode.
A kind of fault-tolerant storage adaptation method of data redundancy, this method include：In management services module configuration cloud storage
Data redundancy fault-tolerant way and its adaptation strategies；
Metadata service module Access Management Access service module simultaneously reads the data redundancy fault-tolerant way of configuration and its fits
With strategy, select the data redundancy fault-tolerant way that is adapted and adaptation adjustment is carried out to the data redundancy fault-tolerant way of itself；
The instruction for being adapted to adjustment its data redundancy fault-tolerant mode is sent to data service module；Data service module
According to the instruction of reception, the data redundancy fault-tolerant way of itself is correspondingly adjusted.
Wherein, the data redundancy fault-tolerant way in the configuration cloud storage and its adaptation strategies specifically include：According to storage
Domain, and/or application, and/or user, and/or data object configure the data redundancy fault-tolerant way and its adaptation strategies.
Wherein, this method further includes：After data service module adjusts the data redundancy fault-tolerant way of itself, returning response is given
The metadata service module；After metadata service module receives the response, corresponding metadata is updated.
Data redundancy fault-tolerant way and its adaptation strategies in management services module configuration cloud storage of the present invention；Metadata takes
Business module accesses management services module simultaneously reads the data redundancy fault-tolerant way of configuration and its adaptation strategies, selection are adapted
Data redundancy fault-tolerant way and adaptation adjustment is carried out to the data redundancy fault-tolerant way of itself；Adjustment its data will be adapted to
The instruction of redundancy fault-tolerant mode is sent to data service module；Data service module is correspondingly adjusted according to the instruction of reception
The whole data redundancy fault-tolerant way of itself.
Using the present invention, due to the data redundancy fault-tolerant way that can alternatively select to be adapted, realize in cloud storage to depositing
Storage mode is neatly adapted to adjustment, rather than uses a single, fixed data redundancy fault-tolerant way, accordingly, it is capable to
Take into account the superiority of various data redundancy fault-tolerant ways.
Brief description of the drawings
Fig. 1 is the composition structure diagram of present system；
Fig. 2 realizes flow chart for the method for the present invention.
Embodiment
The present invention basic thought be：Data redundancy fault-tolerant way and its adaptation in management services module configuration cloud storage
Strategy；Metadata service module Access Management Access service module and the data redundancy fault-tolerant way and its adaptation plan for reading configuration
Slightly, select the data redundancy fault-tolerant way that is adapted and adaptation adjustment is carried out to the data redundancy fault-tolerant way of itself；It will fit
Instruction with adjustment its data redundancy fault-tolerant mode is sent to data service module；Data service module is according to reception
Instruction, correspondingly adjusts the data redundancy fault-tolerant way of itself.
The invention mainly comprises herein below：
The system of the present invention includes：The metadata service module and cloud in management services module, cloud storage in cloud storage
Data service module in storage.Wherein, metadata service module and data service module are located at homonymy, and by Ethernet with
Management services module interconnects.
Management services module in cloud storage, for configuring data redundancy fault-tolerant way and its adaptation strategies in cloud storage,
Select to be adapted to according to parameter such as physical memory space capacity, and/or access efficiency, and/or network bandwidth and combinations thereof to realize
Data redundancy fault-tolerant way carry out data storage, needed so as to reach and take into account data reliability and two aspect performance of availability of data
The effect asked.
Here, the configuration of the adaptation strategies can be configured according to storage domain, application, user, data object.Its
In, for domain is stored, storage domain refers to cloud storage system logical partitioning, and a storage domain includes multiple storage server sections
Point is general to store domain not common storage node.For application, using referring to mailbox, video, picture, office etc., different application
Generally also differed comprising call data storage.For a user, user refers to the user account number that registration logs in cloud storage system to pin,
Capacity, period and storage strategy using the cloud storage for specifying authority is possessed according to its service agreement.For data object
Speech, data object index is according to storing directory or data file class etc..
Here, specifically, scene 1) is served by for video frequency requested program broadcasting-like, management services module is served by by this
When the concrete application of scene is to set the adaptation strategies of data redundancy fault-tolerant way, it is whether video file is that hot spot regards with application
Exemplified by frequency file, if a video file by multiple program request, is referred to as the video and is whether hot video file (is heat
Point video file, can be weighed with for the daily access times of the video), for hot video file, then using copy
Mode；For non-hot video file, then using extended control mode.Moreover, the video file hot spot, when adaptation strategies are set,
During using copy mode, corresponding copy amount can be more.
2) application scenarios of cloud storage equipment fault maintenance management are directed to, management services module, is set by the storage domain
During the adaptation strategies of data redundancy fault-tolerant way, a storage domain is set to reserve multiple secure data blocks more than needed.Copy mode and
Extended control mode is all suitable for, and only, for copy mode, which refers to duplicate copy database, and is directed to EC side
Formula, then refer to verification data block.
3) few for remaining amount of physical memory, system can alert, and cannot implement dilatation in time for some reason
Application scenarios, management services module, during by the storage domain to set the adaptation strategies of data redundancy fault-tolerant way, reduces more than needed
The quantity of secure data block, data of other secure data blocks more than needed remain unchanged, for temperature it is relatively low, original use
The data that copy mode stores, are changed to the extended control mode using memory space utilization rate higher.
Metadata service module in cloud storage, for reading the data configured in management services module.
Here, which includes data redundancy fault-tolerant way, and how to select different pieces of information redundancy fault-tolerant side
The adaptation strategies of formula.
Metadata service module in cloud storage, is additionally operable to after reading configured data, by the adaptation strategies logarithm
It is scanned according to redundancy fault-tolerant mode, and selects a suitable data redundancy fault-tolerant way.
Data service module in cloud storage, for the instruction according to the metadata service module, adjusts the number of itself
, can be consistent with the selected data redundancy fault-tolerant way of metadata service module according to redundancy fault-tolerant mode, and returning response is given
Metadata service module.Wherein, the response returned is specially：It has been adjusted to the sound of consistent data redundancy fault-tolerant way
Should.
The implementation to technical solution is described in further detail below in conjunction with the accompanying drawings.
As shown in Figure 1, Fig. 1 is the composition structure of the fault-tolerant storage adaption system of the data redundancy of the invention based on cloud storage
Schematic diagram, by taking any of cloud storage stores domain as an example, which includes：The fault-tolerant storage management apparatus of data redundancy and service
Equipment, wherein, which includes management services module；The service equipment includes metadata service module and data service mould
Block.
Management services module is used to configuring the adaptation strategies of data redundancy fault-tolerant way in cloud storage, which can be with
It is configured according to storage domain, application, user, data object.Management services module is additionally operable to cloud storage Topology Management, i.e.,：Pipe
Reason service module can manage one or more storage domains.Storage domain has corresponded to a storage logical space definition, includes one
Or multiple cloud storage service devices.Adjustment data redundancy fault-tolerant way is performed using storing domain as base unit.Management services module is also
Include the management such as performance monitoring, fault warning management, system log, statistical analysis, object redundancy list for other.
Any of cloud storage storage domain is logically separated into metadata service module and data service module.Here,
For the relation of metadata and data, metadata can be understood as the data on " data ", i.e. metadata is for describing it
A kind of data of its data.And data can be understood as physical entity, metadata then there is provided each kind relevant to data
Property record, such as record user data physical distribution, for user data read and write Data Physical distributed intelligence etc., first number are provided
According to itself can also be as the data object being described.
Just because of there is the above-mentioned relation between " metadata " and " data ", therefore, in " metadata service module " and " number
According to service module " between, in metadata service module a data redundancy fault-tolerant mode is chosen according to the adaptation strategies of configuration
Afterwards, it can indicate that data service module also adjusts the data redundancy fault-tolerant way of itself.Being additionally, since data, there are different services
On device, each server will use at least identical redundancy fault-tolerant mode to same data, so, metadata service module
After the data redundancy fault-tolerant way that have adjusted itself, data service module can also adjust the data redundancy fault-tolerant way of itself, and
Response is returned into metadata service module, while metadata service module changes its corresponding metadata.
Metadata service module is used to reading the configuration data of management services module, including data redundancy fault-tolerant way and its
Adaptation strategies, and configured data redundancy fault-tolerant way is scanned by adaptation strategies, after have selected data redundancy fault-tolerant way, refer to
Show that data service module performs the corresponding adaptation adjustment of data redundancy fault-tolerant way.Metadata service module is additionally operable to record user
The physical distribution of data；Data Physical distributed intelligence is provided for user data read-write.
Data service module is used for the instruction according to metadata service module, adjusts the data redundancy fault-tolerant way of itself simultaneously
Response.Data service module is used for the data of physical store user in a distributed manner；Distributed Services are provided for user writable data；
Record system log, including physical equipment alarm, system operatio record etc..
As shown in Figure 1, the fault-tolerant storage management apparatus of data redundancy in Fig. 1 includes：Management services module.Such as Fig. 1 institutes
Show, the fault-tolerant storage service equipment of the data redundancy in Fig. 1 includes：Metadata service module and data service module, as this
Management services module, the description such as content described in said system of the metadata service module and the data service module,
Do not repeated secondary.
As shown in Fig. 2, Fig. 2 is the implementation method of the fault-tolerant storage adaptation method of the data redundancy of the invention based on cloud storage
Flow chart, this method comprise the following steps：
Step 101, management services module set data redundancy fault-tolerant way and its adaptation strategies.
Step 102, metadata service module read above-mentioned preset strategy, and carry out Scanning Detction.
Step 103, metadata service module judge whether to need to adjust data redundancy fault-tolerant way, if it is, performing
Step 104, otherwise, step 102 is performed.
Step 104, data service module receive the instruction of metadata service module, perform the suitable of data redundancy fault-tolerant way
Match somebody with somebody, and return to implementing result, metadata service module updates corresponding metadata.
Hereinafter, the adaptation in a manner of redundancy fault-tolerant is described by three kinds of concrete application scenes.
Application scenarios one, the access temperature for video frequency requested program broadcasting-like service carry out the adaptation of data redundancy fault-tolerant way
In the service of video frequency requested program broadcasting-like, the access temperature of each video file differs greatly.So-called temperature is evaluation one
The index of the visit capacity of a video file by multiple program request, the video file are exactly the hot spot networks video of user's common concern
Fragment.
Management services module is set the adaptation strategies of data redundancy fault-tolerant way by the application.
Such as, on the one hand, the access temperature of video file is set, is weighed with daily access times, more than the visit of setting
Ask number threshold values (such as 10 times), the data redundancy fault-tolerant way used is copy mode；Hot spot, the copy amount of use
Can be more, such as 2 copies are adjusted to 3 copies.To (referring to less than the redundancy fault-tolerant specification and foregoing 2 copies being adjusted to 3 copies
This specification) data, data service module can receive the instruction of metadata service module, perform data redundancy fault-tolerant way
Adaptation adjustment, and implementing result is returned, metadata service module updates corresponding metadata, because after data adaptation adjustment, member
Data are to record the description data that data change, and therefore, metadata is also required to update, and renewal is retouched for what data changed
State.
On the other hand, be not belonging to hot spot video file (evaluate whether for hot spot video file or be not belonging to hot spot
Video file, the index that can be used for：Such as the video that can be described as be not belonging to hot spot text of the daily access times less than 2 times
Part, is exactly the video file of hot spot in addition), such as more deserted video file, using extended control mode, more deserted video
File, the ratio of used verification data block is lower, for example, the parameter configuration of extended control mode is adjusted to 10+2 by 9+3.To height
In the data of the redundancy fault-tolerant specification (refer to the foregoing parameter configuration by extended control mode be adjusted to by 9+3 10+2 specification), data clothes
Business module can receive the instruction of metadata service module, perform the adaptation adjustment of data redundancy fault-tolerant way, and return and perform knot
Fruit, metadata service module update corresponding metadata.
Application scenarios two, for cloud storage equipment fault maintenance management data redundancy fault-tolerant way adaptation
Cloud storage uses general server apparatus, in daily engineering maintenance, can run into hard disk corruptions, memory node not
Can use etc. abnormal conditions, as the time elapses, ageing equipment situation is also changing, and physical store environment is to data redundancy fault-tolerant
It is required that therefore also changing.Particularly Backup Data often negligence of management, the adaptation strategies of configuration data redundancy fault-tolerant mode, are fitted
When adjust automatically data redundancy fault-tolerant way it is just very valuable.
Management services module is set the adaptation strategies of data redundancy fault-tolerant way by the storage domain.
For example, setting a storage domain to reserve 2 secure data blocks more than needed (refers to that reduce can vitiable redundant data block
Afterwards, it is also act against the data number of blocks of damage), then counted according to equipment fault, in a period of time when small (such as 24), this is deposited
It is 2 to store up the hard disk quantity damaged at the same time in domain, then needs to store 4 parts of data for the data.If using copy mode, it is provided with
4 copies；If using extended control mode, check block is provided with more than 4 pieces.To (referring to foregoing using secondary less than the redundancy fault-tolerant specification
The manner, is provided with 4 copies；If using extended control mode, be provided with the specification of more than 4 pieces of check block) data, cloud storage number
It can receive the instruction of metadata service module according to service module, perform the adaptation adjustment of data redundancy fault-tolerant way, and return and hold
Row is as a result, metadata service module updates corresponding metadata.
Application scenarios three, the adaptation for the few data redundancy fault-tolerant way of remaining amount of physical memory
Although cloud storage system can increase according to capacity requirement amount, elastic on-line rapid estimation is carried out, sometime
(after referring to cloud storage some engineering construction, there is a definite software and hardware configuration moment), itself still there is a capacity
The upper limit.When the increase of capacity requirement amount, remaining amount of physical memory is few, and system can alert, and for some reason cannot be timely
Online real-time dilatation, it is possible to win the time of online dilatation in real time using the adaptation adjustment of cloud storage redundancy fault-tolerant mode.
Management services module is set the adaptation strategies of data redundancy fault-tolerant way by the storage domain.
For example, when setting a storage domain residual memory space less than 10%, 3 number is more than for secure data block of having more than needed
According to secure data number of blocks more than needed is reduced to 2, other remain unchanged；For temperature is relatively low, the number that is stored using copy mode
According to using the extended control mode of memory space utilization rate higher.
In management services module, the redundancy list of data object can be checked, specified data object can also be configured
The adaptation strategies of data redundancy fault-tolerant way., can be by for example, data object index is according to storing directory or data file class etc.
Adaptation strategies are specified according to file directory and/or type.
Management services module can also carry out ruling to the adaptation strategies of overlapping data redundancy fault-tolerant way, can be certainly
Dynamic or prompting manually selects the adaptation strategies of data redundancy fault-tolerant way.For example, the secure data block more than needed according to user setting
Quantity is 2, but it is 3 that the user's data object is provided with secure data number of blocks more than needed again, then can set system automatic cutting
Certainly, handled by the strategy of safer mode, scope smaller, i.e., to the data object of the user according to safety more than needed
Data number of blocks is 3 processing.Judge to need to carry out adaptation strategies adjustment for another example working as system, but be related to the more (example of data object
Such as, unnecessary 100 data files), when may influence online service performance, administrator can be reminded to intervene control, systems and management
Member's session includes whether interaction needs adjustment now or set adjustment time；Whether under catalogue All Files perform adjustment or
Person is to adjust this document redundancy strategy, such etc..
In conclusion the present invention provides when using cloud storage, be adjusted flexibly the system of data redundancy fault-tolerant way with
Method.A variety of data redundancy fault-tolerant ways can be taken into account using the present invention, contrast the different implementations of extended control mode and copy mode
Understand：Extended control mode possesses flexible parameter configuration and (supports 9+3 as configurable, a variety of extended control modes such as 10+2,10+6, meet different
The needs of reliability and disk utilization), configured according to different parameters and support different memory space utilization rates and different numbers
According to reliability requirement.Extended control mode is improving availability of data, has a clear superiority in terms of saving memory space, but reliable in data
Property aspect, data reading performance using redundancy and save calculate processing expense in terms of or copy mode have a clear superiority.And copy mode can
To possess higher reading efficiency, save and calculate processing expense, but memory space utilization rate is not high.And the present invention is more due to that can take into account
Kind data redundancy fault-tolerant way, in the performance and unbalanced space utilisation of cloud storage system, is adapted to application scenarios
Need and online adaptation adjustment in time, so as to select an optimal data redundancy fault-tolerant way.It is single than using the prior art
Data redundancy fault-tolerant way there is no washability, have a clear superiority.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention.
Claims (14)
1. the fault-tolerant storage adaptation management equipment of a kind of data redundancy, it is characterised in that the equipment includes：Management services module,
For configuring data redundancy fault-tolerant way and its adaptation strategies in cloud storage, by the data redundancy fault-tolerant way and its adaptation
Strategy is supplied to service side to access and indicates the data redundancy fault-tolerant way that the selection of service side is adapted, wherein, the data are superfluous
Remaining fault-tolerant way includes at least：Copy mode and extended control mode；
The management services module is additionally operable to carry out ruling to the adaptation strategies of overlapping data redundancy fault-tolerant way, automatically or carries
Show the adaptation strategies for manually selecting data redundancy fault-tolerant way；
The management services module, is further used for being matched somebody with somebody according to storage domain, and/or application, and/or user, and/or data object
Put the data redundancy fault-tolerant way and its adaptation strategies.
2. equipment according to claim 1, it is characterised in that the management services module, is further used in video point
Broadcast when the data redundancy fault-tolerant way and its adaptation strategies are configured under the application scenarios of class service, to hot video file, need
Configuration selection copy mode；To non-hot video file, configuration selection correcting and eleting codes extended control mode is needed.
3. equipment according to claim 1, it is characterised in that the management services module, is further used in equipment event
When configuring the data redundancy fault-tolerant way and its adaptation strategies under the application scenarios of barrier maintenance management, a storage domain pair is configured
Multiple secure data blocks more than needed should be reserved, and configure copy mode and extended control mode is all suitable for.
4. equipment according to claim 1, it is characterised in that the management services module, is further used in residue
When reason insufficient memory causes to configure the data redundancy fault-tolerant way and its adaptation strategies under the application scenarios of system alarm,
The quantity that one storage domain of configuration corresponds to reserved multiple secure data blocks more than needed needs to reduce, and configures copy mode and EC side
Formula is all suitable for；When detecting that its correspondence temperature of the data block stored in a manner of copy reduces, it is changed to select extended control mode.
5. the fault-tolerant storage adaptation services equipment of a kind of data redundancy, it is characterised in that the equipment includes：Metadata Service mould
Block, data service module；Wherein,
The metadata service module, for Access Management Access side and reads the data redundancy fault-tolerant way of configuration and its is adapted to plan
Slightly, select the data redundancy fault-tolerant way that is adapted and adaptation adjustment is carried out to the data redundancy fault-tolerant way of itself；It will fit
Instruction with adjustment its data redundancy fault-tolerant mode is sent to the data service module, wherein, the data redundancy is fault-tolerant
Mode includes at least：Copy mode and extended control mode；
The data service module, for the instruction according to reception, correspondingly adjusts the data redundancy fault-tolerant way of itself.
6. equipment according to claim 5, it is characterised in that the data service module, is further used for adjusting itself
Data redundancy fault-tolerant way after, returning response gives the metadata service module；
The metadata service module, is further used for after receiving the response, updates corresponding metadata.
7. the fault-tolerant storage adaption system of a kind of data redundancy, it is characterised in that the system includes：The management service mould of management-side
Block, services the metadata service module and data service module of side；Wherein,
The management services module, for configuring data redundancy fault-tolerant way and its adaptation strategies in cloud storage, by the number
It is supplied to service side to access according to redundancy fault-tolerant mode and its adaptation strategies and indicates that the data redundancy that the selection of service side is adapted holds
Wrong mode；The management services module is additionally operable to carry out ruling to the adaptation strategies of overlapping data redundancy fault-tolerant way, automatically
Or prompting manually selects the adaptation strategies of data redundancy fault-tolerant way, wherein, the data redundancy fault-tolerant way includes at least：It is secondary
The manner and extended control mode；
The metadata service module, for Access Management Access side and reads the data redundancy fault-tolerant way and its adaptation of configuration
Strategy, selects the data redundancy fault-tolerant way that is adapted and carries out adaptation adjustment to the data redundancy fault-tolerant way of itself；By
The instruction of adaptation adjustment its data redundancy fault-tolerant mode is sent to the data service module；
The data service module, for the instruction according to reception, correspondingly adjusts the data redundancy fault-tolerant way of itself.
8. system according to claim 7, it is characterised in that the management services module, is further used for according to storage
Domain, and/or application, and/or user, and/or data object configure the data redundancy fault-tolerant way and its adaptation strategies.
9. the system according to claim 7 or 8, it is characterised in that the management services module, is further used in video
When the data redundancy fault-tolerant way and its adaptation strategies are configured under the application scenarios of program request class service, to hot video file,
Need configuration selection copy mode；To non-hot video file, configuration selection extended control mode is needed.
10. the system according to claim 7 or 8, it is characterised in that the management services module, is further used for setting
When configuring the data redundancy fault-tolerant way and its adaptation strategies under the application scenarios of standby Breakdown Maintenance management, a storage is configured
Domain, which corresponds to, reserves multiple secure data blocks more than needed, and configures copy mode and extended control mode is all suitable for.
11. the system according to claim 7 or 8, it is characterised in that the management services module, is further used for surplus
Excess reason insufficient memory causes to configure the data redundancy fault-tolerant way and its adaptation plan under the application scenarios of system alarm
When slightly, configuring the quantity that a storage domain corresponds to reserved multiple secure data blocks more than needed needs to reduce, and configures copy mode
It is all suitable for extended control mode；When detecting that its correspondence temperature of the data block stored in a manner of copy reduces, it is changed to select extended control mode.
12. the fault-tolerant storage adaptation method of a kind of data redundancy, it is characterised in that this method includes：Management services module configures
Data redundancy fault-tolerant way and its adaptation strategies in cloud storage；
Metadata service module Access Management Access service module and the data redundancy fault-tolerant way and its adaptation plan for reading configuration
Slightly, select the data redundancy fault-tolerant way that is adapted and adaptation adjustment is carried out to the data redundancy fault-tolerant way of itself, wherein, institute
Data redundancy fault-tolerant way is stated to include at least：Copy mode and extended control mode；
The instruction for being adapted to adjustment its data redundancy fault-tolerant mode is sent to data service module；Data service module according to
The instruction received, correspondingly adjusts the data redundancy fault-tolerant way of itself.
13. according to the method for claim 12, it is characterised in that the data redundancy fault-tolerant way in the configuration cloud storage
And its adaptation strategies specifically include：The data are configured according to storage domain, and/or application, and/or user, and/or data object
Redundancy fault-tolerant mode and its adaptation strategies.
14. the method according to claim 12 or 13, it is characterised in that this method further includes：Data service module adjusts
After the data redundancy fault-tolerant way of itself, returning response gives the metadata service module；Metadata service module receives institute
After stating response, corresponding metadata is updated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210305197.0A CN103631666B (en) | 2012-08-24 | 2012-08-24 | The fault-tolerant adaptation management equipment of data redundancy, service equipment, system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210305197.0A CN103631666B (en) | 2012-08-24 | 2012-08-24 | The fault-tolerant adaptation management equipment of data redundancy, service equipment, system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103631666A CN103631666A (en) | 2014-03-12 |
| CN103631666B true CN103631666B (en) | 2018-04-20 |
Family
ID=50212755
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210305197.0A Active CN103631666B (en) | 2012-08-24 | 2012-08-24 | The fault-tolerant adaptation management equipment of data redundancy, service equipment, system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103631666B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103942116B (en) * | 2014-04-24 | 2017-02-01 | 深圳市中博科创信息技术有限公司 | Method for conducting fault-tolerance processing on data and data processing system |
| CN104281533B (en) * | 2014-09-18 | 2018-03-20 | 深圳市中博科创信息技术有限公司 | A kind of method and device of data storage |
| CN105095352B (en) * | 2015-06-19 | 2019-03-05 | 曙光信息产业股份有限公司 | Data processing method and device applied to distributed system |
| CN106791863B (en) * | 2015-11-19 | 2019-07-16 | 浙江大华技术股份有限公司 | A kind of storage method and device of SVC video data |
| CN105630418A (en) * | 2015-12-24 | 2016-06-01 | 曙光信息产业(北京)有限公司 | Data storage method and device |
| CN106202396A (en) * | 2016-07-08 | 2016-12-07 | 乐视控股（北京）有限公司 | Object storage method and object storage system |
| CN106681656A (en) * | 2016-11-16 | 2017-05-17 | 南京云创大数据科技股份有限公司 | Dynamically-adjustable cloud storage system fault tolerance method |
| CN108319618B (en) * | 2017-01-17 | 2022-05-06 | 阿里巴巴集团控股有限公司 | Data distribution control method, system and device of distributed storage system |
| CN109150792B (en) | 2017-06-15 | 2022-03-08 | 杭州海康威视数字技术股份有限公司 | Method and device for improving data storage security |
| CN112015583B (en) | 2017-12-20 | 2022-06-28 | 华为云计算技术有限公司 | Data storage method, device and system |
| CN110389715B (en) * | 2018-04-23 | 2021-07-09 | 杭州海康威视系统技术有限公司 | Data storage method, storage server and cloud storage system |
| CN111399755A (en) * | 2019-09-10 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Data storage management method and device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201726424U (en) * | 2009-08-18 | 2011-01-26 | 升东网络科技发展(上海)有限公司 | Distributed storage system |
| CN102622184A (en) * | 2011-01-27 | 2012-08-01 | 北京东方广视科技股份有限公司 | Data storage system and method |
| CN102088491B (en) * | 2011-02-01 | 2013-06-26 | 西安建筑科技大学 | Distributed storage oriented cloud storage security architecture and data access method thereof |
| CN102270161B (en) * | 2011-06-09 | 2013-03-20 | 华中科技大学 | Methods for storing, reading and recovering erasure code-based multistage fault-tolerant data |
-
2012
- 2012-08-24 CN CN201210305197.0A patent/CN103631666B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN103631666A (en) | 2014-03-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103631666B (en) | The fault-tolerant adaptation management equipment of data redundancy, service equipment, system and method | |
| CN108780386B (en) | Data storage method, device and system | |
| US8612284B1 (en) | Quality of service differentiated cloud storage | |
| CN102546256B (en) | System and method used for monitoring cloud computation service | |
| US20220215875A1 (en) | Multi-level cell page writes | |
| CN102882909B (en) | Cloud computing service monitoring system and method thereof | |
| CN106933868B (en) | Method for adjusting data fragment distribution and data server | |
| CN103607424B (en) | Server connection method and server system | |
| WO2007028679A1 (en) | Operational risk control apparatus and method for data processing | |
| CN105095103B (en) | For the storage device management method and apparatus under cloud environment | |
| CN105072201A (en) | Distributed storage system and storage quality control method and device thereof | |
| CN108319618B (en) | Data distribution control method, system and device of distributed storage system | |
| CN110198346B (en) | Data reading method and device, electronic equipment and readable storage medium | |
| CN110445822A (en) | A kind of object storage method and device | |
| CN104243195A (en) | Remote disaster recovery processing method and device | |
| CN109117292A (en) | A kind of method and apparatus and cluster storage system of cluster-based storage | |
| CN112130761A (en) | Capacity expansion method and system for storage system and related device | |
| CN110291505A (en) | Reduce the recovery time of application | |
| CN109510730A (en) | Distributed system and its monitoring method, device, electronic equipment and storage medium | |
| CN102340437A (en) | Network fault tolerance method and device | |
| CN103685359B (en) | Data processing method and device | |
| CN115604294A (en) | Method and device for managing storage resources | |
| CN115756955A (en) | Data backup and data recovery method and device and computer equipment | |
| KR101527058B1 (en) | Distributed file management apparatus and method | |
| KR101725192B1 (en) | Resource information management and data storage system through the management of the resource template |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |