CN103201736A - Method and system for extending data storage system functions - Google Patents
Method and system for extending data storage system functions Download PDFInfo
- Publication number
- CN103201736A CN103201736A CN2011800509497A CN201180050949A CN103201736A CN 103201736 A CN103201736 A CN 103201736A CN 2011800509497 A CN2011800509497 A CN 2011800509497A CN 201180050949 A CN201180050949 A CN 201180050949A CN 103201736 A CN103201736 A CN 103201736A
- Authority
- CN
- China
- Prior art keywords
- data
- storage
- intercepting
- blocker
- storage system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/16—Protection against loss of memory contents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/188—Virtual file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/16—Error detection or correction of the data by redundancy in hardware
- G06F11/20—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
- G06F11/2053—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant
- G06F11/2056—Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements where persistent mass storage functionality or persistent mass storage control functionality is redundant by mirroring
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A method and system for extending the functionality of a data storage system, the data storage system including a data organization means and a data storage resource, the method including the steps of dissociating the data storage functions of the data storage system from other functions of the data storage system and transferring at least a portion of said data storage functions to an intercepting system.
Description
The application requires to enjoy and comes from the U.S. Provisional Application No.61/376 that submitted on August 25th, 2010, and 905 right of priority, the content of this provisional application are all incorporated this paper by reference into.
Technical field
The present invention relates generally to data-storage system, and relate more specifically to a kind of method and system of the function for the growth data storage system.
Background technology
File system on the computing machine makes operation system of computer can serve as believable third party, and it implements safety and name agreement between the process that communicates, even is not also to be like this under the simultaneously movable situation in these processes.A function of file system is to the data layout catalogue that offers it and organizes, makes described data can be retrieved after a while.In order to carry out this function, file system must managing memory source.For file system, general storage resources shows as continuous byte sequence in the target storage system for example, continuous piece sequence or is the key value storage in essence.
File system is the part of the operating system on the computing machine normally, but can also be as the expansion of operating system in application, perhaps as the pure application of using the client file system protocol by the network connection visit.These various examples are well known in the art.In all situations, file system all is designed to be used in the storage resources (for example, disk) of particular types and will has the constraint relevant with this storage resources.For example, these constraints can be relevant with performance, capacity, concurrency, scalability, physical location etc.The feature of its data of storing being operated in view of the permission of file system, in file system self, also there is extra constraint, for example with when not having in advance encryptions to be provided, to compress, to copy or to comprise other variation of processing (data polymorphism) of context-sensitive to file system, use these functions.In the art, the phrase context-sensitive typically refers to the performance of program of making according to you that changes in program.For example, the documents and materials provide you to be in this special characteristic of use are provided context-sensitive, and the context of data or context-sensitive are handled and allowed according to will how to use data and will where use data to come differently deal with data.
Use these some intrafascicular approximately existing systems that copy and method to use virtual unit, described virtual unit is that file system is created the virtual store resource that will use.Owing to lack and be present in the file system and available contexts that be not sent to storage resources, this method can not provide the context-sensitive of data to handle, and can not remove since on the data seen of file system and the storage resources 1:1 of in esse data (for example, total data storage capacities) shine upon and the constraint that causes.
Thereby, an object of the present invention is to provide a kind of innovative system and method for the function for the growth data storage system, for example the function of growth data storage system is handled with the data that allow context-sensitive.
Summary of the invention
According to one embodiment of the invention, a kind of method of the function for the growth data storage system is provided, described data-storage system comprises data molded tissue block and data storage resource, and described method comprises: the data storage function of described data-storage system is separated with other function of described data-storage system; And at least a portion of described data storage function transferred to intercepting system.
Scheme according to the present invention, described intercepting system are to give (complimentary) storage resources.Described data molded tissue block can be selected from the group that comprises file system, key value storage and database.
Another program according to the present invention, separating step is carried out by the blocker module is provided, and described blocker module and described data molded tissue block, described data storage resource and described intercepting system communicate; Described blocker module interception file system operation is to be handled or to be handled by described intercepting system by described data storage resource with the function of determining described operation.
Another program according to the present invention, described blocker module will be tackled described operation when file system operation still has context and in described operation otherwise before will being broken down into the independent operation that is suitable for described data storage resource.
Another program according to the present invention, another part at least and the described data-storage system of described data storage function are retained in together.Another program according to the present invention is transferred to described intercepting system with described data storage function whole.
Another program according to the present invention, described data-storage system are to select from the group that comprises file system, key value storage, object storage and procotol.
Another program according to the present invention, described data-storage system are shared between a plurality of external interfaces.
Another program according to the present invention, described blocker module comprise the user space application with the facility cooperation of operating system or file system.Another program according to the present invention, described blocker module is included in the file system protocol agent application of carrying out on the network.Another program according to the present invention, described blocker module comprises the microfilter driver, it is used for tackling file system operation at operating system kernel.
Another program according to the present invention, described intercepting system use one or more storage resources of giving to carry out its function, and the described storage resources of giving and described storage resources are independent.
Another program according to the present invention, described intercepting system is realized the capacity extension of described data-storage system.Another program according to the present invention, described intercepting system improve the performance of described data-storage system by the one or more characteristics that change the data on the described storage resources.Described one or more characteristic is preferably selected from the group that comprises storage format, memory location and storage order.
Another program according to the present invention, described method also comprises: carry out the step that cancellation repeats by described intercepting system.Another program according to the present invention, described method also comprises: carry out the polymorphic step of data by described intercepting system.Another program according to the present invention, described method also comprises: by the step of described intercepting system realization at the independent access controlling mechanism of data.Another program according to the present invention, described method also comprise the step of data being carried out Version Control by described intercepting system.
Another program according to the present invention, described method also comprises: realize in the single-stage buffer memory of described data-storage system or the multi-level buffer; Wherein, described performing step is carried out by described blocker module.Another program according to the present invention, described method also comprises: be pushed to data-storage system at a distance and be drawn near data-storage system by the data with more use by the data with less use and realize local optimum; Wherein, this performing step is carried out by described blocker module.The data-storage system of described distant place comprises data-storage system a long way off or that need the more time visit physically.
Another program according to the present invention, described method also comprises: realize the virtualized step based on title, described virtual by making that with reference to the data on other data-storage system Invalid filename claims to look like effectively in the current data storage system based on title; Wherein, this performing step is carried out by the blocker module.
Another program according to the present invention, described method also comprises: realize data backup and data copy step; Wherein, this performing step is carried out by described blocker module.
Another program according to the present invention, described method also comprises: realize the step of data virtualization, described data virtualization comprises that the data that allow same names are physically located in the different data-storage systems; Wherein, this performing step is carried out by described blocker module.
Another program according to the present invention, described method also comprises: with compare at described storage resources place, provide different abilities at described intercepting system place for selected data, wherein, described different ability is from comprising that Performance Characteristics, cancellation repeat, data are polymorphic, independent access controlling mechanism, Version Control, buffer memory, localizing, copy the group with data virtualization and select.Preferably, use selection mechanism to identify selected data, described selection mechanism adopts one or the multinomial metadata model coupling of selecting from the group that comprises the following: title, timestamp, size, historical information, physical message and contextual information.
According to another embodiment of the present invention, a kind of system of the function for the growth data storage system is provided, described data-storage system comprises data molded tissue block and data storage resource, and described system comprises: be used for the separation module that the data storage function with described data-storage system separates with other function of described data-storage system; And the module that is used at least a portion of described data storage function is transferred to intercepting system.
According to the scheme of this embodiment, described intercepting system can be the storage resources of giving.
According to another program of this embodiment, described separation module comprises blocker, and described blocker and described data molded tissue block, described data storage resource and described intercepting system communicate; Described blocker is used for the interception file system operation should be handled or be handled by described intercepting system with the function of determining described operation by described data storage resource.
According to another program of this embodiment, described blocker will be tackled described operation when file system operation still has context and in described operation otherwise before will being broken down into the independent operation that is suitable for described data storage resource.
According to another program of this embodiment, another part at least and the described data-storage system of described data storage function are retained in together.Replacedly, described data storage function whole are transferred to described intercepting system.
According to another program of this embodiment, described data-storage system is to select from the group that comprises file system, key value storage, object storage and procotol.
According to another program of this embodiment, described data-storage system is shared between a plurality of external interfaces.
According to another program of this embodiment, described blocker comprises the user space application with the facility cooperation of operating system or file system.
According to another program of this embodiment, described blocker is included in the file system protocol agent application of carrying out on the network.
According to another program of this embodiment, described blocker comprises the microfilter driver, is used for tackling file system operation at operating system kernel.
According to another program of this embodiment, described intercepting system uses one or more storage resources of giving to carry out its function, and the described storage resources of giving and described storage resources are independent.
According to another program of this embodiment, described intercepting system is realized the capacity extension of described data-storage system.
According to another program of this embodiment, described intercepting system improves the performance of described data-storage system by the one or more characteristics that change the data on the described storage resources.
According to another program of this embodiment, described one or more characteristics are to select from the group that comprises storage format, memory location and storage order.
According to another program of this embodiment, described intercepting system is used for carrying out the cancellation repetition of data.
According to another program of this embodiment, described intercepting system is used for carrying out the polymorphic of data.
According to another program of this embodiment, described intercepting system is used for carrying out the independent access controlling mechanism at data.
According to another program of this embodiment, described intercepting system is used for carrying out the Version Control of data.
According to another program of this embodiment, described blocker is used for carrying out the single-stage buffer memory of described data-storage system or of multi-level buffer.
According to another program of this embodiment, described blocker is carried out local optimum for being pushed to data-storage system at a distance by the data with less use and being drawn near data-storage system by the data with more use.
According to another program of this embodiment, the data-storage system of described distant place comprises physically a long way off or needs the data-storage system that the more time conducts interviews.
Another program according to this embodiment, described blocker be used for to be carried out virtual based on title, described virtual by making that with reference to the data on other data-storage system Invalid filename claims to look like effectively in the current data storage system based on title.
According to another program of this embodiment, described blocker is for carrying out one that data backup and data copy.
According to another program of this embodiment, it is virtual that described blocker is used for carrying out data, and described data virtualization comprises that the data that allow same names are physically located in the different data-storage systems.
According to another program of this embodiment, described intercepting system be used for to carry out from the group that comprises the following select one or more: improve that Performance Characteristics, cancellation repeat, data are polymorphic, independent access controlling mechanism, Version Control, buffer memory, localize, copy and data virtualization.
According to another program of this embodiment, described data molded tissue block is to select from the group that comprises file system, key value storage and database.
Description of drawings
Only by the mode of example embodiment is described referring now to accompanying drawing, in the accompanying drawings:
Fig. 1 shows the high level architecture according to system of the present invention;
Fig. 2 shows and can use and/or realize computer system of the present invention;
Fig. 3 shows the embodiment of the method according to this invention.
Embodiment
The invention provides a kind ofly for imitation (affect) all innovative system and method for the constraint relevant with data, the described constraint relevant with data includes but not limited to those constraints that can use the virtual store resource to imitate.The present invention can provide this function for the data-storage system except file system, described data-storage system provides the more senior abstract main interface that is used as them on the actual storage resource, for example database and specific object database, key value storage, some procotol and some shared data system.
In addition, the present invention allows all traditional constraints of data storage system are changed, and does not need the cooperation of file system itself or file system itself is changed.Thereby, the present invention does not need to revise fully or realize new file system, and under the situation of needs exploitation new file system, perhaps do not needing to develop under the situation of new file system, allowing realization advantage described herein and expanded function in the interface between legacy system and new system on the existing system and in the process of the new system of exploitation.Broadly, the invention provides a kind of intercepting system that is attached to file system, described intercepting system can optionally be tackled these file system operations when these file system operations still comprise contextual information before file system operation is broken down into the operation that is suitable for storage resources.That is, context operation and the instruction that is used for carrying out these operations that is provided by operation system of computer are provided file system.When file system and storage resources were mutual, these contexts operations and instruction were lost, and this is because file system-storage resources only retrieval of focused data, storage and catalog compiling alternately.Intercepting system according to the present invention provides the function of file system at intercept point place needs, and at described intercept point place, the operation of intercepting system is transparent for file system.Therefore, the invention provides a kind of method and system, described method and system is by making the storage responsibility of data-storage system separate with its other function (for example file designation, lock, share, safety etc.), and make actual storage responsibility be carried out by the assembly that separates alternatively, come the function of growth data storage resources and system.
With reference to figure 1, show one embodiment of the present of invention, wherein, show data-storage system 100, blocker 200 and intercepting system 300.Hereinafter describe the details of preferred embodiment, in a preferred embodiment, the various distribution and data-storage system of the function of existing data-storage system separate itself, and are divided into by data-storage system or intercepting system and carry out.In the art, it is novel that the function separation of novelty and the subsequent allocations of function between data-storage system 100 and intercepting system 300 are considered to, and especially provides the function between data-storage system 100 and the intercepting system 300 to distribute in non-co-operative environment.Yet, though the preferred embodiments of the present invention relate to the additional mechanism (add-on mechanism) of existing non-cooperation data-storage system, but it also can be to provide extendability so that architectural features expansion and function enhancing to be provided to the cooperation data-storage system, hereinafter will describe the example about it in more detail.
With reference to figure 3, show another embodiment of the present invention, in this embodiment, a kind of method of the function for the growth data storage system is provided, comprise in the following manner other function separation steps 310 with data storage function and the described data-storage system of data-storage system: tackle 315 file system operations and should be handled still by data storage resource with the function of determining described operation and handled by intercepting system, and at least a portion of described data storage function is shifted 320 to described intercepting system.Replacedly, the data storage function that is blocked continues to arrive data storage resource 325.
The present invention moves in the context of computer system usually, and is used for providing the expansion of the data storage capacities that is associated with general computer system, figure 2 illustrates an exemplary computer system.As directed, computer system 20 has a plurality of physics and logic module, comprises CPU (central processing unit) (" CPU ") 24, random access memory (" RAM ") 28, I/O (" I/O ") interface 32, network interface 36, nonvolatile memory 40 and makes the local bus 44 that CPU24 can communicate with other assembly.CPU24 executive operating system and a plurality of software systems.RAM28 provides responsive relatively (relatively-responsive) volatile memory to CPU24.I/O interface 32 allows from one or more equipment (for example keyboard, mouse etc.) reception input and to output device (for example display and/or loudspeaker) output information.Network interface 36 permissions are communicated by letter with other system.Nonvolatile memory 40 storage operating systems and program.In the operating period of computer system 20, can obtain operating system, program and data from nonvolatile memory 40, and they are put into RAM28 to help execution.
Data-storage system 100 is normally known in the art, and for the purposes of the present invention, data-storage system 100 is any machine, equipment or device, and can use given identifier (for example filename) to store data, and can obtain at least a portion of described data after a while by this identifier desirably.Data-storage system 100 comprises data molded tissue block 105, and but to use in operation be the independently Embedded machine of physics or module (be referred to herein as storage resources 110(and also be illustrated as nonvolatile memory 40 in Fig. 2) alternatively) store data according to the characteristic of storage resources with primitive form, data-storage system itself can obtain the data of storing from storage resources after a while.Data-storage system 100 can be shared between a plurality of external interfaces.
The representative example of data molded tissue block 105 comprises file system, key value storage, database or level other machine on these, for example webpage buffer memory and pagefile, combination with it.The representative example of storage resources 110 comprises physical disks, the storer such as RAM, RAID array, paper tape, document etc.In the representative example of preferred embodiment, storage system 100 can be Windows
TMNew technology file system, and storage resources can be hard disk drive.
According to representative example discussed above, blocker module 200 is microfilter or conventional filter driver, and it is designed and otherwise is arranged at Windows
TMTackle file system operation at suitable layer place in the operating system kernel.This driver can be realized independently, perhaps this driver can be obtained commercially.In other example, can use application layer file system supplier's (for example file system in the FUSE(user's space), be used for STREAMS or the physics Transparent Proxy machine of network interception) provide interception mechanism by the facility (such as but not limited to the file system stack) that is structured in other operating environment.The blocker module can also be the software module that has merged the Custom Design of function described herein.Blocker module 200(is called blocker convertibly) can also be the file system protocol agent application.
Engaging between blocker module 200 management data storage systems 100 and the intercepting system 300.Some operation sense interest of 300 pairs of data storage systems of intercepting system.In representative example, this can be all data I/O, in order to data are redirected to another memory device.Intercepting system 300 can be configured to interception mechanism all data I/O operations laterally are delivered to intercepting system 300, rather than these operations is delivered to the low layer of data-storage system 100 as traditionally.
When receiving the operation that is blocked, intercepting system 300 has a plurality of options: (a) intercepting system 300 can determine that it does not handle this operation, and indication blocker module 200 continues, just look like not exist interception the same, make the blocker module this operation is delivered to the low layer of data-storage system 100; (b) intercepting system 300 can determine that it need revise this operation before the low layer that this operation is delivered to data-storage system 100; Perhaps (c) intercepting system 300 can determine itself to be responsible for handling this operation, and take over the responsibility at it, in this case, interception mechanism 200 must suitably operate high level and low layer to the data storage system to react at this, comprises all mistake processing.
Intercepting system 300 can be the hardware device of any amount, the storage resources of for example giving, multimedia expander, multimedia server, home server, specific to the controller of using or can carry out various tasks any similar system of (comprising being carried out by storage resources usually of task).
According to the present invention, blocker module 200 is provided, by blocker module 200, intercepting system can namely in the unrecognizable mode of data-storage system, be responsible for being blocked operation pellucidly.When data storage resource separated with file system, this allowed to use new function to expand existing system.On the contrary, the existing interlude (for example virus checking device and encryption pad (shim)) of carrying out some data interception functions uses original data-storage system that their function is provided.
The present invention also provides a kind of existing data-storage system, and it separates the data of storing or position and the data-storage system of metadata, keeps the original semanteme of this data-storage system simultaneously.Thereby, data-storage system is responsible available data or metadata of having stored still, but intercepting system can be taken at any data new or that revise or the responsibility of metadata, as long as the external semantic of available data storage system remains unchanged according to its oneself strategy.
In the representative example of Miao Shuing, original file systems still is responsible for all metadata, and intercepting system is taken over the responsibility at some or all data in the text.Particularly, the present invention allows to keep new technology file system metadata (content that comprises catalogue, MFT (MFT) and each MFT item) at the storage resources of file system, just as what take place under the normal condition, but data itself can be operated individually, and are independent of the configuration of primary data storage system and are stored.
The invention provides a plurality of advantages, be included as traditional unmodified NTFS and realize providing a kind of and support modern file system feature (for example copying, cancel repetition, buffer memory, merging (pooling)) and usually by the mode of storage resources (RAID rank) feature that provides and new specific characteristic.The present invention is to make it become possibility with the existing mode of using compatibility, and this is to have presented unmodified semanteme because of the applications interface in file system.That is, file system remains unchanged, and can be used by operating system, and does not need to revise.
An existing example attempting to solve similar problem solved by the invention is Microsoft
TMWindows home server project (WHS)-at domestic consumer's the data-storage system with simple extensibility.WHS initially comprises and is called the function that drives expander, it expands NTFS in the Windows system by using normal form based on file, namely whole file can be placed on the different destination new technology file systems, and leave with reference to (" gravestone ") at original new technology file system.This method is failed, this is not transparent fully (left reference is different with the source document performance) because of it for application, and in fact, the function that the driving expander is just attempting to provide initially is to use dummy block will storage resources model (that is, by looking like storage resources for file system) to realize again.Removed this function fully from WHS subsequently.Other example of original driving expander method is clearly in the art, for example in the hierarchical storage management system.On the other hand, the present invention allows file system semantics and metadata to keep with the original storage resource, and is cited as required, but data itself are independently operated and stored, and does not need the constraint of the configuration of primary data storage system.
Though the example of aforementioned paragraphs is provided at disk drive storage system, also can have used method of the present invention at non-cooperation file system being expanded to other non-cooperation data-storage system (for example key value storage, object storage, various Cache, database etc.).Compare with the design of primary data storage system, providing independently, individually or even only use extra processing to come now just can use the available data storage system to provide a plurality of new features and corresponding function to application under the situation of ability of locator data.The applicant thinks that prior art does not have open following method and system, described method and system is used for that data are stored responsibility and separates with other function of data-storage system, thereby the function of growth data storage system is to exceed the constraint of file system and data-storage system itself.
An advantage incorporating system and a method according to the invention into is: possible performance improves.The data that redirect can be stored in than the employed storage resources of primary data storage system faster on the storage resources, perhaps it can be stored to cause the quicker mode that reads and/or write data, perhaps can side by side use this two mechanism.For example, can keep the metadata of storage system in the original storage resource, but data itself are stored in faster on the storage resources.In this example, blocker module policy paper system is from original storage resource acquisition metadata, but obtains data from storage resources faster.Subsequently information and data are being delivered to file system when handling for operating system, data seem similarly to be that it is directly from the original storage resource acquisition, but data are obtained fasterly.For example can also improve performance by on the secondary storage resource data being carried out buffer memory, described secondary storage resource may be quicker or be had a characteristic of other improvement.
In one embodiment of the invention, the data that redirect are stored in the sparse file on the quick storage resource more.Sparse file is the file that comprises the hole, in described hole, does not have distribute data at storage resources, but data are represented virtually.A problem in the file cache system is to represent the file that is buffered with the effective form in space, needs to create data packing module usually and store real data, and change between internal file position and the place of storing at the data of these positions.For the rank with son file as required data are carried out buffer memory, the data of not obtaining from source document should be represented virtually rather than physically be represented as yet.Use sparse file to allow to support that any file system of sparse file provides this function, and exempted and to have developed and to provide this function in the present invention.The applicant believes that it is novel using sparse file to come the data of buffer memory actual file, and those skilled in the art will know the ability of using existing force that this built-in function of son file buffer is provided.
The present invention can also be used in the data compression context, in the data compression context, and packed data in every way, described variety of way is not subjected to the restriction of the characteristic of primary data storage system.For example, the stream of standard compression or cancellation repeated compression, if wherein data Already in the system somewhere, do not store these data so again.Use the instruction of prior art, such data compression can not directly occur on the aspect of data-storage system usually.Those skilled in the art will know the different advantages that data compression is provided in any file system and data storage resource system, and no matter the file system of using and data storage resource system how.
Another possible application is in data are polymorphic, and in data were polymorphic, according to the context of planning to use data, data can show differently.Though the polymorphic application of various data is known in the art, these are normally carried out in application layer, and are subjected to the constraint of data-storage system.The method according to this invention, can be at the intercepting system place (namely on the aspect at data-storage system) service data, and can present data to file system in the mode that context is modified, so do not need operating system and application resource or modification.
In the context of document file management system, the present invention allows to keep the semantic but mode that provide the semanteme of expansion of data-storage system and comes the data that redirect are carried out Version Control, the semanteme of described expansion allow to obtain data than early version.Current system need new data more than early version with incorporate into the semantic of expansion or need with old data and than the mode that new data distinguishes obtain data than early version.
The data I/O that redirects can also provide input to distributed data-storage system, and where described distributed data-storage system controls source bit of data in based on the serviceability relevant with the time for data-storage system of source data.Again, in this respect, can realize these features in the aspect of data-storage system, and for the application of access file system, will can not show any difference.
In the title virtualization context, data may no longer be positioned on the storage resources of primary data storage system, and may be positioned at diverse place, and the adaptation for file system needn't be provided.The data virtualization possibility that also becomes, wherein, single named data fragment may no longer be positioned on the identical storage resources, but may be distributed to a plurality of data-storage systems and/or storage resources.Copy in the context in data, the data I/O that redirects can drive reproduction process to guarantee the redundancy of data.
Though the data described above and operation of data is had known realization (mainly is on the layer of using, for example by using virtual machine to realize), but the shortcoming of these realizations has been described in the technical background part, the present invention allows to provide these features and function in the aspect of data-storage system, thereby makes these features and function be independent of file system or operating system.
Under the situation of the spirit that does not break away from claim subsequently, can make various adjustment and realization to the present invention, comprise by blocker and realize in the single-stage of data-storage system or the multi-level buffer one; Wherein, performing step is carried out by the blocker module, and by the data with less use be pushed to data-storage system at a distance and be drawn to by the data with more use near data-storage system realize local optimum.Data-storage system at a distance comprises the data-storage system that physically is in the distant place or the data-storage system that needs more time to conduct interviews.In addition, realize based on the virtual of title it being possible by blocker, make that based on the data on virtual other data-storage system by reference of title Invalid filename looks like effectively in the current data storage system.By making blocker directly and file system communicates, data backup and to copy also be possible.Also make the data virtual on the data-storage system aspect change into to possibility, comprise that the data that allow same names are physically located in the different data-storage systems; Wherein, performing step is carried out by the blocker module.
According to another aspect of the present invention, method also comprises: with compare at the storage resources place, provide different abilities at the intercepting system place for selected data, wherein, described different ability be from comprise that Performance Characteristics, cancellation repeat, data are polymorphic, independently access control mechanisms, Version Control, buffer memory, localize, copy the group with data virtualization and select.Preferably use selection mechanism to identify selected data, described selection mechanism adopts or multinomial metadata model coupling to selecting from the group that comprises the following: title, timestamp, size, historical information, physical message and contextual information.
Above-described embodiment is intended to as example of the present invention, and those skilled in the art can carry out substitutions and modifications to it without departing from the present invention, and the scope of the invention only is defined by the following claims.
Claims (59)
1. method that is used for the function of growth data storage system, described data-storage system comprises data molded tissue block and data storage resource, and described method comprises: the data storage function of described data-storage system is separated with other function of described data-storage system; And at least a portion of described data storage function transferred to intercepting system.
2. method according to claim 1, wherein, described intercepting system comprises the storage resources of giving.
3. method according to claim 1, wherein, described separating step is carried out by the blocker module is provided, and described blocker module and described data molded tissue block, described data storage resource and described intercepting system communicate; Described blocker module interception file system operation should be handled still by described data storage resource with the function of determining described operation and be handled by described intercepting system.
4. method according to claim 3, wherein, described blocker module described file system operation of interception when file system operation still has context and in described operation otherwise before will being broken down into the independent operation that is suitable for described data storage resource.
5. method according to claim 1, wherein, another part at least and the described data-storage system of described data storage function are retained in together.
6. method according to claim 1, wherein, described data storage function whole are transferred to described intercepting system.
7. method according to claim 1, wherein, described data-storage system is to select from the group that comprises file system, key value storage, object storage and procotol.
8. method according to claim 1, wherein, described data-storage system is shared between a plurality of external interfaces.
9. method according to claim 3, wherein, described blocker module comprises the user space application with facility cooperation operating system or file system.
10. method according to claim 3, wherein, described blocker module is included in the file system protocol agent application of carrying out on the network.
11. method according to claim 3, wherein, described blocker module comprises the microfilter driver, and described microfilter driver is used for tackling file system operation at operating system kernel.
12. method according to claim 3, wherein, described intercepting system uses one or more storage resources of giving to carry out its function, and the described storage resources of giving and described storage resources are independent.
13. method according to claim 1, wherein, described intercepting system is realized the capacity extension of described data-storage system.
14. method according to claim 1, wherein, described intercepting system is by the one or more characteristics that change the data on the described storage resources or the performance of improving described data-storage system by the storage resources that use is given.
15. method according to claim 14, wherein, described one or more characteristics are to select from the group that comprises storage format, memory location and storage order.
16. method according to claim 1 also comprises: carry out the step that cancellation repeats by described intercepting system.
17. method according to claim 1 also comprises: carry out the polymorphic step of data by described intercepting system.
18. method according to claim 1 also comprises: by the step of described intercepting system realization at the independent access controlling mechanism of data.
19. method according to claim 1 also comprises: the step of data being carried out Version Control by described intercepting system.
20. method according to claim 12 also comprises: realize the single-stage buffer memory of described data-storage system or one step in the multi-level buffer; Wherein, described performing step is carried out by described blocker module.
21. method according to claim 12, wherein, the described storage resources of giving is the operating system memory cache device that just is being usually used in data are carried out buffer memory.
22. method according to claim 12, also comprise: by the data with less use be pushed to data-storage system at a distance and be drawn to by the data with more use near storage resources or the data-storage system step that realizes local optimum, wherein, described performing step is carried out by described blocker module.
23. method according to claim 21, wherein, the data-storage system of described distant place comprises the data-storage system that is physically located in the distant place or the data-storage system that needs more time to conduct interviews.
24. method according to claim 12, also comprise: realize the virtualized step based on title, describedly make that based on the data on virtual other data-storage system by reference of title Invalid filename seems effective in the current data storage system; Wherein, described performing step is carried out by described blocker module.
25. method according to claim 12 also comprises: one step during realization data backup and data copy; Wherein, described performing step is carried out by described blocker module.
26. method according to claim 12 also comprises: realize the step of data virtualization, described data virtualization comprises that the data that allow same names are physically located in the different data-storage systems; Wherein, described performing step is carried out by described blocker module.
27. method according to claim 1, also comprise: with compare at described storage resources place, provide different abilities at described intercepting system place for selected data, wherein, described different ability is from comprising that Performance Characteristics, cancellation repeat, data are polymorphic, independent access controlling mechanism, Version Control, buffer memory, localizing, copy the group with data virtualization and select.
28. method according to claim 26, wherein, described selected data is to use selects mechanism to identify, described one or the multinomial metadata model coupling of mechanism employing to selecting from the group that comprises the following of selecting: title, timestamp, size, historical information, physical message and contextual information.
29. method according to claim 1, wherein, described data molded tissue block is to select from the group that comprises file system, key value storage and database.
30. method according to claim 12, wherein, described additional storage resources is sparse file.
31. system that is used for the function of growth data storage system, described data-storage system comprises data molded tissue block and data storage resource, and described system comprises: be used for the separation module that the data storage function with described data-storage system separates with other function of described data-storage system; And the module that is used at least a portion of described data storage function is transferred to intercepting system.
32. system according to claim 31, wherein, described intercepting system comprises the storage resources of giving.
33. system according to claim 31, wherein, described separation module comprises blocker, and described blocker and described data molded tissue block, described data storage resource and described intercepting system communicate; Described blocker is used for the interception file system operation should be handled or be handled by described intercepting system with the function of determining described operation by described data storage resource.
34. system according to claim 33, wherein, described blocker will be tackled described file system operation when file system operation still has context and in described operation otherwise before will being broken down into the independent operation that is suitable for described data storage resource.
35. system according to claim 31, wherein, another part at least and the described data-storage system of described data storage function are retained in together.
36. system according to claim 31, wherein, described data storage function whole are transferred to described intercepting system.
37. system according to claim 31, wherein, described data-storage system is to select from the group that comprises file system, key value storage, object storage and procotol.
38. system according to claim 31, wherein, described data-storage system is shared between a plurality of external interfaces.
39. system according to claim 33, wherein, described blocker comprises the user space application with facility cooperation operating system or file system.
40. system according to claim 33, wherein, described blocker is included in the file system protocol agent application of carrying out on the network.
41. system according to claim 33, wherein, described blocker comprises microfilter or conventional filter driver, is used for tackling file system operation at operating system kernel.
42. system according to claim 33, wherein, described intercepting system uses one or more storage resources of giving to carry out its function, and the described storage resources of giving and described storage resources are independent.
43. system according to claim 33, wherein, described intercepting system is realized the capacity extension of described data-storage system.
44. system according to claim 33, wherein, described intercepting system improves the performance of described data-storage system by the one or more characteristics that change the data on the described storage resources.
45. system according to claim 33, wherein, described one or more characteristics are to select from the group that comprises storage format, memory location and storage order.
46. system according to claim 31, wherein, described intercepting system is used for carrying out the cancellation repetition of data.
47. system according to claim 31, wherein, described intercepting system is used for carrying out the polymorphic of data.
48. system according to claim 31, wherein, described intercepting system is used for carrying out the independent access controlling mechanism at data.
49. system according to claim 31, wherein, described intercepting system is used for carrying out the Version Control of data.
50. system according to claim 33, wherein, described blocker is used for carrying out the single-stage buffer memory of described data-storage system or of multi-level buffer.
51. system according to claim 33, wherein, the described storage resources of giving is the operating system memory cache device that just is being usually used in data are carried out buffer memory.
52. system according to claim 33, wherein, described blocker is carried out local optimum for being pushed to data-storage system at a distance by the data with less use and being drawn to neighbouring storage resources or data-storage system by the data with more use.
53. according to the described system of claim 51, wherein, the data-storage system of described distant place comprises the data-storage system that is physically located in the distant place or the data-storage system that needs more time to conduct interviews.
54. system according to claim 33, wherein, described blocker be used for to be carried out virtual based on title, describedly makes that based on the data on virtual other data-storage system by reference of title Invalid filename seems effective in the current data storage system.
55. system according to claim 33, wherein, described blocker is for carrying out one that data backup and data copy.
56. system according to claim 33, wherein, it is virtual that described blocker is used for carrying out data, and described data virtualization comprises that the data that allow same names are physically located in the different data-storage systems.
57. system according to claim 31, wherein, described intercepting system be used for to carry out from comprising improve that Performance Characteristics, cancellation repeat, data are polymorphic, independent access controlling mechanism, Version Control, buffer memory, localize, copy with the group of data virtualization select one or more.
58. system according to claim 31, wherein, described data molded tissue block is to select from the group that comprises file system, key value storage and database.
59. according to the described system of claim 42, wherein, described additional storage resources is sparse file.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US37690510P | 2010-08-25 | 2010-08-25 | |
| US61/376,905 | 2010-08-25 | ||
| PCT/CA2011/050514 WO2012024800A1 (en) | 2010-08-25 | 2011-08-24 | Method and system for extending data storage system functions |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103201736A true CN103201736A (en) | 2013-07-10 |
Family
ID=45722795
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011800509497A Pending CN103201736A (en) | 2010-08-25 | 2011-08-24 | Method and system for extending data storage system functions |
Country Status (6)
| Country | Link |
|---|---|
| EP (1) | EP2609528A4 (en) |
| JP (1) | JP2013536514A (en) |
| KR (1) | KR101510025B1 (en) |
| CN (1) | CN103201736A (en) |
| AU (1) | AU2011293014B2 (en) |
| WO (1) | WO2012024800A1 (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9424267B2 (en) | 2013-01-02 | 2016-08-23 | Oracle International Corporation | Compression and deduplication layered driver |
| CN104077374B (en) * | 2014-06-24 | 2018-09-11 | 华为技术有限公司 | A kind of method and device for realizing the storage of IP disk files |
| US10802928B2 (en) | 2015-09-10 | 2020-10-13 | International Business Machines Corporation | Backup and restoration of file system |
| US10423331B2 (en) * | 2016-02-02 | 2019-09-24 | Samsung Electronics Co., Ltd. | Polymorphic storage devices |
| US11287973B2 (en) | 2016-02-02 | 2022-03-29 | Samsung Electronics Co., Ltd. | Polymorphic storage devices |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0856803A2 (en) * | 1997-01-31 | 1998-08-05 | Informix Software, Inc. | File system interface to a database |
| CN1647047A (en) * | 2002-04-03 | 2005-07-27 | 鲍尔凯斯特公司 | Using disassociated images for computer and storage resource management |
| CN1894672A (en) * | 2003-12-19 | 2007-01-10 | 维里塔斯管理公司 | Method and apparatus for performing operations on selected data in a storage area |
| US7165057B2 (en) * | 2001-11-29 | 2007-01-16 | Veritas Operating Corporation | Methods and systems to access storage objects |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6269382B1 (en) | 1998-08-31 | 2001-07-31 | Microsoft Corporation | Systems and methods for migration and recall of data from local and remote storage |
| CA2646776C (en) * | 1999-08-05 | 2014-07-08 | Oracle International Corporation | Internet file system |
| US9390102B2 (en) * | 2008-09-29 | 2016-07-12 | Oracle International Corporation | Client application program interface for network-attached storage system |
-
2011
- 2011-08-24 WO PCT/CA2011/050514 patent/WO2012024800A1/en active Application Filing
- 2011-08-24 AU AU2011293014A patent/AU2011293014B2/en not_active Ceased
- 2011-08-24 JP JP2013525098A patent/JP2013536514A/en active Pending
- 2011-08-24 EP EP11819254.1A patent/EP2609528A4/en not_active Withdrawn
- 2011-08-24 CN CN2011800509497A patent/CN103201736A/en active Pending
- 2011-08-24 KR KR20137007468A patent/KR101510025B1/en not_active IP Right Cessation
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0856803A2 (en) * | 1997-01-31 | 1998-08-05 | Informix Software, Inc. | File system interface to a database |
| US7165057B2 (en) * | 2001-11-29 | 2007-01-16 | Veritas Operating Corporation | Methods and systems to access storage objects |
| CN1647047A (en) * | 2002-04-03 | 2005-07-27 | 鲍尔凯斯特公司 | Using disassociated images for computer and storage resource management |
| CN1894672A (en) * | 2003-12-19 | 2007-01-10 | 维里塔斯管理公司 | Method and apparatus for performing operations on selected data in a storage area |
Also Published As
| Publication number | Publication date |
|---|---|
| EP2609528A4 (en) | 2017-07-19 |
| JP2013536514A (en) | 2013-09-19 |
| WO2012024800A1 (en) | 2012-03-01 |
| EP2609528A1 (en) | 2013-07-03 |
| AU2011293014A1 (en) | 2013-03-21 |
| KR20130046441A (en) | 2013-05-07 |
| KR101510025B1 (en) | 2015-04-08 |
| AU2011293014B2 (en) | 2014-08-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3117326B1 (en) | Reducing data volume durability state for block-based storage | |
| JP5620581B2 (en) | Method and system for cache tiering | |
| JP7065108B2 (en) | A system for storing data in a tape volume container | |
| US8572136B2 (en) | Method and system for synchronizing a virtual file system at a computing device with a storage device | |
| US20140304513A1 (en) | Storage drive processing multiple commands from multiple servers | |
| JP5387757B2 (en) | Parallel data processing system, parallel data processing method and program | |
| US20140245026A1 (en) | System and method for resource sharing across multi-cloud arrays | |
| US8566297B1 (en) | Method to spoof data formats from image backups | |
| US20070220029A1 (en) | System and method for hierarchical storage management using shadow volumes | |
| US20210064486A1 (en) | Access arbitration to a shared cache storage area in a data storage management system for live browse, file indexing, backup and/or restore operations | |
| US8131957B2 (en) | Splitting writes between a storage controller and replication engine | |
| JP2008539520A (en) | System and method for caching a network file system | |
| JP5352490B2 (en) | Server image capacity optimization | |
| CN103201736A (en) | Method and system for extending data storage system functions | |
| US11442894B2 (en) | Methods for scalable file backup catalogs and devices thereof | |
| US7415480B2 (en) | System and method for providing programming-language-independent access to file system content | |
| US8024534B2 (en) | Replication engine communicating with a splitter to split writes between a storage controller and replication engine | |
| JP5517224B2 (en) | Storage device | |
| US20080270480A1 (en) | Method and system of deleting files from a remote server | |
| EP2544084A1 (en) | Storage device | |
| US8726299B1 (en) | Image-oriented, plugin-based API to storage server appliances | |
| WO2014165451A2 (en) | Key/value storage device and method | |
| US7734869B1 (en) | Interfaces for flexible storage management | |
| JP2010117791A (en) | Shared storage system and data keeping method and data keeping program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| ASS | Succession or assignment of patent right |
Owner name: INTEL CORP . Free format text: FORMER OWNER: NEVEX VIRTUAL TECHNOLOGIES INC. Effective date: 20131216 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20131216 Address after: American California Applicant after: Intel Corporation Address before: Ontario Applicant before: Nevex Virtual Technologies Inc. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130710 |