CN100437586C - A resource allocation method and system for shared memory - Google Patents
A resource allocation method and system for shared memory Download PDFInfo
- Publication number
- CN100437586C CN100437586C CNB2006101499366A CN200610149936A CN100437586C CN 100437586 C CN100437586 C CN 100437586C CN B2006101499366 A CNB2006101499366 A CN B2006101499366A CN 200610149936 A CN200610149936 A CN 200610149936A CN 100437586 C CN100437586 C CN 100437586C
- Authority
- CN
- China
- Prior art keywords
- file
- rule
- attribute
- resource
- heterogeneous
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
This invention relates to a materials align method and system for common memory, wherein, the method comprises the following steps: abnormal memory device registering; dividing files according to property of memory device; establishing mapping relationship between file property and memory property; materials aligning according to rules. The align system comprises the following parts: memory pool connected to materials management device; control device connected to the device; rules management device; one rules database; one interface with different file property; user operation visit interface.
Description
Technical field
The present invention relates to share the resource allocation techniques of storage system, particularly a kind of resource allocation system and distribution method based on the file storage attribute.
Background technology
Along with expansion of storage capacity, the storage resources that has inevitably occurred various isomeries in the system, all exist difference as FC equipment, iscsi device, IDE equipment, SATA equipment, TAPE equipment etc., these heterogeneous resources at aspects such as capacity, performance, stability, reliability, securities, how in system, effectively to use these heterogeneous resources to become important day by day.
And, widespread use along with computing machine, along with network, Development of Multimedia Technology, meeting and deposit several data in the storage system of sharing, particularly in the environment of IDC (Internet Data Center), be inevitable several data and deposited, as WEB, EMAIL, stream medium data etc.Every type of data characteristic difference is 13KB as WEB data mean size, a large amount of read operations; Stream medium data is substantially at tens of MB, and data do not need buffer memory, but needs to guarantee certain amount of reading in advance.Like this, all there is the problem of supporting the different attribute data in the environment such as IDC, enterprise at present.
Existing storage system and file system are not carried out the good solution of system to above two problems, the storage resources that often in system, has only isomorphism, and can not provide strong support to the file of different attribute, in the time of can not well solving multiple property file and share, effectively support the problem of different attribute file high-performance visit, as local system UFS, XFS, JFS, FFS, EXT2, EXT3 etc.; In addition, at present the distributed network Design of File Systems is based on local file system substantially, and as NFS, the rear end is fully based on local system; Lustre, the rear end is based on EXT3; CXFS, the rear end is based on XFS; Storage Tank, the rear end is based on JFS.Therefore the resource management allocation strategy of local system has directly determined the allocation strategy of storage system.
At the deficiencies in the prior art, the present invention proposes and a kind ofly can effectively use heterogeneous resource and effectively support different attribute file high-performance accessed resources distribution method.
Summary of the invention
The technical problem to be solved in the present invention is the problem that storage system can not effectively be used isomerism storage resources and only single memory attribute be supported in the prior art, provides a kind of and can effectively support different attribute file high-performance accessed resources distributor and distribution method.
In order to achieve the above object, the present invention takes following technical scheme:
A kind of being used for shared the stored resource distribution method, comprises the steps:
Step 1), heterogeneous storage devices is registered;
Step 2), the storage resources of heterogeneous storage devices is divided file volume according to the attribute of heterogeneous storage devices;
Step 3), according to mapping relations and the file attribute accessing operation of file attribute with memory attribute, set up rule base;
Step 4), carry out resources allocation according to rule.
In technique scheme, further, described step 4) specifically comprises the steps:
Step 4-1), the step of abstract file attribute;
Step 4-2), according to the abstract file attribute that obtains, the rule searching storehouse obtains rule;
Step 4-3), according to the rule obtained, file is carried out resource allocation operations;
Step 4-4), when deleted file carries out resource when reclaiming, discharge according to the release granularity in the described rule.
A kind of being used for shared the stored resource distribution system, comprising: memory resource pool; The resource management apparatus that is connected with described memory resource pool; The control device that is connected with described resource management apparatus; The regulation management device that being used for of being connected with described control device manages described rule; One rule base that is used to store described rule is connected to described regulation management device; Described control device has the interface that receives the pending file set with different attribute; Described resource management apparatus and described regulation management device are provided with user's operational access interface; Described memory resource pool is made up of a plurality of heterogeneous storage devices, and the attribute according to different heterogeneous storage devices on memory resource pool is divided different file volumes; Described rule base is made up of at least one rule; Rule expression file attribute is with the mapping relations of memory attribute and the accessing operation of file attribute.
In technique scheme, further, described resource management apparatus, comprise heterogeneous resource adding, withdraw from, functional unit such as distribution recovery of the inquiry of heterogeneous resource information, heterogeneous resource, file volume division.
In technique scheme, further, described control device by file attribute is carried out abstract parts, according to file attribute rule searching storehouse with obtain the parts of suitable rule and carry out the parts that the distribution of heterogeneous resource reclaims according to rule and form.
In technique scheme, further, described file set is that the different attribute file is through the set by the file behind the attributive classification, file attribute comprises: owner, group, ACL (Access Control List), filename, catalogue, extension name, status requirement, permission type, creation-time, modification time etc., the file of different attribute has different processing modes, need be stored in the storage resources that satisfies documentation requirements, and conduct interviews according to the rule request mode.
In technique scheme, further, described heterogeneous storage devices comprises dissimilar storage resources, and this storage resources has different attributes, comprising: capacity, data transmission performance, reliability, physical location, security etc.
In technique scheme, further, described rule base management devices comprises coming into force of the foundation of setting up functional unit, rule of initialization rule base functional unit, default rule and deletion action parts, rule and quiescing parts, receives rule query and return results functional unit.
The present invention is according to different file attributes, according to determining good policing rule in advance, provide corresponding heterogeneous resource to distribute and access interface to file, thereby effectively use isomerism storage resources, satisfy the performance requirement of different attribute file, improve the performance of file system on the whole.
Compared with prior art, beneficial effect of the present invention is:
By the different attribute file is carried out different resources allocations according to strategy, make that all types file can both have high performance in multiple property file shared environment, and effectively managed isomerism storage resources.The present invention has adaptability widely, and in different storage environments, at different storage demands, the present invention satisfies storage demand according to the cooperation of attribute and resource.
Description of drawings
Fig. 1 is the resource allocation system structural drawing that the present invention is used for sharing storage system.
Embodiment
Below in conjunction with the drawings and specific embodiments the present invention is described in further detail.
As shown in Figure 1, be used for sharing the resource allocation system of storage system, comprise: memory resource pool 1; The resource management apparatus 6 that is connected with described memory resource pool 1; The control device 3 that is connected with described resource management apparatus 6; The regulation management device 5 that is connected with described control device 3; One rule base 4 is connected to described regulation management device 5; Described control device 3 has the interface that receives the pending file set with different attribute 2; Described resource management apparatus 6 and described regulation management device 5 are provided with user's operational access interface.Below each several part is elaborated:
Described memory resource pool 1 is made up of a plurality of heterogeneous storage devices, and such as equipment such as disk, tape, iSCSI, FC, the attribute according to different heterogeneous storage devices on memory resource pool is divided different file volumes.Isomerism storage resources, comprise dissimilar storage resources, storage resources has different attributes, comprise: capacity, data transmission performance, reliability, physical location, security etc., memory resource pool 1 is the set of above storage resources, and the division of storage resources is exactly that different attribute according to storage resources is divided into memory resource pool that different classes of part---file volume manages respectively.File volume is the set of memory resources with same services quality (QoS) (physical attribute), and resource operation carries out based on file volume; File volume provides routine interface, for program provides visit this document volume resource operation interface.
Described resource management apparatus 6, be responsible for the heterogeneous resource in the maintenance system, comprise heterogeneous resource adding, withdraw from, functional unit such as distribution recovery of the inquiry of heterogeneous resource information, heterogeneous resource, file volume division, this device offers user's operational access interface.
Described control device 3 by file attribute is carried out abstract parts, according to file attribute rule searching storehouse with obtain the parts of suitable rule and carry out the parts that the distribution of heterogeneous resource reclaims according to rule and form.Control device 3 is realized the different attribute file carried out the allocation manager of isomerism storage resources by strategy, control device at first carries out abstract to file attribute, according to file attribute rule searching storehouse, obtain suitable rule then, and carry out the operations such as distribution recovery of heterogeneous resource according to rule.
Described rule base 4 is made up of at least one rule; Rule expression file attribute is with the mapping relations of memory attribute and the accessing operation of file attribute.Rule is user configured allocation strategy, and allocation strategy is specialized, and the expression file attribute is with the mapping relations of memory attribute and the accessing operation of file attribute.
Described file set 2 is that the different attribute file is through the set by the file behind the attributive classification, file attribute comprises: owner, group, ACL (Access Control List), filename, catalogue, extension name, status requirement, permission type, creation-time, modification time etc., the file of different attribute has different processing modes, need be stored in the storage resources that satisfies documentation requirements, and conduct interviews according to the rule request mode.
Described rule base management devices 5 comprises coming into force of the foundation of setting up functional unit, rule of initialization rule base functional unit, default rule and deletion action parts, rule and quiescing parts, receives rule query and return results functional unit.Rule in 5 responsible maintenances of rule base management devices and the rule searching storehouse specifically comprises: the foundation operation of the operation of initialization rule base, default rule, the foundation of rule and deletion action, regular coming into force and quiescing, the rule query that receives other modules and return results operation etc.This device 5 also offers user's operational access interface.
Among the present invention, form memory resource pool 1 by heterogeneous resource, be divided into different file volumes according to different Resource Properties on memory resource pool, each file volume is inherited the respective attributes of memory device, to the resource operation of the file of different attribute all based on file volume; By heterogeneous resource management devices 6, the user can handle heterogeneous resource adding, withdraw from, divide operations such as file volume.By various resource operations rule composition rules storehouse 4, the user can be by the interface of rule base management devices 5, to the rule in the rule base 4 add, operation such as deletion, and rule base receives the rule query in the resource operation.File set 2 with different attribute is pending files with different attribute.Control device 3 comprises parts such as file attribute is abstract, rule query, resource operation, finishes according to the file attribute rule searching, and carries out the control function of resource operation according to rule.Rule base management devices 5 provides user interface, receives user's configuration management order, receives the querying command of control module; Resource management apparatus 6 provides user interface, receives user's configuration management order, receives the resource operation of control module.
In the embodiment, be used for sharing the resource allocation methods of storage system, comprise the steps:
One, the step that heterogeneous resource management, file volume are divided
1. isomery facility registration step
At first, need be in storage pool the isomery facility registration; Registration process can be used that command-line tool that resource management apparatus provides is manual to be finished or allow this module finish automatically by configuration file by the user.In the registration process, general need provide device name to get final product, and can obtain the various attributes of equipment in subsequent process, perhaps provides various attributes by hand by the user in registration process.The attribute of memory device comprises following several:
(1) data transmission performance: IOPS, bandwidth postpones etc.;
(2) reliability: the RAID rank, duplicate (telecopy), backup etc.;
(3) physical location: be positioned at which cluster (physical connection) etc.;
(4) security: whether encrypt etc.
(5) physical characteristics: the capacity of memory device, the block size of memory device, storage device media, accessing storage device interface etc.
After resource management apparatus receives above facility registration order, above attribute is quantized respectively, and the data structure of establishment representative equipment, name is called DEVICE, the various attributes that comprise device name and quantification, store the equipment chained list into, name is called in the chained list of DEVICE_LIST, as the storage resources in the storage pool.So far, the facility registration process finishes.
2. file volume partiting step
The file volume partiting step, the various isomerism storage resources according in the storage pool are divided into different file volumes according to certain requirement, for file system provides concrete storage resources.File volume can comprise one or more memory devices, and the equipment in the file volume can have single attribute, perhaps has multiple attribute; Such as: only have the attribute of performance, and do not have security attributes, or the both has.The division of file volume can be divided by the user interface initial conditions is manual, perhaps carries out file volume by user's direct control memory device and divides, and is perhaps finished automatically by the Storage Resource Management (SRM) device by configuration file.After being divided into file volume, create the data structure of representation file volume, name is called FILESET, is connected to the file volume chained list, and name is called in the chained list of FILESET_LIST.
File volume can dynamically be accelerated, and also can dynamically increase capacity.File volume can be carved dynamic division when needed, to accelerate; File volume needn't carry out by integral dispensing memory device, can only use a part of resource of memory device, divides as requested; And the division of file volume can dynamic expansion, and when certain file volume off-capacity or performance can not meet the demands, file volume can be expanded to meet the demands.
Two, rule base management process;
1. regular establishment step
Rule can be represented with five-tuple: (file attribute, file volume set, partition size, presort granularity, discharge granularity), expression has the file of " file attribute ", should be assigned in " file volume set ", and distribute release according to " partition size ", " presorting granularity ", " release granularity ".Rule can be created by the various piece parameter of rule base management devices interface input five-tuple by the user, perhaps creates by duplicating the rule that has existed, and perhaps creates rule automatically by configuration file.
2. rule query step
Rule can also can be used by the resource distribution module inquiry by interface by user interface by user inquiring.In the rule query process, may there be the situation of corresponding many rules of file attribute, at this moment needs to determine optimal rules according to priority; During optimal rules is determined, according to the attribute of paying close attention to most, determine many rules, the attribute according to other carries out optimum choosing from above many definite rules then.
3. redundant rule elimination step
The user can delete out-of-date rule by interface.
Three, resource allocation step
1. the step of abstract file attribute
Abstract the requirement of file is carried out, abstract is the specific object of file, thus for the distribution based on attribute provide the basis.From the following aspect, carried out abstract:
(1), takes out the user property of file according to file attribute differences such as owner, group, ACL;
(2), take out the built-in attribute of file according to file name, catalogue, extension name file attribute difference.
(3), take out security attribute, the position attribution of file according to position (which cluster is client computer be positioned at), permission type documentation requirements differences such as (as security considerations).
And above requirement is quantized ordering according to degree of concern, thereby for selecting optimal strategy ready.
2. based on the resources allocation flow process of memory attribute
According to the abstract different file attributes that come out, determine the storage demand of file, carry out resources allocation; In application, can there be different storage demands to the file of different attribute, as:
The owner need encrypt and be stored on other medium of RAID6 higher level for the file of abc;
The granularity of the each distribute data piece of file of expansion .mp3 by name is 64MB, and is stored on other storage medium of RAID0 level, and its IOPS and random access performance are the highest in the storage system;
Authority is that the file of read-only type need be stored on the medium with timed backup function;
The file of other attributes.
Therefore at different storage demands, file attribute, Resource Properties, formulate corresponding resource allocation policy and method, so just have extensive applicability.
The strategy of resources allocation and method mainly solve file attribute and resource mapping, and the file of different attribute is corresponding mutually with different file volumes, and the management granularity is a unit with file and file volume.The idiographic flow of resources allocation is as follows:
(1) according to abstract file attribute, the rule searching database management module is obtained optimum rule.
(2), file is carried out resource allocation operations according to the rule of obtaining.According to rule, the access destination file volume if file destination volume can not allocation space, then returns and makes mistakes; Otherwise in the file destination volume, resource is distributed the resource situation in the modify target file volume.
(3) work as deleted file, when carrying out the resource recovery, discharge according to the release granularity in the strategy.
In a word, by the cooperation of above step, at first, and user definition Resource Properties, file attribute and strategy, system just can carry out the allocation manager of resource according to strategy then, reaches the purpose that satisfies the files in different types storage demand.Thereby the present invention can satisfy the demand of sharing aspects such as storage under the storage condition, security, performance at files in different types, has extensive applicability.
It should be noted last that above embodiment is only unrestricted in order to technical scheme of the present invention to be described.Although the present invention is had been described in detail with reference to embodiment, those of ordinary skill in the art is to be understood that, technical scheme of the present invention is made amendment or is equal to replacement, do not break away from the spirit and scope of technical solution of the present invention, it all should be encompassed in the middle of the claim scope of the present invention.
Claims (8)
1, a kind of being used for shared the stored resource distribution method, comprises the steps:
Step 1), heterogeneous storage devices is registered;
Step 2), the storage resources of heterogeneous storage devices is divided file volume according to the attribute of heterogeneous storage devices;
Step 3), according to mapping relations and the file attribute accessing operation of file attribute with memory attribute, set up rule base;
Step 4), carry out resources allocation according to rule.
2, be used for sharing the stored resource distribution method according to claim 1 is described, it is characterized in that described step 4) specifically comprises the steps:
Step 4-1), the step of abstract file attribute;
Step 4-2), according to the abstract file attribute that obtains, the rule searching storehouse obtains rule;
Step 4-3), according to the rule obtained, file is carried out resource allocation operations;
Step 4-4), when deleted file carries out resource when reclaiming, discharge according to the release granularity in the described rule.
3, a kind of being used for shared the stored resource distribution system, comprises memory resource pool; It is characterized in that, also comprise the resource management apparatus that is connected with described memory resource pool; The control device that is connected with described resource management apparatus; The regulation management device that being used for of being connected with described control device manages rule; One rule base that is used for storage rule is connected to described regulation management device; Described control device has the interface that receives the pending file set with different attribute; Described resource management apparatus and described regulation management device have user's operational access interface; Described memory resource pool is made up of a plurality of heterogeneous storage devices, and the attribute according to different heterogeneous storage devices on memory resource pool is divided different file volumes; Described rule base is made up of with the rule of the accessing operation of the mapping relations of memory attribute and file attribute at least one expression file attribute.
4, be used for sharing the stored resource distribution system according to claim 3 is described, it is characterized in that described resource management apparatus comprises the distribution reclaimer operation parts and the file volume division operation parts of the adding functional unit of heterogeneous resource, the query manipulation parts that withdraw from functional unit, heterogeneous resource information, heterogeneous resource.
5, be used for sharing the stored resource distribution system according to claim 3 is described, it is characterized in that, described control device comprise to file attribute carry out abstract parts, according to file attribute rule searching storehouse with obtain the parts of suitable rule and carry out the parts that the distribution of heterogeneous resource is reclaimed according to rule.
6, be used for sharing the stored resource distribution system according to claim 3 is described, it is characterized in that described file set is that the different attribute file is through the set by the file behind the attributive classification.
7, be used for sharing the stored resource distribution system according to claim 3 is described, it is characterized in that, described heterogeneous storage devices comprises dissimilar storage resources, and this storage resources has different attributes, and attribute comprises capacity, data transmission performance, reliability, physical location, security.
8, be used for sharing the stored resource distribution system according to claim 3 is described, it is characterized in that described rule base management devices comprises coming into force of the foundation of setting up functional unit, rule of initialization rule base functional unit, default rule and deletion action parts, rule and quiescing parts, receives rule query and return results functional unit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2006101499366A CN100437586C (en) | 2005-10-21 | 2006-10-17 | A resource allocation method and system for shared memory |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200510109459.6 | 2005-10-21 | ||
| CN200510109459 | 2005-10-21 | ||
| CNB2006101499366A CN100437586C (en) | 2005-10-21 | 2006-10-17 | A resource allocation method and system for shared memory |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1952941A CN1952941A (en) | 2007-04-25 |
| CN100437586C true CN100437586C (en) | 2008-11-26 |
Family
ID=38059282
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2006101499366A Active CN100437586C (en) | 2005-10-21 | 2006-10-17 | A resource allocation method and system for shared memory |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100437586C (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102426552B (en) | 2011-10-31 | 2014-06-04 | 华为数字技术（成都）有限公司 | Storage system service quality control method, device and system |
| KR102034736B1 (en) * | 2012-05-30 | 2019-10-22 | 삼성에스디에스 주식회사 | Managing apparatus and method for Machine-to-Machine communications |
| CN103399875A (en) * | 2013-07-12 | 2013-11-20 | 深圳市金立通信设备有限公司 | File managing method and device |
| CN105791370B (en) * | 2014-12-26 | 2019-02-01 | 华为技术有限公司 | A kind of data processing method and associated server |
| CN105049443A (en) * | 2015-08-12 | 2015-11-11 | 北京因特信安软件科技有限公司 | Method for constructing trusted resource pool |
| CN105573679B (en) * | 2015-12-18 | 2019-10-11 | 国云科技股份有限公司 | A method of suitable for distributed memory system storage pool resource impact rule |
| CN107783734B (en) * | 2017-11-17 | 2021-03-16 | 北京联想超融合科技有限公司 | Resource allocation method, device and terminal based on super-fusion storage system |
| CN110968264B (en) * | 2019-10-16 | 2022-09-16 | 南京基石数据技术有限责任公司 | Method for distributing discrete groups under ASM framework |
| CN111694520B (en) * | 2020-06-11 | 2021-04-27 | 邦尼集团有限公司 | Method and device for optimizing big data storage |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000242805A (en) * | 1999-02-22 | 2000-09-08 | Hitachi Ltd | Method of detecting alteration of feature model |
| US20050204040A1 (en) * | 2004-03-10 | 2005-09-15 | International Business Machines Corporation | Facilitating allocation of resources in a heterogeneous computing environment |
-
2006
- 2006-10-17 CN CNB2006101499366A patent/CN100437586C/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2000242805A (en) * | 1999-02-22 | 2000-09-08 | Hitachi Ltd | Method of detecting alteration of feature model |
| US20050204040A1 (en) * | 2004-03-10 | 2005-09-15 | International Business Machines Corporation | Facilitating allocation of resources in a heterogeneous computing environment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1952941A (en) | 2007-04-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100437586C (en) | A resource allocation method and system for shared memory | |
| US11153380B2 (en) | Continuous backup of data in a distributed data store | |
| US10437721B2 (en) | Efficient garbage collection for a log-structured data store | |
| JP5798248B2 (en) | System and method for implementing a scalable data storage service | |
| US8046422B2 (en) | Automatic load spreading in a clustered network storage system | |
| US8312242B2 (en) | Tracking memory space in a storage system | |
| US7743038B1 (en) | Inode based policy identifiers in a filing system | |
| US7870116B2 (en) | Method for administrating data storage in an information search and retrieval system | |
| US20220114064A1 (en) | Online restore for database engines | |
| US9519653B2 (en) | Techniques for efficiently enforcing resource quotas in a multi-tenant cloud storage system | |
| US8996490B1 (en) | Managing logical views of directories | |
| JP2004070403A (en) | File storage destination volume control method | |
| US11080253B1 (en) | Dynamic splitting of contentious index data pages | |
| US20090070382A1 (en) | System and Method for Performing a File System Operation on a Specified Storage Tier | |
| US10885023B1 (en) | Asynchronous processing for synchronous requests in a database | |
| US20080021865A1 (en) | Method, system, and computer program product for dynamically determining data placement | |
| US20100082546A1 (en) | Storage Tiers for Database Server System | |
| CN105608224A (en) | Orthogonal multilateral Hash mapping indexing method for improving massive data inquiring performance | |
| US10909143B1 (en) | Shared pages for database copies | |
| US10719554B1 (en) | Selective maintenance of a spatial index | |
| CN110825704A (en) | Data reading method, data writing method and server | |
| EP3788501B1 (en) | Data partitioning in a distributed storage system | |
| Cha et al. | Effective metadata management in exascale file system | |
| US20190026732A1 (en) | Aggregated storage file service | |
| US11409733B2 (en) | Automatic reclamation of storage space for database systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |