CN104360961A - Object storage-based self-adaptive graded processing method and object storage-based self-adaptive graded processing system - Google Patents
Object storage-based self-adaptive graded processing method and object storage-based self-adaptive graded processing system Download PDFInfo
- Publication number
- CN104360961A CN104360961A CN201410757697.7A CN201410757697A CN104360961A CN 104360961 A CN104360961 A CN 104360961A CN 201410757697 A CN201410757697 A CN 201410757697A CN 104360961 A CN104360961 A CN 104360961A
- Authority
- CN
- China
- Prior art keywords
- water level
- system capacity
- power system
- upper limit
- capacity water
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims abstract description 159
- 238000000034 method Methods 0.000 claims abstract description 88
- 230000008569 process Effects 0.000 claims abstract description 74
- 230000005012 migration Effects 0.000 claims abstract description 33
- 238000013508 migration Methods 0.000 claims abstract description 33
- 238000012544 monitoring process Methods 0.000 claims abstract description 32
- 230000003044 adaptive effect Effects 0.000 claims description 13
- 238000012546 transfer Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000002567 autonomic effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
Landscapes
- Processing Of Solid Wastes (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an object storage-based self-adaptive graded processing method and an object storage-based self-adaptive graded processing system. The method comprises the following steps: when an object storage operation is implemented, firstly calculating heat of an object in a host process; calculating a system capacity water level according to current data of a system; after calculation, continuously carrying out a normal host process storage operation in the host process and putting a message that the system capacity water level reaches the upper limit of the water level into a water level monitoring array when the system capacity water level reaches the upper limit of the water level; reading the message that the system capacity water level reaches the upper limit of the water level in the water level monitoring array in a grading process; and executing a one-way migration operation of object data from a high performance storage pond to a low performance storage pond according to the heat of the calculated object. According to the scheme, the visiting efficiency of the system can be improved.
Description
Technical field
The present invention relates to cloud field of storage, be specifically related to a kind of adaptive hierarchical disposal route based on object storage and system.
Background technology
Classification storage is the index such as importance, access frequency, retention time, capacity, performance according to data, data taked different storage modes to be stored in respectively on the memory device of different performance, realize the Autonomic Migration Framework of Data object between memory device by hierarchical storage management.The principle of work that data staging stores is the locality based on data access, by the data of infrequently accessing are moved on to level lower in memory hierarchy automatically, the storage space discharging higher cost, to the data of frequently accessing, can obtain better cost performance.The space of insignificant data shared by one-level local disk can be greatly reduced like this, also can accelerate the memory property of whole system.
Object storage system take object as the network store system of access interface, object storage file system provides a set of access interface being similar to file operation, and file operation is converted into object accesses, make the multiple application program of user not need to carry out any amendment and just may operate on object storage system.Adopt object interface can make object storage system cross-platform data share, security, performance and intelligent in there is advantage.In object storage system, data are deposited in units of object, and object can provide the integrality of data.In object storage system with object for granularity designs grading function, be that point level framework of granularity is no longer file but focus object in file due to the object of migration with object, do the data volume just decreasing migration like this, simultaneously because object is less, enormously simplify the process to data in transition process, accelerate transition process.
Disk is polymerized to the different storage pool of two performances according to performance by grading design general at present, and the business idle periods of specifying user carries out the migration of data between storage pool according to the temperature of data.Although high-performance storage pool access efficiency is high under normal circumstances, cost is very high usually, therefore only configures a small amount of high-performance storage space and is used for depositing hot spot data.When hot spot data too much or subscriber policy arrange not too reasonably in situation, be easy to cause high-performance storage pool space not enough, at this moment common disposal route is the request of access of refusal user, or carry out the dilatation of high-performance storage pool by the mode reminding user given warning in advance or manually carry out the migration of partial data, the experience of user can be had a strong impact on like this.
Summary of the invention
In order to solve the problem, the present invention proposes a kind of adaptive hierarchical disposal route based on object storage and system, the access efficiency of system can be improved.
In order to achieve the above object, the present invention proposes a kind of adaptive hierarchical disposal route stored based on object, the method comprises:
When object stores operation arrival, host process is calculating object temperature first, the data computing system capacity water level current according to system, after completing calculating, host process proceeds normal host process and stores operation, and to reach on water level in limited time at power system capacity water level, message power system capacity water level being reached the water level upper limit puts into monitoring water level queue.
The classification process power system capacity water level read in monitoring water level queue reaches the message of the water level upper limit, according to the object temperature calculated, performs and is operated by the object data One Direction Transfer of high-performance storage pool to low performance storage pool.
Preferably, power system capacity water level refers to: the ratio in storage space between occupied space and the total volume of storage space.
Preferably, classification process is according to the object temperature calculated, perform the operation of data One Direction Transfers to refer to, the object in classification process traverse object storage system, according to the object temperature of object preferentially by object data minimum for object temperature by high-performance storage pool migration to low performance storage pool.
Preferably, the method also comprises: after the object migration with minimum object temperature completes by classification process, host process is computing system capacity water level again, if power system capacity water level is still more than or equal to the described water level upper limit, then the message that power system capacity water level is reached the water level upper limit by host process again puts into monitoring water level queue; Classification process reads the message in monitoring queue, and proceed migration operation based on object temperature, if power system capacity water level is less than the water level upper limit, then host process proceeds normal host process and stores operation, and carry out the calculating of power system capacity water level next time in the given time, classification process timing scan monitoring queue, waits for that next power system capacity water level reaches the message of the water level upper limit.
The present invention also proposes a kind of adaptive hierarchical disposal system stored based on object, and this system comprises: host process module, classification scheduler module.
Host process module, during for storing operation arrival at object, first calculating object temperature, the data computing system capacity water level current according to system, after completing calculating, proceed normal host process and store operation, and to reach on water level in limited time at power system capacity water level, message power system capacity water level being reached the water level upper limit puts into monitoring water level queue.
Classification scheduler module, reaches the message of the water level upper limit for the power system capacity water level read in monitoring water level queue, according to the object temperature calculated, perform and operated by the object data One Direction Transfer of high-performance storage pool to low performance storage pool.
Preferably, classification scheduler module also for, the object in traverse object storage system, according to the object temperature of object preferentially by object data minimum for object temperature by high-performance storage pool migration to low performance storage pool.
Preferably,
Host process module also for: after the object migration with minimum object temperature completes by classification scheduler module, computing system capacity water level again, if power system capacity water level is still more than or equal to the water level upper limit, then the message again power system capacity water level being reached the water level upper limit puts into monitoring water level queue; If power system capacity water level is less than the water level upper limit, then proceeds normal host process and store operation, and carry out the calculating of power system capacity water level next time in the given time.
Classification scheduler module also for: if power system capacity water level is still more than or equal to the water level upper limit, then reads the message in monitoring queue, and proceed migration operation based on object temperature; If power system capacity water level is less than the water level upper limit, then timing scan monitoring queue, waits for that next power system capacity water level reaches the message of the water level upper limit.
Compared with prior art, the present invention includes: when object stores operation arrival, host process is calculating object temperature first, the data computing system capacity water level current according to system, after completing calculating, host process proceeds normal host process and stores operation, and to reach on water level in limited time at power system capacity water level, and message power system capacity water level being reached the water level upper limit puts into monitoring water level queue.The classification process power system capacity water level read in monitoring water level queue reaches the message of the water level upper limit, according to the object temperature calculated, performs and is operated by the object data One Direction Transfer of high-performance storage pool to low performance storage pool.By the solution of the present invention, the access efficiency of system can be improved.
Accompanying drawing explanation
Be described the accompanying drawing in the embodiment of the present invention below, the accompanying drawing in embodiment is for a further understanding of the present invention, is used from explanation the present invention, does not form limiting the scope of the invention with instructions one.
Fig. 1 is the adaptive hierarchical process flow figure stored based on object of the present invention;
Fig. 2 is the adaptive hierarchical disposal system block diagram stored based on object of the present invention;
Fig. 3 is the adaptive hierarchical disposal route schematic diagram stored based on object of the present invention.
Embodiment
For the ease of the understanding of those skilled in the art, below in conjunction with accompanying drawing, the invention will be further described, can not be used for limiting the scope of the invention.
Based on the various problems existed in prior art, this method proposes a kind of adaptive hierarchical processing method, when high-performance storage pool reaches maximum size, and the migration of automatic trigger data, by Data Migration minimum for temperature to low performance storage pool, thus the access efficiency of the system of guarantee.
The grading function of the inventive method Object-based storage system, add the judgment mechanism of capacity water level, the capacity water level of current system all will be calculated when operating each time or after Data Migration, at capacity close to water level triggering in limited time by high-performance storage pool to the data sheet of low performance storage pool to migration operation, thus the data of the system of guarantee high-performance storage pool maintain below certain water level all the time, ensure the operability of high-performance storage pool.
The grading function of the inventive method Object-based storage system, make use of the hot statistics of grading function in data migration process, the data that priority migration temperature is minimum when moving, thus ensure that the access performance of high-performance storage pool hot spot data.
The realization of this method is mainly outside host process and classification treatment progress, increase by one is carried out Data Migration thread according to water level and temperature, and in host process, embed water level computing, here host process refers to the manipulation process of high-performance storage pool, and the thread of Data Migration is also arranged in the process of high-performance storage pool.
Particularly, the present invention proposes a kind of adaptive hierarchical disposal route stored based on object, as shown in Figure 1,3, the method comprises:
S101, object store operation arrive time, host process is calculating object temperature first, the data computing system capacity water level current according to system, after completing calculating, host process proceeds normal host process and stores operation, and to reach on water level in limited time at power system capacity water level, message power system capacity water level being reached the water level upper limit puts into monitoring water level queue.
Preferably, power system capacity water level refers to: the ratio in storage space between occupied space and the total volume of storage space.
S102, the classification process power system capacity water level read in monitoring water level queue reaches the message of the water level upper limit, according to the object temperature calculated, performs and is operated by the object data One Direction Transfer of high-performance storage pool to low performance storage pool.
Preferably, classification process is according to the object temperature calculated, perform the operation of data One Direction Transfers to refer to, the object in classification process traverse object storage system, according to the object temperature of object preferentially by object data minimum for object temperature by high-performance storage pool migration to low performance storage pool.
Preferably, the method also comprises: after the object migration with minimum object temperature completes by classification process, host process is computing system capacity water level again, if power system capacity water level is still more than or equal to the described water level upper limit, then the message that power system capacity water level is reached the water level upper limit by host process again puts into monitoring water level queue; Classification process reads the message in monitoring queue, and proceed migration operation based on object temperature, if power system capacity water level is less than the water level upper limit, then host process proceeds normal host process and stores operation, and carry out the calculating of power system capacity water level next time in the given time, classification process timing scan monitoring queue, waits for that next power system capacity water level reaches the message of the water level upper limit.
The present invention also proposes a kind of adaptive hierarchical disposal system 01 stored based on object, and as shown in Figure 2, this system comprises: host process module 02, classification scheduler module 03.
Host process module 02, during for storing operation arrival at object, first calculating object temperature, the data computing system capacity water level current according to system, after completing calculating, proceed normal host process and store operation, and to reach on water level in limited time at power system capacity water level, message power system capacity water level being reached the water level upper limit puts into monitoring water level queue.
Classification scheduler module 03, reaches the message of the water level upper limit for the power system capacity water level read in monitoring water level queue, according to the object temperature calculated, perform and operated by the object data One Direction Transfer of high-performance storage pool to low performance storage pool.
Preferably, classification scheduler module 03 also for, the object in traverse object storage system, according to the object temperature of object preferentially by object data minimum for object temperature by high-performance storage pool migration to low performance storage pool.
Preferably,
Host process module 02 also for: after the object migration with minimum object temperature completes by classification scheduler module 03, computing system capacity water level again, if power system capacity water level is still more than or equal to the water level upper limit, then the message again power system capacity water level being reached the water level upper limit puts into monitoring water level queue; If power system capacity water level is less than the water level upper limit, then proceeds normal host process and store operation, and carry out the calculating of power system capacity water level next time in the given time.
Classification scheduler module 03 also for: if power system capacity water level is still more than or equal to the water level upper limit, then reads the message in monitoring queue, and proceed migration operation based on object temperature; If power system capacity water level is less than the water level upper limit, then timing scan monitoring queue, waits for that next power system capacity water level reaches the message of the water level upper limit.
Such process decreases the impact on host process performance as much as possible, thus ensure that the access performance of system.
It should be noted that; above-described embodiment is only understand for the ease of those skilled in the art; be not limited to protection scope of the present invention; under the prerequisite not departing from inventive concept of the present invention, any apparent replacement and improvement etc. that those skilled in the art make the present invention are all within protection scope of the present invention.
Claims (7)
1., based on the adaptive hierarchical disposal route that object stores, it is characterized in that, described method comprises:
When object stores operation arrival, first host process calculates described object temperature, the data computing system capacity water level current according to system, after the calculation is completed, described host process proceeds normal host process and stores operation, and to reach on water level in limited time at described power system capacity water level, the message that described power system capacity water level reaches the water level upper limit is put into monitoring water level queue;
The classification process described power system capacity water level read in described monitoring water level queue reaches the message of the water level upper limit, according to the described object temperature calculated, performs and is operated by the object data One Direction Transfer of high-performance storage pool to low performance storage pool.
2. the method for claim 1, is characterized in that, described power system capacity water level refers to: the ratio in storage space between occupied space and the total volume of described storage space.
3. the method for claim 1, it is characterized in that, described classification process is according to the described object temperature calculated, perform described data sheet to refer to migration operation, described classification process travels through the object in described object storage system, according to the object temperature of described object preferentially by object data minimum for described object temperature by described high-performance storage pool migration to described low performance storage pool.
4. the method for claim 1, it is characterized in that, described method also comprises: after the object migration with minimum object temperature completes by described classification process, described host process calculates described power system capacity water level again, if described power system capacity water level is still more than or equal to the described water level upper limit, then the message that described power system capacity water level reaches the water level upper limit is put into described monitoring water level queue by described host process again; Described classification process reads the message in described monitoring queue, and proceed described migration operation based on described object temperature, if described power system capacity water level is less than the described water level upper limit, then described host process proceeds normal host process and stores operation, and carry out the calculating of described power system capacity water level next time in the given time, monitor queue described in described classification process timing scan, wait for that next described power system capacity water level reaches the message of the water level upper limit.
5., based on the adaptive hierarchical disposal system that object stores, it is characterized in that, described system comprises: host process module, classification scheduler module;
Described host process module, during for storing operation arrival at object, first described object temperature is calculated, the data computing system capacity water level current according to system, after the calculation is completed, proceed normal host process and store operation, and to reach on water level in limited time at described power system capacity water level, the message that described power system capacity water level reaches the water level upper limit is put into monitoring water level queue;
Described classification scheduler module, reaches the message of the water level upper limit for the described power system capacity water level read in described monitoring water level queue, according to the described object temperature calculated, perform and operated by the object data One Direction Transfer of high-performance storage pool to low performance storage pool.
6. system as claimed in claim 4, it is characterized in that, described classification scheduler module also for, travel through the object in described object storage system, according to the object temperature of described object preferentially by object data minimum for described object temperature by described high-performance storage pool migration to described low performance storage pool.
7. system as claimed in claim 4, is characterized in that,
Described host process module also for: after the object migration with minimum object temperature completes by described classification scheduler module, again calculate described power system capacity water level, if described power system capacity water level is still more than or equal to the described water level upper limit, then again the message that described power system capacity water level reaches the water level upper limit is put into described monitoring water level queue; If described power system capacity water level is less than the described water level upper limit, then proceeds normal host process and store operation, and carry out the calculating of described power system capacity water level next time in the given time;
Described classification scheduler module also for: if described power system capacity water level is still more than or equal to the described water level upper limit, then read the message in described monitoring queue, and proceed described migration operation based on described object temperature; If described power system capacity water level is less than the described water level upper limit, then monitor queue described in timing scan, wait for that next described power system capacity water level reaches the message of the water level upper limit.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410757697.7A CN104360961A (en) | 2014-12-10 | 2014-12-10 | Object storage-based self-adaptive graded processing method and object storage-based self-adaptive graded processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410757697.7A CN104360961A (en) | 2014-12-10 | 2014-12-10 | Object storage-based self-adaptive graded processing method and object storage-based self-adaptive graded processing system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104360961A true CN104360961A (en) | 2015-02-18 |
Family
ID=52528225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410757697.7A Pending CN104360961A (en) | 2014-12-10 | 2014-12-10 | Object storage-based self-adaptive graded processing method and object storage-based self-adaptive graded processing system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104360961A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106502576A (en) * | 2015-09-06 | 2017-03-15 | 中兴通讯股份有限公司 | Migration strategy method of adjustment, capacity change suggesting method and device |
| CN107103203A (en) * | 2017-05-22 | 2017-08-29 | 郑州云海信息技术有限公司 | A kind of storage AUTOMATIC ZONING system and method applied to PACS |
| CN108170789A (en) * | 2017-12-27 | 2018-06-15 | 郑州云海信息技术有限公司 | A kind of classification storage method and relevant apparatus based on distributed file system |
| CN111984549A (en) * | 2020-07-25 | 2020-11-24 | 苏州浪潮智能科技有限公司 | Hierarchical storage method and system for splitting data according to objects |
-
2014
- 2014-12-10 CN CN201410757697.7A patent/CN104360961A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106502576A (en) * | 2015-09-06 | 2017-03-15 | 中兴通讯股份有限公司 | Migration strategy method of adjustment, capacity change suggesting method and device |
| CN106502576B (en) * | 2015-09-06 | 2020-06-23 | 中兴通讯股份有限公司 | Migration strategy adjusting method and device |
| CN107103203A (en) * | 2017-05-22 | 2017-08-29 | 郑州云海信息技术有限公司 | A kind of storage AUTOMATIC ZONING system and method applied to PACS |
| CN108170789A (en) * | 2017-12-27 | 2018-06-15 | 郑州云海信息技术有限公司 | A kind of classification storage method and relevant apparatus based on distributed file system |
| CN111984549A (en) * | 2020-07-25 | 2020-11-24 | 苏州浪潮智能科技有限公司 | Hierarchical storage method and system for splitting data according to objects |
| CN111984549B (en) * | 2020-07-25 | 2022-07-19 | 苏州浪潮智能科技有限公司 | Hierarchical storage method and system for splitting data according to objects |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105407055B (en) | A kind of consumption control method of content center network | |
| JP2015534157A5 (en) | ||
| CN107276827A (en) | Qos implementation method and device in a kind of distributed memory system | |
| CN104360961A (en) | Object storage-based self-adaptive graded processing method and object storage-based self-adaptive graded processing system | |
| CN104573119A (en) | Energy-saving-oriented Hadoop distributed file system storage policy in cloud computing | |
| CN104536701A (en) | Realizing method and system for NVME protocol multi-command queues | |
| US11093410B2 (en) | Cache management method, storage system and computer program product | |
| CN109710175A (en) | Device and method for data storage management | |
| CN104571954A (en) | Method and device for storing data | |
| CN104202253B (en) | Network-on-chip jamming control method based on dynamic routing table | |
| CN102982122A (en) | Repeating data deleting method suitable for mass storage system | |
| CN103631894A (en) | Dynamic copy management method based on HDFS | |
| CN103345451A (en) | Data buffering method in multi-core processor | |
| CN106959893A (en) | Accelerator, EMS memory management process and data handling system for accelerator | |
| CN102136993A (en) | Data transfer method, device and system | |
| CN104572498B (en) | The buffer memory management method and device of message | |
| CN105302830A (en) | Map tile caching method and apparatus | |
| CN107491272A (en) | A kind of method, apparatus of Data Migration, equipment and storage medium | |
| CN106547477A (en) | For the method and apparatus for reducing buffer memory device online | |
| CN109445695A (en) | A kind of data migration method | |
| CN104503924B (en) | Region allocation method and device in a kind of Bedding storage system | |
| CN105242955A (en) | Virtual machine disk input and output scheduling method and system | |
| CN104951242B (en) | Method and apparatus for relocating data automatically between storage array | |
| CN106155923B (en) | The method and apparatus of memory sharing | |
| CN108388529A (en) | A kind of method that peripheral hardware actively realizes the data exchange between CPU |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150218 |
|
| WD01 | Invention patent application deemed withdrawn after publication |