WO2015184710A1

WO2015184710A1 - Method and device for processing magnetic disk hibernation

Info

Publication number: WO2015184710A1
Application number: PCT/CN2014/087721
Authority: WO
Inventors: 刘文文
Original assignee: 中兴通讯股份有限公司
Priority date: 2014-06-05
Filing date: 2014-09-28
Publication date: 2015-12-10
Also published as: CN105138416A

Abstract

A method and device for processing magnetic disk hibernation. The method comprises: counting the access frequency for accessing each storage block (chunk) in a first magnetic disk; judging whether the access frequency for accessing the chunk reaches a predetermined threshold value; in the case where the judgement result is yes, migrating the chunk of which the access frequency exceeds the predetermined threshold value in the first magnetic disk into a second magnetic disk, wherein the performance of the second magnetic disk is higher than that of the first magnetic disk; and conducting hibernation on the first magnetic disk. By means of this solution, the problem in the related art that the system performance is wasted due to hibernation conducted only at a fixed time and unnecessary fragment repair existing in the magnetic disk hibernation technology is solved, thereby achieving the effects of realizing the dynamic hibernation of a magnetic disk and increasing the system performance of the magnetic disk.

Description

Disk sleep processing method and device

Technical field

The present invention relates to the field of communications, and in particular to a disk sleep processing method and apparatus.

Background technique

Erasure Coding (EC) is a data protection method that divides data into segments, expands, encodes, and stores them in different locations, such as disks, storage nodes, or Other locations. It creates a mathematical function that describes a set of numbers so that they can be checked for accuracy and can be recovered once one of the numbers is lost. Polynomial interpolation or oversampling is the key technique used by erasure codes.

EC mode can be used in applications or systems that have large amounts of data and any fault tolerance, such as disk array systems, data grids, distributed storage applications, object storage, or archive storage. Currently, a common use case for erasure codes is object-based cloud storage.

In the related art, the disk array disk hibernation is empirically configured, and the disk sleep is selected at a fixed time when the system read operation is less, so that file access is also performed during the sleep process, which is particularly obvious in the EC mode. Because sleep will greatly increase the amount of file access calculations, causing unnecessary fragmentation repairs a lot of system performance and improving maintenance costs, especially in EC environments.

Therefore, in the disk sleep scheme of the related art, there are the following disadvantages: Disk hibernation can only sleep the disk at a fixed time. It is not guaranteed that during the long-term operation, the system is idle for a fixed period of time, or the idle period is not used properly. As a result, the disk hibernation function is basically unavailable. For the EC environment, sleeping disks are also accessed due to different storage rules. This results in a large number of unnecessary fragmentation repairs, which consumes a lot of system performance. The shortcomings of the prior art are particularly serious.

Therefore, in the related art, there is a defect in system performance waste caused by only fixed-time sleep and unnecessary fragmentation repair in the disk sleep technology.

Summary of the invention

The invention provides a disk sleep processing method and device, which can at least solve the defects in the related art that the system performance waste caused by only fixed time sleep and unnecessary fragment repair exists in the disk sleep technology.

According to an aspect of the present invention, a disk sleep processing method is provided, including: calculating an access heat of accessing each memory block chunk in the first disk; determining whether the access heat of accessing the chunk reaches a predetermined threshold; If the result of the determination is yes, the chunk of the first disk whose access heat exceeds the predetermined threshold is migrated to the second disk, wherein the performance of the second disk is higher than the first disk; The disk is dormant.

Preferably, if there are at least two of the first disks, sleeping the first disk comprises: sleeping the first disk in a predetermined time by using a predetermined idle sleep.

Preferably, after the sleeping of the first disk, the method further comprises: waking up the first disk when at least one of the following conditions is met: the second disk reaches a predetermined storage threshold, exists in the first disk A chunk that exceeds the predetermined threshold.

Preferably, if the second disk reaches the predetermined storage threshold, the waking up the first disk includes: sorting the chunks in the second disk according to the access heat of the chunk; The chunk whose access heat is queued at the end of the queue is migrated to the first disk.

Preferably, the migrating the chunk of the first disk with the access heat exceeding the predetermined threshold to the second disk comprises: counting the chunks in the first disk, sorting according to the access heat; A chunk exceeding the predetermined threshold is migrated to the second disk.

According to another aspect of the present invention, a disk dormancy processing apparatus is provided, comprising: a statistics module configured to count access heat for accessing each memory block chunk in the first disk; and a determining module configured to determine access to the chunk Whether the access heat reaches a predetermined threshold; the migration module is configured to migrate the chunk of the first disk whose access heat exceeds the predetermined threshold to the second disk if the determination result of the determining module is yes The performance of the second disk is higher than the first disk; and the hibernation module is configured to sleep the first disk.

Preferably, the hibernation module includes: a hibernation unit configured to sleep the first disk in a predetermined time in a predetermined time in a case where the first disk has at least two.

Preferably, the apparatus further comprises: a wake-up module configured to wake up the first disk when at least one of the following conditions is met: the second disk reaches a predetermined storage threshold, and the first disk exceeds the predetermined The chunk of the threshold.

Preferably, the waking module includes: a first sorting unit, configured to: in case the second disk reaches the predetermined storage threshold, the chunk in the second disk is in accordance with the access heat of the chunk High Low sorting; the first migration unit is configured to migrate the chunk whose access heat is queued at the end of the queue to the first disk.

Preferably, the migration module includes: a second sorting unit configured to count the chunks in the first disk according to the access heat; and the second migration unit is configured to exceed the predetermined threshold The chunk is migrated to the second disk.

According to the present invention, the access heat of the access to each memory block chunk in the first disk is statistically determined; whether the access heat of accessing the chunk reaches a predetermined threshold is determined; if the determination result is yes, the first The chunk of the disk whose access heat exceeds the predetermined threshold is migrated to the second disk, wherein the performance of the second disk is higher than that of the first disk; and the first disk is dormant, which solves the related art, the disk dormancy technology There are only defects in system performance waste caused by fixed time sleep and unnecessary fragmentation repair, thereby achieving the effect of realizing dynamic sleep of the disk and improving the performance of the disk system.

DRAWINGS

The drawings described herein are intended to provide a further understanding of the invention, and are intended to be a part of the invention. In the drawing:

1 is a flowchart of a disk sleep processing method according to an embodiment of the present invention;

2 is a block diagram showing the structure of a disk sleep processing apparatus according to an embodiment of the present invention;

3 is a block diagram showing a preferred structure of a sleep module 28 in a disk sleep processing apparatus according to an embodiment of the present invention;

4 is a block diagram showing a preferred structure of a disk sleep processing apparatus according to an embodiment of the present invention;

FIG. 5 is a block diagram showing a preferred structure of a wake-up module 42 in a disk sleep processing apparatus according to an embodiment of the present invention; FIG.

6 is a block diagram showing a preferred structure of a migration module 26 in a disk sleep processing apparatus according to an embodiment of the present invention;

7 is a schematic diagram of a hierarchical storage dynamic sleep disk system architecture according to a preferred embodiment of the present invention;

FIG. 8 is a schematic diagram of interaction of a dynamic disk sleep method according to a preferred embodiment of the present invention; FIG.

9 is a flow chart of a method for dynamically sleeping a disk in accordance with a preferred embodiment of the present invention;

FIG. 10 is a schematic diagram of interaction of a dynamic disk wakeup method according to a preferred embodiment of the present invention; FIG.

11 is a flow chart of a method for dynamically awakening a disk in accordance with a preferred embodiment of the present invention.

detailed description

The invention will be described in detail below with reference to the drawings in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments may be combined with each other without conflict.

A disk sleep processing method is provided in this embodiment. FIG. 1 is a flowchart of a disk sleep processing method according to an embodiment of the present invention. As shown in FIG. 1 , the process includes the following steps:

Step S102, collecting the access heat of accessing each chunk of the chunk in the first disk;

Step S104, determining whether the access heat of accessing the chunk reaches a predetermined threshold;

In step S106, if the result of the determination is yes, the chunk of the first disk whose access heat exceeds a predetermined threshold is migrated to the second disk, wherein the performance of the second disk is higher than that of the first disk;

Step S108, sleeping on the first disk.

Through the above steps, the chunks are migrated between disks according to the access heat of the chunks in the disk, so that the chunks with high access can be migrated to the high-performance disks, and the chunks that access the low chunks or even the non-accesses can be concentrated to the hot ones. The disk can further hibernate the less hot disk, which not only solves the defect of system performance waste caused by only fixed-time sleep and unnecessary fragmentation in the disk sleep technology, thereby achieving the realization. The dynamic sleep of the disk improves the performance of the disk system.

It should be noted that, in the case that there are at least two first disks, when the first disk is dormant, the following processing may be adopted: the first disk is dormant in a predetermined time in a predetermined time.

It should be noted that the dormancy of the first disk is to achieve dynamic sleep of the first disk, and when the storage of the hot disk, that is, the storage of the second disk reaches a predetermined storage threshold, or the predetermined threshold is exceeded in the first disk, When the chunk is hibernated, the first disk can be woken up after the first disk is hibernated. On the one hand, the hot disk overload can be effectively avoided, and on the other hand, the idle disk can be obtained. use efficiently.

When the disk is dynamically hibernated, it involves the dynamic migration of chunks on the disk. You can use a variety of methods to dynamically migrate a chunk of a disk. Here is a description of how to handle queue priority.

For example, in the case that the second disk reaches the predetermined storage threshold, the following processing may be adopted when waking up the first disk: the chunks in the second disk are sorted according to the access heat of the chunk (that is, a queue with a certain priority is formed). ); migrate the chunks whose access heat is queued at the end of the queue to the first disk.

For another example, a similar process may be adopted for migrating a chunk of the first disk whose access heat exceeds a predetermined threshold to the second disk: sorting the chunks in the first disk according to the access heat (that is, forming a priority) Queue); migrating chunks that exceed a predetermined threshold to the second disk. The queue priority processing method can not only improve the processing efficiency, but also is simple and convenient.

In the embodiment, a disk sleep processing device is also provided, which is used to implement the above embodiments and preferred embodiments, and has not been described again. As used below, the term "module" may implement a combination of software and/or hardware of a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.

2 is a structural block diagram of a disk sleep processing apparatus according to an embodiment of the present invention. As shown in FIG. 2, the apparatus includes a statistics module 22, a determination module 24, a migration module 26, and a sleep module 28. The device will be described below.

The statistic module 22 is configured to count the access heat of accessing the chunks of the first disk; the determining module 24 is connected to the statistic module 22, and is configured to determine whether the access heat of accessing the chunk reaches a predetermined threshold; 26, connected to the foregoing determining module 24, configured to migrate the chunk of the first disk whose access heat exceeds a predetermined threshold to the second disk, wherein the performance of the second disk is higher than the determination result of the determining module is YES The first disk; the hibernation module 28, connected to the migration module 26, is configured to sleep on the first disk.

3 is a block diagram showing a preferred structure of the hibernation module 28 in the disk hibernation processing apparatus according to the embodiment of the present invention. As shown in FIG. 3, the hibernation module 28 includes a sleep unit 32, and the sleep unit 32 will be described below.

The sleep unit 32 is configured to sleep the first disk in a predetermined time in a predetermined time in the case where there are at least two of the first disks.

FIG. 4 is a block diagram showing a preferred structure of a disk sleep processing apparatus according to an embodiment of the present invention. As shown in FIG. 4, the apparatus includes a wake-up module 42 in addition to all the structures shown in FIG. Be explained.

The wake-up module 42 is connected to the above-described hibernation module 28, and is configured to wake up the first disk when at least one of the following conditions is met: the second disk reaches a predetermined storage threshold, and a chunk exceeding a predetermined threshold exists in the first disk.

FIG. 5 is a block diagram of a preferred structure of the wake-up module 42 in the disk sleep processing apparatus according to the embodiment of the present invention. As shown in FIG. 5, the wake-up module 42 includes: a first sorting unit 52 and a first migration unit 54, The wakeup module 42 is described.

The first sorting unit 52 is configured to, in the case that the second disk reaches a predetermined storage threshold, sort the chunks in the second disk according to the access heat of the chunk; the first migration unit 54 is connected to the first sort. Unit 52 is configured to migrate the chunks whose access heat is queued at the end of the queue to the first disk.

FIG. 6 is a block diagram showing a preferred structure of the migration module 26 in the disk sleep processing apparatus according to the embodiment of the present invention. As shown in FIG. 6, the migration module 26 includes: a second sorting unit 62 and a second migration unit 64. The migration module 26 is described.

The second sorting unit 62 is configured to sort the chunks in the first disk according to the access heat; the second migration unit 64 is connected to the second sorting unit 62, and is configured to migrate the chunks exceeding the predetermined threshold to the second Disk.

In the related art, in the disk sleep technology, there is only a fixed-time sleep, and the system performance waste caused by unnecessary fragmentation repair. In this embodiment, a dynamic sleep of the disk is implemented on the basis of hierarchical storage. Program. The architecture of a disk-based dynamic sleep disk system based on hierarchical storage is described below.

FIG. 7 is a schematic diagram of a hierarchical storage dynamic sleep disk system architecture according to a preferred embodiment of the present invention. As shown in FIG. 7, the system includes: a chunk access statistics module 70 and an IO load reporting module 72. 22), chunk access statistics management module 74 (functioning with the above-mentioned judging module 24), chunk hot and cold scheduling module 76 (functioning with the above-mentioned migration module 26) and disk sleep control module 78 (functioning with the above-described hibernation module 28). The following is an example in which the first disk is a Serial Advanced Technology Attachment (SATA) and the second disk is a Solid State Disk (SSD). The performance of the second disk is higher than that of the first disk, that is, the second disk has higher performance than the first disk in terms of read/write capability, storage capability, and ability to process statistical information.

The chunk access statistic module 70, on the FAC side, mainly reports the statistics of the read byte count of the user to the chunk access statistics management module 74. When the user reads the file, the module reports the access statistics of the user to the file to the chunk access statistical management module 74 when the file is closed and the file is closed. The access statistics of chunk include, chunky (chunk), the number of bytes accessed. Multiple accesses accumulate the number of bytes per access.

The chunk access statistics management module 74 calculates the current revenue based on the statistical information reported by the FAC on the FLR side, and modifies the revenue field in the database. The module collects access statistics on the chunk, calculates the access gain to the chunk in the statistical period, and invokes the interface of the database to modify the revenue field of the chunk table chunk.

The chunk hot and cold dispatching module 76, on the database side, calculates the chunk's revenue according to the chunk's historical access revenue and the current access revenue, and constructs a chunk upgrade and downgrade priority queue according to the chunk's access revenue. And notify FLR to carry out the migration of chunks between SSD and SATA.

The disk sleep control module 78 is on the FAS side. After the hotspot chunk is successfully migrated, some SATA disks are hibernated. When the SSD space reaches the threshold, when the hotspot chunk needs to be downgraded, the sleeping SATA disk is dynamically woken up.

The SSD management module is a support module. You need to add management to the SSD disk, including disk identification, volume creation, volume deletion, and alarm reporting recovery.

FIG. 8 is a schematic diagram of interaction of a dynamic sleep mode of a disk according to a preferred embodiment of the present invention. As shown in FIG. 8 , the dynamic sleep method of the disk includes the following process: a cache upgrade request is initiated by a scheduling module of the FLR, and hotspot data of the SATA disk is migrated. Go to the SSD and delete the data on the SATA disk. And modify the chunk location information in the database. After the hotspot migration is completed, the disk that is about to sleep is scanned as a chunk and reported to the chunk-level access statistics module of the FLR. N blocks SATA disks in sequence. After the migration is successful, the chunk is placed in the chunk revenue degradation queue. In the tiered storage migration period (in hours) set by the system, the migration traffic between the hot and cold chunks is initiated. The database collects the length of the migration queue on the FAS and the queue length of the normal read and write access. The chunk whose read/write queue length is within the threshold and whose migration queue length is within the set threshold initiates the migration process, notifying the FLR that the chunk will be chunked. Migrate from SATA storage to L1 cache. After the migration, sleep N non-hotspot storage SATA disks.

FIG. 9 is a flowchart of a method for dynamically sleeping a disk according to a preferred embodiment of the present invention. As shown in FIG. 9, the process includes the following steps:

Step S902, starting to classify the chunk;

Step S904, it is determined whether the threshold of degradation is reached, if the determination result is yes, proceed to step S906, otherwise return to step S902;

Step S906, enabling the downgrade process;

Step S908, obtaining the volume load situation, including: migration queue length, IO read/write queue length;

Step S910, determining whether the read/write queue length is less than a preset threshold, if the determination result is yes, proceeding to step S912, otherwise returning to step S902;

Step S912, determining whether the migration queue length is less than a predetermined threshold, if the determination result is yes, proceeding to step S914, otherwise returning to step S902;

Step S914, taking a chunk from the head of the migration queue;

Step S916, notifying the FLR to migrate to the level 1 cache destination volume;

Step S918, it is determined whether the migration is successful, if the determination result is yes, proceed to step S920, otherwise return to step S912;

Step S920, deleting the chunk from the migration upgrade queue;

Step S922, placing the chunk into the degraded priority queue;

Step S924, modifying the location information of the chunk, and deleting the source chunk;

Step S926, counting all the chunks on the SATA disc and recording;

In step S928, the SATA disk is hibernated, and the process returns to step S912.

FIG. 10 is a schematic diagram of interaction of a dynamic wake-up method of a disk according to a preferred embodiment of the present invention. As shown in FIG. 10, the dynamic wake-up method of the disk includes the following process: when the SSD disk space reaches a threshold, the chunk needs to be downgraded. The downgrade request is initiated by the FLR debug module, and the FAS responds to the dynamic wakeup of the dormant disk. After the migration is successful, modify the chunk location information in the FLR database. And remove the chunk from the downgrade queue. When the disk chunk gain in the sleep state reaches the hotspot threshold, the sleep disk is woken up. In the level 1 cache, the space reaches a set threshold, such as 80%, and then the degradation processing of the hotspot chunk needs to be initiated. The FAS is notified to wake up the dormant disk. The downgrade process removes the low-revenue chunk from the tail end of the chunk demotion queue, migrates from the SSD disk to the SATA disk, deletes the data on the SSD disk, and notifies the FLR to modify the chunk location information.

11 is a flowchart of a method for dynamically waking up a disk according to a preferred embodiment of the present invention. As shown in FIG. 11, the process includes the following steps:

Step S1102: Obtain a list of all level 1 cache volumes;

Step S1104, taking a volume;

Step S1106, it is determined whether the volume space usage has reached a predetermined threshold, if the determination result is yes, proceeds to step S1108, otherwise returns to step S1104;

Step S1108, taking out a chunk with low profit from the tail of the chunk demotion queue;

Step S1110, waking up the sleeping SATA disk;

Step S1112, notifying the FLR to migrate the chunk to the SATA from the primary storage volume SSD;

Step S1114, it is determined whether the migration is successful, if the determination result is yes, proceed to step S1116, otherwise return to step S1106;

Step S1116, deleting the chunk from the tail of the migration queue;

Step S1118, deleting the physical chunk in the SSD;

In step S1120, the location information of the database chunk is modified.

In addition, it should be noted that the above embodiments and preferred embodiments can be widely used in disk storage that uses mass storage for a long time.

It will be apparent to those skilled in the art that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein. The steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module. Thus, the invention is not limited to any specific combination of hardware and software.

The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention, and various modifications and changes can be made to the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and scope of the present invention are intended to be included within the scope of the present invention.

Industrial applicability

As described above, the above embodiments and the preferred embodiments solve the defects in the related art that the system performance waste caused by only fixed-time sleep and unnecessary fragmentation repair exists in the disk sleep technology, thereby achieving the realization of the disk. Dynamic sleep, improving the performance of the disk system.

Claims

A disk sleep processing method includes:

Counting the access heat of accessing each chunk of the chunk in the first disk;

Determining whether the access heat of accessing the chunk reaches a predetermined threshold;

If the result of the determination is yes, the chunk of the first disk whose access heat exceeds the predetermined threshold is migrated to the second disk, wherein the performance of the second disk is higher than that of the first disk;

Sleeping the first disk.
The method of claim 1, wherein, if there are at least two of the first disks, sleeping the first disk comprises:

The first disk is hibernated in a predetermined time by a predetermined period of sleep.
The method of claim 1, wherein after the first disk is hibernated, the method further comprises:

The first disk is woken up when at least one of the following conditions is met: the second disk reaches a predetermined storage threshold, and a chunk exceeding the predetermined threshold exists in the first disk.
The method of claim 3, wherein, if the second disk reaches the predetermined storage threshold, waking up the first disk comprises:

Sorting the chunks in the second disk according to the access heat of the chunks;

The chunk whose access heat is queued at the end of the queue is migrated to the first disk.
The method of claim 1, wherein migrating a chunk of the first disk with a heat of access exceeding the predetermined threshold to the second disk comprises:

And counting the chunks in the first disk according to the access heat;

A chunk exceeding the predetermined threshold is migrated to the second disk.
A disk sleep processing device includes:

The statistics module is configured to count the access heat of accessing each chunk of the storage block in the first disk;

a determining module, configured to determine whether the access heat of accessing the chunk reaches a predetermined threshold;

a migration module, configured to migrate a chunk of the first disk whose access heat exceeds the predetermined threshold to a second disk, where the result of the determining by the determining module is YES, wherein the performance of the second disk Higher than the first disk;

The hibernation module is configured to sleep on the first disk.
The apparatus of claim 6, wherein the hibernation module comprises:

The sleep unit is configured to sleep the first disk in a predetermined time in a predetermined time when there is at least two of the first disks.
The apparatus of claim 6 further comprising:

The wake-up module is configured to wake up the first disk when at least one of the following conditions is met: the second disk reaches a predetermined storage threshold, and a chunk exceeding the predetermined threshold exists in the first disk.
The apparatus of claim 8, wherein the wake-up module comprises:

a first sorting unit, configured to sort the chunks in the second disk according to the access heat of the chunks when the second disk reaches the predetermined storage threshold;

The first migration unit is configured to migrate the chunk whose access heat is queued at the end of the queue to the first disk.
The apparatus of claim 6 wherein the migration module comprises:

a second sorting unit, configured to sort the chunks in the first disk according to the access heat;

And a second migration unit configured to migrate a chunk exceeding the predetermined threshold to the second disk.