WO2022062015A1

WO2022062015A1 - Data storage device and stored data migration method

Info

Publication number: WO2022062015A1
Application number: PCT/CN2020/123033
Authority: WO
Inventors: 王毅; 杨淏; 陈家贤; 廖好; 周池; 毛睿
Original assignee: 深圳大学
Priority date: 2020-09-28
Filing date: 2020-10-23
Publication date: 2022-03-31
Also published as: CN112214169B; CN112214169A

Abstract

Disclosed in the present application are a data storage device and a stored data migration method. The data storage device comprises a volatile memory module and a shingled magnetic recording disk module located below the volatile memory module, wherein the shingled magnetic recording disk module comprises multiple layers of storage areas, each storage area consists of a plurality of basic storage units, the storage space of the storage area on the next layer is N times that of the storage area on the adjacent upper layer, and the size of each storage area is dynamically adjusted according to the total capacity of the shingled magnetic recording disk to automatically adapt to an optimal read and write state. The stored data migration method provided by the present application first analyzes and stores data, and re-migrates the frequently accessed hot data according to preset conditions to a place where the data can be found more easily, increasing search speed, improving the storage stability and reliability of data to a great extent, and also increasing the speed of adding, deleting, checking and modifying.

Description

A data storage device and method for migrating stored data

technical field

The present application relates to the technical field of storage data processing, and in particular, to a data storage device and a method for migrating stored data.

Background technique

Various data storage systems have been widely used in data management production systems to handle a large number of write operations. And it has very efficient applications in solid state drives and mechanical hard drives. Most of the existing optimization methods for improving data storage only consider the direct combination of data and hardware, but do not consider the problem from software application, which makes the read and write overhead of stored data high, and the read and write efficiency needs to be improved.

SUMMARY OF THE INVENTION

Therefore, the technical problem to be solved by the present application is to overcome the defects in the prior art that the data storage and read-write overhead is high and the read-write efficiency needs to be improved, so as to provide a data storage device and a method for migrating stored data.

To achieve the above purpose, the application provides the following technical solutions:

In a first aspect, embodiments of the present application provide a data storage device, including: a volatile memory module and a shingled magnetic recording disk module located under the volatile memory module, wherein the shingled magnetic recording disk module includes multiple layers Storage area, each storage area is composed of several basic storage units. The storage space of the storage area located in the next layer is N times that of the storage area immediately above it. According to the total capacity of the shingled magnetic recording disk, the dynamic Adjust the size of each layer's storage area.

In one embodiment, the number of layers in which the shingled magnetic recording disk module divides the storage area and the storage basic unit included in each layer can be adjusted.

In one embodiment, the following formula is used to adjust the size of each storage area:

θ _i =ρ _i +η×δ _i +η×Δ

Among them, the parameter θ _i represents the storage area size of the i-th layer after the change, the parameter ρ _i represents the initial storage area size, the parameter δ _i represents the change of the constraints in the i-th layer storage area, and the parameter η represents the data The read-write ratio of , the parameter Δ represents the correction amount.

In a second aspect, an embodiment of the present application provides a method for migrating stored data. Based on the data storage device described in the first aspect of the embodiment of the present application, the method includes the following steps:

When there is a command to write, delete and modify data, it will be written to the volatile memory module first. When the data in the volatile memory module is greater than the preset threshold, data migration will be performed, and the data will be migrated to the volatile memory module. In the storage area of the first layer in the adjacent shingled magnetic recording disk module of the flexible memory module, when the storage area of the first layer is full, the data will be migrated down to the storage area of the second layer. , and so on;

When the data storage device receives the query instruction, it first searches in the volatile memory module, and returns the result if found; otherwise, searches layer by layer starting from the first layer of storage area in the shingled magnetic recording disk module, and finally return result;

Obtain the data access frequency of each layer of storage area in the shingled magnetic recording disk module, when the access frequency satisfies the first preset condition, move up one layer, and directly migrate to the volatile memory if the access frequency satisfies the second preset condition in the module.

In one embodiment, when the data access frequency f _i satisfies the first preset condition expressed by the following formula, it is moved to the upper layer:

Among them, i>1, f _i-1 is the access frequency of data on the i-1 layer storage area, γ is a constant parameter that controls the migration floating frequency, and η is the read-write ratio of the current workload.

In one embodiment, when the data access frequency satisfies the second preset condition expressed by the following formula, the data is directly migrated to the volatile memory module:

Among them, i>1, f _{i-1, max} represents the maximum access frequency of data in the storage area of the i-1 layer, γ is a constant parameter that controls the floating frequency of migration, and η is the read-write ratio of the current workload.

In a third aspect, the embodiments of the present application provide a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the first aspect of the embodiments of the present application. Migration method of stored data.

In a fourth aspect, an embodiment of the present application provides a computer device, including: a memory and a processor, the memory and the processor are connected in communication with each other, the memory stores computer instructions, and the processor executes the The computer instructions are executed, thereby executing the method for migrating stored data described in the first aspect of the embodiments of the present application.

The technical solution of the present application has the following advantages:

The present application provides a data storage device and a method for migrating stored data. The data storage device includes a volatile memory module and a shingled magnetic recording disk module located under the volatile memory module, wherein the shingled magnetic recording disk module Including multi-layer storage areas, each storage area is composed of several basic storage units, the storage space of the storage area located in the next layer is N times that of the storage area immediately above it, according to the shingled magnetic recording disk. The total capacity dynamically adjusts the size of each storage area so that it automatically adapts to the optimal read and write state. The storage data migration method provided by the present application performs a re-migration operation for hot data that is accessed frequently according to preset conditions, migrates it to a place where it is easier to find, speeds up the search, and can greatly improve the The storage stability and reliability of the data can also speed up the speed of addition, deletion and modification.

Description of drawings

In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the accompanying drawings that need to be used in the description of the specific embodiments or the prior art will be briefly introduced below. The drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is a schematic structural diagram of a data storage device provided in an embodiment of the present application;

Fig. 2 is the working flow chart of reading data provided in the embodiment of this application;

FIG. 3 is a workflow diagram of query data provided by the embodiment of the present application;

FIG. 4 is a composition diagram of a specific example of a computer device provided by an embodiment of the present application.

detailed description

The technical solutions of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as there is no conflict with each other.

Example 1

An embodiment of the present application provides a data storage device, which is mainly used in a data storage system, such as a storage database system in personal business storage, data server, etc., as shown in FIG. The shingled magnetic recording disk module below the flexible memory module, wherein the shingled magnetic recording disk module includes multiple storage areas, each storage area is composed of several basic storage units, and the storage space of the storage area located in the lower layer The size of each storage area is dynamically adjusted according to the total capacity of the shingled magnetic recording disk to be N times the storage area of the layer immediately above it.

In the embodiment of the present application, the difference between the volatile memory module and the shingled magnetic recording disk module is that the volatile memory module has a faster read and write speed than the shingled magnetic recording disk module, but data is easily lost when power is lost , Although the shingled magnetic recording disk module has a slower read and write speed, the stored data is not easy to lose, and the data is stored in layers inside, and the storage space in the lower layer is larger than the next layer. The storage space of the volatile memory module and the shingled magnetic recording disk module can be the same or different, which is determined according to the user-defined settings, which is not limited here.

In a specific embodiment, the shingled magnetic recording disk module is divided into 8 layers of storage areas, each layer of storage area is composed of several storage basic units, and the storage space of the storage area located in the next layer is the storage area immediately adjacent to the upper layer. 16 times the storage area, for example, the size of the first layer of storage area is t, the size of the second layer of storage area is 16t, the size of the third storage area is 16*16t, and so on. The size of all the storage areas is accumulated to be the size of the entire shingled magnetic recording disk module. The above number of layers and the size of each layer area are only examples and not limited thereto.

In the embodiment of the present application, the number of layers of the storage area divided by the shingled magnetic recording disk module and the basic storage unit contained in each layer can be adjusted, and the design can be made according to the characteristics of the file size and the capacity of the shingled magnetic recording disk. The size of each storage area is dynamically adjusted according to the total capacity of the magnetic recording disk. Specifically, the following formula is used to adjust the size of each storage area:

θ _i =ρ _i +η×δ _i +η×Δ

Among them, the parameter θ _i represents the storage area size of the i-th layer after the change, the parameter ρ _i represents the initial storage area size, the parameter δ _i represents the change of the constraints in the i-th layer storage area, and the parameter η represents the data read Write ratio, parameter Δ represents the correction amount. It should be noted that the parameters in the above formula are determined according to actual application requirements or according to experimental experience values, and are not limited herein.

In the embodiment of the present application, the number of layers in the storage area divided by the shingled magnetic recording disk module and the basic storage unit included in each layer can be adjusted, and the specific characteristics such as file size and data type can be stored according to the needs or the storage needs of practical applications. , make reasonable adjustments.

In the data storage device provided by the embodiment of the present application, the size of each storage area is dynamically adjusted according to the total capacity of the shingled magnetic recording disk, so that it can be automatically adjusted to an optimal read-write state, which can improve data processing efficiency.

Example 2

An embodiment of the present application provides a method for migrating stored data, including the following steps:

Step S10: When a command to write, delete and modify data occurs, write to the volatile memory module first, and perform data migration when the data in the volatile memory module is greater than a preset threshold, and migrate the data to the volatile memory module. In the storage area of the first layer in the shingled magnetic recording disk module adjacent to the volatile memory module, when the storage area of the first layer is full, the data is migrated down to the second layer of storage area. tier storage area, and so on. The above preset thresholds are adapted according to actual application requirements.

Figure 2 shows the workflow of data writing as an example. The deletion and modification commands are similar to the data writing process. The difference is that the deleted workflow needs to be written by marking. In the embodiment of the present application, the involved addition, deletion and modification commands include:

INSERT Key Value//Insert KV pair;

UPDATE Key Value//Update KV pair;

READ Key//Data read as Key;

DELETE Key//Delete the data that is the key. When deleting the data, the Value value is null (NULL, that is, does not exist). The Value value in the delete command is used as a marker to determine whether the data is really stored.

Step S20: when the data storage device receives the query instruction, it first searches in the volatile memory module, if found, returns the result; otherwise, starts from the first layer storage area in the shingled magnetic recording disk module layer by layer. Search, and finally return the result, and its workflow is shown in Figure 3.

Step S30: Acquire the data access frequency of each layer of storage areas in the shingled magnetic recording disk module, move up one layer when the access frequency meets the first preset condition, and directly migrate to the easy-to-use storage area if the access frequency meets the second preset condition. in the volatile memory module. When the access frequency of data is greater than a certain value, it indicates that it is relatively important hot data, and it is necessary to migrate the data to the upper layer to a storage layer that is easier to read to reduce processing overhead and improve reading efficiency.

In this embodiment of the present application, when the data access frequency f _i satisfies the first preset condition expressed by the following formula, it is moved to the upper layer:

In this embodiment of the present application, when the data access frequency satisfies the second preset condition expressed by the following formula, the data is directly migrated to the volatile memory module:

In the storage data migration method provided by the present application, for hot data with high access frequency, the re-migration operation is performed according to preset conditions such as the load read-write ratio, the constant parameter of the migration floating frequency, the maximum access frequency of the data in the upper-layer storage area, etc. Migrating it to a place where it is easier to find and speeding up the search can greatly improve the storage stability and reliability of the data, and at the same time, it can speed up the speed of additions, deletions, and changes.

Example 3

An embodiment of the present application provides a computer device. As shown in FIG. 4 , the device may include a processor 51 and a memory 52, where the processor 51 and the memory 52 may be connected through a bus or in other ways. FIG. 4 takes the connection through a bus as an example .

The processor 51 may be a central processing unit (Central Processing Unit, CPU). The processor 51 can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or Other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components and other chips, or a combination of the above types of chips.

As a non-transitory computer-readable storage medium, the memory 52 can be used to store non-transitory software programs, non-transitory computer-executable programs and modules, such as corresponding program instructions/modules in the embodiments of the present application. The processor 51 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 52 , ie, implements the stored data migration method in the above method embodiment 1.

The memory 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor 51, and the like. Additionally, memory 52 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 52 may optionally include memory located remotely from processor 51 , which may be connected to processor 51 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, intranets, mobile communication networks, and combinations thereof.

One or more modules are stored in the memory 52, and when executed by the processor 51, execute the stored data migration method in Embodiment 1.

The specific details of the above computer equipment can be understood by referring to the corresponding related descriptions and effects in Embodiment 1, and details are not repeated here.

Those skilled in the art can understand that the realization of all or part of the processes in the methods of the above embodiments is a program that can be completed by instructing relevant hardware through a computer program and can be stored in a computer-readable storage medium. When the program is executed , which may include the processes of the above-mentioned method embodiments. The storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive) , abbreviation: HDD) or solid-state drive (Solid-State Drive, SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memories.

Obviously, the above-mentioned embodiments are only examples for clear description, and are not intended to limit the implementation manner. For those of ordinary skill in the art, on the basis of the above description, other different forms of changes or modifications can also be made. There is no need and cannot be exhaustive of all implementations here. However, the obvious changes or changes derived from this are still within the protection scope of the present invention.

Claims

A data storage device, characterized by comprising: a volatile memory module and a shingled magnetic recording disk module located at a lower layer of the volatile memory module, wherein the shingled magnetic recording disk module includes a multi-layer storage area, each The storage area is composed of several basic storage units. The storage space of the storage area located in the next layer is N times that of the storage area immediately above it. The storage area of each layer is dynamically adjusted according to the total capacity of the shingled magnetic recording disk. the size of.
The data storage device according to claim 1, wherein the number of layers for dividing the storage area by the shingled magnetic recording disk module and the storage basic unit included in each layer are adjustable.
The data storage device according to claim 1, wherein the following formula is used to adjust the size of each storage area:

θ i =ρ i +η×δ i +η×Δ

Among them, the parameter θ i represents the storage area size of the i-th layer after the change, the parameter ρ i represents the initial storage area size, the parameter δ i represents the change of the constraints in the i-th layer storage area, and the parameter η represents the data read Write ratio, parameter Δ represents the correction amount.
A method for migrating stored data, characterized in that, based on the data storage device according to any one of claims 1-3, comprising the following steps:

When there is a command to write, delete and modify data, it will be written to the volatile memory module first. When the data in the volatile memory module is greater than the preset threshold, data migration will be performed, and the data will be migrated to the volatile memory module. In the storage area of the first layer in the adjacent shingled magnetic recording disk module of the flexible memory module, when the storage area of the first layer is full, the data will be migrated down to the storage area of the second layer. , and so on;

When the data storage device receives the query instruction, it first searches in the volatile memory module, and returns the result if found; otherwise, searches layer by layer starting from the first layer of storage area in the shingled magnetic recording disk module, and finally return result;

Obtain the data access frequency of each layer of storage area in the shingled magnetic recording disk module, when the access frequency satisfies the first preset condition, move up one layer, and directly migrate to the volatile memory if the access frequency satisfies the second preset condition in the module.
The method for migrating stored data according to claim 4, wherein when the data access frequency f i satisfies the first preset condition expressed by the following formula, it is moved to the upper layer:

Among them, i>1, f i-1 is the access frequency of data on the i-1 layer storage area, γ is a constant parameter that controls the migration floating frequency, and η is the read-write ratio of the current workload.
The method for migrating stored data according to claim 4, wherein when the data access frequency satisfies the second preset condition expressed by the following formula, the data is directly migrated to the volatile memory module:

Among them, i>1, f i-1, max represents the maximum access frequency of data in the storage area of the i-1 layer, γ is a constant parameter that controls the floating frequency of migration, and η is the read-write ratio of the current workload.
A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the data storage process according to any one of claims 4-6. Migration method.
A computer device, characterized in that it comprises: a memory and a processor, wherein the memory and the processor are connected in communication with each other, the memory stores computer instructions, and the processor executes the computer instructions, thereby The method for migrating stored data according to any one of claims 4-6 is performed.