CN117348789A

CN117348789A - Data access method, storage device, hard disk, storage system and storage medium

Info

Publication number: CN117348789A
Application number: CN202210737293.6A
Authority: CN
Inventors: 项伟杰; 杨滢; 陈俊伟; 史仁仁; 谢鑫
Original assignee: Chengdu Huawei Technology Co Ltd
Current assignee: Chengdu Huawei Technology Co Ltd
Priority date: 2022-06-27
Filing date: 2022-06-27
Publication date: 2024-01-05

Abstract

A data access method, a storage device, a hard disk, a storage system and a storage medium relate to the field of data storage. The storage device sequentially writes hot data to the hot-fluid cooling area of the logical disk, and the hot-fluid cooling area is also sequentially written to a first area of the hard disk, the first area containing a continuous address of a segment of the storage space provided by the hard disk. The self-adaptive heat flow cooling area is used for enabling the heat data in the heat flow cooling area to be just invalid, so that the problem that the write amplification of a logic disk is increased due to the fact that the heat data is repeatedly moved without being invalid and the cold flow area is extruded due to the fact that the heat flow cooling area is too large is avoided. When the thermal data in the thermal flow cooling area is recovered in sequence, all the data stored in the Block in the first area are also sequentially invalid, and the storage device recovers the invalid data in the Block in sequence, so that invalid moving of valid data in the hard disk is avoided, WA value in the hard disk is reduced, and the data access performance of the storage device and the service life of the hard disk are further improved.

Description

Data access method, storage device, hard disk, storage system and storage medium

Technical Field

The present disclosure relates to the field of data storage, and in particular, to a data access method, a storage device, a hard disk, a storage system, and a storage medium.

Background

A disk array (redundant arrays of independent drives, RAID) is a redundant array of multiple disks, which may be implemented as a stand-alone mass storage device. RAID can fully exploit the advantages of multiple hard disks, such as improving hard disk speed, providing fault tolerance to ensure data security, etc. If the storage media in RAID are all flash memories, the RAID is called a full flash memory storage array. Since the data erasure granularity of flash memory is larger than the data write granularity, the full flash memory array has the problem of write amplification (write amplification, WA).

Taking a full flash memory array as an example, a device master in the full flash memory array divides data into hot data or cold data, writes a plurality of hot data into a logical Block group (CKG) 1, writes a plurality of cold data into a CKG 2, and writes data in the CKG 1 and the CKG 2 into one or more data blocks (blocks) in a hard disk of the full flash memory array according to an address mapping relationship indicated by a flash memory conversion layer (flash translation layer, FTL). And the GC time of CKG 1 corresponding to hot data and the GC time of CKG 2 corresponding to cold data are different, so that the data failure time in the Block is different, and when the Block performs GC, the effective data in the Block occupy a relatively large amount, and the effective data need to be moved to other blocks, so that WA values between the device master controller and the blocks provided by the hard disk are increased, and the data access performance of the full flash memory array is reduced.

Therefore, how to provide a more efficient data access method is a problem that needs to be solved.

Disclosure of Invention

The application provides a data access method, storage equipment, a hard disk, a storage system and a storage medium, and solves the problem that WA values are large in the data access process.

In a first aspect, a data access method is provided, which is applicable to a data storage system or to a storage device supporting the data storage system to implement the data access method, e.g. the storage device comprising a chip. Here, a description will be given taking, as an example, a case where a storage device performs a data access method provided in this embodiment, the storage device including at least one hard disk for providing a storage space of a logical disk, the data access method including: firstly, a storage device acquires an Input Output (IO) stream, and takes data with access frequency greater than or equal to an access threshold value in the IO stream as hot data in a statistical period; secondly, the storage device sequentially writes the hot data into a hot-flow cooling area, wherein the hot-flow cooling area is a storage space which is provided by a logic disk and supports sequential writing; finally, the storage device writes the data in the heat flow cooling area into a first area of the hard disk in sequence according to the writing sequence of the data in the heat flow cooling area; the first area contains a continuous address in a storage space provided by the hard disk, and the continuous address is used for storing hot data.

In this embodiment, the storage device sequentially writes the hot data to the hot-stream cooling area of the logical disk, and the data in the hot-stream cooling area is also sequentially written to the first area of the hard disk, and the distribution of the plurality of hot data in the hard disk is continuous because the first area contains a continuous address in the storage space provided by the hard disk. During GC of the storage device, for the logical disk, a plurality of thermal data will be recovered sequentially within the thermal flow cooling zone; when the CKG of the heat flow cooling area comprises a plurality of heat data, the heat data are sequentially written into the first area of the hard disk, so that after the heat data in the heat flow cooling area are sequentially recovered, all data stored in the Block in the first area are sequentially invalid, and the storage device sequentially garbage recovers the invalid data in the Block, so that invalid moving of the valid data in the hard disk is avoided, WA value in the hard disk is reduced, data access performance of the storage device is improved, and service life of the hard disk is prolonged.

In an alternative implementation, the storage capacity of the hot-stream cooling zone is determined based on the time to failure of the thermal data; the storage capacity of the hot-stream cooling zone supports access to the hot data at least twice during the statistical period. In this embodiment, the storage device may dynamically adjust the storage capacity of the hot-flow cooling area according to the failure time of the hot data, so that the hot data in the hot-flow cooling area may be fully cooled and then GC may be performed, thereby avoiding the problem that the hot data is invalidity moved to the cold-flow area or other storage areas, and reducing the write amplification problem of the hot-flow cooling area.

For example, the expiration time may be determined based on a workload (workload) in the storage device. For example, the workload may be characterized by a write reuse distance of the thermal data, e.g., the storage device may determine the storage capacity of the thermal flow cooling zone based on the write reuse distance of the thermal data. The write reuse distance is the distance that two adjacent accesses of thermal data occur in the IO stream. The heat flow cooling area is a storage space for storing heat data in the logic disk provided by the embodiment, and because the storage capacity of the heat flow cooling area can be determined according to the write reuse distance of the heat data, and the write reuse distance is at least two times of the foregoing heat data accesses in the interval, after the heat data of the heat flow cooling area can be sufficiently cooled (failed), GC is performed, so that the problem that the heat data is not moved to the cold flow area or other storage areas by the storage device in an invalid condition is avoided, the write amplification problem of the heat flow cooling area is reduced, and the data access performance of the storage device is improved.

In an optional implementation manner, the data access method provided in this embodiment may further include: firstly, in a statistical period, the storage device takes data with access frequency smaller than an access threshold value in an IO stream as cold data; secondly, the storage device writes cold data into a cold flow area, wherein the cold flow area is a storage space which is provided by a logic disk and supports random writing; finally, the storage device randomly writes the data in the cold flow area into a second area of the hard disk; the second area is used for storing cold data. In this embodiment, the cold data is written into the storage space (cold flow area) supporting random writing in the logic disk by the storage device, and the hot data and the cold data are written into different storage spaces in the logic disk by the storage device respectively, so that the problem that more cold data need to be moved when the hot data GC exists is avoided, and the problem of write amplification of the storage device is alleviated. In addition, because the cold data is written into the second area of the hard disk in the storage device, the second area and the first area are two different areas provided by the hard disk, so that the cold data to be moved in the second area cannot be moved into the first area in the GC process of the hard disk, the zonal storage of the hot data and the cold data in the hard disk is realized, the hot data is stored in the first area supporting sequential writing and sequential release, the problem of larger WA value caused by random distribution of the hot data is avoided, and the data access performance of the storage device and the service life of the hard disk are improved.

In an optional implementation manner, the data access method provided in this embodiment may further include: and in the statistical period, if the storage capacity of the heat flow cooling area is occupied, the storage device performs GC according to the writing sequence of the data in the heat flow cooling area. Because the data in the heat flow cooling area is written by the storage device in sequence, the data stored in the heat flow cooling area is subjected to GC in sequence in the GC process of the heat flow cooling area, and the problem of the increase of the write amplification of the storage device caused by different failure time of the heat data is avoided.

It should be understood that after the data GC in the hot-fluid cooling zone, the data recovered in the hot-fluid cooling zone stored in the first zone fails, the data GC in the hot-fluid cooling zone is sequentially failed, and the data stored in the first zone also sequentially fails, so that the storage device performs GC on the sequentially failed data in the first zone. Because the storage space in the first area is continuous, for the Block included in the first area, the storage device does not need to move the invalid data stored in the Block, and the WA value between the hard disk and the logic disk is reduced.

In an optional implementation manner, the data access method provided in this embodiment may further include: if the hot-stream cooling zone also contains valid data that has not failed while the GC is performed in the hot-stream cooling zone, the storage device writes the valid data into the cold-stream zone. When the data in the heat flow cooling area is not accessed for a long time, the storage device sequentially writes the effective data in the heat flow cooling area into the cold flow area of the logic disk, and the storage device writes the cold data into the cold flow area of the logic disk, so that the problem of write amplification increase caused by invalid movement of the storage device in the GC process of the heat flow cooling area of the cold data is avoided, and the accurate cooling of the heat data in the heat flow cooling area of the storage device is facilitated.

In an optional implementation manner, the data access method provided in this embodiment may further include: and if the remaining space of the hard disk is smaller than or equal to the first threshold value, the storage device performs GC according to the writing sequence of the data in the first area. The storage device performs GC on a first area storing hot data according to the size of the remaining space of the hard disk, and since the data is sequentially written in the first area and the storage space included in the first area is a continuous address, hot data included in the continuous address is sequential. In other words, for one data Block (Block) in the hard disk, the thermal data is stored in the Block, the failure time of the thermal data is short, the thermal data in the Block can be failed together after the data GC in the thermal flow cooling area, the storage device performs GC according to the writing sequence of the data in the first area, the problem that the data which are not failed and the failure data exist in the data Block (Block) of the hard disk at the same time can be avoided, the GC can be performed after the data which are not failed in the Block are moved, the problem that the WA value between the logic disk and the hard disk is large is caused, and the data access performance of the storage device and the service life of the hard disk are improved.

In an optional implementation manner, the data access method provided in this embodiment may further include: after the GC is completed in the first area, if the residual space of the hard disk is smaller than or equal to a first threshold value, performing GC on blocks meeting the set conditions in the second area; the set conditions include: in the blocks of the second area, the weight of the failed data reaches a second threshold; the second threshold is associated with the first threshold. Because the access frequency of the storage device to the hot data is higher, the update speed of the hot data is higher, so that the GC is preferentially performed on the first area, and more storage space can be rapidly obtained to realize that the residual space of the hard disk is larger than or equal to the first threshold value. And the life cycle of the cold data in the second area is longer, and then GC is carried out on the second area, and the residual space in the second area is adjusted so as to realize the requirement that the hard disk is larger than or equal to the first threshold value.

In a second aspect, a data access apparatus is provided for use in a storage device in a data storage system, the data access apparatus may comprise means for performing any one of the implementations of the first aspect.

Illustratively, the storage device to which the data access apparatus applies includes one or more hard disks for providing physical storage space to the logical disk, the data access apparatus comprising: the device comprises an acquisition module, a hot spot distribution identification module, a first writing module and a second writing module.

And the acquisition module is used for acquiring the IO stream. And the hot spot distribution identification module is used for taking the data with the access frequency larger than or equal to the access threshold value in the IO stream as hot data in the statistical period. And the first writing module is used for sequentially writing the hot data into a hot-flow cooling area, wherein the hot-flow cooling area is a storage space which is provided by the logic disk and supports sequential writing. The second writing module is used for writing the data in the heat flow cooling area into the first area of the hard disk in sequence according to the writing sequence of the data in the heat flow cooling area; the first area contains a continuous address in a storage space provided by the hard disk, and the continuous address is used for storing hot data.

In an alternative implementation, the storage capacity of the hot-stream cooling zone is determined based on the time to failure of the thermal data; the storage capacity of the hot-stream cooling zone supports access to the hot data at least twice during the statistical period. Illustratively, the expiration time of the hot data is determined by the storage device based on a workload, which may be characterized by a write reuse distance of the hot data, the write reuse distance being the distance that the hot data has been accessed twice in an IO stream. For example, a hotspot distribution identification module may be used to determine the aforementioned write reuse distance.

In a possible implementation manner, the hotspot distribution identification module is further configured to use, as cold data, data in the IO stream, where the access frequency is less than the access threshold, in a statistical period. And the second writing module is used for writing the cold data into a cold flow area, wherein the cold flow area is a storage space which is provided by the logic disk and supports random writing. The second writing module is also used for randomly writing the data in the cold flow area into a second area of the hard disk; the second area is for storing cold data.

In a possible implementation manner, the data access device further includes: a GC module for: and if the storage capacity of the heat flow cooling area is occupied in the statistical period, performing GC according to the writing sequence of the data in the heat flow cooling area.

In a possible implementation, the GC module is configured to: when the GC is executed in the hot-flow cooling zone, if the hot-flow cooling zone also contains non-failure effective data, the effective data is written into the cold-flow zone.

In a possible implementation, the GC module is configured to: and if the remaining space of the hard disk is smaller than or equal to the first threshold value, performing GC according to the writing sequence of the data in the first area.

In a possible implementation, the GC module is configured to: after the first area completes GC, if the remaining space of the hard disk is less than or equal to the first threshold, GC is performed on the data blocks Block in the second area that satisfy the set condition. The set conditions include: in the blocks of the second area, the weight of the failed data reaches a second threshold; the second threshold is associated with the first threshold.

The advantages may be seen in the description of any one of the first aspects, which is not repeated here. The data access device has the functionality to implement the actions in the method instances of any of the above-described first aspects. The functions may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above.

In a third aspect, there is provided a storage device comprising: the device comprises a controller and at least one hard disk, wherein the at least one hard disk is used for providing physical storage space for a logic disk; the controller acquires IO streams and takes data with access frequency greater than or equal to an access threshold value in the IO streams in a statistical period as thermal data; the logic disk is provided with a heat flow cooling zone, wherein the heat flow cooling zone comprises a storage space supporting sequential writing in the logic disk and is used for storing heat data; the at least one hard disk comprises a first area, wherein the first area is a section of continuous reading address which supports sequential writing in a storage space provided by the at least one hard disk; the one-piece read-through address is used for storing hot data sequentially written by the controller in the hot-stream cooling area.

In an alternative implementation, the storage capacity of the hot-stream cooling zone is determined based on the time to failure of the thermal data; the storage capacity of the hot-stream cooling zone supports access to the hot data at least twice during the statistical period. Illustratively, the expiration time of the hot data is determined by the storage device based on a workload, which may be characterized by a write reuse distance of the hot data, the write reuse distance being the distance that the hot data has been accessed twice in an IO stream.

In an alternative implementation manner, the logic disk is further provided with a cold flow area, and the cold flow area is a storage space which is provided by the logic disk and supports random writing; the cold flow area is used for storing cold data, and the cold data comprises data with access frequency smaller than an access threshold value in a statistical period; the aforementioned at least one hard disk further comprises a second area for storing cold data randomly written by the controller in the cold flow area.

In an alternative implementation, if the storage capacity of the thermal flow cooling zone is already occupied during the statistical period, the controller performs GC according to the writing sequence of the data in the thermal flow cooling zone.

In an alternative implementation, when the GC is performed in the hot-stream cooling zone, if the hot-stream cooling zone still contains valid data that has not failed, the controller writes the valid data into the cold-stream zone.

In an alternative implementation manner, if the remaining space of the hard disk is less than or equal to the first threshold, the controller performs GC according to the writing sequence of the data in the first area.

In an optional implementation manner, after the GC is completed in the first area, if the remaining space of the hard disk is smaller than or equal to the first threshold, the controller performs GC on the data Block in the second area that meets the set condition; the set conditions include: in the blocks of the second area, the weight of the failed data reaches a second threshold; the second threshold is associated with the first threshold.

In a fourth aspect, there is provided a hard disk comprising: a first region and a second region; the first area comprises a section of continuous addresses supporting sequential writing in a storage space of the hard disk, and is used for storing hot data; the second area includes a storage space supporting random writing in a storage space of the hard disk, and is used for storing cold data.

In an alternative implementation manner, the first area is associated with a first interface provided by the hard disk, the first interface is used for providing data access for a computing device in communication with the hard disk, and the first interface supports a data access mode of sequential writing; the second area is associated with a second interface provided by the hard disk, the second interface being for providing data access to a computing device in communication with the hard disk, the second interface supporting a random write data access mode.

Illustratively, the data access manner of the first region includes writing data sequentially in physical block addresses (physical block address, PBA) by a first interface providing a semantic implementation.

In a fifth aspect, there is provided a storage system comprising a storage device provided by any implementation manner of the second aspect.

In a sixth aspect, a chip is provided, including: control circuitry and interface circuitry. The interface circuit is used for receiving signals from other chips outside the chip and transmitting the signals to the control circuit or sending the signals from the control circuit to the other chips outside the chip. The control circuitry and interface circuitry are operable to implement the steps of the method described in the first aspect or any one of the possible implementations of the first aspect by logic circuitry or executing code instructions.

In a seventh aspect, there is provided a computer readable storage medium comprising: computer software instructions; the computer software instructions, when executed in a storage device, cause the computing device to perform the operational steps of the method as described in the first aspect or any one of the possible implementations of the first aspect.

In an eighth aspect, there is provided a computer program product for, when run on a computer, causing a computing device to perform the operational steps of the method as described in the first aspect or any one of the possible implementations of the first aspect.

In a ninth aspect, a chip system is provided, the chip system comprising a processor for implementing the functions of the memory device or the controller in the memory device in the method of the first aspect. In one possible design, the chip system further includes a memory for holding program instructions and/or data. The chip system can be composed of chips, and can also comprise chips and other discrete devices.

Further combinations of the present application may be made to provide further implementations based on the implementations provided in the above aspects.

Drawings

FIG. 1 is a schematic diagram of a data storage system provided herein;

FIG. 2 is a schematic management diagram of a storage device provided in the present application;

FIG. 3 is a flowchart illustrating a method for accessing data according to the present application;

FIG. 4 is a second flow chart of the data access method provided in the present application;

FIG. 5 is a third flow chart of the data access method provided in the present application;

fig. 6 is a schematic structural diagram of a data access device provided in the present application.

Detailed Description

The present application provides a data access method performed by a storage device including at least one hard disk for providing a storage space of a logical disk, the data access method comprising: firstly, a storage device acquires an IO stream, and takes data with access frequency greater than or equal to an access threshold value in the IO stream as hot data in a statistical period; secondly, the storage device sequentially writes the hot data into a hot-flow cooling area, wherein the hot-flow cooling area is a storage space which is provided by a logic disk and supports sequential writing; finally, the storage device writes the data in the heat flow cooling area into a first area of the hard disk in sequence according to the writing sequence of the data in the heat flow cooling area; the first area contains a continuous address in a storage space provided by the hard disk, and the continuous address is used for storing hot data. In this embodiment, the storage device sequentially writes the hot data to the hot-stream cooling area of the logical disk, and the data in the hot-stream cooling area is also sequentially written to the first area of the hard disk, and the distribution of the plurality of hot data in the hard disk is continuous because the first area contains a continuous address in the storage space provided by the hard disk. In the data access process, the storage device can access data according to the writing sequence of a plurality of hot data, so that the data access performance of the storage device is improved. Moreover, during GC of the storage device, for the logical disk, a plurality of thermal data will be sequentially recovered within the thermal flow cooling zone; when the CKG of the heat flow cooling area comprises a plurality of heat data, the heat data are sequentially written into the first area of the hard disk, so that after the heat data in the heat flow cooling area are sequentially recovered, all data stored in the Block in the first area are sequentially invalid, and the storage device sequentially garbage recovers the invalid data in the Block, so that invalid moving of the valid data in the hard disk is avoided, WA values in the logic disk and the hard disk are reduced, and the data access performance of the storage device and the service life of the hard disk are improved.

To make the above embodiments clearer, an introduction of the related art will be given first.

FIG. 1 is a schematic structural diagram of a data storage system provided in the present application, the data storage system comprising: computing device 100 and storage device 120. In the application scenario shown in fig. 1, a user accesses data through an application program. A computer running these applications may be referred to as a "computing device. Computing device 100 may be a physical machine or a virtual machine. Physical computing devices include, but are not limited to, desktop computers, servers, notebook computers, and mobile devices.

In one possible example, computing device 100 accesses storage device 120 over a network to access data, which may include switch 110, for example.

In another possible example, computing device 100 may also communicate with storage device 120 over a wired connection, such as a universal serial bus (universal serial bus, USB) or a peripheral component interconnect express (Peripheral Component Interconnect Express, PCIe) bus, or the like.

The storage device 120 shown in fig. 1 may be a centralized storage system. The centralized storage system is characterized by a unified portal through which all data from external devices passes, which is the engine 121 of the centralized storage system. The engine 121 is the most central component of a centralized storage system in which many of the high-level functions of the storage system are implemented.

As shown in FIG. 1, there may be one or more controllers in the engine 121, and FIG. 1 illustrates that the engine 121 includes one controller. In one possible example, if the engine 121 has multiple controllers, any two controllers may have a mirror channel between them, so as to implement the function that any two controllers are backup to each other, so as to avoid hardware failure from causing the whole storage device 120 to be unusable. It should be appreciated that if the engine 121 includes multiple controllers, the engine 121 may also be referred to as an array controller for the storage device 120.

The engine 121 also includes a front end interface 1211 and a back end interface 1214, wherein the front end interface 1211 is used to communicate with the computing device 100 to provide data access services for the computing device 100. And a back-end interface 1214 is used to communicate with the hard disk to expand the capacity of the storage device 120. Through the back-end interface 1214, the engine 121 may connect more hard disks, thereby forming a very large pool of storage resources.

In hardware, as shown in fig. 1, the controller includes at least a processor 1212 and a memory 1213. The processor 1212 is a central processing unit (central processing unit, CPU) for processing data access requests from outside the storage device 120 (server or other storage system) and also for processing requests generated internally to the storage device 120. Illustratively, when the processor 1212 receives write data requests sent by the computing device 100 through the front-end interface 1211, the data in these write data requests is temporarily stored in the memory 1213. When the total amount of data in the memory 1213 reaches a certain threshold, the processor 1212 sends the data stored in the memory 1213 to at least one hard disk 1221, mechanical hard disk 1222, solid state disk (solid state drive, SSD) 1223 or other hard disks 1224 for persistent storage through the back-end port.

Memory 1213 refers to an internal memory that exchanges data directly with the processor, which can read and write data at any time and at a fast rate, as a temporary data store for the operating system or other program in operation. The memory includes at least two types of memories, for example, the memory may be a random access memory (ram) or a Read Only Memory (ROM). For example, the random access memory is a DRAM, or SCM. DRAM is a semiconductor memory, which, like most random access memories (random access memory, RAM), is a volatile memory (volatile memory) device. However, the DRAM and SCM are only exemplary in this embodiment, and the memory may also include other random access memories, such as static random access memories (static random access memory, SRAM), and the like. For read-only memory, for example, it may be a programmable read-only memory (programmable read only memory, PROM), erasable programmable read-only memory (erasable programmable read only memory, EPROM), etc.

In addition, the memory 1213 may be a dual in-line memory module or a dual in-line memory module (DIMM), i.e., a module composed of Dynamic Random Access Memory (DRAM), or an SSD. In practice, multiple memories 1213 may be configured in the controller, as well as different types of memories 1213. The number and type of the memory 1213 are not limited in this embodiment. In addition, the memory 1213 may be configured to provide power conservation. The power-up protection function refers to that when the system is powered down and powered up again, the data stored in the memory 1213 is not lost. The memory having the power-saving function is called a nonvolatile memory.

The memory 1213 stores a software program, and the processor 1212 runs the software program in the memory 1213 to manage the hard disk. For example, abstracting a hard disk into a storage resource pool, and providing the storage resource pool to a server for use in the form of a logical unit number (logical unit number, LUN), etc. The LUN here is in fact the hard disk seen on the server. Of course, some centralized storage systems are themselves file servers, which may be provided with shared file services.

As shown in fig. 1, in this system, the engine 121 may not have a hard disk slot, the hard disk needs to be placed in the hard disk frame 122, and the back-end interface 1214 communicates with the hard disk frame 122. The back-end interface 1214 exists in the engine 121 in the form of an adapter card, and two or more back-end interfaces 1214 may be simultaneously used on one engine 121 to connect a plurality of hard disk frames. Alternatively, the adapter card may be integrated onto the motherboard, in which case the adapter card may communicate with the processor 1212 via a PCIe bus.

It should be noted that, only one engine 121 is shown in fig. 1, however, in practical applications, two or more engines 121 may be included in the storage system, and redundancy or load balancing may be performed between the multiple engines 121.

The hard disk frame 122 includes a control unit 1225 and a number of hard disks. The control unit 1225 may have various forms. In one case, the hard disk frame 122 belongs to an intelligent disk frame, and as shown in fig. 1, the control unit 1225 includes a CPU and a memory. The CPU is used for performing address conversion, data reading and writing and other operations. The memory is used for temporarily storing data to be written into the hard disk or reading data to be transmitted to the controller from the hard disk. In another case, the control unit 1225 is a programmable electronic component, such as a data processing unit (data processing unit, DPU). The DPU has the versatility and programmability of the CPU, but more specialized, and can operate efficiently on network packets, storage requests, or analysis requests. The DPU is distinguished from the CPU by a large degree of parallelism (requiring handling of a large number of requests). Alternatively, the DPU may be replaced by a graphics processing unit (graphics processing unit, GPU), an embedded neural Network Processor (NPU), or the like. In general, the number of the control units 1225 may be one, two, or more. The functionality of the control unit 1225 may be offloaded onto the network card 1226. In other words, in this embodiment, the hard disk frame 122 does not have the control unit 1225 inside, but the network card 1226 performs data reading and writing, address conversion, and other computing functions. In this case, the network card 1226 is an intelligent network card. It may contain a CPU and memory. The CPU is used for performing address conversion, data reading and writing and other operations. The memory is used for temporarily storing data to be written into the hard disk or reading data to be transmitted to the controller from the hard disk. Or may be a programmable electronic component such as a DPU. The network card 1226 in the hard disk frame 122 has no attribution relationship with the hard disk, and the network card 1226 can access any hard disk in the hard disk frame 122 (such as the mechanical hard disk 1221, the mechanical hard disk 1222, the solid state hard disk 1223 and other hard disks 1224 shown in fig. 1), so that it is convenient to expand the hard disk when the storage space is insufficient.

The hard disk frame 122 may be a serial attached small computer system interface (serial attached small computer system interface, SAS) hard disk frame, or may be an NVMe (Non-Volatile Memory express) hard disk frame, or other types of hard disk frames, according to the type of communication protocol between the engine 121 and the hard disk frame 122. SAS hard disk blocks, adopting SAS3.0 protocol, each supporting 25 SAS hard disks. Engine 121 is connected to hard disk frame 122 through an onboard SAS interface or SAS interface module. The NVMe hard disk frame is more like a complete computer system, and the NVMe hard disk is inserted into the NVMe hard disk frame. The NVMe hard disk frame is in turn connected to the engine 121 via an RDMA port.

For example, storage device 120 may refer to a storage array, such as a full flash storage array where the storage medium is all flash memory.

In an alternative implementation, the storage device 120 is a centralized storage system with integrated disk control, the storage device 120 does not have the hard disk frame 122 described above, and the engine 121 is configured to manage multiple hard disks connected through a hard disk slot. The function of the hard disk slot may be performed by the back end interface 1214.

In another alternative implementation, the storage device 120 shown in FIG. 1 is a distributed storage system that includes a cluster of computing devices that include one or more computing devices that can communicate with each other and a cluster of storage devices. The computing device may be a computing device such as a server, a desktop computer, or a controller of a storage array, etc. On hardware, the computing device may include a processor, memory, network cards, and the like. Wherein the processor is a CPU for processing data access requests from outside the computing device or requests generated internally to the computing device. Illustratively, when the processor receives write data requests sent by a user, the data in the write data requests is temporarily stored in memory. When the total amount of data in the memory reaches a certain threshold value, the processor sends the data stored in the memory to the storage device for persistent storage. In addition, the processor is used for data calculation or processing, such as metadata management, data de-duplication, data compression, virtualized storage space, address translation, and the like. In one example, any one computing device may access any one of the storage devices in the cluster of storage devices over a network. The storage device cluster includes a plurality of storage devices. A storage device includes one or more controllers, a network card, and a plurality of hard disks, the network card for communicating with a computing device.

Taking the storage device 120 shown in fig. 1 as an example, a hard disk (or magnetic disk) management manner of a hard disk and a storage resource pool (logic disk) in the storage device is provided, taking the storage device 120 in the data storage system shown in fig. 1 as an example for illustration, as shown in fig. 2, fig. 2 is a management schematic diagram of the storage device provided in the application, where the storage device 120 provides a storage space for an external (such as a computing device) by using the logic disk 120B, and the storage space of the logic disk 120B is provided by the hard disk 120A. Among them, the hard DISK 120A may be referred to as a DISK side (DISK) of the storage device 120, and the logical DISK 120B may be referred to as a SYSTEM side (SYSTEM) of the storage device 120.

As shown in fig. 2, hard disk 120A may refer to any of the hard disks of fig. 1, such as mechanical hard disk 1221, mechanical hard disk 1222, solid state hard disk 1223, or other hard disk 1224 shown in fig. 1. In some cases, hard disk 120A may also be referred to as a physical disk of storage device 120.

In some alternative cases, the hard disk 120A may also refer to a collection of multiple hard disks in fig. 1, for example, the hard disk 120A refers to a collection of hard disks including the mechanical hard disk 1221, the mechanical hard disk 1222, the solid state hard disk 1223, and the other hard disks 1224 in fig. 1, which are not limited in this application.

As shown in fig. 2, the hard disk 120A includes a first area (area 1) and a second area (area 2).

Region 1 contains a segment of contiguous addresses in the memory space of hard disk 120A that support sequential writing, region 1 being used to store hot data.

Region 2 contains a storage space supporting random writing in the storage space of the hard disk 120A, and region 2 is used to store cold data.

It should be understood that hot data and cold data refer to data that differs in the access frequency of the data by the storage device 120 over a certain data access period, with the access frequency of hot data being greater than the access frequency of cold data.

For example, for an IO stream obtained by the storage device 120, in a statistical period, a controller of the storage device (e.g., a controller included in the engine 121 shown in fig. 1) may use data in the IO stream with an access frequency greater than or equal to an access threshold as hot data, and data in the IO stream with an access frequency less than the access threshold as cold data. The access threshold may be preset, or may be adjusted by the controller according to the data access requirement of the user, for example, the access threshold is 2, 3, or other values.

In one possible scenario, the statistical period is determined by the length of time that the data is accessed. Such as a statistical period of 10 seconds(s), 5 minutes (min), 2 hours (hours, h), or the like.

In another possible scenario, the statistical period is determined in terms of the amount of data accessed by the data. The data volume of the data access corresponding to the statistical period is as follows: 10 ten thousand data, 20 ten thousand data, 50 ten thousand data, or others.

In the embodiments provided herein, the statistical period is described by taking as an example that the data amount of the data access is determined.

The hard disk 120A includes a storage space having a plurality of data blocks (blocks), such as blocks 1 to 6 shown in fig. 2; it should be understood that fig. 2 is only an example of the storage device 120 provided in this embodiment, and the hard disk 120A may include more or fewer blocks, which is not limited in this application. Where each Block includes 5 data pages (pages), block 1 as shown in fig. 2 includes 5 data pages of written data.

Optionally, in the hard disk 120A, a plurality of different data access interfaces may be provided, so as to implement data access to the storage area corresponding to each Block in the storage space of the hard disk. Illustratively, the hard disk 120A is provided with a first interface and a second interface, which support different data access semantics, such as the first interface supporting sequential writing and the second interface supporting random writing.

For example, a first interface provided by hard disk 120A may be associated with region 1 for providing data access to a computing device in communication with hard disk 120A, the first interface supporting a sequential write data access scheme. The computing device in communication with the hard disk 120A may refer to a control unit included in the hard disk frame 122 in fig. 1, or the computing device 100.

For another example, a second interface provided by hard disk 120A may be associated with region 2 for providing data access to a computing device in communication with hard disk 120A, the second interface supporting at least a random write data access mode.

For hard disk 120A and logical disk 120B, a Block may be used to provide storage space for one or more CKGs, and each hard disk is illustratively divided into physical blocks (blocks) that are mapped into logical storage blocks (blocks) to form a storage pool, also referred to herein as a logical disk.

The storage pool is used to provide up storage space that is actually sourced from the hard disk contained in the storage device. Of course, not all hard disks need to provide space to the storage pool. In practice, the storage device may include one or more storage pools, where one storage pool includes some or all of the storage space provided by the hard disk. Multiple chunks from different hard disks or different storage nodes form a logical block group (CKG), which is the smallest allocation unit of the storage pool. When a controller applies for storage space from the storage pool, the storage pool may provide one or more logical block groups to the controller. The controller further virtualizes the storage space provided by the logical block group into Logical Units (LU) for use by the computing device. Each logical unit has a unique logical unit number (logical unit number, LUN). Since a computing device can directly perceive a logical unit number, those skilled in the art often refer to a logical unit directly with a LUN. Each LUN has a LUN ID that identifies the LUN. The specific location where data is located within a LUN may be determined by the starting address and the length (length) of the data. For the start address, those skilled in the art are generally referred to as logical block addresses (logical block address, LBA). It will be appreciated that three factors, LUN ID, LBA and length, identify a certain address field. The data access request generated by the application server typically carries the LUN ID, LBA and length in the request.

The number of chunks that a logical block group contains depends on what mechanism (also known as redundancy mode) is employed to ensure data reliability. In general, in order to ensure reliability of data, a storage device stores data using a multi-copy mechanism or an Erasure Coding (EC) check mechanism. Multiple copy mechanisms refer to storing at least two identical copies of data, where when one copy of the data is lost, other copies of data may be used for recovery. If a multiple copy mechanism is employed, a logical block group contains at least two chunks, each on a different hard disk or on a different storage node. The EC check mechanism is to divide the data to be stored into at least two data fragments, calculate the check fragments of the at least two data fragments according to a certain check algorithm, and when one data fragment is lost, the other data fragment and the check fragment can be utilized to recover the data. If an EC check mechanism is employed, a logical block group contains at least three chunks, each located on a different hard disk or on a different storage node.

With continued reference to FIG. 2, the logical disk 120B, which provides physical storage space for the hard disk 120A, may be provided with a hot-fluid cooling zone and a cold-fluid zone, wherein the hot-fluid cooling zone comprises storage space in the logical disk 120B that supports sequential writing, and the hot-fluid cooling zone may be used to store hot data; the cold flow area, which is a storage space provided for the logical disk to support random writing, may be used to store cold data.

In an alternative example, the hot-flow cooling zone may also be referred to as a hot data storage area in a logical disk, and the cold-flow zone may also be referred to as a cold data storage area in a logical disk, and the names of the two areas in the above logical disk are not limited in this application.

In some possible cases, reserved (OP) space may also be included in the logical disks and hard disks. For a logical disk, the OP space may provide spare storage space for the cold flow area for data access procedures such as data overwrite and data release. For a hard disk, the OP space may provide a spare memory space for performing data access processes such as data writing and data release in the first area (area 1) and the second area (area 2).

In the logical disk 120B, the storage space of the hot-stream cooling zone and the cold-stream zone may be provided by a plurality of logical block groups (CKGs), such as CKG 1 and CKG 2 in the hot-stream cooling zone and CKG 3 and CKG 4 in the cold-stream zone. It should be understood that fig. 2 is only an example of the storage device 120 provided in this embodiment, and the logical disk 120B may also include more or fewer CKGs, which is not limited in this application. As shown in fig. 2, the two data access interfaces that the storage device 120 may provide for the hard disk 120A may be provided with different Identities (IDs), such as id=0 for the first interface and id=1 for the second interface. When the storage device writes data in the logical disk 120B to the hard disk 120A, the controller included in the storage device may add an identification id=0 to the data of the heat flow cooling zone, the id=0 being used to indicate that the data is to be sequentially written by the controller to the area 1 in the hard disk 120A; and, the controller may add an identification id=1 to the data of the cold flow area, the id=1 indicating that the data is to be randomly written by the controller to the area 2 in the hard disk 120A.

The data access method provided in the present application is described in detail below with reference to the data storage system shown in fig. 1 and the storage device shown in fig. 2.

Fig. 3 is a schematic flow chart one of a data access method provided in the present application, which is applicable to the data storage system shown in fig. 1, and which is executed by the storage device 120 shown in fig. 2, and the data access method includes the following steps S310 to S340.

S310, the storage device acquires the IO stream.

An IO stream refers to a set of IO requests that includes a plurality of IO requests. If the IO request is a write request, each write request may carry a part of data and address information of the data; if the IO request is a read request, each read request may carry a memory address or the like of the requested data.

And S320, in the statistical period, the storage device takes the data with the access frequency greater than or equal to the access threshold value in the IO stream as hot data.

The data of the IO stream in the statistical period may be referred to as a data access period including a plurality of data in the data access period, as shown in fig. 3: data 1, data 2, data 3, data 4, data 5, data 6, data 2, data 3 and data 7. If the access threshold is 2, the thermal data of the IO stream in the statistical period includes: data 1, data 2, data 3, data 2 and data 3. For more details regarding the statistics period and access threshold, reference is made to the relevant description of fig. 2, which is not repeated here.

S330, the storage device writes the hot data into the hot-stream cooling area sequentially.

The hot-fluid cooling zone provides storage space for the logical disk that supports sequential writing. The sequential writing refers to that the storage device sequentially writes data into consecutive addresses in the storage space according to the access sequence of the data in the IO stream. If the hot-flow cooling area includes a continuous memory space from address 1 to address N, for data 1 to data N, data i will be stored and written in address i, i being any one of 1 to N.

In the conventional technology, if a storage area for storing the thermal data is too small, the thermal data is not moved; if the storage area storing hot data is too large, it may result in a decrease in the storage space (OP space) reserved for cold data in the logical disk, resulting in an increase in write amplification.

Optionally, the storage device may adjust the storage capacity of the thermal cooling zone according to the expiration time of the thermal data, and the storage capacity of the thermal flow cooling zone supports access to the thermal data at least twice during the statistical period. In this embodiment, the storage device may dynamically adjust the storage capacity of the hot-flow cooling area according to the failure time of the hot data, so that the hot data in the hot-flow cooling area may be fully cooled and then GC may be performed, thereby avoiding the problem that the hot data is invalidity moved to the cold-flow area or other storage areas, and reducing the write amplification problem of the hot-flow cooling area.

For example, the expiration time may be determined based on a workload (workload) in the storage device such that the CKGs exiting the hot cooling zone are just all hot cooled. For example, the workload may be characterized by a write reuse distance of the thermal data, e.g., the storage device may determine the storage capacity of the thermal flow cooling zone based on the write reuse distance of the thermal data. The write reuse distance is the distance that two adjacent accesses of thermal data occur in the IO stream.

The heat flow cooling area is a storage space for storing heat data in the logic disk provided by the embodiment, and because the storage capacity of the heat flow cooling area can be determined according to the write reuse distance of the heat data, and the write reuse distance is at least two times of the foregoing heat data accesses in the interval, after the heat data of the heat flow cooling area can be sufficiently cooled (failed), GC is performed, so that the problem that the heat data is not moved to the cold flow area or other storage areas by the storage device in an invalid condition is avoided, the write amplification problem of the heat flow cooling area is reduced, and the data access performance of the storage device is improved.

Moreover, the storage device can determine the storage capacity of the hot-flow cooling area according to the writing reuse distance of the hot data in the IO stream, so that the problem that the hot-flow cooling area occupies more storage space in the logic disk when the hot data in the IO stream is less, so that the storage space for the logic disk is not occupied is avoided, the storage space of the cold-flow area is further reduced, the cold-flow area can carry out GC in advance, and the writing amplification of the cold-flow area can be increased is avoided; and the problem that when the data in the IO stream is more, the data in the IO stream is invalid and moved to the cold flow area due to the fact that the data in the IO stream is stored in the storage device GC when the data in the IO stream is not invalid is avoided.

As an alternative implementation manner, the process of determining the heat flow cooling area by the storage device according to the writing reuse distance may be implemented by a hotspot distribution identification module deployed in the controller, where the hotspot distribution identification module is a software module or unit deployed in the controller, and is configured to calculate, by calculating the writing reuse distance distribution of data, the space size of the heat flow cooling area, and to instruct how long the CKG of the heat flow cooling area needs to be cooled before recycling garbage. The hot spot distribution identification module adjusts the storage space (storage capacity) of the hot-flow cooling area, so that the hot data is recovered sequentially after being just cooled, the hot data is prevented from being repeatedly moved without failure, meanwhile, the problem that the hot data occupies the storage space in the logic disk for a long time due to the fact that the hot-flow cooling area extrudes the storage space of the cold-flow area is avoided, and the resource utilization rate of the storage space in the logic disk is improved.

It is noted that the storage device is capable of adaptively adjusting the thermal cooling area according to the write reuse distance distribution, and can be applied to the field of data access to avoid invalid movement of thermal data immediately before failure. The method can also be used in the field of data compression, so that the hot data is prevented from being compressed in an invalid way, CPU resources in the storage device are wasted, and the data compression performance of the storage device is improved.

S340, the storage device writes the data in the heat flow cooling area into the first area of the hard disk according to the writing sequence of the data in the heat flow cooling area.

In this embodiment, the storage device sequentially writes the hot data to the hot-stream cooling area of the logical disk, and the data in the hot-stream cooling area is also sequentially written to the first area of the hard disk, and the distribution of the plurality of hot data in the hard disk is continuous because the first area contains a continuous address in the storage space provided by the hard disk.

In the data access process, the storage device can access data according to the writing sequence of a plurality of hot data, so that the data access performance of the storage device is improved. Moreover, during GC of the storage device, multiple thermal data will be recovered sequentially within the thermal flow cooling zone for the logical disk.

When the CKG of the heat flow cooling area comprises a plurality of heat data, the heat data are sequentially written into the first area of the hard disk, so that after the heat data in the heat flow cooling area are sequentially recovered, all data stored in the Block in the first area are sequentially invalid, and the storage device sequentially garbage recovers the invalid data in the Block, so that invalid moving of the valid data in the hard disk is avoided, WA value in the hard disk is reduced, data access performance of the storage device is improved, and service life of the hard disk is prolonged.

Optionally, the storage device may further write the cold data in the IO stream to other areas of the storage device, and on the basis of fig. 3, a possible implementation manner is provided, as shown in fig. 4, fig. 4 is a second schematic flow chart of the data access method provided in the present application, and the data access method provided in this embodiment may further include the following steps S350 to S370.

S350, in the statistical period, the storage device takes the data with the access frequency smaller than the access threshold value in the IO stream as cold data.

The content of the statistical period and the access threshold may be described with reference to fig. 2, and will not be described herein.

S360, the storage device writes cold data into the cold flow area.

Wherein the cold flow area is a storage space supporting random writing provided for the logic disk. It should be noted that in some alternative cases, the cold flow area may support other types of data access modes (e.g., sequential writing, intermittent writing), which are not limited in this application.

And S370, the storage device randomly writes the data in the cold flow area into a second area of the hard disk.

The second area is for storing cold data.

In this embodiment, the cold data is written into the storage space (cold flow area) supporting random writing in the logic disk by the storage device, and the hot data and the cold data are written into different storage spaces in the logic disk by the storage device respectively, so that the problem that more cold data need to be moved when the hot data GC exists is avoided, and the problem of write amplification of the storage device is alleviated.

In addition, since the cold data is written into the second area of the hard disk in the storage device, the second area and the first area are two different areas provided by the hard disk, so that the cold data to be moved in the second area cannot be moved into the first area in the GC process of the hard disk, the zonal storage of the hot data and the cold data in the hard disk is realized, the hot data is stored in the first area supporting sequential writing and sequential release, the problem of larger WA value caused by random distribution of the hot data is avoided, and the data access performance of the storage device and the service life of the hard disk are improved.

In the process of accessing data of the storage device by the computing device, the storage space of the storage device is inevitably occupied, so that the available storage space in the storage device is reduced, part of data in the storage device cannot be used continuously, the data which cannot be used continuously can be called garbage data (or invalid data, invalid data and the like) in the storage device, and therefore, in order to improve the utilization rate of storage resources in the storage device, a controller of the storage device can carry out garbage recycling (GC) on the garbage data.

On the basis of fig. 3 and fig. 4, the present embodiment provides a garbage collection method in a data access process, as shown in fig. 5, fig. 5 is a flowchart illustrating a third embodiment of a data access method provided in the present application, and the data access method provided in the present embodiment may include the following steps S510 to S540.

S510, if the storage capacity of the heat flow cooling area is occupied in the statistical period, the storage device performs GC according to the writing sequence of the data in the heat flow cooling area.

In this embodiment, since the data in the heat flow cooling area is sequentially written by the storage device, in the GC process of the heat flow cooling area, the data stored in the heat flow cooling area is sequentially GC, which avoids the problems that the heat difference of the heat data is large, the failure time is different, the heat data is repeatedly moved without failure, the cold flow area is extruded due to the overlarge heat flow cooling area, the heat data occupies the storage space in the logic disk for a long time, and the write amplification of the logic disk is increased, and the resource utilization rate of the storage resource in the storage device is improved.

S520, when the GC is executed in the hot-flow cooling area, if the hot-flow cooling area also contains non-failure effective data, the storage device writes the effective data into the cold-flow area.

The storage device, or "storage device," migrates valid data in the hot-flow cooling zone that has not failed into the cold-flow zone.

Taking CKG2 of the hot-stream cooling zone as an example in fig. 5, the first 4 data in CKG2 have failed in GC, and the storage device performs GC on the 4 data; since the 5 th data in the CKG2 is not invalid in GC, the CKG2 contains valid data (abbreviated as invalid data) which is not invalid, and the storage device writes the valid data which is not invalid into the cold flow area, so that the problem that the cooled data with a longer life cycle occupies the storage space in the hot flow cooling area, and the write amplification of the storage device is increased is avoided.

During data access, the hot data in the hot-stream cooling zone may remain valid after temperature cooling, and the still valid data in the hot-stream cooling zone of the storage device may be more closely related in life cycle to the cold data, and thus the still valid data may also be referred to as cooled data. The storage device writes the effective data (cooled data) into the cold flow area, so that the problem of increased write amplification of the storage device caused by different failure time of the effective data and other hot data in the hot flow cooling area is avoided.

It is noted that after the data GC in the hot-fluid cooling area, the data recovered in the hot-fluid cooling area stored in the first area fails, the data GC in the hot-fluid cooling area also fails sequentially, so that the data stored in one Block in the hard disk becomes garbage data at the same time, and thus the storage device performs GC on the data sequentially failed in the first area. Because the storage space in the first area is continuous, for the Block included in the first area, the storage device does not need to move the non-failure data stored in the Block, and the WA value between the hard disk and the logic disk is reduced.

S510 and S520 above are described with respect to GC of a logical disk corresponding to a storage device, and please refer to fig. 5, below, a GC process of a hard disk in the storage device is described.

And S530, if the remaining space of the hard disk is smaller than or equal to the first threshold value, the storage device performs GC according to the writing sequence of the data in the first area.

The first threshold refers to a storage margin threshold in the hard disk that is used to ensure that the hard disk supports at least some of the necessary data access services. In some alternative cases, the first threshold may be preset or may be adjusted according to the data access requirement of the user. If the first threshold is 100MB, 50MB, or the like, the present application is not limited thereto.

In the data access process, the storage device performs GC on a first area for storing hot data according to the size of the remaining space of the hard disk, and since the data in the first area is sequentially written and the storage space included in the first area is a segment of continuous addresses, the hot data included in the segment of continuous addresses are all sequential.

For one data Block (Block) in the hard disk, the thermal data is stored in the Block, the failure time of the thermal data is short, the thermal data in the Block can be failed together after the data GC in the thermal flow cooling area, the storage device carries out the GC according to the writing sequence of the data in the first area, the problem that the data access performance of the storage device and the service life of the hard disk are improved because the failure data and the non-failure data exist in the data Block (Block) of the hard disk at the same time, the GC can be carried out after the non-failure data in the Block of the first area are moved, and the WA value between the logic disk and the hard disk is large is solved. For example, the WA value between the thermal data in the hot stream cooling zone and the thermal data in the first zone is 1.

S540, after the first area completes GC, if the remaining space of the hard disk is smaller than or equal to a first threshold value, the storage device performs GC on blocks meeting the set conditions in the second area.

The set conditions include: in the blocks of the second area, the weight of the failed data reaches a second threshold; the second threshold is associated with the first threshold. The failed data refers to the garbage data and the failed data.

As an alternative specific embodiment, the storage device may GC the storage space in the second region according to a greedy release policy. The greedy release means that the storage device sorts the duty ratios of failure data (or called garbage data) of a plurality of blocks in the second area, and sequentially releases the data from the Block with the largest duty ratio of the failure data, so that the remaining space of the hard disk is larger than a first threshold value. Because the writing of the cold flow area is almost completely random, the CKG garbage amount of the cold flow area increases at a similar rate, the storage equipment performs GC on the cold flow area CKG under a greedy releasing strategy, and for a hard disk, the approximate sequential writing and sequential releasing of data are realized, and the writing amplification value of the partial area of the hard disk (disk side) is approximately 1.

Because the access frequency of the storage device to the hot data is higher, the update speed of the hot data is higher, so that the storage device preferentially carries out GC on the first area, and more storage space can be rapidly obtained to realize that the residual space of the hard disk is larger than or equal to the first threshold value. And the life cycle of the cold data in the second area is longer, then the GC is carried out on the second area, and the residual space in the second area is adjusted, so that the residual space of the hard disk is larger than or equal to the first threshold value, the problem that the hard disk in the storage device is occupied by the garbage data (invalid data) for a long time is avoided, and the data access performance of the storage device and the service life of the hard disk are improved.

It will be appreciated that, in order to implement the functions of the above embodiments, the storage device includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the elements and method steps of the examples described in connection with the embodiments disclosed herein may be implemented as hardware or a combination of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application scenario and design constraints imposed on the solution.

The hardware architecture of the storage device, the logical disk, and the deployment of the hard disk may refer to relevant contents of fig. 1, which are not described herein.

When the data access method of the present application is performed by software, this embodiment provides a possible implementation manner, as shown in fig. 6, fig. 6 is a schematic structural diagram of a data access device provided in the present application, where the data access device 600 includes: transceiver module 610, hotspot distribution identification module 620, first write module 630, second write module 640, and GC module 650.

The data access apparatus 600 may be used to implement the functions of the storage device or the controller included in the storage device in the above method embodiment, so that the beneficial effects of the above method embodiment can also be implemented. In this embodiment, the data access apparatus may be a data storage system or a storage device as shown in the foregoing drawings, or may be a module (e.g., a chip) applied to the storage device.

The transceiver module 610 is configured to obtain an IO stream.

The hotspot distribution identification module 620 is configured to take, as hot data, data in the IO stream, where the access frequency is greater than or equal to the access threshold, in a statistical period.

The first writing module 630 is configured to sequentially write the hot data into a hot-fluid cooling area, where the hot-fluid cooling area is a storage space provided by the logical disk and supporting sequential writing.

The second writing module 640 is configured to sequentially write the data in the hot-stream cooling area to the first area of the hard disk according to the writing order of the data in the hot-stream cooling area. The first area contains a continuous address in a storage space provided by the hard disk, and the continuous address is used for storing hot data.

In one possible example, the storage capacity of the hot-stream cooling zone is determined based on a time to failure of the thermal data; the storage capacity of the hot-stream cooling zone supports access to the hot data at least twice during the statistical period. Illustratively, the expiration time of the hot data is determined by the storage device based on a workload, which may be characterized by a write reuse distance of the hot data, the write reuse distance being the distance that the hot data has been accessed twice in an IO stream. For example, the hotspot distribution identification module 620 may be used to determine the write reuse distance described previously.

In a possible example, the hotspot distribution identification module 620 is further configured to use, as cold data, data in the IO stream having an access frequency less than the access threshold in a statistical period. The second writing module 640 is configured to write cold data into a cold flow area, where the cold flow area is a storage space provided by the logical disk and supporting random writing. A second writing module 640, configured to randomly write data in the cold flow area into a second area of the hard disk; the second area is for storing cold data.

In one possible example, GC module 650 is configured to: and if the storage capacity of the heat flow cooling area is occupied in the statistical period, performing GC according to the writing sequence of the data in the heat flow cooling area.

In one possible example, GC module 650 is configured to: when the GC is executed in the hot-flow cooling zone, if the hot-flow cooling zone also contains non-failure effective data, the effective data is written into the cold-flow zone.

In one possible example, GC module 650 is configured to: and if the remaining space of the hard disk is smaller than or equal to the first threshold value, performing GC according to the writing sequence of the data in the first area.

In one possible example, GC module 650 is configured to: after the first area completes GC, if the remaining space of the hard disk is less than or equal to the first threshold, GC is performed on the data blocks Block in the second area that satisfy the set condition. The set conditions include: in the blocks of the second area, the weight of the failed data reaches a second threshold; the second threshold is associated with the first threshold.

The more detailed descriptions of the transceiver module 610, the hotspot distribution identification module 620, the first writing module 630, the second writing module 640 and the GC module 650 may be directly obtained by referring to the related descriptions in the method embodiments shown in the foregoing drawings, which are not repeated herein.

It should be appreciated that the data access apparatus 600 of the embodiments of the present application may be implemented by a DPU. The data access apparatus 600 according to the embodiments of the present application may correspond to performing the methods described in the embodiments of the present application, and the above and other operations and/or functions of each unit in the data access apparatus 600 are respectively for implementing the corresponding flow of each method in the foregoing drawings, which are not repeated herein for brevity.

In some possible cases, the data access device provided by the application may be provided for the user to use in a form of a virtual machine, a container, a micro service (micro service), a cloud service (cloud service), or the like, so as to implement the data access method shown in the application. For example, when the data access device is provided to the user for use through a cloud service, the service provider may provide an access interface through which the user implements the data access method provided in the present application.

When the data access device implements the data access method shown in any of the foregoing drawings through software, the data access device and its respective units may also be software modules. The software module is called by the processor to realize the data access method. The processor may be a CPU, ASIC implementation, or programmable logic device (programmable logic device, PLD), which may be a complex program logic device (complex programmable logical device, CPLD), field programmable gate array (field programmable gate array, FPGA), general-purpose array logic (generic array logic, GAL), or any combination thereof.

For more detailed description of the data access device, reference may be made to the related description in the embodiment shown in the foregoing drawings, which is not repeated here. It will be appreciated that the data access device shown in the foregoing drawings is merely an example provided in this embodiment, and that different data access devices according to the data access process or service may include more or less units, which is not limited in this application.

When the data access means is implemented by hardware, the hardware may be implemented by a processor or a chip. The chip includes an interface circuit and a control circuit. The interface circuit is used for receiving data from other devices outside the processor and transmitting the data to the control circuit or sending the data from the control circuit to the other devices outside the processor.

The control circuitry and interface circuitry are operable to implement the methods of any of the possible implementations of the above embodiments by logic circuitry or executing code instructions. The advantages may be seen from the description of any of the above embodiments, and are not repeated here.

It is to be appreciated that the processor in embodiments of the present application may be a CPU, NPU, or GPU, but may also be other general purpose processor, digital signal processor (digital signal processor, DSP), ASIC, FPGA or other programmable logic device, transistor logic device, hardware components, or any combination thereof. The general purpose processor may be a microprocessor, but in the alternative, it may be any conventional processor.

In addition, the data access apparatus 600 shown in fig. 6 may also be implemented by a storage device, such as the storage device 120 shown in fig. 1, or a data storage system including a storage device, or the like.

The method steps in the embodiments of the present application may also be implemented by way of a processor executing software instructions. The software instructions may be comprised of corresponding software modules that may be stored in random access memory (random access memory, RAM), flash memory, ROM, programmable ROM, erasable Programmable ROM (EPROM), electrically erasable programmable EPROM (EEPROM), registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. In addition, the ASIC may reside in a communication device. The processor and the storage medium may reside as discrete components in a communication device.

The application also provides a chip system which comprises a processor and is used for realizing the functions of the storage device in the method. In one possible design, the chip system further includes a memory for holding program instructions and/or data. The chip system can be composed of chips, and can also comprise chips and other discrete devices.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are performed in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, a network device, a user device, or other programmable apparatus. The computer program or instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer program or instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center by wired or wireless means. The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that integrates one or more available media. The usable medium may be a magnetic medium, e.g., floppy disk, hard disk, tape; optical media, such as digital video discs (digital video disc, DVD); but also semiconductor media such as solid state disks (solid state drive, SSD).

While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of data access, the method performed by a storage device comprising one or more hard disks for providing physical storage space for logical disks, the method comprising:

acquiring IO streams;

in the statistical period, taking the data with the access frequency larger than or equal to the access threshold value in the IO stream as hot data;

sequentially writing the hot data into a hot-flow cooling area, wherein the hot-flow cooling area is a storage space which is provided by the logic disk and supports sequential writing;

sequentially writing the data in the heat flow cooling area into a first area of the hard disk according to the writing sequence of the data in the heat flow cooling area; the first area contains a continuous address in a storage space provided by the hard disk, and the continuous address is used for storing hot data.

2. The method of claim 1, wherein the storage capacity of the hot-stream cooling zone is determined based on a time to failure of the thermal data; the storage capacity of the thermal flow cooling zone supports access to the thermal data at least twice during the statistical period.

3. The method according to claim 1 or 2, characterized in that the method further comprises:

in the statistical period, taking the data with the access frequency smaller than the access threshold value in the IO stream as cold data;

writing the cold data into a cold flow area, wherein the cold flow area is a storage space supporting random writing provided by the logic disk;

randomly writing data in the cold flow area into a second area of the hard disk; the second area is for storing cold data.

4. A method according to claim 3, characterized in that the method further comprises:

and in the statistical period, if the storage capacity of the heat flow cooling area is occupied, performing garbage collection GC according to the writing sequence of the data in the heat flow cooling area.

5. The method according to claim 4, wherein the method further comprises:

and when the GC is executed in the hot-flow cooling zone, if the hot-flow cooling zone also contains non-failure effective data, writing the effective data into the cold-flow zone.

6. The method according to any one of claims 3 to 5, further comprising:

and if the remaining space of the hard disk is smaller than or equal to a first threshold value, performing GC according to the writing sequence of the data in the first area.

7. The method of claim 6, wherein the method further comprises:

after the first area completes GC, if the remaining space of the hard disk is smaller than or equal to the first threshold, GC is performed on the data blocks Block in the second area that satisfy the set condition;

the set conditions include: in the blocks of the second area, the weight of the failed data reaches a second threshold; the second threshold is associated with the first threshold.

8. A memory device, comprising: the device comprises a controller and at least one hard disk, wherein the at least one hard disk is used for providing physical storage space for a logic disk;

the controller acquires IO streams and takes data with access frequency greater than or equal to an access threshold value in the IO streams in a statistical period as hot data;

the logic disk is provided with a heat flow cooling zone, and the heat flow cooling zone comprises a storage space supporting sequential writing in the logic disk; the heat flow cooling zone is used for storing the heat data;

The at least one hard disk comprises a first area, wherein the first area is a section of continuous reading address which supports sequential writing in a storage space provided by the at least one hard disk; the one-segment read-through address is used for storing hot data sequentially written by the controller in the hot-flow cooling area.

9. The storage device of claim 8, wherein a storage capacity of the thermal flow cooling zone is determined based on a time to failure of the thermal data; the storage capacity of the thermal flow cooling zone supports access to the thermal data at least twice during the statistical period.

10. The storage device according to claim 8 or 9, wherein,

the logic disk is also provided with a cold flow area, and the cold flow area is a storage space which is provided by the logic disk and supports random writing; the cold flow area is used for storing cold data, and the cold data comprises data with access frequency smaller than the access threshold value in the statistical period;

the at least one hard disk further comprises a second area for storing cold data randomly written by the controller in the cold flow zone.

11. The storage device of claim 10, wherein the controller performs garbage collection GC in the order of writing data in the hot-stream cooling zone if the storage capacity of the hot-stream cooling zone has been occupied during the statistical period.

12. The storage device of claim 11, wherein the controller writes valid data to the cold flow zone if the hot flow cooling zone also contains non-failed valid data while the GC is performed by the hot flow cooling zone.

13. The storage device according to any one of claims 10 to 12, wherein if a remaining space of the hard disk is less than or equal to a first threshold, the controller performs GC in the order of writing data in the first area.

14. The storage device of claim 13, wherein after the first area completes GC, if the remaining space of the hard disk is less than or equal to the first threshold, the controller performs GC on the data Block in the second area that satisfies the set condition;

15. A hard disk comprising: a first region and a second region;

the first area comprises a section of continuous addresses supporting sequential writing in a storage space of the hard disk, and is used for storing hot data;

The second area comprises a storage space supporting random writing in the storage space of the hard disk, and is used for storing cold data.

16. The hard disk of claim 15, wherein the first region is associated with a first interface provided by the hard disk, the first interface for providing data access to a computing device in communication with the hard disk, the first interface supporting a sequential write data access;

the second area is associated with a second interface provided by the hard disk, the second interface is used for providing data access for computing equipment in communication with the hard disk, and the second interface supports a random writing data access mode.

17. A storage system, comprising: the storage device of any one of claims 8-14.

18. A computer readable storage medium, characterized in that the storage medium has stored therein a computer program or instructions which, when executed by a controller in a storage device, implement the method of any of claims 1-7.

19. A data access apparatus for use with a storage device comprising one or more hard disks for providing physical storage space for logical disks, the data access apparatus comprising:

The acquisition module is used for acquiring the IO stream;

the hotspot distribution identification module is used for taking data with the access frequency greater than or equal to an access threshold value in the IO stream as hot data in a statistical period;

the first writing module is used for sequentially writing the hot data into a hot-flow cooling area, wherein the hot-flow cooling area is a storage space which is provided by the logic disk and supports sequential writing;

the second writing module is used for writing the data in the heat flow cooling area into the first area of the hard disk in sequence according to the writing sequence of the data in the heat flow cooling area; the first area contains a continuous address in a storage space provided by the hard disk, and the continuous address is used for storing hot data.