CN113467699B - Method and device for improving available storage capacity - Google Patents

Method and device for improving available storage capacity Download PDF

Info

Publication number
CN113467699B
CN113467699B CN202010239040.7A CN202010239040A CN113467699B CN 113467699 B CN113467699 B CN 113467699B CN 202010239040 A CN202010239040 A CN 202010239040A CN 113467699 B CN113467699 B CN 113467699B
Authority
CN
China
Prior art keywords
data
information
memory
computing device
solid state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010239040.7A
Other languages
Chinese (zh)
Other versions
CN113467699A (en
Inventor
杜阁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202010239040.7A priority Critical patent/CN113467699B/en
Publication of CN113467699A publication Critical patent/CN113467699A/en
Application granted granted Critical
Publication of CN113467699B publication Critical patent/CN113467699B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604Improving or facilitating administration, e.g. storage management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/0652Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0662Virtualisation aspects
    • G06F3/0664Virtualisation aspects at device level, e.g. emulation of a storage device or system
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0668Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671In-line storage system
    • G06F3/0673Single storage device
    • G06F3/0679Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]

Abstract

The application discloses a method and a device for improving available storage capacity, wherein the method comprises the following steps: transmitting a write command and first data to a memory, wherein the write command is used for indicating the memory to write the first data; receiving first information from a memory; in response to the first information, a virtual disk is created for the memory. By implementing the embodiment of the application, the high data compression rate can be achieved without affecting the performance of the storage system, and the unit capacity cost of the memory is reduced, so that the available storage capacity of the system is indirectly improved.

Description

Method and device for improving available storage capacity
Technical Field
The present application relates to the field of computer data storage, and in particular, to a method and apparatus for improving available storage capacity.
Background
Compared with the traditional mechanical Hard Disk (Hard Disk Drive), the solid state Disk (Solid State Drive, SSD) has the advantages of quick start, strong shock resistance, faster data read-write speed, low power consumption, basically no noise during operation, smaller volume of the same-capacity Hard Disk, and the like, and therefore, the SSD is far higher than the traditional mechanical Hard Disk in performance and reliability. Although SSDs have obvious performance advantages, the cost per unit capacity of SSDs is far higher than that of mechanical hard disks, and the large-scale application of SSDs is limited due to the small capacity and high price of SSDs. Therefore, in order to increase the cost performance of the SSD and reduce the cost per unit capacity of the SSD, it is often adopted to indirectly increase the storage capacity of the SSD of the system through a data compression function.
The existing storage system mainly comprises four data compression methods, namely central processing unit (Central Processing Unit, CPU) online compression, hardware acceleration engine online compression, CPU offline compression and simple SSD internal compression. The CPU online compression occupies system computing resources and influences the overall performance of the system; the hardware acceleration engine online compression does not occupy the system computing resource, but the integrated compression algorithm cannot be updated to the existing optimal algorithm in time, so that the optimal data compression rate cannot be achieved; the CPU offline compression needs to read and compress the data stored in the storage medium and then re-store the data, so that not only is the system computing resource occupied, but also the read/write times of the SSD are increased, and the life cycle of the SSD is reduced; the internal compression of the SSD only needs to report the virtual storage space in advance, and the expansibility is poor, so that the problem that partial storage space cannot be used to cause system abnormality or storage space waste easily occurs.
Therefore, how to achieve data compression and achieve the optimal data compression rate while also reducing the cost per unit capacity of the SSD is an urgent issue to be addressed while ensuring that system performance is not affected.
Disclosure of Invention
The embodiment of the application discloses a method and a device for improving the available storage capacity of a system, which realize the purposes of achieving high data compression rate and reducing the unit capacity cost of a memory under the condition of not influencing the performance of the storage system, thereby indirectly improving the available storage capacity of the system.
In a first aspect, an embodiment of the present application provides a method for improving available storage capacity, applied to a computing device, the method including: transmitting a write command and first data to a memory, wherein the write command is used for indicating the memory to write the first data; receiving first information from the memory; and responding to the first information, and creating a virtual disk for the memory.
In the embodiment of the present application, the first data may be text, a program, an image, a media video file (for example, MP3, high definition movie, etc.), etc., where the first data may be compressible data, for example: text, characters, images, etc., the first data may also be incompressible data, such as: MP3, high definition movies, compressed data, etc. If the first data is compressible data, the data volume of the first data is changed before and after the first data is compressed; if the first data is incompressible data, the data volume of the first data is unchanged before and after the first data is compressed.
In the embodiment of the application, the memory is a generic term comprising a solid state disk SSD, a mechanical hard disk HDD, a virtual disk or other disks capable of storing data. The memory has compression and decompression functions and information receiving and transmitting functions, and can realize communication between the memory and the computing equipment. The embodiment of the application uses the solid state disk as an example for carrying out the exemplary explanation of the scheme, but the application is not limited to the type of the memory which is the solid state disk, and the scheme disclosed by the embodiment of the application can be realized by any memory with a compression function and a receiving and transmitting function.
In the embodiment of the application, the first information is feedback information sent to the computing device after the memory executes the write command, and the first information reflects the change of the data volume of the first data after the memory writes the first data according to the write command because the memory has a compression function, and indirectly reflects the change of the available storage capacity of the memory after the memory writes the first data according to the write command.
In the embodiment of the application, the write command also carries the logic address of the write data, so that the memory establishes a corresponding physical address according to the logic address of the write data to store the compressed write data after receiving the write command.
In the embodiment of the application, the write command and the first data can be simultaneously and jointly sent to the memory, and the write command and the first data can be respectively sent to the memory according to the sequence.
The virtual disk created by the computing device for the memory is an idle, unoccupied virtual disk. Virtual disks created by a computing device are mapped in physical space in memory. After the computing device creates the virtual disk for the memory, the available storage capacity of the memory corresponding to the record of the computing device is increased, and the increased available storage capacity is the capacity (or virtual capacity) of the virtual disk.
It can be seen that after the computing device issues the write command to the memory, feedback information (i.e., first information) reported after the memory executes the write command is received, and a virtual disk is created according to the first information, so that the capacity expansion operation of the available storage capacity of the memory is realized, the available storage capacity of the memory is indirectly improved, and the unit capacity cost of the memory (for example, a solid state disk) is reduced. In addition, the writing operation business issued by the computing equipment is decoupled from the compression process of writing data in the memory, so that the performance of the system where the computing equipment is located is not influenced while the high compression rate is achieved.
Based on the first aspect, in a possible embodiment, the first information is used to indicate an amount of difference between a first amount of data before the first data compression and a second amount of data after the first data compression; the creating a virtual disk for the memory in response to the first information includes: and creating the virtual disk for the memory according to the difference indicated by the first information.
In the embodiment of the present application, the write command indicates that the memory writes the first data, and since the memory has a compression function, when the first data is written into the memory, the writing means that the memory compresses and stores the first data sequentially, and the data amount of the first data may change before and after writing, so that the first information is used to indicate the difference between the first data amount before the first data compression and the second data amount after the first data compression, in other words, the first information indicates the reduced data amount after the first data is compressed by the memory. If the first data is compressible data, the first data volume before the first data compression is larger than the second data volume after the first data compression, namely the difference volume indicated by the first information is larger than 0; if the first data is incompressible data, the first data amount is equal to the second data amount, i.e. the difference indicated by the first information is 0. Therefore, the first data amount before the first data compression is equal to or larger than the second data amount after the first data compression.
For example, taking a solid state disk as an example, assuming that the capacity of the original physical space of the solid state disk is 100G, initially, the available storage capacity of the solid state disk recorded by the computing device is recorded as 100G. The method comprises the steps that the computing device sends 10G write data to the solid state disk to indicate the solid state disk to write, the 10G write data is compressed to 6G through the solid state disk, after the write data is successfully written to the solid state disk, the computing device records that the available storage capacity of the solid state disk is 90G, meanwhile, the computing device receives first information from the solid state disk, the first information indicates the difference of 4G, if the first preset storage capacity set by the computing device is 4G, the computing device creates a virtual disk for the solid state disk, the capacity of the virtual disk is 4G, the available storage capacity of the solid state disk recorded by the computing device is 94G, the solid state disk occupies only 6G storage capacity, and the remaining available storage capacity of the solid state disk is 94G, wherein the available storage capacity of 4G is reported to the computing device to enable the computing device to create the virtual disk for the solid state disk.
In the embodiment of the application, the computing equipment creates a virtual disk for the memory according to the difference indicated by the first information. In one implementation, when the computing device receives the first information, creating a virtual disk with the same capacity for the memory according to the difference indicated by the first information, wherein the capacity of the virtual disk is determined by the difference indicated by the first information each time; in another implementation, a virtual disk creating condition may be set, that is, the difference indicated by the first information reaches a certain fixed threshold value, and then the virtual disk is created, where the capacity of the virtual disk is the fixed threshold value; in another implementation, the virtual disk may be created only when the cumulative sum of the difference amounts indicated by the received first information multiple times reaches a fixed threshold. The creation conditions of the virtual disk in the embodiment of the application are not particularly limited.
It can be seen that after the computing device issues a write command to the memory, the computing device receives the data compression change information (i.e., the first information) reported after the memory performs the write operation, and creates a virtual disk according to the data compression change information, so as to implement the capacity expansion operation on the available storage capacity of the memory, indirectly improve the available storage capacity of the memory, and reduce the unit capacity cost of the memory.
Based on the first aspect, in a possible embodiment, after the receiving the first information from the memory, the method further includes: recording the difference indicated by the first information to a storage pool; the creating a virtual disk for the memory according to the difference indicated by the first information includes: and when the storage pool reaches a preset storage capacity, creating the virtual disk for the memory.
In the embodiment of the application, after the computing device receives the first information sent by the solid state disk, recording the difference indicated by the first information into the storage pool means that the storage pool is built for the virtual capacity equivalent to the difference indicated by the first information (the data amount which is compressed and reduced), so that the address space corresponding to the free and scattered virtual capacity becomes a continuous address space. When the difference indicated by the first information is smaller than the preset storage capacity, the difference indicated by the first information can be accumulated in the storage pool, and when the sum of the difference indicated by the plurality of times of first information accumulated in the storage pool reaches the preset storage capacity, the computing device can create a virtual disk. The preset storage capacity may be 8G, 10G, 20G, 25G, 50G or other values, and the present application is not particularly limited.
The storage pool reaching the preset storage capacity may be: the difference indicated by a certain first information recorded in the storage pool is larger than or equal to the preset storage capacity; or the cumulative sum of the difference amounts indicated by the plurality of first information recorded in the storage pool is greater than or equal to the preset storage capacity. When the difference indicated by the first information at a certain time or the sum of the differences indicated by the first information at a plurality of times is equal to the preset storage capacity, the capacity of the created virtual disk is the preset storage capacity; the capacity of the created virtual disk may be the preset storage capacity when the difference indicated by the first information or the sum of the differences indicated by the first information is greater than the preset storage capacity a certain time, in which case the portion exceeding the preset storage capacity is used to accumulate the next preset storage capacity, and of course, in some possible embodiments, the capacity of the created virtual disk may be the capacity of the current storage pool if the capacity of the current storage pool is greater than the preset storage capacity after the difference indicated by the first information is recorded to the storage pool a certain time. It should be noted that, the number of memories is at least one, and the virtual capacity accumulated in the memory pool may be accumulated for multiple write operations of one memory or may be accumulated for multiple write operations of multiple memories. Therefore, the virtual capacity of the created virtual disk may be the sum of the differences reported after multiple write operations of one memory, or the sum of the differences reported after multiple write operations of multiple memories.
It can be seen that the computing device sets a trigger condition for creating the virtual disk, so that control signaling overhead of the computing device is saved, the computing device is prevented from frequently creating the virtual disk for the memory, a corresponding virtual disk is created when the difference amount indicated by the first information reported by the memory is accumulated to a certain extent, and management pressure of the computing device on the created virtual disk is reduced.
Based on the first aspect, in a possible embodiment, after the write command and the first data are sent to the memory, the method further includes sending a delete command or a change command to the memory, where the delete command is used to instruct the memory to delete target data, and the change command is used to instruct the memory to change the target data, and the target data is part or all of the second data; receiving second information from the memory; and deleting the virtual disk in response to the second information.
In the embodiment of the present application, the second data is data written into or stored in the memory.
After the write command, the computing device issues a delete command to the memory, the delete command to instruct the memory to delete the target data, the target data being part or all of the second data. In some possible embodiments, the delete command may also be issued to the memory based on the change command.
The actual increase in available storage capacity of the memory after execution of the delete command is inconsistent with the increase in available storage capacity of the memory by the computing device. Since the memory has a compression function, the amount of data of the target data (compressed) deleted by the memory is smaller than the amount of data before the target data is compressed, and thus the available storage capacity actually increased by the memory after the deletion operation is performed is smaller than the available storage capacity increased by the computing device to the memory. Thus, the second information received by the computing device from the memory is feedback after the delete command is performed on the memory.
After the write command, the computing device issues a change command to the memory, the change command being used to instruct the memory to change the target data, wherein the change command is used to instruct the memory to change part or all of the second data, because the target data is part or all of the second data. In some possible embodiments, the change command may also be issued to the memory based on the delete command.
It should be noted that, the solid state disk executes the change command, and the data source is changed due to the change command, so that the compression rate of the changed second data is reduced, so that the data volume of the changed second data after compression is increased compared with that before compression, but the actual data volume of the second data before and after the change is equal. Thus, the change command may cause a virtual volume datagram to exist in the virtual disk created by the computing device, and thus the computing device receives the second information sent by the memory, where the second information is feedback after the memory has executed the change command.
The change command also carries the logic address of the second data, so that the memory establishes a physical address corresponding to the second data and stores the data after the change of the second data after compression according to the physical address.
It can be seen that, based on the write command, the computing device issues a delete command to the memory, then changes the command, receives the dynamic storage information (i.e. the second information) reported after the memory executes the delete command or changes the command, and according to the dynamic storage information, updates the available storage capacity of the memory recorded on the computing device side by deleting the added virtual disk, thereby improving the flexibility of disk management, realizing the dynamic management of redundant storage capacity caused by deleting or changing the compressed data in the memory, avoiding the waste of virtual capacity in the virtual disk, and improving the utilization rate of the system storage capacity.
Based on the first aspect, in a possible embodiment, the second information is used to indicate an amount of difference caused before and after the memory executes the delete command or the change command; after the receiving the second information from the memory, the method further comprises: marking virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk; said deleting said virtual disk in response to said second information comprises: and deleting the virtual disk according to the unavailable virtual capacity.
In the embodiment of the application, whether the memory executes the deletion command or the change command, a part of virtual capacity in the virtual disk created by the computing device is unavailable, and the size of the unavailable part of virtual capacity is the difference amount caused by the second information indicating the memory to execute the deletion command or the change command.
It should be noted that, after the memory executes the delete command, the virtual capacity of the virtual disk already created in the computing device has a partial datagram, so the computing device needs to mark the partial datagram as unavailable in the virtual disk according to the second information fed back after the memory executes the delete command. Because no data is occupied in the portion of virtual capacity that the delete command is marked as unavailable, only a portion of the virtual capacity that is free in the virtual disk is marked as unavailable. The method effectively solves the problem of reduced available virtual capacity in the virtual disk caused by the fact that the memory executes the deletion command, and avoids data writing failure or system breakdown of the computing equipment caused by insufficient available storage capacity of the memory.
It should be noted that, the reason why the virtual capacity of the corresponding portion is marked as unavailable in the virtual disk according to the second information fed back by the computing device after the memory executes the change command is that: after the memory performs the change operation, a part of the spare virtual capacity in the virtual disk is occupied, so that the occupied part of the virtual capacity is marked as unavailable. I.e., the portion of virtual disk that is marked as unavailable due to the change command has data stored therein. The method effectively solves the problem of reduced available virtual capacity in the virtual disk caused by the execution of the change command by the memory, and avoids the failure of data writing or system breakdown of the computing equipment caused by insufficient available storage capacity of the memory.
After the computing device marks the virtual capacity equivalent to the difference amount indicated by the second information as unavailable in the virtual disk, the virtual disk may be deleted according to the unavailable virtual capacity. In one implementation, after marking the virtual capacity corresponding to the difference indicated by the second information as unavailable, the computing device may delete the virtual disk with the same capacity according to the unavailable virtual capacity, where the capacity of the deleted virtual disk is determined by the difference indicated by the second information; in another implementation, the deletion condition of the virtual disk may be set, that is, the unavailable virtual capacity reaches a certain specified threshold, so that the virtual disk can be deleted, where the capacity of the virtual disk is the specified threshold, and the unavailable virtual capacity reaches a certain specified threshold, or the unavailable virtual capacity according to the second information mark reaches the specified threshold at a certain time, or the sum of the accumulated unavailable virtual capacities according to the second information mark reaches the specified threshold multiple times. It should be noted that, the above specified threshold is equal to or less than the capacity of the virtual disk to be deleted (i.e., the preset storage capacity), and the deletion condition of the virtual disk in the embodiment of the present application is not specifically limited.
It can be seen that according to the received second information, after the virtual capacity which is equal to the difference indicated by the second information is marked as unavailable in the virtual disk, the available storage capacity which is externally displayed by the virtual disk is reduced, so that the computing device is helped to know the remaining available storage capacity of the memory, and the problems that the data writing failure or system abnormality is caused by the fact that the data quantity of the data written by the memory indicated by the computing device is larger than the available storage capacity of the memory are effectively avoided.
Based on the first aspect, in a possible embodiment, the deleting the virtual disk according to the unavailable virtual capacity includes: and deleting the virtual disk when the unavailable virtual capacity reaches a disk deletion threshold, wherein the disk deletion threshold is smaller than or equal to the preset storage capacity.
In the embodiment of the application, the computing device can determine whether to delete the virtual disk by judging the relation between the unavailable virtual capacity and the disk deletion threshold. In some possible embodiments, the difference amount indicated by the second information at a certain time is smaller than the disc deletion threshold, that is, the condition of deleting the virtual disc is not reached, the virtual disc is marked as unavailable, and when the sum of the difference amounts indicated by the second information at a plurality of times reaches the disc deletion threshold, the virtual disc can be deleted after the corresponding unavailable virtual capacity reaches the disc deletion threshold.
In the embodiment of the application, the disc deleting threshold value is smaller than or equal to the preset storage capacity, and the preset storage capacity is the capacity of the virtual disc created by the computing equipment. In some possible embodiments, the ratio of the disc deletion threshold to the preset storage capacity may be 0.45, 0.5, 0.6, 0.8 or less than or equal to 1. The disc deleting threshold value is set to leave a margin for the available capacity of the created virtual disc, so that the operation efficiency of the system is improved.
Based on the first aspect, in a possible embodiment, when the delete command instructs the memory to delete the target data, the amount of difference indicated by the second information is an amount of difference between a third amount of data before the target data compression and a fourth amount of data after the target data compression; or when the change command indicates that the memory changes the target data, the difference indicated by the second information is the difference between the fifth data amount after the second data is compressed after the change and the sixth data amount after the second data is compressed.
In the embodiment of the application, the computing device sends the deleting command to the memory to enable the memory to delete the target data, wherein the target data is the data deleted by the memory, and in this case, the second information received by the computing device is used for indicating the difference between the third data amount before the target data is compressed and the fourth data amount after the target data is compressed.
In the embodiment of the application, the computing device sends a change command to the memory to enable the memory to change target data, wherein the target data is part or all of second data, namely, the change command indicates the memory to change part or all of the second data, and in this case, second information received by the computing device is used for indicating the difference between the fifth data amount after the second data is compressed after the change and the sixth data amount after the second data is compressed.
Based on the first aspect, in a possible embodiment, the virtual disk is mapped to a first physical space of the memory, the first physical space being used to store data of a magnitude of a difference between the fifth data amount and the sixth data amount; before the deleting the virtual disk according to the unavailable virtual capacity, the method further comprises: and sending third information to the memory, wherein the third information is used for indicating the memory to migrate the data of the first physical space to a second physical space of the memory.
In the embodiment of the application, the virtual disk is mapped to a first physical space of the memory, and when the computing device sends a change command to the memory, the first physical space is used for storing data with a difference between a fifth data amount and a sixth data amount, in other words, the first physical space is used for storing part of data compressed by second data after the change of the memory, and the data amount of the part of data is the difference between the fifth data amount and the sixth data amount.
For example, the amount of data after the second data compression is 6G, for the amount of data after the second data compression after the modification is 7G, after the modification command is completed, 6G in the data after the second data compression after the modification is stored in the original position in the memory, and the remaining 1G is stored in the first physical space of the memory, and because of the mapping between the first physical space and the virtual disk created by the computing device, the remaining 1G data can be considered to be stored in the virtual disk.
In the embodiment of the application, when the virtual capacity marked as unavailable in the virtual disk reaches the disk deletion threshold value due to the memory executing the change command, the computing device needs to delete the virtual disk, but because the deleted virtual disk stores data, before the computing device deletes the virtual disk, third information is also sent to the memory to enable the memory to migrate the data in the virtual disk to be deleted, and because the virtual disk is mapped in the first physical space of the memory, namely the computing device instructs the memory to migrate the data in the first physical space to the second physical space of the memory, the computing device can execute the virtual disk deletion after the data is migrated. Therefore, dynamic management of the virtual disk is realized, and the operation efficiency of the system is improved.
It should be noted that, after the computing device deletes a certain created virtual disk, the corresponding virtual disk is no longer mapped to the first physical space in the memory, so that the first physical space of the memory is restored to the local storage space of the memory.
Based on the first aspect, in a possible embodiment, the virtual disk is mapped to a first physical space of the memory, and when third data is stored in the first physical space, the method further includes: receiving fourth information from the memory; and deleting the virtual disk in response to the fourth information.
In some possible embodiments, after the computing device creates the virtual disk for the memory, the computing device sends a write command to the memory, and the memory stores the compressed third data to the first physical space mapped with the virtual disk created by the computing device after executing the write command, which means that the data is stored in the virtual disk, so the computing device also receives fourth information from the memory, where the fourth information is feedback after the memory executes the write command and stores the data to the virtual disk.
It can be seen that, in the case that the computing device has created a virtual disk, if the memory stores the written data to the first physical space mapped with the virtual disk, the memory will trigger the computing device to delete the virtual disk.
Based on the first aspect, in a possible embodiment, the fourth information is used to indicate the data amount after the third data compression; after said receiving the fourth information from the memory, the method further comprises: marking virtual capacity equivalent to the data amount indicated by the fourth information as unavailable in the virtual disk; said deleting said virtual disk in response to said fourth information comprises: and deleting the virtual disk according to the unavailable virtual capacity.
In some possible embodiments, since the memory stores the third data to the first physical space mapped with the virtual disk after executing the write command, the computing device needs to mark the portion of the virtual disk occupied as unavailable after receiving the information from the fourth information, that is, mark a virtual volume in the virtual disk equal to the amount of data indicated by the fourth information as unavailable.
In some possible embodiments, a condition may be set for deletion of the virtual disk, that is, when the virtual capacity, marked according to the data amount indicated by the fourth information, that is, the virtual capacity that is not available in the virtual disk reaches the deletion threshold, the description about the deletion threshold may refer to the description in the foregoing, and will not be repeated herein. The setting of the deletion threshold can leave a margin for the available capacity of the created virtual disk, which is beneficial to improving the operation efficiency of the system.
Based on the first aspect, in a possible embodiment, the virtual disk is mapped to a first physical space of the memory, the first physical space storing the third data; before said deleting said virtual disk, said method further comprises: and when the unavailable virtual capacity reaches a disk deletion threshold, sending fifth information to the memory, wherein the fifth information is used for indicating the memory to migrate the data of the first physical space to a second physical space of the memory.
In some possible embodiments, if the memory stores data to the virtual disk according to the write command, when the virtual capacity marked as unavailable in the virtual disk reaches the disk deletion threshold due to the write command, because the virtual disk to be deleted stores data, the computing device further needs to send fifth information to the memory before deleting the corresponding virtual disk, where the fifth information is used to instruct the memory to migrate the data in the first physical space to the second physical space of the memory. It should be noted that the second physical space is different from the first physical space in that: the second physical space has no virtual disk mapped to it.
In a second aspect, an embodiment of the present application provides a method for improving available storage capacity, the method including: receiving a write command and first data from a computing device; compressing and storing the first data; first information is sent to the computing device to cause the computing device to create a virtual disk for the memory.
In the embodiment of the present application, the first data may be text, a program, an image, a media video file (for example, MP3, high definition movie, etc.), etc., where the first data may be compressible data, for example: text, characters, images, etc., the first data may also be incompressible data, such as: MP3, high definition movies, compressed data, etc. If the first data is compressible data, the first data volume before the first data compression is larger than the second data volume after the first data compression; if the first data is compressible data, the data volume of the first data is changed before and after the first data is compressed; if the first data is incompressible data, the data volume of the first data is unchanged before and after the first data is compressed.
The write command instructs the memory to write the first data, and since the memory has a compression function, the writing process of the first data in the memory means: the memory compresses the first data by using an internal compression algorithm, and stores the compressed first data. In the embodiment of the application, the write command also carries the logical address of the first data, the memory establishes a physical address corresponding to the logical address, and the compressed first data is stored in a physical space where the physical address is located. It should be noted that, the process of writing the first data by the memory according to the received write-once command may also be called that the memory performs a write-once operation.
According to the embodiment of the application, the compression algorithm integrated in the memory can be Huffman coding, run-length coding or other compression algorithms with higher compression rate, and the compression algorithm in the memory can be updated in time according to the data type and the industry compression algorithm so as to achieve the optimal data compression rate, thereby indirectly improving the available storage capacity of the memory.
In the embodiment of the application, the memory is a generic term comprising a solid state disk SSD, a mechanical hard disk HDD, a virtual disk or other disks capable of storing data. The memory has compression and decompression functions and information receiving and transmitting functions, and can realize communication between the memory and the computing equipment. The embodiment of the application uses the solid state disk as an example for carrying out the exemplary explanation of the scheme, but the application is not limited to the type of the memory which is the solid state disk, and the scheme disclosed by the embodiment of the application can be realized by any memory with a compression function and a receiving and transmitting function.
In the embodiment of the application, the first information is feedback information sent to the computing device after the memory executes the write command, and the first information reflects the change of the data volume of the first data after the memory writes the first data according to the write command because the memory has a compression function, and indirectly reflects the change of the available storage capacity of the memory after the memory writes the first data according to the write command.
Based on the second aspect, in a possible embodiment, the first information is used to indicate an amount of difference between a first amount of data before the first data compression and a second amount of data after the first data compression; the sending, to the computing device, first information, where the first information is used to instruct the computing device to create a virtual disk for the memory, includes: and sending the first information to the computing device, wherein the first information is used for indicating the computing device to create the virtual disk for the memory based on the difference indicated by the first information.
In the embodiment of the application, since the memory has a compression function, the data amount before and after the first data compression is changed after the memory executes the write command, the first information is used for indicating the difference amount between the first data amount before the first data compression and the second data amount after the first data compression, in other words, the first information indicates the reduced data amount after the first data is compressed by the memory. If the first data is compressible data, the first data volume before the first data compression is larger than the second data volume after the first data compression, namely the difference volume indicated by the first information is larger than 0; if the first data is incompressible data, the first data amount is equal to the second data amount, i.e. the difference indicated by the first information is 0. Therefore, the first data amount before the first data compression is equal to or larger than the second data amount after the first data compression.
It can be seen that the compression process of the memory to the data does not consume the computing power of the computing device under the write command, so that the performance of the system is not affected. The compression algorithm adopted by the memory can be updated in time and customized according to the need, so that the memory achieves the optimal data compression rate, the data volume of the compressed data is reduced as much as possible, and feedback information (namely first information) is reported to the computing equipment after a write command is executed, so that the computing equipment creates a virtual disk for the memory, thereby indirectly improving the available storage capacity of the memory and reducing the unit capacity cost of the memory.
Based on the second aspect, in a possible embodiment, after the compressing and storing the first data, the method further includes: marking the difference amount caused by the first data before and after compression as a reduced data amount; the sending first information to the computing device includes: and sending the first information to the computing device when the reduced data amount reaches a preset threshold.
In the embodiment of the application, a trigger condition can be set for the transmission of the first information, that is, when the reduced data volume reaches the preset threshold value, the memory transmits the first information to the computing device. The reduced data amount may be the difference amount marked after the write-once command is executed, or may be the sum of the difference amounts marked after the write-many commands are executed. The preset threshold may be 2G, 3G, 5G, 6G, 10G, or other value, the preset threshold being less than or equal to the preset storage capacity.
For example, when the amount of compressed and reduced data of the first data in a single write operation is smaller than a preset threshold value, the memory does not send the first information to the computing device, but the difference caused by the write operation can be marked first; when the sum of the difference amounts accumulated by the marked write operations is greater than or equal to a preset threshold, the memory transmits first information to the computing device, wherein the first information is used for indicating the difference amount between the first data amount before the first data compression and the second data amount after the first data compression.
It can be seen that setting the trigger condition for the transmission of the first information can effectively reduce the number of times that the memory transmits the first information to the computing device, avoid frequent reporting of the first information, and reduce the system control signaling overhead.
Based on the second aspect, in a possible embodiment, after the compressing and storing the first data, the method further includes: receiving a deletion command or a change command sent by the computing equipment; deleting target data according to the deleting command or modifying the target data according to the modifying command, wherein the target data is part or all of second data; and sending second information to the computing device, wherein the second information is used for indicating the computing device to delete the virtual disk.
It should be noted that the second data is data that has been written or stored into the memory. The second information is feedback after the memory has executed the delete command or the change command.
And after the write command is based on the write command, deleting the target data by receiving a deleting command issued by the computing equipment, wherein the target data is part or all of the second data. The deleting command carries the logical address of the second data, and the solid state disk finds the corresponding physical address according to the logical address, and the stored compressed write data can be obtained from the physical space corresponding to the physical address.
For example, when the target data is all the second data, that is, the memory deletes all the second data according to the received delete command, the stored compressed second data is found in the physical space corresponding to the physical address, and then the second data is directly deleted.
When the target data is part of the second data, the memory deletes the part of the second data according to the received delete command, the memory finds the stored compressed second data in the physical space corresponding to the physical address, decompresses the compressed second data, and then deletes the target data.
After the write command is based on the change command issued by the computing device, the memory receives the change command, and changes the target data according to the change command, wherein the target data is part or all of the second data, namely, the memory changes part or all of the second data according to the change command. The change command carries a logic address of the write data, the memory finds a corresponding physical address according to the logic address, and the stored compressed second data can be obtained from a physical space corresponding to the physical address.
It should be noted that, since the data stored in the memory are all compressed data, in order to implement the modification of the second data, the memory needs to decompress and then modify the compressed second data. Whether the partial or all data of the second data is modified, but because the whole second data is decompressed, after the modification of the partial or all data of the second data is completed, the modified second data needs to be recompressed and stored, that is, the data after the modification of the second data is compressed is stored in the physical space of the memory.
It should be noted that, since the memory has a compression function, after the memory deletes the target data according to the delete command or changes the target data according to the change command, the virtual capacity of the virtual disk created by the computing device has a partial virtual report, so the memory sends second information to the computing device, where the second information is feedback information to the computing device after the memory executes the delete command or the change command, so that the computing device deletes the virtual disk. Therefore, the method is beneficial to realizing the dynamic management of the redundant storage capacity of the computing device caused by deleting or changing the compressed data in the memory, avoiding the waste of the virtual capacity in the virtual disk and improving the utilization rate of the storage capacity of the system.
Based on the second aspect, in a possible embodiment, the second information is further used to indicate an amount of difference caused before and after the memory executes the delete command or the change command.
In the embodiment of the application, the memory can reduce the available storage capacity of the virtual disk no matter the memory executes the deleting command or the changing command, and the memory can send second information to the computing device, wherein the second information is used for indicating the difference amount caused before and after the memory executes the deleting command or the changing command, so that the reduced available storage capacity in the virtual disk is equal to the difference amount indicated by the second information.
Based on the second aspect, in a possible embodiment, when the memory deletes the target data, the difference amount indicated by the second information is a difference amount between a third data amount before the target data compression and a fourth data amount after the target data compression; or when the memory alters the target data, the difference indicated by the second information is the difference between the fifth data amount after the second data is compressed after the alteration and the sixth data amount after the second data is compressed.
When deleting target data according to a deleting command in a memory, the difference indicated by the second information is the difference between the third data amount before the deleted target data is compressed and the fourth data amount after the target data is compressed; when the memory changes the target data (i.e., part or all of the second data) according to the change command, the difference indicated by the second information is the difference between the fifth data amount after the second data compression after the change and the sixth data amount after the second data compression, which is equivalent to the difference between the data amount after the change and the data amount after the target data compression. It should be noted that, recompression of the data after modification may result in an increase in the data amount, because the data source is changed after modification of the data, resulting in a decrease in the compression rate of the modified second data, and thus, the fifth data amount is larger than the sixth data amount. The changed second data volume becomes larger, which means that the physical space occupied in the memory is increased, namely more available storage capacity is consumed in the memory, the available storage capacity consumed in the memory is the difference between the fifth data volume and the sixth data volume, and the available storage capacity of the memory recorded on the side of the computing device after the change operation is correspondingly reduced, so that the memory sends second information indicating the difference between the fifth data volume and the sixth data volume to the computing device, and the computing device reduces the available storage capacity of the virtual disk according to the second information, thereby achieving the purpose of reducing the available storage capacity of the memory recorded on the side of the computing device, and avoiding the data writing failure or system abnormality caused by insufficient available storage capacity of the memory of the computing device.
Based on a second aspect, the virtual disk maps to a first physical space of the memory, the first physical space for storing data of a difference magnitude between the fifth amount of data and the sixth amount of data; after the sending of the second information to the computing device while the memory alters the target data, the method further comprises: third information from the computing device is received, and data of the first physical space is migrated to a second physical space of the memory according to the third information.
In the embodiment of the present application, the virtual disk is mapped in a first physical space of the memory, and when the computing device sends a change command to the memory, the first physical space is used for storing data of a difference amount between the fifth data amount and the sixth data amount, in other words, the first physical space is used for storing a part of data after the second data is compressed after the change, and the data amount of the part of data is the difference amount between the fifth data amount and the sixth data amount.
Illustratively, after writing the command, the memory reports the second data to the computing device in a compressed, reduced amount of data to cause it to create a corresponding virtual disk that maps to a first physical space of the memory. After the memory executes the change command, the memory stores the redundant data after the second data compression after the change to the first physical space of the memory preferentially, and simultaneously sends second information to the computing device, wherein the second information is used for indicating the difference between the fifth data volume and the sixth data volume after the second data compression after the change, so that the computing device marks the corresponding virtual capacity in the virtual disk as unavailable according to the second information. It can be seen that dynamic management of excess storage capacity caused by changing compressed data stored in memory is achieved, and data write failure or system crash of the computing device due to insufficient available storage capacity of the memory is avoided.
In the embodiment of the application, the memory receives the third information sent by the computing device, namely, the computing device is explained to pre-delete the created virtual disk, the memory transfers the data volume of the first physical space to the second physical space of the memory according to the third information, the data in the virtual disk to be deleted is transferred to different physical spaces of the memory, and the physical space where the data is located after the data is transferred is not mapped with the virtual disk.
It should be noted that, after the memory completes migration of the data according to the third information, if the computing device deletes the virtual disk, the first physical space in the memory does not have the virtual disk and the mapping thereof, and the first physical space is restored to the local storage space of the memory. Therefore, dynamic management of the virtual disk is realized, and the operation efficiency of the system is improved.
Based on the second aspect, in a possible embodiment, the virtual disk is mapped to a first physical space of the solid state disk, and when the memory stores third data to the first physical space according to the write command, the method further includes: and sending fourth information to the computing device, wherein the fourth information is used for indicating the computing device to delete the virtual disk.
In some possible embodiments, in the case that the computing device has created the virtual disk, the memory may further store the third data (compressed) into the first physical space mapped with the virtual disk after the memory has executed the write command, that is, equivalent to storing the data in the virtual disk, so that the memory further needs to send fourth information to the computing device to inform the computing device how much virtual capacity in the virtual disk is occupied by the third data.
Based on the second aspect, in a possible embodiment, the fourth information is used to indicate the third data compressed data amount; the sending fourth information to the computing device to cause the computing device to delete the virtual disk includes: and fourth information is sent to the computing equipment, wherein the fourth information is used for indicating the computing equipment to delete the virtual disk according to the data quantity indicated by the fourth information.
Based on the second aspect, in a possible embodiment, after sending fourth information to the computing device, the method further includes: and receiving fifth information from the computing equipment, and migrating the data of the first physical space to a second physical space of the solid state disk according to the fifth information.
In some possible embodiments, in the case that the computing device has created the virtual disk, since the memory stores the compressed third data into the first physical space mapped with the virtual disk after executing the write command, when the computing device needs to delete the virtual disk, the memory will migrate the data in the virtual disk to be deleted from the first physical space according to the indication information (i.e. the fifth information) sent by the computing device.
In a third aspect, an embodiment of the present application provides an apparatus, applied to a computing device, including: a transmitting unit configured to transmit a write command and first data to a memory, the write command being configured to instruct the memory to write the first data; a receiving unit configured to receive first information from the memory; and the processing unit is used for responding to the first information and creating a virtual disk for the memory.
Based on the third aspect, in a possible implementation, the first information is used to indicate an amount of difference between a first amount of data before the first data compression and a second amount of data after the first data compression; the processing unit is specifically configured to: and creating the virtual disk for the memory according to the difference indicated by the first information.
Based on the third aspect, in a possible implementation, after the receiving the first information from the memory, the processor is further configured to: recording the difference indicated by the first information to a storage pool; the processor is specifically configured to: and when the storage pool reaches a preset storage capacity, creating a virtual disk for the memory.
Based on the third aspect, in a possible implementation manner, the sending unit is further configured to send a delete command or a change command to the memory, where the delete command is used to instruct the memory to delete target data, and the change command is used to instruct the memory to change the target data, and the target data is part or all of second data; the receiving unit is further used for receiving second information from the memory; the processing unit is further configured to delete the virtual disk in response to the second information.
Based on the third aspect, in a possible implementation, the second information is used to indicate an amount of difference caused before and after the memory executes the delete command or the change command; the processing unit is further configured to: marking virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk; the processing unit is specifically configured to: and deleting the virtual disk according to the unavailable virtual capacity.
Based on the third aspect, in a possible implementation, the processing unit is specifically configured to delete the virtual disk when the unavailable virtual capacity reaches a disk deletion threshold, where the disk deletion threshold is smaller than or equal to the preset storage capacity.
Based on the third aspect, in a possible implementation, when the delete command instructs the memory to delete the target data, the amount of difference indicated by the second information is an amount of difference between a third amount of data before the target data compression and a fourth amount of data after the target data compression; or when the change command indicates that the memory changes the target data, the difference indicated by the second information is the difference between the fifth data amount after the second data is compressed after the change and the sixth data amount after the second data is compressed.
Based on the third aspect, in a possible implementation, the virtual disk is mapped to a first physical space of the memory, the first physical space being used to store data of a difference magnitude between the fifth data amount and the sixth data amount; the transmitting unit is further configured to: and sending third information to the memory, wherein the third information is used for indicating the memory to migrate the data of the first physical space to a second physical space of the memory.
Based on the third aspect, in a possible implementation, the virtual disk is mapped to a first physical space of the memory, and when third data is stored in the first physical space, the receiving unit is further configured to receive fourth information from the memory; the processing unit is specifically configured to: and deleting the virtual disk in response to the fourth information.
Based on the third aspect, in a possible implementation, the fourth information is used to indicate an amount of data after the third data compression; after said receiving the fourth information from the memory, the processing unit is further configured to: marking virtual capacity equivalent to the data amount indicated by the fourth information as unavailable in the virtual disk; the processing unit is specifically configured to: and deleting the virtual disk according to the unavailable virtual capacity.
Based on the third aspect, in a possible embodiment, the virtual disk is mapped to a first physical space of the memory, the first physical space storing the third data; before the deleting the virtual disk, the sending unit is further configured to: and when the unavailable virtual capacity reaches a disk deletion threshold, sending fifth information to the memory, wherein the fifth information is used for indicating the memory to migrate the data of the first physical space to a second physical space of the memory.
In a fourth aspect, an embodiment of the present application provides an apparatus, applied to a memory, including: a receiving unit for receiving a write command and first data from a computing device; the processing unit is used for compressing and storing the first data; and the sending unit is used for sending first information to the computing device, wherein the first information is used for indicating the computing device to create a virtual disk for the memory.
Based on the fourth aspect, in a possible embodiment, after said compressing and storing of said first data, said processing unit is further configured to: marking the difference amount caused by the first data before and after compression as a reduced data amount; the processing unit is specifically configured to: and when the reduced data amount reaches a preset threshold value, sending first information to the computing device.
Based on the fourth aspect, in a possible embodiment, the first information is used to indicate an amount of difference between a first amount of data before the first data compression and a second amount of data after the first data compression; the sending unit is specifically configured to: and sending the first information to the computing device, wherein the first information is used for indicating the computing device to create the virtual disk for the memory based on the difference indicated by the first information.
Based on the fourth aspect, in a possible embodiment, the receiving unit is further configured to receive a delete command or a change command from the computing device; the processing unit is further configured to delete target data according to the deletion command or change the target data according to the change command, where the target data is part or all of second data; the sending unit is further configured to send second information to the computing device, where the second information is used to instruct the computing device to delete the virtual disk.
Based on the fourth aspect, in a possible embodiment, the second information is further used to indicate an amount of difference caused before and after the memory executes the delete command or the change command.
Based on the fourth aspect, in a possible embodiment, when the memory deletes the target data, the difference amount indicated by the second information is a difference amount between a third data amount before the target data compression and a fourth data amount after the target data compression; or when the memory alters the target data, the difference indicated by the second information is the difference between the fifth data amount after the second data is compressed after the alteration and the sixth data amount after the second data is compressed.
Based on the fourth aspect, in a possible embodiment, the virtual disk is mapped to a first physical space of the memory, the first physical space being used to store data of a magnitude of a difference between the fifth data amount and the sixth data amount; when the memory alters the target data, the receiving unit is further configured to: receiving third information from the computing device; the processing unit is specifically configured to: and migrating the data of the first physical space to a second physical space of the memory according to the third information.
Based on the fourth aspect, in a possible embodiment, the virtual disk is mapped to a first physical space of the solid state disk, and when the memory stores third data into the first physical space according to the write command, the sending unit is further configured to send fourth information to the computing device, where the fourth information is used to instruct the computing device to delete the virtual disk.
Based on a fourth aspect, in a possible embodiment, the fourth information is used to indicate the third data compressed data amount; the sending unit is specifically configured to: and fourth information is sent to the computing equipment, wherein the fourth information is used for indicating the computing equipment to delete the virtual disk according to the data quantity indicated by the fourth information.
Based on the fourth aspect, in a possible embodiment, the receiving unit is further configured to receive fifth information from the computing device after sending fourth information to the computing device; and the processing unit is further used for migrating the data in the first physical space to a second physical space of the solid state disk according to the fifth information.
In a fifth aspect, an embodiment of the present application provides an apparatus, applied to a computing device, including: at least one processor, a transmitter and a receiver, wherein the transmitter is configured to send a write command and first data to a memory, the write command being configured to instruct the memory to write the first data; the receiver is used for receiving the first information from the memory; the processor is configured to create a virtual disk for the memory in response to the first information. The apparatus is for implementing the method of the first aspect or any of the possible embodiments of the first aspect.
In a sixth aspect, an embodiment of the present application provides an apparatus, applied to a memory, including: at least one processor, a transmitter, and a receiver, wherein the receiver is to receive a write command and first data from a computing device; the processor is used for compressing and storing the first data; the transmitter is configured to transmit first information to the computing device, where the first information is configured to instruct the computing device to create a virtual disk for the memory. The apparatus is for implementing the method of the second aspect or any of the possible embodiments of the second aspect.
In a seventh aspect, embodiments of the present application provide a computer readable storage medium storing program code for execution by an apparatus, the program code comprising instructions for performing the method of the first aspect or any of the possible implementations of the first aspect.
In an eighth aspect, embodiments of the present application provide a computer readable storage medium storing program code for execution by an apparatus, the program code comprising instructions for performing the second aspect or any one of the possible implementations of the second aspect.
In a ninth aspect, embodiments of the present application provide a computer program product comprising program instructions which, when executed by an apparatus, performs the method of the first aspect or any of the possible embodiments of the first aspect. The computer software product may be a software installation package which may be downloaded and executed on a device to implement the method of the first aspect or any of the possible embodiments of the first aspect, in case the method provided by any of the possible designs of the first aspect is required.
In a tenth aspect, embodiments of the present application provide a computer program product comprising program instructions which, when executed by an apparatus, performs the method of the second aspect or any of the possible embodiments of the second aspect. The computer software product may be a software installation package which may be downloaded and executed on a device to implement the method of the second aspect or any of the possible embodiments of the second aspect, in case the method provided by any of the possible designs of the second aspect described above is required.
It can be seen that, by implementing the embodiment of the application, the memory integrated with the latest compression algorithm is adopted to execute the write command on the data and execute other data operation instructions (such as change and delete commands) on the written data, and the dynamic storage information caused by executing the corresponding operation is reported to the computing device, so that the computing device realizes the dynamic management of the redundant storage capacity caused by executing the corresponding operation through the creation and deletion of the virtual disk, thus improving the flexibility of the regulation and control of the storage system, realizing the data compression and achieving the highest data compression rate under the condition of not affecting the performance of the storage system, and indirectly improving the available capacity of the system disk, thereby realizing the optimal overall cost performance of the storage system.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a system architecture diagram of an application provided by an embodiment of the present application;
FIG. 2 is a flow chart of a method for increasing available storage capacity based on a write command according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a scenario for a write command according to an embodiment of the present application;
FIG. 4 is a flow chart of a method for increasing available storage capacity based on a delete command provided by an embodiment of the present application;
FIG. 5 is a schematic diagram of a scenario for a delete command according to an embodiment of the present application;
FIG. 6 is a schematic diagram of yet another scenario for a delete command according to an embodiment of the present application;
FIG. 7 is a flow chart of a method for increasing available storage capacity based on a change command provided by an embodiment of the present application;
FIG. 8 is a schematic diagram of a scenario for application to a change command according to an embodiment of the present application;
FIG. 9 is a schematic diagram of yet another scenario for application to a change command provided by an embodiment of the present application;
FIG. 10 is a flow chart of a method for applying to a read command according to an embodiment of the present application;
FIG. 11 is a schematic view of an apparatus according to an embodiment of the present application;
FIG. 12 is a functional schematic of yet another apparatus according to an embodiment of the present application;
FIG. 13 is a schematic view of an apparatus according to an embodiment of the present application;
fig. 14 is a functional structural diagram of yet another device according to an embodiment of the present application.
Detailed Description
The terminology used in the embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The terms first, second and the like in the description and in the claims, are used for distinguishing between different objects and not for describing a particular sequential order. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.
For ease of understanding, related terms and the like that may be involved in embodiments of the present application are described first.
(1) Mechanical hard disk
The mechanical Hard Disk (HDD) is a conventional common Hard Disk, and uses a magnetic Disk as a storage medium, and is generally composed of mechanical components such as a Disk and a magnetic head, wherein the magnetic head can move along the radial direction of the Disk, and the magnetic head can be positioned at a specified position of the Disk to perform data reading and writing operations in addition to high-speed rotation of the Disk of thousands of revolutions per minute. Vibration and noise can be generated during working, the bearable temperature range is small, and the hard disk is easy to damage during movement or vibration. However, the mechanical hard disk has high read/write times, long service life and easy manufacture into large capacity.
(2) Solid state disk
The solid state disk (Solid State Drive, SSD) is a hard disk generally composed of a memory and a controller, and because no mechanical structure exists, seek is not needed during reading and writing, the speed of reading and writing is high, the efficiency is high, the noise-free shock resistance is high during operation, the low-power consumption standby function is realized, and the temperature range which can be born during operation is larger than that of a mechanical hard disk. However, compared with a mechanical hard disk, the magnetic disk has small capacity, high unit cost and relatively short service life due to the limit of read-write times.
The storage of the SSD can be divided into two types, one is a dynamic random access memory (Dynamic Random Access Memory, DRAM), a DRAM chip is adopted as a storage medium for storing temporary data of program operation, the SSD based on the DRAM chip can be set and managed by most of file system tools of an operating system, and PCI and FC interfaces of industry standards are provided for connecting a host or a server, but an independent power supply is needed for protecting data security; the other is a FLASH memory (FLASH) memory, which adopts a FLASH chip as a storage medium, wherein the FLASH chip can be divided into a NOR (NOR) type and a NAND (NAND) type, the NOR type storage content is mainly encoded, the function of the NOR type storage content is more related to operation, the NOR type FLASH chip has the advantages of high reliability and high random reading speed, but has slower erasing and programming speeds and small capacity, and is mainly used for storing executable program codes; the NAND type flash memory has large capacity and high erasing speed, and is suitable for data storage, such as data storage. It should be noted that the Flash chip belongs to a nonvolatile memory, i.e. the data stored after power failure is not lost. The DRAM chip is used for storing temporary data of program operation and belongs to a volatile memory.
Taking NAND-Flash as a storage medium, there are typically multiple NAND-Flash in an SSD, each NAND-Flash containing multiple blocks, each Block containing multiple pages. Because of the NAND nature, the access data must be in pages, i.e., at least one Page per read and write. Typically, each Page is 4K or 8K in size. Another characteristic of NAND is that only a single Page can be read and written, no overwriting of data is supported, i.e. write operations can only be performed in free or erased cells.
(3) Storage pool
The storage pool is a high-efficiency storage management mechanism of the application program, and has the advantages of high allocation efficiency, low management cost, capability of eliminating hidden danger of memory leakage and the like. The allocation and management of storage space is performed by the operating system or compiler. Taking an operating system as an example, the operating system divides a large area (called a heap) with continuous addresses in the storage space for use by memory allocation functions (such as malloc, free, etc.) of the system. The conventional malloc/free memory allocation method easily causes fragmentation of the heap, and there may be a large total number of free spaces in the heap, but these free spaces are scattered and discontinuous, and thus cannot be combined to accommodate large allocation blocks.
The storage pool is a space for accommodating variables allocated by the storage pool by taking a Block as a unit, namely the storage pool is equivalent to an available space table established by a system, scattered free blocks in the storage space are integrated in a linked list form with continuous addresses, each free Block is an node in the linked list, if the free Block is required to be allocated, a proper node is found from the linked list to be allocated, and the node is deleted after the allocation is successful; if a new idle block appears, the idle block is inserted into the linked list, so that the management of the idle storage capacity space is realized, and the problem of space waste caused by idle space dispersion is effectively solved.
At present, compared with a mechanical hard disk, a solid state hard disk is better in terms of read-write speed, power consumption, shock resistance, occupied volume and the like, so that in storage application scenes with higher requirements on performance and reliability of the hard disk, such as finance, a core database and the like, the solid state hard disk gradually starts to replace the traditional mechanical hard disk to be used as a main memory of data. However, the disadvantages of small storage capacity, high unit capacity cost and the like of the solid state disk limit the wide-range application of the solid state disk, so that only high-end application scenes use the solid state disk at present. Therefore, the storage space of the data can be reduced by means of data compression, and the effect of increasing the storage capacity of the solid state disk is indirectly achieved, so that the unit capacity cost of the solid state disk is reduced.
The existing storage system mainly comprises four data compression methods, namely central processing unit (Central Processing Unit, CPU) online compression, hardware acceleration engine online compression, CPU offline compression and simple SSD internal compression. The two methods are different in that the CPU online compression is that the CPU executes a compression algorithm to complete the compression and decompression of the data, which occupies system computing resources and affects the overall performance of the system, and the hardware acceleration engine online compression is that the hardware acceleration engine (for example, an ASIC chip) executes the compression algorithm to complete the compression and decompression of the data, which does not occupy the computing resources of the system and does not affect the performance of the system, but the compression algorithm integrated in the hardware cannot be updated to the existing optimal algorithm in time due to the limited process time of the hardware acceleration engine, so that the optimal data compression rate cannot be achieved. The CPU offline compression means that data is firstly stored in a storage medium, then the data compression module re-reads the data from the storage medium for compression, and finally the compressed data is re-stored in the storage medium, and the data compression module runs on the CPU, so that not only is the computing resource occupied to influence the system performance, but also the read-write times of SSD are increased, and the service life of SSD is reduced.
The following simple SSD internal compression is specifically described herein: because Firmware in the SSD is newly added with the compression and decompression functions, the Firmware of the SSD hard disk can report part of virtual storage space to the system according to experience during initialization, and after data is written into the SSD, the Firmware of the SSD compresses the data and stores the compressed data into a storage medium. In the method, the compression algorithm is executed by Firmware, the performance of the system is not affected, but because the virtual storage space is reported empirically and is reported only once in the initialization process, the expandability is poor, the compression algorithm cannot be changed, and the phenomenon that the multiple reported storage spaces cannot be used to cause system abnormality or partial storage space waste is easily caused. For example, assuming that the actual capacity of the SSD is 1T and the compression rate of the SSD may reach 50%, the SSD empirically reports the storage capacity of the SSD to the system as 2T (i.e. more virtual storage space than 1T is reported), so the system considers the storage capacity of the SSD as 2T, and when the write data input by the system is already compressed (data cannot be compressed for the second time) or the write data is low in compressibility, the data writing failure or the virtual storage space reported by the SSD may not be used.
In summary, the existing method for reducing the cost per unit capacity of the SSD by the data compression method cannot solve the problem that the performance of the system is not affected and the optimal compression rate of the data is achieved at the same time, and cannot reduce the cost per unit capacity of the SSD to the greatest extent under the condition that the performance of the system is not affected, so that the storage capacity of the SSD is indirectly improved.
The system architecture to which the embodiments of the present application are applied is described below, where the system to which the present application is applied includes a computing device and a memory, and the memory may exist independently of the computing device or may be integrated inside the computing device, and the present application is not limited in detail. The memory is a generic term comprising solid state disk SSD, mechanical hard disk HDD, virtual disk or other disk capable of storing data. The memory has compression and decompression functions and information receiving and transmitting functions, and can realize communication between the memory and the computing equipment. The memory may take a solid state disk as an example to describe the structure of the system, but the application is not limited to the solid state disk as the type of the memory in the system architecture, and any memory with compression function and transceiver function can be applied to the system disclosed in the embodiment of the application.
Referring to fig. 1, fig. 1 is a block diagram of a system provided by an embodiment of the present application, where the system includes a computing device 10 and a solid state disk 20, where the solid state disk 20 may exist independently of the computing device 10, and the solid state disk 20 may also be integrated into the computing device 10, and the embodiment of the present application is not specifically limited, and the solid state disk 20 is connected to the computing device 10 for communication through a wired or wireless manner. The computing device 10 may issue operation commands, such as a read command, a write command, a change command, or a delete command, to the solid state disk 20, and may also interact with data of the solid state disk, such as reading data in the solid state disk, writing data to the solid state disk, receiving available capacity change information from the solid state disk, and so on.
The computing device 20 at least includes storage software, an SSD drive, a compression middleware and a compression drive, where the storage software, the SSD drive, the compression middleware and the compression drive are all running on a processor of the computing device 10, the storage software is used to create files for users and control writing, reading, modifying, deleting, etc. of the files, the SSD drive is used to implement communication between the computing device and the solid state disk, the compression middleware is used to manage storage pools and create and delete virtual disks, and the compression drive is used as an information transfer interface between the solid state disk and the compression middleware.
In one implementation, the compression middleware may be a hardware on-chip software function running on a computing device processor. In another implementation, the compression middleware may also be a hardware chip integrated in the computing device processor in the form of hardware circuitry.
The solid state disk 20 at least includes an SSD controller, in which a compression Firmware (Firmware) is running, for performing specific data compression and/or decompression operations and reporting of relevant capacity information after compression/decompression, and a storage medium for storing data compressed by the compression Firmware (Firmware). The storage medium may be a plurality of memories, each of which may also be referred to as a Flash grain or NAND Flash or DIE or LUN or PU or NAND memory or NAND device (NAND device), or the like. The SSD controller can execute operations such as command issuing, data storage, reading, changing, deleting and the like on the storage medium through an Input/Output (IO) channel.
In one implementation, the compressed Firmware (Firmware) may be a hardware on-chip software function running on the SSD controller. In another implementation, the compression Firmware (Firmware) may also be a hardware chip integrated in the SSD controller in the form of a hardware circuit.
The SSD controller may include, for example, one or more processors, e.g., a central processing unit (Central Processing Unit, CPU), which may be integrated on the same hardware chip.
In general, data is stored in flash memory granules according to a fixed address mapping relationship, where the address mapping relationship refers to a mapping relationship (Map) of a logical address to a physical address. And storing the data in the solid state disk according to the physical address. When the computing device instructs the solid state disk to perform writing/reading/changing/deleting operation, the computing device sends a writing/reading/changing/deleting command to the solid state disk, wherein the writing/reading/changing/deleting command carries a logical address of corresponding data.
The SSD controller is provided with a flash memory conversion layer (Flash Translation Layer, FTL) module which is used for completing conversion from a logical address of the computing device to a physical address of the solid state disk. The mapping relation between the logical address and the physical address of the data is recorded when the solid state disk writes the data into the storage medium every time, so that after the solid state disk receives a command of reading/changing/deleting operation sent by the computing device, the logical address of the data is resolved into the physical address according to the fixed address mapping relation, and the position of the data is searched in the storage medium.
In one implementation, the FTL module may be a hardware on-chip software function running on the SSD controller. In another implementation, the FTL module may also be a hardware chip integrated in the SSD controller in the form of a hardware circuit.
For the computing device 20, on the one hand, the computing device 20 is configured to execute a system service, that is, the storage software is configured to send a data operation command (for example, a write command, a read command/change command, a delete command, etc.) and a logic address of write data or corresponding data to the solid state disk 10 through the SSD driver, and the storage software is further configured to receive a result returned after the solid state disk completes the data operation command; on the other hand, the computing device 20 is configured to manage the available storage capacity of the system, that is, the storage software receives the available capacity change information of the solid state disk 20 reported by the compression middleware, and manages the available capacity of the system according to the information, where the available capacity change information is reported to the compression driver by the compression Firmware (Firmware) of the solid state disk 20, and is transferred to the compression middleware via the compression driver.
For the solid state disk 20, a compression algorithm is integrated in a compression Firmware (Firmware), and the main function of the compression Firmware (Firmware) is to perform data compression and/or decompression operations and report dynamic storage information to the computing device 10 after compression/decompression. Specifically, the compression Firmware (Firmware) performs a corresponding operation according to a data operation command transmitted from the computing device 20, counts a change in the available capacity before and after the operation, and reports the change to the compression middleware via the compression driver of the computing device 20. Compressed Firmware (Firmware) may also store the compressed data in a storage medium. It should be noted that, the compression algorithm of the compression Firmware (Firmware) may be updated in time according to the data service and industry algorithm to maintain the optimal data compression rate. The storage medium may be a DRAM chip or a FLASH chip. The FLASH chip may be further classified into NOR type and NAND type, and the related description about the storage medium may refer to the related description about the storage medium in the solid state disk, which is not repeated herein for brevity of description.
In the embodiment of the application, the compression and decompression of the data are completed in the solid state disk, and the calculation power of the calculation equipment is not consumed, so that the compression and decompression processing and the decoupling of system business (writing, reading, changing, deleting and the like) are realized, and the system performance is not influenced. In addition, the compression algorithm adopted in the solid state disk can be updated in time according to service requirements to achieve the highest compression rate of data, so that the unit capacity cost of the solid state disk is greatly reduced, and the available storage capacity of the system is indirectly improved. In addition, the solid state disk can be timely reported to the system when the capacity is changed, so that the dynamic management of the system on the available capacity of the disk is realized.
Based on the system architecture described above, the description of the method for improving available storage capacity of the present application will be made below taking a write command, a delete command, a change command, a read command as examples, respectively.
For the following description of the method embodiments, for convenience, these are all expressed as a combination of a series of action steps, but those skilled in the art should appreciate that the implementation of the technical solution of the present application is not limited by the order of the series of action steps described.
It should be noted that, in the following description of the embodiments in fig. 2, fig. 4, fig. 7 and fig. 10, the memory may not be a solid state disk for example for exemplary description of the scheme, but the application is not limited to the type of the memory, and any memory having a compression function and a transceiver function may implement the scheme disclosed in the embodiments of the application.
First, taking a write command as an example, referring to fig. 2, fig. 2 is a flowchart of a method for improving available storage capacity according to an embodiment of the present application. The method includes, but is not limited to, the steps of:
and S101, the computing equipment sends a write command and first data to the solid state disk.
In the embodiment of the application, the computing device sends a write command and first data to the solid state disk, and the write command instructs the solid state disk to write the first data. The first data may be text, programs, images, media audiovisual files (e.g., MP3, high definition movies, etc.), and the like.
It should be noted that, the storage of data in the storage medium of the solid state disk is often based on a fixed address mapping relationship, where the address mapping relationship refers to a mapping relationship of a logical address to a physical address. And storing the data in the solid state disk according to the physical address. Therefore, the write command also carries the logical address of the first data, so that the solid state disk establishes a physical address corresponding to the logical address when the first data is stored.
In one embodiment of the present application, when the amount of data to be written is small, the computing device may send only the write-once command and the first data to the solid state disk. In some possible embodiments, if the data size of the data to be written is large, the data to be written may be divided into multiple groups of first data, and the computing device may send the write command and the first data to the solid state disk for multiple times.
In one embodiment of the present application, when the computing device sends the write command and the first data to the solid state disk multiple times, each sending of the computing device may include both the write command and the first data. In some possible embodiments, the computing device may also send the write-once command to the solid state disk first, and then send the first data to the solid state disk multiple times.
It should be noted that, in some possible embodiments, the write command and the first data may be sent together at the same time, and the embodiment of the present application is not limited in particular.
S102, the solid state disk receives the write command and the first data, compresses the first data, and stores the compressed first data.
In the embodiment of the application, after the solid state disk receives the write command and the first data from the computing device, the first data is written according to the write command. Because the solid state disk has a compression function, the writing process of the first data in the solid state disk means that: the solid state disk compresses the first data by utilizing an internal compression algorithm, and stores the compressed first data. It is easy to understand that the second data amount of the first data compressed by the solid state disk is less than or equal to the first data amount before the first data is compressed. It should be noted that, the write command also carries a logical address corresponding to the first data. The solid state disk completes the process of writing data once according to the write command, and can also be called the solid state disk executing write-once operation.
Specifically, the writing process of the first data in the solid state disk is as follows: the method comprises the steps that received first data are firstly placed into a cache of a solid state disk by the solid state disk, the first data are compressed by a compression algorithm, a physical address is determined according to the storage capacity of the compressed first data, the mapping relation between the logical address and the physical address is recorded, and finally the compressed first data are stored into a physical space corresponding to the physical address by the solid state disk, so that the solid state disk completes the compression of the first data.
It should be noted that the first data may be compressible data, for example: text, characters, images, etc., the first data may also be incompressible data, such as: MP3, high definition movies, compressed data, etc. For compressible data, the amount of data after compression (or the occupied storage capacity) is reduced, and for incompressible data, the amount of data before and after compression (or the occupied storage capacity) is unchanged, i.e. the amount of data before compression is equal to the amount of data after compression.
The compressed data is incompressible data, in other words, the compressed data cannot be recompressed. This is because the essence of compression is the imbalance in the distribution of specific characters, with short codes assigned to high frequency characters and long codes for low frequency characters achieving optimization in length. After the data is compressed once, the character distribution is averaged, and it is difficult to compress the data further. Thus, the data that has been compressed is incompressible data.
The data compression refers to a technical method for reducing the data volume to reduce the storage space and improving the transmission, storage and processing efficiency of the data without losing information. Or reorganizing the data according to a certain algorithm, so as to reduce redundancy and storage space of the data. A process of representing information with fewer data bits (or other information-related units) than are not encoded according to a particular encoding scheme. For example, if we encode "expression" as "comp", then "expression" may be represented with fewer data bits. Virtually all data (including operating systems, applications, games, etc.) can be compressed to varying degrees, except that data that does not contain discernable patterns (e.g., MP3, high definition movies, etc. multimedia audiovisual files) cannot be compressed by any compression algorithm.
The compressibility of the data model corresponding to different data is different, for example, for a database application model, the compressibility of the data is 8%, that is, the data can be compressed to 8% of the original volume, that is, 92% can not be compressed; for the application model, the compressibility of the data is 46%, i.e. the data can be compressed to 46% of the original volume. The greater the compressibility of the data model, the higher the compressibility of the data, and for the same amount of data, the greater the compressibility of the data, the smaller the storage capacity of the compressed data.
The compression algorithm integrated in the solid state disk can be Huffman coding, and can be applied to compression of images, characters and characters. The Huffman coding is a lossless compression method, and the principle is that a Huffman binary tree is constructed by using the occurrence times of data, the data with more occurrence times is arranged at the upper layer of the tree, the data with less occurrence times is arranged at the lower layer of the tree, and the path from a root node to each data can be used for coding and realizing compression. It can be simply understood that the higher the frequency with which a length is represented by a symbol, the shorter the length of the binary code after the symbol is replaced.
For example, the compression algorithm integrated in the solid state disk may be run-length encoding (RLE), also called run-length encoding or run-length encoding, which is applied to compression of an image, and the principle is to replace consecutive symbols having the same value with one symbol value or string length, so that the symbol length is smaller than the length of the original data. Only when the code of each row or each column of data changes, the code and the repeated number of the same code are recorded at one time, thereby realizing the compression of the data.
It should be noted that the compression algorithm integrated in the solid state disk may be LZW compression, LZ77 algorithm or any other compression algorithm with a higher compression rate, and the compression algorithm in the solid state disk may be updated in time according to the data type and the industry compression algorithm to achieve the optimal data compression rate.
S103, calculating the difference between the first data volume before the first data compression and the second data volume after the first data compression by the solid state disk.
In the embodiment of the application, since the solid state disk has the compression function, after the writing operation, the second data volume after the first data compression is less than or equal to the first data volume before the first data compression, namely the storage capacity occupied by the first data in the solid state disk after the first data compression is reduced, the increase of the available storage capacity of the solid state disk is indirectly indicated, and the difference between the first data volume and the second data volume is calculated, wherein the difference also represents the reduced data volume after the first data compression.
For example, the computing device writes the first data into the solid state disk, assuming that the current available storage capacity of the solid state disk is 100G, the data size before the first data compression is 10G, and assuming that the data size of the first data after the compression is reduced to 6G. For the solid state disk, if the solid state disk does not have the compression function, the data written into the solid state disk consumes 10G of storage capacity, but because the solid state disk has the compression function, the data written into the solid state disk only consumes 6G of storage capacity, and the remaining available storage capacity of the solid state disk is 94G, so that the available storage capacity of the solid state disk is saved by 4G after the first data of 10G is compressed by the solid state disk.
S104, the solid state disk sends first information to the computing device.
In the embodiment of the application, after the writing operation, the second data volume of the first data after the compression of the solid state disk is smaller than or equal to the first data volume before the compression of the first data, after the solid state disk obtains the reduced data volume (or the saved available storage capacity) of the first data after the compression, the first information is sent to the computing device, and the first information is used for indicating the difference volume between the first data volume before the compression of the first data and the second data volume after the compression of the first data, and the difference volume can also represent the reduced data volume of the first data after the compression of the solid state disk.
In one embodiment of the application, each time the solid state disk writes first data, the solid state disk sends first information to the computing device, wherein the first information represents the difference between the first data volume before the first data compression of the writing and the second data volume after the first data compression of the solid state disk.
In one embodiment of the present application, the solid state disk sends the first information to the computing device under the trigger condition that is set, for example, a difference between the first data amount before the first data compression and the second data amount after the first data compression is greater than or equal to a preset threshold, for example, 2G, 5G, 6G, 10G or other values, which is not specifically limited in the embodiment of the present application. If the reduced data volume of the first data after the second writing is compressed by the solid state disk is smaller than the preset threshold, the first information is not sent to the computing device because the first data volume and the second data volume are inconsistent after the second writing, but the solid state disk does not send the first information because the triggering condition is not met, the difference volume caused by the writing can be marked as the reduced data volume, when the data volume to be reduced (namely, the sum of the difference volumes accumulated by the writing for a plurality of times) is larger than or equal to the preset threshold, the solid state disk sends the first information to the computing device, and the first information represents the difference volume between the first data volume before the first writing data is compressed by the solid state disk and the second data volume after the first writing data is compressed by the solid state disk, and at the moment, the first data is the sum of the data written by the writing for a plurality of times. In some possible embodiments, the difference between the first data amount before the first data compression and the second data amount after the first data compression of the first information characterization may be a set fixed value.
For example, in the case where the solid state disk sends the first information to the computing device that the set trigger condition needs to be met, the amount of difference between the first data amount before the first data compression and the second data amount after the first data compression, which are characterized in the first information, may be dynamic, for example: the preset threshold value is 10G, the difference amount after the current writing operation is 12G, and the difference amount indicated by the first information is 12G; if the difference amount after the write operation is 15G, the difference amount indicated by the first information is 15G. Of course, the amount of difference indicated by the first information may be a fixed value, for example: the preset threshold value is 10G, the difference after writing operation is 12G, the difference indicated by the first information can be 10G, the difference caused by the rest 2G and the subsequent writing operation is accumulated, and the first information is sent to the computing equipment to report 10G after the accumulation of 10G. In some possible embodiments, the difference in the first information may also be an indication bit representing a specific value, the indication bit may be a number or a character, and taking a number as an example, that is, the indication bit is "1" to represent the difference as 10G.
It should be noted that, the triggering condition for the solid state device to send the first information to the computing device may or may not be set, and the embodiment of the present application is not limited specifically.
S105, the computing device records the difference indicated by the first information to a storage pool.
In the embodiment of the application, the computing device receives first information from the solid state disk, the first information indicates the difference between the first data volume before the first data compression and the second data volume after the first data compression, and the computing device establishes a management map for the storage capacity equivalent to the difference indicated by the first information. The storage capacity equivalent to the difference indicated by the first information is an idle and available virtual capacity, and the virtual capacity is mapped in the solid state disk.
It should be noted that, the virtual capacity with the same difference indicated by the first information is managed by the storage pool, so that the scattered free storage space can be integrated in a continuous address form, and space waste caused by the scattered space storage space is avoided.
After the difference indicated by the first information is recorded in the storage pool, judging whether the current storage pool reaches the preset storage capacity, if not, waiting to receive the next first information from the solid state disk and recording the difference indicated by the first information into the storage pool; if the preset storage capacity is reached, S106 is performed. The preset storage capacity is greater than or equal to the preset threshold in S104 (in the case where the trigger condition exists in the transmission of the first information in S104), and the preset storage capacity is, for example, 30G, 50G, 100G or other values, which are not specifically limited in the embodiment of the present application.
In one implementation, in connection with the system architecture shown in the embodiment of fig. 1, the compression middleware in the computing device receives first information from a compression Firmware (Firmware) of the solid state disk, and records a difference indicated by the first information into the storage pool. Assuming that the preset storage capacity is set to 50G, the storage pool is accumulated from 0, and each time the first information from the solid state disk is received, the difference indicated by the first information is added or recorded to the storage pool, and the steps are circularly executed until the storage pool reaches the preset storage capacity.
For example, the preset storage capacity is set to 50G, after the first writing operation, the computing device receives the first information indicating the 10G difference sent by the solid state disk, records the first information in the storage pool, and then the capacity of the storage pool is 10G, after the second writing operation, the computing device receives the first information indicating the 5G difference, records the first information in the storage pool, and then the capacity of the storage pool is changed to 15G, and the steps are performed until the storage pool reaches the preset storage capacity.
And S106, when the storage pool reaches a preset storage capacity, the computing equipment creates a virtual disk for the solid state disk.
In the embodiment of the application, after the computing equipment receives the first information from the solid state disk, the computing equipment records the difference indicated by the first information to the storage pool, and when the storage pool reaches the preset storage capacity, a virtual disk is created for the solid state disk. In one implementation, the capacity of the created virtual disk is a preset storage capacity. The preset storage capacity is, for example, 30G, 50G, 100G or other values, and embodiments of the present application are not particularly limited.
It should be noted that, when the storage pool reaches the preset storage capacity, the creation of the virtual disk may be: the initial capacity of the storage pool is 0, the difference indicated by the first information received for a certain time is larger than or equal to the preset storage capacity, after the difference indicated by the first information is recorded in the storage pool for the next time, the computing equipment can create a virtual disk, the capacity of the virtual disk is the preset storage capacity, and the part exceeding the preset storage capacity can be used for accumulating the next preset storage capacity in the storage pool; in some possible embodiments, when the difference indicated by the first information is less than the preset storage capacity at a time, the difference indicated by the first information may be accumulated in the storage pool, and when the sum of the differences indicated by the first information accumulated in the storage pool reaches the preset storage capacity, the computing device may create a virtual disk.
For example, assuming that the preset storage capacity is 50G, the computing device records the difference indicated by the first information into the storage pool after receiving the first information, and assuming that the capacity of the storage pool is 45G after recording for a plurality of times, at this time, the computing device receives the first information indicating the 10G difference sent by the solid state disk, and records the first information into the storage pool, the capacity of the storage pool is 55G, the current storage pool has reached the preset storage capacity 50G, the computing device creates a virtual disk for the solid state disk, and the capacity of the virtual disk becomes 50G, and simultaneously the computing device increases by 50G on the recorded available storage capacity of the solid state disk. After the virtual disk is created, the capacity of the storage pool becomes 5G, and the difference indicated by the first information is waited for recording. It is simply understood that, each time the storage pool reaches the preset storage capacity, the computing device creates a virtual disk, and the capacity of the virtual disk is the preset storage capacity.
Specifically, when the storage pool reaches a preset storage capacity, the computing device creates a virtual disk for the solid state disk, and the capacity (or virtual capacity) of the created virtual disk is the preset storage capacity. Note that the preset storage capacity is equal to or greater than the preset threshold in S104 (in the case where the trigger condition exists for transmission of the first information in S104).
It should be noted that, the virtual disk is mapped in the first physical space of the solid state disk. After the writing operation, the solid state disk obtains the reduced data volume after the first data is compressed, first information indicating the reduced data volume is reported to the computing device, and a physical space (namely a first physical space) corresponding to the reduced data volume in the solid state disk is used as a mapping of the virtual disk created by the computing device.
In a specific implementation, in combination with the system architecture shown in the embodiment of fig. 1, the compression middleware of the computing device receives first information from the solid state disk, records the difference indicated by the first information into the storage pool, creates a virtual disk when the storage pool reaches a preset storage capacity, and reports the created virtual disk to the storage software of the computing device, and then adds the capacity of the virtual disk to the recorded available storage capacity of the solid state disk, that is, adds the first preset storage capacity to the recorded available storage capacity of the solid state disk. It should be noted that, the specific numerical value corresponding to the preset storage capacity may be reported by the compression middleware to the storage software, for example, "50G", or may be an indication bit indicating the preset storage capacity, where the indication bit may be a number or a character, and the number is taken as an example, for example, the indication bit is "1", and represents that the preset storage capacity is 50G.
In some possible embodiments, the creation condition of the virtual disk may not be limited, for example, each time the computing device receives the first information, after recording the difference indicated by the first information into the storage pool, a virtual disk with a capacity equal to the difference indicated by the first information is created.
In some possible embodiments, after the computing device creates the virtual disk, the computing device issues a write command to the solid state disk, if the solid state disk stores third data to the virtual disk after executing the write command, the computing device receives fourth information sent by the solid state disk in addition to the first information sent by the solid state disk, where the fourth information indicates a data amount after the third data is compressed, and the computing device marks a virtual capacity equal to the data amount indicated by the fourth information as unavailable in the virtual disk. Because the virtual disk is mapped in the first physical space of the solid state disk, in this case, the solid state disk stores the compressed third data into the first physical space mapped with the virtual disk, but not other physical spaces without virtual disk mapping in the solid state disk. When the virtual capacity marked as unavailable due to writing operation in the virtual disk reaches a disk deletion threshold, the computing device sends fifth information to the solid state disk, wherein the fifth information is used for indicating the solid state disk to migrate data in the first physical space to the second physical space of the solid state disk, the second physical space is not mapped with the virtual disk, after the data migration is completed, the computing device deletes the created virtual disk, and meanwhile, the computing device reduces the capacity of the virtual disk on the recorded available storage capacity of the solid state disk, namely reduces the preset storage capacity. It should be noted that the disc deletion threshold is less than or equal to the preset storage capacity.
It can be seen that by implementing the embodiment of the application, the data compression change information caused by data compression is reported to the computing device after the writing operation is executed by the solid state disk, so that the computing device realizes the capacity expansion operation on the available storage capacity of the solid state disk by creating the virtual disk, thereby indirectly improving the available storage capacity of the solid state disk and reducing the unit capacity cost of the solid state disk. In addition, the writing operation business issued by the computing equipment is decoupled from the compression process of the data in the solid state disk, so that the performance of the system where the computing equipment is located is not influenced while the high compression rate is achieved.
Referring to fig. 3, the method described in the embodiment of fig. 2 is described below in some specific application scenarios.
As shown in fig. 3, it is assumed that the available storage capacity of the solid state disk B recorded by the computing device a is equal to the actual available storage capacity of the solid state disk B, and is 100G before any operation is performed. The method comprises the steps that a preset storage capacity set by a computing device A is 8G, the computing device A indicates a solid state disk B to execute a first write operation (reference numeral 1), the data volume of first data sent to the solid state disk B by the computing device A is 10G, the solid state disk B stores the compressed first data, the data volume of the compressed first data is 6G, the solid state disk B reports first information indicating 4G difference to a compression middleware of the computing device, a physical space where the 4G is located is called a first physical space, the computing device records the difference indicated by the first information to a storage pool, and the capacity of the storage pool is changed into 4G which is smaller than the preset storage capacity (the computing device is set to 8G); assuming that the computing device a instructs the solid state disk B to execute writing operation again, the data amount of the first data is 10G, the solid state disk B stores the compressed first data, the data amount of the compressed first data is 6G, the solid state disk B reports first information indicating the 4G difference amount to a compression middleware of the computing device, the storage pool after the difference amount indicated by the first information is recorded reaches a preset storage capacity of 8G, at this time, the computing device creates a virtual disk with the capacity of 8G for the solid state disk, and the virtual disk is mapped in a first physical space of the solid state disk. As shown in fig. 3, after the two writing operations, the available storage capacity of the solid state disk B is 88G, and the available storage capacity of the solid state disk B recorded by the computing device a is also 88G.
It should be noted that, before the computing device does not create the virtual disk, after the solid state disk performs the write operation, the compressed first data is stored into the physical space without virtual disk mapping in the solid state disk. After the virtual disk is created by the computing device, the solid state disk performs writing operation, and the solid state disk may store the compressed first data into a physical space without virtual disk mapping in the solid state disk, or may store the compressed first data into a physical space with virtual disk mapping in the solid state disk.
It should be noted that, when the computing device stores the compressed first data into the physical space mapped with the virtual disk in the solid state disk, the solid state disk further sends fifth information indicating the second data volume after the first data is compressed to the computing device, so that the computing device marks, as unavailable, a virtual capacity equivalent to the data volume indicated by the fifth information in the virtual disk.
The method for improving the available storage capacity provided by the embodiment of the application is further described below by taking a deletion operation as an example. Referring to fig. 4, fig. 4 is a flowchart of a method for increasing available storage capacity applied to a delete command according to an embodiment of the present application. The embodiment of fig. 4 is complementary to the embodiment of fig. 2. The method includes, but is not limited to, the steps of:
And S201, the computing equipment sends a deleting command to the solid state disk.
In the embodiment of the application, the computing device sends the deleting command to the solid state disk, wherein the deleting command is used for indicating the solid state disk to delete the target data, and the target data is part or all of the second data. It should be noted that the deletion command also carries a logical address corresponding to the write data. The second data is the data stored in the solid state disk.
It is easy to understand that if the computing device instructs the solid state disk to delete the target data, the target data must have been written to the solid state disk before. Therefore, the deletion command sent by the computing device to the solid state disk is valid only after the solid state disk in the embodiment of fig. 2 executes the write command, that is, after the data is written into the solid state disk, the deletion command sent by the computing device to the solid state disk.
S202, the solid state disk receives the deleting command and deletes the target data.
In the embodiment of the application, the solid state disk receives the deleting command from the computing device, the deleting command indicates the solid state disk to delete the target data, the target data is part or all of the second data, the physical address corresponding to the second data is found in the mapping table of the solid state disk according to the logical address carried in the deleting command, and the second data stored in the storage space corresponding to the physical address is subjected to subsequent deleting operation.
In one embodiment of the present application, when the solid state disk deletes all the second data according to the delete command, the solid state disk finds that the second data (compressed) is directly deleted in the storage medium.
In one embodiment of the present application, when the solid state disk deletes part of the second data according to the delete command, the solid state disk finds the second data (compressed) in the storage medium, decompresses the compressed second data, and then deletes the target data, where after deleting the target data in the second data, the solid state disk needs to recompress and store the remaining data in the second data, and the recompressed remaining data after deleting the second data is referred to as the deleted remaining compressed data.
It should be noted that, if part of the data of the second data is deleted, after the deleting operation is completed, the solid state disk updates the physical address corresponding to the logical address of the second data in its mapping table, that is, the physical address of the remaining compressed data; and if all the data of the second data are deleted, deleting the group of mapping relations corresponding to the logical addresses of the second data from the mapping table by the solid state disk after the deleting operation is completed.
S203, calculating the difference between the third data volume before target data compression and the fourth data volume after target data compression by the solid state disk.
In the embodiment of the application, after the solid state disk completes the deleting operation of the target data, the solid state disk calculates and calculates the difference between the third data volume before the target data is compressed and the fourth data volume after the target data is compressed. The fourth data amount after the target data compression is a difference between the data amount after the second data compression and the data amount of the remaining compressed data obtained by compressing the data remaining after deleting part or all of the second data.
For example, when the target data is written, the data amount before the target data compression is 10G, and the data amount after the target data compression is 6G, that is, after the target data is written into the solid state disk, the target data occupies only 6G of storage space, but is characterized by 10G of data information. At this time, the solid state disk deletes the target data, calculates the difference amount caused before and after the deletion operation is executed, and changes the difference amount to 4G.
S204, the solid state disk sends second information to the computing device.
In the embodiment of the application, after the solid state disk executes the deletion command, the solid state disk sends second information to the computing device, and under the condition that the deletion command is executed, the second information is feedback information sent to the computing device after the deletion command is executed on the solid state disk, the second information indicates the difference amount caused before and after the deletion command is executed on the solid state disk, specifically, the second information is used for indicating the difference amount between a third data amount before target data compression and a fourth data amount after target data compression, and the third data amount is larger than the fourth data amount.
In one implementation of the present application, after each execution of the delete command, the solid state disk sends a second message to the computing device, where the second message indicates a difference between the third data size and the fourth data size under the delete command.
In one implementation of the present application, the solid state disk sends the second information to the computing device under the set triggering condition, for example: the difference between the third data amount and the fourth data amount is equal to or greater than a preset threshold (e.g., 2G, 5G, 6G, 10G, or other value). For example, when the difference between the third data amount and the fourth data amount after the solid state disk executes the deletion command is smaller than a preset threshold, and the sum of the difference amounts accumulated by the solid state disk to be executed for multiple times is greater than or equal to the preset threshold, the solid state disk sends second information to the computing device. In this case, the difference amount indicated in the second information is the sum of difference amounts obtained after the plurality of deletion commands are executed.
It should be noted that, the triggering condition for the solid state device to send the second information to the computing device may or may not be set, and the embodiment of the present application is not limited specifically.
S205, the computing device receives the second information, and marks, in the virtual disk, a virtual capacity equivalent to the difference indicated by the second information as unavailable.
In the embodiment of the application, after receiving the second information sent by the solid state disk, the computing device marks the virtual capacity equivalent to the difference indicated by the second information as unavailable, namely that the virtual disk created by the computing device has partial virtual capacity false alarm.
It should be noted that, because the solid state disk has a compression function, the data volume of the target data (compressed) deleted by the solid state disk is smaller than the data volume before the target data is compressed, so that the actually increased available storage capacity of the solid state disk after the deletion operation is performed is smaller than the available storage capacity of the solid state disk added by the computing device, and the difference between the two is the difference between the third data volume before the target data is compressed and the fourth data volume after the target data is compressed. Therefore, after the computing device receives the second information from the solid state disk, the virtual capacity equivalent to the difference indicated by the second information needs to be marked as unavailable in the virtual disk.
It will be appreciated that the lack of data stored in the virtual capacity marked as unavailable in the virtual disk due to the delete command is essentially the marking of a portion of the free virtual capacity in the virtual disk as unavailable due to the excess storage capacity generated by the solid state disk executing the delete command.
After receiving the second information, the computing device marks the virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk, and then judges whether the unavailable virtual capacity in the current virtual disk reaches a disk deletion threshold, if not, waits for receiving the next second information from the solid state disk and marks the virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk; if the erasure threshold is reached, S206 is performed. The second preset storage capacity is equal to or smaller than the preset storage capacity in S105.
In some possible embodiments, the ratio of the disc deletion threshold to the preset storage capacity may be 0.45, 0.5, 0.6, 0.8 or less than or equal to 1. The disc deleting threshold value is set to leave a margin for the available capacity of the created virtual disc, so that the operation efficiency of the system is improved.
In a specific implementation, in combination with the system architecture shown in the embodiment of fig. 1, the compression middleware of the computing device receives, through a compression driver, second information sent by the solid state disk, and the compression middleware marks a virtual capacity equivalent to a difference indicated by the second information as unavailable. And circularly executing the step until the unavailable virtual capacity in the virtual disk reaches the second preset storage capacity.
For example, assuming that the capacity of the virtual disk is 50G, the deletion threshold is 30G, after the first deletion operation, the compression middleware of the computing device receives second information from the solid state disk, the second information indicates a difference of 10G, and the compression middleware marks, in the virtual disk, a virtual capacity equivalent to the difference indicated by the second information as unavailable, and at the moment, the unavailable virtual capacity is 5G; after the second deleting operation, the computing device receives second information indicating the 5G difference amount, marks the virtual capacity equivalent to the 5G difference amount indicated by the second information as unavailable in the virtual disk, accumulates the unavailable virtual capacity to 15G at the moment, and circulates until the unavailable virtual capacity in the virtual disk reaches a disk deleting threshold.
S206, when the unavailable virtual capacity reaches a disk deletion threshold, the computing device deletes the virtual disk.
In the embodiment of the application, after the computing device receives the second information from the solid state disk, the computing device marks the virtual capacity which is equal to the difference indicated by the second information as unavailable in the virtual disk, and when the virtual capacity marked as unavailable in the virtual disk reaches the disk deletion threshold, the computing device deletes the corresponding virtual disk. The disc deletion threshold is less than or equal to the preset storage capacity, and the embodiment of the application is not particularly limited.
The virtual capacity that is not available may reach the erasure threshold value, or may be the sum of the accumulation of the virtual capacities that are not available according to the second information flag, which is reached a certain time.
In a specific implementation, the system structure shown in fig. 1 is used to explain that after the compression middleware of the computing device receives the second information from the solid state disk, the virtual capacity equivalent to the difference indicated by the second information is marked as unavailable in the virtual disk, the computation history is marked as unavailable virtual capacity, when the unavailable virtual capacity in the virtual disk reaches the disk deletion threshold, the compression middleware deletes the virtual disk, meanwhile, the storage software of the computing device subtracts the deleted virtual disk capacity from the recorded available storage capacity of the solid state disk, and the deleted virtual disk capacity is the preset storage capacity.
It should be noted that, since the deletion command causes that no data is stored in the virtual capacity marked as unavailable in the virtual disk, when the unavailable virtual capacity in the virtual disk reaches the disk deletion threshold, the computing device directly deletes the corresponding virtual disk.
In some possible embodiments, the computing device has created a virtual disk that maps to a first physical space of the solid state disk before issuing the delete command. When the solid state disk executes a write command, third data are stored in a first physical space of the solid state disk mapped with the virtual disk, fourth information is sent to the computing device, the fourth information indicates the data amount after the third data are compressed, the computing device marks virtual capacity which is equal to the difference amount indicated by the fourth information as unavailable in the virtual disk, and when judging whether the unavailable virtual capacity in the virtual disk reaches a disk deletion threshold value, the unavailable virtual capacity includes the virtual capacity marked as unavailable by the second information (under the condition that the delete command is executed) and also includes the virtual capacity marked as unavailable by the fourth information, and the data are stored in the virtual capacity marked as unavailable by the fourth information. Under the condition, when the unavailable virtual capacity reaches the disk deletion threshold, the computing device further needs to send fifth information to the solid state disk, wherein the fifth information is used for indicating the solid state disk to migrate the data in the first physical space to the second physical space of the solid state disk, and the second physical space is not mapped by the virtual disk. After the solid state disk completes data migration, the computing device deletes the created virtual disk again, and meanwhile the computing device reduces the capacity of the virtual disk on the recorded available storage capacity of the solid state disk, namely reduces the first preset capacity.
It can be seen that by implementing the embodiment of the application, the dynamic storage information caused by deleting the compressed data is reported to the computing device after the solid state disk executes the deleting operation, so that the computing device can dynamically manage the redundant storage capacity caused by deleting the compressed data in the solid state disk by deleting the created virtual disk, the flexibility of disk management is improved, the performance of the system is not influenced, the flexibility of adjusting the available storage capacity of the solid state disk is improved, and the problem that the computing device writes in abnormity or crashes due to insufficient available storage capacity of the solid state disk is avoided.
Referring to fig. 5 and 6, the embodiment of fig. 4 is described below in some specific application scenarios.
Referring to fig. 5, fig. 5 is a schematic view of a scene based on a delete command. As shown in fig. 5, the available storage capacity of the solid state disk B initially recorded by the computing device a is the same as the actual available storage capacity of the solid state disk B, and is 100G. The solid state disk B sequentially executes the write commands twice, the data volume before data compression written by each write command is 10G, and the data volume after the data compression by the solid state disk is 6G. After the write command is executed twice, the computing device creates a virtual disk with the capacity of 8G, the virtual disk is mapped in the first physical space of the solid state disk B, the available storage capacity of the solid state disk B recorded by the computing device a is 88G, the available storage capacity of the solid state disk B is 88G, and the process of executing the write command by the solid state disk B and the process of creating the virtual disk by the computing device can be described with reference to the related description of the embodiment of fig. 3, which is not repeated herein.
Let the capacity of the virtual disk (i.e., the preset storage capacity) be 8G and the erasure threshold be 4G. The method comprises the steps that a computing device A indicates a solid state disk B to delete data written for the second time, the solid state disk B deletes compressed data written for the second time, the data volume of the solid state disk B is changed to 6G, the available storage volume of the solid state disk B is changed to 94G, meanwhile, the available storage volume of the solid state disk B recorded by the computing device A is changed to 98G, after a deletion command is executed by the solid state disk B, the solid state disk B sends second information indicating a 4G difference volume to the computing device A, the computing device A marks virtual capacity which is equal to the difference volume indicated by the second information as unavailable in a virtual disk, the virtual disk is pre-deleted by the computing device because the unavailable virtual capacity is 4G, the virtual disk is directly deleted by the computing device A because no data is stored in the virtual disk at the moment, and meanwhile, the available storage volume of the solid state disk B recorded by the computing device A is changed to 90G.
Referring to fig. 6, fig. 6 is a schematic diagram of yet another scenario based on a delete command.
As shown in fig. 6, assuming that the available storage capacity of the solid state hard disk recorded by the computing device a is (y+8), where 8G is the capacity of the virtual disk, and the virtual capacity of 2G in the virtual disk is marked as unavailable, it can be seen that data is stored in the virtual capacity of 2G, that is, the virtual capacity of 2G that is unavailable in the virtual disk. For the solid state disk B, the internal occupied storage capacity is 12G, the available storage capacity is X, wherein a first physical space of 8G is mapped with a virtual disk on the side of the computing device, and 2G of data is stored in the first physical space of 8G.
The solid state disk B deletes the data of 6G (10G before compression) according to the deleting command, the available storage capacity of the solid state disk B is (X+6) after the deleting is completed, and the second information indicating the 4G difference is reported to the computing equipment A; when the solid state disk B executes the delete command, the available storage capacity of the solid state disk B recorded by the computing device a is (y+18), and at the same time, according to the received second information, the virtual capacity equivalent to the 4G difference indicated by the second information is marked as unavailable in the virtual disk, at this time, the unavailable virtual capacity in the virtual disk is 6G, and the delete threshold is reached (assuming to be 5G), and the computing device pre-deletes the virtual disk, because there is data in the virtual disk, before deleting the virtual disk, the computing device a sends information (for example, the fifth information in the above embodiment) indicating that the solid state disk migrates the data in the virtual disk to be deleted to the solid state disk B, and in particular, the information indicates that the solid state disk B migrates the data in the second physical space to the second physical space of the solid state disk, where the virtual disk is not mapped with the second physical space. After the solid state disk B completes data migration, the available storage capacity of the solid state disk B is changed from (X+6) to (X+4), the computing device A deletes the virtual disk, the available storage capacity of the solid state disk B recorded by the computing device A is changed from (Y+18) to (Y+10), and the (Y+10) is smaller than or equal to (X+4).
It should be noted that, the 2G data to be migrated in the virtual disk to be deleted in the computing device a may be written by executing a write command, may be written by executing a change command, or may be written in the virtual disk together by executing the write command and the change command. The related description of the modification operation (or executing the modification command) may refer to the embodiment of fig. 7, and will not be repeated herein for the sake of indirection of the description.
Simply understand that after the solid state disk executes the delete command, reporting related information to the computing device, when the computing device judges that the unavailable virtual capacity in a certain virtual disk reaches the trigger condition of deleting the virtual disk (namely, the unavailable virtual capacity in the virtual disk reaches the delete threshold), before deleting the virtual disk, whether the computing device instructs the solid state disk to perform data migration operation depends on whether data exists in the virtual disk to be deleted, if so, the solid state disk is indicated to have performed change operation before executing the delete command, or has performed write operation after creating the virtual disk and the solid state disk stores write data into the virtual disk, and the solid state disk needs to perform data migration according to the instruction of the computing device; if no data exists, the solid state disk does not need to carry out data migration, and the computing equipment directly deletes the virtual disk.
The method for improving the available storage capacity according to the embodiment of the present application is further described below by taking a change command as an example. Referring to fig. 7, fig. 7 is a flowchart of a method for improving available storage capacity according to an embodiment of the present application. The embodiment of fig. 7 may be independent of the embodiment of fig. 4 or may be complementary to the embodiments of fig. 2 and 4. The method includes, but is not limited to, the steps of:
s301, the computing device sends a change command to the solid state disk.
In the embodiment of the application, the computing device sends a change command to the solid state disk, wherein the change command is used for indicating the solid state disk to change target data, and the target data is part or all of second data, namely, the change command is used for indicating the solid state disk to change part or all of the second data. It should be noted that, the change command also carries a logical address corresponding to the write data.
It is easy to understand that if the computing device instructs the solid state disk to change part or all of the second data, the second data must have been written to the solid state disk before. Therefore, the computing device sends the change command to the solid state disk, and after the solid state disk in the embodiment of fig. 2 executes the write command, that is, after the data is written into the solid state disk, the change command sent by the computing device to the solid state disk is valid.
S302, the solid state disk receives a change command and changes part or all of the second data.
In the embodiment of the application, after the solid state disk receives the change command from the computing device, the solid state disk compresses and stores the second data after the second data is written into the solid state disk, namely, the second data stored in the physical space of the solid state disk is the compressed second data, so that if partial or all data of the second data are to be changed, the solid state disk needs to find the compressed second data stored in the corresponding physical space in the solid state disk according to the logical address carried by the change command, decompress the compressed second data, and then modify partial or all data in the decompressed second data according to the requirement.
S303, compressing and storing the changed second data by the solid state disk.
In the embodiment of the application, after the solid state disk changes part or all of the second data, the changed second data is also required to be compressed and stored. It is easy to understand that the data stored in the physical space of the solid state disk are all compressed data, so that after some or all of the second data are changed, the solid state disk will recompress the changed second data and store the second data.
It should be noted that, the data amount before the second data compression after the modification is equal to the first data amount before the second data compression, but the fifth data amount after the second data compression after the modification is larger than the sixth data amount after the second data compression, because the compression rate is reduced when the second data after the modification is compressed again due to the change of the data source after the second data is modified, the fifth data amount after the modification is larger than the sixth data amount of the compressed second data stored in the solid state disk before the modification.
After the solid state disk recompresses and stores the changed second data, the fifth data volume after the second data is compressed after the second data is changed is larger than the sixth data volume before the second data is changed, so that the physical address corresponding to the second data after the second data is compressed is changed, the physical address corresponding to the logical address of the second data is updated in the mapping table by the solid state disk, and the logical address of the second data after the second data is changed is still the logical address of the second data.
It should be noted that, because the data stored in the solid state disk are all compressed data, in order to implement the modification of the second data, the solid state disk needs to decompress and then modify the compressed second data. Whether the part or the whole of the second data is modified, because the whole writing data is decompressed, after the modification of the part or the whole of the second data is completed, the modified second data is required to be recompressed and stored, namely, the data after the modification of the second data is compressed is stored in the physical space of the solid state disk.
S304, the solid state disk calculates the difference between the fifth data volume after the second data compression after the modification and the sixth data volume after the second data compression.
In the embodiment of the application, the fifth data volume after the second data compression after the modification is larger than the sixth data volume after the second data compression, namely, the storage capacity occupied by the compressed package obtained after the second data compression after the modification is recompressed is increased, correspondingly, the available storage capacity of the solid state disk is reduced, and the reduced available storage capacity is the difference value between the fifth data volume and the sixth data volume.
For example, assuming that the computing device indicates that the solid state disk writes the second data of 10G, after the solid state disk completes the writing operation, the storage capacity actually occupied by the second data of 10G in the solid state disk is 6G, so that the data amount of 4G is reduced, and therefore, the solid state disk reports the first information indicating that the difference amount is 4G to the computing device after the writing operation is completed. At this time, if the computing device indicates that the solid state disk changes the second data of 10G that has just been written in, the solid state disk changes the stored compressed second data (the data amount is 6G), and then recompresses and stores the changed second data, and it is assumed that the data amount of the compressed second data after the change is 7G, that is, the second data occupies more 1G of storage space after the change compared with that before the change.
S305, the solid state disk sends second information to the computing device.
In the embodiment of the application, after the solid state disk executes the change command, the solid state disk sends second information to the computing device, and under the condition that the change command is executed, the second information is feedback information sent to the computing device after the change command is executed on the solid state disk, the second information indicates the difference amount caused before and after the change command is executed on the solid state disk, and specifically, the second information is used for indicating the difference amount between the fifth data amount after the change and the sixth data amount after the second data compression.
In one embodiment of the present application, each time the solid state disk executes the change command, the solid state disk sends a second message to the computing device, where the second message is used to indicate a difference between the fifth data size and the sixth data size under the change command.
In an embodiment of the present application, a trigger condition may be further set for sending the second information, that is, after the solid state disk executes the multiple change commands, and when a difference between the fifth data amount after the second data is compressed after the change and the sixth data amount after the second data is compressed due to the multiple change commands is greater than or equal to a preset threshold (for example, 2G, 5G, 6G, 10G or other values), the solid state disk sends the second information to the computing device, where the second information is used to indicate a difference between the fifth data amount and the sixth data amount under the multiple change commands, and the difference is a sum of differences caused by performing the multiple change commands historically. It should be noted that, the multiple changes to the second data may be multiple changes to the second data written in the same time, or may be changes to the second data written in different times, which is not limited in particular.
For example, the preset threshold set by the solid state disk is 3G, the solid state disk performs first modification on the second data according to the modification command, the difference between the fifth data amount after compression of the second data after modification and the sixth data amount occupied in the solid state disk before modification of the second data is assumed to be 1G, the subsequent solid state disk performs second modification on the second data, if the difference amount of the data amounts corresponding to the second data after modification (recompression) and before modification (before decompression) after the second modification operation is 2G, the sum of the difference amounts caused by the two modification operations is 3G (reaches the preset threshold), the solid state disk sends second information to the computing device once, and the second information indicates the difference amount caused by the modification command to be 3G; if the difference between the storage capacity after the second change operation and the storage capacity before the change is 2.5G, the sum of the difference amounts caused by the two change operations is 3.5G (reaching the preset threshold), the solid state disk sends a second message to the computing device, the second message indicates that the difference amount caused by the change command is 3.5G, and in some possible embodiments, the difference amount indicated by the second message sent by the solid state disk to the computing device may be 3G, and the remaining 0.5G is accumulated until the next preset threshold is met and then reported.
It should be noted that, the triggering condition for the solid state device to send the second information to the computing device may or may not be set, and the embodiment of the present application is not limited specifically.
S306, the computing device receives the second information, and marks virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk.
In the embodiment of the application, after receiving the second information sent by the solid state disk, the computing device marks the virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk, namely that part of capacity in the virtual disk created by the computing device is unavailable.
It should be noted that, the virtual capacity equivalent to the difference indicated by the second information is marked as unavailable by the computing device, because the solid state disk occupies a part of the free virtual capacity in the virtual disk after executing the change command, and the part of the virtual capacity is already used for storing data, so that the occupied part of the virtual capacity needs to be marked as unavailable.
After receiving the second information, the computing device marks the virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk, and then judges whether the unavailable virtual capacity in the current virtual disk reaches a disk deletion threshold, if not, waits for receiving the next second information from the solid state disk and marks the virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk; if the erasure threshold is reached, S307 is performed.
In a specific implementation, in combination with the system architecture shown in the embodiment of fig. 1, the compression middleware of the computing device receives, through a compression driver, second information sent by the solid state disk, marks, in the virtual disk, a virtual capacity equivalent to a difference indicated by the second information as unavailable, and performs this step in a circulating manner until the unavailable virtual capacity in the virtual disk reaches a disk deletion threshold. The disk deletion threshold is less than or equal to a preset storage capacity, wherein the preset storage capacity is the capacity of a virtual disk created by the computing device.
S307, the computing device sends third information to the solid state disk.
In the embodiment of the application, when the unavailable virtual capacity in the virtual disk reaches the disk deletion threshold, the condition that the computing device deletes the virtual disk is met, but before the computing device deletes the virtual disk, the computing device needs to send third information to the computing device firstly, wherein the third information is used for indicating the solid state disk to migrate the data in the first physical space to the second physical space of the solid state disk, the second physical space has no virtual disk and mapping thereof, and the simple understanding is that the fourth information indicates the solid state disk to migrate the data in the virtual disk to be deleted.
It should be noted that, the virtual disk is mapped to a first physical space of the solid state disk, where the first physical space is used for storing data with a difference between the fifth data amount and the sixth data amount, in other words, the first physical space is used for storing a part of data compressed by the second data after being modified by the solid state disk, and the data amount of the part of data is the difference between the fifth data amount and the sixth data amount.
For example, the data amount after the second data compression is 6G, the data amount after the second data compression after the modification is 7G, after the modification operation is completed, 6G in the data obtained after the second data compression after the modification is stored in the original position in the solid state disk, and the remaining 1G is stored in the first physical space of the solid state disk mapped with the virtual disk, so that the remaining 1G data can be considered to be stored in the virtual disk. And the computing equipment can delete the virtual disk until the data migration in the virtual disk to be deleted is completed.
And S308, the solid state disk receives the third information, and data in the first physical space is migrated to a second physical space of the solid state disk according to the third information.
In the embodiment of the application, after the solid state disk receives the third information from the computing device, the computing device is explained to pre-delete the created virtual disk, the solid state disk transfers the data volume of the first physical space to the second physical space of the solid state disk according to the third information, the data in the virtual disk to be deleted is transferred to different physical spaces of the solid state disk, and the physical space where the data is located after the data transfer is not mapped by the virtual disk.
It should be noted that, after the solid state disk completes migration of the data according to the third information, if the computing device deletes the virtual disk, the virtual disk and its mapping are not available in the first physical space in the solid state disk, and the first physical space is restored to the local storage space of the solid state disk. Therefore, dynamic management of the virtual disk is realized, and the operation efficiency of the system is improved.
In some possible embodiments, after the computing device adds the virtual disk, the solid state disk executes the write command and stores the compressed third data to the virtual disk, in which case the unavailable virtual capacity in the virtual disk includes two portions, wherein one portion of the unavailable virtual capacity is marked as unavailable due to the partial write command and the other portion of the unavailable virtual capacity is marked as unavailable due to the change command. In this case, the data in the virtual disk to be deleted, which is migrated according to the third information, includes not only the partially compressed third data, but also the partially compressed modified second data.
In some possible embodiments, the unavailable virtual capacity in the virtual disk may further comprise three portions, the unavailable virtual capacity of the first portion being marked as unavailable due to a partial write command, the unavailable virtual capacity of the second portion being marked as unavailable due to a change command, the unavailable virtual capacity of the third portion being marked as unavailable due to a delete command. In this case, the data in the virtual disk to be deleted, which is migrated according to the third information, includes not only the partially compressed third data, but also the partially compressed modified second data.
S309, when the unavailable virtual capacity reaches a disk deletion threshold, the computing device deletes the virtual disk.
In the embodiment of the application, after the computing device receives the second information from the solid state disk, the computing device marks the virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk, and when the virtual capacity marked as unavailable in the virtual disk reaches the disk deletion threshold, the computing device deletes the corresponding virtual disk. The disc deletion threshold is less than or equal to the preset storage capacity, and the embodiment of the application is not particularly limited.
In a specific implementation, in combination with the system architecture shown in the embodiment of fig. 1, after receiving the second information from the solid state disk, the compression middleware of the computing device marks a virtual capacity equivalent to a difference indicated by the second information as unavailable in the virtual disk, marks a calculation history as an unavailable virtual capacity, and when the unavailable virtual capacity in the virtual disk reaches a disk deletion threshold, the compression middleware deletes the virtual disk, and meanwhile, the storage software of the computing device subtracts the deleted virtual disk capacity from the recorded available storage capacity of the solid state disk, where the deleted virtual disk capacity is a preset storage capacity.
It should be noted that, when the unavailable virtual capacity reaches the disk deletion threshold, and before the computing device pre-deletes the created virtual disk, S307-S308 are also executed.
It can be seen that, by implementing the embodiment of the application, the dynamic storage information caused by changing the compressed data is reported to the computing device after the solid state disk executes the change command, so that the computing device can dynamically manage the redundant storage capacity caused by changing the compressed data in the solid state disk by deleting the created virtual disk, the flexibility of disk management is improved, the performance of the system is not influenced, the flexibility of adjusting the available storage capacity of the solid state disk is improved, and the problem that the computing device writes in abnormity or crashes due to insufficient available storage capacity of the solid state disk is avoided.
Referring to fig. 8 and 9, the embodiment of fig. 7 is described below in some specific application scenarios.
Referring to fig. 8, fig. 8 is a schematic view of a scenario based on a change command. As shown in fig. 8, after the computing device a instructs the solid state disk B to execute the write command twice, the computing device a creates a virtual disk with a capacity of 8G, and the available storage capacity of the solid state disk B recorded by the computing device a is 88G; after the solid state disk B has performed the write command twice, the available storage capacity becomes 88G, with 8G being the first physical space mapped with the virtual disk created by computing device a. In addition, under the supplementary assumption that the data volume of the data written into the solid state disk B each time is 10G, the data volume of the data compressed by the solid state disk is 6G, the preset storage capacity is 8G, and the disk deletion threshold is 4G.
The solid state disk B executes the change command (e.g. reference numeral 1), and it is assumed that the data volume after the data compression after the change is completed is changed from 6G to 8G, and for the data after the data recompression after the change, 6G is stored in the original physical space of the solid state disk B, and the remaining 2G is stored in the first physical space of the solid state disk B, which is mapped with the virtual disk in the computing device a, so that the solid state disk B sends second information (e.g. reference numeral 2) indicating the difference volume of 2G to the computing device a, and the computing device a marks the virtual capacity in the virtual disk, which is equivalent to the difference volume indicated by the second information, as unavailable, and at this time, the unavailable virtual capacity in the virtual disk is 2G and does not reach the deletion threshold.
The solid state disk B executes the change command again (e.g. reference numeral 3), and likewise, the data volume after the data compression after the change is completed is changed from 6G to 8G, the solid state disk B still preferentially stores the 2G exceeding the change into a first physical space mapped with the virtual disk in the solid state disk B, and sends second information (e.g. reference numeral 4) indicating the 2G difference volume to the computing device a, the computing device a marks the virtual capacity equivalent to the difference volume indicated by the second information as unavailable in the virtual disk, at this time, the unavailable virtual capacity in the virtual disk is 4G, the computing device a wants to delete the virtual disk, and because the virtual disk to be deleted has 4G data, the computing device a sends third information (e.g. reference numeral 6) to the solid state disk B, the third information indicates that the solid state disk B migrates the data in the first physical space mapped with the virtual disk to be deleted to a second physical space of the solid state disk B, and the second physical space has no virtual space mapped with the second physical space. After the solid state disk B completes data migration, the computing device A deletes the virtual disk (as shown by reference numeral 7), at this time, the available storage capacity of the solid state disk B recorded by the computing device A is changed from 88G to 80G, and after the solid state disk B completes data migration, the available storage capacity is changed from 88G to 84G.
As shown in fig. 9, fig. 9 is a schematic diagram of yet another scenario based on a change command.
As shown in fig. 9, assuming that the available storage capacity of the solid state disk recorded by the computing device a is (y+8), where 8G is the capacity of the virtual disk, and the virtual capacity of 2G in the virtual disk is marked as unavailable, it can be seen that data is stored in the virtual capacity of 2G, that is, the virtual capacity of 2G that is unavailable in the virtual disk. For the solid state disk B, the available storage capacity is X, where there is a mapping between the first physical space of 8G and the virtual disk on the computing device side, and 2G of data is stored in the first physical space of 8G. Assume that the second preset storage capacity is 4G.
The solid state disk B changes the second data according to the change command, after the change is completed, the original 6G of the data volume after the changed second data compression is changed into 8G, the solid state disk B stores the exceeding 2G into a first physical space mapped with the virtual disk created by the computing device a, and reports second information indicating the 2G difference to the computing device a, the computing device a marks a virtual capacity equivalent to the 2G difference indicated by the second information as unavailable in the virtual disk according to the received second information, at this time, the unavailable virtual capacity in the virtual disk is accumulated into 4G, and a disk deletion threshold (4G) is reached, the computing device pre-deletes the virtual disk, and because the virtual disk has data in the virtual disk, before deleting the virtual disk, the computing device a sends information (for example, the third information in the above embodiment) indicating that the solid state disk migrates the data in the virtual disk to be deleted to the second physical space, specifically, the information indicates that the solid state disk B migrates the data in the second physical space to the second physical space of the solid state disk, and the second physical space has no virtual disk mapped with the virtual disk. After the solid state disk B completes data migration, the available storage capacity of the solid state disk B is changed from X to (X-4), the computing device A deletes the virtual disk, the available storage capacity of the solid state disk B recorded by the computing device A is changed from (Y+8) to Y, and Y is smaller than or equal to (X-4).
In the application scenario shown in fig. 9, it can be seen that, before the solid state disk executes the change operation, the data is stored in the virtual disk, where the data in the virtual disk may be the data stored only by the change command, the data stored only by the write command, or the data stored only by the change command and the write command.
It should be noted that, after the solid state disk executes the change command, the first physical space mapped with the virtual disk created by the computing device will cause data to be stored, which means that the virtual disk stores data. The computing device determines whether a virtual capacity marked as unavailable in a virtual disk reaches a trigger condition for deleting the virtual disk (i.e., the unavailable virtual capacity in the virtual disk reaches a disk deletion threshold), where the unavailable virtual capacity may be marked as unavailable due to at least one of a delete command, a write command, and a change command.
The method for improving the available storage capacity according to the embodiment of the present application is further described below by taking a read command as an example. Referring to fig. 10, fig. 10 is a flowchart of a method for improving available storage capacity applied to a read command according to an embodiment of the present application. The embodiment of fig. 10 may be independent of the embodiment of fig. 4 and 7 or may be complementary to the embodiment of fig. 2, 4 and 7. The method includes, but is not limited to, the steps of:
S401, the computing device sends a read command to the solid state disk.
In the embodiment of the application, the computing device reads the command to the read command, wherein the read command is used for indicating the solid state disk to read the specified data. It should be noted that, the read command also carries a logical address corresponding to the specified data.
It is easy to understand that if the computing device instructs the solid state disk to read certain data, the data must have been written to the solid state disk before. Therefore, the computing device sends the read command to the solid state disk, and the computing device sends the delete command to the solid state disk only after the computing device sends the write command to the solid state disk, i.e. after the data is written into the solid state disk.
It should be noted that the target data to be read by the computing device may be the first data written in the embodiment of fig. 2, or may be the second data modified in the embodiment of fig. 7.
S402, the solid state disk receives a read command, and decompresses the specified data according to the read command.
In the embodiment of the application, after the solid state disk receives the read command from the computing device, the physical address mapped with the read command in the storage medium is found according to the logical address carried in the read command, and because the data stored in the solid state disk are all compressed, the data taken out of the mapped physical address by the solid state disk are the compressed specified data, and the specified data (compressed) are decompressed by the solid state disk, so that the specified data to be read by the computing device can be obtained.
The data decompression means that the compressed specified data is restored to the state before compression, and the storage capacity of the decompressed specified data is the same as the storage capacity before decompression.
In one implementation, the solid state disk finds the physical address mapped in the storage medium according to the read command, and takes out the specified data (compressed) from the physical space corresponding to the physical address, and puts the specified data into the cache of the solid state disk to decompress the data. Taking the system framework shown in fig. 1 as an example, the compression Firmware (Firmware) receives a read command transmitted by the storage software of the computing device through the SSD driver, decompresses the specified data (compressed) in the cache, and returns the decompressed specified data to the storage software of the computing device through the SSD driver.
S403, the solid state disk sends decompressed specified data to the computing device.
In the embodiment of the application, after the computing device obtains the decompressed specified data, the decompressed specified data is sent to the computing device, so that the specified data is read.
It should be noted that, the read command sent by the computing device does not cause the change of the available storage capacity of the solid state disk for the solid state disk, so that the solid state disk does not need to report dynamic storage change information to the computing device under the read operation command, and only needs to return a result after the read operation command is executed to the computing device.
It can be seen that by implementing the embodiment of the application, the compressed data stored in the solid state disk is read, and the decompression processing of the data by the solid state disk and the read operation business of the system are decoupled during the read operation, so that the performance of the system is not affected. In addition, the data to be read are compressed by the solid state disk and then stored, so that the capacity of the solid state disk is indirectly improved, and the unit capacity cost of the solid state disk is reduced.
Referring to fig. 11, an embodiment of the present application further provides a schematic structural diagram of an apparatus, where the apparatus 30 includes at least a processor 310, a memory 311, a receiver 312, and a transmitter 313, and the receiver 312 and the transmitter 313 may be replaced by communication interfaces for providing information input/output to the processor 310. Alternatively, the memory 311, the receiver 312, and the transmitter 313 and the processor 310 may be connected or coupled by a bus. Optionally, the apparatus 30 further includes a display 314, where the display 314 is configured to display the available storage capacity of the solid state disk stored in the local record, the data returned by the solid state disk after the reading operation is completed, and so on. The apparatus 30 may be a computing device in the embodiment of fig. 1.
The transmitter 313 is configured to send a write command and write data or other data operation commands (e.g., a change command, a delete command, and a read command) to the solid state disk, and under the delete command, the transmitter 313 is further configured to send information indicating data migration (i.e., third information in fig. 7) to the solid state disk. The receiver 312 is configured to receive information (e.g., the first information in fig. 2, the second information in fig. 4, and the second information in fig. 7) indicating a change in the available storage capacity, which is sent after the solid state disk executes the corresponding data operation command, and the receiver 312 is further configured to receive a result returned after the data operation command is sent by the solid state disk, for example, decompressed data to be read, which is returned after the read operation is completed in fig. 10. The receiver 312 and the transmitter 313 may include antennas and chipsets for communicating with the solid state disk or other hardware in the solid state disk directly or through an air interface. The receiver 312 and the transmitter 313 constitute a communication module that may be configured to receive and transmit information in accordance with one or more other types of wireless communications (e.g., protocols), such as bluetooth, IEEE 802.11 communication protocols, cellular technology, worldwide interoperability for microwave access (Worldwide Interoperability for Microwave Access, wiMAX) or LTE (Long Term Evolution ), zigBee protocols, dedicated short range communications (Dedicated Short Range Communications, DSRC), and RFID (Radio Frequency Identification ) communications, among others.
The Memory 311 may include Volatile Memory (Volatile Memory), such as random access Memory (Random Access Memory, RAM); the Memory 311 may also include a Non-Volatile Memory (Non-Volatile Memory), such as a Read-Only Memory (ROM), a Flash Memory (Flash Memory), a Hard Disk (HDD), or a Solid State Drive (SSD); the memory 311 may also include a combination of the above. The memory 311 may store programs and data, wherein the stored programs include: data operation commands (write command, read command, change command, and delete command), addition and removal programs of virtual disk, drive programs of solid state disk, and the like, and the stored data includes: presetting storage capacity, a disc deleting threshold value, a logic address for writing data, data to be read, available storage capacity of a solid state disk and the like. The memory 311 may exist alone or may be integrated within the processor 310.
Specific implementations of the operations performed by the processor 310 may refer to specific operations in the method embodiments described above of creating a storage pool based on the first information in fig. 2, creating a virtual disk, marking virtual capacity unavailable based on the second information in fig. 4 or the second information in fig. 7, deleting a virtual disk, and so on. The processor 310 may be comprised of one or more general purpose processors, such as a central processing unit (Central Processing Unit, CPU), or a combination of CPU and hardware chips. The hardware chip may be an Application-specific integrated circuit (ASIC), a programmable logic device (Programmable Logic Device, PLD), or a combination thereof. The PLD may be a complex programmable logic device (Complex Programmable Logic Device, CPLD), a Field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose array logic (Generic Array Logic, GAL), or any combination thereof. The processor 310 is specifically configured to call program code and data in the memory 311 to perform various operations described above. In one implementation, the storage software, compression middleware, SSD driver, and compression driver of FIG. 1 may run in processor 310.
In an embodiment of the present application, the apparatus 30 is configured to implement the method on the computing device side described in the embodiments of fig. 2, fig. 4, fig. 7 and fig. 10.
Referring to fig. 12, an embodiment of the present application further provides a schematic structural diagram of an apparatus, where the apparatus 40 includes at least a processor 410, a memory 411, a receiver 412, and a transmitter 413. The receiver 412 and the transmitter 413 may also be replaced with communication interfaces for providing information input/output to the processor 410. Alternatively, the memory 411, the receiver 412 and the transmitter 413 and the processor 410 may be connected or coupled by a bus. The device 40 may be a solid state disk in the embodiment of fig. 1.
The receiver 412 is configured to receive a write command from the computing device and write data or other data manipulation commands (e.g., a change command, a delete command, and a read command), under which the receiver 412 is further configured to receive information from the computing device indicating data migration (i.e., the third information in fig. 7). The transmitter 413 is configured to send information (e.g., the first information in fig. 2, the second information in fig. 4, and the second information in fig. 7) indicating a change in the available storage capacity after the solid state disk executes the corresponding data operation command to the computing device, and the transmitter 413 is further configured to send a result returned after the data operation command to the computing device, for example, the read operation is completed in fig. 10 to send the decompressed data to be read, and so on. The receiver 412 and transmitter 413 may include antennas and chipsets for communicating with the solid state disk or other hardware in the solid state disk, either directly or through an air interface. The receiver 412 and the transmitter 413 constitute a communication module that may be configured to receive and transmit information according to one or more other types of wireless communication (e.g., protocols), such as bluetooth, IEEE 802.11 communication protocols, cellular technology, worldwide interoperability for microwave access (Worldwide Interoperability for Microwave Access, wiMAX) or LTE (Long Term Evolution ), zigBee protocols, dedicated short range communication (Dedicated Short Range Communications, DSRC), and RFID (Radio Frequency Identification ) communication, among others.
The Memory 411 may include a Non-Volatile Memory (Non-Volatile Memory), for example, may be a Flash chip, which may specifically be a Flash granule (NAND Flash); memory 411 may also be dynamic random access memory (Dynamic Random Access Memory, DRAM), random access memory (Random Access Memory, RAM) or other types of volatile memory; the memory 411 may also include combinations of the above. The memory 411 may store programs including: compression and decompression program, setting program transmitted by capacity change information (for example, first information in fig. 2, second information in fig. 4, and second information in fig. 7), initialization program of solid state disk, and the like, and stored data include: a preset threshold, an address mapping table, first data, second data, modified second data, etc. The memory 411 may exist alone or may be integrated within the processor 410. Memory 411 may be a flash memory granule in fig. 1.
Specific implementation of the operations performed by the processor 410 may refer to the above-described method embodiment for compressing and storing the first data, performing the change command on part or all of the second data, and performing the operations of deleting the command, data migration and storage on part or all of the second data. The processor 410 may be comprised of one or more general purpose processors, such as a central processing unit (Central Processing Unit, CPU), or a combination of CPU and hardware chips. The hardware chip may be an Application-specific integrated circuit (ASIC), a programmable logic device (Programmable Logic Device, PLD), or a combination thereof. The PLD may be a complex programmable logic device (Complex Programmable Logic Device, CPLD), a Field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose array logic (Generic Array Logic, GAL), or any combination thereof. The processor 410 has integrated therein a compression decompression function. Processor 410 may be the SSD controller of FIG. 1. The processor 410 is specifically configured to call program code and data in the memory 411 to perform the operations described above. In one implementation, processor 410 may be the SSD controller of FIG. 1, and a Firmware (Firmware) and FTL module may be run in processor 410.
In the embodiment of the present application, the device 40 is used to implement the method on the solid state disk described in the embodiments of fig. 2, fig. 4, fig. 7 and fig. 10.
Referring to fig. 13, the embodiment of the present application further provides a functional structural schematic of an apparatus, where the apparatus 32 further includes a sending unit 320, a receiving unit 321, and a processing unit 322.
The sending unit 320 is configured to send a write command and first data to the memory, where the write command is used to instruct the memory to write the first data; the receiving unit 321 receives the first information from the memory; the processing unit 322 is configured to create a virtual disk for the memory in response to the first information.
The functional modules of the apparatus 32 may be used to implement the computing device-side method described in the embodiment of fig. 2. In the embodiment of fig. 2, the transmitting unit 320 may be used to perform S101, the receiving unit 321 may be used to perform S105, and the processing unit 322 may be used to perform S105 and S106. The functional modules of the apparatus 32 may also be used to perform the method of computing the device side in the embodiments of fig. 4, 7 and 10, and the present application is not repeated here for brevity of description.
Referring to fig. 14, an embodiment of the present application further provides a functional structural schematic of an apparatus, where the apparatus 42 at least includes the apparatus 42 further includes a receiving unit 420, a processing unit 421, and a transmitting unit 422.
Wherein the receiving unit 420 is configured to receive a write command and first data from a computing device; the processing unit 421 is configured to compress and store the first data; the sending unit 422 is configured to send first information to the computing device, where the first information is used to instruct the computing device to create a virtual disk for the memory.
The functional modules of the device 42 may be used to implement the method on the solid state disk described in the embodiment of fig. 2. In the embodiment of fig. 2, the receiving unit 420 may be used to perform S102, the processing unit 421 may be used to perform S102 and S103, and the transmitting unit 422 may be used to perform S104. The functional modules of the apparatus 42 may also be used to execute the method on the solid state disk in the embodiments of fig. 4, 7 and 10, and for brevity of description, the disclosure is not repeated here.
The embodiment of the application also provides a storage system which comprises a computing device, such as a computer, and further optionally at least one memory, such as a solid state disk. The computing device may be the apparatus described in the embodiment of fig. 11, or the apparatus described in the embodiment of fig. 13. Further, the memory may alternatively be the device described in the embodiment of fig. 12, or the device described in the embodiment of fig. 14.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
It should be noted that, all or part of the steps in the various methods of the foregoing embodiments may be implemented by a program, which may be stored in a computer readable storage medium, including Read-Only Memory (ROM), random-access Memory (Random Access Memory, RAM), programmable Read-Only Memory (Programmable Read-Only Memory, PROM), erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), one-time programmable Read-Only Memory (One-time Programmable Read-Only Memory, OTPROM), electrically erasable programmable Read-Only Memory (EEPROM), compact disc Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM), or other optical disc Memory, magnetic disc Memory, tape Memory, or any other medium capable of being used for carrying or storing data.
The technical solution of the present application may be embodied essentially or partly or all of the technical solution or in the form of a software product stored in a storage medium, comprising several instructions for causing a device (which may be a personal computer, a server, or a network device, a robot, a single-chip microcomputer, a chip, a robot, etc.) to perform all or part of the steps of the method according to the embodiments of the present application.

Claims (24)

1. A method of increasing available storage capacity for use in a computing device, the method comprising:
transmitting a write command and first data to a memory, wherein the write command is used for indicating the memory to write the first data;
receiving first information from the memory, wherein the first information is used for indicating the difference between a first data volume before the first data compression and a second data volume after the first data compression;
and responding to the first information, and creating a virtual disk for the memory.
2. The method of claim 1, wherein said creating a virtual disk for said memory in response to said first information comprises:
And creating the virtual disk for the memory according to the difference indicated by the first information.
3. The method of claim 1 or 2, wherein after the sending of the write command and the first data to the memory, the method further comprises:
sending a deleting command or a changing command to the memory, wherein the deleting command is used for indicating the memory to delete target data, the changing command is used for indicating the memory to change the target data, and the target data is part or all of second data;
receiving second information from the memory;
and deleting the virtual disk in response to the second information.
4. The method of claim 3, wherein the second information is used to indicate an amount of difference caused before and after the memory executes the delete command or the change command;
after the receiving the second information from the memory, the method further comprises:
marking virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk;
said deleting said virtual disk in response to said second information comprises:
And deleting the virtual disk according to the unavailable virtual capacity.
5. The method of claim 4, wherein the step of determining the position of the first electrode is performed,
when the deleting command indicates the memory to delete the target data, the difference indicated by the second information is the difference between the third data amount before the target data is compressed and the fourth data amount after the target data is compressed; or alternatively, the process may be performed,
when the change command instructs the memory to change the target data, the amount of difference indicated by the second information is an amount of difference between a fifth amount of data compressed by the changed second data and a sixth amount of data compressed by the second data.
6. The method of claim 5, wherein the virtual disk is mapped to a first physical space of the memory, the first physical space being used to store data of a magnitude of a difference between the fifth amount of data and the sixth amount of data;
before said deleting said virtual disk according to said unavailable virtual capacity, said method further comprises:
and sending third information to the memory, wherein the third information is used for indicating the memory to migrate the data of the first physical space to a second physical space of the memory.
7. A method of increasing available storage capacity for a memory, the method comprising:
receiving a write command and first data from a computing device;
compressing and storing the first data;
and sending first information to the computing device, wherein the first information is used for indicating the computing device to create a virtual disk for the memory, and the first information is used for indicating the difference between the first data volume before the first data compression and the second data volume after the first data compression.
8. The method of claim 7, wherein the step of determining the position of the probe is performed,
the sending, to the computing device, first information, where the first information is used to instruct the computing device to create a virtual disk for the memory, includes:
and sending the first information to the computing device, wherein the first information is used for indicating the computing device to create the virtual disk for the memory based on the difference indicated by the first information.
9. The method of claim 8, wherein after said compressing and storing said first data, said method further comprises:
receiving a deletion command or a change command sent by the computing equipment;
Deleting target data according to the deleting command or modifying the target data according to the modifying command, wherein the target data is part or all of second data;
and sending second information to the computing device, wherein the second information is used for indicating the computing device to delete the virtual disk.
10. The method of claim 9, wherein the second information is further used to indicate an amount of difference that the memory caused before and after executing the delete command or the change command.
11. The method of claim 10, wherein the step of determining the position of the first electrode is performed,
when the memory deletes the target data, the difference indicated by the second information is the difference between the third data amount before the target data is compressed and the fourth data amount after the target data is compressed; or alternatively, the process may be performed,
when the memory alters the target data, the difference indicated by the second information is the difference between the fifth data amount after the second data compression after the alteration and the sixth data amount after the second data compression.
12. The method of claim 11, wherein the virtual disk is mapped to a first physical space of the memory, the first physical space being used to store data of a magnitude of a difference between the fifth amount of data and the sixth amount of data;
After the sending of the second information to the computing device while the memory alters the target data, the method further comprises:
third information from the computing device is received, and data of the first physical space is migrated to a second physical space of the memory according to the third information.
13. An apparatus, the apparatus comprising:
a transmitting unit configured to transmit a write command and first data to a memory, the write command being configured to instruct the memory to write the first data;
a receiving unit configured to receive first information from the memory, the first information indicating an amount of difference between a first data amount before the first data compression and a second data amount after the first data compression;
and the processing unit is used for responding to the first information and creating a virtual disk for the memory.
14. The apparatus according to claim 13, wherein the processing unit is specifically configured to:
and creating the virtual disk for the memory according to the difference indicated by the first information.
15. The device according to claim 13 or 14, wherein,
The sending unit is further configured to send a deletion command or a change command to the memory, where the deletion command is used to instruct the memory to delete target data, the change command is used to instruct the memory to change the target data, and the target data is part or all of second data;
the receiving unit is further used for receiving second information from the memory;
the processing unit is further configured to delete the virtual disk in response to the second information.
16. The apparatus of claim 15, wherein the second information is used to indicate an amount of difference caused before and after the memory executes the delete command or the change command;
the processing unit is further configured to: marking virtual capacity equivalent to the difference indicated by the second information as unavailable in the virtual disk;
the processing unit is specifically configured to: and deleting the virtual disk according to the unavailable virtual capacity.
17. The apparatus of claim 16, wherein the device comprises a plurality of sensors,
when the deleting command indicates the memory to delete the target data, the difference indicated by the second information is the difference between the third data amount before the target data is compressed and the fourth data amount after the target data is compressed; or alternatively, the process may be performed,
When the change command instructs the memory to change the target data, the amount of difference indicated by the second information is an amount of difference between a fifth amount of data compressed by the changed second data and a sixth amount of data compressed by the second data.
18. The apparatus of claim 17, wherein the virtual disk is mapped to a first physical space of the memory, the first physical space being for storing data of a magnitude of a difference between the fifth amount of data and the sixth amount of data;
the transmitting unit is further configured to: and sending third information to the memory, wherein the third information is used for indicating the memory to migrate the data of the first physical space to a second physical space of the memory.
19. An apparatus, the apparatus comprising:
a receiving unit for receiving a write command and first data from a computing device;
the processing unit is used for compressing and storing the first data;
and the sending unit is used for sending first information to the computing equipment, the first information is used for indicating the computing equipment to create a virtual disk for the memory, and the first information is used for indicating the difference between the first data volume before the first data compression and the second data volume after the first data compression.
20. The apparatus according to claim 19, wherein the transmitting unit is specifically configured to: and sending the first information to the computing device, wherein the first information is used for indicating the computing device to create the virtual disk for the memory based on the difference indicated by the first information.
21. The device according to claim 19 or 20, wherein,
the receiving unit is further configured to receive a deletion command or a change command from the computing device;
the processing unit is further configured to delete target data according to the deletion command or change the target data according to the change command, where the target data is part or all of second data;
the sending unit is further configured to send second information to the computing device, where the second information is used to instruct the computing device to delete the virtual disk.
22. The apparatus of claim 21, wherein the second information is further used to indicate an amount of difference caused before and after the memory executes the delete command or the change command.
23. The apparatus of claim 22, wherein the device comprises a plurality of sensors,
when the memory deletes the target data, the difference indicated by the second information is the difference between the third data amount before the target data is compressed and the fourth data amount after the target data is compressed; or alternatively, the process may be performed,
When the memory alters the target data, the difference indicated by the second information is the difference between the fifth data amount after the second data compression after the alteration and the sixth data amount after the second data compression.
24. The apparatus of claim 23, wherein the virtual disk is mapped to a first physical space of the memory, the first physical space being for storing data of a magnitude of a difference between the fifth amount of data and the sixth amount of data;
when the memory alters the target data, the receiving unit is further configured to: receiving third information from the computing device; the processing unit is specifically configured to: and migrating the data of the first physical space to a second physical space of the memory according to the third information.
CN202010239040.7A 2020-03-30 2020-03-30 Method and device for improving available storage capacity Active CN113467699B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010239040.7A CN113467699B (en) 2020-03-30 2020-03-30 Method and device for improving available storage capacity

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010239040.7A CN113467699B (en) 2020-03-30 2020-03-30 Method and device for improving available storage capacity

Publications (2)

Publication Number Publication Date
CN113467699A CN113467699A (en) 2021-10-01
CN113467699B true CN113467699B (en) 2023-08-22

Family

ID=77865134

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010239040.7A Active CN113467699B (en) 2020-03-30 2020-03-30 Method and device for improving available storage capacity

Country Status (1)

Country Link
CN (1) CN113467699B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI768738B (en) * 2021-03-02 2022-06-21 群聯電子股份有限公司 Memory control method, memory storage device and memory control circuit unit

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FI20125779A (en) * 2012-07-05 2014-01-06 Blancco Oy Ltd Device, arrangement, method, and computer program for erasing data stored in mass storage
WO2017219364A1 (en) * 2016-06-24 2017-12-28 华为技术有限公司 Method for processing data, storage apparatus, solid state disk and storage system
CN108415669A (en) * 2018-03-15 2018-08-17 深信服科技股份有限公司 The data duplicate removal method and device of storage system, computer installation and storage medium
CN108427538A (en) * 2018-03-15 2018-08-21 深信服科技股份有限公司 Storage data compression method, device and the readable storage medium storing program for executing of full flash array

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2528585A (en) * 2013-05-17 2016-01-27 Hitachi Ltd Storage device
WO2015008375A1 (en) * 2013-07-19 2015-01-22 株式会社日立製作所 Storage device, and storage control method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FI20125779A (en) * 2012-07-05 2014-01-06 Blancco Oy Ltd Device, arrangement, method, and computer program for erasing data stored in mass storage
WO2017219364A1 (en) * 2016-06-24 2017-12-28 华为技术有限公司 Method for processing data, storage apparatus, solid state disk and storage system
CN109690681A (en) * 2016-06-24 2019-04-26 华为技术有限公司 Handle method, storage device, solid state hard disk and the storage system of data
CN108415669A (en) * 2018-03-15 2018-08-17 深信服科技股份有限公司 The data duplicate removal method and device of storage system, computer installation and storage medium
CN108427538A (en) * 2018-03-15 2018-08-21 深信服科技股份有限公司 Storage data compression method, device and the readable storage medium storing program for executing of full flash array

Also Published As

Publication number Publication date
CN113467699A (en) 2021-10-01

Similar Documents

Publication Publication Date Title
US20230259452A1 (en) Computing system and method for controlling storage device
CA3028821C (en) Data processing method, storage apparatus, solid state disk, and storage system
US8364929B2 (en) Enabling spanning for a storage device
US8601228B2 (en) Extended utilization area for a memory device
US11321229B2 (en) System controller and system garbage collection method
JP2019079113A (en) Storage device, data management method, and data management program
CN105094709A (en) Dynamic data compression method for solid-state disc storage system
CN111108488B (en) Memory block recovery method and device
CN113419975B (en) Control system of memory, address mapping method and address mapping device
WO2017132797A1 (en) Data arrangement method, storage apparatus, storage controller and storage array
CN115756312A (en) Data access system, data access method, and storage medium
CN107423425B (en) Method for quickly storing and inquiring data in K/V format
KR20220134769A (en) Temperature-based data storage processing
EP3798852B1 (en) System controller and system garbage recovery method
CN113467699B (en) Method and device for improving available storage capacity
US9524236B1 (en) Systems and methods for performing memory management based on data access properties
CN117075810A (en) ZNS solid state disk-based data management method
CN114003169B (en) Data compression method for SSD
CN111597128B (en) Method and device for managing data in solid-state nonvolatile storage device
WO2018142622A1 (en) Computer
CN113312275A (en) Data processing method, device and system of memory equipment
KR102557557B1 (en) Electronic device and computing system including same
KR102561316B1 (en) Electronic device and computing system including same
WO2024001863A1 (en) Data processing method and related device
CN117472285A (en) Intelligent operation acceleration method for solid state disk, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant