CN115878042A - Data storage system, data processing method and storage medium - Google Patents
Data storage system, data processing method and storage medium Download PDFInfo
- Publication number
- CN115878042A CN115878042A CN202211709814.3A CN202211709814A CN115878042A CN 115878042 A CN115878042 A CN 115878042A CN 202211709814 A CN202211709814 A CN 202211709814A CN 115878042 A CN115878042 A CN 115878042A
- Authority
- CN
- China
- Prior art keywords
- data
- storage subsystem
- hard disk
- disk drive
- index
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013500 data storage Methods 0.000 title claims abstract description 114
- 238000003672 processing method Methods 0.000 title claims abstract description 17
- 239000007787 solid Substances 0.000 claims abstract description 79
- 238000012545 processing Methods 0.000 claims description 66
- 238000013508 migration Methods 0.000 claims description 38
- 230000005012 migration Effects 0.000 claims description 38
- 238000000034 method Methods 0.000 claims description 37
- 230000008569 process Effects 0.000 claims description 23
- 230000004044 response Effects 0.000 claims description 11
- 238000012217 deletion Methods 0.000 claims description 8
- 230000037430 deletion Effects 0.000 claims description 8
- 238000013523 data management Methods 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 13
- 238000004891 communication Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a data storage system, a data processing method and a storage medium. The system comprises: the system comprises a copy solid state disk medium storage subsystem, a copy hard disk drive medium storage subsystem and an erasure code hard disk drive medium storage subsystem; the system comprises a copy solid state disk medium storage subsystem, a copy data storage subsystem and a copy data management subsystem, wherein the copy solid state disk medium storage subsystem is used for storing data in a solid state disk medium based on a preset first copy data redundancy strategy; the duplicate hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset second duplicate data redundancy strategy; and the erasure code hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset erasure code data redundancy strategy. The data storage system for realizing data layering, which is disclosed by the embodiment of the invention, combines different storage media and different data redundancy strategies, and has high performance and low cost.
Description
Technical Field
The embodiment of the invention relates to the technical field of data storage, in particular to a data storage system, a data processing method and a storage medium.
Background
In the big data era, data storage is of great importance, and a data storage system which can be used for realizing data layering has become the research and development focus of various industries at present.
However, the data storage systems developed at present cannot achieve both high performance and low cost, and need to be improved.
Disclosure of Invention
The embodiment of the invention provides a data storage system, a data processing method and a storage medium, which solve the problem that the data storage system cannot have high performance and low cost at the same time.
According to an aspect of the present invention, there is provided a data storage system, which may include: the system comprises a copy solid state disk medium storage subsystem, a copy hard disk drive medium storage subsystem and an erasure code hard disk drive medium storage subsystem;
the system comprises a copy solid state disk medium storage subsystem, a copy solid state disk medium storage subsystem and a data processing subsystem, wherein the copy solid state disk medium storage subsystem is used for storing data in a solid state disk medium based on a preset first copy data redundancy strategy;
the duplicate hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset second duplicate data redundancy strategy;
and the erasure code hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset erasure code data redundancy strategy.
According to another aspect of the present invention, there is provided a data processing method comprising:
responding to the data processing request, and processing to-be-processed data associated with a target storage subsystem in the data storage system aiming at the target storage subsystem;
the data storage system comprises a copy solid state disk medium storage subsystem, a copy hard disk drive medium storage subsystem and an erasure code hard disk drive medium storage subsystem, and the target storage subsystem comprises at least one of the copy solid state disk medium storage subsystem, the copy hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem;
the system comprises a copy solid state disk medium storage subsystem, a copy solid state disk medium storage subsystem and a data processing subsystem, wherein the copy solid state disk medium storage subsystem is used for storing data in a solid state disk medium based on a preset first copy data redundancy strategy;
the duplicate hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset second duplicate data redundancy strategy;
and the erasure code hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset erasure code data redundancy strategy.
According to another aspect of the present invention, there is provided a computer-readable storage medium having stored thereon computer instructions for causing a processor to execute a method of processing data provided by any of the embodiments of the present invention.
The data storage system described in the embodiments of the present invention includes: a duplicate SSD media storage subsystem, a duplicate HDD media storage subsystem and an erasure code HDD media storage subsystem; the system comprises a copy SSD medium storage subsystem and a data storage subsystem, wherein the copy SSD medium storage subsystem is used for storing data in an SSD medium based on a preset first copy data redundancy strategy; the duplicate HDD medium storage subsystem is used for storing data in the HDD medium based on a preset second duplicate data redundancy strategy; and the erasure code HDD medium storage subsystem is used for storing data in the HDD medium based on a preset erasure code data redundancy strategy. According to the data storage system, data layering is realized by utilizing the data storage subsystems with non-completely consistent storage media, and on the basis, the SSD media, the HDD media, the duplicate data redundancy strategy and the EC data redundancy strategy are combined, so that the data storage system has high performance and low cost.
It should be understood that the statements in this section are not intended to identify key or critical features of the embodiments of the present invention, nor are they intended to limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a block diagram of a data storage system provided in accordance with an embodiment of the present invention;
FIG. 2 is a block diagram of another data storage system provided in accordance with an embodiment of the present invention;
FIG. 3 is a block diagram of another data storage system provided in accordance with an embodiment of the present invention;
FIG. 4a is a schematic diagram of an example of uploading data in another data storage system provided in accordance with an embodiment of the present invention;
FIG. 4b is a diagram illustrating an example of data download in another data storage system provided in accordance with an embodiment of the present invention;
FIG. 4c is a schematic diagram of an example of data deletion in another data storage system provided in accordance with an embodiment of the present invention;
FIG. 5a is a schematic diagram of a first example of data migration in another data storage system provided in accordance with an embodiment of the present invention;
FIG. 5b is a block diagram illustrating a second example of data migration in another data storage system provided in accordance with an embodiment of the present invention;
FIG. 6 is a flow chart of a data processing method according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an access module for implementing the data processing method according to the embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. The cases of "target", "original", etc. are similar and will not be described in detail herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not expressly listed or inherent to such process, system, article, or apparatus.
Before the embodiment of the present invention is described, an application scenario of the embodiment of the present invention is exemplarily described: a Solid State Disk (SSD) medium has a higher (access) performance, so that a data storage system implemented based on the SSD medium can effectively ensure the access performance of data; hard Disk Drive (HDD) media have a low cost, and thus a data storage system implemented based on HDD media can effectively control the storage cost of data. In addition, in order to ensure the reliability of data, the data storage system may perform data redundancy based on a replica (e.g., multiple copies or double copies), that is, based on a replica data redundancy policy, or may perform data redundancy based on an Erasure-Code (EC), that is, based on an Erasure-Code data redundancy policy. The latter can achieve the same degree of data reliability with a lower number of copies than the former, but can carry a lower read load.
Therefore, the SSD medium, the HDD medium, the copy data redundancy strategy and the EC data redundancy strategy can be combined, so that a data storage system which has the characteristics of high performance (namely high access quality) and low cost is constructed. The data storage system will be described in detail below.
Fig. 1 is a block diagram of a data storage system provided in an embodiment of the present invention. The embodiment can be applied to the situation that the data storage system with the characteristics of high performance and low cost is constructed by combining the SSD medium, the HDD medium, the copy data redundancy strategy and the EC data redundancy strategy, and is particularly applied to the situation that the constructed data storage system can be used for realizing data layering.
As shown in fig. 1, a data storage system 1 according to an embodiment of the present invention includes: a duplicate solid state disk medium storage subsystem 10, a duplicate hard disk drive medium storage subsystem 11 and an erasure code hard disk drive medium storage subsystem 12; wherein,
the duplicate solid state disk medium storage subsystem 10 may be configured to perform data storage in a solid state disk medium based on a preset first duplicate data redundancy policy;
a duplicate hard disk drive media storage subsystem 11, operable to store data in a hard disk drive media based on a preset second duplicate data redundancy policy;
an erasure coding hard drive media storage subsystem 12 is operable to store data in a hard drive media based on a preset erasure coding data redundancy policy.
For the replica solid state disk medium storage subsystem 10, the SSD medium is used and the first replica data redundancy policy is used for data redundancy, so that the access performance is optimal.
For the duplicate hard disk drive media storage subsystem 11, which employs HDD media and uses a second duplicate data policy for data redundancy, both access performance and cost are intermediate between those of the duplicate SSD media storage subsystem 10 and the EC HDD media storage subsystem 12. It should be noted that the first duplicate data redundancy policy and the second duplicate data redundancy policy may be the same duplicate data redundancy policy, or may be different duplicate data redundancy policies, for example, the number of copies redundant by the first duplicate data redundancy policy and the second duplicate data redundancy policy are different, which may be set according to actual situations, and is not specifically limited herein.
For the erasure code hard disk drive medium storage subsystem 12, the HDD medium and EC data redundancy strategy are adopted for data redundancy, which has the lowest cost, but can still maintain good data reliability, and the access performance is relatively common.
On this basis, for convenience of description, the above-mentioned duplicate SSD media storage subsystem 10, duplicate HDD media storage subsystem 11, and EC HDD media storage subsystem 12 may be referred to as data storage subsystems, that is, the above-mentioned data storage system 1 includes three data storage subsystems.
It should be noted that, firstly, the three data storage subsystems in the data storage system 1 are not completely consistent on the storage medium, the storage medium of some data storage subsystems, such as the replica SSD media storage subsystem 10, is the SSD medium, and the storage medium of some data storage subsystems, such as the replica HDD media storage subsystem 11 and the EC HDD media storage subsystem 12, is the HDD medium, so that data layering can be performed between the three data storage subsystems, that is, data transfer can be performed between the data storage subsystems having different storage media; secondly, the data storage system 1 can fully exert the advantages of the SSD media, the HDD media, the copy data redundancy policy, and the EC data redundancy policy, thereby having high performance and low cost; thirdly, combining the first two points, the data storage system 1 has lower cost compared with a data layering scheme of a data storage system based on different storage media of copies, has higher performance compared with a data layering scheme of a data storage system based on different storage media of EC, and can better exert the advantages of a high-performance storage system and a low-cost storage system compared with a two-stage data layering scheme realized based on an SSD medium and an HDD medium; fourth, the data storage system 1 described above is a multi-level heterogeneous storage system, where multi-level is understood to mean that it has different levels of data storage subsystems, and heterogeneous is understood to mean that it has different storage media and data redundancy strategies, so that data layering can be performed between the multi-level heterogeneous software and hardware data storage subsystems.
The data storage system described in the embodiments of the present invention includes: a duplicate SSD media storage subsystem, a duplicate HDD media storage subsystem and an erasure code HDD media storage subsystem; the system comprises a copy SSD medium storage subsystem and a data storage subsystem, wherein the copy SSD medium storage subsystem is used for storing data in an SSD medium based on a preset first copy data redundancy strategy; the duplicate HDD medium storage subsystem is used for storing data in the HDD medium based on a preset second duplicate data redundancy strategy; and the erasure code HDD medium storage subsystem is used for storing data in the HDD medium based on a preset erasure code data redundancy strategy. According to the data storage system, data layering is realized by utilizing the data storage subsystems with non-completely consistent storage media, and on the basis, the SSD media, the HDD media, the duplicate data redundancy strategy and the EC data redundancy strategy are combined, so that the data storage system has high performance and low cost.
Fig. 2 is a block diagram of another data storage system provided in an embodiment of the present invention. The present embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the data storage system further includes: accessing a module; the access module is used for responding to a data processing request and processing to-be-processed data associated with the target storage subsystem; the target storage subsystem comprises at least one of a copy solid state disk medium storage subsystem, a copy hard disk drive medium storage subsystem and an erasure code hard disk drive medium storage subsystem. The same or corresponding terms as those in the above embodiments are not explained in detail herein.
Specifically, referring to fig. 2, the data storage system 2 according to this embodiment includes: a duplicate solid state disk medium storage subsystem 20, a duplicate hard disk drive medium storage subsystem 21, an erasure code hard disk drive medium storage subsystem 22 and an access module 23; wherein,
the duplicate solid state disk medium storage subsystem 20 may be configured to store data in the solid state disk medium based on a preset first duplicate data redundancy policy;
a duplicate hard disk drive media storage subsystem 21 operable to store data in a hard disk drive media based on a preset second duplicate data redundancy policy;
an erasure code hard disk drive media storage subsystem 22 operable to store data in a hard disk drive media based on a preset erasure code data redundancy policy;
and an access module 23, configured to process data to be processed associated with a target storage subsystem in response to the data processing request, where the target storage subsystem includes at least one of the replica solid state disk medium storage subsystem 20, the replica hard disk drive medium storage subsystem 21, and the erasure code hard disk drive medium storage subsystem 22.
The access module 23 provides a uniform access entry and a uniform access mode for the three storage subsystems, and any object needing to access any storage subsystem in the three storage subsystems can be accessed through the access module 23, so that the access entry and the access mode do not need to be set for each storage subsystem, and the access module has better convenience. The object may be a subsystem in the data storage system 2, or may be an object outside the data storage system 2, which is not specifically limited herein.
Specifically, the data processing request may be understood as a request for processing to-be-processed data associated with the target storage subsystem, and a specific data processing manner may be data uploading, data downloading, data deleting, data transferring, or the like, which is not specifically limited herein. The target storage subsystem may be at least one of the three data storage subsystems described above, which may be determined based on a particular data processing request. The data to be processed may be data already stored in the target storage subsystem, or may also be data that needs to be stored in the target storage subsystem, which is related to the actual situation, and is not specifically limited herein. The access module 23 processes the data to be processed associated with the target storage subsystem in response to the data processing request, thereby implementing a data processing procedure related to the target storage subsystem.
The data storage system of the embodiment of the invention also comprises an access module used for providing a uniform access entry and a uniform access mode for each data storage subsystem, so that the data to be processed related to the target storage subsystem can be processed by responding to the data processing request through the access module, and the data processing process related to the target storage subsystem is realized.
Fig. 3 is a block diagram of another data storage system provided in an embodiment of the present invention. The present embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the data storage system may further include: an index storage subsystem; the index storage subsystem is used for storing indexes of data to be processed; and the access module is also used for processing the index of the data to be processed. The explanations of the same or corresponding terms as those in the above embodiments are omitted.
Specifically, referring to fig. 3, the data storage system 3 according to this embodiment includes: a duplicate solid state disk medium storage subsystem 30, a duplicate hard disk drive medium storage subsystem 31, an erasure code hard disk drive medium storage subsystem 32, an access module 33 and an index storage subsystem 34; wherein,
the duplicate solid state disk medium storage subsystem 30 may be configured to perform data storage in the solid state disk medium based on a preset first duplicate data redundancy policy;
a duplicate hard disk drive media storage subsystem 31 operable to store data in a hard disk drive media based on a preset second duplicate data redundancy policy;
an erasure code hard disk drive media storage subsystem 32 operable to store data in a hard disk drive media based on a preset erasure code data redundancy policy;
the index storage subsystem 34 is used for storing an index of the data to be processed, and the index of the data to be processed is used for representing the storage position of the data to be processed in the data storage system;
and the access module 33 is configured to, in response to the data processing request, process to-be-processed data associated with the target storage subsystem, and process an index of the to-be-processed data, where the target storage subsystem includes at least one of the replica solid state disk medium storage subsystem 30, the replica hard disk drive medium storage subsystem 31, and the erasure code hard disk drive medium storage subsystem 32.
Wherein, for the index storage subsystem 34, it can be used to store an index of the data to be processed, and the index can be used to characterize the storage location of the data to be processed in the data storage system 3, such as a duplicate SSD media storage subsystem, a duplicate HDD media storage subsystem, or an EC HDD media storage subsystem; besides, the method can be used for characterizing metadata information required when the data to be processed is accessed. In practical applications, optionally, the index storage subsystem 34 may be used to store, in addition to the index of the data to be processed, an index of stored data that is currently stored in each data storage system, which is not specifically limited herein. And the index storage subsystem 34 is used for facilitating the access module 33 to quickly read the data to be processed.
In order to ensure effective reading of the data to be processed, when responding to the data processing request, the access module 33 needs to process the data to be processed, in addition to the data to be processed, the index of the data to be processed stored in the index storage subsystem 34, so as to facilitate the subsequent reading process of the data to be processed.
The data storage system of the embodiment of the invention further comprises an index storage subsystem for storing the index of the data to be processed, so that the access module can process the index of the data to be processed when processing the data to be processed, thereby facilitating the subsequent reading process of the data to be processed.
On the basis, an optional technical scheme is that the data processing request comprises a data uploading request, the data to be processed comprises the data to be uploaded, and the target storage subsystem comprises a copy solid state disk medium storage subsystem; the access module is specifically used for responding to a data uploading request, acquiring data to be uploaded, writing the data to be uploaded into the replica solid state disk medium storage subsystem, and writing an index of the data to be uploaded determined according to the replica solid state disk medium storage subsystem into the index storage subsystem. In other words, the data to be uploaded is written into the copy SSD medium storage subsystem, so that high-performance access to the data to be uploaded is facilitated; and moreover, the index of the data to be uploaded for representing the copy SSD medium storage subsystem is written into the index storage subsystem, so that the data to be uploaded can be conveniently read from the copy SSD medium storage subsystem according to the index. For example, as shown in fig. 4a, when the access module responds to a data upload request, the access module may first write data to be uploaded into the high-performance replica SSD media storage subsystem, and then write an index of the data to be uploaded into the index storage subsystem, thereby completing the data upload process.
Another optional technical solution is that the data processing request includes a data downloading request, and the data to be processed includes data to be downloaded; the access module is specifically used for reading an index of data to be downloaded from the index storage subsystem in response to a data downloading request, determining a target storage subsystem for storing the data to be downloaded from the replica solid state hard disk medium storage subsystem, the replica hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem according to the index of the data to be downloaded, and downloading the data to be downloaded from the target storage subsystem. In other words, the target storage subsystem storing the data to be downloaded may be determined from the three data storage subsystems according to the index of the data to be downloaded read from the index storage subsystem, and then the data to be downloaded may be downloaded from the target storage subsystem. For example, as shown in fig. 4b, when the access module responds to the data download request, the access module may first read the index of the data to be downloaded from the index storage subsystem, and then read the data to be downloaded from a specific data storage subsystem (i.e., the target storage subsystem) according to the index, thereby completing the data download process.
In another optional technical solution, the data processing request includes a data deletion request, and the data to be processed includes data to be deleted; the access module is specifically configured to delete an index of data to be deleted from the index storage subsystem in response to a data deletion request, determine a target storage subsystem storing the data to be deleted from the replica solid state disk medium storage subsystem, the replica hard disk drive medium storage subsystem, and the erasure code hard disk drive medium storage subsystem according to the index of the data to be deleted, and delete the data to be deleted from the target storage subsystem. In other words, the index of the data to be deleted may be deleted from the index storage subsystem, and the target storage subsystem storing the data to be deleted may be determined from the three data storage subsystems according to the index of the data to be deleted acquired before deletion, and then the data to be deleted may be deleted from the target storage subsystem. For example, as shown in fig. 4c, when the access module responds to the data deletion request, the access module may delete the index of the data to be deleted stored in the index storage subsystem, and then asynchronously delete the data to be deleted from the specific data storage subsystem (i.e., the target storage subsystem) according to the index, thereby completing the data deletion process.
The present embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the data processing request includes a data migration request, the target storage subsystem includes a source storage subsystem and a destination storage subsystem, and the to-be-processed data includes to-be-migrated data in the source storage subsystem; the access module is specifically used for determining a source storage subsystem and a target storage subsystem from the replica solid state disk medium storage subsystem, the replica hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem in response to a data migration request, migrating the data to be migrated in the source storage subsystem to the target storage subsystem, and modifying the index stored in the index storage subsystem based on the target storage subsystem. The same or corresponding terms as those in the above embodiments are not explained in detail herein.
Specifically, the data storage system according to this embodiment includes: the system comprises a copy solid state disk medium storage subsystem, a copy hard disk drive medium storage subsystem, an erasure code hard disk drive medium storage subsystem, an access module and an index storage subsystem; wherein,
the system comprises a copy solid state disk medium storage subsystem, a copy solid state disk medium storage subsystem and a data processing subsystem, wherein the copy solid state disk medium storage subsystem is used for storing data in a solid state disk medium based on a preset first copy data redundancy strategy;
the duplicate hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset second duplicate data redundancy strategy;
the erasure code hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset erasure code data redundancy strategy;
the index storage subsystem is used for storing the index of the data to be processed, and the index of the data to be processed is used for representing the storage position of the data to be processed in the data storage system;
the access module is used for responding to a data migration request, determining a source storage subsystem and a target storage subsystem from the copy solid state hard disk medium storage subsystem, the copy hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem, migrating the data to be migrated in the source storage subsystem to the target storage subsystem, and modifying the index stored in the index storage subsystem based on the target storage subsystem.
The data migration request may be understood as a request for migrating data to be migrated in a source storage subsystem to a destination storage subsystem, where the source storage subsystem and the destination storage subsystem are both derived from three data storage subsystems. Therefore, the access module can determine a source storage subsystem and a destination storage subsystem from the three data storage subsystems by responding to the data migration request, migrate the data to be migrated in the source storage subsystem to the destination storage subsystem, and modify the index stored in the index storage subsystem based on the destination storage subsystem, so as to facilitate subsequent reading of the data to be migrated from the destination storage subsystem.
According to the technical scheme of the embodiment of the invention, the access module can determine the source storage subsystem and the target storage subsystem from the three data storage subsystems by responding to the data migration request, then migrate the data to be migrated in the source storage subsystem to the target storage subsystem, and modify the index stored in the index storage subsystem based on the target storage subsystem so as to facilitate the subsequent reading of the data to be migrated from the target storage subsystem, thereby completing the data migration process.
On this basis, an optional technical solution is that the access module is specifically configured to: responding to a data migration request, reading an index of data to be migrated from an index storage subsystem, and determining a source storage subsystem for storing the data to be migrated from a copy solid state hard disk medium storage subsystem, a copy hard disk drive medium storage subsystem and an erasure code hard disk drive medium storage subsystem according to the index of the data to be migrated;
determining a target storage subsystem from the erasure code hard disk drive medium storage subsystem, the replica solid state hard disk medium storage subsystem and the replica hard disk drive medium storage subsystem;
reading data to be migrated from the source storage subsystem, writing the read data to be migrated into the target storage subsystem, modifying the index stored in the index storage subsystem based on the target storage subsystem, and deleting the data to be migrated in the source storage subsystem.
In other words, according to the index of the data to be migrated read from the index storage subsystem, the storage location of the data to be migrated in the data storage system may be determined, and then the source storage subsystem storing the data to be migrated may be determined from the three data storage subsystems according to the storage location. Further, after the target storage subsystem is determined from the three data storage subsystems, the data to be migrated can be read from the source storage subsystem, the read data to be migrated is written into the target storage subsystem, then the index stored in the index storage subsystem is modified based on the target storage subsystem, and the data to be migrated in the source storage subsystem is deleted, so that the subsequent reading process of the data to be migrated is only carried out in the target storage subsystem.
For example, as shown in fig. 5a, when the access module responds to the data migration request, the access module may first read the index of the data to be migrated from the index storage subsystem, then read the data to be migrated from the source storage subsystem according to the read index, and write the read data to be migrated into the destination storage subsystem. Further, the index stored in the index storage subsystem is modified into the target storage subsystem, and then the data to be migrated in the source storage subsystem is deleted, so that the data migration process is completed.
Another optional technical solution is that the data storage system further includes: the system comprises an acquisition subsystem and a data migration subsystem; wherein,
the acquisition subsystem is used for acquiring access frequency information of stored data respectively stored in the copy solid state hard disk medium storage subsystem, the copy hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem, wherein the stored data comprise data to be migrated;
and the data migration subsystem is used for pulling the access frequency information from the acquisition subsystem and generating a data migration request according to the access frequency information.
The access frequency information of the stored data stored in each data storage subsystem can be counted in each data storage subsystem, so that the access frequency information of the stored data stored in each data storage subsystem can be acquired from each data storage subsystem through the acquisition subsystem. On the basis, the access frequency information of the stored data can be pulled from the acquisition subsystem through the data migration subsystem, then the data to be migrated with the data migration requirement in the stored data is determined according to the access frequency information, so that a corresponding data migration request is generated, and the data migration request is sent to the access module, so that the access module can realize the migration process of the data to be migrated by responding to the data migration request.
Illustratively, referring to fig. 5b, the data storage system of the present example includes: the system comprises a copy SSD medium storage subsystem, a copy HDD medium storage subsystem, an EC HDD medium storage subsystem, an access module, an index storage subsystem, an acquisition subsystem and a data migration subsystem; wherein,
the system comprises a copy SSD medium storage subsystem and a data storage subsystem, wherein the copy SSD medium storage subsystem is used for storing data in an SSD medium based on a preset first copy data redundancy strategy;
the duplicate HDD medium storage subsystem is used for storing data in the HDD medium based on a preset second duplicate data redundancy strategy;
the EC HDD medium storage subsystem is used for storing data in the HDD medium based on a preset EC data redundancy strategy;
the acquisition subsystem is used for acquiring access frequency information of stored data stored in the copy SSD medium storage subsystem, the copy HDD medium storage subsystem and the ECHDD medium storage subsystem respectively;
the data migration subsystem is used for pulling the access frequency information from the acquisition subsystem and generating a data migration request according to the access frequency information;
the index storage subsystem is used for storing an index of the data to be migrated, and the index of the data to be migrated is used for representing the storage position of the data to be migrated in the data storage system;
the access module is used for responding to a data migration request, determining a source storage subsystem and a target storage subsystem from the copy SSD medium storage subsystem, the copy HDD medium storage subsystem and the ECHDD medium storage subsystem, migrating data to be migrated in the source storage subsystem to the target storage subsystem, and modifying an index stored in the index storage subsystem based on the target storage subsystem.
On the basis, optionally, the target storage subsystem corresponding to the stored data with the access frequency information being greater than or equal to the first preset threshold value comprises a copy solid state disk medium storage subsystem; and/or the presence of a gas in the gas,
the target storage subsystem corresponding to the stored data with the access frequency information smaller than a first preset threshold and larger than or equal to a second preset threshold comprises a duplicate hard disk drive medium storage subsystem; and/or the presence of a gas in the atmosphere,
and the destination storage subsystem corresponding to the stored data with the access frequency information smaller than the second preset threshold value comprises an erasure code hard disk drive medium storage subsystem.
In other words, data to be migrated with high frequency of access can be migrated to the replica SSD media storage subsystem, data to be migrated with lower frequency of access can be migrated to the replica HDD media storage subsystem, and data to be migrated with little access can be migrated to the EC HDD media storage subsystem, thereby ensuring that stored data with frequent access has higher access performance and stored data with little access has lower storage cost.
It should be noted that, in the embodiment of the data storage system, the subsystems and modules included in the embodiment are only divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
Fig. 6 is a flowchart of a data processing method provided in an embodiment of the present invention. The present embodiment is applicable to a case where data processing is performed in a data storage system having both high performance and low cost. The method can be executed by the access module provided by the embodiment of the invention, and the module can be realized by software and/or hardware.
Referring to fig. 6, the method of the embodiment of the present invention specifically includes the following steps:
s510, responding to a data processing request, and processing to-be-processed data associated with a target storage subsystem in the data storage system aiming at the target storage subsystem;
the data storage system comprises a copy solid state disk medium storage subsystem, a copy hard disk drive medium storage subsystem and an erasure code hard disk drive medium storage subsystem, and the target storage subsystem comprises at least one of the copy solid state disk medium storage subsystem, the copy hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem;
the system comprises a copy solid state disk medium storage subsystem, a copy solid state disk medium storage subsystem and a data processing subsystem, wherein the copy solid state disk medium storage subsystem is used for storing data in a solid state disk medium based on a preset first copy data redundancy strategy;
the duplicate hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset second duplicate data redundancy strategy;
and the erasure code hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset erasure code data redundancy strategy.
The specific implementation process of the above technical solution has already been described above, and is not described herein again.
According to the technical scheme of the embodiment of the invention, the data to be processed associated with the target storage subsystem is processed aiming at the target storage subsystem in the data storage system by responding to the data processing request, so that the data processing process in the data storage system with the characteristics of high performance and low cost is realized.
According to an optional technical scheme, the data storage system further comprises an index storage subsystem, wherein the index storage subsystem is used for storing an index of the data to be processed, and the index of the data to be processed is used for representing a storage position of the data to be processed in the data storage system;
the data processing method further comprises the following steps: and processing the index of the data to be processed.
On the basis, an optional technical scheme is that the data processing request comprises a data uploading request, the data to be processed comprises the data to be uploaded, and the target storage subsystem comprises a copy solid state disk medium storage subsystem;
responding to the data processing request, aiming at a target storage subsystem in the data storage system, processing data to be processed associated with the target storage subsystem, and processing an index of the data to be processed, wherein the processing comprises the following steps:
and responding to the data uploading request, acquiring data to be uploaded, writing the data to be uploaded into the copy solid state disk medium storage subsystem, and writing the index of the data to be uploaded into the index storage subsystem.
Another optional technical solution is that the data processing request includes a data downloading request, and the data to be processed includes data to be downloaded;
responding to the data processing request, aiming at a target storage subsystem in the data storage system, processing data to be processed associated with the target storage subsystem, and processing an index of the data to be processed, wherein the processing comprises the following steps:
and responding to a data downloading request, reading the index of the data to be downloaded from the index storage subsystem, determining a target storage subsystem for storing the data to be downloaded from the copy solid state hard disk medium storage subsystem, the copy hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem according to the index of the data to be downloaded, and downloading the data to be downloaded from the target storage subsystem.
According to another optional technical scheme, the data processing request comprises a data deleting request, and the data to be processed comprises the data to be deleted; wherein,
responding to the data processing request, aiming at a target storage subsystem in the data storage system, processing data to be processed associated with the target storage subsystem, and processing an index of the data to be processed, wherein the processing comprises the following steps:
and in response to the data deletion request, deleting the index of the data to be deleted from the index storage subsystem, determining a target storage subsystem for storing the data to be deleted from the duplicate solid state disk medium storage subsystem, the duplicate hard disk drive medium storage subsystem and the erasure correcting code hard disk drive medium storage subsystem according to the index of the data to be deleted, and deleting the data to be deleted from the target storage subsystem.
According to another optional technical scheme, the data processing request may include a data migration request, the target storage subsystem includes a source storage subsystem and a destination storage subsystem, and the data to be processed includes the data to be migrated in the source storage subsystem;
responding to the data processing request, aiming at a target storage subsystem in the data storage system, processing data to be processed associated with the target storage subsystem, and processing an index of the data to be processed, wherein the processing comprises the following steps:
responding to a data migration request, determining a source storage subsystem and a target storage subsystem from the copy solid state hard disk medium storage subsystem, the copy hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem, migrating data to be migrated in the source storage subsystem to the target storage subsystem, and modifying an index stored in the index storage subsystem based on the target storage subsystem.
On this basis, an optional method for determining a source storage subsystem and a destination storage subsystem from the replica solid state disk medium storage subsystem, the replica hard disk drive medium storage subsystem, and the erasure code hard disk drive medium storage subsystem, migrating data to be migrated in the source storage subsystem to the destination storage subsystem, and modifying an index stored in the index storage subsystem based on the destination storage subsystem includes:
reading an index of data to be migrated from the index storage subsystem, and determining a source storage subsystem for storing the data to be migrated from the duplicate solid state disk medium storage subsystem, the duplicate hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem according to the index of the data to be migrated;
determining a target storage subsystem from the erasure code hard disk drive medium storage subsystem, the replica solid state hard disk medium storage subsystem and the replica hard disk drive medium storage subsystem;
reading data to be migrated from the source storage subsystem, writing the read data to be migrated into the target storage subsystem, modifying the index stored in the index storage subsystem based on the target storage subsystem, and deleting the data to be migrated in the source storage subsystem.
Optionally, the data storage system further includes: the system comprises an acquisition subsystem and a data migration subsystem; the data processing method further includes:
receiving a data migration request sent by a data migration subsystem;
the data migration request is generated by the data migration subsystem according to access frequency information pulled from the acquisition subsystem, and the access frequency information is obtained by acquiring access frequency information of stored data respectively stored in the duplicate solid state hard disk medium storage subsystem, the duplicate hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem by the acquisition subsystem.
On the basis, optionally, the target storage subsystem corresponding to the stored data with the access frequency information being greater than or equal to the first preset threshold value comprises a copy solid state disk medium storage subsystem; and/or the presence of a gas in the gas,
the target storage subsystem corresponding to the stored data with the access frequency information being less than a first preset threshold and greater than or equal to a second preset threshold comprises a duplicate hard disk drive medium storage subsystem; and/or the presence of a gas in the gas,
and the destination storage subsystem corresponding to the stored data with the access frequency information smaller than the second preset threshold value comprises an erasure code hard disk drive medium storage subsystem.
The specific implementation process of each of the above optional technical solutions has been described in detail above, and is not described herein again.
Fig. 7 shows a schematic structural diagram of an access module 60 that may be used to implement an embodiment of the invention. The access module is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The access module may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 7, the access module 60 includes at least one processor 61, and a memory communicatively connected to the at least one processor 61, such as a Read Only Memory (ROM) 62, a Random Access Memory (RAM) 63, etc., wherein the memory stores computer programs executable by the at least one processor, and the processor 61 may perform various suitable actions and processes according to the computer programs stored in the Read Only Memory (ROM) 62 or the computer programs loaded from the storage unit 68 into the Random Access Memory (RAM) 63. In the RAM 63, various programs and data required for the operation of the access module 60 can also be stored. The processor 61, the ROM 62, and the RAM 63 are connected to each other by a bus 64. An input/output (I/O) interface 65 is also connected to bus 64.
A number of components in the access module 60 are connected to the I/O interface 65, including: an input unit 66 such as a keyboard, a mouse, or the like; an output unit 67 such as various types of displays, speakers, and the like; a storage unit 68 such as a magnetic disk, optical disk, or the like; and a communication unit 69 such as a network card, modem, wireless communication transceiver, etc. The communication unit 69 allows the access module 60 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
In some embodiments, the data processing method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 68. In some embodiments, part or all of the computer program may be loaded and/or installed onto the access module 60 via the ROM 62 and/or the communication unit 69. When the computer program is loaded into the RAM 63 and executed by the processor 61, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the processor 61 may be configured to perform the data processing method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on a machine, as a stand-alone software package partly on a machine and partly on a remote machine or entirely on a remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described herein may be implemented on an access module having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the access module. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome.
It should be understood that various forms of the flows shown above, reordering, adding or deleting steps, may be used. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (13)
1. A data storage system, comprising: the system comprises a copy solid state disk medium storage subsystem, a copy hard disk drive medium storage subsystem and an erasure code hard disk drive medium storage subsystem;
the copy solid state disk medium storage subsystem is used for storing data in a solid state disk medium based on a preset first copy data redundancy strategy;
the duplicate hard disk drive medium storage subsystem is used for storing data in a hard disk drive medium based on a preset second duplicate data redundancy strategy;
and the erasure code hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset erasure code data redundancy strategy.
2. The system of claim 1, further comprising: an access module;
the access module is used for responding to a data processing request and processing to-be-processed data associated with a target storage subsystem;
wherein the target storage subsystem comprises at least one of the replica solid state hard drive media storage subsystem, the replica hard drive media storage subsystem, and the erasure code hard drive media storage subsystem.
3. The system of claim 2, further comprising: an index storage subsystem;
the index storage subsystem is used for storing the index of the data to be processed, and the index of the data to be processed is used for representing the storage position of the data to be processed in the data storage system;
the access module is further configured to process the index of the data to be processed.
4. The system of claim 3, wherein the data processing request comprises a data upload request, the data to be processed comprises data to be uploaded, and the target storage subsystem comprises the replica solid state disk media storage subsystem; wherein,
the access module is specifically configured to, in response to the data upload request, acquire the data to be uploaded, write the data to be uploaded into the replica solid state disk medium storage subsystem, and write an index of the data to be uploaded, which is determined according to the replica solid state disk medium storage subsystem, into the index storage subsystem.
5. The system of claim 3, wherein the data processing request comprises a data download request, and the data to be processed comprises data to be downloaded; wherein,
the access module is specifically configured to, in response to the data download request, read an index of the data to be downloaded from the index storage subsystem, determine, according to the index of the data to be downloaded, the target storage subsystem in which the data to be downloaded is stored from the replica solid state disk medium storage subsystem, the replica hard disk drive medium storage subsystem, and the erasure code hard disk drive medium storage subsystem, and download the data to be downloaded from the target storage subsystem.
6. The system of claim 3, wherein the data processing request comprises a data delete request, and the data to be processed comprises data to be deleted; wherein,
the access module is specifically configured to delete, in response to the data deletion request, the index of the data to be deleted from the index storage subsystem, determine, according to the index of the data to be deleted, the target storage subsystem in which the data to be deleted is stored from the replica solid state disk medium storage subsystem, the replica hard disk drive medium storage subsystem, and the erasure code hard disk drive medium storage subsystem, and delete the data to be deleted from the target storage subsystem.
7. The system of claim 3, wherein the data processing request comprises a data migration request, the target storage subsystem comprises a source storage subsystem and a destination storage subsystem, and the data to be processed comprises data to be migrated in the source storage subsystem; wherein,
the access module is specifically configured to determine, in response to the data migration request, the source storage subsystem and the destination storage subsystem from the replica solid state disk medium storage subsystem, the replica hard disk drive medium storage subsystem, and the erasure code hard disk drive medium storage subsystem, migrate data to be migrated in the source storage subsystem to the destination storage subsystem, and modify an index stored in the index storage subsystem based on the destination storage subsystem.
8. The system of claim 7, wherein the access module is specifically configured to:
responding to the data migration request, reading the index of the data to be migrated from the index storage subsystem, and determining the source storage subsystem storing the data to be migrated from the copy solid state hard disk medium storage subsystem, the copy hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem according to the index of the data to be migrated;
determining the target storage subsystem from the erasure code hard disk drive media storage subsystem, the replica solid state hard disk media storage subsystem, and the replica hard disk drive media storage subsystem;
reading the data to be migrated from the source storage subsystem, writing the read data to be migrated into the target storage subsystem, modifying the index stored in the index storage subsystem based on the target storage subsystem, and deleting the data to be migrated in the source storage subsystem.
9. The system of claim 7, further comprising: the system comprises an acquisition subsystem and a data migration subsystem; wherein,
the acquisition subsystem is used for acquiring access frequency information of stored data respectively stored in the copy solid state hard disk medium storage subsystem, the copy hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem, wherein the stored data comprise the data to be migrated;
and the data migration subsystem is used for pulling the access frequency information from the acquisition subsystem and generating the data migration request according to the access frequency information.
10. The system of claim 9, wherein:
the target storage subsystem corresponding to the stored data of which the access frequency information is greater than or equal to a first preset threshold comprises the copy solid state disk medium storage subsystem; and/or the presence of a gas in the gas,
the target storage subsystem corresponding to the stored data of which the access frequency information is less than the first preset threshold and greater than or equal to a second preset threshold comprises the duplicate hard disk drive medium storage subsystem; and/or the presence of a gas in the atmosphere,
the destination storage subsystem corresponding to the stored data having the access frequency information less than the second preset threshold includes the erasure code hard disk drive media storage subsystem.
11. A data processing method, comprising:
responding to a data processing request, and aiming at a target storage subsystem in a data storage system, processing to-be-processed data associated with the target storage subsystem;
the data storage system comprises a duplicate solid state disk medium storage subsystem, a duplicate hard disk drive medium storage subsystem and an erasure code hard disk drive medium storage subsystem, and the target storage subsystem comprises at least one of the duplicate solid state disk medium storage subsystem, the duplicate hard disk drive medium storage subsystem and the erasure code hard disk drive medium storage subsystem;
the copy solid state disk medium storage subsystem is used for storing data in a solid state disk medium based on a preset first copy data redundancy strategy;
the duplicate hard disk drive medium storage subsystem is used for storing data in a hard disk drive medium based on a preset second duplicate data redundancy strategy;
and the erasure code hard disk drive medium storage subsystem is used for storing data in the hard disk drive medium based on a preset erasure code data redundancy strategy.
12. The method of claim 11, wherein the data storage system further comprises an index storage subsystem for storing an index of the data to be processed, wherein the index of the data to be processed is used to characterize a storage location of the data to be processed within the data storage system;
the method further comprises the following steps:
and processing the index of the data to be processed.
13. A computer-readable storage medium, characterized in that it stores computer instructions for causing a processor to implement a data processing method as claimed in claim 11 or 12 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211709814.3A CN115878042A (en) | 2022-12-29 | 2022-12-29 | Data storage system, data processing method and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211709814.3A CN115878042A (en) | 2022-12-29 | 2022-12-29 | Data storage system, data processing method and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115878042A true CN115878042A (en) | 2023-03-31 |
Family
ID=85757201
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211709814.3A Pending CN115878042A (en) | 2022-12-29 | 2022-12-29 | Data storage system, data processing method and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115878042A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117420969A (en) * | 2023-12-19 | 2024-01-19 | 中电云计算技术有限公司 | Distributed data storage method, device, equipment and storage medium |
-
2022
- 2022-12-29 CN CN202211709814.3A patent/CN115878042A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117420969A (en) * | 2023-12-19 | 2024-01-19 | 中电云计算技术有限公司 | Distributed data storage method, device, equipment and storage medium |
| CN117420969B (en) * | 2023-12-19 | 2024-04-16 | 中电云计算技术有限公司 | Distributed data storage method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9811424B2 (en) | Optimizing restoration of deduplicated data | |
| US9116936B2 (en) | Inline learning-based selective deduplication for primary storage systems | |
| CN108629029B (en) | Data processing method and device applied to data warehouse | |
| CN111309732B (en) | Data processing method, device, medium and computing equipment | |
| US9811577B2 (en) | Asynchronous data replication using an external buffer table | |
| US8843716B2 (en) | Computer system, storage apparatus and data transfer method | |
| US8495166B2 (en) | Optimized caching for large data requests | |
| US8386717B1 (en) | Method and apparatus to free up cache memory space with a pseudo least recently used scheme | |
| US10108644B1 (en) | Method for minimizing storage requirements on fast/expensive arrays for data mobility and migration | |
| US20130080397A1 (en) | Database restore using incremental backups in reverse order | |
| US10394819B2 (en) | Controlling mirroring of tables based on access prediction | |
| JP6774971B2 (en) | Data access accelerator | |
| CN113364877B (en) | Data processing method, device, electronic equipment and medium | |
| CN113961510B (en) | File processing method, device, equipment and storage medium | |
| US11500577B2 (en) | Method, electronic device, and computer program product for data processing | |
| US20130024432A1 (en) | Method and system for storing data in compliance with a compression handling instruction | |
| US10574751B2 (en) | Identifying data for deduplication in a network storage environment | |
| CN115878042A (en) | Data storage system, data processing method and storage medium | |
| CN111506450B (en) | Method, apparatus and computer program product for data processing | |
| CN105068875A (en) | Intelligence data processing method and apparatus | |
| US20150193311A1 (en) | Managing production data | |
| US10452614B2 (en) | Storage data reduction analysis and forecast | |
| US10592123B1 (en) | Policy driven IO scheduler to improve write IO performance in hybrid storage systems | |
| CN113127438B (en) | Method, apparatus, server and medium for storing data | |
| US11386060B1 (en) | Techniques for verifiably processing data in distributed computing systems |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |