CN113296699A - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN113296699A CN113296699A CN202110338957.7A CN202110338957A CN113296699A CN 113296699 A CN113296699 A CN 113296699A CN 202110338957 A CN202110338957 A CN 202110338957A CN 113296699 A CN113296699 A CN 113296699A
- Authority
- CN
- China
- Prior art keywords
- data
- target data
- storage medium
- target
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 41
- 238000000034 method Methods 0.000 claims abstract description 46
- 238000013524 data verification Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims description 26
- 238000012795 verification Methods 0.000 claims description 26
- 238000012986 modification Methods 0.000 claims description 13
- 230000004048 modification Effects 0.000 claims description 13
- 238000012216 screening Methods 0.000 claims description 11
- 238000013507 mapping Methods 0.000 claims description 8
- 230000014759 maintenance of location Effects 0.000 claims description 5
- 230000000717 retained effect Effects 0.000 description 13
- 238000010586 diagram Methods 0.000 description 9
- 125000004122 cyclic group Chemical group 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 230000037430 deletion Effects 0.000 description 5
- 238000012217 deletion Methods 0.000 description 4
- 238000012937 correction Methods 0.000 description 3
- 230000007423 decrease Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 230000001960 triggered effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/062—Securing storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1004—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's to protect a block of data words, e.g. CRC or checksum
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
Abstract
The present specification provides a data processing method and apparatus, wherein the data processing method includes: receiving a data dump instruction for target data; dumping the target data from an initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to first data verification in the dumping process; and in the case that the first data check is passed, retaining the target data in the initial storage medium.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a data processing method. The present specification also relates to a data processing apparatus, a computing device, and a computer-readable storage medium.
Background
With the development of internet technology, the amount of data owned by each enterprise is larger and larger, and some of the data are frequently accessed, and some of the data are not frequently accessed, and the value of the data mostly decreases with the increase of time, for example, the data mostly gradually changes from the stage of being accessed frequently to the stage of not being accessed frequently, and data for different stages can be stored by adopting different types of storage media, so that the storage cost is reduced, and therefore, when the value of data is changed, data is generally required to be dumped between different types of storage media, and currently during the dumping process of the data, data errors may occur for several reasons, such as program code problems, network errors, unavailability of the target storage medium, etc., and thus, a more secure way of data processing is required.
Disclosure of Invention
In view of this, the embodiments of the present specification provide a data processing method. The present specification also relates to a data processing apparatus, a computing device, and a computer-readable storage medium to solve the technical problems in the prior art.
According to a first aspect of embodiments herein, there is provided a data processing method including:
receiving a data dump instruction for target data;
dumping the target data from an initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to first data verification in the dumping process;
and in the case that the first data check is passed, retaining the target data in the initial storage medium.
According to a second aspect of embodiments herein, there is provided a data processing apparatus comprising:
receiving a data dump instruction for target data;
dumping the target data from an initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to first data verification in the dumping process;
and in the case that the first data check is passed, retaining the target data in the initial storage medium.
According to a third aspect of embodiments herein, there is provided a computing device comprising: a memory and a processor;
the memory is to store computer instructions, and the processor is to execute the computer instructions to:
receiving a data dump instruction for target data;
dumping the target data from an initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to first data verification in the dumping process;
and in the case that the first data check is passed, retaining the target data in the initial storage medium.
According to a fourth aspect of embodiments herein, there is provided a computer readable storage medium storing computer instructions which, when executed by a processor, implement the steps of the data processing method.
In the data processing method provided by the present specification, target data is dumped from an initial storage medium to a target storage medium based on a data dump instruction by receiving the data dump instruction for the target data, wherein the target data is subjected to a first data check in the dumping process; and under the condition that the first data check is passed, the target data is retained in the initial storage medium, so that after the target data is dumped to the target storage medium, the target data is retained in the initial storage medium and serves as a buffer for dumping the target data, and the data security of dumping the target data among different types of storage media is improved through the buffer for dumping the target data.
Drawings
Fig. 1 is a schematic diagram of a data processing method provided in an embodiment of the present specification;
FIG. 2 is a flow chart of a data processing method provided in an embodiment of the present specification;
fig. 3 is a schematic diagram of a data processing method applied to an actual scene according to an embodiment of the present specification;
fig. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present disclosure;
fig. 5 is a block diagram of a computing device according to an embodiment of the present disclosure.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, as those skilled in the art will be able to make and use the present disclosure without departing from the spirit and scope of the present disclosure.
The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
First, the noun terms to which one or more embodiments of the present specification relate are explained.
Data Lifecycle Management (DLM) is a policy-based method for managing the flow of data of an information system throughout the lifecycle: from creation and initial storage to its obsolescence is deleted.
Cyclic Redundancy Check (CRC) is a channel coding technique that generates a short fixed bit Check code based on data such as network data packets or computer files, and is mainly used to detect or Check errors that may occur after data transmission or storage. It uses the principle of division and remainder to detect the error.
The low frequency storage (Infrequency Access) is a storage type suitable for data which cannot be accessed frequently, in practical application, the unit storage cost of the low frequency storage is lower than that of standard storage, but the read-write delay is higher than that of the standard storage, and more network overhead is generated in the read-write process than that of the standard storage, wherein the standard storage is also a storage type, provides high-reliability, high-availability and high-performance storage service, and can support frequent data Access.
In order to realize the dumping of data between different storage media, fig. 1 shows a schematic diagram of a data processing method provided according to an embodiment of the present specification.
Specifically, taking the dump of on-line (Online) target data as an example, the storage state of the target data (which may be understood as data to be dumped) before the dump is not performed is an un-dump state, and after receiving a data dump instruction for the target data, dumping the target data from the initial storage medium to the target storage medium, and after the dump is completed, the storage state of the target data is modified from the non-dump state to the dump state, and modifying the dump sub-state of the target data into a None state, wherein after the modification is completed, the storage state of the target data is a dumped state and the storage sub-state is the None state, and if a read-write request for the target data is received during the period that the storage state is the dumped state and the dump sub-state is the None state, triggering data retrieval aiming at the target data (namely retrieving the target data from the target storage medium into the initial storage medium).
After data retrieval, the dump sub-state of the target data is modified from the None state to a Recalled (Recalled) state, after the modification is completed, the storage state of the target data is a dumped state and the dump sub-state is a Recalled (which can be understood as a second state in the following embodiments), if an update for the target data occurs in the process during the state, the target data needs to be re-dumped from the initial storage medium to the target storage medium after the target data meets the condition of re-dumping, and after the re-dumping, the storage sub-state of the target data is modified back to the None state.
In addition, before the target data is re-dumped, the target data which is dumped to the target storage medium can be added into a GC (gas Collection) garbage collection table as the expired data, and the expired data in the table is deleted through a data deletion program.
In order to solve the above problem, further, in this specification, a data processing method is provided, and this specification simultaneously relates to a data processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.
Fig. 2 is a flowchart illustrating a data processing method according to an embodiment of the present specification, which specifically includes the following steps:
step 202: a data dump instruction for target data is received.
In practical applications, the data value of the data determines the length of the data life cycle, and the data value decreases with time. Therefore, in performing data lifecycle management, data is typically divided into hot data and cold data, where hot data may be newer data and data that may be frequently accessed. And the cold data can be old data and data which is accessed infrequently, such as enterprise backup data, operation log data and the like.
In specific implementation, in order to reduce the storage cost of data, different storage media are usually used for cold and hot data, for example, the hot data is stored on a storage medium with a fast read/write speed, and the cold data is stored on a storage medium with a slower read/write speed, specifically, the hot data may be stored in a standard storage, and the cold data may be stored in a low-frequency storage; or storing hot data in memory and cold data in a cheaper storage medium (e.g., SSD or hard disk), etc., without limitation.
Further, the target data may be understood as any data that needs to be converted from the storage medium, such as: hot data or cold data, etc. Accordingly, the data dump instruction may be understood as an instruction for instructing to dump target data, where the instruction may be triggered based on a preset condition (e.g., every 3 days, etc.), or may be submitted by a user, and is not limited herein.
In specific implementation, the data dump in the embodiment of the present application may be performed on target data based on storage modes such as file storage, object storage, block storage, or storage of a database, and is not limited herein.
Step 204: and dumping the target data from the initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to first data verification in the dumping process.
The initial storage medium refers to a storage medium in which current target data is stored, and the target storage medium refers to a storage medium in which the target data is transferred and then stored. Further, dumping the target data from the initial storage medium to the target storage medium based on the data dumping instruction may be understood as copying the target data stored in the initial storage medium, and transferring or transmitting the copy of the target data to the target storage medium for storage.
In a specific implementation, the data dump instruction may carry location information of the target data stored in the initial storage medium, so as to indicate where to acquire the target data for data dump. In addition, the data dump instruction may also carry location information (such as a storage path for storing the target data in the target storage medium) for storing the target data in the target storage medium, so as to indicate a dump location of the target data.
In practical applications, in order to determine whether a transmission error occurs during the transfer or transmission of target data in different storage media, data verification (i.e., first data verification) may be performed on the target data during the dumping process. Specifically, the first data Check may adopt Cyclic Redundancy Check (CRC), parity Check, gray code Check, and the like, which is not limited herein.
The principle of cyclic redundancy check is: a number (i.e., CRC code, abbreviated as cyclic code, which is a check code with error detection and correction capabilities, and may be a binary sequence) is appended to a data frame (which includes data in target data) to be transmitted from a transmitting end (i.e., an initial storage medium), and a new data frame is generated and transmitted to a receiving end (i.e., a target storage medium).
And after receiving the new data frame, the receiving end performs data check on the new data frame through the CRC code to judge whether the received data has errors, and if so, performs error correction. Specifically, the data check at the receiving end uses the characteristics of the CRC code: the new data frame generated can be divided by a certain number selected in common by the transmitting end and the receiving end. If the remainder is 0, the frame is indicated to have no errors in the transmission process; if the remainder is not 0, it indicates that the frame has errors in the transmission process and needs error correction.
In practical applications, in an optional implementation manner provided by the embodiment of the present application, after dumping the target data from the initial storage medium to the target storage medium based on the data dump instruction, the method further includes:
receiving an update request for the target data in the initial storage medium;
updating the target data in the initial storage medium based on the update request.
Specifically, the update request may be understood as a write request for target data, or may be understood as a modification request for target data, in practical applications, after the target data is dumped from the initial storage medium to the target storage medium, or during the process of dumping the target data, an update request for the target data in the initial storage medium may be received, in this case, the target data in the initial storage medium is updated based on the update request, so that the target data is prevented from being updated during the process of dumping the target data, and a dumping experience and an updating experience of a user for the target data are ensured.
It should be noted that, after target data is dumped to a target storage medium, if the target data in an initial storage medium is updated based on an update request, a problem that the target data stored in the initial storage medium is inconsistent with the target data stored in the target storage medium may occur, and based on this, in an optional implementation manner provided by an embodiment of the present application, after the target data in the initial storage medium is updated based on the update request, the method further includes:
receiving a second data dump instruction for the target data;
and deleting the target data in the target storage medium based on the second data dump instruction, and dumping the updated target data from the initial storage medium to the target storage medium.
Specifically, the second data dump instruction, similar to the data dump instruction described above, is an instruction for dumping target data from the initial storage medium to the target storage medium, and the difference is that, because the target storage medium is already stored in the target storage medium, based on the second data dump instruction, it is necessary to delete the target data that has been previously stored in the target storage medium, and then dump the target data in the initial storage medium to the target storage medium again, so as to ensure consistency of the target data in the initial storage medium and the target storage medium.
Further, on the basis that the target data has a corresponding storage state, the update condition in the initial storage medium of the target data dumped into the target storage medium may be state-marked by the storage state, so as to determine whether the target data dumped into the target storage medium has changed based on the storage state, in an optional implementation manner provided in the embodiment of the present application, when the first data check passes, the method further includes:
receiving an update request for the target data in the initial storage medium;
and updating the target data in the initial storage medium based on the updating request, and modifying the storage state into a second state.
Specifically, a specific implementation of receiving an update request for the target data in the initial storage medium and updating the target data in the initial storage medium based on the update request is similar to the specific implementation of receiving an update request for the target data in the initial storage medium and updating the target data in the initial storage medium based on the update request, and reference to the specific implementation is only needed, and details are not repeated here.
Further, the second state may be understood as a dumped state and a state to be dumped again, in the embodiment of the present application, after the update of the target data in the initial storage medium is completed, the storage state corresponding to the target data is modified into the second state, so that the dumped data, which is required to be dumped again, is screened and dumped again based on the second state.
Still further, after the dumping the target data to the target storage medium and on the basis that the target data has the corresponding storage state, if the target data in the initial storage medium is updated based on the update request, the target data stored in the initial storage medium also needs to be dumped again to ensure consistency between the target data of the initial storage medium and the target data in the target storage medium, in an optional implementation manner provided in the embodiment of the present application, after the modifying the storage state to the second state, the method further includes:
receiving a second data dump instruction for the target data;
deleting the target data in the target storage medium based on the second data dump instruction, and dumping the updated target data from the initial storage medium to the target storage medium;
modifying the storage state modification from the second state to the first state.
Specifically, a second data dump instruction for the target data is received, the target data in the target storage medium is deleted based on the second data dump instruction, and the updated target data is dumped from the initial storage medium to the target storage medium, which is similar to the specific implementation of the data dump based on the second data dump instruction, and is not limited here.
It should be noted that, the second data dump instruction may carry a filtering condition for target data that needs to be re-dumped, for example, the filtering condition may include that the storage state is a second state, so as to determine the target data that needs to be re-dumped based on the filtering condition carried in the instruction.
In addition, the screening condition may further include an update time interval of the data, and since when the target data is updated, only a part of the target data may be updated, the updated part of the target data (update data) may be screened out through the update time interval, and the update data in the target data may be re-dumped.
And after the target data is re-dumped, modifying the storage state corresponding to the target data from the second state to the first state, wherein the first state can indicate that the target data is stored in the target storage medium. And the storage state is modified from the second state to the first state, which indicates that the target data after updating is re-dumped, so that whether the target data is re-dumped or not is marked through the storage state, and the dumping situation of the target data is clearer.
Step 206: and in the case that the first data check is passed, retaining the target data in the initial storage medium.
Specifically, under the condition that the first data check is passed, the target data is retained in the initial storage medium for a period of time (for example, 2 hours, 1 day, and the like, which is not limited herein), so that data risks caused by data errors and the like which may occur in a period of time after the target data is dumped, which are caused by directly deleting the target data from the initial storage medium after the target data is dumped, are avoided, and the security of the target data is increased.
Further, if the result of the first data check performed on the target data in the dumping process is failed, which indicates that the target data is in error in the dumping process, dumping can be performed again on the target data.
In practical applications, the target data is retained in the initial storage medium, so as to perform a second data check on the target data, thereby ensuring consistency of the target data before and after dumping, and in an optional implementation manner provided by an embodiment of the present application, the data processing method further includes:
receiving a secondary verification instruction aiming at the target data;
performing second data verification on the target data based on the second verification instruction;
and deleting the target data from the initial storage medium under the condition that the second data check is passed.
The second verification instruction is a verification instruction for indicating second data verification on the target data on the basis of performing first data verification on the target data. In practical application, the verification method for performing the second data verification on the target data may be the same as the verification method for performing the first data verification, or may be different from the verification method for performing the first data verification, which is not limited herein.
In a specific implementation, after the target data is retained in the initial storage medium for a period of time, the second data verification is performed on the target data based on receiving a second verification instruction for the target data, specifically, the second verification instruction may be periodically triggered (for example, a trigger period is set according to a desired retention time of the target data in the initial storage medium), or may be manually triggered by a user, which is not limited herein. And the second data check may include: consistency checks and/or integrity checks, etc., without limitation.
It should be noted that the secondary verification instruction may also carry a screening condition/determination condition (for example, dump time exceeds 1 day or 2 hours, etc.) for data to be subjected to the second data verification, and add information such as a data identifier of the data to be verified, which is determined based on the screening condition/determination condition carried in the instruction, to a processing list for the second data verification, so that when performing the second data verification, the second data verification may be performed on the data to be verified sequentially according to the information in the processing list.
It should be noted that the determined data to be verified may include, in addition to the target data, other dump data that satisfies the screening condition/determination condition, which is not limited herein.
Further, in the case that the second data check is passed, it indicates that the target data in the initial storage medium is consistent with the target data in the target storage medium, and also indicates that the dump for the target data is successful, the target data is deleted from the initial storage medium.
In an optional implementation manner provided by the embodiment of the present application, the second data verification on the target data may be performed by freely combining the following 3 verification manners:
1) and carrying out consistency check on the target data according to the data volume information in the metadata of the target data and the data volume of the target data in the target storage medium.
The metadata may be understood as tags of the target data, and the entry type and number of the tags are not limited, and in particular, the metadata may be various kinds of description information of the target data. For example, if the target data is a photo of a person, the metadata may include the name, nationality, etc. of the person in the photo, and the metadata may also include the location, time, etc. of the photo, without limitation.
It should be noted that the metadata may belong to the target data itself, and be packaged and stored together with the target data. In addition, the metadata may also be separate from the target data and not encapsulated within the target data.
In this embodiment of the application, the metadata of the target data includes data volume information of the target data (for example, 45kb, 10M, etc., which is not limited herein), the data volume information in the metadata is compared with the data volume of the target data in the target storage medium, and if the comparison is consistent, it indicates that the consistency check on the target data passes, the target data is deleted from the initial storage medium; if the comparison is inconsistent, it indicates that the consistency check on the target data fails, and also indicates that the second data check on the target data fails, the target data in the target storage medium needs to be deleted, and the target data needs to be dumped again.
2) And performing consistency check on the target data according to the version information of the target data in the initial storage medium and the version information of the target data in the target storage medium.
In practical application, the version information of the target data is changed at the same time of changing the target data. Therefore, when the version information of the target data in the initial storage medium is inconsistent with the version information of the target data in the target storage medium, the target data in the initial storage medium is indicated to be inconsistent with the target data in the target storage medium. The version information of the target data in the initial storage medium may be updated and stored as data in the metadata of the target data.
In specific implementation, comparing the version information of the target data in the initial storage medium with the version information of the target data in the target storage medium, and deleting the target data from the initial storage medium if the comparison is consistent and the consistency check of the target data is passed; if the comparison is inconsistent, it indicates that the consistency check on the target data fails, and also indicates that the second data check on the target data fails, the target data in the target storage medium needs to be deleted, and the target data needs to be dumped again.
3) Determining a data check code according to the target data in the initial storage medium and a preset check mode;
and carrying out consistency check on the target data based on the data check code.
The data check code may generate different data check codes according to different preset data check modes (preset check modes), for example, when data check is performed by using cyclic redundancy check, a CRC code is generated according to a rule of the cyclic redundancy check and target data in an initial storage medium; when the parity check is used for data check, a parity check code is generated according to a parity check rule and target data in the initial storage medium, which is not limited herein.
Further, performing consistency check on the target data based on the data check code may be understood as performing consistency check on the target data in the initial storage medium and the target data in the target storage medium based on the data check code.
It should be noted that, in addition to the above data verification method, other methods may also be used to perform data verification, and the method is not limited herein.
In practical application, at least one consistency check is carried out on the target data in the initial storage medium and the target data in the target storage medium, so that the consistency of the target data in the initial storage medium and the target storage medium is guaranteed.
In specific implementation, in order to more clearly understand and control the dump condition of the target data, a storage state may be set for the target data, and the storage state is modified to implement state marking on the dump process of the target data, in an optional implementation manner provided in this application embodiment, in the case that the first data check passes, the method further includes:
modifying a storage state of the target data in the metadata in the initial storage medium to a first state;
correspondingly, the performing a second data verification on the target data based on the second verification instruction includes:
determining the target data based on a data screening condition carried in the secondary verification instruction, wherein the data screening condition comprises: the storage state is the first state, and the retention time length in the initial storage medium meets a preset time length;
and carrying out second data verification on the target data.
Specifically, in the case that the first data check is passed, the storage state of the target data is modified to a first state, where the first state may indicate that the target data is already stored in the target storage medium. In practical application, in order to avoid modifying the target data, the storage state of the target data may be recorded in a metadata manner, and the metadata and the target data are independent from each other, so that modifying the metadata does not affect the dumped target data.
Further, after the storage state of the target data in the metadata in the initial storage medium is modified to the first state, when the target data needs to be subjected to second data verification, the data with the storage state in the first state may be screened based on the data screening condition carried in the secondary verification instruction, so as to determine the target data, and further, the target data is subjected to second data verification.
Specifically, the reserved time length may be understood as a time length reserved in the initial storage medium after the target data is dumped to the target storage medium; the preset duration can be set according to experience or actual requirements, and is not limited herein.
According to the data verification method and device, the target data are reserved in the initial storage medium for more than the preset time length and then subjected to the second data verification, namely the target data to be subjected to the second data verification are screened through the storage state and the preset time length, and then the second data verification is performed on the target data, so that the accuracy of performing the second data verification on the target data is improved.
Further, in an optional implementation manner provided by an embodiment of the present application, after deleting the target data from the initial storage medium, the method further includes:
modifying the storage state from the first state to a third state.
The third state may be understood as a dump complete state, and specifically, after the target data is deleted from the initial storage medium, the storage state in the metadata of the target data is modified from the first state to the third state, which indicates that dump is completed for the target data.
In addition to the above distinguishing of the dump condition of the target data by the storage state, in order to make the target data remain in the initial storage medium for a period of time, the storage condition of the target data may also be distinguished by hiding a folder, in an optional implementation manner provided by an embodiment of the present application, the keeping of the target data in the initial storage medium includes:
transferring and retaining the target data to a hidden folder created in advance in the initial storage medium;
correspondingly, the performing of the second data verification on the target data includes:
and carrying out consistency check on the target data in the hidden folder and the target data in the target storage medium.
The hidden folder may be understood as a folder in a hidden state, which is invisible in a user space (which may be understood as a space where a user can access data), but may be indexed to target data transferred to the hidden folder by internal attributes of the target data (such as data keywords, data identifiers, and the like).
Based on this, when performing the second data check on the target data, the second data check (such as consistency check) may be performed on the target data in the hidden folder and the target data in the target storage medium.
According to the embodiment of the application, after the target data is dumped to the target storage medium, the target data is transferred to the hidden folder for reservation, which not only indicates that the target data is dumped, but also can realize the reservation of the target data in the initial storage medium, so that the reserved time is checked for the second time.
Further, on the basis that the target data is transferred and retained in the hidden folder and the second data check is completed on the target data, in an optional implementation manner provided by an embodiment of the present application, in a case that the second data check passes, the deleting the target data from the initial storage medium includes:
and deleting the target data and the hidden folder from the initial storage medium.
According to the embodiment of the application, the hidden folder is used for storing the dumped target data, and the hidden folder is deleted together under the condition that the target data are deleted, so that the hidden folder and the target data are managed together, and the management convenience is improved.
It should be noted that, in addition to the above deletion method for the target data, the target data may be deleted from the hidden folder in the initial storage medium, that is, in the case of deleting the target data, the hidden folder is reserved so as to be used for storing other dumped data, but data needs to be checked for the second time, so that frequent creation/deletion of the hidden folder is avoided, and the utilization rate of the hidden folder is improved.
In an optional implementation manner provided by the embodiment of the present application, after storing the target data in a hidden folder created in advance in the initial storage medium, the method further includes:
receiving an update request for the target data in the initial storage medium;
and updating the target data in the hidden folder based on the updating request and the mapping relation between the target data and the hidden folder which is established in advance.
Specifically, the mapping relationship may be created in advance according to the data identifier of the target data and the folder identifier of the hidden folder, that is, the mapping relationship between the data identifier and the folder identifier is established as the mapping relationship between the target data and the hidden folder.
In specific implementation, the update request may carry a data identifier of the target data or other data information that may represent the target data, so as to determine the hidden folder based on the data identifier or the data information and a mapping relationship established in advance, determine the target data in the hidden folder, and then update the target data.
According to the embodiment of the application, the hidden folder is searched based on the mapping relation between the target data and the hidden folder which is established in advance, and the target data in the hidden folder is updated after the hidden folder is determined.
Further, after updating the target data in the hidden folder in the initial storage medium, if the target data in the hidden folder is inconsistent with the target data in the target storage medium, then the target data needs to be dumped again, in an optional implementation manner provided in an embodiment of the present application, the data processing method further includes, after updating the target data in the hidden folder:
receiving a second data dump instruction for the target data;
and deleting the target data in the target storage medium based on the second data dump instruction, and dumping the updated target data to the target storage medium from the hidden folder.
Specifically, dumping the updated target data from the hidden folder to the target storage medium is similar to the specific implementation of dumping the updated target data from the initial storage medium to the target storage medium, and is not limited herein.
To sum up, in the data processing method provided by the present specification, a data dump instruction for target data is received, and the target data is dumped from an initial storage medium to a target storage medium based on the data dump instruction, where the target data is subjected to a first data check in the dumping process; and under the condition that the first data check is passed, the target data is retained in the initial storage medium, so that after the target data is dumped to the target storage medium, the target data is retained in the initial storage medium and serves as a buffer for dumping the target data, so that the target data is subjected to second data check through the buffer for dumping the target data, and the data security of dumping the target data among different types of storage media is improved.
The following describes the data processing method further by taking an application of the data processing method provided in this specification in an actual scene as an example, with reference to fig. 3. Fig. 3 shows a schematic diagram of a data processing method applied in an actual scene provided in an embodiment of the present specification, which specifically includes the following steps:
taking the dump of Online (Online) target data as an example, the state control of the target data during the dump process will be described in detail.
Specifically, the storage state of the target data before dump is not in the dump state, and after a data dump instruction for the target data is received, the target data is dumped from the initial storage medium to the target storage medium, and after dump is completed, the target data is retained in the initial storage medium, and the storage state of the target data is modified from the dump state to the dump state. And modifying the dump sub-state of the target data into a Grace (hierarchical) state, after the modification is completed, the storage state of the target data is a dumped state and the storage sub-state is a Grace state (which can be understood as a first state in the above embodiment), and if a read request for the target data in the initial storage medium is received while the storage state of the target data is the dumped state and the dump sub-state is the Grace state (which can be understood as a first state in the following embodiment), directly reading the data in the initial storage medium, and not modifying the state.
During this period, a write request for target data is received (which may be understood as an update request in the above embodiment), based on the write request, the target data in the initial storage medium is updated, and the dump sub-state of the target data is modified from the Grace state to a Recalled state, and after the modification is completed, the storage state of the target data is a dumped state and the dump sub-state is a Recalled (which may be understood as a second state in the above embodiment). And after the target data meets the condition of re-dumping (for example, the dumping sub-state is called and the update duration of the target data exceeds the re-dumping period), re-dumping the target data from the initial storage medium to the target storage medium by a dumping program, and after the re-dumping, modifying the storage sub-state of the target data back to the Grace state.
And if the storage state of the target data is a dumped state, the dump sub-state is a Grace state, and the dumping duration of the target data exceeds the verification period of the second data verification, performing the second data verification on the target data, deleting the target data from the initial storage medium if the second data verification is passed, and modifying the dump sub-state of the target data into a None state. After the modification is completed, the storage state of the target data is the dumped state and the dump sub-state is the None state (which may be understood as the third state in the above embodiment).
If the target data is updated during the storage state of the target data is the dumped state and the dump sub-state is the None state, it is necessary to perform data retrieval on the target data (i.e. retrieving data from the target storage medium to the original storage medium) according to the configuration (such as the proportion of data retrieval), and modify the dump sub-state of the retrieved data to be Recalled.
To sum up, in the data processing method provided by the present specification, a data dump instruction for target data is received, and the target data is dumped from an initial storage medium to a target storage medium based on the data dump instruction, where the target data is subjected to a first data check in the dumping process; and under the condition that the first data check is passed, the target data is retained in the initial storage medium, so that after the target data is dumped to the target storage medium, the target data is retained in the initial storage medium and serves as a buffer for dumping the target data, so that the target data is subjected to second data check through the buffer for dumping the target data, and the data security of dumping the target data among different types of storage media is improved.
Corresponding to the above method embodiment, this specification further provides an embodiment of a data processing apparatus, and fig. 4 shows a schematic structural diagram of a data processing apparatus provided in an embodiment of this specification. As shown in fig. 4, the apparatus includes:
a receiving module 402 configured to receive a data dump instruction for target data;
a dumping module 404 configured to dump the target data from an initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to a first data check during the dumping process;
a retaining module 406 configured to retain the target data in the initial storage medium in case the first data check passes.
Optionally, the data processing apparatus further includes:
a second receiving module configured to receive a secondary verification instruction for the target data;
the secondary verification module is configured to perform secondary data verification on the target data based on the secondary verification instruction;
a first deletion module configured to delete the target data from the initial storage medium if the second data check passes.
Optionally, the data processing apparatus further includes:
a third receiving module configured to receive an update request for the target data in the initial storage medium;
a first update module configured to update the target data in the initial storage medium based on the update request.
Optionally, the data processing apparatus further includes:
a fourth receiving module configured to receive a second data dump instruction for the target data;
and the second deleting module is configured to delete the target data in the target storage medium based on the second data dump instruction, and dump the updated target data from the initial storage medium to the target storage medium.
Optionally, the data processing apparatus further includes:
a first modification module configured to modify a storage state of the target data in the metadata in the initial storage medium to a first state;
correspondingly, the secondary verification module is further configured to:
determining the target data based on a data screening condition carried in the secondary verification instruction, wherein the data screening condition comprises: the storage state is the first state, and the retention time length in the initial storage medium meets a preset time length;
and carrying out second data verification on the target data.
Optionally, the data processing apparatus further includes:
a fifth receiving module configured to receive an update request for the target data in the initial storage medium;
a second update module configured to update the target data in the initial storage medium based on the update request and modify the storage state to a second state.
Optionally, the data processing apparatus further includes:
a sixth receiving module configured to receive a second data dump instruction for the target data;
a third deleting module configured to delete the target data in the target storage medium based on the second data dump instruction, and dump the updated target data from the initial storage medium to the target storage medium;
a second modification module configured to modify the storage state modification from the second state to the first state.
Optionally, the data processing apparatus further includes:
a third modification module configured to modify the storage state from the first state to a third state.
Optionally, the secondary verification module is further configured to:
according to data volume information in metadata of the target data and the data volume of the target data in the target storage medium, performing consistency check on the target data; and/or
Performing consistency check on the target data according to the version information of the target data in the initial storage medium and the version information of the target data in the target storage medium; and/or
Determining a data check code according to the target data in the initial storage medium and a preset check mode;
and carrying out consistency check on the target data based on the data check code.
Optionally, the reservation module 406 is further configured to:
transferring and retaining the target data to a hidden folder created in advance in the initial storage medium;
correspondingly, the secondary verification module is further configured to:
and carrying out consistency check on the target data in the hidden folder and the target data in the target storage medium.
Optionally, the first deletion module is further configured to:
and deleting the target data and the hidden folder from the initial storage medium.
Optionally, the data processing apparatus further includes:
a seventh receiving module configured to receive an update request for the target data in the initial storage medium;
a third updating module configured to update the target data in the hidden folder based on the update request and a mapping relationship between the target data and the hidden folder, which is established in advance.
Optionally, the data processing apparatus further includes:
an eighth receiving module configured to receive a second data dump instruction for the target data;
and the fourth deleting module is configured to delete the target data in the target storage medium based on the second data dump instruction, and dump the updated target data from the hidden folder to the target storage medium.
To sum up, the data processing apparatus provided in this specification dumps target data from an initial storage medium to a target storage medium based on a data dump instruction by receiving the data dump instruction for the target data, where the target data is subjected to a first data check during the dump; and under the condition that the first data check is passed, the target data is retained in the initial storage medium, so that after the target data is dumped to the target storage medium, the target data is retained in the initial storage medium and serves as a buffer for dumping the target data, so that the target data is subjected to second data check through the buffer for dumping the target data, and the data security of dumping the target data among different types of storage media is improved.
The above is a schematic configuration of a data processing apparatus of the present embodiment. It should be noted that the technical solution of the data processing apparatus and the technical solution of the data processing method belong to the same concept, and details that are not described in detail in the technical solution of the data processing apparatus can be referred to the description of the technical solution of the data processing method.
FIG. 5 illustrates a block diagram of a computing device 500 provided in accordance with one embodiment of the present description. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. Processor 520 is coupled to memory 510 via bus 530, and database 550 is used to store data.
Computing device 500 also includes access device 540, access device 540 enabling computing device 500 to communicate via one or more networks 560. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. The access device 540 may include one or more of any type of network interface, e.g., a Network Interface Card (NIC), wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
In one embodiment of the present description, the above-described components of computing device 500, as well as other components not shown in FIG. 5, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 5 is for purposes of example only and is not limiting as to the scope of the present description. Those skilled in the art may add or replace other components as desired.
Computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), mobile phone (e.g., smartphone), wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 500 may also be a mobile or stationary server.
Wherein processor 520 is configured to execute the following computer-executable instructions:
receiving a data dump instruction for target data;
dumping the target data from an initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to first data verification in the dumping process;
and in the case that the first data check is passed, retaining the target data in the initial storage medium.
The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the data processing method belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the data processing method.
An embodiment of the present specification also provides a computer readable storage medium storing computer instructions that, when executed by a processor, are operable to:
receiving a data dump instruction for target data;
dumping the target data from an initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to first data verification in the dumping process;
and in the case that the first data check is passed, retaining the target data in the initial storage medium.
The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the data processing method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the data processing method.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present disclosure is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present disclosure. Further, those skilled in the art should also appreciate that the embodiments described in this specification are preferred embodiments and that acts and modules referred to are not necessarily required for this description.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the specification and its practical application, to thereby enable others skilled in the art to best understand the specification and its practical application. The specification is limited only by the claims and their full scope and equivalents.
Claims (16)
1. A method of data processing, comprising:
receiving a data dump instruction for target data;
dumping the target data from an initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to first data verification in the dumping process;
and in the case that the first data check is passed, retaining the target data in the initial storage medium.
2. The data processing method of claim 1, further comprising:
receiving a secondary verification instruction aiming at the target data;
performing second data verification on the target data based on the second verification instruction;
and deleting the target data from the initial storage medium under the condition that the second data check is passed.
3. The data processing method of claim 1, after the dumping the target data from the initial storage medium to the target storage medium based on the data dump instruction, further comprising:
receiving an update request for the target data in the initial storage medium;
updating the target data in the initial storage medium based on the update request.
4. The data processing method of claim 3, further comprising, after updating the target data in the initial storage medium based on the update request:
receiving a second data dump instruction for the target data;
and deleting the target data in the target storage medium based on the second data dump instruction, and dumping the updated target data from the initial storage medium to the target storage medium.
5. The data processing method according to claim 2, further comprising, in case the first data check passes:
modifying a storage state of the target data in the metadata in the initial storage medium to a first state;
correspondingly, the performing a second data verification on the target data based on the second verification instruction includes:
determining the target data based on a data screening condition carried in the secondary verification instruction, wherein the data screening condition comprises: the storage state is the first state, and the retention time length in the initial storage medium meets a preset time length;
and carrying out second data verification on the target data.
6. The data processing method according to claim 5, further comprising, in case the first data check passes:
receiving an update request for the target data in the initial storage medium;
and updating the target data in the initial storage medium based on the updating request, and modifying the storage state into a second state.
7. The data processing method of claim 6, after modifying the storage state to the second state, further comprising:
receiving a second data dump instruction for the target data;
deleting the target data in the target storage medium based on the second data dump instruction, and dumping the updated target data from the initial storage medium to the target storage medium;
modifying the storage state modification from the second state to the first state.
8. The data processing method of claim 5, further comprising, after the deleting the target data from the initial storage medium:
modifying the storage state from the first state to a third state.
9. The data processing method of claim 2, wherein the performing a second data check on the target data comprises:
according to data volume information in metadata of the target data and the data volume of the target data in the target storage medium, performing consistency check on the target data; and/or
Performing consistency check on the target data according to the version information of the target data in the initial storage medium and the version information of the target data in the target storage medium; and/or
Determining a data check code according to the target data in the initial storage medium and a preset check mode;
and carrying out consistency check on the target data based on the data check code.
10. The data processing method of claim 2, the retaining the target data in the initial storage medium, comprising:
transferring and retaining the target data to a hidden folder created in advance in the initial storage medium;
correspondingly, the performing of the second data verification on the target data includes:
and carrying out consistency check on the target data in the hidden folder and the target data in the target storage medium.
11. The data processing method of claim 10, the deleting the target data from the initial storage medium, comprising:
and deleting the target data and the hidden folder from the initial storage medium.
12. The data processing method of claim 10, further comprising, after storing the target data in a hidden folder created in advance in the initial storage medium:
receiving an update request for the target data in the initial storage medium;
and updating the target data in the hidden folder based on the updating request and the mapping relation between the target data and the hidden folder which is established in advance.
13. The data processing method of claim 12, further comprising:
receiving a second data dump instruction for the target data;
and deleting the target data in the target storage medium based on the second data dump instruction, and dumping the updated target data to the target storage medium from the hidden folder.
14. A data processing apparatus comprising:
a receiving module configured to receive a data dump instruction for target data;
the dumping module is configured to dump the target data from an initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to first data verification in the dumping process;
a retention module configured to retain the target data in the initial storage medium if the first data check passes.
15. A computing device, comprising:
a memory and a processor;
the memory is used for storing computer instructions, and the processor is used for executing the computer instructions to realize the following method:
receiving a data dump instruction for target data;
dumping the target data from an initial storage medium to a target storage medium based on the data dumping instruction, wherein the target data is subjected to first data verification in the dumping process;
and in the case that the first data check is passed, retaining the target data in the initial storage medium.
16. A computer readable storage medium storing computer instructions which, when executed by a processor, carry out the steps of the data processing method of any one of claims 1 to 13.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110338957.7A CN113296699A (en) | 2021-03-30 | 2021-03-30 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110338957.7A CN113296699A (en) | 2021-03-30 | 2021-03-30 | Data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113296699A true CN113296699A (en) | 2021-08-24 |
Family
ID=77319288
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110338957.7A Pending CN113296699A (en) | 2021-03-30 | 2021-03-30 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113296699A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190220528A1 (en) * | 2018-01-18 | 2019-07-18 | International Business Machines Corporation | Effective handling of hsm migrated files and snapshots |
| US10733145B1 (en) * | 2014-12-16 | 2020-08-04 | Amazon Technologies, Inc. | Techniques for distributing verification checks incident to data deletion |
| WO2021051563A1 (en) * | 2019-09-17 | 2021-03-25 | 平安科技（深圳）有限公司 | Target data archiving method and apparatus, and electronic device and computer non-volatile readable storage medium |
-
2021
- 2021-03-30 CN CN202110338957.7A patent/CN113296699A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10733145B1 (en) * | 2014-12-16 | 2020-08-04 | Amazon Technologies, Inc. | Techniques for distributing verification checks incident to data deletion |
| US20190220528A1 (en) * | 2018-01-18 | 2019-07-18 | International Business Machines Corporation | Effective handling of hsm migrated files and snapshots |
| WO2021051563A1 (en) * | 2019-09-17 | 2021-03-25 | 平安科技（深圳）有限公司 | Target data archiving method and apparatus, and electronic device and computer non-volatile readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6373328B2 (en) | Aggregation of reference blocks into a reference set for deduplication in memory management | |
| CN102880663B (en) | The optimization of the file of part deduplication | |
| JP4778061B2 (en) | Efficient database synchronization by comparing representative information | |
| US20150006475A1 (en) | Data deduplication in a file system | |
| US8250040B2 (en) | Storage or removal actions based on priority | |
| US11093387B1 (en) | Garbage collection based on transmission object models | |
| US8935212B2 (en) | Discovery of non-standard folders for backup | |
| US9292387B2 (en) | Medium, control method, and information processing apparatus | |
| CN113296696A (en) | Data access method, computing device and storage medium | |
| US10628298B1 (en) | Resumable garbage collection | |
| US20170123678A1 (en) | Garbage Collection for Reference Sets in Flash Storage Systems | |
| WO2019062572A1 (en) | Data processing method, device and system | |
| US20170123689A1 (en) | Pipelined Reference Set Construction and Use in Memory Management | |
| CN106708825A (en) | Data file processing method and system | |
| US20170123677A1 (en) | Integration of Reference Sets with Segment Flash Management | |
| WO2024041434A1 (en) | Storage system and data processing method | |
| CN113296699A (en) | Data processing method and device | |
| CN114625695A (en) | Data processing method and device | |
| CN112286947B (en) | Method and device for keeping data consistency of different storage systems | |
| US11943294B1 (en) | Storage medium and compression for object stores | |
| CN115422135A (en) | Data processing method and device | |
| CN113886350A (en) | Data processing method and system | |
| US11645333B1 (en) | Garbage collection integrated with physical file verification | |
| CN113918096A (en) | Method and device for uploading algorithm mirror image packet and application | |
| CN114063935B (en) | Method and device for processing data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40059143 Country of ref document: HK |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240302 Address after: # 03-06, Lai Zan Da Building 1, 51 Belarusian Road, Singapore Applicant after: Alibaba Innovation Co. Guo jiahuodiqu after: Xin Jiapo Address before: Room 01, 45th Floor, AXA Building, 8 Shanton Road Applicant before: Alibaba Singapore Holdings Ltd. Guo jiahuodiqu before: Xin Jiapo |