CN117762693A - Data recovery method and device - Google Patents
Data recovery method and device Download PDFInfo
- Publication number
- CN117762693A CN117762693A CN202311830457.0A CN202311830457A CN117762693A CN 117762693 A CN117762693 A CN 117762693A CN 202311830457 A CN202311830457 A CN 202311830457A CN 117762693 A CN117762693 A CN 117762693A
- Authority
- CN
- China
- Prior art keywords
- data
- change information
- recovery
- change
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011084 recovery Methods 0.000 title claims abstract description 174
- 238000000034 method Methods 0.000 title claims abstract description 85
- 230000008859 change Effects 0.000 claims abstract description 230
- 238000012545 processing Methods 0.000 claims abstract description 47
- 238000004088 simulation Methods 0.000 claims abstract description 38
- 230000002776 aggregation Effects 0.000 claims abstract description 21
- 238000004220 aggregation Methods 0.000 claims abstract description 21
- 238000012360 testing method Methods 0.000 claims description 45
- 238000003860 storage Methods 0.000 claims description 21
- 238000012986 modification Methods 0.000 claims description 13
- 230000004048 modification Effects 0.000 claims description 13
- 230000004931 aggregating effect Effects 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 27
- 238000012217 deletion Methods 0.000 description 18
- 230000037430 deletion Effects 0.000 description 18
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 9
- 230000006872 improvement Effects 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 230000001360 synchronised effect Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 238000012958 reprocessing Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 230000003362 replicative effect Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013481 data capture Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 229920001296 polysiloxane Polymers 0.000 description 1
- 238000013102 re-test Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000010979 ruby Substances 0.000 description 1
- 229910001750 ruby Inorganic materials 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the specification provides a data recovery method and device, wherein the data recovery method comprises the following steps: acquiring data change information of data change of the data in the mirror image database by a plurality of simulation flow rates, and extracting at least one data position corresponding to the data change information; performing aggregation processing on the data change information according to each data position to obtain position data change information of each data position; analyzing the position data change information of each data position, obtaining at least one data operation of each data position, and determining a data recovery operation corresponding to the at least one data operation; and carrying out data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
Description
Technical Field
The present disclosure relates to the field of data processing technologies, and in particular, to a data recovery method and apparatus.
Background
With the development of network technology, more and more services are provided on line, and more information is reached to users in an on-line mode; how to ensure the user's participation experience in the service is an increasing focus of the user and the service provider, so the service provider tests whether the service provided to the user is valid by testing the data; in the testing process, the data stored in the mirror database needs to be restored to realize more accurate and more effective testing, and how to restore the data stored in the mirror database is an important point of attention of service providers.
Disclosure of Invention
One or more embodiments of the present specification provide a data recovery method. The data recovery method comprises the following steps: and acquiring data change information of data change of the data in the mirror image database by the plurality of simulation flow rates, and extracting at least one data position corresponding to the data change information. And carrying out aggregation processing on the data change information according to each data position to obtain the position data change information of each data position. And analyzing the position data change information of each data position, obtaining at least one data operation of each data position, and determining a data recovery operation corresponding to the at least one data operation. And carrying out data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
One or more embodiments of the present specification provide a data recovery apparatus, including: the information acquisition module is configured to acquire data change information of data change of the data in the mirror database by the plurality of simulation flow rates and extract at least one data position corresponding to the data change information. And the aggregation processing module is configured to aggregate the data change information according to each data position to obtain the position data change information of each data position. The information analysis module is configured to analyze the position data change information of each data position, obtain at least one data operation of each data position, and determine a data recovery operation corresponding to the at least one data operation. And the data recovery module is configured to perform data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
One or more embodiments of the present specification provide a data recovery apparatus including: a processor; and a memory configured to store computer-executable instructions that, when executed, cause the processor to: and acquiring data change information of data change of the data in the mirror image database by the plurality of simulation flow rates, and extracting at least one data position corresponding to the data change information. And carrying out aggregation processing on the data change information according to each data position to obtain the position data change information of each data position. And analyzing the position data change information of each data position, obtaining at least one data operation of each data position, and determining a data recovery operation corresponding to the at least one data operation. And carrying out data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
One or more embodiments of the present specification provide a storage medium storing computer-executable instructions that, when executed by a processor, implement the following: and acquiring data change information of data change of the data in the mirror image database by the plurality of simulation flow rates, and extracting at least one data position corresponding to the data change information. And carrying out aggregation processing on the data change information according to each data position to obtain the position data change information of each data position. And analyzing the position data change information of each data position, obtaining at least one data operation of each data position, and determining a data recovery operation corresponding to the at least one data operation. And carrying out data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
Drawings
For a clearer description of one or more embodiments of the present description or of the solutions of the prior art, the drawings that are needed in the description of the embodiments or of the prior art will be briefly described below, it being obvious that the drawings in the description that follow are only some of the embodiments described in the present description, from which other drawings can be obtained, without inventive faculty, for a person skilled in the art;
FIG. 1 is a schematic diagram of an environment in which a data recovery method according to one or more embodiments of the present disclosure may be implemented;
FIG. 2 is a process flow diagram of a data recovery method according to one or more embodiments of the present disclosure;
FIG. 3 is a flow diagram of a process for a data change log provided in one or more embodiments of the present disclosure;
FIG. 4 is a process flow diagram of a data recovery method for a data recovery scenario for a mirrored database according to one or more embodiments of the present disclosure;
FIG. 5 is a schematic diagram of one or more embodiments of a data recovery device according to the present disclosure;
fig. 6 is a schematic structural diagram of a data recovery device according to one or more embodiments of the present disclosure.
Detailed Description
In order to enable a person skilled in the art to better understand the technical solutions in one or more embodiments of the present specification, the technical solutions in one or more embodiments of the present specification will be clearly and completely described below with reference to the drawings in one or more embodiments of the present specification, and it is obvious that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one or more embodiments of the present disclosure without inventive effort, are intended to be within the scope of the present disclosure.
The data recovery method provided in one or more embodiments of the present disclosure may be applied to an implementation environment of data recovery, as shown in fig. 1, where the implementation environment includes at least a server 101, a mirror database 102, and a test platform 103;
the server 101 may be one or more servers, a server cluster formed by a plurality of servers, or a cloud server of a cloud computing platform, and is configured to perform data recovery on changed data in the mirror database 102 caused by performing test processing on the test platform 103 based on the data in the mirror database 102, so that the test platform 103 may perform multiple test playback based on unit data in the mirror database 102;
The mirror database 102 is used for storing data copied from the database of the real environment, and the test platform 103 is used for performing test tasks based on the data in the mirror database 102.
In this implementation environment, during the process of recovering data in the mirror database 102, the server 101 first obtains data change information of a plurality of simulation flows for performing data change on the data in the mirror database, extracts at least one data location corresponding to the data change information, then performs aggregation processing on the data change information according to each data location to obtain location data change information of each data location, analyzes the location data change information of each data location, obtains at least one data operation of each data location, and determines a data recovery operation corresponding to the at least one data operation; and carrying out data recovery on the current data unknown to each data in the mirror database based on the data of each data position and the data recovery operation.
One or more embodiments of a data recovery method provided in the present specification are as follows:
according to the data recovery method provided by the embodiment, data operation of a plurality of simulation flows on the same data position is aggregated by aggregating data change information of the same data position in the mirror database, at least one data operation of each data position is obtained, a data recovery strategy of each data position is determined based on the at least one data operation of each data position, and data in the mirror database is recovered based on the data recovery strategy; specifically, according to the data positions, aggregation processing is carried out on the data change information, and position data change information of each data position is obtained; analyzing the change information of each data position to obtain at least one data operation of each data position, determining a data recovery operation corresponding to the at least one data operation, and carrying out data recovery on the current data of each data position in the mirror image database based on the data of each data position and the data recovery operation, so that the data recovery is carried out from the full dimension, and the data recovery efficiency is improved.
Referring to fig. 2, the data recovery method provided in this embodiment specifically includes steps S202 to S208.
Step S202, obtaining data change information of data change of a plurality of simulation flow rates on data in a mirror database, and extracting at least one data position corresponding to the data change information.
In practical application, a mirror image database consistent with data in a database in a real environment is established to test services provided in the real environment, so that a real data processing flow of a test image is prevented from being directly based on the data in the database in the real environment, but in the test process, the data in the mirror image database is changed due to the fact that the test is performed according to the simulation flow, and the retest cannot be performed after the change occurs; therefore, in order to meet the requirement of multiple playback of the test traffic, the present embodiment provides an efficient method for full-volume data recovery.
The simulated traffic in the present embodiment includes traffic in the replicated real environment, for example, simulated transaction requests obtained by replicating transaction requests in the real environment, and simulated verification requests obtained by replicating verification requests in the real environment. In the process of processing based on the simulation flow, certain change of the data in the mirror database can be caused, corresponding data change logs can be generated for each data change to record the data change condition, and the data change information of any simulation flow in the embodiment comprises information representing the data change of the simulation flow to the data in the mirror database.
The mirror image database comprises a database for storing and copying data in the database in the real environment; in order to improve the effectiveness of data management in the mirror database, the data in the mirror database can be divided according to data positions, for example, the data in the mirror database is divided according to data positions, that is, each row is taken as a data position; or, carrying out position division on the data in the mirror image database according to the punctuation marks, namely taking one specific punctuation mark to the next specific punctuation mark as a data position; in addition, the data locations may be divided in other manners, which are not limited in this embodiment.
In particular, in order to improve the efficiency of data recovery, the problem that the efficiency of data recovery is affected because data recovery is not needed due to the fact that data recovery is performed at all data positions in the mirror database is avoided, and the data recovery resources are wasted; in this process, since the embodiment is applicable to a simulation environment, and the test processing based on the data in the mirror database is performed by the test platform, the test platform maintains a data change log; in order to improve the convenience of management of the data change log, the configurable test platform can maintain the data change log according to a table form; for example, the stream data change log is stored in the form of a stream data change table, and the transaction record is stored in the form of a transaction data change table, so that convenience and effectiveness of data change log management are improved, and convenience is provided for subsequent data recovery.
The test platform in this embodiment may be a complete data stream transmission and conversion platform integrating online database incremental data capture, distributed data stream storage, schema management, external indexing and internal data computing capabilities.
After obtaining a change log table synchronized by a test platform, firstly judging whether to perform data recovery based on the change log table according to a pre-configured data recovery admittance category and a category label of the change log table, if so, executing steps S202 to S208 based on the change log table to perform data recovery on a mirror database based on the change log table; if not, the processing is not needed.
For example, after a change log table with a class label synchronized by the test platform as a running class is obtained, checking that the running class is not matched with a pre-configured data recovery admittance class, and not processing; for another example, after the change log table with the class label synchronized by the test platform as the balance class is obtained, after verification, the balance class is matched with the pre-configured data recovery admittance class, and then the data recovery is performed on the mirror database based on the change log table with the class label as the balance class.
It should be noted that the preconfigured data recovery admittance category may be a category characterizing a service state, for example, an account category, a balance category, and the like, which are related to the service state.
In addition to directly storing the data change logs based on the categories, after the data change log set synchronized by the test platform is obtained, log screening can be performed based on the data categories of the data change logs in the data change logs, so that the data change logs with the data categories matched with the data recovery admittance categories can be obtained, and the data recovery can be performed based on the data change logs obtained by screening. Alternatively, the data category may be determined based on the data in the data change log.
In a specific execution process, data change information of data change of the mirror database by the plurality of simulation flows is obtained, that is, a process of aggregating the data change information of the simulation flows based on the obtained data change log is obtained, and in an optional implementation provided in this embodiment, the data change information of data change of the mirror database by the plurality of simulation flows is obtained by adopting the following manner:
Acquiring a data change log corresponding to the mirror image database;
and aggregating the data change logs according to the flow marks in the data change logs to obtain data change information of each simulation flow in the plurality of simulation flows.
Optionally, the data change log corresponding to the mirror database includes a data change log matching the data category of the change data with the data recovery admittance category.
Specifically, a data change log meeting the data recovery admittance condition is obtained, namely: and (3) aggregating the data change logs according to the flow marks in the data change logs to obtain the data change information of each simulation flow.
In this embodiment, after the data change log is obtained, on one hand, data change information of each simulation flow is obtained by aggregating the dimensions of the data change log from the simulation flow, that is, the combination of the data change logs corresponding to the simulation flow, and on the other hand, determination of original data of each change data position is required to be performed, so that data recovery is performed based on the original data subsequently; in an alternative implementation manner provided in this embodiment, the determination of the original data of each changed data location is performed in the following manner:
Extracting at least one change data position corresponding to the data change log, and reading a first data change log of each change data position in the data change log;
and determining the original data of each change data position based on the first data change log.
Specifically, the first data change log of each change data position is read from the data change log, and the original data of each change data position is determined based on the first data change log of each change data position.
For example, log 1 is the first data change log for location 1, and based on log 1: deleting the data xxx of the data position 1 according to the flow 1, and determining the original data of the data position 1 as xxx. For another example, log 2 is the first data change log for data location 2, based on log 2: and adding the data aaa of the data position 2 according to the flow 1 to obtain aaaaaa, and determining the original data of the position 2 as aaaa.
In this embodiment, by the above manner, data change information for performing data change on the data in the mirror database by the plurality of simulation flows is obtained, and original data of each changed data position can also be obtained. After the data change information is acquired, extracting the data position corresponding to each data change information, and performing de-duplication processing on the extracted data position to acquire at least one data position.
An alternative embodiment is described in detail below, with reference to fig. 3, and specifically includes the following steps.
Step S202-2, a change log table sent by the test platform is obtained.
Step S202-4, judging whether the class label of the change log table is matched with the data recovery admittance class;
if yes, go to step S202-6 to step S202-8;
if not, the processing is not carried out.
Step S202-6, reading the data change log in the change log table and carrying out aggregation processing on the data change log according to each simulation flow to obtain the data change information of each simulation flow.
Step S202-8, reading the data position in each data change information.
In addition, step S202 may be replaced by obtaining the data change information, extracting at least one data location corresponding to the data change information, determining the original data of each data location, and forming a new implementation manner with other processing steps provided in this embodiment.
Step S204, the data change information is aggregated according to each data position, and position data change information of each data position is obtained.
And the position data change information of any one of the at least one data position is composed of data change information corresponding to the data position in the data change information.
In order to avoid a lot of time consumption caused by sequentially recovering the data of the same data location according to the data change data of the data location, in this embodiment, the data change information of each data location is obtained by performing aggregation processing on the data change information of each data location, the data recovery operation corresponding to each data location is determined based on the position data change information of each data location, and the current data of each data location in the mirror database is recovered based on the data of each data location and the data recovery operation.
In the specific execution process, in the process of carrying out aggregation processing on the data change information of each data position to obtain the position data change information of each data position, carrying out aggregation processing on the data change information according to the extracted data position to obtain the position data change information of each data position. That is, the data change information for data change for the same data position in the data change information is aggregated to obtain the position data change information for each data position.
For example, the extracted data locations include data location 1, data location 2, data location 3, data location 4 and data location 5, and the aggregation processing is performed to obtain location data change information corresponding to the data location 1, where the location data change information includes deletion of data xxx of the data location 1 by the flow 1 and addition of data xxx1 to the data location 1 by the flow 2, and further obtain location data change information corresponding to other data locations.
The obtained positional data change information of each data position may be the same as the form of the data change information or may be different from the form of the data change information, but the present embodiment is not limited thereto, and the positional data change information and the data change information represent the same content.
In addition, step S202 to step S204 may be replaced by obtaining the data change information, and performing aggregation processing on the data change information according to the data positions to obtain the position data change information of each data position, and form a new implementation manner with other processing steps provided in this embodiment.
Step S206, analyzing the position data change information of each data position, obtaining at least one data operation of each data position, and determining a data recovery operation corresponding to the at least one data operation.
After the position data change information of each data position is obtained, the position data change information of each data position is analyzed to obtain at least one data operation of each data position, and a data recovery operation corresponding to the at least one data operation is determined, so that the data recovery operation is obtained by combining a plurality of data operations of each data position and obtaining a corresponding data recovery operation, the one-time data recovery of the plurality of data operations is realized, the data recovery efficiency is improved, and the data recovery event is saved.
In the implementation, after the position data change information of each data position is obtained, the data operation corresponding to each position data change information is read from at least one position data change information of each data position, at least one data operation is obtained, and then the data recovery operation of each data position is determined based on the at least one data operation.
In order to improve the accuracy of data recovery, determining a target data operation of each data position based on at least one data operation for each data position, namely combining a plurality of data operations for each data position into one data operation, and then determining the data operation corresponding to the data operation as a corresponding data recovery operation; in an optional implementation manner provided in this embodiment, in a process of determining a data recovery operation corresponding to at least one data operation, the following operations are performed:
determining a target data operation for each data location based on the at least one data operation;
and reading the data recovery operation corresponding to the target data operation.
Optionally, the target data operation includes a data operation of the current data of the data location compared with the original data obtained after merging the at least one data operation.
Specifically, after determining the target data operation of each data location based on at least one data operation, the data operation corresponding to the target data operation may be determined as the data recovery operation.
The following specifically describes the determination process of the target data operation of any one of the at least one data location, and it should be noted that, for each data location in the at least one data, the determination of the target data operation is performed in a similar manner as follows.
In an optional implementation manner provided in this embodiment, the target data operation of any one of the at least one data location is determined in the following manner:
performing de-duplication processing on at least one data operation of any data position to obtain a position data operation;
if the operation number of the position data operation is one, determining the position data operation as a target data operation of any data position;
and if the operation number of the position data operation is greater than one, sequencing at least one data operation of any data position according to the operation time to obtain an operation sequence, and determining a target data operation based on the operation sequence.
Specifically, at least one data operation of the data position is subjected to de-duplication processing, a position data operation is obtained, if the operation number of the position data operation is 1, the position data operation is determined to be a target data operation, and if the operation number is greater than 1, the target data operation is determined based on an operation sequence of the at least one data operation.
Further, in an alternative implementation manner provided in this embodiment, in determining the target data operation based on the operation sequence, the following operations are performed:
reading the last bit data operation of the operation sequence;
reading a first data operation of the operation sequence under the condition that the last data operation is a first data operation; if the first data operation is the second data operation corresponding to the first data operation, determining that the target data operation is empty; and if the first data operation is not the second data operation, determining the first data operation as a target data operation.
Determining candidate data operation in the position data operation according to priority configuration corresponding to the operation number under the condition that the last data operation is not the first data operation; and determining a target data operation in the candidate data operation based on a determination mode corresponding to the operation number of the candidate data operation.
Optionally, the first data operation may be a delete operation; correspondingly, the second data operation may be a new data operation.
Specifically, in determining the target data operation based on the operation sequence:
if the last data operation of the operation sequence is a deletion operation and the first data operation is a new data operation, determining that the target data operation is empty, namely, not processing;
if the last data operation of the operation sequence is not the deletion operation, determining candidate data operation in the position data operation according to the priority configuration corresponding to the operation number of the position data operation, and determining target data operation in the candidate data operation based on the determination mode corresponding to the operation number of the candidate data operation;
further, in determining candidate data operations in the position data operations according to the priority configuration corresponding to the number of operations of the position data operations:
if the operation number of the position data operation is a preset threshold value, reading a first priority configuration, and determining candidate data operation in the position data operation based on the first priority configuration; optionally, in the first priority configuration, the deleting operation and the adding operation are in a first order, and the modifying operation is in a second order;
If the operation number of the position data operation is larger than the preset threshold value, reading a second priority configuration, and determining candidate data operation in the position data operation based on the second priority configuration; optionally, in the second priority configuration, the modifying operation is at a first level, and the deleting operation and the adding operation are at a second level;
alternatively, the preset threshold may be 2.
Based on the determination mode corresponding to the operation number of the candidate data operation, in the process of determining the target data operation in the candidate data operation:
if the operation number of the candidate data operation is 1, determining the candidate data operation as a target data operation;
if the number of candidate data operations is greater than 1, determining the target data operation as a modification operation.
Further, since the modification operation includes at least three of adding, deleting and replacing, in order to determine an accurate and effective data recovery operation, in determining that the target data operation is the modification operation, determining the target data operation of the data location based on the original data and the current data of the data location; the determined target data operation may be a deletion operation, an addition operation, a deletion operation, or a replacement operation.
After the target data operation is obtained, determining the data operation corresponding to the target data operation as a data recovery operation; optionally, the deleting operation corresponds to the adding operation, the adding operation corresponds to the deleting operation, and the replacing operation corresponds to the replacing operation.
For example, by performing the reprocessing, the position data operation of the data position 1 is obtained as a deletion operation and a new operation, the deletion operation-the new operation is obtained by sorting the times of the deletion operation and the new operation, that is, the deletion operation is performed first and then the new operation is performed, since the last data operation is not the deletion operation, the number of position data operations is equal to 2, and in the first priority configuration, the deletion operation and the new operation are in the same order, the obtained candidate data operation includes two deletion operations and the new operation, the candidate data operation is greater than 1, and thus the original data and the current data of the data position 1 are read, the original data is xxx, the current data is xxx1, the current data is incremented by "1" with respect to the original data, the target data operation of the data position 1 is determined as the increment operation, and the corresponding data recovery operation is determined as the deletion operation.
For another example, the location data operation of the data location 2 is obtained as the new operation and the modification operation through the reprocessing, and since the last location data operation is not the deletion operation and the deletion operation in the first priority configuration is located at the first order, the target data operation of the data location 2 is determined as the new operation and the corresponding data recovery operation is determined as the deletion operation.
For another example, after the reprocessing, the position data operation of the data position 3 is obtained as a new operation and a delete operation, the new operation and the delete operation are ordered according to time, and the new operation-delete operation is obtained, and since the last data operation is the delete operation and the first data operation is the corresponding new operation, the target data operation of the data position 3 is determined to be null, and the corresponding data recovery operation is determined to be null.
For another example, the operation of obtaining the location data of the data location 4 includes an adding operation, a modifying operation and a deleting operation, and the adding operation, the modifying operation and the deleting operation are obtained by sorting according to time, and since the last data operation is the deleting operation and the first data operation is the corresponding adding operation, the target data operation of the data location 4 is determined to be null, and the corresponding data recovery operation is determined to be null.
For another example, the obtaining the location data operation of the data location 5 includes adding an operation, modifying an operation, and deleting an operation, sorting by time, obtaining a delete operation-adding an operation-modifying an operation, reading a second priority configuration according to the number of the location data operations since the last location data operation is not a delete operation, determining the modifying operation as a target data operation of the data location 5 if the first bit in the second priority configuration is a modifying operation, and determining a corresponding data recovery operation; further, if the modification operation is only one modification mode, the modification mode is determined to be the target data operation, and if the modification operation includes at least two modification modes, the target modification mode is determined to be the target data operation based on the original data and the current data of the data position 5.
The new addition in the technical scheme of the application is a process from no to no, the addition is an addition based on data, the deletion is a process from no to no, and the deletion is a reduction of partial data based on data.
It should be further noted that the above provided process of determining the target data operation is merely exemplary, and one target data operation may be determined based on a plurality of data operations according to an actual configuration, which is not described herein again and is not limited thereto.
Step S208, performing data recovery on the current data of each data location in the mirror database based on the data of each data location and the data recovery operation.
The data of each data position comprises the current data and the original data of each data position, and the data recovery is realized by recovering the current data of each data position into the original data of each data position. Optionally, the current data of each data position in the mirror database includes data obtained by data change after test processing based on the simulation flow; after the data is recovered, the data of each data position in the mirror database is original data.
And determining the data recovery operation corresponding to at least one data operation, and in this step, performing data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
In a specific implementation, the simulation environment needs to restore data in the mirror database, and based on this, in an optional implementation provided in this embodiment, based on the data of each data location and a data restoration operation, in a process of restoring current data of each data location in the mirror database, the following operations are executed:
reading the original data and the current data of each data position;
generating an operation expression of each data position based on the original data, the current data and the data recovery operation of each data position;
and executing the operation expression to restore the current data of each data position in the mirror database.
Specifically, after the data recovery operation of each data location is obtained, an operation expression of each data location is generated based on the original data, the current data and the data recovery operation of each data location, and the operation expression is executed to perform data recovery on the current data of each data location in the mirror database.
Alternatively, the operation expression may be an sql ((Structured Query Language) structured query language) expression, or may be another expression for performing data processing on the database, which is not limited herein.
In the process of executing the operation expression to restore the current data of each data position in the mirror database, the operation expression can be executed based on the data restoring mode to restore the current data of each data position in the mirror database; alternatively, the data recovery method is, for example, JDBC (Java DataBase Connectivity, java database connection) direct database method.
After data recovery is performed on the data at each data location in the mirror database, the data at each data location in the mirror database is original data, and a test request can be performed based on the original data, and in an optional implementation provided in this embodiment, the following manner is adopted to implement the test:
acquiring a test request, and reading corresponding test data from the mirror image database based on the test data position carried in the test request;
and performing test processing according to the test data.
It should be noted that the read test data includes the original data of the test data location.
It should be further noted that, the data recovery method provided in this embodiment may be performed when a preset time point arrives, for example, the data in the mirror database is to be tested during the day, and the data in the mirror database is recovered at eight points in the evening every day, so that the test process may be performed on the basis of the original data in the next day.
In summary, in the data restoration method provided in this embodiment, for data in the mirror database, by aggregating a plurality of simulated flows with respect to data change information of the same data location, determining a corresponding one of data restoration operations by at least one data operation in the location data change information of each data location based on the obtained location data change information of each data location, and then restoring the current data of each data location in the mirror database to the original data based on the original data, the current data and the data restoration operations of each data location, so as to refine the original data of the data location by sensing the data location commonly affected by the plurality of flows, performing accurate reverse derivation based on the original data and the current data, obtaining one data restoration operation, performing data restoration on the current data in the mirror database based on the one data restoration operation, and implementing DB (DoggaByte) level baseline restoration.
The following further describes the data recovery method provided in this embodiment by taking an application of the data recovery method provided in this embodiment to a data recovery scene of a mirror database as an example, and referring to fig. 4, the data recovery method applied to the data recovery scene of the mirror database specifically includes the following steps.
Step S402, a data change log for carrying out data change on the data in the mirror database is obtained.
Step S404, the data change logs are aggregated according to the simulated flow to obtain the data change information of each data flow.
In step S406, the first data change log of each data location is read from the data change log, and the original data of each data location is determined based on the first change log.
Alternatively, the execution order of step S404 and step S406 is not limited.
Step S408, the data change information is aggregated according to each data location, so as to obtain location data change information of each data location.
Alternatively, step S404 and step S408 may be replaced by performing aggregation processing on the data change log according to the data location, to obtain location data change information (data change information) of each data location.
Step S410, determining a corresponding data recovery operation based on at least one data operation in the position data change information of each data position.
Step S412, generating the data recovery strategy of each data position according to the data recovery operation of each data position and the original data of each data position.
Step S414, executing the data recovery strategy of each data position, and carrying out data recovery on the current data of each data position in the mirror database in a pre-configured data recovery mode.
One or more embodiments of a data recovery device provided in the present specification are as follows:
in the above-described embodiments, a data recovery method is provided, and a data recovery apparatus is provided corresponding thereto, and is described below with reference to the accompanying drawings.
Referring to fig. 5, a schematic diagram of an embodiment of a data recovery device provided in this embodiment is shown.
Since the apparatus embodiments correspond to the method embodiments, the description is relatively simple, and the relevant portions should be referred to the corresponding descriptions of the method embodiments provided above. The device embodiments described below are merely illustrative.
The present embodiment provides a data recovery apparatus, including:
The information obtaining module 502 is configured to obtain data change information of data change of the data in the mirror database by the plurality of simulation flow rates, and extract at least one data position corresponding to the data change information;
an aggregation processing module 504 configured to aggregate the data change information according to each data location, and obtain location data change information of each data location;
an information parsing module 506 configured to parse the location data change information of each data location, obtain at least one data operation of each data location, and determine a data recovery operation corresponding to the at least one data operation;
and the data recovery module 508 is configured to perform data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
One or more embodiments of a data recovery device provided herein are as follows:
in correspondence to the above-described data recovery method, one or more embodiments of the present disclosure further provide a data recovery device, based on the same technical concept, for performing the above-provided data recovery method, and fig. 6 is a schematic structural diagram of a data recovery device provided by one or more embodiments of the present disclosure.
The data recovery device provided in this embodiment includes:
as shown in fig. 6, the data recovery device may have a relatively large difference due to different configurations or performances, and may include one or more processors 601 and a memory 602, where the memory 602 may store one or more storage applications or data. Wherein the memory 602 may be transient storage or persistent storage. The application programs stored in the memory 602 may include one or more modules (not shown), each of which may include a series of computer-executable instructions in the data recovery device. Still further, the processor 601 may be arranged to communicate with the memory 602 and execute a series of computer executable instructions in the memory 602 on a data recovery device. The data recovery device may also include one or more power supplies 603, one or more wired or wireless network interfaces 604, one or more input/output interfaces 605, one or more keyboards 606, and the like.
In a particular embodiment, a data recovery device includes a memory, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions in the data recovery device, and configured to be executed by one or more processors, the one or more programs comprising computer-executable instructions for:
Acquiring data change information of data change of a plurality of simulation flow rates on data in a mirror image database, and extracting at least one data position corresponding to the data change information;
performing aggregation processing on the data change information according to each data position to obtain position data change information of each data position;
analyzing the position data change information of each data position, obtaining at least one data operation of each data position, and determining a data recovery operation corresponding to the at least one data operation;
and carrying out data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
One or more embodiments of a storage medium provided in the present specification are as follows:
corresponding to the above-described data recovery method, one or more embodiments of the present disclosure further provide a storage medium based on the same technical concept.
The storage medium provided in this embodiment is configured to store computer executable instructions that, when executed by a processor, implement the following flow:
acquiring data change information of data change of a plurality of simulation flow rates on data in a mirror image database, and extracting at least one data position corresponding to the data change information;
Performing aggregation processing on the data change information according to each data position to obtain position data change information of each data position;
analyzing the position data change information of each data position, obtaining at least one data operation of each data position, and determining a data recovery operation corresponding to the at least one data operation;
and carrying out data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
It should be noted that, the embodiments related to the storage medium in the present specification and the embodiments related to the data recovery method in the present specification are based on the same inventive concept, so that the specific implementation of this embodiment may refer to the implementation of the foregoing corresponding method, and the repetition is not repeated.
In this specification, each embodiment is described in a progressive manner, and the same or similar parts of each embodiment are referred to each other, and each embodiment focuses on the differences from other embodiments, for example, an apparatus embodiment, and a storage medium embodiment, which are all similar to a method embodiment, so that description is relatively simple, and relevant content in reading apparatus embodiments, and storage medium embodiments is referred to the part description of the method embodiment.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
In the 30 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented by using "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but not just one of the hdds, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.
The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC 625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each unit may be implemented in the same piece or pieces of software and/or hardware when implementing the embodiments of the present specification.
One skilled in the relevant art will recognize that one or more embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
One or more embodiments of the present specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. One or more embodiments of the specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.
The foregoing description is by way of example only and is not intended to limit the present disclosure. Various modifications and changes may occur to those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. that fall within the spirit and principles of the present document are intended to be included within the scope of the claims of the present document.
Claims (13)
1. A data recovery method, comprising:
acquiring data change information of data change of a plurality of simulation flow rates on data in a mirror image database, and extracting at least one data position corresponding to the data change information;
performing aggregation processing on the data change information according to each data position to obtain position data change information of each data position;
analyzing the position data change information of each data position, obtaining at least one data operation of each data position, and determining a data recovery operation corresponding to the at least one data operation;
And carrying out data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
2. The data recovery method according to claim 1, wherein the obtaining data change information for performing data change on data in the mirror database by the plurality of simulation flows includes:
acquiring a data change log corresponding to the mirror image database;
and aggregating the data change logs according to the flow marks in the data change logs to obtain data change information of each simulation flow in the plurality of simulation flows.
3. The data recovery method of claim 2, further comprising:
extracting at least one change data position corresponding to the data change log, and reading a first data change log of each change data position in the data change log;
and determining the original data of each change data position based on the first data change log.
4. The data recovery method according to claim 1, wherein the determining a data recovery operation corresponding to the at least one data operation includes:
determining a target data operation for each data location based on the at least one data operation;
And reading the data recovery operation corresponding to the target data operation.
5. The data recovery method of claim 4, wherein the target data operation for any of the at least one data location is determined by:
performing de-duplication processing on at least one data operation of any data position to obtain a position data operation;
if the operation number of the position data operation is one, determining the position data operation as a target data operation of any data position;
and if the operation number of the position data operation is greater than one, sequencing at least one data operation of any data position according to the operation time to obtain an operation sequence, and determining a target data operation based on the operation sequence.
6. The data recovery method of claim 5, the determining a target data operation based on the sequence of operations comprising:
reading the last bit data operation of the operation sequence;
reading a first data operation of the operation sequence under the condition that the last data operation is a first data operation;
if the first data operation is the second data operation corresponding to the first data operation, determining that the target data operation is empty;
And if the first data operation is not the second data operation, determining the first data operation as a target data operation.
7. The data recovery method of claim 6, further comprising:
determining candidate data operation in the position data operation according to priority configuration corresponding to the operation number under the condition that the last data operation is not the first data operation;
and determining a target data operation in the candidate data operation based on a determination mode corresponding to the operation number of the candidate data operation.
8. The data recovery method according to claim 1, wherein the performing data recovery on the current data of each data location in the mirror database based on the data of each data location and the data recovery operation includes:
reading the original data and the current data of each data position;
generating an operation expression of each data position based on the original data, the current data and the data recovery operation of each data position;
and executing the operation expression to restore the current data of each data position in the mirror database.
9. The data recovery method according to claim 1, wherein the current data of each data location in the mirror database includes data obtained by data modification after test processing based on a simulation flow;
After the data is recovered, the data of each data position in the mirror database is original data.
10. The data recovery method according to claim 1, wherein after the performing the data recovery step on the current data of each data location in the mirror database based on the data of each data location and the data recovery operation, the method further comprises:
acquiring a test request, and reading corresponding test data from the mirror image database based on the test data position carried in the test request;
and performing test processing according to the test data.
11. A data recovery apparatus comprising:
the information acquisition module is configured to acquire data change information of data change of the data in the mirror image database by a plurality of simulation flow rates and extract at least one data position corresponding to the data change information;
the aggregation processing module is configured to aggregate the data change information according to each data position to obtain position data change information of each data position;
the information analysis module is configured to analyze the position data change information of each data position, obtain at least one data operation of each data position, and determine a data recovery operation corresponding to the at least one data operation;
And the data recovery module is configured to perform data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
12. A data recovery apparatus comprising:
a processor; and a memory configured to store computer-executable instructions that, when executed, cause the processor to:
acquiring data change information of data change of a plurality of simulation flow rates on data in a mirror image database, and extracting at least one data position corresponding to the data change information;
performing aggregation processing on the data change information according to each data position to obtain position data change information of each data position;
analyzing the position data change information of each data position, obtaining at least one data operation of each data position, and determining a data recovery operation corresponding to the at least one data operation;
and carrying out data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
13. A storage medium storing computer-executable instructions that when executed by a processor implement the following:
Acquiring data change information of data change of a plurality of simulation flow rates on data in a mirror image database, and extracting at least one data position corresponding to the data change information;
performing aggregation processing on the data change information according to each data position to obtain position data change information of each data position;
analyzing the position data change information of each data position, obtaining at least one data operation of each data position, and determining a data recovery operation corresponding to the at least one data operation;
and carrying out data recovery on the current data of each data position in the mirror database based on the data of each data position and the data recovery operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311830457.0A CN117762693A (en) | 2023-12-27 | 2023-12-27 | Data recovery method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311830457.0A CN117762693A (en) | 2023-12-27 | 2023-12-27 | Data recovery method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117762693A true CN117762693A (en) | 2024-03-26 |
Family
ID=90318006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311830457.0A Pending CN117762693A (en) | 2023-12-27 | 2023-12-27 | Data recovery method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117762693A (en) |
-
2023
- 2023-12-27 CN CN202311830457.0A patent/CN117762693A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110705214A (en) | Automatic coding method and device | |
| CN110503435B (en) | Transaction early warning method, device and equipment based on blockchain | |
| CN110569428A (en) | recommendation model construction method, device and equipment | |
| CN110635962B (en) | Abnormity analysis method and device for distributed system | |
| CN111930810B (en) | Data rule mining method and device | |
| CN111753328B (en) | Private data leakage risk detection method and system | |
| CN110888756A (en) | Diagnostic log generation method and device | |
| CN110008419A (en) | Removing duplicate webpages method, device and equipment | |
| CN116610568A (en) | Method, device, equipment and medium for identifying dependency relationship of codes | |
| CN110245137B (en) | Index processing method, device and equipment | |
| CN110245136B (en) | Data retrieval method, device, equipment and storage equipment | |
| CN109582300A (en) | Code analysis on altered project method, device and equipment based on path | |
| CN109614393A (en) | Verification of data method and device | |
| CN117421214A (en) | Batch counting method, device, electronic equipment and computer readable storage medium | |
| CN116048977B (en) | Test method and device based on data reduction | |
| CN110008252B (en) | Data checking method and device | |
| CN116822606A (en) | Training method, device, equipment and storage medium of anomaly detection model | |
| CN110119442A (en) | A kind of dynamic searching method, device, equipment and medium | |
| CN110502551A (en) | Data read-write method, system and infrastructure component | |
| CN117762693A (en) | Data recovery method and device | |
| CN115391015A (en) | Batch processing method and device based on test framework, electronic equipment and medium | |
| CN111967767A (en) | Business risk identification method, device, equipment and medium | |
| CN112182510B (en) | Method, device and equipment for measuring product coverage degree | |
| CN117743189A (en) | Code analysis method and device | |
| CN117035695B (en) | Information early warning method and device, readable storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |