CN114461454A - Data recovery method and device, storage medium and electronic equipment - Google Patents
Data recovery method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN114461454A CN114461454A CN202210128039.6A CN202210128039A CN114461454A CN 114461454 A CN114461454 A CN 114461454A CN 202210128039 A CN202210128039 A CN 202210128039A CN 114461454 A CN114461454 A CN 114461454A
- Authority
- CN
- China
- Prior art keywords
- data
- statement
- rollback
- manipulation
- transaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011084 recovery Methods 0.000 title claims abstract description 82
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000004458 analytical method Methods 0.000 claims abstract description 95
- 230000004044 response Effects 0.000 claims abstract description 6
- 238000004590 computer program Methods 0.000 claims description 9
- 238000003780 insertion Methods 0.000 claims description 5
- 230000037431 insertion Effects 0.000 claims description 5
- 238000012217 deletion Methods 0.000 claims description 3
- 230000037430 deletion Effects 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000012545 processing Methods 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 239000002253 acid Substances 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 231100000957 no side effect Toxicity 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000010926 purge Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/253—Grammatical analysis; Style critique
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/30—Arrangements for executing machine instructions, e.g. instruction decode
- G06F9/38—Concurrent instruction execution, e.g. pipeline or look ahead
- G06F9/3861—Recovery, e.g. branch miss-prediction, exception handling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/466—Transaction processing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Quality & Reliability (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure relates to the field of computer processing technologies, and in particular, to a data recovery method, a data recovery apparatus, a storage medium, and an electronic device. The data recovery method comprises the following steps: when the opening of the transaction is detected, analyzing the data manipulation language of the transaction to obtain an analysis result; generating a manipulation rollback statement of the data manipulation language according to the analysis result to obtain rollback information of the transaction; and acquiring the rollback information to traverse and execute the manipulation rollback statement in response to a data recovery instruction of the transaction. The data recovery method provided by the disclosure can enable the data recovery applicability to be strong and is convenient to manage and maintain.
Description
Technical Field
The present disclosure relates to the field of computer processing technologies, and in particular, to a data recovery method, a data recovery apparatus, a storage medium, and an electronic device.
Background
The internet produces massive data at present, and data storage can bring the performance bottleneck in a database, and data often can be stored to a plurality of databases by the split, and an operational data falls to a plurality of databases, means to a plurality of database operations, and the affair can not be guaranteed to database itself, consequently can appear unpredictable condition often in the implementation process, leads to the data error, needs the recovery.
Data recovery has no uniform standard and needs to be processed by a developer, the granularity of Data recovery technology of each database product is coarse, and a Data Management Language (DML) -based operation Data recovery technology is rarely provided, so that the developer has to develop reverse business recovery Data.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The present disclosure aims to provide a data recovery method, a data recovery device, a storage medium, and an electronic apparatus, and aims to solve the problems of high applicability of data recovery and convenience in management and maintenance.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.
According to an aspect of the embodiments of the present disclosure, there is provided a data recovery method, including: when the opening of the transaction is detected, analyzing the data manipulation language of the transaction to obtain an analysis result; generating a manipulation rollback statement of the data manipulation language according to the analysis result to obtain rollback information of the transaction; and acquiring the rollback information to traverse and execute the manipulation rollback statement in response to a data recovery instruction of the transaction.
According to some embodiments of the present disclosure, based on the foregoing solution, the parsing the data manipulation language of the transaction to obtain a parsing result includes: performing lexical analysis on the data manipulation language to obtain a lexical analysis result; and carrying out syntactic analysis on the lexical analysis result by using an analysis tool syntactic analysis generator, and generating an abstract syntactic tree as the analysis result.
According to some embodiments of the present disclosure, based on the foregoing solution, the generating a manipulation rollback statement of the data manipulation language according to the parsing result includes: when the data manipulation language is an insert statement, generating the manipulation rollback statement according to a deletion statement grammar according to a data table and condition information in the analysis result; when the data manipulation language is an update statement, generating the manipulation rollback statement according to a data table, field information and condition information in the analysis result and according to an update statement grammar; or when the data manipulation language is a delete statement, generating the manipulation rollback statement according to the insertion statement syntax according to the data table in the analysis result.
According to some embodiments of the present disclosure, based on the foregoing scheme, when the data manipulation language is an update statement or a delete statement, the method further includes: generating a query statement according to the parsing result and the query statement syntax; executing the query statement to obtain operation data information; saving the manipulation data information to a multi-version concurrency control to generate manipulation snapshot data, and adding the manipulation snapshot data to the rollback information.
According to some embodiments of the present disclosure, based on the foregoing scheme, when the data manipulation language is an update statement and a primary key field is missing in the parsing result of the update statement, the method further includes: and adding a main key field to generate the query statement according to the main key field and the analysis result.
According to some embodiments of the disclosure, based on the foregoing scheme, when the data manipulation language is a delete statement, the method further comprises: and inquiring all field information of a data table in the analysis result so as to generate the inquiry statement according to all the field information and the analysis result.
According to some embodiments of the present disclosure, based on the foregoing scheme, the obtaining the rollback information to traverse the manipulation rollback statement includes: and acquiring the rollback information to traverse and execute the manipulation rollback statement of the data manipulation language based on the manipulation snapshot data of the data manipulation language.
According to a second aspect of the embodiments of the present disclosure, there is provided a data recovery apparatus including: the analysis module is used for analyzing the data manipulation language of the transaction to obtain an analysis result when the transaction is detected to be started; the rollback module is used for generating a manipulation rollback statement of the data manipulation language according to the analysis result so as to obtain rollback information of the transaction; and the recovery module is used for responding to a data recovery instruction of the transaction, and acquiring the rollback information to traverse and execute the manipulation rollback statement.
According to a third aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the data recovery method as in the above embodiments.
According to a fourth aspect of the embodiments of the present disclosure, there is provided an electronic apparatus, including: one or more processors; a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the data recovery method as in the above embodiments.
Exemplary embodiments of the present disclosure may have some or all of the following benefits:
in the technical scheme provided by some embodiments of the disclosure, when a transaction is started, an analysis result is obtained by analyzing a data manipulation language in a forward direction, and then a reverse rollback statement is generated according to the analysis result to obtain rollback information, so that when the transaction needs to be restored, the manipulation rollback statement can be executed according to the rollback information to complete data restoration; on the other hand, data recovery control is performed by using a general logic for generating rollback information according to an analysis result to perform distribution execution of data recovery, so that development efficiency can be improved, and maintenance is facilitated.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:
fig. 1 schematically illustrates a flow chart of a data recovery method in an exemplary embodiment of the present disclosure;
FIG. 2 schematically illustrates a diagram of an abstract syntax tree in an exemplary embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow diagram of a method of generating a manipulated rollback statement in an exemplary embodiment of the present disclosure;
FIG. 4 schematically illustrates a flow chart of a data recovery method in an exemplary embodiment of the present disclosure;
FIG. 5 is a schematic diagram illustrating components of a data recovery apparatus according to an exemplary embodiment of the present disclosure;
FIG. 6 schematically illustrates a schematic diagram of a computer-readable storage medium in an exemplary embodiment of the disclosure;
fig. 7 schematically shows a structural diagram of a computer system of an electronic device in an exemplary embodiment of the disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the embodiments of the disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known methods, devices, implementations, or operations have not been shown or described in detail to avoid obscuring aspects of the disclosure.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
The internet generates massive data, but unpredictable conditions often occur in the implementation process, so that data errors occur and recovery is needed. Data recovery does not have a uniform standard and needs to be processed by a developer, the granularity of the data recovery technology of each database product is relatively coarse, and the DML-based operation data recovery technology is rarely provided, so that the developer has to develop reverse business recovery data.
In the prior art, data recovery can be performed by analyzing binlog of mysql in a log, but only a few databases are in an open log format, and a commercial database is not in an open log data format, so that data recovery of all databases cannot be realized by analyzing log recovery data; still adopt oracle database flash back technique, but this kind of technique needs to use specific sql statement, if the table structure is modified before the flash back technique recovers the data, can not recover the data, and only oracle database has this technique; in addition, a business developer can develop reverse business logic to recover data, and the reverse logic invades each business, but the method cannot provide a toolkit and is not universal.
Therefore, aiming at the defects in the prior art, the present disclosure provides a general data recovery method, which generates a rollback statement using a DML statement analysis result by performing forward analysis on business operation data, generates a query statement using a DML statement analysis result when a data manipulation language is an update statement or a delete statement, and generates a snapshot using the query statement. When the transaction is abnormally interrupted, the transaction snapshot is quickly recovered through a rollback mechanism, and the characteristics of the database transaction are ensured.
Implementation details of the technical solution of the embodiments of the present disclosure are set forth in detail below.
Fig. 1 schematically illustrates a flow chart of a data recovery method in an exemplary embodiment of the present disclosure. As shown in fig. 1, the data recovery method includes steps S101 to S103:
step S101, when detecting that the transaction is started, analyzing a data manipulation language of the transaction to obtain an analysis result;
step S102, generating a manipulation rollback statement of the data manipulation language according to the analysis result to obtain rollback information of the transaction;
step S103, in response to the data recovery instruction of the transaction, obtaining the rollback information to traverse and execute the manipulation rollback statement.
In the technical scheme provided by some embodiments of the present disclosure, when a transaction is started, an analysis result is obtained by analyzing a data manipulation language in a forward direction, and then a reverse rollback statement is generated according to the analysis result to obtain rollback information, so that when the transaction needs to be restored, the manipulation rollback statement can be executed according to the rollback information to complete data restoration, on one hand, the data manipulation language is analyzed in a general manner, the compatibility of language analysis independent of a database is good, the method is suitable for each database, and meanwhile, database data restoration under a heterogeneous database is supported; on the other hand, data recovery control is performed by using a general logic for generating rollback information according to an analysis result to perform distribution execution of data recovery, so that development efficiency can be improved, and maintenance is facilitated.
Hereinafter, each step of the data recovery method in the present exemplary embodiment will be described in more detail with reference to the drawings and examples.
In step S101, when it is detected that a transaction is started, a data manipulation language of the transaction is parsed to obtain a parsing result.
Wherein a transaction (transaction) in the database is a collection of operations that constitute a single logical unit of work. A transaction may contain one or more database operations, but these operations form a logical whole, and these database operations that form the logical whole are either all successfully executed or all not executed. In order to ensure that a transaction is correct and reliable during a data writing or updating process of a database management system (DBMS), ACID characteristics, i.e., atomicity (or indivisible), consistency (consistency), isolation (or independence), and durability (durability), must be provided.
Therefore, when the detected transaction is opened, a series of database operations of the transaction can be acquired to obtain a data Manipulation language DML (data Manipulation language), and the DML statements are forward operation statements of the business.
And then, the DML statement is analyzed to obtain an analysis result, and then the reverse statement of the service can be generated according to the analysis result for data recovery.
In step S102, a manipulation rollback statement of the data manipulation language is generated according to the parsing result, so as to obtain rollback information of the transaction.
The operation rollback statement is a statement for data recovery, and data can be recovered by executing the operation rollback statement.
Specifically, the content of the generated steering rollback statement is different due to the difference of the DML statements. For example, DML statements mainly include Update statements, Insert statements, and Delete statements.
When the manipulation rollback statement is generated, for example, data a is updated to data B for the Update statement, then the corresponding manipulation rollback statement should also be the Update statement, and data B is updated to data a; for an Insert statement, for example, Insert data C, then the corresponding rollback manipulation statement should be a Delete statement, deleting data C; for a Delete statement, for example, data D is deleted, then the corresponding rollback manipulation statement should be an Insert statement, inserting data D.
And obtaining the rollback information of the transaction when the operation rollback statement is generated. It should be noted that the manipulation rollback statement is integrated into rollback information in transaction units, i.e., a complete transaction contains multiple rows of manipulation rollback statements. When the transaction execution is finished, the rollback information can be cleared and can be additionally stored to other positions.
In step S103, in response to the data recovery instruction of the transaction, the rollback information is acquired to traverse the execution of the manipulation rollback statement.
The data recovery instruction triggering the transaction is generated when the transaction operation is abnormal, for example, when the transaction operation cannot be completed normally due to extreme conditions such as data conflict, network interruption, abnormal downtime, and the like.
Each piece of forward operation data of the transaction is analyzed in advance to obtain rollback information, so that when the transaction needs data recovery, the rollback information of the transaction can be extracted, a control rollback statement is executed for each piece of operation data according to the rollback information, and the rollback operation is completed through a reverse statement, a database route and a database execution module so as to achieve the effect of data recovery.
In an embodiment of the present disclosure, when step S101 is executed, the specific content of parsing the data manipulation language of the transaction to obtain a parsing result is as follows: performing lexical analysis on the data manipulation language to obtain a lexical analysis result; and carrying out syntactic analysis on the lexical analysis result by using an analysis tool syntactic analysis generator, and generating an abstract syntactic tree as the analysis result.
Firstly, lexical analysis is carried out on DML sentences of the data manipulation language.
Lexical Analysis (Lexical Analysis) is a process in computer science that converts a sequence of characters into a sequence of words (Token). The rules of lexical analysis are relatively simple, the rules of the lexical units are well defined, branch selection rarely occurs, and the lexical units can be determined only by searching one character in advance. However, lexical analysis is the basis of syntactic analysis, and if word segmentation is wrong, syntactic analysis is difficult to process correctly.
Lexical parsing can be accomplished using a lexical analyzer, also called a scanner, which typically exists in the form of a function for invocation by a parser.
Then, grammar parsing is performed based on the lexical parsing result.
In one embodiment of the present disclosure, a parser can be utilized to convert a data manipulation language into an abstract syntax tree. There are many third party tools for generating abstract syntax trees and ANTLR is a good choice. It generates the Java code of the abstract syntax tree from the rules defined by the developer and provides the visitor interface.
Specifically, ANTLR may be used as an engine for SQL parsing, and it is planned to sequentially replace the original parsing engine according to the order of DDL → TCL → DAL → DCL → DML → DQL.
Fig. 2 schematically illustrates a diagram of an abstract syntax tree in an exemplary embodiment of the present disclosure. Taking the DML statement "SELECT id, name FROM t _ user WHERE status ═ ACTIVE' AND > 18" as an example, the abstract syntax tree after parsing is shown in fig. 2. Referring to fig. 2, for convenience of explanation, the Token of the keyword in the abstract syntax tree is represented by a solid box, such as SELECT, FROM, etc., the Token of the variable is represented by a dotted box, such as id, name, etc., and the dotted box represents a word that needs to be further split, such as FIELDS, tabes, etc.
The data table, the field information, the condition information and the like related to the DML statement can be acquired through the abstract syntax tree. Using the abstract syntax tree shown in fig. 2, for example, the data table is "t _ user", the field information is "id" AND "name", AND the condition information is "ACTIVE' AND > 18".
Compared with the existing SQL analysis engines, such as jsqlpaser, pipeline and the like, when SQL compatibility is not comprehensive, errors can be directly reported when the SQL analysis engines are used, the ANTLR serving as the syntax analysis engine can be better compatible with the SQL, and the friendliness of analysis and understanding of SQL sentences such as complex expressions, recursion and sub-queries is greatly improved.
It should be noted that, compared to code generation, the handwriting abstract syntax tree is more efficient in terms of execution efficiency, but the workload is also larger. Therefore, in a scenario with high performance requirements, a customized abstract syntax tree may be considered. The analysis process may select a suitable method as needed, and the disclosure is not limited in detail herein.
When step S102 is executed, the specific content of the manipulation rollback statement of the data manipulation language generated according to the parsing result is as follows:
when the data manipulation language is an insert statement, generating the manipulation rollback statement according to a deletion statement grammar according to a data table and condition information in the analysis result; when the data manipulation language is an update statement, generating the manipulation rollback statement according to a data table, field information and condition information in the analysis result and according to an update statement grammar; or when the data manipulation language is a delete statement, generating the manipulation rollback statement according to the insertion statement syntax according to the data table in the analysis result.
In an embodiment of the present disclosure, when the data manipulation language is an insert statement, the manipulation rollback statement is generated according to a delete statement syntax according to a data table and condition information in the parsing result.
Insert is the addition of data, no data in the database before execution. The rollback statement for Insert is Delete, for example, the generated rollback statement is: from product where id? .
In another embodiment of the present disclosure, when the data manipulation language is an update statement, the manipulation rollback statement is generated according to an update statement syntax based on the data table, the field information, and the condition information in the parsing result.
The rollback of the update statement may be referred to as follows, for example: the update statement is: update product set name is 'handset', print is 100where id is 10. And analyzing the DML statement to obtain an analysis result: "operation data table: product; operation field information: name, price; operating condition information: and id is 10 ". The rollback statement for Update remains Update, then the resulting manipulation rollback statement: is update product set name? Price? where id? . Wherein "? "precompiled placeholders for jdbc ((Java Data Base Connectivity, Java database Connectivity).
In a further embodiment of the present disclosure, when the data manipulation language is a delete statement, the manipulation rollback statement is generated according to an insert statement syntax from a data table in the parsing result.
The rollback of the delete statement may be referred to as follows, for example: the delete statement is: delete from product name ═ cell phone'; and analyzing the DML statement to obtain an analysis result: "operation data table: product; operating condition information: name ═ cell phone'. The rollback operation of Delete is Insert, and the generated rollback statement is: insert in product values (.
FIG. 3 schematically illustrates a flowchart of a method of generating a manipulation rollback statement in an exemplary embodiment of the present disclosure. In an embodiment of the present disclosure, when the data manipulation language is an update statement or a delete statement, a specific step of generating a method for manipulating a rollback statement is shown in fig. 3, and includes:
step S301, generating a query statement according to the parsing result and the query statement syntax;
step S302, executing the query statement to obtain operation data information;
step S303, saving the manipulation data information to a multi-version concurrency control to generate manipulation snapshot data, and adding the manipulation snapshot data to the rollback information.
Specifically, when the data manipulation language is an update statement or a delete statement, the database has data before the operation, and therefore, snapshot data during the operation needs to be saved as well, so that the manipulation rollback statement has performability.
In step S301, data processed at the time of data operation is first queried, and a query statement may be generated according to the query statement syntax based on the parsing result.
In one embodiment of the present disclosure, when the data manipulation language is an update statement and a primary key field is missing in the parsing result of the update statement, the method further comprises: and adding a main key field to generate the query statement according to the main key field and the analysis result.
Specifically, the primary key field is used for subsequently generating the manipulation rollback statement, so when the field information in the parsing result has no primary key, the primary key field needs to be added to the query list, and then the manipulation rollback statement can be subsequently generated based on the primary key field for data recovery.
The step of generating the query statement by the Update statement comprises the following steps: (1) analyzing the table name of the operation statement; (2) analyzing an operation field; (3) resolving a where condition; (4) and generating the query statement according to the query statement grammar.
For example, reference examples are as follows: the update statement is: update product set name is 'handset', print is 100where id is 10. And analyzing the DML statement to obtain an analysis result: "operation data table: product; operation field information: name, price; operating condition information: and id is 10 ". And generating a query statement: select name, price, id from product where id is 10. Here, the data table used in data recovery relates to "id", so the "id" field needs to be added when generating the query statement.
Based on the method, the query statement is generated by the main key after the main key field is added, so that the control snapshot data is obtained, the problems of full-table scanning or broadcast query and the like caused by the fact that the data table has no index II and uses other columns for rollback operation are avoided, the operation data cannot be expanded, the stability of the database is improved, no side effect exists, the service details do not need to be concerned during data recovery, and the integration is convenient.
In one embodiment of the present disclosure, when the data manipulation language is a delete statement, the method further comprises: and inquiring all field information of a data table in the analysis result so as to generate the inquiry statement according to all the field information and the analysis result.
Specifically, the rollback operation of Delete is Insert, the Insert contains all fields, the Insert statement can be generated only according to the number of the columns of the query result, and the generated Insert contains all fields and does not need to specify the field names. Therefore, when generating the manipulation snapshot data, all fields of the operation data table need to be queried.
A Delete statement generating query statement step: (1) analyzing the deleted table name; (2) inquiring all fields of the operation table; (3) resolving a where condition; (4) and generating the query statement according to the query statement grammar.
For example, reference examples are as follows: the Delete statement is: the delete from product name ═ mobile phone', analyzes the DML statement to obtain an analysis result: "operation data table: product; operating condition information: name ═ cell phone'. And generating a query statement: a select from product name is 'handset'. Wherein, replacing the query field, all the fields are queried.
In step S303, the manipulation data information is saved to multi-version concurrency control to generate manipulation snapshot data, and the manipulation snapshot data is added to the rollback information.
After the operation data information is obtained after the query statement is executed, the query result is stored in the MVCC (Multi-Version Concurrency Control), and is used as a data snapshot of a concurrent query request Select on one hand and undo data of a Delete transaction rollback on the other hand, so that the data snapshot is used as a data basis for data recovery.
It should be noted that steps S401 to S403 may be completed in an actual kernel parsing process, and after completing the lexical and grammatical parsing of the DML statement of the structured query language, the kernel parser first rewrites and generates a query statement, and then performs a query through the executor, and stores the result in the MVCC multi-version concurrency control. If the Delete transaction execution succeeds, then the snapshot version of mvcc is deleted by the purge thread.
In one embodiment of the present disclosure, the obtaining the rollback information to traverse the executing of the manipulation rollback statement includes: and acquiring the rollback information to traverse and execute the manipulation rollback statement of the data manipulation language based on the manipulation snapshot data of the data manipulation language.
When the operation rollback statement is executed, traversing each operation snapshot data, executing the operation rollback statement of the row according to the operation snapshot data in the single row of data, further acquiring the version data before the execution of the corresponding transaction, and thus finishing the execution of the operation rollback statement according to the acquired data.
Based on the method, the rollback information comprises the accessible reverse operation statements and the manipulated snapshot data, a unified recovery scheme is adopted, the development efficiency of the data recovery scheme is improved, and the maintenance is convenient.
Therefore, according to the above disclosure, a rollback statement is generated by forward parsing the service data using the DML statement parsing result. And simultaneously, generating a query statement by using a DML statement analysis result, and generating a manipulation data snapshot by using the query statement. When the transaction is abnormally interrupted, the rapid recovery of the transaction snapshot is realized through a rollback mechanism, and the ACID characteristic of the database transaction is ensured.
Fig. 4 is a schematic flow chart illustrating a data recovery method according to an exemplary embodiment of the disclosure, and referring to fig. 4, a complete data recovery method includes the following steps:
step S401: initializing a service, and opening a method for generating a rollback statement at the moment; step S402, executing DML operation as normal operation; step S403, analyzing the acquired DML statement; step S404, generating a query statement and a rollback statement according to the analysis result; step S405, generating a snapshot; step S406, determining whether the DML operation is successfully executed, if not, executing step S407, that is, executing a rollback statement according to the snapshot to perform data recovery, and then executing step S408; if the execution is successful, directly execute step S408; step S408, determining whether the DML operation is finished, if so, executing step S409, deleting the snapshot and the rollback statement, releasing the memory, and if not, returning to step S402 to continue to repeat the execution until the execution is finished.
Fig. 5 schematically illustrates a composition diagram of a data recovery apparatus in an exemplary embodiment of the disclosure, and as shown in fig. 5, the data recovery apparatus 500 may include a parsing module 501, a rollback module 502, and a recovery module 503. Wherein:
the analysis module 501 is configured to, when it is detected that a transaction is started, analyze a data manipulation language of the transaction to obtain an analysis result;
a rollback module 502, configured to generate a manipulation rollback statement of the data manipulation language according to the analysis result, so as to obtain rollback information of the transaction;
a recovery module 503, configured to, in response to the data recovery instruction of the transaction, obtain the rollback information to traverse and execute the manipulation rollback statement.
According to an exemplary embodiment of the present disclosure, the parsing module 501 is further configured to perform lexical parsing on the data manipulation language to obtain a lexical parsing result; and carrying out syntactic analysis on the lexical analysis result by using an analysis tool syntactic analysis generator, and generating an abstract syntactic tree as the analysis result.
According to an exemplary embodiment of the present disclosure, the rollback module 502 is configured to, when the data manipulation language is an insertion statement, generate the manipulation rollback statement according to a delete statement syntax according to a data table and condition information in the parsing result; when the data manipulation language is an update statement, generating the manipulation rollback statement according to a data table, field information and condition information in the analysis result and according to an update statement grammar; or when the data manipulation language is a delete statement, generating the manipulation rollback statement according to the insertion statement syntax according to the data table in the analysis result.
According to an exemplary embodiment of the present disclosure, the rollback module 502 further includes a snapshot unit, where the snapshot unit is configured to generate a query statement according to a query statement syntax according to the parsing result when the data manipulation language is an update statement or a delete statement; executing the query statement to obtain operation data information; saving the manipulation data information to a multi-version concurrency control to generate manipulation snapshot data, and adding the manipulation snapshot data to the rollback information.
According to an exemplary embodiment of the present disclosure, the snapshot unit is further configured to add a primary key field when the data manipulation language is an update statement and a primary key field is missing in the parsing result of the update statement, so as to generate the query statement according to the primary key field and the parsing result.
According to an exemplary embodiment of the disclosure, the snapshot unit is further configured to, when the data manipulation language is a delete statement, the method further comprises: and inquiring all field information of a data table in the analysis result so as to generate the inquiry statement according to all the field information and the analysis result.
According to an exemplary embodiment of the present disclosure, the recovery module 503 is further configured to obtain the rollback information to traverse and execute the manipulation rollback statement of the data manipulation language based on the manipulation snapshot data of the data manipulation language.
The details of each module in the data recovery apparatus 500 are already described in detail in the corresponding data recovery method, and therefore are not described herein again.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
In an exemplary embodiment of the present disclosure, there is also provided a storage medium capable of implementing the above-described method. Fig. 6 schematically illustrates a schematic diagram of a computer-readable storage medium in an exemplary embodiment of the disclosure, and as shown in fig. 6, a program product 600 for implementing the above method according to an embodiment of the disclosure is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a mobile phone. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided. Fig. 7 schematically shows a structural diagram of a computer system of an electronic device in an exemplary embodiment of the disclosure.
It should be noted that the computer system 700 of the electronic device shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of the application of the embodiments of the present disclosure.
As shown in fig. 7, the computer system 700 includes a Central Processing Unit (CPU)701, which can perform various appropriate actions and processes according to a program stored in a Read-Only Memory (ROM) 702 or a program loaded from a storage section 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data necessary for system operation are also stored. The CPU 701, ROM702, and RAM 703 are connected to each other via a bus 704. An Input/Output (I/O) interface 705 is also connected to the bus 704.
The following components are connected to the I/O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output section 707 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a Network interface card such as a LAN (Local Area Network) card, a modem, or the like. The communication section 709 performs communication processing via a network such as the internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 710 as necessary, so that a computer program read out therefrom is mounted into the storage section 708 as necessary.
In particular, the processes described below with reference to the flowcharts may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 709, and/or installed from the removable medium 711. When the computer program is executed by a Central Processing Unit (CPU)701, various functions defined in the system of the present disclosure are executed.
It should be noted that the computer readable medium shown in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM), a flash Memory, an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer-readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
As another aspect, the present disclosure also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method described in the above embodiments.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A method for data recovery, comprising:
when the opening of the transaction is detected, analyzing the data manipulation language of the transaction to obtain an analysis result;
generating a manipulation rollback statement of the data manipulation language according to the analysis result to obtain rollback information of the transaction;
and acquiring the rollback information to traverse and execute the manipulation rollback statement in response to a data recovery instruction of the transaction.
2. The data recovery method of claim 1, wherein parsing the data manipulation language of the transaction to obtain a parsed result comprises:
performing lexical analysis on the data manipulation language to obtain a lexical analysis result;
and carrying out syntactic analysis on the lexical analysis result by using an analysis tool syntactic analysis generator, and generating an abstract syntactic tree as the analysis result.
3. The data recovery method of claim 1, wherein the generating a manipulation rollback statement for the data manipulation language from the parsing result comprises:
when the data manipulation language is an insert statement, generating the manipulation rollback statement according to a deletion statement grammar according to a data table and condition information in the analysis result;
when the data manipulation language is an update statement, generating the manipulation rollback statement according to a data table, field information and condition information in the analysis result and according to an update statement grammar; or
And when the data manipulation language is a delete statement, generating the manipulation rollback statement according to the insertion statement syntax according to the data table in the analysis result.
4. The data recovery method of claim 3, wherein when the data manipulation language is an update statement or a delete statement, the method further comprises:
generating a query statement according to the parsing result and the query statement syntax;
executing the query statement to obtain operation data information;
saving the manipulation data information to a multi-version concurrency control to generate manipulation snapshot data, and adding the manipulation snapshot data to the rollback information.
5. The data recovery method of claim 4, wherein when the data manipulation language is an update statement and a primary key field is missing in the parsing result of the update statement, the method further comprises: and adding a main key field to generate the query statement according to the main key field and the analysis result.
6. The data recovery method of claim 4, wherein when the data manipulation language is a delete statement, the method further comprises: and inquiring all field information of a data table in the analysis result so as to generate the inquiry statement according to all the field information and the analysis result.
7. The data recovery method of claim 4, wherein said obtaining the rollback information to traverse through the execution of the manipulation rollback statement comprises:
and acquiring the rollback information to traverse and execute the manipulation rollback statement of the data manipulation language based on the manipulation snapshot data of the data manipulation language.
8. A data recovery apparatus, comprising:
the analysis module is used for analyzing the data manipulation language of the transaction to obtain an analysis result when the transaction is detected to be started;
the rollback module is used for generating a manipulation rollback statement of the data manipulation language according to the analysis result so as to obtain rollback information of the transaction;
and the recovery module is used for responding to a data recovery instruction of the transaction, and acquiring the rollback information to traverse and execute the manipulation rollback statement.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the data recovery method of any one of claims 1 to 7.
10. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out a data recovery method as claimed in any one of claims 1 to 7.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210128039.6A CN114461454A (en) | 2022-02-11 | 2022-02-11 | Data recovery method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202210128039.6A CN114461454A (en) | 2022-02-11 | 2022-02-11 | Data recovery method and device, storage medium and electronic equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114461454A true CN114461454A (en) | 2022-05-10 |
Family
ID=81413605
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202210128039.6A Pending CN114461454A (en) | 2022-02-11 | 2022-02-11 | Data recovery method and device, storage medium and electronic equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114461454A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115145831A (en) * | 2022-09-05 | 2022-10-04 | 湖南长银五八消费金融股份有限公司 | Non-invasive test data recovery method and system |
-
2022
- 2022-02-11 CN CN202210128039.6A patent/CN114461454A/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115145831A (en) * | 2022-09-05 | 2022-10-04 | 湖南长银五八消费金融股份有限公司 | Non-invasive test data recovery method and system |
CN115145831B (en) * | 2022-09-05 | 2022-11-08 | 湖南长银五八消费金融股份有限公司 | Non-invasive test data recovery method and system |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN107038222B (en) | Database cache implementation method and system | |
US7822710B1 (en) | System and method for data collection | |
CN109614432B (en) | System and method for acquiring data blood relationship based on syntactic analysis | |
CN110795455A (en) | Dependency relationship analysis method, electronic device, computer device and readable storage medium | |
CN109491989B (en) | Data processing method and device, electronic equipment and storage medium | |
CN111813796B (en) | Data column level blood margin processing system and method based on Hive data warehouse | |
CN111324610A (en) | Data synchronization method and device | |
CN111752959B (en) | Real-time database cross-database SQL interaction method and system | |
CN103514223A (en) | Data synchronism method and system of database | |
CN109086382B (en) | Data synchronization method, device, equipment and storage medium | |
CN111522631A (en) | Distributed transaction processing method, device, server and medium | |
CN115543402B (en) | Software knowledge graph increment updating method based on code submission | |
CN113204571B (en) | SQL execution method and device related to write-in operation and storage medium | |
CN117093599A (en) | Unified SQL query method for heterogeneous data sources | |
CN111367893A (en) | Method and device for database version iteration | |
CN111125213A (en) | Data acquisition method, device and system | |
CN114461454A (en) | Data recovery method and device, storage medium and electronic equipment | |
CN112395307A (en) | Statement execution method, statement execution device, server and storage medium | |
CN117421302A (en) | Data processing method and related equipment | |
CN110399378B (en) | Database system lock operation analysis method and device | |
CN117131230A (en) | Data blood edge analysis method, device, equipment and storage medium | |
CN109460416B (en) | Data processing method and device, electronic equipment and storage medium | |
US20070088684A1 (en) | Partial updating in a database proxy driver | |
CN116955393A (en) | Data processing method and device, electronic equipment and storage medium | |
CN117312420A (en) | Data sharing method and related system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |