CN113626423B - Log management method, device and system of business database - Google Patents
Log management method, device and system of business database Download PDFInfo
- Publication number
- CN113626423B CN113626423B CN202110725298.2A CN202110725298A CN113626423B CN 113626423 B CN113626423 B CN 113626423B CN 202110725298 A CN202110725298 A CN 202110725298A CN 113626423 B CN113626423 B CN 113626423B
- Authority
- CN
- China
- Prior art keywords
- sql
- database
- data
- semantic
- sql statement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000007726 management method Methods 0.000 title abstract description 24
- 238000004458 analytical method Methods 0.000 claims abstract description 45
- 238000000034 method Methods 0.000 claims abstract description 17
- 238000013507 mapping Methods 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 7
- 230000014509 gene expression Effects 0.000 description 7
- 230000006399 behavior Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 241000590419 Polygonia interrogationis Species 0.000 description 1
- 241000289690 Xenarthra Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24564—Applying rules; Deductive queries
- G06F16/24566—Recursive queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/14—Tree-structured documents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a log management method, device and system of a business database. The method comprises the following steps: acquiring SQL sentences aiming at a service database, and carrying out semantic analysis on the SQL sentences to obtain semantic analysis results; executing the SQL sentence in a data tracking database so as to synchronize the data of the data tracking database and the data of the service database, and determining the data change information of the data tracking database, wherein the data tracking database and the data structure information of the service database are the same; and generating the SQL execution log of the service database according to the semantic analysis result and the data change information. Therefore, the business database does not need to be invaded, the data change condition before and after the business database executes the SQL statement can be generated, the behavior of the SQL statement can be perceived, and the SQL execution log can be generated.
Description
Technical Field
The present invention relates to the field of database technologies, and in particular, to a method, an apparatus, a system, an electronic device, and a storage medium for log management of a service database.
Background
The log is used for business personnel to view the business changes and for developers to understand the system behavior. The current database log only records SQL sentences executed by the database, analysis on SQL behaviors is lacking, the data change condition of the database before and after the SQL sentences are executed cannot be known according to the database log, and if a business system needs to check the data change condition of the database, for example, the order amount is updated from 10 yuan to 20 yuan, the intrusion development of business codes needs to be carried out, and the development cost and the subsequent maintenance cost are increased.
Disclosure of Invention
The invention aims to overcome the defect that the database log in the prior art cannot record SQL behaviors of SQL sentences executed by a database, and provides a log management method, device, system, electronic equipment and storage medium of a service database.
The invention solves the technical problems by the following technical scheme:
in a first aspect, a log management method of a service database is provided, including:
acquiring SQL sentences aiming at a service database, and carrying out semantic analysis on the SQL sentences to obtain semantic analysis results;
executing the SQL sentence in a data tracking database so as to synchronize the data of the data tracking database and the data of the service database, and determining the data change information of the data tracking database, wherein the data tracking database and the data structure information of the service database are the same;
and generating the SQL execution log of the service database according to the semantic analysis result and the data change information.
Optionally, the SQL statement comprises a semantic unit and an SQL parameter;
semantic analysis is performed on the SQL statement, which comprises the following steps:
parsing the SQL statement into an abstract syntax tree, each tree node of the abstract syntax tree corresponding to a semantic unit or SQL parameter;
and generating the semantic analysis result according to each tree node.
Optionally, generating the semantic analysis result according to the tree nodes includes:
traversing the abstract syntax tree, determining semantic description information corresponding to the semantic units of the SQL statement according to the mapping relation between the semantic units and the semantic descriptions, and determining parameter description information corresponding to the SQL parameters of the SQL statement according to the mapping relation between the SQL parameters and the parameter descriptions;
and generating the semantic analysis result according to the semantic description information and the parameter description information.
Optionally, executing the SQL statement includes:
analyzing the SQL statement into an abstract syntax tree, and recursively calling a virtual machine or an interpreter according to the hierarchical structure of the abstract syntax tree to execute the SQL statement.
Optionally, determining the data change information of the data tracking database includes:
before the data tracking database executes the SQL statement, a first snapshot of the data tracking database is obtained;
after executing the SQL statement, obtaining a second snapshot of the data tracking database;
and determining data change information of the data tracking database according to the first snapshot and the second snapshot.
In a second aspect, there is provided a log management apparatus of a service database, including:
the acquisition module is used for acquiring SQL sentences aiming at the service database and carrying out semantic analysis on the SQL sentences to obtain semantic analysis results;
the execution module is used for executing the SQL sentence in a data tracking database so as to synchronize the data of the data tracking database and the data of the service database and determine the data change information of the data tracking database, wherein the data tracking database and the data structure information of the service database are the same;
and the generating module is used for generating the SQL execution log of the service database according to the semantic analysis result and the data change information.
Optionally, the SQL statement comprises a semantic unit and an SQL parameter;
the acquisition module comprises:
the parsing unit is used for parsing the SQL sentence into an abstract syntax tree, and each tree node of the abstract syntax tree corresponds to one semantic unit or SQL parameter;
and the generation unit is used for generating the semantic analysis result according to the tree nodes.
Optionally, the generating unit is configured to:
traversing the abstract syntax tree, determining semantic description information corresponding to the semantic units of the SQL statement according to the mapping relation between the semantic units and the semantic descriptions, and determining parameter description information corresponding to the SQL parameters of the SQL statement according to the mapping relation between the SQL parameters and the parameter descriptions;
and generating the semantic analysis result according to the semantic description information and the parameter description information.
Optionally, in executing the SQL statement, the execution module is configured to:
analyzing the SQL statement into an abstract syntax tree, and recursively calling a virtual machine or an interpreter according to the hierarchical structure of the abstract syntax tree to execute the SQL statement.
Optionally, in determining the data change information of the data tracking database, the execution module is configured to:
before the data tracking database executes the SQL statement, a first snapshot of the data tracking database is obtained;
after executing the SQL statement, obtaining a second snapshot of the data tracking database;
and determining data change information of the data tracking database according to the first snapshot and the second snapshot.
The invention has the positive progress effects that:
according to the embodiment of the invention, the business database does not need to be invaded, the data change condition before and after the SQL statement is executed by the business database can be generated, the behavior of the SQL statement can be perceived, the SQL execution log is generated for operation and maintenance personnel and business personnel to review, and the SQL execution log is easy to understand.
Drawings
FIG. 1 is a flow chart of a method for log management of a business database according to an exemplary embodiment of the present invention;
FIG. 2a is a schematic diagram of an abstract syntax tree generated during log management of a business database according to an exemplary embodiment of the present invention;
FIG. 2b is a diagram of a user interface for SQL execution log showing a method for log management of a business database according to an exemplary embodiment of the invention;
FIG. 3 is a block diagram of a log management device of a service database according to an exemplary embodiment of the present invention;
FIG. 4 is a block diagram of a log management system for a business database according to an exemplary embodiment of the present invention;
fig. 5 is a schematic structural view of an electronic device according to an exemplary embodiment of the present invention.
Detailed Description
The invention is further illustrated by means of the following examples, which are not intended to limit the scope of the invention.
Fig. 1 is a flowchart of a log management method of a service database according to an exemplary embodiment of the present invention, where the log management method may manage logs of one service database, or may manage logs of a plurality of service databases, and referring to fig. 1, the log management method includes the following steps:
step 101, acquiring SQL sentences aiming at a service database, and carrying out semantic analysis on the SQL sentences to obtain a semantic analysis result.
The SQL statement is used to instruct the operation on the business database. Operations on the business database may include, but are not limited to: operations on database objects (libraries, tables, columns, indexes); operations on database records, and the like. The service database stores a plurality of SQL sentences, and based on the service operation request of the user on the service application program, the service database determines the SQL sentences corresponding to the service operation request and executes the SQL sentences to complete the operation corresponding to the service operation request, for example, change the numerical value of a certain field in the data table.
In the embodiment of the invention, under the condition that the service database executes the SQL statement, the SQL statement is obtained, the SQL statement is subjected to semantic analysis, and the obtained semantic analysis result is used for generating the SQL execution log of the service database.
In one embodiment, SQL statements for the business database may be obtained, but are not limited to, by Mybatis plug-in.
In one embodiment, a plurality of SQL sentences and mapping relations between the SQL sentences and operation requests are stored in a data tracking database, the service operation requests received by the service database are obtained, and the SQL sentences corresponding to the service operation requests are determined according to the mapping relations between the SQL sentences and the operation requests.
Typically, an SQL statement includes a plurality of semantic units and at least one SQL parameter. The semantic units comprise keywords such as 'SELECT', 'FROM', 'WHERE', 'order' and the like in the SQL statement, operator or database fields, database tables and the like which meet semantic rules.
In one embodiment, the semantic analysis is performed on the SQL statement through an Abstract Syntax Tree (AST), wherein the abstract syntax tree is a tree representation of the SQL statement structure, so that the understanding of the syntax structure hierarchy of the SQL statement is facilitated, and one SQL statement corresponds to one abstract syntax tree.
For example, assume that the SQL statement obtained in step 101 is as follows:
update order
set status＝？，version＝version+1
where code＝？AND company＝？
resolving the SQL sentence, and generating an abstract syntax tree, as shown in fig. 2a, wherein the abstract syntax tree comprises 19 tree nodes, each tree node corresponds to a semantic unit or SQL parameter of the SQL sentence, and the semantic unit or SQL parameter is respectively: update, order, set, tatus, =, version, 1, where, code, AND, company; parameter 1 node corresponds to set status=? Is "? "SQL parameters, parameter 2 node corresponds to where code=? Is "? "SQL parameters, parameter 3 nodes correspond to company =? Is "? "SQL parameters".
The semantic analysis result is used for generating the SQL execution log of the service database, and in order to facilitate the understanding of the SQL execution log by a user, the semantic analysis result is described by using a popular and easily understood language.
In one embodiment, the mapping relationship between the semantic units and the semantic descriptions and the mapping relationship between the SQL parameters and the parameter descriptions are predefined, and the semantic descriptions and the parameter descriptions are described in a popular and easily understood manner. The mapping relation between the semantic units and the semantic descriptions and the mapping relation between the SQL parameters and the parameter descriptions can be set according to actual requirements. The semantic descriptions may include, but are not limited to, interpretation of semantic elements, description of data processing rules for field configuration, association between fields.
The following table illustrates exemplary mappings of semantic units to semantic descriptions:
| database field semantic unit | Semantic description |
| So | Order form |
| orderCoder | Order encoding |
| orderStatus | Order status |
| orderAmount | Order amount |
| GoodReceiverAddress | Detailed address |
From the above table, it can be known that the semantic description of the "So" database field semantic unit is an order table, and the semantic description of the "orderemount" database field semantic unit is an order amount.
The mapping relationship can also be expressed as follows:
string or numeric literal quantity: directly returning the corresponding Java basic object;
addition, subtraction, multiplication, division operators: recursively calling interpreters for all the child nodes, and then performing addition, subtraction, multiplication and division on values obtained by parsing of all the child nodes;
brackets (.+ -.): the brackets indicate that the operation performed earlier, e.g., (a+b) C performs the addition of a and B first;
question marks? : the SQL parameter corresponding to the order is represented and taken, and the node directly returns;
column name: in the UPDATE statement, the column value of the current processing snapshot is taken;
function: support NOW, IFNULL, CONCAT and other common functions, and the value functions used by INSERT sentences;
comparison operator (equal to, not equal to, greater than, less than, greater than or equal to, less than or equal to): the present implementation supports only comparison operations with column names on the left and expressions on the right, e.g., age > =14. Recursively invoking the interpreter on the right child node and then returning the generated query condition (criterion);
logical operator (AND OR): recursively calling interpreters for all the child nodes, and then analyzing all the child nodes to obtain query conditions which are combined into the query conditions after logic combination and returning (criterion);
set clause: the left node of each assigned expression in the set clause of the UPDATE is the column name and the right node is the expression. Recursively calling an interpreter for the right node, and sorting out the updated value of each column name;
the where clause: the sphere clause of the UPDATE has only one child node, and an interpreter is recursively called for the child node to obtain a query condition for matching the snapshot record;
the mapping relation between the semantic unit and the semantic description, and the mapping relation between the SQL parameter and the parameter description can be represented in a list form, and can also be represented in a YAML (format for expressing data serialization) configuration file.
Traversing the abstract syntax tree when generating a semantic analysis result, determining semantic description information of a semantic unit corresponding to the SQL sentence according to a mapping relation between the semantic unit and the semantic description, and determining parameter description information of an SQL parameter corresponding to the SQL sentence according to a mapping relation between the SQL parameter and the parameter description; and generating a semantic analysis result according to the semantic description information and the parameter description information.
The value of the SQL parameter is determined according to the execution result of the SQL statement in step 102.
And 102, executing SQL sentences in the data tracking database, and determining the data change information of the data tracking database.
Wherein the data tracking database may be, but is not limited to, a MongoDB database. The data tracking database is the same as the database structure information of the business database, and the database structure information comprises database names, data table names and fields in the data tables.
By executing SQL statements in the data tracking database, data synchronization of the data tracking database with the business database can be achieved.
Because the execution condition of SQL in the service database cannot be obtained, namely the data change information of the service database before and after executing the SQL statement cannot be obtained, in the embodiment of the invention, a data tracking database is arranged, if the SQL statement is executed in the service database, the same SQL statement is executed by the data tracking database, so that the data synchronization of the two databases is realized, and the data change condition of the service database can be obtained by analyzing the data change condition of the data tracking database.
In one embodiment, the SQL statement is executed based on a virtual machine or interpreter. Under the condition that the virtual machine or the interpreter cannot directly execute the original SQL statement, the SQL statement is parsed into an abstract syntax tree, and the virtual machine or the interpreter is recursively called according to the hierarchical structure of the abstract syntax tree, so that the SQL statement execution is realized. For example, the query condition parsed by the SQL interpreter window clause matches the snapshot record of the data tracking database, and updates the snapshot according to the column name and update value relationship determined by the abstract syntax tree, thereby realizing SQL statement execution.
Taking an INSERT statement as an example, the procedure executed by the SQL virtual machine or the interpreter on the AST is described as follows:
1. traversing each expression of the values clause in the AST;
2. analyzing the expression to obtain a corresponding result;
3. match the result with the column name (field) in the AST and sort out the output.
Taking UPDATE statement as an example, the procedure executed by the SQL virtual machine or the interpreter on the AST is described as follows:
1. analyzing a sphere clause in AST, and matching a data snapshot of a data tracking database;
2. taking the obtained data snapshot as a context;
3. traversing each assignment expression in the set clause in context;
4. analyzing the assignment expression to obtain a result of assignment of the corresponding column;
5. the update data tracks the snapshot of the database and collates and outputs the values before and after the update.
In one embodiment, a data tracking database is used to save and update snapshots of data records. Before the SQL statement is executed by the data tracking database, a first snapshot of the data tracking database is obtained, after the SQL statement is executed, a second snapshot of the data tracking database is obtained, and data change information of the data tracking database is determined according to the first snapshot and the second snapshot.
The data change information comprises data before updating, data after updating and data change description of the data tracking database, and the data change information characterizes the behavior of the SQL statement.
And step 103, generating SQL execution logs of the service database according to the semantic analysis result and the data change information.
In one embodiment, in the process of generating the SQL execution log, determining whether a field related to the SQL statement is configured with a value converter, if the value converter is set, converting and replacing the field value, for example, if 1010- > to-be-paid is configured for an orderttatus (order state) field of a So (order) table, 1020- > to-be-paid converter, the value conversion and replacement are required when the SQL execution log is generated, that is, when the value of the field orderttatus is 1010, the description in the SQL execution log is replaced with "to-be-paid"; when the value of field orderStatus is 1020, the description in the SQL execution log is replaced with "paid". In connection with semantic analysis results, the SQL behavioral description of the SQL execution log record may be, for example, "order status set to pending payment".
The value converter is configured in the following ways:
direct configuration mapping, e.g., 1010- > to pay; 1020- > paid;
calling a conversion method in the software application;
mapping relation configuration in a persistent container such as a pointing database.
In one embodiment, when the SQL execution log is generated, the semantic analysis result and the data change information can be respectively filled in the corresponding positions of the templates by means of a pre-configured log template to generate the SQL execution log. Different templates are used according to different scenarios.
For example, if there is a change in the data of the field, the application template may be, for example, according to the name of the field in the configuration, the data before update, the data after update: the field < field name > is changed from < pre-update data > to < post-update data >, and the generated SQL execution log includes a description of "field" order amount "changed from 10.00 to 5.00".
If the field is first written, i.e., the pre-update data is empty and only the post-update data is present, the template field < field name > is set to < post-update data > and the generated SQL execution log includes a description of the "" order tag "field set to vip.
Because the data tracking data implements data synchronization with the service database, in one embodiment, if the service database fails, the service database can be restored by the tracking database.
In one embodiment, the SQL execution log may be combined with a full link log. Specifically, whether the current service database has full link information is judged, if yes, the SQL execution log and the full link information are integrated, and the full link log is obtained. The association relation between the SQL execution log and the full-link log can be established, for example, the association relation between the SQL execution log and the full-link log is established through a link or a log ID, so that mutual inquiry is convenient, and the full-link information and the data change condition of a database are convenient.
In one embodiment, a display interface is further provided, referring to fig. 2b, which may be but not limited to development using SpringMVC, and encapsulates an interface for querying "SQL execution log for business personnel", and the user may query the corresponding SQL execution log (operation log in the figure) using "operation time", "operator" as a query condition.
According to the embodiment of the invention, the business database does not need to be invaded, the data change condition before and after the SQL statement is executed by the business database can be generated, the behavior of the SQL statement can be perceived, the SQL execution log is generated for operation and maintenance personnel and business personnel to review, and the SQL execution log is easy to understand.
Corresponding to the embodiment of the log management method of the service database, the invention also provides an embodiment of a log management device of the service database.
Fig. 3 is a schematic block diagram of a log management device of a service database according to an exemplary embodiment of the present invention, where the log management device includes:
the acquiring module 31 is configured to acquire an SQL statement for a service database, and perform semantic analysis on the SQL statement to obtain a semantic analysis result;
an execution module 32, configured to execute the SQL statement in a data tracking database to synchronize data of the data tracking database with data of the service database, and determine data change information of the data tracking database, where the data tracking database is identical to the database structure information of the service database;
and the generating module 33 is configured to generate an SQL execution log of the service database according to the semantic analysis result and the data change information.
Optionally, the SQL statement comprises a semantic unit and an SQL parameter;
the acquisition module comprises:
the parsing unit is used for parsing the SQL sentence into an abstract syntax tree, and each tree node of the abstract syntax tree corresponds to one semantic unit or SQL parameter;
and the generation unit is used for generating the semantic analysis result according to the tree nodes.
Optionally, the generating unit is configured to:
traversing the abstract syntax tree, determining semantic description information corresponding to the semantic units of the SQL statement according to the mapping relation between the semantic units and the semantic descriptions, and determining parameter description information corresponding to the SQL parameters of the SQL statement according to the mapping relation between the SQL parameters and the parameter descriptions;
and generating the semantic analysis result according to the semantic description information and the parameter description information.
Optionally, in executing the SQL statement, the execution module is configured to:
analyzing the SQL statement into an abstract syntax tree, and recursively calling a virtual machine or an interpreter according to the hierarchical structure of the abstract syntax tree to execute the SQL statement.
Optionally, in determining the data change information of the data tracking database, the execution module is configured to:
before the data tracking database executes the SQL statement, a first snapshot of the data tracking database is obtained;
after executing the SQL statement, obtaining a second snapshot of the data tracking database;
and determining data change information of the data tracking database according to the first snapshot and the second snapshot.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present invention. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
Fig. 4 is a schematic block diagram of a log management system of a service database according to an exemplary embodiment of the present invention, where the log management system includes a data tracking database and a log management device provided in any one of the foregoing embodiments. The data tracking database may be, but is not limited to, a MongoDB database, and the relevant parts of the log management device are referred to in the description of the above embodiments, and are not repeated here. It should be noted that, the log management device may be disposed on an independent server, or may be disposed on the same server as the data tracking database.
Fig. 5 is a schematic diagram of an electronic device, showing an exemplary electronic device 50 suitable for use in implementing embodiments of the present invention, in accordance with an exemplary embodiment of the present invention. The electronic device 50 shown in fig. 5 is merely an example and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 5, the electronic device 50 may be embodied in the form of a general purpose computing device, which may be a server device, for example. Components of electronic device 50 may include, but are not limited to: the at least one processor 51, the at least one memory 52, a bus 53 connecting the different system components, including the memory 52 and the processor 51.
The bus 53 includes a data bus, an address bus, and a control bus.
Memory 52 may include volatile memory such as Random Access Memory (RAM) 521 and/or cache memory 522, and may further include Read Only Memory (ROM) 523.
Memory 52 may also include a program tool 525 (or utility) having a set (at least one) of program modules 524, such program modules 524 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
The processor 51 executes various functional applications and data processing, such as the methods provided in any of the embodiments described above, by running a computer program stored in the memory 52.
The electronic device 50 may also communicate with one or more external devices 54 (e.g., keyboard, pointing device, etc.). Such communication may occur through an input/output (I/O) interface 55. Also, model-generated electronic device 50 may also communicate with one or more networks such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet via network adapter 56. As shown, the network adapter 56 communicates with other modules of the model-generated electronic device 50 via the bus 53. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with model-generating electronic device 50, including, but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, data backup storage systems, and the like.
It should be noted that although several units/modules or sub-units/modules of an electronic device are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more units/modules described above may be embodied in one unit/module in accordance with embodiments of the present invention. Conversely, the features and functions of one unit/module described above may be further divided into ones that are embodied by a plurality of units/modules.
The embodiment of the present invention also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method provided by any of the embodiments described above.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the principles and spirit of the invention, but such changes and modifications fall within the scope of the invention.
Claims (9)
1. A method for log management of a business database, comprising:
acquiring SQL sentences aiming at a service database, and carrying out semantic analysis on the SQL sentences to obtain semantic analysis results;
executing the SQL sentence in a data tracking database so as to synchronize the data of the data tracking database and the data of the service database, and determining the data change information of the data tracking database, wherein the data tracking database and the data structure information of the service database are the same;
according to the semantic analysis result and the data change information, generating an SQL execution log of the service database;
the SQL statement comprises a semantic unit and SQL parameters;
semantic analysis is performed on the SQL statement, which comprises the following steps:
parsing the SQL statement into an abstract syntax tree, each tree node of the abstract syntax tree corresponding to a semantic unit or SQL parameter;
generating the semantic analysis result according to each tree node;
the generating the semantic analysis result according to the tree nodes comprises the following steps:
traversing the abstract syntax tree, determining semantic description information corresponding to the semantic units of the SQL statement according to the mapping relation between the semantic units and the semantic descriptions, and determining parameter description information corresponding to the SQL parameters of the SQL statement according to the mapping relation between the SQL parameters and the parameter descriptions;
and generating the semantic analysis result according to the semantic description information and the parameter description information.
2. The method of claim 1, wherein executing the SQL statement comprises:
analyzing the SQL statement into an abstract syntax tree, and recursively calling a virtual machine or an interpreter according to the hierarchical structure of the abstract syntax tree to execute the SQL statement.
3. The method for log management of a traffic database according to claim 1, wherein determining data change information of the data tracking database comprises:
before the data tracking database executes the SQL statement, a first snapshot of the data tracking database is obtained;
after executing the SQL statement, obtaining a second snapshot of the data tracking database;
and determining data change information of the data tracking database according to the first snapshot and the second snapshot.
4. A log management apparatus for a business database, comprising:
the acquisition module is used for acquiring SQL sentences aiming at the service database and carrying out semantic analysis on the SQL sentences to obtain semantic analysis results;
the execution module is used for executing the SQL sentence in a data tracking database so as to synchronize the data of the data tracking database and the data of the service database and determine the data change information of the data tracking database, wherein the data tracking database and the data structure information of the service database are the same;
the generation module is used for generating an SQL execution log of the service database according to the semantic analysis result and the data change information;
the SQL statement comprises a semantic unit and SQL parameters;
the acquisition module comprises:
the parsing unit is used for parsing the SQL sentence into an abstract syntax tree, and each tree node of the abstract syntax tree corresponds to one semantic unit or SQL parameter;
the generation unit is used for generating the semantic analysis result according to each tree node;
the generating unit is used for:
traversing the abstract syntax tree, determining semantic description information corresponding to the semantic units of the SQL statement according to the mapping relation between the semantic units and the semantic descriptions, and determining parameter description information corresponding to the SQL parameters of the SQL statement according to the mapping relation between the SQL parameters and the parameter descriptions;
and generating the semantic analysis result according to the semantic description information and the parameter description information.
5. The log management device of the service database according to claim 4, wherein in executing the SQL statement, the execution module is configured to:
analyzing the SQL statement into an abstract syntax tree, and recursively calling a virtual machine or an interpreter according to the hierarchical structure of the abstract syntax tree to execute the SQL statement.
6. The log management device of the service database according to claim 4, wherein in determining the data change information of the data tracking database, the execution module is configured to:
before the data tracking database executes the SQL statement, a first snapshot of the data tracking database is obtained;
after executing the SQL statement, obtaining a second snapshot of the data tracking database;
and determining data change information of the data tracking database according to the first snapshot and the second snapshot.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any one of claims 1 to 3 when executing the computer program.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method of any one of claims 1 to 3.
9. A log management system for a business database, comprising: a data tracking database and the log management device of any of claims 4-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110725298.2A CN113626423B (en) | 2021-06-29 | 2021-06-29 | Log management method, device and system of business database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110725298.2A CN113626423B (en) | 2021-06-29 | 2021-06-29 | Log management method, device and system of business database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113626423A CN113626423A (en) | 2021-11-09 |
| CN113626423B true CN113626423B (en) | 2024-01-30 |
Family
ID=78378492
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110725298.2A Active CN113626423B (en) | 2021-06-29 | 2021-06-29 | Log management method, device and system of business database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113626423B (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944114A (en) * | 2010-09-16 | 2011-01-12 | 深圳天源迪科信息技术股份有限公司 | Data synchronization method between memory database and physical database |
| CN105938492A (en) * | 2016-04-14 | 2016-09-14 | 北京思特奇信息技术股份有限公司 | Data synchronization increment tracking method and system |
| CN108536743A (en) * | 2018-03-09 | 2018-09-14 | 中国电子科技集团公司第二十八研究所 | A kind of Combat Command System database in phase system |
| CN110209684A (en) * | 2019-06-03 | 2019-09-06 | 深圳前海微众银行股份有限公司 | Tracking, device, system and the medium of database D DL change operation |
| CN110309196A (en) * | 2019-05-22 | 2019-10-08 | 深圳壹账通智能科技有限公司 | Block chain data storage and query method, apparatus, equipment and storage medium |
| CN110750582A (en) * | 2018-07-23 | 2020-02-04 | 阿里巴巴集团控股有限公司 | Data processing method, device and system |
| CN110990420A (en) * | 2019-11-27 | 2020-04-10 | 腾讯科技（深圳）有限公司 | Data query method and device |
| CN111241125A (en) * | 2020-01-08 | 2020-06-05 | 成都嗨学洛子教育科技有限公司 | Method and device for recording operation log, electronic equipment and storage medium |
| CN111597257A (en) * | 2020-05-09 | 2020-08-28 | 远光软件股份有限公司 | Database synchronization method and device, storage medium and terminal |
| CN112231396A (en) * | 2020-09-15 | 2021-01-15 | 山东浪潮通软信息科技有限公司 | Method for incrementally synchronizing data across databases |
| CN112445867A (en) * | 2019-08-16 | 2021-03-05 | 成都翰通新能源有限公司 | Intelligent analysis method and system for data relationship |
| CN113032579A (en) * | 2021-03-25 | 2021-06-25 | 中国建设银行股份有限公司 | Metadata blood margin analysis method and device, electronic equipment and medium |
| CN113032421A (en) * | 2021-04-29 | 2021-06-25 | 电子科技大学 | MongoDB-based distributed transaction processing system and method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9916538B2 (en) * | 2012-09-15 | 2018-03-13 | Z Advanced Computing, Inc. | Method and system for feature detection |
-
2021
- 2021-06-29 CN CN202110725298.2A patent/CN113626423B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944114A (en) * | 2010-09-16 | 2011-01-12 | 深圳天源迪科信息技术股份有限公司 | Data synchronization method between memory database and physical database |
| CN105938492A (en) * | 2016-04-14 | 2016-09-14 | 北京思特奇信息技术股份有限公司 | Data synchronization increment tracking method and system |
| CN108536743A (en) * | 2018-03-09 | 2018-09-14 | 中国电子科技集团公司第二十八研究所 | A kind of Combat Command System database in phase system |
| CN110750582A (en) * | 2018-07-23 | 2020-02-04 | 阿里巴巴集团控股有限公司 | Data processing method, device and system |
| CN110309196A (en) * | 2019-05-22 | 2019-10-08 | 深圳壹账通智能科技有限公司 | Block chain data storage and query method, apparatus, equipment and storage medium |
| CN110209684A (en) * | 2019-06-03 | 2019-09-06 | 深圳前海微众银行股份有限公司 | Tracking, device, system and the medium of database D DL change operation |
| CN112445867A (en) * | 2019-08-16 | 2021-03-05 | 成都翰通新能源有限公司 | Intelligent analysis method and system for data relationship |
| CN110990420A (en) * | 2019-11-27 | 2020-04-10 | 腾讯科技（深圳）有限公司 | Data query method and device |
| CN111241125A (en) * | 2020-01-08 | 2020-06-05 | 成都嗨学洛子教育科技有限公司 | Method and device for recording operation log, electronic equipment and storage medium |
| CN111597257A (en) * | 2020-05-09 | 2020-08-28 | 远光软件股份有限公司 | Database synchronization method and device, storage medium and terminal |
| CN112231396A (en) * | 2020-09-15 | 2021-01-15 | 山东浪潮通软信息科技有限公司 | Method for incrementally synchronizing data across databases |
| CN113032579A (en) * | 2021-03-25 | 2021-06-25 | 中国建设银行股份有限公司 | Metadata blood margin analysis method and device, electronic equipment and medium |
| CN113032421A (en) * | 2021-04-29 | 2021-06-25 | 电子科技大学 | MongoDB-based distributed transaction processing system and method |
Non-Patent Citations (2)
| Title |
|---|
| Data-mining based SQL injection attack detection using internal query trees;Mi-Yeon Kim 等;《Expert Systems with Applications》;第41卷(第11期);5416-5430 * |
| 大数据量下异构数据库的数据同步系统的设计与实现;冼啟铭;《中国优秀硕士学位论文全文数据库 信息科技辑》(第02期);I138-772 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113626423A (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11907247B2 (en) | Metadata hub for metadata models of database objects | |
| US8984485B2 (en) | Analysis of source code changes | |
| US8065315B2 (en) | Solution search for software support | |
| Hartig | SPARQL for a Web of Linked Data: Semantics and computability | |
| US20140101632A1 (en) | Open application lifecycle management framework | |
| US11487742B2 (en) | Consistency checks between database systems | |
| US7634766B2 (en) | Method and apparatus for pattern-based system design analysis using a meta model | |
| CN107766353B (en) | Method and device for migrating statistical information of database | |
| CN113220633B (en) | Unified file coding management method and system | |
| Fisun et al. | Query parsing in order to optimize distributed DB structure | |
| CN113901083A (en) | Heterogeneous data source operation resource analysis positioning method and equipment based on multiple analyzers | |
| US11561976B1 (en) | System and method for facilitating metadata identification and import | |
| CN112347120B (en) | Automatic optimization method and device based on complex SQL | |
| CN113032642A (en) | Data processing method, device and medium for target object and electronic equipment | |
| EP2797001A2 (en) | System and method for creating variants in a test database during various test stages | |
| US10223086B2 (en) | Systems and methods for code parsing and lineage detection | |
| US20230289331A1 (en) | Model generation service for data retrieval | |
| US11194785B2 (en) | Universal self-learning database recovery | |
| CN113127460B (en) | Evaluation method of data cleaning frame, device, equipment and storage medium thereof | |
| CN113626423B (en) | Log management method, device and system of business database | |
| US20050066235A1 (en) | Automated fault finding in repository management program code | |
| US11709726B2 (en) | Error dynamics analysis | |
| CN114519071A (en) | Generation method, matching method, system, device and medium of rule matching model | |
| EP0990983A1 (en) | Automation of the design recovery and forward engineering of legacy applications | |
| CN112835887A (en) | Database management method, database management device, computing equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |