CN115269563B - Method and device for performing log analysis on database system - Google Patents
Method and device for performing log analysis on database system Download PDFInfo
- Publication number
- CN115269563B CN115269563B CN202211169601.6A CN202211169601A CN115269563B CN 115269563 B CN115269563 B CN 115269563B CN 202211169601 A CN202211169601 A CN 202211169601A CN 115269563 B CN115269563 B CN 115269563B
- Authority
- CN
- China
- Prior art keywords
- log
- data
- metadata snapshot
- stream
- time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
Abstract
In the method for log analysis, a log analysis system determines a log analysis time and determines a target time based on the log analysis time, wherein the target time is a starting time of a metadata snapshot persistence event closest to the log analysis time in a system table log stream. And sending a system table log stream acquisition request to the database system, wherein the system table log stream acquisition request at least comprises a target time and a log analysis time. And receiving a system log data segment which is extracted from the system log stream by the database system from the target time to the log analysis time. And analyzing a system log data segment, and acquiring the latest metadata snapshot obtained by the latest metadata snapshot persistence event and operation log data of data definition language DDL operation. Based on the latest metadata snapshot and the operation log data, a temporary metadata snapshot corresponding to the time of log analysis is created for log analysis.
Description
Technical Field
One or more embodiments of the present disclosure relate to the field of databases, and in particular, to a method and an apparatus for log analysis for a database system.
Background
In order to perform log analysis on a database system, a snapshot of metadata of a database object in the database system (referred to as a metadata snapshot) is generally required.
Conventionally, a database user usually takes a metadata snapshot in two ways: first, a database object is queried by SQL statements to create a metadata snapshot of the database object based on the query results. Second, a metadata snapshot of a database object is created based on a log stream of the database system. However, when creating a metadata snapshot based on a log stream, it is often necessary to read a large amount of user table data, which may include private data. This may affect the efficiency of creating the metadata snapshot and, in turn, the efficiency of log analysis.
Disclosure of Invention
One or more embodiments of the present specification describe a method and an apparatus for performing log analysis on a database system, which do not require sensing user table data, and thus can greatly improve the efficiency of log analysis.
In a first aspect, a method for log analysis for a database system is provided, including:
determining the log analysis time;
determining a target time based on the log analysis time, wherein the target time is the starting time of a metadata snapshot persistence event which is closest to the log analysis time in the system table log stream;
sending a system table log stream acquisition request to a database system, wherein the system table log stream acquisition request at least comprises the target time and the log analysis time;
receiving a system log data segment which is extracted from the system log stream by the database system from the target time to the log analysis time;
analyzing the system log data segment, and acquiring the latest metadata snapshot obtained by the latest metadata snapshot persistence event and operation log data of Data Definition Language (DDL) operation;
and creating a temporary metadata snapshot corresponding to the log analysis time for log analysis based on the latest metadata snapshot and the operation log data.
In a second aspect, a method for log analysis for a database system is provided, including:
the log analysis system determines the log analysis time;
the log analysis system determines a target time based on the log analysis time, wherein the target time is the starting time of a metadata snapshot persistence event which is closest to the log analysis time in the system table log stream;
the log analysis system sends a system table log stream acquisition request to a database system, wherein the system table log stream acquisition request at least comprises the target time and the log analysis time;
in response to the system table log stream acquisition request, the database system extracts a system log data segment from the target time to the log analysis time from the system table log stream;
the log analysis system analyzes the system log data segment, and acquires the latest metadata snapshot obtained by the latest metadata snapshot persistence event and operation log data operated by a Data Definition Language (DDL);
the log analysis system creates a temporary metadata snapshot corresponding to the log analysis time for log analysis based on the latest metadata snapshot and the operation log data.
In a third aspect, a method for constructing a data dictionary is provided, which includes:
determining the log analysis time;
determining a target time based on the log analysis time, wherein the target time is the starting time of a metadata snapshot persistence event which is closest to the log analysis time in the system table log stream;
sending a system table log stream acquisition request to a database system, wherein the system table log stream acquisition request at least comprises the target time and the log analysis time;
receiving a system log data segment which is extracted from the system table log stream by the database system and is from the target time to the log analysis time;
analyzing the system log data segment, and acquiring the latest metadata snapshot obtained by the latest metadata snapshot persistence event and operation log data of Data Definition Language (DDL) operation;
and constructing a data dictionary based on the latest metadata snapshot and the operation log data.
In a fourth aspect, an apparatus for log analysis for a database system is provided, comprising:
a determining unit for determining a log analysis time;
the determining unit is further configured to determine a target time based on the log analysis time, where the target time is a start time of a metadata snapshot persistence event closest to the log analysis time in the system table log stream;
a sending unit, configured to send a system table log stream acquisition request to a database system, where the request at least includes the target time and the log analysis time;
a receiving unit, configured to receive a system log data segment extracted by the database system from the system table log stream from the target time to the log analysis time;
the analysis unit is used for analyzing the system log data segment, acquiring the latest metadata snapshot obtained by the latest metadata snapshot persistence event and the operation log data operated by a Data Definition Language (DDL);
and the creating unit is used for creating a temporary metadata snapshot corresponding to the log analysis time based on the latest metadata snapshot and the operation log data for log analysis.
In a fifth aspect, an apparatus for constructing a data dictionary is provided, including:
a determining unit for determining a log analysis time;
the determining unit is further configured to determine a target time based on the log analysis time, where the target time is a start time of a metadata snapshot persistence event closest to the log analysis time in the system table log stream;
a sending unit, configured to send a request for obtaining a system table log stream to a database system, where the request at least includes the target time and the log analysis time;
a receiving unit, configured to receive a system log data segment extracted from the system table log stream by the database system from the target time to the log analysis time;
the analysis unit is used for analyzing the system log data segment, acquiring the latest metadata snapshot obtained by the latest metadata snapshot persistence event and the operation log data operated by a Data Definition Language (DDL);
and the construction unit is used for constructing a data dictionary based on the latest metadata snapshot and the operation log data.
In a sixth aspect, there is provided a computer storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of the first or second or third aspect.
In a seventh aspect, there is provided a computing device comprising a memory and a processor, the memory having stored therein executable code, the processor when executing the executable code implementing the method of the first, second or third aspect.
One or more embodiments of the present specification provide a method and an apparatus for performing log analysis on a database system, where the database system may write a log related to system metadata into a system table log stream. Here, the log related to the system metadata includes a metadata snapshot of a database object related to the system metadata and operation log data of a DDL operation performed on the database object. Then, when the log analysis system wants to perform log analysis on the database system, the starting time of a metadata snapshot persistent event which is closest to the log analysis time in the system table log stream is determined, then a system log data segment from the starting time to the log analysis time is pulled from the system table log stream and analyzed, and based on the analysis result, a temporary metadata snapshot corresponding to the log analysis time is created and log analysis is performed based on the temporary metadata snapshot. It should be noted that, because the system table log stream is only used for recording the change data of the system table, and the data volume of the system table log stream is small, when the temporary metadata snapshot is created based on the system table log stream, the data volume required to be obtained can be reduced, which can improve the creation efficiency of the metadata snapshot, and further improve the log analysis efficiency.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the drawings used in the description of the embodiments will be briefly described below, and it is obvious that the drawings in the description below are only some embodiments of the present specification, and it is obvious for those skilled in the art that other drawings may be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic diagram illustrating an implementation scenario of an embodiment disclosed herein;
FIG. 2 shows a schematic diagram of a data writing method in one example;
FIG. 3 is a diagram showing a data writing method in another example;
FIG. 4 illustrates a method interaction diagram for log analysis for a database system, according to one embodiment;
FIG. 5 shows a schematic diagram of an apparatus for log analysis for a database system, according to one embodiment;
FIG. 6 illustrates an apparatus diagram for building a data dictionary, according to one embodiment.
Detailed Description
The scheme provided by the specification is described below with reference to the accompanying drawings.
Before describing the solutions provided in the embodiments of the present specification, the following description will be made on the inventive concept of the present solution.
As mentioned above, a database user typically takes a snapshot of the metadata of a database object in two ways, wherein the metadata of the database object includes, but is not limited to, a database name, a table name, a column name, a number of columns of attributes contained in the table, a type of the columns of attributes, and a database to which the columns of attributes belong.
First, a database object is queried by SQL statements to create a metadata snapshot of the database object based on the query results. In this manner, however, a corresponding metadata snapshot can typically only be created for a particular time (e.g., a time when the database object has not changed as compared to the current time), and for portions of the database system having historical versions of the recorded database object, a corresponding metadata snapshot can be created for any time.
Second, a metadata snapshot of a database object is created based on a log stream of the database system. In this way, the database system may periodically create metadata snapshots of the database objects, and write the periodically created metadata snapshots into a log stream of the database system, where the log stream is used to record change information of the user table and the system table at the same time. Then, when the database user wants to create a corresponding temporary metadata snapshot for any time, the log data is pulled from the log stream, and then a corresponding temporary metadata snapshot is created based on the pulled log data. However, in this manner, the database user is typically required to pull a large amount of user table data from the database system, which can affect the efficiency of metadata snapshot creation and, in turn, log analysis.
The present invention is mainly directed to the second method, and specifically, the inventors of the present application propose that a log related to system metadata can be written into a system table log stream. The system table log stream herein includes an Online redo (Online) log or Archive (Archive) log. In addition, the log associated with the system metadata may include a metadata snapshot of a database object associated with the system metadata and operation log data of a DDL operation performed on the database object, wherein the metadata snapshot may be periodically written into a system table log stream. Then, when the log analysis system wants to perform log analysis on the database system, the starting time of a metadata snapshot persistent event which is closest to the log analysis time in the system table log stream is determined, then a system log data segment from the starting time to the log analysis time is pulled from the system table log stream and analyzed, and based on the analysis result, a temporary metadata snapshot corresponding to the log analysis time is created and log analysis is performed based on the temporary metadata snapshot.
In addition, the inventor of the present application further proposes that, during the writing of the metadata snapshot into the system table log stream, the DDL operation performed on the database system is not disabled. It should be noted that, in order to ensure atomicity and durability of data operations in a database system, a Write-Ahead Logging (WAL) mechanism is generally adopted, that is, a log is written first and then data is written. Therefore, the scheme does not disable the concept of DDL operation, and can avoid the influence on the usability of the database system.
The present invention has been conceived as above, and the present invention will be described in detail below.
Fig. 1 is a schematic view of an implementation scenario of an embodiment disclosed in this specification. In fig. 1, several database nodes may be included in the database system. In each database node, several tenants (Tenant) can be deployed, and the tenants are basic units for resource allocation and object management of the database system.
In addition, in each database node, a corresponding dictionary acquisition component may be assigned in advance for each tenant. The dictionary acquisition component is used for acquiring a metadata snapshot of a database object under a corresponding tenant and writing the metadata snapshot into a system table log stream under the corresponding tenant. In one example, the metadata snapshot is recorded in a system table log stream by a log entry of a first type.
It should be noted that, for each database node in fig. 1, one database node may also be selected as a tenant-level root node, and is used to receive and execute a DDL operation on a database object under a certain tenant, and write operation log data of the DDL operation into a system table log stream under the tenant. In one example, the operation log data of the DDL operation is recorded in the system table log stream by a log entry of the second type.
The log analysis system of fig. 1 is used to obtain system log data segments from system log streams under a tenant of the database system and parse them through an analysis component therein to create temporary metadata snapshots corresponding to any one time. And the database system is also used for acquiring user log data from a user table log stream of the tenant of the database system, analyzing the user log data according to the created temporary metadata snapshot and further outputting the change information of the user table.
It should be understood that fig. 1 is only an exemplary illustration, and in practical applications, the database system or the log analysis system may further include other components, which are not limited in this specification.
The following description will first describe a method of writing metadata snapshot and operation log data (collectively referred to as data) of the DDL operation. Since the data writing method for each tenant is similar, the following description will be given of a method for writing corresponding data by taking any tenant (tenant 1) as an example.
It should be understood that, in the present solution, since the DDL operation performed on the database system is not disabled in the process of writing the metadata snapshot into the system table log stream, the data writing method corresponding to tenant 1 may be as follows.
Fig. 2 shows a schematic diagram of a data writing method in one example. In fig. 2, at time T1, the dictionary obtaining component 1 corresponding to tenant 1 obtains the metadata snapshot 1 of the database object under tenant 1 by snapshot reading. After that, when the dictionary obtaining component 1 writes the metadata snapshot 1 into the system table log stream, the tenant-level root node receives a DDL1 operation for the database object under tenant 1, so that operation log data of the DDL1 operation is inserted into the log entry section of the metadata snapshot 1 when being written into the system table log stream.
In fig. 2, metadata snapshot 1 written into the system table log stream consists of log entries of a first type, which may be marked by boxes with cross-hatching. The oplog data written into the system table log stream consists of log entries of a second type, which may be illustrated by the boxes with vertical lines.
It should be understood that after writing the metadata snapshot corresponding to the time T1 into the system table log stream, the dictionary obtaining component 1 corresponding to the tenant 1 may obtain the log entry interval corresponding to the metadata snapshot persistence event. In one example, the Log entry interval can be comprised of a first Log Sequence Number (LSN) of a starting Log entry and a second LSN of a terminating Log entry. Then, the metadata snapshot persistence event, the time T1, and the log entry interval may be stored in a data snapshot information table, and the data snapshot information table is maintained as a system table in the database node where the tenant 1 is located, and a database user (e.g., a log analysis system) may query the data snapshot information table based on an SQL statement.
In one example, the data snapshot information table may be as shown in table 1.
It should be noted that in table 1, T2, and T3 are 3 times (snapshot times for short) of obtaining a metadata snapshot of a database object, and the snapshot time is stored in the data snapshot information table as a start time of a metadata snapshot persistence event.
It should be further noted that the start time in table 1 can be divided into two cases: first, when the metadata snapshot is written into the system table log stream, the operation log data of the DDL operation is not inserted before the start log entry of the metadata snapshot, so that the start time here is the actual start time of the metadata snapshot persistence event. Secondly, when the metadata snapshot is written into the system table log stream, the operation log data of the DDL operation is inserted before the start log entry of the metadata snapshot, so that the start time here is the log recording time of the inserted operation log data (see fig. 3), that is, the actual start time of the metadata snapshot persistence event is later than the start time of the storage.
It should be understood that table 1 is only an exemplary illustration, and in practical applications, the metadata snapshot persistence event may not be stored in table 1, and the description does not limit this.
Fig. 3 is a schematic diagram showing a data writing method in another example. In fig. 3, at time T1, in the process that the dictionary obtaining component 1 corresponding to tenant 1 obtains the metadata snapshot 1, the tenant-level root node receives a DDL1 operation for the database object under tenant 1, so that the operation log data of the DDL1 operation is written into the system table log stream before the metadata snapshot 1.
In fig. 3, metadata snapshot 1 written into the system table log stream consists of log entries of a first type, which may be marked by boxes with cross-hatching. The oplog data written into the system table log stream consists of log entries of a second type, which may be illustrated by the boxes with vertical lines.
It should be understood that fig. 2 and fig. 3 are only exemplary, and in practical applications, the operation log data corresponding to the metadata snapshot at other snapshot time and other DDL operations may also be persisted in the system table log stream, which is not limited in this specification.
Further, based on the writing method shown in fig. 2 and fig. 3, which disturbs the real time sequence between the metadata snapshot 1 and the operation log data of the DDL1 operation, recovery needs to be performed during playback (a specific recovery method will be described later).
Further, after the writing method shown in fig. 3 is performed, a data snapshot information table as shown in table 1 may also be obtained.
The above is an example of any tenant and a description of a data writing method thereof, and the following is an example of any tenant (i.e., tenant 1) and a description of a method of creating a temporary metadata snapshot at any time.
It should be noted that, for simplifying the description, in the following description of the present specification, the description of "tenant" will be omitted, and those skilled in the art should understand "database object under tenant 1", "system table log stream under tenant 1", "user table log stream under tenant 1", and "system log data segment under tenant 1" and the like.
FIG. 4 illustrates an interaction diagram of a method for log analysis for a database system, according to one embodiment. As shown in fig. 4, the method may include the following steps.
Step 402, determining a log analysis time.
The log analysis time may be any time, or may be the current time, or may be any historical time.
In step 404, the log analysis system determines a target time based on the log analysis time, where the target time is a starting time of a metadata snapshot persistence event closest to the log analysis time in the system table log stream.
It should be understood that, when the log analysis time is any historical time, the target time is specifically the start time of a metadata snapshot persistence event which is in the system table log stream and is before and closest to the log analysis time.
In one example, the log analysis system may select the target time from start times corresponding to respective metadata snapshot persistence events cached locally.
In another example, the log analysis system may query a data snapshot information table in the database system, in which several metadata snapshot persistence events and their corresponding start times are stored. And obtaining the target time according to the query result.
Wherein the log analysis system can query the data snapshot information table using SQL statements.
Taking the snapshot information table shown in table 1 as an example, assuming that the log analysis time is Ts, and the log analysis time Ts is located between the snapshot time T1 and the snapshot time T2, the obtained target time is T1.
In addition, as described above, log entry intervals corresponding to a plurality of metadata snapshot persistence events may also be stored in the data snapshot information table. In the case that the data snapshot information table further stores log entry intervals, the query result may further include a target log entry interval corresponding to a metadata snapshot persistence event that is closest to the log analysis time.
Step 406, the log analysis system sends a system table log stream acquisition request to the database system, wherein the system table log stream acquisition request at least comprises a target time and a log analysis time.
The database system maintains a system table log stream and a user table log stream. The system table log stream is used to record a change log of a system table (also referred to as an internal table). Since the system table is usually only used inside the database system, the change log of the system table is the internal operation data of the system. Of course, in this scenario, the system table log stream is also used to record logs related to system metadata. Wherein the log associated with the system metadata comprises a metadata snapshot of a database object associated with the system metadata and operation log data of a DDL operation performed on the database object associated with the system metadata.
The user table log stream is used for recording a change log of the user table. The user table is usually externally used, and thus the change log of the user table is external operation data of an external user.
It should be noted that in a database system, the system tables are usually small in number, while the user tables are large in number, so that the data size of the system table log stream is small compared to the user table log stream. Therefore, the metadata snapshot and the operation log data of the DDL operation are innovatively written into the system table log stream, and the data volume acquired by a database user when the temporary metadata snapshot is created can be reduced.
In response to the request for obtaining the system log stream, the database system extracts the data segment of the system log from the target time to the log analysis time from the system log stream, and provides the data segment to the log analysis system, step 408.
The system table log stream described in this specification is further configured to record a log recording time and a log sequence number LSN corresponding to each log entry, so that the database system can convert the target time into a start LSN recorded in the system table log stream and convert the log analysis time into an end LSN recorded in the system table log stream by analyzing the system table log stream. And then, extracting the system log data segment from the system table log stream according to the starting LSN and the ending LSN.
For example, assuming that the target time is T1, the log analysis time is Ts, and the start LSN corresponding to T1 is 80, the end LSN corresponding to Ts is 260, the above-mentioned system day number data segment is composed of log entries from LSN =80 to LSN =260 in the system table log stream.
Step 410, the log analysis system analyzes the system log data segment, obtains the latest metadata snapshot obtained from the latest metadata snapshot persistent event at the time of log analysis, and the operation log data of the data definition language DDL operation executed on the database object related to the system metadata.
It should be noted that, in this solution, because the DDL operation performed on the database system is not disabled during the process of writing the metadata snapshot into the system table log stream, the obtained system log data segment may be divided into the following cases.
First, the system log data segment includes a first log section, a second log section, and a third log section arranged in chronological order, wherein the first log section and the third log section are configured of a first type of log entry, and the second log section is configured of a second type of log entry.
Referring to fig. 2, in the case that the target time is T1 and the log analysis time is Ts, the obtained system log data segment is the system log data segment in the first case. Specifically, the second log section is operation log data of the DDL1 operation located in the log entry interval of the metadata snapshot 1, and the second log section is formed by a second type of log entry. The first log section and the third log section are two parts of the metadata snapshot 1 divided by the operation log data operated by the DDL 1.
Secondly, the system log data segment comprises a fourth log section and a fifth log section which are arranged in time sequence, wherein the fourth log section is formed by log entries of a second type, and the starting log recording time is the target time. The fifth journal segment is composed of a journal entry of the first type whose start journal recording time is the actual start time of the latest metadata snapshot persistence event.
Referring to fig. 3, in the case that the target time is T1 and the log analysis time is Ts, the obtained system log data segment is the system log data segment in the second case. Specifically, the fourth log section is operation log data of the DDL1 operation, which is composed of log entries of the second type, and the starting log recording time of the fourth log section is T1. The fifth log segment is the metadata snapshot 1 corresponding to T1, and is formed by log entries of the first type, and the starting log recording time of the fifth log segment is the actual starting time of the metadata snapshot persistent event (i.e., the latest metadata snapshot persistent event) used for writing the metadata snapshot 1.
It should be noted that, for the above system log data segment, the log analysis system may directly and sequentially analyze the whole system log data segment to obtain the above metadata snapshot and operation log data.
Or, the log analysis system may also extract the target log entry from the system log data segment according to the target log entry interval, and then analyze the target log entry and the remaining log entries respectively. And analyzing the target log entry to obtain the latest metadata snapshot obtained by the latest metadata snapshot persistence event. By analyzing the remaining log entries, the operation log data of the DDL operation can be acquired.
It should be noted that, in the process of analyzing the system log data segment, if the operation log data of the DDL operation is obtained through analysis, the log analysis system firstly saves the operation log data until the latest metadata snapshot is obtained, and then sequentially plays back the operation log data of each DDL operation according to a storage sequence (hereinafter referred to as a sequence), so that a real time sequence between the latest metadata snapshot and the operation log data of each DDL operation can be restored.
Referring to fig. 2, assuming that the starting LSN corresponding to the target time T1 is 100, and the ending LSN corresponding to the log analysis time Ts is 200, the system log data segment is composed of log entries from LSN =100 to LSN = 200. In addition, assume that the target log entry interval corresponding to the latest metadata snapshot persistence event: [100,200]. Because the upper and lower limits of the target log entry interval are the same as the starting LSN and the ending LSN, the whole system log data segment is used as the target log entry to be analyzed, operation log data of DDL1 operation can be obtained firstly in the analyzing process, then the operation log data are stored, and then the analysis is continued, so that the complete metadata snapshot 1 can be obtained.
Referring to fig. 3, assuming that the starting LSN corresponding to the target time T1 is 80 and the ending LSN corresponding to the log analysis time Ts is 260, the system log data segment is composed of log entries from LSN =80 to LSN = 260. In addition, assume that the target log entry interval corresponding to the latest metadata snapshot persistence event: 100,200, based on the target log entry interval, a target log entry can be retrieved. Thereafter, the target log entry may be parsed first, then the metadata snapshot 1 may be obtained, then the remaining log entries (including the log entries from LSN =80 to LSN =100 and the log entries from LSN =200 to LSN = 260) may be parsed, the operation log data of the DDL1 operation may be obtained, and then the operation log data may be saved.
Of course, in practical applications, the remaining log entries may be analyzed first, and then the target log entries may be analyzed, which is not limited in this specification.
In step 412, the log analysis system creates a temporary metadata snapshot corresponding to the time of log analysis based on the latest metadata snapshot and the operation log data for log analysis.
Specifically, the saved operation log data is played back, and the change data of the system metadata is acquired. And taking the latest metadata snapshot as reference data, taking the changed data as incremental data of the system metadata, and creating a temporary metadata snapshot corresponding to the log analysis time.
It should be understood that, when the operation log data of a plurality of DDL operations are stored, the operation log data are sequentially played back in the storage order of the operation log data.
As in the foregoing example, assuming that the log entries from LSN =200 to LSN =260 further include the operation log data of the DDL2 operation, after the metadata snapshot 1 is acquired, the operation log data of the DDL1 operation is played back first, and then the operation log data of the DDL2 operation is played back.
It should be noted that, the DDL operation described in this specification may include, but is not limited to, an add/delete modify operation on a database object related to system metadata, taking the DDL operation as an example of adding a new column in the data table t1, and by playing back corresponding operation log data, the data table t1 after adding a new column may be obtained, that is, the modified data table t1 is obtained. Thereafter, the latest metadata snapshot may be adjusted based on the modified data table t1. For example, add 1 to the number of columns of data table t1 in the latest metadata snapshot, and add the newly added data type, and so on.
It should be appreciated that by adjusting the most recent metadata snapshot, a temporary metadata snapshot corresponding to the time of log analysis may be obtained. For example, a temporary metadata snapshot is created that corresponds to the log analysis time Ts. It should also be appreciated that if multiple oplog data are played back in sequence, an adjustment is made to the most recent metadata snapshot after the current oplog data playback is complete. And then playing back next operation log data, and carrying out next adjustment on the adjusted latest metadata snapshot until the playback of all the operation log data is finished.
It should be noted that the data dictionary can be formed based on the temporary metadata snapshot created as described above and corresponding to the time of log analysis. Of course, in practical applications, the data dictionary may also include other target data for log analysis, and may also include other temporary metadata snapshots created corresponding to other log analysis times, and the like.
In addition, based on the created temporary metadata snapshot, a log analysis may also be performed on the database system, and the log analysis method will be described below.
The log analysis system may send a user log stream acquisition request to the database system including at least a log analysis time (e.g., ts). User log data extracted by the database system from the user table log stream from the time of log analysis is received. And analyzing the user log data, wherein at least the internal identification of the database object in the user log data is converted into a real object name according to the created temporary metadata snapshot. For example, table _ id/column _ id, etc. are converted into real table _ name/column _ name, etc.
It should be noted that the method for the database system to pull the user log data from the user log stream may refer to the pulling method of the system log data segment.
It should be noted that, due to the difference between the data amount of the user table log stream and the system table log stream, the LSNs converted for the same time (for example, ts) are also different. For example, the log analysis time Ts may have a corresponding LSN of 260 in the system table log stream, but may have a corresponding LSN of 5000 in the user table log stream. Therefore, the metadata snapshot and the operation log data are written into the log stream of the system table, and the data volume obtained when the temporary metadata snapshot is created can be reduced.
Of course, in practical applications, the database system may further continuously send system log data from the time of log analysis to the log analysis system in response to the user table log stream acquisition request, so that the log analysis system determines whether operation log data of a new DDL operation is stored therein, and if the operation log data of the new DDL operation is stored, the temporary metadata snapshot is adjusted based on the operation log data of the new DDL operation, and subsequent analysis is performed based on the adjusted temporary metadata snapshot. And if not, directly discarded.
In summary, according to the scheme provided in this embodiment, when a log analysis system wants to perform log analysis on a database system, a start time of a latest metadata snapshot persistent event in a system table log stream is determined, then a system log data segment from the start time of the event to a log analysis time is pulled from the system table log stream and analyzed, and based on an analysis result, a temporary metadata snapshot corresponding to the log analysis time is created and log analysis is performed based on the temporary metadata snapshot.
It should be noted that, because the system table log stream is only used for recording the change log of the system table, the data volume of the system table log stream is small, so that the data volume acquired in the process of creating the data snapshot can be reduced, which can improve the creation efficiency of the data snapshot, and further improve the log analysis efficiency. In addition, in the scheme, in the process of writing the metadata snapshot into the log stream of the system table, the DDL operation executed for the database system is not disabled, so that the usability of the database system can be prevented from being affected.
Correspondingly to the method for performing log analysis on a database system, an embodiment of the present specification further provides an apparatus for performing log analysis on a database system, as shown in fig. 5, the apparatus may include:
a determining unit 502 for determining a log analysis time.
The determining unit 502 is further configured to determine a target time based on the log analysis time, where the target time is a start time of a metadata snapshot persistence event closest to the log analysis time in the system table log stream.
The determining unit 502 is specifically configured to:
inquiring a data snapshot information table in a database system, wherein a plurality of metadata snapshot persistence events and corresponding starting moments are stored;
and obtaining the target moment according to the query result.
A sending unit 504, configured to send a system table log stream obtaining request to a database system, where the system table log stream obtaining request at least includes a target time and a log analysis time.
A receiving unit 506, configured to receive a system log data segment extracted by the database system from the system log stream from the target time to the log analysis time.
The system log data segment from the target time to the log analysis time is obtained by the following steps:
respectively converting the target time and the log analysis time into an initial LSN and a termination LSN in the system table log stream by analyzing the system table log stream;
and extracting the system log data segment from the system table log stream according to the starting LSN and the ending LSN.
And in the process of writing the latest metadata snapshot in the system log data segment into the system table log stream, the DDL operation executed for the database system is not forbidden.
In the process of writing the latest metadata snapshot in the system log data segment into the system table log stream, the DDL operation performed on the database system is not disabled, and the method includes:
when the latest metadata snapshot is written into the system table log stream, the database system receives the DDL operation, so that the operation log data of the DDL operation is inserted into the log entry interval of the latest metadata snapshot when written into the system table log stream.
The parsing unit 508 is configured to parse the system log data segment, and obtain a latest metadata snapshot obtained by a latest metadata snapshot persistence event, and operation log data of a data definition language DDL operation.
A creating unit 510, configured to create a temporary metadata snapshot corresponding to a log analysis time for log analysis based on the latest metadata snapshot and the operation log data.
The creating unit 510 includes:
the playback submodule 5102 is configured to play back the operation log data, and obtain change data of the system metadata;
the creating submodule 5104 is configured to create a temporary metadata snapshot corresponding to the log analysis time, using the latest metadata snapshot as reference data and the change data as incremental data.
Optionally, the data snapshot information table further stores log entry intervals corresponding to respective metadata snapshot persistent events; the query result further comprises a target log entry interval corresponding to the latest metadata snapshot persistence event;
the parsing unit 508 includes:
the extracting submodule 5082 is used for extracting a target log entry from the system log data segment according to the target log entry interval;
and the parsing submodule 5084 is used for parsing the target log entry and obtaining the latest metadata snapshot.
In one example, the target log entry interval consists of a first log sequence number LSN of the start log entry and a second log sequence number LSN of the end log entry.
Optionally, the apparatus further comprises: an analysis unit 512;
a sending unit 504, configured to send a user table log stream obtaining request to a database system, where the user table log stream obtaining request at least includes a log analysis time;
a receiving unit 506, further configured to receive user log data extracted by the database system from the user table log stream from the time of log analysis;
the analyzing unit 512 is configured to analyze the user log data, which at least includes converting the internal identifier of the database object in the user log data into a real object name according to the temporary metadata snapshot.
The functions of each functional module of the device in the above embodiments of the present description may be implemented through each step of the above method embodiments, and therefore, a specific working process of the device provided in one embodiment of the present description is not repeated herein.
In the device for analyzing logs for a database system provided in one embodiment of the present specification, user table data does not need to be sensed in the process of creating a data snapshot, so that the creation efficiency of the data snapshot can be greatly improved, and the log analysis efficiency can be further improved.
Corresponding to the above method for constructing a data dictionary, an embodiment of the present specification further provides an apparatus for constructing a data dictionary, which may include, as shown in fig. 6:
a determining unit 602, configured to determine a log analysis time.
The determining unit 602 is further configured to determine a target time based on the log analysis time, where the target time is a start time of a metadata snapshot persistence event closest to the log analysis time in the system table log stream.
The sending unit 604 is configured to send a request for obtaining a system table log stream to the database system, where the request includes at least a target time and a log analysis time.
A receiving unit 606, configured to receive a system log data segment extracted by the database system from the system table log stream from the target time to the log analysis time.
The parsing unit 608 is configured to parse the system log data segment, and obtain a latest metadata snapshot obtained by a latest metadata snapshot persistence event, and operation log data of a data definition language DDL operation.
The constructing unit 610 is configured to construct a data dictionary based on the latest metadata snapshot and the operation log data.
The functions of each functional module of the device in the above embodiments of the present description may be implemented through each step of the above method embodiments, and therefore, a specific working process of the device provided in one embodiment of the present description is not repeated herein.
According to an embodiment of another aspect, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method described in connection with fig. 4.
According to an embodiment of yet another aspect, there is also provided a computing device comprising a memory and a processor, the memory having stored therein executable code, the processor, when executing the executable code, implementing the method described in connection with fig. 4.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The steps of a method or algorithm described in connection with the disclosure herein may be embodied in hardware or may be embodied in software instructions executed by a processor. The software instructions may consist of corresponding software modules that may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an ASIC. Additionally, the ASIC may reside in a server. Of course, the processor and the storage medium may reside as discrete components in a server.
Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in this invention may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The above-mentioned embodiments, objects, technical solutions and advantages of the present specification are further described in detail, it should be understood that the above-mentioned embodiments are only specific embodiments of the present specification, and are not intended to limit the scope of the present specification, and any modifications, equivalent substitutions, improvements and the like made on the basis of the technical solutions of the present specification should be included in the scope of the present specification.
Claims (18)
1. A method for log analysis for a database system that maintains a system table log stream and a user table log stream, the system table log stream for logging logs related to system metadata, the method comprising:
determining the log analysis time;
determining a target time based on the log analysis time, wherein the target time is the starting time of a metadata snapshot persistence event which is closest to the log analysis time in the system table log stream;
sending a system table log stream acquisition request to a database system, wherein the system table log stream acquisition request at least comprises the target time and the log analysis time;
receiving a system log data segment which is extracted from the system log stream by the database system from the target time to the log analysis time;
analyzing the system log data segment, and acquiring the latest metadata snapshot obtained by the latest metadata snapshot persistence event and operation log data of Data Definition Language (DDL) operation;
creating a temporary metadata snapshot corresponding to the log analysis time for log analysis based on the latest metadata snapshot and the operation log data;
wherein, during writing into the system table log stream, the DDL operation performed on the database system by the latest metadata snapshot in the system log data segment is not disabled, and the DDL operation includes: and when the latest metadata snapshot is written into the system table log stream, the database system receives a DDL operation, so that operation log data of the DDL operation is inserted into a log entry interval of the latest metadata snapshot when the operation log data is written into the system table log stream.
2. The method of claim 1, wherein said creating a temporary metadata snapshot corresponding to said log analysis time comprises:
playing back the operation log data to obtain the change data of the system metadata;
and taking the latest metadata snapshot as reference data, taking the changed data as incremental data, and creating a temporary metadata snapshot corresponding to the log analysis time.
3. The method of claim 1, wherein the determining a target time based on the log analysis time comprises:
inquiring a data snapshot information table in the database system, wherein a plurality of metadata snapshot persistent events and corresponding starting moments thereof are stored;
and obtaining the target time according to the query result.
4. The method according to claim 3, wherein the data snapshot information table further stores log entry intervals corresponding to the metadata snapshot persistence events respectively; the query result further comprises a target log entry interval corresponding to the latest metadata snapshot persistence event;
the obtaining of the latest metadata snapshot obtained by the latest metadata snapshot persistence event includes:
extracting a target log entry from the system log data segment according to the target log entry interval;
and analyzing the target log entry and acquiring the latest metadata snapshot.
5. The method of claim 4, wherein the target log entry interval consists of a first Log Sequence Number (LSN) of a starting log entry and a second Log Sequence Number (LSN) of a terminating log entry.
6. The method of claim 1, further comprising:
sending a user table log stream acquisition request to the database system, wherein the user table log stream acquisition request at least comprises the log analysis time;
receiving user log data which is extracted from the user table log stream by the database system and starts from the log analysis time;
analyzing the user log data, wherein at least including converting the internal identification of the database object in the user log data into a real object name according to the temporary metadata snapshot.
7. A method for log analysis for a database system that maintains a system table log stream and a user table log stream, the system table log stream for logging logs related to system metadata, the method comprising:
the log analysis system determines the log analysis time;
the log analysis system determines a target time based on the log analysis time, wherein the target time is the starting time of a metadata snapshot persistence event which is closest to the log analysis time in the system table log stream;
the log analysis system sends a system table log stream acquisition request to a database system, wherein the system table log stream acquisition request at least comprises the target time and the log analysis time;
in response to the system table log stream acquisition request, the database system extracts a system log data segment from the target time to the log analysis time from the system table log stream;
the log analysis system analyzes the system log data segment, and acquires the latest metadata snapshot obtained by the latest metadata snapshot persistence event and operation log data operated by a Data Definition Language (DDL);
the log analysis system creates a temporary metadata snapshot corresponding to the log analysis time based on the latest metadata snapshot and the operation log data for log analysis;
wherein, during writing into the system table log stream, the DDL operation performed on the database system by the latest metadata snapshot in the system log data segment is not disabled, and the DDL operation includes: and when the latest metadata snapshot is written into the system table log stream, the database system receives a DDL operation, so that operation log data of the DDL operation is inserted into a log entry interval of the latest metadata snapshot when the operation log data is written into the system table log stream.
8. The method of claim 7, wherein the system table log stream is further configured to record a log recording time and a Log Sequence Number (LSN) corresponding to each log entry;
the extracting the system log data segment from the target time to the log analysis time comprises:
converting the target time and the log analysis time into a start LSN and a stop LSN in the system table log stream respectively by analyzing the system table log stream;
and extracting the system log data segment from the system table log stream according to the starting LSN and the ending LSN.
9. A method of building a data dictionary performed based on a system table log stream maintained in a database system, the system table log stream for logging data related to system metadata, the method comprising:
determining the log analysis time;
determining a target time based on the log analysis time, wherein the target time is the starting time of a metadata snapshot persistence event which is closest to the log analysis time in the system table log stream;
sending a system table log stream acquisition request to a database system, wherein the acquisition request at least comprises the target time and the log analysis time;
receiving a system log data segment which is extracted from the system table log stream by the database system and is from the target time to the log analysis time;
analyzing the system log data segment, and acquiring the latest metadata snapshot obtained by the latest metadata snapshot persistence event and operation log data of Data Definition Language (DDL) operation;
constructing a data dictionary based on the latest metadata snapshot and the operation log data;
wherein, during writing into the system table log stream, the DDL operation performed on the database system by the latest metadata snapshot in the system log data segment is not disabled, and the DDL operation includes: when the latest metadata snapshot is written into the system table log stream, the database system receives a DDL operation, so that operation log data of the DDL operation is inserted into a log entry interval of the latest metadata snapshot when the operation log data of the DDL operation is written into the system table log stream.
10. An apparatus for log analysis for a database system maintaining a system table log stream and a user table log stream, the system table log stream for logging related to system metadata, the apparatus comprising:
a determining unit for determining a log analysis time;
the determining unit is further configured to determine a target time based on the log analysis time, where the target time is a start time of a metadata snapshot persistence event closest to the log analysis time in the system table log stream;
a sending unit, configured to send a system table log stream acquisition request to a database system, where the request at least includes the target time and the log analysis time;
a receiving unit, configured to receive a system log data segment extracted from the system table log stream by the database system from the target time to the log analysis time;
the analysis unit is used for analyzing the system log data segment, and acquiring the latest metadata snapshot obtained by the latest metadata snapshot persistence event and operation log data operated by a Data Definition Language (DDL);
a creating unit for creating a temporary metadata snapshot corresponding to the log analysis time for log analysis based on the latest metadata snapshot and the operation log data;
wherein, during writing into the system table log stream, the DDL operation performed on the database system by the latest metadata snapshot in the system log data segment is not disabled, and the DDL operation includes: and when the latest metadata snapshot is written into the system table log stream, the database system receives a DDL operation, so that operation log data of the DDL operation is inserted into a log entry interval of the latest metadata snapshot when the operation log data is written into the system table log stream.
11. The apparatus of claim 10, wherein the creating unit comprises:
the playback submodule is used for playing back the operation log data and acquiring the change data of the system metadata;
and the creating sub-module is used for taking the latest metadata snapshot as reference data, taking the changed data as incremental data and creating a temporary metadata snapshot corresponding to the log analysis time.
12. The apparatus according to claim 10, wherein the determining unit is specifically configured to:
inquiring a data snapshot information table in the database system, wherein a plurality of metadata snapshot persistent events and corresponding starting moments thereof are stored;
and obtaining the target moment according to the query result.
13. The apparatus according to claim 12, wherein the data snapshot information table further stores log entry intervals corresponding to the respective metadata snapshot persistence events; the query result further comprises a target log entry interval corresponding to the latest metadata snapshot persistence event;
the analysis unit includes:
the extraction submodule is used for extracting a target log entry from the system log data segment according to the target log entry interval;
and the analysis sub-module is used for analyzing the target log entry and acquiring the latest metadata snapshot.
14. The apparatus of claim 13, wherein the target log entry interval is comprised of a first Log Sequence Number (LSN) of a starting log entry and a second Log Sequence Number (LSN) of a terminating log entry.
15. The apparatus of claim 10, further comprising: an analysis unit;
the sending unit is further configured to send a user table log stream acquisition request to the database system, where the user table log stream acquisition request at least includes the log analysis time;
the receiving unit is further configured to receive user log data extracted by the database system from the user table log stream from the log analysis time;
the analysis unit is used for analyzing the user log data, and at least comprises the step of converting the internal identification of the database object in the user log data into a real object name according to the temporary metadata snapshot.
16. An apparatus for constructing a data dictionary, comprising:
a determining unit for determining a log analysis time;
the determining unit is further configured to determine a target time based on the log analysis time, where the target time is a start time of a metadata snapshot persistence event closest to the log analysis time in a system table log stream;
a sending unit, configured to send a system table log stream acquisition request to a database system, where the request at least includes the target time and the log analysis time;
a receiving unit, configured to receive a system log data segment extracted from the system table log stream by the database system from the target time to the log analysis time;
the analysis unit is used for analyzing the system log data segment, acquiring the latest metadata snapshot obtained by the latest metadata snapshot persistence event and the operation log data operated by a Data Definition Language (DDL);
the construction unit is used for constructing a data dictionary based on the latest metadata snapshot and the operation log data;
wherein, during writing into the system table log stream, the DDL operation performed on the database system by the latest metadata snapshot in the system log data segment is not disabled, and the DDL operation includes: and when the latest metadata snapshot is written into the system table log stream, the database system receives a DDL operation, so that operation log data of the DDL operation is inserted into a log entry interval of the latest metadata snapshot when the operation log data is written into the system table log stream.
17. A computer-readable storage medium, on which a computer program is stored, wherein the computer program causes a computer to carry out the method of any one of claims 1-9, when the computer program is carried out in the computer.
18. A computing device comprising a memory and a processor, wherein the memory has stored therein executable code that when executed by the processor implements the method of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211169601.6A CN115269563B (en) | 2022-09-26 | 2022-09-26 | Method and device for performing log analysis on database system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211169601.6A CN115269563B (en) | 2022-09-26 | 2022-09-26 | Method and device for performing log analysis on database system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115269563A CN115269563A (en) | 2022-11-01 |
| CN115269563B true CN115269563B (en) | 2022-12-09 |
Family
ID=83756450
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211169601.6A Active CN115269563B (en) | 2022-09-26 | 2022-09-26 | Method and device for performing log analysis on database system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115269563B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105205053A (en) * | 2014-05-30 | 2015-12-30 | 阿里巴巴集团控股有限公司 | Method and system for analyzing database incremental logs |
| CN112445863A (en) * | 2020-11-30 | 2021-03-05 | 永辉云金科技有限公司 | Real-time data synchronization method and system |
| CN114090538A (en) * | 2020-07-30 | 2022-02-25 | 华为云计算技术有限公司 | Data backtracking method and device |
| CN114328374A (en) * | 2020-09-29 | 2022-04-12 | 华为云计算技术有限公司 | Snapshot method, device, related equipment and database system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10698921B2 (en) * | 2017-02-28 | 2020-06-30 | Sap Se | Persistence and initialization of synchronization state for serialized data log replay in database systems |
-
2022
- 2022-09-26 CN CN202211169601.6A patent/CN115269563B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105205053A (en) * | 2014-05-30 | 2015-12-30 | 阿里巴巴集团控股有限公司 | Method and system for analyzing database incremental logs |
| CN114090538A (en) * | 2020-07-30 | 2022-02-25 | 华为云计算技术有限公司 | Data backtracking method and device |
| CN114328374A (en) * | 2020-09-29 | 2022-04-12 | 华为云计算技术有限公司 | Snapshot method, device, related equipment and database system |
| CN112445863A (en) * | 2020-11-30 | 2021-03-05 | 永辉云金科技有限公司 | Real-time data synchronization method and system |
Non-Patent Citations (1)
| Title |
|---|
| Oracle数据库系统日志文件的分析方法;刘惠琴;《宁夏机械》;20041115(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115269563A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109241175B (en) | Data synchronization method and device, storage medium and electronic equipment | |
| US10565208B2 (en) | Analyzing multiple data streams as a single data object | |
| CN108399256B (en) | Heterogeneous database content synchronization method and device and middleware | |
| CN110879813B (en) | Binary log analysis-based MySQL database increment synchronization implementation method | |
| CN110569214B (en) | Index construction method and device for log file and electronic equipment | |
| US20170322972A1 (en) | Database workload capture and replay | |
| US9753977B2 (en) | Method and system for managing database | |
| CN110334326B (en) | A kind of method and system for identifying recipe file and being converted into XML file | |
| CN106874281B (en) | Method and device for realizing database read-write separation | |
| CN106407360B (en) | Data processing method and device | |
| CN111209344A (en) | Data synchronization method and device | |
| CN109086382B (en) | Data synchronization method, device, equipment and storage medium | |
| CN106649722B (en) | Monitoring system high-frequency data storage and query method | |
| CN114691704A (en) | Metadata synchronization method based on MySQL binlog | |
| KR100899616B1 (en) | Method and system of management metadata using relational database management system | |
| CN110515958A (en) | Data consistency method, apparatus, equipment and storage medium based on big data | |
| CN112395307A (en) | Statement execution method, statement execution device, server and storage medium | |
| CN108038253B (en) | Log query processing method and device | |
| CN115269563B (en) | Method and device for performing log analysis on database system | |
| CN115391457B (en) | Cross-database data synchronization method, device and storage medium | |
| CN113348450A (en) | Vehicle-mounted data storage method and system | |
| US20150269277A1 (en) | Storing method and apparatus for data aquisition | |
| CN114816247A (en) | Logic data acquisition method and device | |
| CN115658815A (en) | CDC (control data center) -based data synchronization method | |
| KR101024494B1 (en) | Extraction method of modified data using meta data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |