CN109471851B - Data processing method, device, server and storage medium - Google Patents
Data processing method, device, server and storage medium Download PDFInfo
- Publication number
- CN109471851B CN109471851B CN201811208366.2A CN201811208366A CN109471851B CN 109471851 B CN109471851 B CN 109471851B CN 201811208366 A CN201811208366 A CN 201811208366A CN 109471851 B CN109471851 B CN 109471851B
- Authority
- CN
- China
- Prior art keywords
- version information
- data table
- log
- target
- target log
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The embodiment of the invention discloses a data processing method, a device, a server and a storage medium, wherein the method comprises the following steps: reading a target log in a pre-stored log set, wherein the target log is generated when a user applies for target operation on a data table; acquiring initial version information and current version information of a data table associated with the target log; and if the initial version information is the same as the current version information, triggering and executing target operation on the data table. The embodiment of the invention can ensure the accuracy of data processing, avoid the damage of the data table and improve the efficiency of data processing.
Description
Technical Field
The embodiment of the invention relates to the technical field of databases, in particular to a data processing method, a data processing device, a server and a storage medium.
Background
In a database, some operations performed on the records of the data table are not performed truly immediately, but rather are performed in a delayed manner. For example, when a user executes a delete operation on a data table record, the database management system records a rollback log of the delete operation, marks a delete mark on the deleted data table record, and performs data cleaning by an independent data cleaning thread according to the rollback log record of the system after the user transaction executing the delete operation is submitted, namely, the real data is deleted.
Assuming that a transaction executes a Data deletion operation of a Data table and submits the Data deletion operation, before a Data cleaning thread performs a related cleaning operation, if a system executes a database schema Definition statement (DDL) for modifying a Data structure, such a statement often causes reconstruction of an aggregate index of the Data table and modification of a table structure in a Data dictionary of the system, so that when the Data cleaning thread performs a cleaning operation according to a rollback log of the transaction, the cleaning operation may cause Data of the Data table to be damaged and serious consequences may be caused due to mismatching of the parsed rollback log and a current table structure.
The existing solution is to forcibly clean data before executing a DDL statement for modifying a data table structure, but this causes that the execution of the DDL statement is performed only after the completion of a "slow" operation of data cleaning, which reduces system efficiency and user experience.
Disclosure of Invention
The embodiment of the invention provides a data processing method, a data processing device, a server and a storage medium, which can solve the problem of low efficiency in the prior art.
In a first aspect, an embodiment of the present invention provides a data processing method, including:
reading a target log in a pre-stored log set, wherein the target log is generated when a user applies for target operation on a data table;
acquiring initial version information and current version information of a data table associated with the target log;
and if the initial version information is the same as the current version information, triggering and executing target operation on the data table.
In a second aspect, an embodiment of the present invention further provides a data processing apparatus, where the apparatus includes:
the reading module is used for reading a target log in a pre-stored log set, and the target log is generated when a user applies for target operation on the data table;
the version module is used for acquiring initial version information and current version information of a data table associated with the target log;
and the operation module is used for triggering and executing the target operation on the data table if the initial version information is the same as the current version information.
In a third aspect, an embodiment of the present invention further provides a server, where the server includes:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a data processing method as described above.
In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the data processing method described above.
According to the embodiment of the invention, the initial version information and the current version information of the data table associated with the target log are obtained by reading the target log in the pre-stored log set, and if the initial version information is the same as the current version information, the target operation on the data table is triggered and executed. The technical scheme provided by the embodiment of the invention adds the version information to the data table, and the version information changes along with the modification of the data table, so that the modified data table can be identified and not operated when the data table is operated, the accuracy of data processing is ensured, the data table is prevented from being damaged, and the efficiency of data processing is improved.
Drawings
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present invention;
fig. 2 is a flowchart of a data processing method according to a second embodiment of the present invention;
fig. 3 is a preferred example of a data processing method provided in the third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a data processing apparatus according to a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of a server according to a fifth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present invention, where the present embodiment is applicable to a data processing situation in a database, and the method may be executed by a data processing apparatus, and the apparatus may be implemented in a software and/or hardware manner, for example, the apparatus may be configured in a server. The method specifically comprises the following steps:
s110, reading a target log in the pre-stored log set, wherein the target log is generated when a user applies for target operation on the data table.
The pre-stored log set is a set of target logs in the database, and may be an entity log or a virtual set of target logs in an entity log. The target log is generated when a user applies for a target operation on the data table, the target operation is an operation that does not need to be executed immediately in the database and needs to be executed later, a specific type of the target operation is not limited in this embodiment, and the target operation may be, for example, deletion or logic analysis. Correspondingly, if the user applies for the target operation to the data table at present, a corresponding target log is generated, and the target log stores the operation to be performed by the user to the data table instead of the operation which is already completed by the user to the data table. When the type of the target operation is not limited, the type of the target log is not limited, but corresponds to the target operation. In this embodiment, a target operation is described as an example of a delete operation.
Optionally, reading a target log in a set of pre-stored logs, including: and acquiring a batch of target logs from the pre-stored log set at set time intervals and processing the target logs one by one. Wherein, the set time interval can be set according to the requirement. If there is a target log that needs to be forcibly executed in the database, the set time interval may not be set.
And S120, acquiring initial version information and current version information of the data table associated with the target log.
In this embodiment, the attribute of each data table in the database may be added with an attribute of version information, and the attribute is recorded in a data dictionary of the database together with other metadata information of the data table. The version information of each data table in the data dictionary can be generated when the database is updated, and can also be generated when a user applies for a target operation for each data table for the first time after the database is updated. When a database schema Definition (DDL) statement is executed to modify the structure of a Data table, the version information corresponding to the Data table is changed, for example, increased by one.
Acquiring current version information of a data table associated with a target log, wherein the current version information comprises the following steps: and acquiring second version information of the data table from a data dictionary of the database based on the ID of the data table, and recording the second version information as the current version information.
In this embodiment, a manner of adding a new version log is adopted, the type of the target log may include an operation log and a version log, the operation log and the version log are respectively generated when a user applies for a target operation on the data table, and correspondingly, before acquiring the initial version information and the current version information of the data table associated with the target log, the method further includes: and judging whether the type of the target log is the operation log or not, and if the type of the target log is the operation log, executing to obtain initial version information and current version information of a data table associated with the target log.
The operation log is used for performing target operation of the data table, the version log is a new log type in this embodiment, and is used for storing version information of the data table, and the recording content format of the log of this type is as follows: log type, data table ID, and data table version information. When a user applies for target operation on a data table, an operation log is generated, whether the version information of the data table exists in a data dictionary of a database or not can be judged, if not, the version information of the data table is registered and stored in the data dictionary of the database, and the version log is generated; and if so, reading the version information of the data table and generating a version log. For example, when the target operation is delete, an operation log, i.e. a rollback log, is generated, and the rollback log may be used for data cleaning and data rollback, and a version log is generated according to the version information of the data table. In this embodiment, a new log type is added to store version information, so that data processing can be downward compatible, and compatibility is stronger.
Because the target log is read one by one, when the version log is newly added, the type of the target log needs to be identified, and the method can be specifically realized by reading the value of the field corresponding to the version information in the log type. If the type of the target log is the operation log, acquiring initial version information and current version information of a data table associated with the target log; and if the type of the target log is the version log, not performing any operation.
Optionally, obtaining initial version information of the data table associated with the target log includes: and acquiring third version information in the version log based on the ID of the data table, and recording the third version information as initial version information.
And S130, if the initial version information is the same as the current version information, triggering and executing target operation on the data table.
Specifically, if the initial version information is the same as the current version information, it indicates that the data table is not modified, and the target operation on the data table is triggered and executed; if the initial version information is different from the current version information, the fact that the DDL modifies the table structure of the data table after the user applies for the target operation on the data table means that the modification on the table structure can rebuild the aggregation index of the data table, the target operation on the data table is not needed, and the reading operation of the target log is returned to be executed.
According to the technical scheme of the embodiment, the initial version information and the current version information of the data table associated with the target log are obtained by reading the target log in the pre-stored log set, and if the initial version information is the same as the current version information, the target operation on the data table is triggered and executed. According to the technical scheme, a new version log is added for storing version information, the version information is added for the data table, and the version information changes along with modification of the data table, so that the modified data table can be identified according to the version information in the version log and the current version information when the data table is operated, operation is not performed, the accuracy of data processing is guaranteed, the data table is prevented from being damaged, and the efficiency of data processing is improved.
Example two
Fig. 2 is a flowchart of a data processing method according to a second embodiment of the present invention. On the basis of the above embodiments, the present embodiment further optimizes the data processing method. Correspondingly, the method of the embodiment specifically includes:
s210, reading a target log in the pre-stored log set, wherein the target log is generated when a user applies for target operation on the data table.
Optionally, reading a target log in a set of pre-stored logs, including: and acquiring a batch of target logs from the pre-stored log set at set time intervals and processing the target logs one by one.
S220, extracting first version information in the target log, and recording the first version information as initial version information.
In this embodiment, a new content attribute, that is, version information, is added to the record content in the target log, and the type of the target log is only a log generated when the user applies for a target operation on the data table, and does not include logs of other operations in the database. The attribute of each data table in the database is added with the attribute of version information, and the attribute is recorded in a data dictionary of the database together with other metadata information of the data table. The version information of each data table in the data dictionary can be generated when the database is updated, and can also be generated when a target log is generated for the first time after the database is updated. When a database schema Definition (DDL) statement is executed to modify the structure of a Data table, the version information corresponding to the Data table is changed, for example, increased by one.
When a user applies for target operation on a data table, judging whether the version information of the data table exists in a data dictionary of a database, if not, registering the version information of the data table, storing the version information in the data dictionary of the database and generating a target log comprising the version information; and if so, reading the version information of the data table and generating a target log comprising the version information.
The first version information is the version information in the target log, and is extracted and recorded as the initial version information.
And S230, acquiring second version information of the data table from a data dictionary of the database based on the ID of the data table, and recording the second version information as the current version information.
And S240, if the initial version information is the same as the current version information, triggering and executing target operation on the data table.
Specifically, if the initial version information is the same as the current version information, it indicates that the data table is not modified, and the target operation on the data table is triggered to be executed.
And S250, if the initial version information is different from the current version information, returning to execute the reading operation of the next target log.
If the initial version information is different from the current version information, the fact that the DDL modifies the table structure of the data table after the user applies for the target operation on the data table means that the modification on the table structure can rebuild the aggregation index of the data table, the target operation on the data table is not needed, and the reading operation of the target log is returned to be executed.
According to the technical scheme of the embodiment, the initial version information and the current version information of the data table associated with the target log are obtained by reading the target log in the pre-stored log set, and if the initial version information is the same as the current version information, the target operation on the data table is triggered and executed. According to the technical scheme provided by the embodiment, the content of the version information is added in the target log, the version information is added to the data table, and the version information changes along with the modification of the data table, so that the modified data table can be identified according to the version information in the target log and the current version information when the data table is operated, further, the operation is not performed, the accuracy of data processing is ensured, the data table is prevented from being damaged, and the efficiency of data processing is improved.
EXAMPLE III
Fig. 3 is a preferred example of a data processing method according to a third embodiment of the present invention. The present embodiment may exemplarily describe the data processing method with the target operation being a delete operation as an example, on the basis of the above-described embodiments.
In this embodiment, a data table version information array may be added to the memory structure of the database transaction, and version information of the data table operated in the transaction may be stored in the version information array. The transaction memory structure of the database is a memory structure created when a transaction starts, so that when version information exists in the data table version information array, the version information does not need to be acquired from a database dictionary, and the efficiency is improved. Each transaction has its corresponding log buffer for temporarily storing a log corresponding to each operation performed in the transaction.
If a new version log is adopted, the target log comprises a rollback log and a version log. When a deleting operation is executed in a transaction, firstly, searching whether the version information of the data table exists in a data table version information array of the current transaction, if not, acquiring the version information of the data table in a data dictionary based on the ID of the data table, generating a version log, storing the version log in a log buffer area of the transaction, and storing the table ID and the version information in the data table version information array; if the transaction exists, the version log exists in the log buffer of the transaction, and the generation of the version log is not carried out. And then generating a rollback log corresponding to the deletion operation. When a transaction commits, the rollback log and the version log of the log buffer corresponding to the transaction are written into a rollback section (i.e., a pre-stored log set) of the system.
If the mode of adding the version information content in the target log is adopted, the target log is only a rollback log. The data table version information array does not need to be added to the memory structure of the database transaction.
Specifically, referring to fig. 3, fig. 3 is a flowchart of data deletion in the third embodiment of the present invention, and a manner of adding a new version log is adopted, and the data deletion process specifically includes:
s301, the data cleaning thread starts to work.
After the user transaction for executing the deleting operation is submitted, data cleaning is carried out according to the target log of the pre-stored log set through an independent data cleaning thread, namely, the real data is deleted.
S302, generating a data cleaning memory structure.
When the data cleaning thread executes data cleaning work, the data cleaning thread can sequentially acquire a target log corresponding to each completed transaction in the rollback section for processing, and generate a data cleaning memory structure to assist in cleaning work, wherein one data cleaning memory structure corresponds to the data cleaning work of a plurality of completed transactions.
S303, whether all the target logs of the completed transaction are processed or not is judged. If yes, go to S304; if not, S305 is executed.
And S304, ending.
S305, sequentially acquiring a target log of completed transactions. The target log comprises two types of rollback logs and version logs.
S306, whether the target log is a version log or not. If yes, go to S311; if not, go to step S307.
S307, acquiring the initial version information and the current version information of the data table.
And when the target log is a rollback log, acquiring the version information of the data table stored in the data cleaning memory structure according to the ID of the data table, and recording the version information as initial version information, wherein the version information of the data table in a data dictionary of the database is current version information.
And S308, whether the initial version information and the current version information are the same or not. If not, go to S309; if so, go to step S310.
S309, skipping the rollback log;
and S310, executing the deleting operation of the rollback log.
If the target log is a version log, the target operation is not required, but the version information in the data cleaning memory structure needs to be replaced, and the specific process comprises the following steps:
s311, whether the version information of the data table is registered in the data cleaning memory structure.
When the data cleaning thread processes the version log, whether the version information of the data table is stored in the data cleaning memory structure before is judged based on the ID of the data table, if so, S312 is executed, and if not, S314 is executed.
And S312, whether the version information is consistent with the version information in the version log or not. If yes, executing S303; if not, S313 is executed.
S313, the version information is modified into the version information in the version log.
If the version information in the data cleaning memory structure is inconsistent with the version information in the version log, a transaction executes a deleting operation on the data table before, and the DDL modifies the table structure of the data table after, so that the version information in the data cleaning memory structure is modified into the version information in the version log based on the ID of the data table.
And S314, registering version information in the version log in the data cleaning memory structure.
If the version information of the data table is not already stored in the data cleaning memory structure, the version information in the version log and the ID of the data table are registered in the data cleaning memory structure, and then S303 is executed.
It is to be understood that the data cleaning memory structure is a platform constructed for executing data deletion in this embodiment, and the data may be deleted directly without constructing the platform when deleting data in the database.
In the technical solution of this embodiment, an exemplary description is performed on the data processing method by taking the target operation as a deletion operation as an example, and further, it is described that by determining consistency between the initial version information and the current version information of the data table, the modified data table can be identified and not operated when the operation is performed on the data table, so that accuracy of data processing is ensured, the data table is prevented from being damaged, and efficiency of data processing is improved.
Example four
Fig. 4 is a schematic structural diagram of a data processing apparatus according to a fourth embodiment of the present invention, which is applicable to a data processing situation in a database. The data processing device provided by the embodiment of the invention can execute the data processing method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. The device specifically comprises a reading module 410, a version module 420 and an operation module 430, wherein:
the reading module 410 is configured to read a target log in a pre-stored log set, where the target log is generated when a user applies for a target operation on a data table;
the version module 420 is configured to obtain initial version information and current version information of a data table associated with the target log;
and an operation module 430, configured to trigger execution of a target operation on the data table if the initial version information is the same as the current version information.
According to the technical scheme of the embodiment, the initial version information and the current version information of the data table associated with the target log are obtained by reading the target log in the pre-stored log set, and if the initial version information is the same as the current version information, the target operation on the data table is triggered and executed. The technical scheme provided by the embodiment of the invention adds the version information to the data table, and the version information changes along with the modification of the data table, so that the modified data table can be identified and not operated when the data table is operated, the accuracy of data processing is ensured, the data table is prevented from being damaged, and the efficiency of data processing is improved.
Optionally, the apparatus further comprises:
and the return module is used for returning and executing the reading operation of the target log if the initial version information is different from the current version information.
Optionally, the target log includes version information of the associated data table; accordingly, the version module includes:
the initial version unit is used for extracting first version information in the target log and recording the first version information as initial version information;
and the current version unit is used for acquiring second version information of the data table from a data dictionary of the database based on the ID of the data table and recording the second version information as the current version information.
Optionally, the type of the target log includes an operation log and a version log, where the operation log and the version log are respectively generated when the user applies for performing a target operation on the data table, and correspondingly, the apparatus further includes:
and the judging module is used for judging whether the type of the target log is an operation log before acquiring the initial version information and the current version information of the data table associated with the target log, and executing the acquisition of the initial version information and the current version information of the data table associated with the target log if the type of the target log is the operation log.
Optionally, the version log includes version information of the data table, and correspondingly, the version module is configured to: and acquiring third version information in the version log based on the ID of the data table, and recording the third version information as initial version information.
The data processing device provided by the embodiment of the invention can execute the data processing method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE five
Fig. 5 is a schematic structural diagram of a server according to a fifth embodiment of the present invention. FIG. 5 illustrates a block diagram of an exemplary server 512 suitable for use in implementing embodiments of the present invention. The server 512 shown in fig. 5 is only an example and should not bring any limitations to the function and scope of the use of the embodiments of the present invention.
As shown in FIG. 5, the server 512 is in the form of a general purpose server. Components of server 512 may include, but are not limited to: one or more processors 516, a storage device 528, and a bus 518 that couples the various system components including the storage device 528 and the processors 516.
The server 512 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by server 512 and includes both volatile and nonvolatile media, removable and non-removable media.
A program/utility 540 having a set (at least one) of program modules 542 may be stored, for example, in storage 528, such program modules 542 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may include an implementation of a network environment. The program modules 542 generally perform the functions and/or methods of the described embodiments of the invention.
The server 512 may also communicate with one or more external devices 514 (e.g., keyboard, pointing terminal, display 524, etc.), with one or more terminals that enable a user to interact with the server 512, and/or with any terminals (e.g., network card, modem, etc.) that enable the server 512 to communicate with one or more other computing terminals. Such communication may occur via input/output (I/O) interfaces 522. Further, server 512 may communicate with one or more networks (e.g., a Local Area Network (LAN), Wide Area Network (WAN), and/or a public Network such as the Internet) via Network adapter 520. As shown in FIG. 5, the network adapter 520 communicates with the other modules of the server 512 via the bus 518. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the server 512, including but not limited to: microcode, end drives, Redundant processors, external disk drive Arrays, RAID (Redundant Arrays of Independent Disks) systems, tape drives, and data backup storage systems, among others.
The processor 516 executes various functional applications and data processing by executing programs stored in the storage device 528, for example, implementing a data processing method provided by the embodiment of the present invention, the method includes:
reading a target log in a pre-stored log set, wherein the target log is generated when a user applies for target operation on a data table;
acquiring initial version information and current version information of a data table associated with a target log;
and if the initial version information is the same as the current version information, triggering to execute target operation on the data table.
EXAMPLE six
An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements a data processing method provided in an embodiment of the present invention, where the method includes:
reading a target log in a pre-stored log set, wherein the target log is generated when a user applies for target operation on a data table;
acquiring initial version information and current version information of a data table associated with a target log;
and if the initial version information is the same as the current version information, triggering to execute target operation on the data table.
Computer storage media for embodiments of the invention may employ any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or terminal. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (9)
1. A data processing method, comprising:
reading a target log in a pre-stored log set, wherein the target log is generated when a user applies for target operation on a data table;
acquiring initial version information and current version information of a data table associated with the target log;
if the initial version information is the same as the current version information, triggering and executing target operation on the data table;
the types of the target logs comprise operation logs and version logs, the operation logs and the version logs are respectively generated when a user applies for target operation on a data table, and the version logs are used for storing version information of the data table;
correspondingly, before the obtaining of the initial version information and the current version information of the data table associated with the target log, the method further includes:
and judging whether the type of the target log is the operation log or not, and if the type of the target log is the operation log, executing the acquisition of the initial version information and the current version information of the data table associated with the target log.
2. The method of claim 1, wherein after obtaining the initial version information and the current version information of the data table associated with the target log, further comprising:
and if the initial version information is different from the current version information, returning to execute the reading operation of the next target log.
3. The method of claim 1, wherein the target log includes version information of the associated data table;
correspondingly, the acquiring of the initial version information and the current version information of the data table associated with the target log includes:
extracting first version information in the target log, and recording the first version information as initial version information;
and acquiring second version information of the data table from a data dictionary of a database based on the ID of the data table, and recording the second version information as current version information.
4. The method of claim 1, wherein the version log includes version information for the data table,
correspondingly, obtaining the initial version information of the data table associated with the target log includes:
and acquiring third version information in the version log based on the ID of the data table, and recording the third version information as initial version information.
5. A data processing apparatus, comprising:
the reading module is used for reading a target log in a pre-stored log set, and the target log is generated when a user applies for target operation on the data table;
the version module is used for acquiring initial version information and current version information of a data table associated with the target log;
the operation module is used for triggering and executing target operation on the data table if the initial version information is the same as the current version information;
the types of the target logs comprise operation logs and version logs, the operation logs and the version logs are respectively generated when a user applies for target operation on a data table, and the version logs are used for storing version information of the data table;
the device, still include: a judgment module;
the judging module is used for judging whether the type of the target log is the operation log before the version module acquires the initial version information and the current version information of the data table associated with the target log, and if the type of the target log is the operation log, acquiring the initial version information and the current version information of the data table associated with the target log.
6. The apparatus of claim 5, further comprising:
and the return module is used for returning and executing the reading operation of the target log if the initial version information is different from the current version information.
7. The apparatus of claim 5, wherein the target log includes version information of the associated data table; accordingly, the version module includes:
the initial version unit is used for extracting first version information in the target log and recording the first version information as initial version information;
and the current version unit is used for acquiring second version information of the data table from a data dictionary of a database based on the ID of the data table and recording the second version information as the current version information.
8. A server, characterized in that the server comprises:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a data processing method as claimed in any one of claims 1-4.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the data processing method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811208366.2A CN109471851B (en) | 2018-10-17 | 2018-10-17 | Data processing method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811208366.2A CN109471851B (en) | 2018-10-17 | 2018-10-17 | Data processing method, device, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109471851A CN109471851A (en) | 2019-03-15 |
| CN109471851B true CN109471851B (en) | 2021-05-28 |
Family
ID=65664698
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811208366.2A Active CN109471851B (en) | 2018-10-17 | 2018-10-17 | Data processing method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109471851B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110059096A (en) * | 2019-03-16 | 2019-07-26 | 平安城市建设科技(深圳)有限公司 | Data version management method, apparatus, equipment and storage medium |
| CN110955539A (en) * | 2019-11-26 | 2020-04-03 | 杭州迪普信息技术有限公司 | Process quitting method and device, electronic equipment and machine-readable storage medium |
| CN111090663B (en) * | 2019-12-25 | 2023-07-07 | 上海金仕达软件科技股份有限公司 | Transaction concurrency control method, device, terminal equipment and medium |
| CN111666166B (en) * | 2020-06-03 | 2023-10-31 | 中国建设银行股份有限公司 | Service providing method, device, equipment and storage medium |
| CN113111071A (en) * | 2021-05-11 | 2021-07-13 | 星辰天合（北京）数据科技有限公司 | Object processing method, device, nonvolatile storage medium and processor |
| CN114253944B (en) * | 2021-12-16 | 2022-10-04 | 深圳壹账通科技服务有限公司 | Database bidirectional synchronization method and device and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102651024A (en) * | 2012-04-01 | 2012-08-29 | 华为技术有限公司 | Method and device for data conversion |
| CN104217174A (en) * | 2014-09-05 | 2014-12-17 | 四川长虹电器股份有限公司 | Safety storage system and safety storage method for distributed files |
| CN104301360A (en) * | 2013-07-19 | 2015-01-21 | 阿里巴巴集团控股有限公司 | Method, log server and system for recording log data |
| CN106528557A (en) * | 2015-09-10 | 2017-03-22 | 北京国双科技有限公司 | Method and device for detecting data rows deleted from data table |
| CN106708427A (en) * | 2016-11-17 | 2017-05-24 | 华中科技大学 | Storage method suitable for key value pair data |
| CN107766354A (en) * | 2016-08-17 | 2018-03-06 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus for being used to ensure data correctness |
-
2018
- 2018-10-17 CN CN201811208366.2A patent/CN109471851B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102651024A (en) * | 2012-04-01 | 2012-08-29 | 华为技术有限公司 | Method and device for data conversion |
| CN104301360A (en) * | 2013-07-19 | 2015-01-21 | 阿里巴巴集团控股有限公司 | Method, log server and system for recording log data |
| CN104217174A (en) * | 2014-09-05 | 2014-12-17 | 四川长虹电器股份有限公司 | Safety storage system and safety storage method for distributed files |
| CN106528557A (en) * | 2015-09-10 | 2017-03-22 | 北京国双科技有限公司 | Method and device for detecting data rows deleted from data table |
| CN107766354A (en) * | 2016-08-17 | 2018-03-06 | 阿里巴巴集团控股有限公司 | A kind of method and apparatus for being used to ensure data correctness |
| CN106708427A (en) * | 2016-11-17 | 2017-05-24 | 华中科技大学 | Storage method suitable for key value pair data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109471851A (en) | 2019-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109471851B (en) | Data processing method, device, server and storage medium | |
| CN107391653B (en) | Distributed NewSQL database system and picture data storage method | |
| CN107506451B (en) | Abnormal information monitoring method and device for data interaction | |
| CN108519967B (en) | Chart visualization method and device, terminal and storage medium | |
| CN109524070B (en) | Data processing method and device, electronic equipment and storage medium | |
| CN110688544A (en) | Method, device and storage medium for querying database | |
| JP2012113706A (en) | Computer-implemented method, computer program, and data processing system for optimizing database query | |
| CN112231407B (en) | DDL synchronization method, device, equipment and medium of PostgreSQL database | |
| CN110413413A (en) | A kind of method for writing data, device, equipment and storage medium | |
| CN112463800A (en) | Data reading method and device, server and storage medium | |
| CN111694866A (en) | Data searching and storing method, data searching system, data searching device, data searching equipment and data searching medium | |
| US10083192B2 (en) | Deleted database record reuse | |
| CN113760894A (en) | Data calling method and device, electronic equipment and storage medium | |
| CN112487025A (en) | Data query method and device, electronic equipment and storage medium | |
| CN111198917A (en) | Data processing method, device, equipment and storage medium | |
| CN110753136B (en) | Domain name resolution method, device, equipment and storage medium | |
| CN114564354A (en) | Database performance monitoring method and device, electronic equipment and storage medium | |
| CN114090514A (en) | Log retrieval method and device for distributed system | |
| CN115374074A (en) | Log processing method and device, storage medium and electronic equipment | |
| US20200167152A1 (en) | Identification of a partial code to be refactored within a source code | |
| CN111427902A (en) | Metadata management method, device, equipment and medium based on lightweight database | |
| CN108628909B (en) | Information pushing method and device | |
| CN110750569A (en) | Data extraction method, device, equipment and storage medium | |
| CN111291130A (en) | Hive table consistency checking method, system, equipment and storage medium | |
| CN117076515B (en) | Metadata tracing method and device in medical management system, server and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |