CN111737203A - Database history log backtracking method, device, system, equipment and storage medium - Google Patents
Database history log backtracking method, device, system, equipment and storage medium Download PDFInfo
- Publication number
- CN111737203A CN111737203A CN202010520272.XA CN202010520272A CN111737203A CN 111737203 A CN111737203 A CN 111737203A CN 202010520272 A CN202010520272 A CN 202010520272A CN 111737203 A CN111737203 A CN 111737203A
- Authority
- CN
- China
- Prior art keywords
- log
- log file
- backtracked
- backtracking
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000004044 response Effects 0.000 claims description 4
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000004891 communication Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 13
- 238000012550 audit Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 8
- 230000006837 decompression Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001174 ascending effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005201 scrubbing Methods 0.000 description 1
- 238000010845 search algorithm Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/14—Details of searching files based on file metadata
- G06F16/144—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
Abstract
Database history log backtracking method, device, system, equipment and storage medium, wherein the method comprises the following steps: responding to a log backtracking instruction of a user, inquiring a log file set stored by database instance operation equipment of the user, and positioning a log file to be backtracked; and acquiring a log file to be backtracked, and writing the log file to be backtracked into the database storage system of the user, so that the user backtracks the log file written into the database storage system. By adopting the scheme of the embodiment of the specification, the data security of the database can be further improved.
Description
Technical Field
The embodiment of the specification relates to the technical field of database security, in particular to a database history log backtracking method, device, system, equipment and storage medium.
Background
The database stores the core data of the system, and the security problem of the database becomes a significant source of leakage and tampering in the traditional environment. And at the cloud end, the threats faced by the database are further amplified. Therefore, the operation behavior of the cloud database, especially the audit log of the execution record of the full-quantity Structured Query Language (SQL), is very important, and is the most basic requirement for ensuring the security of the cloud database.
The SQL audit function of the existing cloud database is usually value-added service, only SQL logs after the SQL audit function is started can be inquired, and many users do not start the audit function when the system is started due to cost consideration. Once the data is tampered with, since the SQL audit function is not opened, a tamperer or a tampering event is required to be traced and the data is recovered, so that the user often repels.
In the face of such a scenario, the existing cloud database usually recommends a user to open SQL audit by default when a database instance is created, so as to improve the security level of the instance and deal with the problem of tracing when data is tampered. However, this approach increases the operation cost of the user, and is not acceptable for a large number of users with weak security awareness.
Therefore, how to obtain the historical log before the user starts the auditing function to guarantee the data security of the database is a technical problem which needs to be solved urgently.
Disclosure of Invention
In view of this, embodiments of the present disclosure provide a method, an apparatus, a system, a device, and a storage medium for database history log backtracking, so as to further improve data security of a database.
An embodiment of the present specification provides a database history log backtracking method, including:
responding to a log backtracking instruction of a user, inquiring a log file set stored by database instance operation equipment of the user, and positioning a log file to be backtracked;
and acquiring a log file to be backtracked, and writing the log file to be backtracked into the database storage system of the user, so that the user backtracks to the log file written into the database storage system.
Optionally, the querying, in response to a log backtracking instruction of a user, a log file set stored in the database instance running device of the user, and locating a log file to be backtracked includes:
obtaining a backtracking time range indicated by the log backtracking instruction;
and inquiring a log file matched with the backtracking time range in a log file set stored by the database instance operation equipment of the user, and determining the log file to be backtracked.
Optionally, the log file set is stored in a redundant storage space of the database instance operating device.
Optionally, the log file set is stored in a storage space of the database instance operating device in a compressed log file list form.
Optionally, the compressing log file list includes a first row of data recording log time information, and the querying a log file matched with the backtracking time range in a log file set stored in the database instance running device of the user determines the log file to be backtracked, and includes:
traversing a compressed log list stored by the database instance operation equipment, decompressing first row data of the compressed log list, and acquiring log time information recorded by each log file in the log file list;
and inquiring the log file of which the log time information is matched with the backtracking time range, and determining the log file to be backtracked.
Optionally, the querying a log file of which the log time information is matched with the backtracking time range, and determining the log file to be backtracked includes:
according to the time sequence, sequencing the log files in the log file set;
and determining a compressed log file to be backtracked by adopting a binary search method according to the log time information recorded in the log file and the backtracking time range.
Optionally, the obtaining a log file to be backtracked and writing the log file to be backtracked into the database storage system of the user includes:
decompressing the compressed log file to be backtracked to obtain a decompressed log file to be backtracked;
and collecting the decompressed log file to be backtracked, and writing the decompressed log file to be backtracked into the database storage system of the user.
Optionally, the method further comprises: and compressing and storing the backtracked historical log file in a database storage system of the user.
An embodiment of the present specification further provides a history log backtracking apparatus, where the apparatus includes:
the backtracking log positioning unit is suitable for responding to a log backtracking instruction of a user, inquiring a log file set stored in database instance operation equipment of the user and positioning a log file to be backtracked;
and the log backtracking unit is suitable for acquiring a log file to be backtracked and writing the log file to be backtracked into the database storage system of the user.
An embodiment of the present specification further provides a database history log backtracking system, where the system includes:
the database instance operation equipment cluster comprises a plurality of database instance operation equipment, and each database instance operation equipment is suitable for storing a log file set corresponding to a database instance operated on the database instance operation equipment by a user;
the historical log backtracking device is suitable for responding to a log backtracking instruction of a user, inquiring a log file set stored by the database instance operation equipment, positioning and acquiring a log file to be backtracked, and writing the log file to be backtracked into a database storage system of the user so that the user backtracks the log file written into the database storage system;
and the database storage system is suitable for storing the log files of the users, including the log files written by the historical log backtracking device.
The present specification further provides an electronic device, which includes a memory and a processor, where the memory stores computer instructions executable on the processor, and the processor executes the computer instructions to perform the steps of the method according to any one of the foregoing embodiments.
The present specification also provides a computer readable storage medium, on which computer instructions are stored, and the computer instructions execute the steps of the method of any one of the foregoing embodiments when executed.
Compared with the prior art, the technical scheme of the embodiment of the specification has the following beneficial effects:
the method comprises the steps of adopting database instance operation equipment to store a log file set, responding to a log backtracking instruction of a user, obtaining the stored log file set of the user from the database instance operation equipment of the user to be positioned and obtain a log file to be backtracked, and then writing the log file to be backtracked into a database storage system of the user, so that the user backtracks the log file written into the database storage system. By adopting the scheme of the embodiment of the specification, when a user has a historical log backtracking requirement, the log file stored by the database instance operation equipment of the user can be inquired according to the log backtracking instruction of the user, the log file to be backtracked is positioned, and the positioned log file to be backtracked is written into the database system of the user, so that the user backtracks the log file written into the database storage system, the data safety of the database can be guaranteed as much as possible under the condition that the safety consciousness of the user is weak, and the operation cost of the user is not increased.
Furthermore, by acquiring the log backtracking time range indicated by the log instruction, inquiring the log file matched with the backtracking time range in the log file set stored by the database instance operation equipment of the user, and determining the log file to be backtracked, the user requirement can be met more accurately, and the log file set outside the log backtracking time range is prevented from occupying the storage space of the database storage system of the user.
Furthermore, the log file is stored in the redundant storage space of the database instance operation equipment, so that the redundant space of the database instance operation machine can be fully utilized, and the storage resource and the operation and maintenance cost of a user are not increased.
Furthermore, the data volume of the storable historical log file can be expanded by adopting the compressed log list stored in the redundant space of the database instance operation machine, so that the time range of the log file which can be backtracked by the user can be expanded as much as possible, and the backtracking requirement of the historical log of the user can be met as much as possible.
Furthermore, as the log time information is recorded in the first row of data of the compressed log file list, the compressed log list stored by the database instance running device is traversed, and only the first row of data of the compressed log list is decompressed, so that the situation that the whole compressed log list is decompressed to take a large amount of decompression time and occupy a large amount of storage space can be avoided, the log time information recorded by each log file in the log list can be quickly obtained, and the historical log files can be quickly traced back.
Furthermore, according to the time sequence, the log files in the log file set are sequenced, and according to the log time information recorded in the log files and the backtracking time range, a binary search method is adopted to determine the compressed log files to be backtracked, so that the operation amount can be reduced, the compressed log files to be backtracked can be quickly positioned, and the backtracking efficiency of the historical log files can be improved.
Furthermore, the compressed log file to be backtracked is decompressed to obtain a decompressed log file to be backtracked, the decompressed log file to be backtracked is collected and written into the database storage system of the user to obtain a backtracked historical log file, so that backtracking of the historical log file can be realized, and the safety of the database is guaranteed.
Furthermore, in the database storage system of the user, the backtracked historical log file is compressed and stored, so that the storage resource of the user can be saved.
Drawings
FIG. 1 is an architectural diagram of a database log audit system;
FIG. 2 is a schematic structural diagram of a database history log backtracking system in an embodiment of the present disclosure;
FIG. 3 is a flowchart of a database history log backtracking method according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram illustrating a process flow of database history log backtracking in a scenario in an embodiment of the present specification;
fig. 5 is a schematic structural diagram of a history log backtracking apparatus in an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device in an embodiment of the present specification.
Detailed Description
Currently, as shown in fig. 1, an architecture diagram of a database log auditing system 10 includes: a database instance execution device 11x, a message queue 12, and a database storage system 13. Referring to fig. 1, the database instance running device cluster 11 is formed by a plurality of database instance running devices 11x, and the cloud database SQL auditing scheme generally includes the following steps:
on the database instance running device 11x, the SQL log file is written into a local storage space, for example, into a local disk, and then, in a push or pull manner, the SQL log file local to the database instance running device 11x, which opens the SQL audit function instance, is written into the message queue 12, and the SQL in the message queue 12 may be written into the database storage system 13 in a message forwarding manner (for example, may be written by a message forwarder broker, specifically, may be written by a plurality of message forwarders, such as broker 1-broker n). The user can query the log file stored in the database storage system 13 through the query interface.
However, as described in the background section, the SQL auditing function of the existing cloud database is usually a value-added service, and only the SQL log after the SQL auditing function is started can be queried, and many users do not start the auditing function when starting to use the database system due to cost considerations. Once the data is tampered with, since the SQL audit function is not opened, a tamperer or a tampering event is required to be traced and the data is recovered, so that the user often repels.
In the face of such a scenario, the existing cloud database usually recommends a user to open SQL audit by default when a database instance is created, so as to improve the security level of the instance and deal with the problem of tracing when data is tampered. However, this approach increases the operation cost of the user, and is not acceptable for a large number of users with weak security awareness.
In view of the above problems, embodiments of the present specification provide a database history log backtracking scheme, so that a user can backtrack a history log before an audit function is not started, and the security performance of a database is further improved.
In order to make those skilled in the art better understand and implement the embodiment of the present specification, a history log backtracking scheme adopted by the embodiment of the present specification is described below with reference to a schematic structural diagram of a database history log backtracking system shown in fig. 2.
As shown in fig. 2, the database history log backtracking system 20 may include: the system comprises a database instance running device 21x, a database storage system 22 and a history log backtracking device 23, wherein a plurality of database instance running devices 21x can form a database instance running device cluster 21.
In a specific implementation, the database instance running device 21x is adapted to store a log file set corresponding to a database instance of a user on the database instance running device; a database storage system 22 adapted to store log files of users, including the log files traced back by the history log tracing device 23; the historical log backtracking device 23 is adapted to query a log file set stored in the database instance operating device in response to a log backtracking instruction of a user, locate and acquire a log file to be backtracked, and write the log file to be backtracked into the database storage system of the user, so that the user backtracks the log file written into the database storage system.
The database instance running device 21x may store the SQL log by using the redundant storage space after the SQL log is landed, so that the redundant space of the database instance running machine may be fully used without increasing the storage resource and operation and maintenance cost of the user.
Referring to the drawings, a database history log backtracking method adopted in the embodiments of the present specification is described in detail by specific embodiments, and as shown in a flowchart of the database history log backtracking method shown in fig. 3, the following steps may be specifically adopted:
s31, responding to the log backtracking instruction of the user, inquiring the log file set stored by the database instance operation equipment of the user, and positioning the log file to be backtracked.
In particular implementations, a user may interact with the database system through an interactive interface such as a database console.
The database system can generate the log backtracking instruction based on any log backtracking interactive operation of a user through a keyboard, touch, voice and the like. It should be understood that, in the embodiment of the present specification, no limitation is imposed on what way a user specifically interacts with a database system, and the above interaction way is merely an example.
In specific implementation, a user can give a backtracking time range of a log file through interactive operation, correspondingly, a log backtracking instruction can include backtracking time range information, a database system can obtain the backtracking time range indicated by the log backtracking instruction, query a log file matched with the backtracking time range in a log file set stored by database instance running equipment of the user, and determine the log file to be backtracked.
As a specific example, in response to a log backtracking instruction of a user, based on a backtracking time range indicated by the backtracking instruction, traversing a log file set stored by the database instance running device of the user, acquiring log time information recorded by each log file in the log file set, querying a log file whose log time information matches the backtracking time range, and determining the log file as a log file to be backtracked.
The backtracking instruction may include a specific backtracking time range indicated by the user, and the time range may be a specific time interval, or may be a start or end of a time when a certain event occurs. For example, a certain user indicated a time range of: 1 month 1 of 2018-to date. As another example, another user indicates a time range of: a database instance is created from the time of day to the time of the monday (where monday may be the time at which the database system failed).
In a specific implementation, there may be multiple database instance operating devices, for example, in a cloud database system, there may be a large number of database instance operating devices forming an equipment cluster, and the equipment cluster provides services for a large number of users in a unified manner.
According to research, a large amount of redundant storage space exists in the database instance operation equipment, and statistics shows that the utilization rate of disks of a large amount of cloud database users is less than 50%, so that historical file logs can be stored by using the redundant storage space of the database instance operation equipment. After the SQL log is landed, the SQL log does not need to be cleaned up immediately, and the SQL log can be stored by utilizing the redundant storage space of the database instance operation equipment. In order to save the storage space and make the redundant storage space more fully utilized, the log file can be compressed and then stored.
In specific implementation, in order to ensure that the database instance operation device has enough redundancy space, a corresponding cleaning strategy can be set. As an example, a scrubbing policy may include the following three components:
1) storing SQL log files of preset days;
2) storing SQL log files in a preset space range;
3) if the utilization rate of the disk reaches a preset threshold (for example, 90%), the SQL log file is automatically cleaned without observing the policy 1 and the policy 2, so as to ensure the availability of the database instance.
In specific implementation, the log file set stored in the redundant storage space of the database instance operation device may be stored in a list form to form a log list, which may be stored in a time sequence. And positioning the log file to be backtracked according to the log time information recorded in the log file and the backtracking time range.
In a specific implementation, in order to improve the positioning efficiency, the log files in the log file set may be sorted according to a time sequence. And then, according to the time information recorded in the log file and the backtracking time range, determining a compressed log file to be backtracked by adopting a binary search method.
In a specific implementation, the log time information recorded in the log file is usually stored in the first record of the log file, so that the first record of the log file can be matched with the backtracking time range to determine whether the log file is a log file to be backtracked by the user.
Among them, the Binary Search algorithm is also called Binary Search. In the log list stored according to the time sequence, it is assumed that the elements in the log list are arranged in ascending order according to the time sequence, the time information recorded in the log list is matched with the searched backtracking time range, and the time range is divided into the start time and the end time. Finding out a corresponding start file by using a binary search method according to the time sequence of the log files by using the start time; and finding the corresponding end file by using the end time through a binary search method. Then, the log files from the start file to the end file are obtained as the log files in the backtracking time range.
And S32, acquiring a log file to be backtracked, and writing the log file to be backtracked into the database storage system of the user, so that the user backtracks to the log file written into the database storage system.
In a specific implementation, the log file to be backtracked may be pulled from the database instance running device, and then the log file to be backtracked may be collected and written into the database storage system to obtain a backtracked historical log file.
The above steps can be specifically executed by a special history log backtracking device or a backtracking module.
Therefore, by adopting the scheme of the embodiment of the specification, the log file stored by the database instance operation equipment can be fully utilized, when a user has a historical log backtracking requirement, the log file stored by the database instance operation equipment of the user can be inquired according to a log backtracking instruction of the user, the log file to be backtracked is positioned, and the positioned log file to be backtracked is written into the database system of the user, so that the user backtracks the log file written into the database storage system, the data safety of the database can be guaranteed as much as possible under the condition that the safety awareness of the user is weak, and the operation cost of the user is not increased.
In addition, the log file matched with the backtracking time range in the log file set stored by the database instance operation equipment of the user is inquired by obtaining the log backtracking time range indicated by the log instruction, and the log file is determined as the log file to be backtracked, so that the user requirement can be met more accurately, and the log file set outside the log backtracking time range is prevented from occupying the storage space of the database storage system of the user.
In addition, the log file is stored in the redundant storage space of the database instance operation equipment, so that the redundant space of the database instance operation machine can be fully utilized, and the storage resource and the operation and maintenance cost of a user are not increased.
In order to make those skilled in the art better understand and implement the retrospective solution of the embodiments of the present disclosure, the following description is provided by way of a specific application scenario.
Referring to fig. 4, a schematic diagram of a database history log backtracking processing flow in a scenario is shown, where the database history log backtracking system 40 may include: the database instance runs an equipment cluster 41, a database storage system 42 and a history log backtracking device 43. In addition, in order to ensure the security of the database, a message queue 44 may be further disposed between the database instance operating device cluster 41 and the database storage system 42, and is used for collecting log files on the database instance operating device cluster 41 in real time and writing the log files into the database storage system 42.
For example, for a cloud database, a user may select one or more database instances to start SQL auditing functions through a cloud database console. As an alternative, at this time, the user may select whether to import the historical SQL log file, and if not, directly perform real-time SQL log collection, and if necessary, as shown in fig. 4, step S41 may be executed, where the user inputs a database historical log backtracking instruction, where the instruction indicates a time range of backtracking logs, and performs historical log data backtracking while acquiring the real-time SQL logs.
The database instance execution device cluster 41 includes a plurality of database instance execution devices 41x, and each database instance execution device 41x is adapted to store a log file set corresponding to a database instance executed on the database instance execution device 41x by a user.
In a specific implementation, the database instance running device cluster 41 may be composed of a plurality of database instance running devices 41x, and each database instance running device 41x may first log the SQL log, and specifically, may write the SQL log into the local disk in a synchronous or asynchronous log printing manner.
It should be noted that, in this embodiment of the present disclosure, after the SQL log is written into the local disk, it is not cleaned immediately, but may be stored for a period of time, and the redundant storage space of the database instance running device 41x is fully utilized to store the historical log file. In order to make the redundant space more fully utilized, after the SQL log is landed, the SQL log may be stored on the database instance device in a compressed manner, for example, in a compressed log list manner.
Thereafter, the history log backtracking means 43 may execute step S42 to locate the backtracking file. In order to improve the positioning efficiency, the log files can be sorted first, and then the positioning is performed accurately. As a specific implementation example, the following is specific:
the history log backtracking device 43 may traverse a compressed log list stored in the redundant storage space of the database instance operation device, decompress time data in a first row of the compressed log list, obtain log time information recorded by each log file in the log file list, and sort the log files in the log file set according to a time sequence; then, the history log backtracking device 43 determines the compressed log file to be backtracked by using a binary search method according to the log time information recorded in the log file and the backtracking time range.
Thereafter, the history log backtracking means 43 may execute step S43 to obtain the log file to be backtracked. As a specific implementation example, the specific steps are as follows: and decompressing the compressed log file to be backtracked according to the log file list located in the step S42 to obtain a decompressed log file to be backtracked.
Finally, the history log backtracking means 43 may execute step S44 to write the acquired log file to be backtracked into the database storage system 42. Specifically, the history log backtracking device 43 may collect the decompressed log file to be backtracked, and write the decompressed log file to be backtracked into the database storage system 42, so as to obtain a backtracked history log file.
To make storage resources more fully utilized, the history log file written to the database storage system 42 may be compressed for storage.
If the user starts the SQL auditing function, the log file stored on each device in the database instance running device cluster 41 may be pulled in real time through the message queue 44, or the log file generated in real time on each device may be pushed to the message queue 44 by the database instance running device cluster 41. Specifically, a corresponding proxy client may be provided on the message queue 44 and the database instance execution device, and used as a message forwarder (e.g., broker) to write the log file generated on the database instance execution device 41x into the database storage system 42 in real time.
An embodiment of the present specification further provides a history log backtracking apparatus, as shown in fig. 5, the history log backtracking apparatus 50 may include: a backtracking log locating unit 51 and a log backtracking unit 52, wherein:
a backtracking log positioning unit 51, adapted to respond to a log backtracking instruction of a user, query a log file set stored in the database instance running device of the user to position a log file to be backtracked;
the log backtracking unit 52 is adapted to obtain a log file to be backtracked, and write the log file to be backtracked into the database storage system of the user.
In a specific implementation, the backtracking log locating unit 51 may obtain a backtracking time range indicated by the log backtracking instruction, query a log file matched with the backtracking time range in a log file set stored in the database instance running device of the user, and determine the log file to be backtracked.
In a specific implementation, the log file set may be stored in a redundant storage space of the database instance execution device.
As an alternative, the log file collection is stored in the storage space of the database instance running device in the form of a compressed log file list. Wherein the first line of data of the compressed log file list may record log time information.
As a specific example, the compressed log list stored by the database instance running device is traversed, the first row of data of the compressed log list is decompressed, log time information recorded by each log file in the log file list is acquired, a log file of which the log time information is matched with the backtracking time range is queried, and the log file is determined to be a log file to be backtracked.
In a specific implementation, the backtracking log locating unit 51 may traverse a compressed log list stored in a redundant space of the database instance operation device, decompress time data in a first row of the compressed log list, and obtain time information recorded by each log file in the log file list.
In a specific implementation, the backtracking log locating unit 51 may include: a sorting subunit 511 and a positioning subunit 512, wherein:
the sorting subunit 511 is adapted to sort the log files in the log file set according to a time sequence;
the positioning sub-unit 512 is adapted to determine a compressed log file to be backtracked by using a binary search method according to the log time information recorded in the log file and the backtracking time range.
In a specific implementation, the log backtracking unit 52 may include: a decompression sub-unit 521 and a writing sub-unit 522, wherein:
the decompression sub-unit 521 is adapted to decompress the compressed log file to be backtracked to obtain a decompressed log file to be backtracked;
the writing subunit 522 is adapted to collect the decompressed log file to be traced back, and write the decompressed log file to be traced back into the database storage system of the user.
An embodiment of the present specification further provides an electronic device, where the history log backtracking apparatus in the foregoing embodiment may be loaded in the electronic device in a form of a computer-executable instruction (such as a program), so as to implement the database history log backtracking method in the embodiment of the present specification.
In an alternative implementation, referring to fig. 6, the electronic device 60 comprises a memory 61 and a processor 62, the memory 61 stores computer instructions executable on the processor, and the processor 62 executes the computer instructions to perform the steps of the method of any of the foregoing method embodiments. Further, the electronic device 60 may also include a communication bus 63, a communication interface 64, and a user interaction interface 65;
in a specific implementation, the number of the memory 61, the processor 62, the communication bus 63 and the communication interface 64 may be one or more, and the memory 61, the processor 62, the communication interface 64 and the user interaction interface 65 may complete communication with each other through the communication bus 63. It will be apparent that the illustrated communication connections of the memory 61, processor 62, communication bus 63 and communication interface 64 are merely optional.
As an alternative example, the communication interface 64 may be an interface of a communication module for performing network communication; the processor 62 may be a central processing unit CPU or an application specific integrated circuit asic or one or more integrated circuits configured to implement embodiments of the present invention.
The memory 61 may comprise a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
The memory 61 stores one or more computer-executable instructions, and the processor 62 calls the one or more computer-executable instructions to execute the database history log backtracking method provided in the embodiment of the present specification.
The present specification further provides a computer readable storage medium, on which computer instructions are stored, where the computer instructions are adapted to execute the steps of the database history log backtracking method according to any one of the foregoing embodiments when executed. For specific steps, reference may be made to the foregoing embodiments, which are not described herein again.
Specific implementation manners, operation principles, specific actions and effects of each device, system, equipment or system in the embodiments of the present invention may be referred to in the detailed descriptions of the corresponding method embodiments.
In this embodiment of the present specification, the Database may be a relational Database, or may also be a non-relational Database, for example, a cloud Database, and more specifically, a Distributed Relational Database Service (DRDS) Database. It should be noted that, a specific database type has no influence on the backtracking scheme of the embodiment of the present specification, and the embodiment of the present specification is applicable to various types of databases.
Although the embodiments of the present invention are disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected by one skilled in the art without departing from the spirit and scope of the embodiments of the invention as defined in the appended claims.
Claims (12)
1. A database history log backtracking method comprises the following steps:
responding to a log backtracking instruction of a user, inquiring a log file set stored by database instance operation equipment of the user, and positioning a log file to be backtracked;
and acquiring a log file to be backtracked, and writing the log file to be backtracked into the database storage system of the user, so that the user backtracks to the log file written into the database storage system.
2. The method of claim 1, wherein the querying a log file set stored by a database instance running device of a user in response to a log backtracking instruction of the user and locating a log file to be backtracked comprises:
obtaining a backtracking time range indicated by the log backtracking instruction;
and inquiring a log file matched with the backtracking time range in a log file set stored by the database instance operation equipment of the user, and determining the log file to be backtracked.
3. The method of claim 1 or 2, wherein the set of log files is stored in a redundant storage space of the database instance runtime device.
4. The method of claim 2, wherein the collection of log files is stored in a compressed log file list in a storage space of the database instance runtime.
5. The method according to claim 4, wherein the first row of data of the compressed log file list records log time information, and the querying a log file matching the backtracking time range in the log file set stored by the database instance running device of the user determines the log file to be backtracked, and includes:
traversing a compressed log list stored by the database instance operation equipment, decompressing first row data of the compressed log list, and acquiring log time information recorded by each log file in the log file list;
and inquiring the log file of which the log time information is matched with the backtracking time range, and determining the log file to be backtracked.
6. The method according to claim 5, wherein the querying the log file whose each log time information matches the backtracking time range, and determining the log file to be backtracked includes:
according to the time sequence, sequencing the log files in the log file set;
and determining a compressed log file to be backtracked by adopting a binary search method according to the log time information recorded in the log file and the backtracking time range.
7. The method of claim 4, wherein the obtaining the log file to be backtracked and writing the log file to be backtracked into the database storage system of the user comprises:
decompressing the compressed log file to be backtracked to obtain a decompressed log file to be backtracked;
and collecting the decompressed log file to be backtracked, and writing the decompressed log file to be backtracked into the database storage system of the user.
8. The method of claim 7, further comprising:
and compressing and storing the backtracked historical log file in a database storage system of the user.
9. A history log backtracking apparatus, comprising:
the backtracking log positioning unit is suitable for responding to a log backtracking instruction of a user, inquiring a log file set stored in database instance operation equipment of the user and positioning a log file to be backtracked;
and the log backtracking unit is suitable for acquiring a log file to be backtracked and writing the log file to be backtracked into the database storage system of the user.
10. A database history log backtracking system, comprising:
the database instance operation equipment cluster comprises a plurality of database instance operation equipment, and each database instance operation equipment is suitable for storing a log file set corresponding to a database instance operated on the database instance operation equipment by a user;
the historical log backtracking device is suitable for responding to a log backtracking instruction of a user, inquiring a log file set stored by the database instance operation equipment, positioning and acquiring a log file to be backtracked, and writing the log file to be backtracked into a database storage system of the user so that the user backtracks the log file written into the database storage system;
and the database storage system is suitable for storing the log files of the users, including the log files written by the historical log backtracking device.
11. An electronic device comprising a memory and a processor, the memory having stored thereon computer instructions executable on the processor, wherein the processor, when executing the computer instructions, performs the steps of the method of any one of claims 1 to 8.
12. A computer readable storage medium having stored thereon computer instructions which, when executed, perform the steps of the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010520272.XA CN111737203A (en) | 2020-06-09 | 2020-06-09 | Database history log backtracking method, device, system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010520272.XA CN111737203A (en) | 2020-06-09 | 2020-06-09 | Database history log backtracking method, device, system, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111737203A true CN111737203A (en) | 2020-10-02 |
Family
ID=72650090
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010520272.XA Pending CN111737203A (en) | 2020-06-09 | 2020-06-09 | Database history log backtracking method, device, system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111737203A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112445773A (en) * | 2021-02-01 | 2021-03-05 | 北京志翔科技股份有限公司 | Method and device for generating decompression log of compressed file and readable storage medium |
| CN112596998A (en) * | 2020-12-30 | 2021-04-02 | 福州掌中云科技有限公司 | SQL (structured query language) auditing method and equipment based on application layer |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011013792A (en) * | 2009-06-30 | 2011-01-20 | Fujitsu Ltd | Device, method and program for control of database in program model inspection |
| CN104516959A (en) * | 2014-12-18 | 2015-04-15 | 杭州华为数字技术有限公司 | Method and device for managing database logs |
| CN104679841A (en) * | 2015-02-11 | 2015-06-03 | 北京京东尚科信息技术有限公司 | Consumption terminal data flow copying method and system |
| CN107145403A (en) * | 2017-04-20 | 2017-09-08 | 浙江工业大学 | The relevant database data retrogressive method of web oriented development environment |
| CN107918621A (en) * | 2016-10-10 | 2018-04-17 | 阿里巴巴集团控股有限公司 | Daily record data processing method, device and operation system |
| CN109274540A (en) * | 2018-11-16 | 2019-01-25 | 四川长虹电器股份有限公司 | A kind of web access log processing method based on storm |
| CN110309130A (en) * | 2018-03-21 | 2019-10-08 | 中国人民财产保险股份有限公司 | A kind of method and device for host performance monitor |
| CN110674079A (en) * | 2019-09-10 | 2020-01-10 | 杭州数梦工场科技有限公司 | Log file recording method and device |
-
2020
- 2020-06-09 CN CN202010520272.XA patent/CN111737203A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011013792A (en) * | 2009-06-30 | 2011-01-20 | Fujitsu Ltd | Device, method and program for control of database in program model inspection |
| CN104516959A (en) * | 2014-12-18 | 2015-04-15 | 杭州华为数字技术有限公司 | Method and device for managing database logs |
| CN104679841A (en) * | 2015-02-11 | 2015-06-03 | 北京京东尚科信息技术有限公司 | Consumption terminal data flow copying method and system |
| CN107918621A (en) * | 2016-10-10 | 2018-04-17 | 阿里巴巴集团控股有限公司 | Daily record data processing method, device and operation system |
| CN107145403A (en) * | 2017-04-20 | 2017-09-08 | 浙江工业大学 | The relevant database data retrogressive method of web oriented development environment |
| CN110309130A (en) * | 2018-03-21 | 2019-10-08 | 中国人民财产保险股份有限公司 | A kind of method and device for host performance monitor |
| CN109274540A (en) * | 2018-11-16 | 2019-01-25 | 四川长虹电器股份有限公司 | A kind of web access log processing method based on storm |
| CN110674079A (en) * | 2019-09-10 | 2020-01-10 | 杭州数梦工场科技有限公司 | Log file recording method and device |
Non-Patent Citations (2)
| Title |
|---|
| 刘浩阳;: "MS SQL数据库在线取证研究", 信息网络安全, no. 09, 10 September 2016 (2016-09-10) * |
| 王伟平;张玮;: "Informix数据库安全审计系统设计与实现", 计算技术与自动化, no. 04, 15 December 2007 (2007-12-15) * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112596998A (en) * | 2020-12-30 | 2021-04-02 | 福州掌中云科技有限公司 | SQL (structured query language) auditing method and equipment based on application layer |
| CN112445773A (en) * | 2021-02-01 | 2021-03-05 | 北京志翔科技股份有限公司 | Method and device for generating decompression log of compressed file and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109034993B (en) | Account checking method, account checking equipment, account checking system and computer readable storage medium | |
| US11455217B2 (en) | Transaction consistency query support for replicated data from recovery log to external data stores | |
| US10810074B2 (en) | Unified error monitoring, alerting, and debugging of distributed systems | |
| US9727625B2 (en) | Parallel transaction messages for database replication | |
| CN109284251A (en) | Blog management method, device, computer equipment and storage medium | |
| US8738767B2 (en) | Mainframe management console monitoring | |
| US10628250B2 (en) | Search for information related to an incident | |
| CN111581054A (en) | ELK-based log point-burying service analysis and alarm system and method | |
| CN111563130A (en) | Data credible data management method and system based on block chain technology | |
| US8738768B2 (en) | Multiple destinations for mainframe event monitoring | |
| CN113688288B (en) | Data association analysis method, device, computer equipment and storage medium | |
| CN111737203A (en) | Database history log backtracking method, device, system, equipment and storage medium | |
| CN107656999B (en) | Call history tracing method and device, electronic equipment and storage medium | |
| CN109992476A (en) | A kind of analysis method of log, server and storage medium | |
| CN109902070B (en) | WiFi log data-oriented analysis storage search method | |
| CN111342994A (en) | Network management system and method | |
| CN114385551B (en) | Log time-sharing management method, device, equipment and storage medium | |
| CN116204540A (en) | Operation log recording method, device, equipment and storage medium | |
| CN106802922B (en) | Tracing storage system and method based on object | |
| CN111813833B (en) | Real-time two-degree communication relation data mining method | |
| US8775528B2 (en) | Computer readable recording medium storing linking keyword automatically extracting program, linking keyword automatically extracting method and apparatus | |
| CN112579552A (en) | Log storage and calling method, device and system | |
| CN109992475A (en) | A kind of processing method of log, server and storage medium | |
| CN117240505B (en) | Early warning processing method and system based on data management platform | |
| US11630818B2 (en) | Iterative performance analysis with interval expansion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |