CN115470082A - Log breakpoint and log text data acquisition method of Oracle database - Google Patents
Log breakpoint and log text data acquisition method of Oracle database Download PDFInfo
- Publication number
- CN115470082A CN115470082A CN202210908654.9A CN202210908654A CN115470082A CN 115470082 A CN115470082 A CN 115470082A CN 202210908654 A CN202210908654 A CN 202210908654A CN 115470082 A CN115470082 A CN 115470082A
- Authority
- CN
- China
- Prior art keywords
- text data
- log text
- log
- scn
- starting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of computers, in particular to a log breakpoint of an Oracle database, a log text data acquisition method, a log text data acquisition device and a log text data acquisition medium, and aims to solve the problems that the analysis efficiency is low and log text data is easy to be omitted and analyzed in the breakpoint continuous acquisition process of the log text data. For this purpose, the method can regularly save the starting SCN of each piece of log text data generated by the OracleLogMiner obtained by analysis into a preset distributed database. When the analysis thread is restarted, the starting SCN of the last log text data obtained before the analysis thread is restarted and stored in the distributed database can be read, and the starting SCN is used as the initial position of the log text data after the analysis thread is restarted to analyze the log text data so as to realize the acquisition of Oracle log text data, avoid the repeated analysis of the log text data, avoid the omission of some log text data and improve the acquisition efficiency of the log text data.
Description
Technical Field
The invention relates to the technical field of computers, and particularly provides a method, a device and a medium for collecting log breakpoints and log text data of an Oracle database.
Background
In the data synchronization process of the heterogeneous database from the Oracle database to other databases (such as MySQL database), the data analysis of the Oracle database and the storage of the data in the MySQL database are included. For the first step, logMiner developed by Oracle can be used to help achieve parsing and archiving of log text data.
However, during the parsing of the log text data, the parsing thread may be interrupted due to some reasons (such as a thread down machine or a human restart). In this case, the log text data may be repeatedly parsed in a large amount already parsed, because the log text data needs to be re-parsed from the beginning; or restarting parsing based on the SCN actually generated by Oracle may result in some log text data being missed and not parsed.
The patent application with the application number of CN201810150258.8 and the publication number of CN110162448A and the patent name of the method and the device for collecting the logs discloses a scheme for saving the number of collecting sub-nodes during log collection so as to accurately and quickly determine the breakpoint position of a certain log during previous collection and realize breakpoint continuous collection. However, when the breakpoint continuous mining is realized, the analyzed log needs to be locally stored and then analyzed, and the corresponding SCN state is modified after the analysis is completed. When the analysis process is restarted and the original log file is repeatedly analyzed, the log file corresponding to the SCN may need to be completely repeatedly analyzed, and the analysis efficiency is low.
Accordingly, there is a need in the art for a new log breakpoint resume collection scheme that addresses the above-mentioned problems.
Disclosure of Invention
In order to overcome the defects, the invention is provided to solve or at least partially solve the problems that the analysis efficiency is low and the log text data is easy to be missed and analyzed in the breakpoint continuous collection process of the log text data.
In a first aspect, the present invention provides a log breakpoint collection method for an Oracle database, where the method includes:
acquiring a starting SCN of each piece of log text data according to an analysis result of each piece of log text data generated by the Oracle LogMiner by an analysis thread;
and storing the starting SCN of the last piece of currently analyzed log text data into a preset distributed database every preset time.
In a technical solution of the log breakpoint collection method for the Oracle database, the obtaining a starting SCN of each log text data according to an analysis result of each log text data generated by an analysis thread to the Oracle LogMiner includes:
analyzing the log text data by applying the analysis thread to obtain an analysis result of the log text data;
acquiring a starting SCN of the log text data according to the analysis result;
and storing the starting SCN of the log text data into a message queue.
In a technical solution of the log breakpoint collection method for the Oracle database, storing the starting SCN of the last piece of currently parsed log text data into a preset distributed database every preset time includes:
and reading the starting SCN of the last piece of currently analyzed log text data from the message queue at preset time intervals, and storing the SCN into the distributed database.
In a technical solution of the log breakpoint collection method of the Oracle database, the method further includes:
when the starting SCN of the last piece of log text data which is analyzed at present is not stored successfully, one starting SCN is set manually as the starting SCN of the last piece of log text data.
In a technical solution of the log breakpoint collection method for an Oracle database, the storing the starting SCN of the last log text data in a preset distributed database further includes:
storing the starting SCN of the last log text data into the distributed database and covering the starting SCN stored in the distributed database in advance
In a second aspect, the present invention provides a method for collecting log text data of an Oracle database, wherein the method includes:
when an analysis thread is applied to analyze the log text data generated by the Oracle LogMiner, if the analysis thread is restarted, reading a starting SCN (sequence coding network) which stores the last log text data obtained by analysis before the restart of the analysis thread in a preset distributed database; the SCN for starting the last log text data is obtained according to the log breakpoint collection method of the Oracle database in any one of the claims 1 to 5;
taking the data position corresponding to the SCN as the initial position of the log text data after the analysis thread is restarted;
analyzing each piece of log text data according to the initial position to obtain an analysis result of each piece of log text data;
and acquiring service application data according to the analysis result so as to realize the acquisition of Oracle log text data.
In one technical solution of the above method for collecting log text data, the analysis result includes service data to be processed of each log text data and a start SCN;
the method further comprises the following steps:
and storing the starting SCN obtained by analyzing each piece of log text data into a message queue.
In a technical solution of the above method for acquiring log text data, the acquiring service application data according to the analysis result includes:
performing service processing on the service data to be processed to obtain service application data;
and storing the service application data into a message queue for consumption by a consumer.
In a third aspect, a control device is provided, which includes a processor and a storage device, where the storage device is adapted to store a plurality of program codes, and the program codes are adapted to be loaded and run by the processor to execute the method for collecting breakpoint log data of Oracle database according to any one of the above technical solutions, or the method for collecting text log data of Oracle database according to any one of the above technical solutions.
In a fourth aspect, a computer-readable storage medium is provided, in which a plurality of program codes are stored, and the program codes are suitable for being loaded and executed by a processor to execute the log breakpoint collecting method of the Oracle database according to any one of the above technical solutions or the log text data collecting method of the Oracle database according to any one of the above technical solutions.
One or more technical schemes of the invention at least have one or more of the following beneficial effects:
in the technical scheme of the invention, the method can regularly save the starting SCN of the last log text data generated by the Oracle LogMiner obtained by analysis to a preset distributed database. When the analysis thread is restarted, the starting SCN of the last log text data obtained before the analysis thread is restarted and stored in the distributed database can be read, and the starting SCN is used as the initial position of the log text data after the analysis thread is restarted to analyze the log text data so as to realize the acquisition of Oracle log text data. Through the configuration mode, the method and the device can avoid the situation that the analyzed log text data needs to be repeatedly analyzed when the analysis thread is restarted for some reasons, and also avoid the situation that some log text data are missed after the analysis thread re-enlightens. The method for saving and starting SCN does not need to store log text data, and effectively improves the log text data acquisition efficiency of the Oracle database on the premise of avoiding log text data omission.
Drawings
The disclosure of the present invention will become more readily understood with reference to the accompanying drawings. As is readily understood by those skilled in the art: these drawings are for illustrative purposes only and are not intended to constitute a limitation on the scope of the present invention. Wherein:
FIG. 1 is a flow chart illustrating the main steps of a log breakpoint collection method of an Oracle database according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating the main steps of a log text data collection method of an Oracle database according to an embodiment of the present invention;
fig. 3 is a flowchart illustrating main steps of a method for collecting log text data of an Oracle database according to an embodiment of the present invention.
Detailed Description
Some embodiments of the invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only for explaining the technical principle of the present invention, and are not intended to limit the scope of the present invention.
In the description of the present invention, a "module" or "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, may comprise software components such as program code, or may be a combination of software and hardware. The processor may be a central processing unit, a microprocessor, a digital signal processor, or any other suitable processor. The processor has data and/or signal processing functionality. The processor may be implemented in software, hardware, or a combination thereof. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random-access memory, and the like. The term "A and/or B" denotes all possible combinations of A and B, such as only A, only B or both A and B. The term "at least one A or B" or "at least one of A and B" means similar to "A and/or B" and may include only A, only B, or both A and B. The singular forms "a", "an" and "the" may include plural forms as well.
Referring to fig. 1, fig. 1 is a flow chart illustrating main steps of a log breakpoint collection method for an Oracle database according to an embodiment of the present invention. As shown in fig. 1, the log breakpoint collection method of the Oracle database in the embodiment of the present invention mainly includes the following steps S101 to S102.
Step S101: and acquiring the starting SCN of each piece of log text data according to the analysis result of each piece of log text data generated by the Oracle LogMiner by the analysis thread.
In this embodiment, the analysis thread may be applied to analyze the log text data generated by Oracle LogMiner, and the starting SCN of the log text data may be obtained according to the analysis result. The SCN (System Change Number) is a Number that is automatically maintained by a DBMS (Database Management System) to accumulate and increase after the Oracle Database is updated. In the Oracle database, there are four SCNs, which are: the method comprises the steps of checking a system check point SCN, checking a data file SCN, starting the SCN and stopping the SCN. In the present embodiment, what needs to be acquired is the start SCN of each piece of log text data. LogMiner is an analysis tool provided by Oracle corporation, with which specific content in an Oracle archive log file can be easily obtained.
In one embodiment, the resolution thread may be a single resolution thread and a batch resolution thread.
Step S102: and storing the starting SCN of the last piece of currently analyzed log text data into a preset distributed database every preset time.
In this embodiment, the starting SCN of the last piece of log text data that is currently analyzed may be stored in the preset distributed database at preset time intervals.
In one embodiment, the preset time period may be 5 minutes.
In one embodiment, the distributed database may be zookeeper. Among them, zookeeper is a kind of distributed database, which mainly solves some data management problems often encountered in distributed applications, such as: unified naming services, state synchronization services, cluster management, management of distributed application configuration items, and the like.
Based on the above steps S101 to S102, the embodiment of the present invention can store the starting SCN obtained by analyzing the last log text data generated by the Oracle LogMiner by the analysis thread into the preset distributed database at regular time, so that the starting SCN of the log text data analyzed by the analysis thread can be effectively obtained at any time.
Step S101 and step S102 will be further described below.
In one implementation of the embodiment of the present invention, step S101 may further include the following steps S1011 to S1013:
step S1011: and analyzing the log text data by using an analysis thread to obtain an analysis result of the log text data.
In this embodiment, the log text data may be analyzed by the analysis thread to obtain an analysis result of each piece of log text data. The parsing result may include the to-be-processed service data of each piece of log text data and the start SCN.
Step S1012: and acquiring the starting SCN of the log text data according to the analysis result.
Step S1013: storing the starting SCN of the log text data in a message queue.
In this embodiment, the analysis result of each piece of log text data may be stored in a message queue, that is, the to-be-processed service data and the SCN of each piece of log text data are stored in the message queue.
In one embodiment, the message queue may be kafka.
In one implementation of the embodiment of the present invention, the step S102 may further include the following step S1021:
step S1021: and reading the starting SCN of the last log text data which is analyzed currently from the message queue at preset time intervals, and storing the starting SCN into a distributed database.
In this embodiment, at preset time intervals, the starting SCN of the last piece of currently analyzed log text data may be read from the message queue and stored in the distributed database.
In one embodiment, when the starting SCN of the last piece of currently parsed log text data is not successfully stored, a starting SCN is manually set as the starting SCN of the last piece of log text data.
In one embodiment, when the starting SCN of the last log text data is stored in the distributed database, the starting SCN stored in the distributed database in advance is overwritten. That is, only one startup SCN of the log text data is stored in the distribution database, which is the last startup SCN stored.
Furthermore, the invention also provides a log text data acquisition method of the Oracle database.
Referring to fig. 2, fig. 2 is a flow chart illustrating main steps of a method for collecting log text data of an Oracle database according to an embodiment of the present invention. As shown in fig. 2, the method for collecting log text data of an Oracle database in the embodiment of the present invention mainly includes the following steps S201 to S204.
Step S201: when the analysis thread is applied to analyze the log text data generated by the Oracle LogMiner, if the analysis thread is restarted, reading a starting SCN (sequence coding network) which stores the last log text data obtained by analysis before the restart of the analysis thread in a preset distributed database; the starting SCN of the last log text data is obtained according to the embodiment of the log breakpoint collection method of the Oracle database.
In this embodiment, if the parsing thread is restarted for some reason (e.g., a parsing thread down machine or an artificial reason), the stored starting SCN of the last log text data before restarting the parsing thread may be read from the preset distributed database.
In one embodiment, the parsing thread may be a single parsing thread or a batch parsing thread.
In one embodiment, the distributed database may be zookeeper.
Step S202: and taking the data position corresponding to the starting SCN as the initial position of the log text data after the restart of the analysis thread.
In this embodiment, the acquired data location corresponding to the start SCN may be used as the start location of the log text data after the analysis thread is restarted, that is, the start location of the log text data after the analysis thread is restarted is determined according to the start SCN.
Step S203: and analyzing each piece of log text data according to the initial position to obtain an analysis result of each piece of log text data.
In this embodiment, each piece of log text data generated by Oracle LogMiner may be parsed from the start position to obtain a parsing result.
In one embodiment, the parsing result may include to-be-processed service data and a start SCN corresponding to each piece of log text data.
Step S204: and acquiring service application data according to the analysis result so as to realize the acquisition of Oracle log text data.
In this embodiment, the service application data can be acquired according to the parsing result, so as to realize the acquisition of the Oracle log text data.
Based on the above steps S201 to S204, in the embodiment of the present invention, when the parsing thread is restarted, the starting SCN of the last log text data obtained before restarting the parsing thread stored in the distributed database may be read, and the starting SCN is used as the start position of the log text data after restarting the parsing thread to parse the log text data, so as to implement acquisition of Oracle log text data. Through the configuration mode, the method and the device can avoid the situation that the analyzed log text data needs to be repeatedly analyzed when the analysis thread is restarted for some reasons, and also avoid the situation that some log text data are missed after the analysis thread re-enlightens. The method for saving and starting SCN does not need to store log text data, and effectively improves the log text data acquisition efficiency of the Oracle database on the premise of avoiding log text data omission.
In one implementation of the embodiment of the present invention, in addition to the above steps S201 to S204, the present invention may further include the following step S205:
step S205: and storing the starting SCN obtained by analyzing each piece of log text data into a message queue.
In this embodiment, the starting SCN obtained by parsing the parsing thread may be stored in the message queue.
In one embodiment, the message queue may be kafka, or the like.
In one implementation manner of the embodiment of the present invention, the step S204 may further include the following steps S2041 and S2042:
step S2041: and performing service processing on the service data to be processed to obtain service application data.
Step S2042: and storing the business application data into a message queue for consumption by the consumer.
In this embodiment, the service processing may be performed on the service data to be processed to obtain the service application data, and the service application data may be stored in the message queue to be consumed by the consumer.
In one embodiment, the business process may be converting the business data to be processed into business application data.
A specific example of an embodiment of the present invention will be described with reference to fig. 3. Fig. 3 is a flowchart illustrating main steps of a method for collecting log text data of an Oracle database according to an embodiment of the present invention.
As shown in fig. 3, the Oracle LogMiner component generates log text data, and the log text data needs to be parsed, and the parsing process can use a single parsing thread and a batch parsing thread to parse the log text data. The single analysis thread pushes one piece of log text data in each batch; the batch parsing thread pushes 20000 pieces of log text data per batch.
When the parsing thread is started, the starting SCN needs to be acquired from the zookeeper.
And the analysis thread analyzes, processes, verifies and deduplicates data, converts, screens and the like for each piece of log text data, obtains service application data, stores the service application data into kafka, and periodically gets the latest starting SCN stored in the kafka by job and stores the latest starting SCN into a zk (zookeeper) node.
The local cache (cache) generates a unique cache identifier of the log text data according to the ssid, ssn, rowid and the like of the log text data, and realizes data verification deduplication of an analysis result of the log text data through the unique cache identifier. Wherein rowid is a pseudo column used for uniquely marking a row in the table; ssid is a log text identifier; ssn is the log text location identification.
Under the condition of not using the log text data acquisition method of the Oracle database of the embodiment of the invention, when a single parsing thread and a batch parsing thread are restarted due to some reasons, one condition is that the single parsing thread and the batch parsing thread parse the log text data from the beginning, that is, the starting SCN =1, but actually before the restart, the parsing is completed and the business application data corresponding to part of the log text data is stored, for example, the log text data of the starting SCN =3 is parsed and stored, and then the log text data at the positions of the starting SCN =1 and 2 are repeatedly parsed.
Another situation is that the actual parsing position of the parsing thread is recorded, and if the starting SCN =6 of the log text data parsed by the current parsing thread, the parsing thread is restarted at this time, and the parsing thread continues parsing the log text data according to the actual parsing position. In this case, although the parsing thread has already parsed the data location where SCN =6 is started before restarting, since the parsing result needs to be processed, data verification and deduplication, converted, filtered, and stored to kafka after the parsing step in the parsing log text data collection process, there is a high possibility that even though the parsing of the log text data of the data location corresponding to SCN =6 is completed, the service application data corresponding to only the log text data of the data location corresponding to SCN =4 may be successfully stored to kafka. Then continuing parsing from the start SCN =6 after the parsing thread is restarted may result in loss of business application data for the data location corresponding to SCN = 5.
By applying the embodiment of the invention, the condition of losing the service application data can be effectively avoided, and the analysis is not required to be started from the beginning every time after the analysis thread is restarted, so that the analysis efficiency is ensured on the premise of ensuring that the service application data is not lost in the process of acquiring the log text data.
It should be noted that, although the foregoing embodiments describe each step in a specific sequence, those skilled in the art can understand that, in order to achieve the effect of the present invention, different steps do not have to be executed in such a sequence, and they may be executed simultaneously (in parallel) or in other sequences, and these changes are all within the scope of the present invention.
It will be understood by those skilled in the art that all or part of the flow of the method according to the above-described embodiment may be implemented by a computer program, which may be stored in a computer-readable storage medium and used to implement the steps of the above-described embodiments of the method when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device capable of carrying said computer program code, media, usb disk, removable hard disk, magnetic diskette, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunication signals, software distribution media, etc. It should be noted that the computer readable storage medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable storage media that does not include electrical carrier signals and telecommunications signals in accordance with legislation and patent practice.
Furthermore, the invention also provides a control device. In an embodiment of the control device according to the present invention, the control device includes a processor and a storage device, the storage device may be configured to store a program for executing the log breakpoint collection method of the Oracle database of the above-mentioned method embodiment, and the processor may be configured to execute a program in the storage device, the program including but not limited to a program for executing the log breakpoint collection method of the Oracle database of the above-mentioned method embodiment. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed. The control device may be a control device apparatus formed including various electronic apparatuses.
Further, the invention also provides a computer readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for executing the log breakpoint collection method of the Oracle database of the above-described method embodiment, and the program can be loaded and executed by a processor to implement the log breakpoint collection method of the Oracle database. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed. The computer readable storage medium may be a storage device formed by including various electronic devices, and optionally, the computer readable storage medium is a non-transitory computer readable storage medium in the embodiment of the present invention.
Furthermore, the invention also provides a control device. In an embodiment of the control device according to the present invention, the control device comprises a processor and a storage device, the storage device may be configured to store a program for executing the method for collecting log text data of the Oracle database of the above-mentioned method embodiment, and the processor may be configured to execute a program in the storage device, the program including but not limited to a program for executing the method for collecting log text data of the Oracle database of the above-mentioned method embodiment. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed. The control device may be a control device apparatus formed including various electronic apparatuses.
Further, the invention also provides a computer readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium may be configured to store a program for executing the method for collecting log text data of the Oracle database of the above-mentioned method embodiment, and the program may be loaded and executed by a processor to implement the method for collecting log text data of the Oracle database of the above-mentioned method embodiment. For convenience of explanation, only the parts related to the embodiments of the present invention are shown, and details of the specific techniques are not disclosed. The computer readable storage medium may be a storage device formed by including various electronic devices, and optionally, the computer readable storage medium is a non-transitory computer readable storage medium in the embodiment of the present invention.
Further, it should be understood that, since the configuration of each module is only for explaining the functional units of the apparatus of the present invention, the corresponding physical devices of the modules may be the processor itself, or a part of software, a part of hardware, or a part of a combination of software and hardware in the processor. Thus, the number of individual blocks in the figures is merely illustrative.
Those skilled in the art will appreciate that the various modules in the apparatus may be adaptively split or combined. Such splitting or combining of specific modules does not cause the technical solutions to deviate from the principle of the present invention, and therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is apparent to those skilled in the art that the scope of the present invention is not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.
Claims (10)
1. A log breakpoint collection method of an Oracle database is characterized by comprising the following steps:
acquiring a starting SCN of each piece of log text data according to an analysis result of each piece of log text data generated by the Oracle LogMiner by an analysis thread;
and storing the starting SCN of the last piece of currently analyzed log text data into a preset distributed database every preset time.
2. The log breakpoint collection method of an Oracle database according to claim 1,
the method for acquiring the starting SCN of each log text data according to the analysis result of each log text data generated by the analysis thread to Oracle LogMiner comprises the following steps:
analyzing the log text data by applying the analysis thread to obtain an analysis result of the log text data;
acquiring a starting SCN of the log text data according to the analysis result;
and storing the starting SCN of the log text data into a message queue.
3. The Oracle database log breakpoint collection method according to claim 2,
storing the starting SCN of the last log text data which is analyzed currently to a preset distributed database every preset time interval, wherein the method comprises the following steps:
and reading the starting SCN of the last piece of currently analyzed log text data from the message queue at preset time intervals, and storing the SCN into the distributed database.
4. The Oracle database log breakpoint collection method according to claim 1, wherein the method further comprises:
and when the starting SCN of the last piece of the analyzed log text data is not successfully stored, manually setting a starting SCN as the starting SCN of the last piece of the log text data.
5. The Oracle database log breakpoint collection method according to any one of claims 1 to 4,
the storing the starting SCN of the last log text data into a preset distributed database further includes:
and storing the starting SCN of the last log text data into the distributed database and covering the starting SCN stored in the distributed database.
6. A log text data acquisition method of an Oracle database is characterized by comprising the following steps:
when an analysis thread is applied to analyze the log text data generated by the Oracle LogMiner, if the analysis thread is restarted, reading a starting SCN (sequence coding network) which stores the last log text data obtained by analysis before the restart of the analysis thread in a preset distributed database; the SCN for starting the last log text data is obtained according to the log breakpoint collection method of the Oracle database of any one of the claims 1-5;
taking the data position corresponding to the SCN as the initial position of the log text data after the analysis thread is restarted;
analyzing each piece of log text data according to the initial position to obtain an analysis result of each piece of log text data;
and acquiring service application data according to the analysis result so as to realize the acquisition of Oracle log text data.
7. The method for collecting log text data according to claim 6, wherein the analysis result includes the service data to be processed and the starting SCN of each log text data;
the method further comprises the following steps:
and storing the starting SCN obtained by analyzing each piece of log text data into a message queue.
8. The method of collecting journal text data of claim 7,
the acquiring the service application data according to the analysis result includes:
performing service processing on the service data to be processed to obtain service application data;
and storing the service application data into a message queue for consumption by a consumer.
9. A control device comprising a processor and a storage device adapted to store a plurality of program codes, wherein said program codes are adapted to be loaded and run by said processor to perform the method of collecting log breakpoints of an Oracle database according to any one of claims 1 to 5 or the method of collecting log text data of an Oracle database according to any one of claims 6 to 8.
10. A computer readable storage medium having stored therein a plurality of program codes, wherein said program codes are adapted to be loaded and run by a processor to perform the method for collecting log breakpoints of an Oracle database according to any one of claims 1 to 5 or the method for collecting log text data of an Oracle database according to any one of claims 6 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210908654.9A CN115470082A (en) | 2022-07-29 | 2022-07-29 | Log breakpoint and log text data acquisition method of Oracle database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210908654.9A CN115470082A (en) | 2022-07-29 | 2022-07-29 | Log breakpoint and log text data acquisition method of Oracle database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115470082A true CN115470082A (en) | 2022-12-13 |
Family
ID=84368197
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210908654.9A Pending CN115470082A (en) | 2022-07-29 | 2022-07-29 | Log breakpoint and log text data acquisition method of Oracle database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115470082A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116701426A (en) * | 2023-08-07 | 2023-09-05 | 荣耀终端有限公司 | Data processing method, electronic device and storage medium |
-
2022
- 2022-07-29 CN CN202210908654.9A patent/CN115470082A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116701426A (en) * | 2023-08-07 | 2023-09-05 | 荣耀终端有限公司 | Data processing method, electronic device and storage medium |
| CN116701426B (en) * | 2023-08-07 | 2024-04-05 | 荣耀终端有限公司 | Data processing method, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107506451B (en) | Abnormal information monitoring method and device for data interaction | |
| CN110019239B (en) | Storage method and device of reported data, electronic equipment and storage medium | |
| CN107040578B (en) | Data synchronization method, device and system | |
| CN109298924B (en) | Timing task management method, computer readable storage medium and terminal device | |
| CN110109741B (en) | Method and device for managing circular tasks, electronic equipment and storage medium | |
| CN111008246A (en) | Database log synchronization method and device, computer equipment and readable storage medium | |
| CN110737594A (en) | Database standard conformance testing method and device for automatically generating test cases | |
| CN103716384A (en) | Method and device for realizing cloud storage data synchronization in cross-data-center manner | |
| CN115470082A (en) | Log breakpoint and log text data acquisition method of Oracle database | |
| CN111611207B (en) | State data processing method and device and computer equipment | |
| CN113468196B (en) | Method, apparatus, system, server and medium for processing data | |
| CN111309548A (en) | Timeout monitoring method and device and computer readable storage medium | |
| CN112395307A (en) | Statement execution method, statement execution device, server and storage medium | |
| CN112671878B (en) | Block chain information subscription method, device, server and storage medium | |
| CN112328602B (en) | Method, device and equipment for writing data into Kafka | |
| CN104317820B (en) | Statistical method and device for report forms | |
| CN109446272B (en) | Data processing method and device for server | |
| CN113760950B (en) | Index data query method, device, electronic equipment and storage medium | |
| CN115952238A (en) | Data synchronization method and device | |
| CN107422991B (en) | Storage strategy management system | |
| CN112685370B (en) | Log collection method, device, equipment and medium | |
| CN112235332B (en) | Method and device for switching reading and writing of clusters | |
| CN113760600A (en) | Database backup method, database restoration method and related device | |
| CN115470083A (en) | Log text data analysis and collection method of oracle database, server and medium | |
| CN115118773B (en) | Data file pushing method, server and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |