CN111124839A - Distributed log data monitoring method and device - Google Patents
Distributed log data monitoring method and device Download PDFInfo
- Publication number
- CN111124839A CN111124839A CN201911424538.4A CN201911424538A CN111124839A CN 111124839 A CN111124839 A CN 111124839A CN 201911424538 A CN201911424538 A CN 201911424538A CN 111124839 A CN111124839 A CN 111124839A
- Authority
- CN
- China
- Prior art keywords
- log data
- data
- distributed
- standard deviation
- tool
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3006—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3051—Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/22—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/04—Processing captured monitoring data, e.g. for logfile generation
- H04L43/045—Processing captured monitoring data, e.g. for logfile generation for graphical visualisation of monitoring data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a distributed log data monitoring method and a distributed log data monitoring device, wherein the method comprises the following steps: acquiring log data of each distributed node; and monitoring distributed log data according to the log data and pre-generated baseline data, wherein the baseline data is pre-generated according to historical log data. The invention can monitor the distributed log data, saves labor and improves efficiency, thereby accurately finding the log data abnormality in time.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a distributed log data monitoring method and device.
Background
In a modern distributed system, in order to protect the operation safety of the system and discover abnormal fluctuation of data, monitoring of generated log data is necessary.
In the prior art, usually, an administrator regularly checks and analyzes log data under each server, but as the scale of server nodes increases in a geometric index manner, manual analysis of a large amount of log data scattered under each server one by one consumes a large amount of manpower and has extremely low efficiency, and log data abnormity is difficult to find accurately in time.
Disclosure of Invention
The embodiment of the invention provides a distributed log data monitoring method, which is used for monitoring distributed log data, saving labor and improving efficiency so as to timely and accurately find log data abnormity, and comprises the following steps:
acquiring log data of each distributed node;
and monitoring distributed log data according to the log data and pre-generated baseline data, wherein the baseline data is pre-generated according to historical log data.
The embodiment of the invention provides a distributed log data monitoring device, which is used for monitoring distributed log data, saving labor and improving efficiency, thereby timely and accurately finding log data abnormity, and comprises:
the acquisition module is used for acquiring the log data of each distributed node;
and the monitoring module is used for monitoring the distributed log data according to the log data and pre-generated baseline data, wherein the baseline data is pre-generated according to historical log data.
Compared with the scheme that an administrator checks and analyzes the log data under each server regularly in the prior art, the embodiment of the invention acquires the log data of each distributed node; and monitoring distributed log data according to the log data and pre-generated baseline data, wherein the baseline data is pre-generated according to historical log data. According to the embodiment of the invention, log data do not need to be analyzed manually, pre-generated baseline data are generated in advance according to historical log data, and then distributed log data monitoring can be carried out according to the acquired log data of each distributed node and the pre-generated baseline data, so that labor is saved, the efficiency is improved, and log data abnormity can be found accurately in time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
FIG. 1 is a diagram illustrating a distributed log data monitoring method according to an embodiment of the present invention;
FIGS. 2-11 are schematic diagrams of a distributed log data monitoring method according to an embodiment of the invention;
fig. 12 is a structural diagram of a distributed log data monitoring apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
First, terms related to embodiments of the present invention are described:
ELK: abbreviations for three open source software, respectively: elastic search, Logstash, Kibana;
elastic search: the distributed storage and index engine supporting full-text index based on Lucene is mainly responsible for indexing and storing logs, and is convenient for a business party to retrieve and query, and the ElasticSearch has the characteristics that: the method comprises the following steps of distributed type, zero configuration, automatic discovery, automatic index fragmentation, an index copy mechanism, a restful style interface, multiple data sources, automatic search load and the like;
logstash: the tool for collecting, analyzing and filtering the logs supports a large number of data acquisition modes. The general working mode is a c/s architecture, a client end is installed on a host computer needing log collection, and a server end is responsible for filtering, modifying and the like operations of received logs of all nodes and sending the received logs to an elastic search;
kibana: the visualization tool is mainly responsible for inquiring the data of the ElasticSearch and displaying the data to a business party in a visualization mode, such as various pie charts, histograms, area charts and the like;
filebeat: after the client is installed on the server, the Filebeat monitors the log directory or the designated log files, tracks and reads the files (tracks the change of the files and continuously reads the files), and forwards the information to the ElasticSearch or logstack for storage. The Filebeat is lighter than Logstash, and occupies little system resource;
kafka: an open-source distributed message queue, supporting a subscription-publication model, is used as a message bus in a distributed architecture to deliver messages.
In order to monitor distributed log data, save manpower, and improve efficiency, thereby accurately finding log data abnormality in time, an embodiment of the present invention provides a distributed log data monitoring method, as shown in fig. 1, which may include:
102, monitoring distributed log data according to the log data and pre-generated baseline data, wherein the baseline data is pre-generated according to historical log data.
As shown in fig. 1, in the embodiment of the present invention, log data of each distributed node is obtained; and monitoring distributed log data according to the log data and pre-generated baseline data, wherein the baseline data is pre-generated according to historical log data. According to the embodiment of the invention, log data do not need to be analyzed manually, pre-generated baseline data are generated in advance according to historical log data, and then distributed log data monitoring can be carried out according to the acquired log data of each distributed node and the pre-generated baseline data, so that labor is saved, the efficiency is improved, and log data abnormity can be found accurately in time.
In specific implementation, the log data of each distributed node is obtained.
In an embodiment, the obtaining log data of each distributed node includes: collecting log data of each distributed node; preprocessing the log data, wherein the preprocessing comprises: filtering, converting and structuring; and acquiring the processed log data.
In this embodiment, collecting log data of each distributed node includes: and collecting log data of each distributed node by using a Filebeat tool. Compared with the existing scheme adopting logstash, the log data acquisition of the embodiment of the invention uses a Filebeat tool, so that the Filebeat is lighter than logstach, occupies fewer resources and has smaller influence on service application than logstash.
In this embodiment, acquiring the log data of each distributed node further includes: after log data of each distributed node are collected by a Filebeat tool, storing the log data into a message queue; preprocessing the log data, including: and pulling the log data in the message queue, and preprocessing the log data.
In this embodiment, after the filebed tool is used to collect the log data of each distributed node, the storing of the log data into the message queue includes: after the Filebeat tool is used for collecting log data of each distributed node, storing the log data into a message queue of a Kafka tool; preprocessing the log data, including: utilizing a Logstash tool to pull log data in a message queue of the Kafka tool, preprocessing the log data, and storing a processing result into an ElasticSearch tool; the processed log data is obtained from the ElasticSearch tool.
It should be noted that, storing the log data into the message queue of the Kafka tool may function as a buffer pool, which may alleviate the peak processing pressure of the log data, and may reduce the possibility of message loss.
Specifically, Filebeat needs to be deployed and started on each distributed node application server which needs to collect log data, input is defined, Path points to a log file Path output by an application, wildcard is supported, output is defined as Kafka, version of Kafka needs to be configured, otherwise, output cannot be output to Kafka, and then a Filebeat process is started. And pulling the log data in the message queue of the Kafka tool by using a Logstash tool, and preprocessing the log data. And the hook is a filtering plug-in of logstack, supports parsing a text log line according to a mode, and is split into fields. The ElasticSearch provides 120 built-in grok built-in types, and if the built-in pattern cannot be met, the pattern needs to be customized, and the mode of customizing the pattern file is as follows: 1. creating a file postfix named patterns, wherein the file name can be set according to actual needs, and in the file, writing a required pattern as a pattern name, a space and then a regular expression of the pattern; 2. the logstack directory is then notified that it is a custom mode using the patterns _ dir setting in this plug-in.
In this embodiment, the distributed log data monitoring method further includes: and acquiring the processed log data from an ElasticSearch tool, and performing visual retrieval processing on the log data.
In this embodiment, after acquiring the processed log data from the ElasticSearch tool, performing a visual retrieval process on the log data includes: and after the processed log data are acquired from the ElasticSearch tool, performing visual retrieval processing on the log data by using a Kibana tool. It should be noted that, using the Kibana tool, a report or a dashboard can be customized to the log data.
The inventor finds that manual analysis of a large amount of log data scattered under each server one by one consumes a large amount of manpower and has extremely low efficiency, and the log data abnormity is difficult to find accurately in time. Therefore, the method is based on the open source ELK log framework, and combines the requirements of the work order reverse checking scene and the operation monitoring, so that the log data acquisition suitable for the distributed architecture is realized.
In specific implementation, distributed log data monitoring is performed according to the log data and pre-generated baseline data, wherein the baseline data is pre-generated according to historical log data.
In an embodiment, the baseline data includes: standard deviation range data of historical log data; according to the log data and pre-generated baseline data, distributed log data monitoring is carried out, wherein the baseline data is pre-generated according to historical log data and comprises the following steps: and monitoring the distributed log data according to the log data and pre-generated standard deviation range data, wherein the standard deviation range data is pre-generated according to historical log data.
In this embodiment, the standard deviation range data is generated in advance according to the history log data as follows: determining the mean value and the standard deviation of the historical log data according to the historical log data; and generating standard deviation range data according to the mean value and the standard deviation of the historical log data.
In an embodiment, the baseline data may further include: time point data of historical log data, time interval data, interface name, system name, maximum value, minimum value, mean value, standard deviation.
In an embodiment, the generation of baseline data includes two modes: a manual generation mode and a timed automatic generation mode, wherein the manual generation mode is to specify a historical date and generate baseline data of the historical date, the timed automatic generation mode is to automatically generate a baseline for D-n days according to the configuration, and default baseline data for D-1 and D-7 days are automatically generated every morning.
In an embodiment, the standard deviation range data is generated according to the mean and standard deviation of the historical log data as follows: the mean of the historical log data plus twice the standard deviation is the upper bound of the standard deviation range and the mean of the historical log data minus twice the standard deviation is the lower bound of the standard deviation range.
In the embodiment, the log data is compared with the pre-generated standard deviation range data, an alarm is given when the log data exceeds the upper limit of the standard deviation range or exceeds the lower limit of the standard deviation range, the log data is stored in an ElasticSearch, and a maintenance department is shown and prompted through kibana, so that distributed log data monitoring is realized.
In the embodiment, taking a system time-consuming alarm as an example, the log data is counted by using the same aggregation latitude, and if the lower bound is a negative number, the lower bound needs to be replaced by a time-consuming minimum value in the baseline data.
In an embodiment, visualization is performed in kibana, and data that currently takes more than the upper bound of the standard deviation is written into the index of the abnormal alarm, which is shown in kibana.
A specific embodiment is given below to describe a specific application of the distributed log data monitoring method in the embodiment of the present invention. In this specific embodiment, Filebeat needs to be deployed and started on each application server that needs to collect logs, and defines input, as shown in fig. 2, and defines output as Kafka, as shown in fig. 3, a filebot process is started,/filebot-c/ipps/install-p 903/elastic/filebot-ipps-marchant.yml is created, Kafka theme is created, upload file/Kafka/create _ topocs.sh is uploaded to any broker of Kafka, create _ topocs.sh is customized and modified, then log is configured, and logs are pulled from Kafka, as shown in fig. 4, a manner of a pattern file is defined: 1. creating a file postfix named patterns, wherein the file name can be set according to actual needs, writing a required pattern in the file as a pattern name, a space and then a regular expression of the pattern, as shown in fig. 5; 2. the logstack directory is then notified that it is a custom schema using the patterns _ dir setting in this plug-in, a complete example log as shown in figure 6. And applying python on the basis of an elk collection log platform to realize a monitoring function. The configuration of the pre-generated baseline data is shown in fig. 7, the time consumption baseline data of the background system is shown in fig. 8, the configuration of the log data is shown in fig. 9, the transaction alarm with the current time consumption exceeding the baseline threshold value is written into the index of the abnormal alarm and is shown in kibana, as shown in fig. 10, the current time consumption and the baseline data are compared in real time, and the obtained result is shown in fig. 11.
Based on the same inventive concept, the embodiment of the present invention further provides a distributed log data monitoring apparatus, as described in the following embodiments. Because the principles of solving the problems are similar to the distributed log data monitoring method, the implementation of the device can refer to the implementation of the method, and repeated details are not repeated.
Fig. 12 is a structural diagram of a distributed log data monitoring apparatus in an embodiment of the present invention, and as shown in fig. 12, the apparatus includes:
an obtaining module 1201, configured to obtain log data of each distributed node;
a monitoring module 1202, configured to perform distributed log data monitoring according to the log data and pre-generated baseline data, where the baseline data is pre-generated according to historical log data.
In one embodiment, the obtaining module 1201 is further configured to:
collecting log data of each distributed node;
preprocessing the log data, wherein the preprocessing comprises: filtering, converting and structuring;
and acquiring the processed log data.
In one embodiment, the obtaining module 1201 is further configured to: and collecting log data of each distributed node by using a Filebeat tool.
In one embodiment, the baseline data includes: standard deviation range data of historical log data;
the monitoring module 1202 is further configured to: and monitoring the distributed log data according to the log data and pre-generated standard deviation range data, wherein the standard deviation range data is pre-generated according to historical log data.
In one embodiment, the standard deviation range data is pre-generated from historical log data as follows:
determining the mean value and the standard deviation of the historical log data according to the historical log data;
and generating standard deviation range data according to the mean value and the standard deviation of the historical log data.
In summary, in the embodiments of the present invention, log data of each distributed node is obtained; and monitoring distributed log data according to the log data and pre-generated baseline data, wherein the baseline data is pre-generated according to historical log data. According to the embodiment of the invention, log data do not need to be analyzed manually, pre-generated baseline data are generated in advance according to historical log data, and then distributed log data monitoring can be carried out according to the acquired log data of each distributed node and the pre-generated baseline data, so that labor is saved, the efficiency is improved, and log data abnormity can be found accurately in time.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (16)
1. A distributed log data monitoring method is characterized by comprising the following steps:
acquiring log data of each distributed node;
and monitoring distributed log data according to the log data and pre-generated baseline data, wherein the baseline data is pre-generated according to historical log data.
2. The distributed log data monitoring method of claim 1, wherein obtaining log data for each distributed node comprises:
collecting log data of each distributed node;
preprocessing the log data, wherein the preprocessing comprises: filtering, converting and structuring;
and acquiring the processed log data.
3. The distributed log data monitoring method of claim 2, wherein collecting log data for each distributed node comprises: and collecting log data of each distributed node by using a Filebeat tool.
4. The distributed log data monitoring method of claim 3, wherein obtaining log data for each distributed node further comprises: after log data of each distributed node are collected by a Filebeat tool, storing the log data into a message queue;
preprocessing the log data, including: and pulling the log data in the message queue, and preprocessing the log data.
5. The distributed log data monitoring method of claim 4, wherein storing the log data into a message queue after collecting the log data of each distributed node by using a filebed tool comprises: after the Filebeat tool is used for collecting log data of each distributed node, storing the log data into a message queue of a Kafka tool;
preprocessing the log data, including: utilizing a Logstash tool to pull log data in a message queue of the Kafka tool, preprocessing the log data, and storing a processing result into an ElasticSearch tool;
the processed log data is obtained from the ElasticSearch tool.
6. The distributed log data monitoring method of claim 5, further comprising: and acquiring the processed log data from an ElasticSearch tool, and performing visual retrieval processing on the log data.
7. The distributed log data monitoring method of claim 6, wherein after the processed log data is obtained from an elastic search tool, performing a visual retrieval process on the log data, comprising: and after the processed log data are acquired from the ElasticSearch tool, performing visual retrieval processing on the log data by using a Kibana tool.
8. The distributed log data monitoring method of claim 1, wherein the baseline data comprises: standard deviation range data of historical log data;
according to the log data and pre-generated baseline data, distributed log data monitoring is carried out, wherein the baseline data is pre-generated according to historical log data and comprises the following steps: and monitoring the distributed log data according to the log data and pre-generated standard deviation range data, wherein the standard deviation range data is pre-generated according to historical log data.
9. The distributed log data monitoring method of claim 8, wherein the standard deviation range data is pre-generated from historical log data as follows:
determining the mean value and the standard deviation of the historical log data according to the historical log data;
and generating standard deviation range data according to the mean value and the standard deviation of the historical log data.
10. A distributed log data monitoring apparatus, comprising:
the acquisition module is used for acquiring the log data of each distributed node;
and the monitoring module is used for monitoring the distributed log data according to the log data and pre-generated baseline data, wherein the baseline data is pre-generated according to historical log data.
11. The distributed log data monitoring apparatus of claim 10, wherein the obtaining module is further to:
collecting log data of each distributed node;
preprocessing the log data, wherein the preprocessing comprises: filtering, converting and structuring;
and acquiring the processed log data.
12. The distributed log data monitoring apparatus of claim 11, wherein the obtaining module is further to: and collecting log data of each distributed node by using a Filebeat tool.
13. The distributed log data monitoring apparatus of claim 10, wherein the baseline data comprises: standard deviation range data of historical log data;
the monitoring module is further configured to: and monitoring the distributed log data according to the log data and pre-generated standard deviation range data, wherein the standard deviation range data is pre-generated according to historical log data.
14. The distributed log data monitoring apparatus of claim 13, wherein the standard deviation range data is pre-generated from historical log data as follows:
determining the mean value and the standard deviation of the historical log data according to the historical log data;
and generating standard deviation range data according to the mean value and the standard deviation of the historical log data.
15. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of claims 1 to 9 when executing the computer program.
16. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911424538.4A CN111124839A (en) | 2019-12-31 | 2019-12-31 | Distributed log data monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911424538.4A CN111124839A (en) | 2019-12-31 | 2019-12-31 | Distributed log data monitoring method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111124839A true CN111124839A (en) | 2020-05-08 |
Family
ID=70507126
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911424538.4A Pending CN111124839A (en) | 2019-12-31 | 2019-12-31 | Distributed log data monitoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111124839A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111726252A (en) * | 2020-06-17 | 2020-09-29 | 浪潮集团有限公司 | Abnormal data reporting method and tool for industrial equipment |
| CN112256530A (en) * | 2020-10-27 | 2021-01-22 | 广州助蜂网络科技有限公司 | Log service method capable of tracking link and system thereof |
| CN112579326A (en) * | 2020-12-29 | 2021-03-30 | 北京五八信息技术有限公司 | Offline data processing method and device, electronic equipment and computer readable medium |
| CN117033334A (en) * | 2023-10-08 | 2023-11-10 | 吉林省高速公路集团有限公司 | Expressway toll lane log acquisition processing method and system |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017071134A1 (en) * | 2015-10-28 | 2017-05-04 | 北京汇商融通信息技术有限公司 | Distributed tracking system |
| CN108874642A (en) * | 2018-05-25 | 2018-11-23 | 平安科技（深圳）有限公司 | SQL method for monitoring performance, device, computer equipment and storage medium |
| CN109167672A (en) * | 2018-07-13 | 2019-01-08 | 腾讯科技（深圳）有限公司 | One kind returning source location of mistake method, apparatus, storage medium and system |
| CN110191000A (en) * | 2019-06-10 | 2019-08-30 | 中汇信息技术（上海）有限公司 | A kind of data processing method, message tracing monitoring method and distributed system |
| CN110309030A (en) * | 2019-07-05 | 2019-10-08 | 亿玛创新网络(天津)有限公司 | Log analysis monitoring system and method based on ELK and Zabbix |
-
2019
- 2019-12-31 CN CN201911424538.4A patent/CN111124839A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017071134A1 (en) * | 2015-10-28 | 2017-05-04 | 北京汇商融通信息技术有限公司 | Distributed tracking system |
| CN108874642A (en) * | 2018-05-25 | 2018-11-23 | 平安科技（深圳）有限公司 | SQL method for monitoring performance, device, computer equipment and storage medium |
| CN109167672A (en) * | 2018-07-13 | 2019-01-08 | 腾讯科技（深圳）有限公司 | One kind returning source location of mistake method, apparatus, storage medium and system |
| CN110191000A (en) * | 2019-06-10 | 2019-08-30 | 中汇信息技术（上海）有限公司 | A kind of data processing method, message tracing monitoring method and distributed system |
| CN110309030A (en) * | 2019-07-05 | 2019-10-08 | 亿玛创新网络(天津)有限公司 | Log analysis monitoring system and method based on ELK and Zabbix |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111726252A (en) * | 2020-06-17 | 2020-09-29 | 浪潮集团有限公司 | Abnormal data reporting method and tool for industrial equipment |
| CN112256530A (en) * | 2020-10-27 | 2021-01-22 | 广州助蜂网络科技有限公司 | Log service method capable of tracking link and system thereof |
| CN112579326A (en) * | 2020-12-29 | 2021-03-30 | 北京五八信息技术有限公司 | Offline data processing method and device, electronic equipment and computer readable medium |
| CN117033334A (en) * | 2023-10-08 | 2023-11-10 | 吉林省高速公路集团有限公司 | Expressway toll lane log acquisition processing method and system |
| CN117033334B (en) * | 2023-10-08 | 2023-12-22 | 吉林省高速公路集团有限公司 | Expressway toll lane log acquisition processing method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111124839A (en) | Distributed log data monitoring method and device | |
| US10860596B2 (en) | Employing external data stores to service data requests | |
| CN113360554B (en) | Method and equipment for extracting, converting and loading ETL (extract transform load) data | |
| CN108985981B (en) | Data processing system and method | |
| CN107784026B (en) | ETL data processing method and device | |
| CN108521339B (en) | Feedback type node fault processing method and system based on cluster log | |
| CN104951512A (en) | Public sentiment data collection method and system based on Internet | |
| US11416278B2 (en) | Presenting hypervisor data for a virtual machine with associated operating system data | |
| CN104537015A (en) | Log analysis computer implementation method, computer and system | |
| CN108829505A (en) | A kind of distributed scheduling system and method | |
| CN110851234A (en) | Log processing method and device based on docker container | |
| CN112529528A (en) | Workflow monitoring and warning method, device and system based on big data flow calculation | |
| CN111106965A (en) | Intelligent log analysis method, tool, equipment and medium for complex system | |
| CN111563131A (en) | Database entity relation generation method and device, computer equipment and storage medium | |
| CN109635022B (en) | Visual elastic search data acquisition method and device | |
| CN104991836A (en) | Database recovery method and apparatus and database server | |
| CN113094154A (en) | Big data processing method and system based on Aliyun | |
| CN112650739A (en) | Data storage processing method and device for coal mine data middling station | |
| CN111708576A (en) | Cross-operating-system equipment network card adapting and managing method, device and system | |
| CN110019045B (en) | Log floor method and device | |
| CN109033196A (en) | A kind of distributed data scheduling system and method | |
| CN115794064A (en) | Configuration method and device of task processing flow, electronic equipment and storage medium | |
| CN115757642A (en) | Data synchronization method and device based on filing log file | |
| KR20130068633A (en) | Apparatus and method for visualizing data | |
| CN113672307A (en) | Data processing method, electronic device, and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |