CN111274095B - Log data processing method, device, equipment and computer readable storage medium - Google Patents
Log data processing method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111274095B CN111274095B CN202010115818.3A CN202010115818A CN111274095B CN 111274095 B CN111274095 B CN 111274095B CN 202010115818 A CN202010115818 A CN 202010115818A CN 111274095 B CN111274095 B CN 111274095B
- Authority
- CN
- China
- Prior art keywords
- data
- log data
- time
- aggregated
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
- G06F11/3082—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting the data filtering being achieved by aggregating or compressing the monitored data
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a log data processing method, a device, equipment and a computer readable storage medium, wherein the method comprises the following steps: collecting service log data of a plurality of systems, analyzing each service log data according to the log data types of the plurality of systems, and generating standard log data; aggregating all standard log data to generate aggregated data, and performing time zone standardization processing and data completion processing on the aggregated data to obtain aggregated time sequence data; and counting abnormal keywords in the aggregated time sequence data, and early warning the abnormal keywords according to a preset monitoring type. The invention analyzes the service log data of each type into the standard log data to be aggregated, avoids the deployment of a plurality of sets of system frameworks for processing aiming at the log data of different types, and can embody the abnormal time by the time sequence of the abnormal keyword in the aggregated time sequence data, thereby simplifying the processing process and saving the cost for the operation and maintenance processing of a plurality of sets of system frameworks.
Description
Technical Field
The present invention relates to the field of financial technology (Fintech) technologies, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for processing log data.
Background
With the continuous development of financial technology (Fintech), especially internet technology and finance, more and more technologies (such as big data, cloud storage, distributed systems and the like) are applied to the financial field, but the financial industry also puts higher requirements on various technologies, such as the requirement for uniformly processing various log data stored on various devices in the distributed systems.
Currently, in order to support uniform processing of different types of log data on devices, a system architecture is generally built by means of an open source tool such as Filebeat (log collection), logstation (log collection processing), and elastic search (log storage).
One of the problems brought by the system architecture is that it is difficult to support multi-type log data source input under a heterogeneous system, and if heterogeneous log data such as common logs, messages, remote services and the like need to be processed in a distributed system, multiple sets of system architectures need to be deployed to process different types of log data, which causes increased system complexity and increased operation and maintenance cost.
Disclosure of Invention
The invention mainly aims to provide a log data processing method, a log data processing device, log data processing equipment and a computer readable storage medium, and aims to solve the technical problems that in the prior art, a plurality of sets of systems are used for processing different types of log data, the systems are complex, and the operation and maintenance cost is high.
In order to achieve the above object, the present invention provides a log data processing method, including the steps of:
collecting service log data of a plurality of systems, and analyzing each service log data according to the log data types of the plurality of systems to generate standard log data;
aggregating all the standard log data to generate aggregated data, and performing time zone standardization processing and data completion processing on the aggregated data to obtain aggregated time sequence data;
and counting abnormal keywords in the aggregation time sequence data, and early warning the abnormal keywords according to a preset monitoring type.
Optionally, the step of performing early warning on the abnormal keyword according to a preset monitoring type includes:
if the preset monitoring type is keyword monitoring, generating the abnormal keywords as early warning information for early warning;
if the preset monitoring type is frequency monitoring, judging whether the number of the abnormal keywords counted in a preset time interval is larger than a preset threshold value or not;
and if the number of the abnormal keywords is larger than a preset threshold value, generating the abnormal keywords as early warning information to carry out early warning.
Optionally, the step of performing time zone standardization processing and data completion processing on the aggregated data to obtain aggregated time series data includes:
acquiring generation time of each item of service log data, and dividing the aggregated data into first data and second data according to each generation time, wherein the generation time corresponding to each first data is a null value, and the generation time corresponding to each second data is a non-null value;
acquiring aggregation time for aggregating each item of standard log data, and adjusting the aggregation time corresponding to each item of second data according to the generation time corresponding to each item of second data so as to perform time zone standardization processing on the aggregated data;
according to the current time, completing the generation time corresponding to each first data to perform data completion processing on the aggregated data, and adjusting the aggregation time corresponding to each first data to the completed generation time;
generating the adjusted each of the first data and each of the second data as the aggregated time-series data.
Optionally, the step of generating the adjusted respective first data and respective second data as the aggregated time-series data is followed by:
storing each first data and each second data which generate the aggregation time sequence data according to the adjusted time sequence;
when a time sequence retrieval request is received, reading a time period carried in the time sequence retrieval request, and retrieving target data in the aggregated time sequence data according to the time period;
and outputting the target data.
Optionally, the step of analyzing each item of service log data according to the log data types of the multiple systems to generate standard log data includes:
determining analysis rules corresponding to the log data types of a plurality of the systems, and establishing a mapping relation between the analysis rules corresponding to the same system and the service log data;
executing the following steps for each mapping relation:
and analyzing the service log data in the mapping relation based on a preset standard format according to the analysis rule in the mapping relation to generate the standard log data corresponding to the mapping relation.
Optionally, the analyzing the service log data in the mapping relationship based on a preset standard format according to the analysis rule in the mapping relationship, and the step of generating the standard log data corresponding to the mapping relationship includes:
analyzing the service log data in the mapping relation according to the analysis rule in the mapping relation to generate log sub-data;
screening each log subdata and determining target log subdata corresponding to the preset standard format;
and arranging the sub data of the target logs according to the preset standard format to generate the standard log data corresponding to the mapping relation.
Optionally, the step of collecting service log data of a plurality of systems includes:
when the detection reaches a preset acquisition period, judging whether a newly accessed system to be acquired exists or not;
if the newly accessed system to be acquired exists, updating the systems and acquiring the updated service log data of the systems;
and if the newly accessed system to be acquired does not exist, executing the step of acquiring the service log data of the plurality of systems.
Further, to achieve the above object, the present invention also provides a log data processing apparatus including:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring service log data of a plurality of systems, analyzing each service log data according to log data types of the systems and generating standard log data;
the aggregation module is used for aggregating the standard log data to generate aggregated data;
and the processing module is used for processing the aggregated data according to a preset processing flow.
Further, to achieve the above object, the present invention also provides a log data processing device, which includes a memory, a processor, and a log data processing program stored on the memory and operable on the processor, and when executed by the processor, the log data processing program implements the steps of the log data processing method as described above.
Further, to achieve the above object, the present invention also provides a computer readable storage medium having stored thereon a log data processing program, which when executed by a processor, implements the steps of the log data processing method as described above.
The log data processing method of the invention generates standard log data by analyzing the collected service log data of each system according to the log data type of each system; then, aggregating all standard log data to obtain aggregated data with a standard format, and performing time zone standardization processing and data completion processing on the aggregated data to obtain aggregated time sequence data with a generation time sequence; then, abnormal keywords in the aggregation time sequence data are counted, and early warning is carried out on the abnormal keywords according to a preset monitoring type; therefore, the monitoring processing of multiple sets of system frameworks is avoided being deployed aiming at different types of log data of each system, the abnormal time can be reflected by the time sequence of the abnormal keywords in the aggregated time sequence data, the monitoring process is simplified, and the cost for the operation and maintenance processing of the multiple sets of system frameworks is saved.
Drawings
FIG. 1 is a schematic structural diagram of a hardware operating environment of a device according to an embodiment of the log data processing device of the present invention;
FIG. 2 is a flowchart illustrating a log data processing method according to a first embodiment of the present invention;
FIG. 3 is a functional block diagram of a log data processing apparatus according to a preferred embodiment of the invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides a log data processing device, and referring to fig. 1, fig. 1 is a schematic structural diagram of a device hardware operating environment according to an embodiment of the log data processing device of the invention.
As shown in fig. 1, the log data processing apparatus may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to implement connection communication among these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory such as a disk memory. The memory 1005 may alternatively be a memory device separate from the processor 1001 described above.
Those skilled in the art will appreciate that the hardware configuration of the log data processing apparatus shown in fig. 1 does not constitute a limitation of the log data processing apparatus, and may include more or less components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, the memory 1005, which is a kind of computer-readable storage medium, may include therein an operating system, a network communication module, a user interface module, and a log data processing program. The operating system is a program for managing and controlling log data processing equipment and software resources and supports the running of a network communication module, a user interface module, a log data processing program and other programs or software; the network communication module is used to manage and control the network interface 1004; the user interface module is used to manage and control the user interface 1003.
In the hardware structure of the log data processing device shown in fig. 1, the network interface 1004 is mainly used for connecting to a background server and performing data communication with the background server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; the processor 1001 may call the log data handler stored in the memory 1005 and perform the following operations:
acquiring service log data of a plurality of systems, analyzing each service log data according to log data types of the plurality of systems, and generating standard log data;
aggregating all the standard log data to generate aggregated data, and performing time zone standardization processing and data completion processing on the aggregated data to obtain aggregated time sequence data;
and counting abnormal keywords in the aggregation time sequence data, and early warning the abnormal keywords according to a preset monitoring type.
Further, the step of performing early warning on the abnormal keyword according to a preset monitoring type includes:
if the preset monitoring type is keyword monitoring, generating the abnormal keywords as early warning information for early warning;
if the preset monitoring type is frequency monitoring, judging whether the number of the abnormal keywords counted in a preset time interval is larger than a preset threshold value or not;
and if the number of the abnormal keywords is larger than a preset threshold value, generating the abnormal keywords as early warning information to carry out early warning.
Further, the step of performing time zone standardization processing and data completion processing on the aggregation data to obtain aggregation time series data includes:
acquiring generation time of each item of service log data, and dividing the aggregated data into first data and second data according to each generation time, wherein the generation time corresponding to each first data is a null value, and the generation time corresponding to each second data is a non-null value;
acquiring aggregation time for aggregating each item of standard log data, and adjusting the aggregation time corresponding to each item of second data according to the generation time corresponding to each item of second data, so as to perform time zone standardization processing on the aggregated data;
according to the current time, completing the generation time corresponding to each first data so as to perform data completion processing on the aggregated data, and adjusting the aggregation time corresponding to each first data to the completed generation time;
generating the adjusted each of the first data and each of the second data as the aggregated time-series data.
Further, after the step of generating the adjusted first data and the second data as the aggregation time-series data, the processor 1001 may call a log data processing program stored in the memory 1005, and perform the following operations:
storing each first data and each second data which generate the aggregation time sequence data according to the adjusted time sequence;
when a time sequence retrieval request is received, reading a time period carried in the time sequence retrieval request, and retrieving target data in the aggregated time sequence data according to the time period;
and outputting the target data.
Further, the step of analyzing each item of service log data according to the log data types of the plurality of systems to generate standard log data includes:
determining analysis rules corresponding to the log data types of a plurality of the systems, and establishing a mapping relation between the analysis rules corresponding to the same system and the service log data;
executing the following steps for each mapping relation:
and analyzing the service log data in the mapping relation based on a preset standard format according to the analysis rule in the mapping relation to generate the standard log data corresponding to the mapping relation.
Further, the step of analyzing the service log data in the mapping relation based on a preset standard format according to the analysis rule in the mapping relation, and generating the standard log data corresponding to the mapping relation includes:
analyzing the service log data in the mapping relation according to the analysis rule in the mapping relation to generate log sub-data;
screening each log subdata, and determining target log subdata corresponding to the preset standard format;
and arranging the sub data of the target logs according to the preset standard format to generate the standard log data corresponding to the mapping relation.
Further, before the step of collecting the service log data of multiple systems, the processor 1001 may call a log data processing program stored in the memory 1005, and perform the following operations:
when the detection reaches a preset acquisition period, judging whether a newly accessed system to be acquired exists or not;
if the newly accessed system to be acquired exists, updating the systems and acquiring the updated service log data of the systems;
and if the newly accessed system to be acquired does not exist, executing the step of acquiring the service log data of the plurality of systems.
The specific implementation of the log data processing device of the present invention is basically the same as the following embodiments of the log data processing method, and is not described herein again.
The invention also provides a log data processing method.
Referring to fig. 2, fig. 2 is a schematic flowchart of a log data processing method according to a first embodiment of the present invention.
It should be noted that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be performed in an order different from the order shown or described herein. Specifically, the log data processing method in this embodiment includes:
step S10, collecting service log data of a plurality of systems, analyzing each service log data according to log data types of the plurality of systems, and generating standard log data.
The log data processing method in the embodiment is applied to a log aggregation processing system, so that the log aggregation processing system can perform standardized post-aggregation processing on various types of service log data from various systems; the log aggregation processing system is a system which is generated in advance and is preferably a DAG (Database Availability Group) type system. For convenience of description, the log aggregation processing system is hereinafter referred to as an aggregation system, and the aggregation system is communicatively connected to a plurality of other systems of different types in the distributed architecture to collect the service log data generated by the other systems in the distributed architecture for aggregation processing.
Understandably, in consideration of the resource limitation of the aggregation system, the aggregation system usually does not collect the service log data generated by other systems in real time, but preset collection periods are preset. And when the preset acquisition period is reached, acquiring the service log data of the accessed multiple systems, wherein the acquired service log data is only the data generated in the preset acquisition period, so that the repeated acquisition processing of the previously processed service log data is avoided.
In addition, considering the characteristic that the system accessed by the aggregation system changes at different time periods, more systems may need to be accessed into the aggregation system for processing the service log data, so that before the service log data of each system is collected each time, whether a new system is accessed is detected, and omission of the new accessed system is avoided. Specifically, the step of collecting the service log data of a plurality of systems comprises:
step a, when the detection reaches a preset acquisition period, judging whether a newly accessed system to be acquired exists;
b, if the newly accessed system to be acquired exists, updating the systems and acquiring the updated service log data of the systems;
and c, if the newly accessed system to be acquired does not exist, executing the step of acquiring the service log data of a plurality of systems.
Further, when the detection reaches the preset acquisition period, whether a newly accessed system to be acquired exists is judged, and the system to be acquired is a new system which is accessed to the aggregation system in the preset acquisition period and needs to acquire data. All systems accessed in the aggregation system are distinguished through system codes, and a historical system code aimed by the last acquisition operation and a current system code accessed currently are read before acquisition. And comparing the two groups of system codes, judging whether codes which are not contained in the historical system codes exist in the current system codes, and judging that a newly accessed system to be acquired exists if the codes which are not contained in the historical system codes exist.
Furthermore, the system to be acquired is added to the plurality of systems which are accessed originally, the plurality of systems which are accessed originally are updated, and the service log data is acquired on the basis of the plurality of updated systems, that is, the service log data generated by the plurality of systems which are added with the system to be acquired in the preset acquisition period are acquired. If the current system code is completely consistent with the historical system code, it is indicated that no new system accessed to the aggregation system exists in the preset acquisition period, that is, no system to be acquired exists, and the service log data generated by each of the plurality of systems accessed originally in the preset acquisition period are acquired.
It should be noted that, in addition to adding a new access system, the change characteristics of the systems accessed by the aggregation system at different time periods may also be the case that the original access system is no longer accessed. In this case, on one hand, it is determined whether there is a case that the original access system is no longer accessed by determining whether there is a code that is not present in the current system code in the historical system codes. If codes which are not contained in the current system codes exist in the historical system codes, the historical system codes indicate that some originally accessed systems are not accessed into the aggregation system any longer, and the aggregation system does not have the authority of collecting service log data generated by the system. Before the collection operation, the system code of the system is deleted from the aggregation system, so that the collection processing of the generated service log data is avoided.
Understandably, the types of the service log data generated by different systems are different, and may be any one of a json type, a log file type, a web page type, and the like. After various types of service log data generated by each system are collected, one piece of service log data containing various information such as generation time, service content, service account and the like is used as one piece of service log data, and each piece of service log data carries an identifier representing a source system of the service log data. And then, analyzing all the service log data according to the log data types of all the systems to generate corresponding all the standard log data. The method comprises the steps of analyzing json type service log data, analyzing a text for the service log data of a log file type, and analyzing XML (extensive Makeup language) for the service log data of a webpage type.
It should be noted that the log data type of the system can be determined by the system type identifier, after the aggregation system collects a certain service log data of a certain system, the system type identifier of the system is searched, and the log data type of the system is determined according to the system type identifier obtained by searching; and then analyzing the service log data to generate standard log data in accordance with a standard format. The standard format is preset according to requirements, for example, the log data is set to include generation time, service operation content, and a time format of the generation time, a format of the content, and the like.
Step S20, aggregating all the standard log data to generate aggregated data, and performing time zone standardization processing and data completion processing on the aggregated data to obtain aggregated time sequence data;
further, after the service log data of the accessed system is obtained and analyzed to obtain various standard log data, the various standard log data are aggregated to obtain aggregated data. The aggregation can be distinguished and aggregated according to the system from which each standard log data comes, so as to reflect the operation condition of each system in the distributed architecture; aggregation can also be performed according to the time sequence of the generation time of each service log data so as to reflect the overall operation condition of the distributed architecture; the setting can be specifically set according to the requirements.
Understandably, the time for generating each service log data by each system is different, and the aggregation time for aggregating each standard log data has the same, so as to represent the generation time of aggregated data after aggregation, time zone standardization needs to be performed on the aggregated data, that is, the aggregation time of each service log data in the aggregated data is modified into the respective generation time, so as to reflect the actual generation time of each service log data. Meanwhile, the method considers that some service log data may not carry time due to some reasons in the generation process, namely, the generation time is not generated; and for the data, data supplementing processing is required, namely the generation time of the data is supplemented, and the aggregation time of the data is modified by using the supplemented generation time, so that aggregation time sequence data carrying the respective generation time of the characteristics is obtained.
And S30, counting abnormal keywords in the aggregation time sequence data, and early warning the abnormal keywords according to a preset monitoring type.
Furthermore, the aggregation system is preset with a preset processing flow for uniformly processing various types of service log data, and after the aggregated data is obtained through aggregation, the preset processing flow is called so as to process the aggregated data through the preset processing flow. The preset processing flow is set according to the processing requirement of the accessed system, if the requirement is monitored, the monitoring flow is set, and if the requirement is filtered, the filtering flow is set. In view of the differences in processing requirements of the systems, a modular processing flow may be provided in order to accommodate the unique processing requirements of the systems. And setting modules for realizing various processing requirements, and configuring respective modules for processing by different systems according to the requirements. If the processing requirements of the system m1 are A1 and A2, the modules A1 and A2 are configured into a preset processing flow, so as to process the service log data from the system m1 in the aggregated data. The processing requirements of each system are configured in the preset processing flow in a module mode, so that the compatibility of different processing requirements among a plurality of systems is realized, and the unique processing requirements among the systems are favorably met while the aggregated data is uniformly processed.
In order to ensure the normal operation of each system, the implementation is at least provided with an early warning monitoring process for monitoring through aggregating time sequence data. Specifically, a plurality of abnormal keywords representing abnormal system operation are preset, the aggregation time sequence data are identified, whether the abnormal keywords are carried or not is judged, and if the abnormal keywords are carried, the number of the abnormal keywords is counted. And presetting different preset monitoring types, wherein the preset monitoring types at least comprise keyword monitoring for monitoring according to the keyword types and frequency monitoring for monitoring according to the number of the keywords. After the abnormal keywords in the aggregation time sequence data are obtained through statistics, early warning is conducted on the abnormal keywords according to the preset monitoring type so as to prompt that the system is abnormal, and the system operation can be processed and recovered in time conveniently.
The log data processing method of the invention generates standard log data by analyzing the collected service log data of each system according to the log data type of each system; then, aggregating all standard log data to obtain aggregated data with a standard format, and performing time zone standardization processing and data completion processing on the aggregated data to obtain aggregated time sequence data with a generation time sequence; then, counting abnormal keywords in the aggregated time sequence data, and early warning the abnormal keywords according to a preset monitoring type; therefore, the monitoring processing of multiple sets of system frameworks is avoided being deployed aiming at different types of log data of each system, the abnormal time can be reflected by the time sequence of the abnormal keywords in the aggregated time sequence data, the monitoring process is simplified, and the cost for the operation and maintenance processing of the multiple sets of system frameworks is saved.
Further, based on the first embodiment of the log data processing method of the present invention, a second embodiment of the log data processing method of the present invention is proposed.
The difference between the second embodiment of the log data processing method and the first embodiment of the log data processing method is that the step of performing early warning on the abnormal keyword according to a preset monitoring type comprises the following steps:
step S31, if the preset monitoring type is keyword monitoring, generating the abnormal keyword as early warning information for early warning;
step S32, if the preset monitoring type is frequency monitoring, judging whether the number of the abnormal keywords counted in a preset time interval is larger than a preset threshold value;
and step S33, if the number of the abnormal keywords is larger than a preset threshold value, generating the abnormal keywords as early warning information to carry out early warning.
In this embodiment, a node of the excepting monitor may be set in the aggregation system to implement a monitoring process, so as to monitor whether abnormal log data exists in the service log data generated by each system, and further obtain abnormal keywords from the abnormal log data for statistics. When each system generates respective service log data, a distinguishing mechanism is arranged between the log data representing abnormal service and the log data representing normal service, for example, an abnormal identifier is added to the abnormal log data, so as to identify the abnormal log data in each service log data.
Further, the preset monitoring type comprises keyword monitoring and frequency monitoring; the keyword monitoring takes whether abnormal keywords exist as a monitoring standard, and the frequency monitoring takes the number of the abnormal keywords as the monitoring standard. Meanwhile, the keyword monitoring and the frequency monitoring are distinguished through monitoring identification, and the server determines the preset monitoring type through determining the type of the monitoring identification which is in an activated state at present. Namely, if the monitoring identifier in the activated state is the keyword identifier, judging that the preset monitoring type is the keyword monitoring; and if the monitoring identifier in the activated state is the frequency identifier, judging that the preset monitoring type is frequency monitoring. For the keyword monitoring, once abnormal keywords exist in the aggregation time sequence data through statistics, the system is judged to be abnormal, and the abnormal keywords are generated as early warning information to carry out early warning.
Furthermore, for frequency monitoring, a preset time interval for counting the number of abnormal keywords is set in advance according to requirements, and when the monitoring reaches the preset time interval, abnormal log data containing the abnormal keywords in the aggregated time series data are counted. In the process of analyzing the service log data to generate standard log data, the abnormal identification used for distinguishing the abnormality in the service log data still exists in the standard log data; after aggregation, abnormal log data existing in the aggregation time sequence data can be found out through the abnormal identifier, and then counting statistics is carried out on abnormal keywords of the abnormal log data.
Further, in order to represent the number of times of the abnormal keyword appearing in the abnormal log data, a preset threshold value is preset. After counting that abnormal keywords exist in the aggregation time sequence data, comparing the counted number of the abnormal keywords with the preset threshold value, judging whether the number of the abnormal keywords is larger than the preset threshold value, if so, indicating that the distributed architecture has more abnormal times in a preset time interval and possibly has the problem of abnormal operation, and generating the abnormal keywords as early warning information to output for early warning.
The method includes the steps that early warning information is generated by abnormal keywords, and the essence of the early warning information is that the abnormal information represented by the abnormal keywords is generated into early warning information; and if the abnormal type and the abnormal time are represented by the type and the generation time of the abnormal keyword, generating early warning information. The early warning information is transmitted to the operation and maintenance personnel for early warning, so that the operation and maintenance personnel can conveniently and quickly determine the type and time of the abnormity, the abnormity can be eliminated in time, and the normal operation of the system can be recovered.
In the embodiment, the preset monitoring type is set to comprise keyword monitoring and frequency monitoring, and for the abnormity with relatively serious degree and high risk degree, the preset monitoring type can be set to be suitable for keyword monitoring, and once the abnormal keywords exist in the aggregation time sequence data, early warning is carried out; and setting the abnormal conditions with relatively light degree and low risk degree to be suitable for frequency monitoring, and monitoring only when the number of keywords in the aggregation time sequence data is counted to be larger than a preset threshold value. Therefore, early warning of different risk degrees is achieved through different monitoring types, and early warning can be more accurate. Meanwhile, the embodiment can also perform independent monitoring on each system, namely generating early warning information of each system by aggregating abnormal keywords from each system in the time sequence data, and performing early warning on each system to realize independent early warning of the system under the distributed architecture.
Further, based on the first embodiment of the log data processing method of the present invention, a third embodiment of the log data processing method of the present invention is proposed.
The third embodiment of the log data processing method is different from the first embodiment of the log data processing method in that the step of performing time zone standardization processing and data completion processing on the aggregated data to obtain aggregated time series data includes:
step S21, obtaining the generation time of each item of service log data, and dividing the aggregated data into first data and second data according to each generation time, wherein the generation time corresponding to each first data is a null value, and the generation time corresponding to each second data is a non-null value;
step S22, acquiring aggregation time for aggregating each item of standard log data, and adjusting the aggregation time corresponding to each item of second data according to the generation time corresponding to each item of second data, so as to perform time zone standardization processing on the aggregated data;
step S23, according to the current time, complementing the generation time corresponding to each first data to perform data complementing processing on the aggregated data, and adjusting the aggregation time corresponding to each first data to the complemented generation time;
step S24 of generating each of the adjusted first data and each of the adjusted second data as the aggregation time-series data.
When the time zone standardization processing is performed on the aggregated data, the generation time of each item of service log data is obtained first, and then the aggregated data is divided into each item of first data and each item of second data on the basis of the generation time of each item of service log data. When the service log data carries time in the generating process, namely the generated time acquired by the service log data is not a null value, the service log data is used as second data in the aggregated data; when the service log data do not carry time in the generating process, namely the acquired generating time is a null value, the service log data are used as various first data in the aggregated data; therefore, according to whether the generation time of each item of service log data is null or not, the aggregated data generated by each item of service log data is divided into each item of first data and each item of second data, so that each item of first data is the data which does not carry the generation time in each item of service data which generates the aggregated data, and each item of second data is the data which carries the generation time in each item of service data which generates the aggregated data.
Further, the time of generating the aggregated data by aggregating the standard log data is used as the aggregation time, and the aggregation time of the aggregated data is read. And then carrying out time zone standardization processing on each item of second data carrying the generation time in the aggregated data, adjusting the aggregation time of each item of second data according to the generation time corresponding to each item of second data, and adjusting the aggregation time of each item of second data into the generation time so as to represent the actual time for generating each item of second data by using the generation time. It should be noted that the aggregated data is aggregated according to various standard log data, and various standard log data are obtained by analyzing various service log data; therefore, the generation time of each acquired service log data is also carried in each standard log data. When aggregation data is obtained by aggregating various standard log data, the aggregation data also carries the generation time of various service log data; therefore, the generation time carried by each item of second data divided by the aggregated data can be adjusted as the generation time corresponding to each item of second data.
Furthermore, when the data completion processing is performed on the aggregated data, the current processing time is directly read as the current time, and the generation time corresponding to each item of the first data is completed. For each item of first data which does not carry generation time in the aggregated data, arranging the current time according to a standard time format as the generation time of the item of first data, and completing the current time of each item of first data. And then, adjusting the respective aggregation time according to the generation time of each item of supplemented first data, and adjusting the aggregation time of each item of second data to the generation time after the supplementation.
Furthermore, after each item of first data is supplemented and adjusted and each item of second data is adjusted, each item of first data and each item of second data are generated into aggregation time sequence data so as to represent the generation time of each item of data in the aggregation data.
Understandably, the aggregation time sequence data which is subjected to time adjustment and contains the generation time embodies the operation process of the system, and in order to facilitate the follow-up search of various data in the aggregation time sequence data to determine whether the system is normal or not, the embodiment is provided with a mechanism for storing the aggregation time sequence data according to the adjusted time sequence. Specifically, the step of generating the adjusted first data and second data as the aggregation time-series data includes:
step S25, storing each first data and each second data which generate the aggregation time sequence data according to the adjusted time sequence;
step S26, when a time sequence retrieval request is received, reading a time period carried in the time sequence retrieval request, and retrieving target data in the aggregated time sequence data according to the time period;
and S27, outputting the target data.
Further, after the aggregation time is adjusted for each item of first data and each item of second data in the aggregated data to obtain aggregated time series data, the first data and the second data are sorted according to the sequence of the adjusted time of each first data and each second data in the aggregated time series data. The subdata with the aggregation time after adjustment is arranged in front, and the subdata with the aggregation time after adjustment is arranged behind to form aggregation time sequence data with time sequence.
Further, the arranged aggregation time sequence data is stored in a storage unit of the aggregation system, and when a time sequence retrieval request which is characterized by retrieval according to time is received, the time period carried in the time sequence retrieval request is read. The representation of the time period requires to acquire the service log data generated in the time period, so that various items of data in the arranged aggregation time sequence data are retrieved, the target data of the aggregation time in the time period is searched out, the searched target data is output, and the retrieval requirement according to the time period retrieval is realized.
The time zone standardization processing and the data completion processing in this embodiment are both used to realize the standardization of the aggregated data in the time dimension, which is beneficial to processing the standardized aggregated data in the same processing manner, and speeding up the processing efficiency. Meanwhile, for the time-adjusted aggregation time sequence data, a mechanism arranged according to the time sequence after adjustment is set, so that retrieval is facilitated according to the time period, the situation that sub-data generated in the time period is searched one by one in the retrieval process is avoided, and the retrieval efficiency is improved.
Further, a fourth embodiment of the log data processing method of the present invention is proposed based on the first, second, or third embodiment of the log data processing method of the present invention.
The fourth embodiment of the log data processing method is different from the first, second, or third embodiments of the log data processing method in that the step of analyzing each item of the service log data according to the log data types of a plurality of the systems to generate standard log data includes:
step S11, determining analysis rules corresponding to the log data types of a plurality of systems, and establishing a mapping relation between the analysis rules corresponding to the same system and the service log data;
executing the following steps for each mapping relation:
and S12, analyzing the service log data in the mapping relation based on a preset standard format according to the analysis rule in the mapping relation, and generating the standard log data corresponding to the mapping relation.
In the embodiment, in the process of analyzing each service log data according to the log data type of each system, an analysis rule for analyzing each log data type is determined first, and then the analysis rule is called to analyze each service log data. Specifically, the service log data generated by each system is different, and the analysis rule for analyzing each type of service log data is also different. The system type identifier for determining the log data type can be used for determining the analysis rule besides the log data type. The incidence relation between the log data type and the analysis rule can be established through the system type identification, after the system type identification of the system from which the service log data comes is found and the log data type is determined according to the system type identification, the analysis rule corresponding to the log data type can be determined through the incidence relation. And determining analysis rules respectively corresponding to the log data types so as to analyze the service log data of the log data types.
Understandably, each item of service log data originates from each system, and the analysis rule for analyzing each item of service log data is different according to different systems, so that the mapping relationship between the analysis rule and the service log data can be established according to the systems, that is, the mapping relationship is established between the analysis rule and the service log data originating from the same system, so as to analyze the service log data therein through the analysis rule in the mapping relationship.
Furthermore, a preset standard format for representing the required format is preset, wherein the preset standard format comprises required information, the existence form and the arrangement sequence of each item of information and the like. The information required for the setting includes time, IP address, thread, source log file, content, etc., where the required time is expressed in "year-month-day: dividing into: seconds "and is arranged in the order of" time, IP address, thread, source log file, content ". Analyzing the service log data in the mapping relation through an analysis rule in the mapping relation, wherein the essence of the analysis is that according to a preset standard format, information represented by the preset standard format is firstly obtained from the service log data, then the obtained information is converted into an information form represented by the preset standard format, and then all the converted information is arranged according to an arrangement sequence represented by the preset standard format, so that the service log data in the mapping relation is analyzed into the standard log data. For example, the service log data of the system a is: the service log data of the' 2019-10-21: { date: '2019-11-03', time: '10PM', action: 'Pay', amont: '10.00' }; the standard log data generated by analysis are respectively as follows: 2019-10-21-00; and B, system: 2019-11-03.
Furthermore, considering that the service log data contains many information items, there may be information that is not needed by the preset standard format, so that before the standard log data is arranged and generated, the information items in the service log data need to be filtered. Specifically, according to an analysis rule in the mapping relationship, analyzing the service log data in the mapping relationship based on a preset standard format, and generating standard log data corresponding to the mapping relationship includes:
step S121, analyzing the service log data in the mapping relation according to the analysis rule in the mapping relation to generate log subdata;
step S122, screening each log subdata and determining target log subdata corresponding to the preset standard format;
step S123, arranging each of the target log subdata according to the preset standard format, and generating the standard log data corresponding to the mapping relationship.
Furthermore, the service log data in the mapping relationship is analyzed according to the analysis rule in the mapping relationship, and the service log data is divided into a plurality of log sub-data, namely, each item of information contained in the service log data, according to the field name or the information partition item of the service log data. And screening the sub-data of each log according to a preset standard format, and determining the information required by representation of the preset standard format as the target sub-data of the log. And then arranging the target log subdata according to the arrangement sequence represented by the preset standard format, and detecting whether the existing form of the target log subdata is consistent with the form required by the current arrangement position or not before reading the target log subdata required to be arranged at the current arrangement position according to the arrangement sequence and arranging the target log subdata. If not, the conversion is carried out, and if the conversion is consistent, the conversion is directly arranged. And finishing the arrangement of each target subdata, namely generating standard log data of the service log data in the mapping relation. And after the business log data in each mapping relation are analyzed to generate respective standard log data, completing the analysis of each business log data from each system to generate each standard data, and performing aggregation processing.
According to the embodiment, the mapping relation between the analysis rule and the service log data from the same system is established for analysis, so that the accuracy of the analysis rule on the service log data is ensured. And meanwhile, the analysis is carried out according to the preset standard format, so that the standard log data obtained by analysis all meet the requirements on the information type, the information form and the arrangement sequence, and the convenience is improved for the subsequent aggregation processing.
The invention also provides a log data processing device.
Referring to fig. 3, fig. 3 is a functional block diagram of a log data processing apparatus according to a first embodiment of the present invention. The log data processing apparatus includes:
the system comprises an acquisition module 10, a storage module and a processing module, wherein the acquisition module is used for acquiring service log data of a plurality of systems, analyzing each service log data according to log data types of the plurality of systems and generating standard log data;
the aggregation module 20 is configured to aggregate the standard log data to generate aggregated data, and perform time zone standardization processing and data completion processing on the aggregated data to obtain aggregated timing data;
and the processing module 30 is configured to count abnormal keywords in the aggregation time series data, and perform early warning on the abnormal keywords according to a preset monitoring type.
Further, the processing module 30 includes:
the first generating unit is used for generating the abnormal keywords as early warning information for early warning if the preset monitoring type is keyword monitoring;
a judging unit, configured to judge whether the number of the abnormal keywords counted in a preset time interval is greater than a preset threshold if the preset monitoring type is frequency monitoring;
and the second generation unit is used for generating the abnormal keywords into early warning information for early warning if the number of the abnormal keywords is greater than a preset threshold value.
Further, the processing module 30 further includes:
an obtaining unit, configured to obtain generation time of each item of service log data, and divide the aggregated data into first data and second data according to each generation time, where the generation time corresponding to each first data is a null value, and the generation time corresponding to each second data is a non-null value;
an adjusting unit, configured to obtain aggregation time for aggregating each item of the standard log data, and adjust the aggregation time corresponding to each item of the second data according to the generation time corresponding to each item of the second data, so as to perform time zone standardization processing on the aggregated data;
a completion unit, configured to complete the generation time corresponding to each of the first data according to the current time, perform data completion processing on the aggregated data, and adjust the aggregation time corresponding to each of the first data to the completed generation time;
a third generating unit configured to generate the adjusted each of the first data and each of the second data as the aggregation timing data.
Further, the processing module further comprises:
the storage unit is used for storing each first data and each second data which generate the aggregation time sequence data according to the adjusted time sequence;
the retrieval unit is used for reading a time period carried in a time sequence retrieval request when the time sequence retrieval request is received, and retrieving target data in the aggregated time sequence data according to the time period;
and the output unit is used for outputting the target subdata.
Further, the acquisition module 10 further includes:
the system comprises an establishing unit, a processing unit and a processing unit, wherein the establishing unit is used for determining analysis rules corresponding to log data types of a plurality of systems and establishing a mapping relation between the analysis rules corresponding to the same systems and the service log data;
executing the following steps for each mapping relation:
and the analysis unit is used for analyzing the service log data in the mapping relation based on a preset standard format according to the analysis rule in the mapping relation to generate the standard log data corresponding to the mapping relation.
Further, the parsing unit is further configured to:
analyzing the service log data in the mapping relation according to the analysis rule in the mapping relation to generate log subdata;
screening each log subdata, and determining target log subdata corresponding to the preset standard format;
and arranging the sub data of the target logs according to the preset standard format to generate the standard log data corresponding to the mapping relation.
Further, the log data processing apparatus further includes:
the judging module is used for judging whether a newly accessed system to be acquired exists or not when the detection reaches the preset acquisition period;
the updating module is used for updating the systems if the newly accessed system to be acquired exists and acquiring the updated service log data of the systems;
and the execution module is used for executing the step of acquiring the service log data of a plurality of systems if the newly accessed system to be acquired does not exist.
The specific implementation of the log data processing apparatus of the present invention is substantially the same as that of each embodiment of the log data processing method described above, and is not described herein again.
In addition, the embodiment of the invention also provides a computer readable storage medium.
The computer readable storage medium has stored thereon a log data processing program which, when executed by a processor, implements the steps of the log data processing method as described above.
The computer-readable storage medium of the present invention may be a computer-readable storage medium, and the specific implementation manner of the computer-readable storage medium is substantially the same as that of each embodiment of the log data processing method, and is not described herein again.
While the embodiments of the present invention have been described with reference to the accompanying drawings, the present invention is not limited to the above embodiments, which are illustrative only and not restrictive, and those skilled in the art can make various changes without departing from the spirit and scope of the invention as claimed.
Claims (9)
1. A log data processing method is characterized by comprising the following steps:
acquiring service log data of a plurality of systems, analyzing each service log data according to log data types of the plurality of systems, and generating standard log data;
aggregating all the standard log data to generate aggregated data, and performing time zone standardization processing and data completion processing on the aggregated data to obtain aggregated time sequence data;
counting abnormal keywords in the aggregation time sequence data, and early warning the abnormal keywords according to a preset monitoring type;
the step of performing time zone standardization processing and data completion processing on the aggregation data to obtain aggregation time sequence data comprises:
acquiring generation time of each item of service log data, and dividing the aggregated data into first data and second data according to each generation time, wherein the generation time corresponding to each first data is a null value, and the generation time corresponding to each second data is a non-null value;
acquiring aggregation time for aggregating each item of standard log data, and adjusting the aggregation time corresponding to each item of second data according to the generation time corresponding to each item of second data so as to perform time zone standardization processing on the aggregated data;
according to the current time, completing the generation time corresponding to each first data so as to perform data completion processing on the aggregated data, and adjusting the aggregation time corresponding to each first data to the completed generation time;
generating the adjusted each of the first data and each of the second data as the aggregated time-series data.
2. The log data processing method of claim 1, wherein the pre-warning of the abnormal keyword according to a preset monitoring type comprises:
if the preset monitoring type is keyword monitoring, generating the abnormal keywords as early warning information for early warning;
if the preset monitoring type is frequency monitoring, judging whether the number of the abnormal keywords counted in a preset time interval is larger than a preset threshold value or not;
and if the number of the abnormal keywords is larger than a preset threshold value, generating the abnormal keywords as early warning information to carry out early warning.
3. The log data processing method of claim 1, wherein the step of generating the adjusted each of the first data and each of the second data as the aggregated time-series data is followed by:
storing each first data and each second data which generate the aggregation time sequence data according to the adjusted time sequence;
when a time sequence retrieval request is received, reading a time period carried in the time sequence retrieval request, and retrieving target data in the aggregated time sequence data according to the time period;
and outputting the target data.
4. The log data processing method according to claim 1, wherein the step of parsing each item of the service log data according to log data types of a plurality of the systems to generate standard log data comprises:
determining analysis rules corresponding to the log data types of a plurality of the systems, and establishing a mapping relation between the analysis rules corresponding to the same system and the service log data;
executing the following steps for each mapping relation:
and analyzing the service log data in the mapping relation based on a preset standard format according to the analysis rule in the mapping relation to generate the standard log data corresponding to the mapping relation.
5. The log data processing method according to claim 4, wherein the step of analyzing the service log data in the mapping relationship based on a preset standard format according to the analysis rule in the mapping relationship, and generating the standard log data corresponding to the mapping relationship comprises:
analyzing the service log data in the mapping relation according to the analysis rule in the mapping relation to generate log subdata;
screening each log subdata, and determining target log subdata corresponding to the preset standard format;
and arranging the sub data of the target logs according to the preset standard format to generate the standard log data corresponding to the mapping relation.
6. The log data processing method of any of claims 1-5, wherein the step of collecting traffic log data for a plurality of systems is preceded by:
when the detection reaches a preset acquisition period, judging whether a newly accessed system to be acquired exists or not;
if the newly accessed system to be acquired exists, updating the systems and acquiring the updated service log data of the systems;
and if the newly accessed system to be acquired does not exist, executing the step of acquiring the service log data of the plurality of systems.
7. A log data processing apparatus characterized by comprising:
the acquisition module is used for acquiring service log data of a plurality of systems, analyzing each service log data according to log data types of the plurality of systems and generating standard log data;
the aggregation module is used for aggregating all the standard log data to generate aggregated data, and performing time zone standardization processing and data completion processing on the aggregated data to obtain aggregated time sequence data;
the aggregation module is further configured to:
acquiring generation time of each item of service log data, and dividing the aggregated data into first data and second data according to each generation time, wherein the generation time corresponding to each first data is a null value, and the generation time corresponding to each second data is a non-null value;
acquiring aggregation time for aggregating each item of standard log data, and adjusting the aggregation time corresponding to each item of second data according to the generation time corresponding to each item of second data so as to perform time zone standardization processing on the aggregated data;
according to the current time, completing the generation time corresponding to each first data so as to perform data completion processing on the aggregated data, and adjusting the aggregation time corresponding to each first data to the completed generation time;
generating the adjusted each of the first data and each of the second data as the aggregated time-series data;
and the processing module is used for counting abnormal keywords in the aggregation time sequence data and carrying out early warning on the abnormal keywords according to a preset monitoring type.
8. A log data processing apparatus comprising a memory, a processor, and a log data processing program stored on the memory and executable on the processor, the log data processing program realizing the steps of the log data processing method according to any one of claims 1 to 6 when executed by the processor.
9. A computer-readable storage medium, characterized in that a log data processing program is stored thereon, which when executed by a processor implements the steps of the log data processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010115818.3A CN111274095B (en) | 2020-02-24 | 2020-02-24 | Log data processing method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010115818.3A CN111274095B (en) | 2020-02-24 | 2020-02-24 | Log data processing method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111274095A CN111274095A (en) | 2020-06-12 |
| CN111274095B true CN111274095B (en) | 2023-01-24 |
Family
ID=70997238
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010115818.3A Active CN111274095B (en) | 2020-02-24 | 2020-02-24 | Log data processing method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111274095B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111526060B (en) * | 2020-06-16 | 2023-02-28 | 网易（杭州）网络有限公司 | Method and system for processing service log |
| CN111881153A (en) * | 2020-07-24 | 2020-11-03 | 北京金山云网络技术有限公司 | Data processing method and device, electronic equipment and machine-readable storage medium |
| CN114301768A (en) * | 2020-09-23 | 2022-04-08 | 中国移动通信集团广东有限公司 | Anomaly detection method and device for Network Function Virtualization (NFV) equipment |
| CN112286757A (en) * | 2020-10-12 | 2021-01-29 | 浙江深大智能科技有限公司 | Data synchronization monitoring method and device, electronic equipment and storage medium |
| CN113312194B (en) * | 2021-06-10 | 2024-01-23 | 中国民航信息网络股份有限公司 | Service data acquisition method and device |
| CN114153389A (en) * | 2021-11-22 | 2022-03-08 | 浙江大华技术股份有限公司 | Automatic abnormal zone supplementing method, device and equipment |
| CN113849337B (en) * | 2021-11-30 | 2022-03-01 | 飞狐信息技术（天津）有限公司 | System exception handling method and device |
| CN114629786A (en) * | 2022-03-22 | 2022-06-14 | 康键信息技术（深圳）有限公司 | Log real-time analysis method, device, storage medium and system |
| CN117033470B (en) * | 2023-10-08 | 2024-01-30 | 天津市天河计算机技术有限公司 | Data generation method, device, equipment and medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003141075A (en) * | 2001-11-06 | 2003-05-16 | Infoscience Corp | Log information management device and log information management program |
| CN107992398A (en) * | 2017-12-22 | 2018-05-04 | 宜人恒业科技发展（北京）有限公司 | The monitoring method and monitoring system of a kind of operation system |
| CN110347547A (en) * | 2019-05-27 | 2019-10-18 | 中国平安人寿保险股份有限公司 | Log method for detecting abnormality, device, terminal and medium based on deep learning |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10331659B2 (en) * | 2016-09-06 | 2019-06-25 | International Business Machines Corporation | Automatic detection and cleansing of erroneous concepts in an aggregated knowledge base |
-
2020
- 2020-02-24 CN CN202010115818.3A patent/CN111274095B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003141075A (en) * | 2001-11-06 | 2003-05-16 | Infoscience Corp | Log information management device and log information management program |
| CN107992398A (en) * | 2017-12-22 | 2018-05-04 | 宜人恒业科技发展（北京）有限公司 | The monitoring method and monitoring system of a kind of operation system |
| CN110347547A (en) * | 2019-05-27 | 2019-10-18 | 中国平安人寿保险股份有限公司 | Log method for detecting abnormality, device, terminal and medium based on deep learning |
Non-Patent Citations (2)
| Title |
|---|
| 任肖肖.基于多源报警日志的网络安全威胁态势感知关键技术研究.《中国优秀博硕士学位论文全文数据库(硕士)》.2016,正文第1-87页. * |
| 基于多源报警日志的网络安全威胁态势感知关键技术研究;任肖肖;《中国优秀博硕士学位论文全文数据库(硕士)》;20160715;正文第1-87页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111274095A (en) | 2020-06-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111274095B (en) | Log data processing method, device, equipment and computer readable storage medium | |
| US20180365085A1 (en) | Method and apparatus for monitoring client applications | |
| CN110888783B (en) | Method and device for monitoring micro-service system and electronic equipment | |
| CN111083225A (en) | Data processing method and device in Internet of things platform and Internet of things platform | |
| CN111740884B (en) | Log processing method, electronic equipment, server and storage medium | |
| CN104007994B (en) | Updating method, upgrading method and upgrading system based on strategy storeroom interaction | |
| CN102999314A (en) | Immediate delay tracker tool | |
| CN105184886A (en) | Cloud data center intelligence inspection system and cloud data center intelligence inspection method | |
| CN112073393B (en) | Flow detection method based on cloud computing and user behavior analysis | |
| CN111858251B (en) | Data security audit method and system based on big data computing technology | |
| US20180095819A1 (en) | Incident analysis program, incident analysis method, information processing device, service identification program, service identification method, and service identification device | |
| CN109408330A (en) | Log analysis method, device, terminal device and readable storage medium storing program for executing | |
| CN112612680A (en) | Message warning method, system, computer equipment and storage medium | |
| CN109409948B (en) | Transaction abnormity detection method, device, equipment and computer readable storage medium | |
| CN106250397B (en) | User behavior characteristic analysis method and device | |
| US9645877B2 (en) | Monitoring apparatus, monitoring method, and recording medium | |
| CN113138906A (en) | Call chain data acquisition method, device, equipment and storage medium | |
| KR102203322B1 (en) | Method for monitoring system based on log, apparatus and system thereof | |
| CN110011845B (en) | Log collection method and system | |
| US8429458B2 (en) | Method and apparatus for system analysis | |
| CN115757045A (en) | Transaction log analysis method, system and device | |
| US11487746B2 (en) | Business impact analysis | |
| CN106777010B (en) | Log providing method and device and log obtaining method, device and system | |
| CN112311679A (en) | State detection method and device, electronic equipment and readable storage medium | |
| CN112866044B (en) | Network equipment state information acquisition method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |