CN111224807A - Distributed log processing method, device, equipment and computer storage medium - Google Patents
Distributed log processing method, device, equipment and computer storage medium Download PDFInfo
- Publication number
- CN111224807A CN111224807A CN201811427214.1A CN201811427214A CN111224807A CN 111224807 A CN111224807 A CN 111224807A CN 201811427214 A CN201811427214 A CN 201811427214A CN 111224807 A CN111224807 A CN 111224807A
- Authority
- CN
- China
- Prior art keywords
- log
- analyzed
- service
- distributed
- running
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a distributed log processing method, a distributed log processing device, distributed log processing equipment and a computer storage medium. The distributed log processing method comprises the following steps: acquiring an operation log and identification information of the operation log from an application host through message middleware; selecting a log to be analyzed from the running logs according to the identification information; and determining index data of the business processing process corresponding to the log to be analyzed according to the log to be analyzed. According to the embodiment of the invention, the real-time performance of data acquisition can be improved, the load of the application host can be reduced, and the index data of the service processing process can be monitored.
Description
Technical Field
The invention belongs to the technical field of information processing, and particularly relates to a distributed log processing method, a distributed log processing device, distributed log processing equipment and a computer storage medium.
Background
The Operation Management for the mobile service support system has been developed for many years, and a mature support platform, i.e. a china mobile service support network Operation Management system (BOMC), is formed. When the BOMC monitors each service application system, it generally collects the process state and middleware queue of each service application system through a collection Agent (Agent), or collects the service index of each service application system through accessing a database storing service information. However, this method can only monitor the performance of the key link of the successful transaction of each service application system, and cannot monitor the failed transaction and determine the fault location.
In order to improve the operation and maintenance means of the service application system, the real-time operation status of the service application system needs to be monitored, and one way is to collect and analyze the operation log of the service application system.
Generally, the log has the following characteristics:
1. the data volume is large: the execution log will record all internal interface calls, which will typically be 10 orders of magnitude larger than the access log.
2. The distribution servers are multiple: the operation logs of each service application system are distributed in different servers and cannot be stored uniformly.
3. The operation environment is complex: the service application system may fall on a different application host environment to run.
In the prior art, a mode of directly capturing an operation log is generally adopted to read log information, namely, after a service application system writes the log information into the operation log and outputs the log information to a specified log directory, the log directory is monitored by an acquisition agent installed on each application host, the log information in the log directory within a certain period is read at regular time, and then the log information is uniformly sent to a log server. Such an approach has the following problems:
1. poor real-time performance: the running log needs to be read by the collection agent and sent to the log server after a certain period.
2. Increasing the load of the application host: the log information needs to be written into the running log firstly and then read by the collection agent, so that the operation of writing and reading the log information can be increased, and the performance of the service application system can be influenced when the service application system is busy.
3. Increasing deployment complexity: when the application host environment is inconsistent or the log directory is changed, the collection agent and the directory configuration need to be adjusted correspondingly.
4. Acquisition cannot be automatically extended: when a newly expanded application host exists, a collection agent needs to be manually installed on the application host, otherwise, log information in the application host cannot be collected.
Disclosure of Invention
Embodiments of the present invention provide a distributed log processing method, apparatus, device, and computer storage medium, which can improve the real-time performance of data acquisition, reduce the load of an application host, and monitor index data in a service processing process.
In one aspect, an embodiment of the present invention provides a distributed log processing method, including:
acquiring an operation log and identification information of the operation log from an application host through message middleware;
selecting a log to be analyzed from the running logs according to the identification information;
and determining index data of the service processing process corresponding to the log to be analyzed according to the log to be analyzed.
Further, before selecting a log to be analyzed from the running logs according to the identification information, the method further includes:
and screening the running logs conforming to the format information according to the preset format information.
Further, still include:
and storing the running log which does not conform to the format information.
Further, the identification information includes a service ID and an interface ID, and selecting the log to be analyzed from the running log according to the identification information includes:
and selecting a log to be analyzed corresponding to the service ID or the interface ID from the running logs according to the service ID or the interface ID.
Further, selecting a log to be analyzed from the running logs according to the identification information further includes:
acquiring relation data of the running log and the acquisition time of the running log;
and selecting a log to be analyzed from the running logs according to the relationship data and the identification information.
Further, according to the log to be analyzed, determining the index data of the service processing process corresponding to the log to be analyzed includes:
acquiring field information related to the service processing process in the log to be analyzed;
and determining the index data of the service processing process according to the field information.
Further, the index data at least includes data of correspondence between a service ID and an interface ID, service index data corresponding to the service processing procedure, and service index data corresponding to the service processing procedure.
In another aspect, an embodiment of the present invention provides a distributed log processing apparatus, where the apparatus includes:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is configured to acquire an operation log and identification information of the operation log from an application host through message middleware;
the processing unit is configured to select a log to be analyzed from the running logs according to the identification information;
and the analysis unit is configured to determine index data of the service processing process corresponding to the log to be analyzed according to the log to be analyzed.
In another aspect, an embodiment of the present invention provides a distributed log processing apparatus, where the apparatus includes: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements the distributed log processing method as described above.
In still another aspect, an embodiment of the present invention provides a computer storage medium, where computer program instructions are stored, and when executed by a processor, the computer program instructions implement the distributed log processing method described above.
The distributed log processing method, the device, the equipment and the computer storage medium of the embodiment of the invention can acquire the running log from the application host through the message middleware, improve the real-time performance of data acquisition and reduce the load of the application host; meanwhile, the logs to be analyzed can be selected from the running logs, and then the index data of the service processing process corresponding to the logs to be analyzed is determined according to the logs to be analyzed, so that the successful transaction service and the failed transaction service in the service processing process can be determined by monitoring the index data of the service processing process, and the failure reason of the failed transaction service is determined according to the index data, so that the fault position is determined.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow diagram of a distributed log processing method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an application side interface structure of the BOMC in the embodiment of the present invention;
FIG. 3 is a flowchart illustrating a distributed log processing method according to another embodiment of the invention;
fig. 4 is a detailed flowchart illustrating an example of step S120 of the distributed log processing method shown in fig. 1;
FIG. 5 is a detailed flowchart of step S130 of the distributed log processing method shown in FIG. 1;
fig. 6 is a schematic structural diagram of a distributed log processing apparatus according to an embodiment of the present invention;
fig. 7 is a schematic hardware structure diagram of a distributed log processing apparatus according to an embodiment of the present invention.
Detailed Description
Features and exemplary embodiments of various aspects of the present invention will be described in detail below, and in order to make objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not to be construed as limiting the invention. It will be apparent to one skilled in the art that the present invention may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the present invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
In order to solve the problem of the prior art, embodiments of the present invention provide a distributed log processing method, apparatus, device, and computer storage medium. First, a distributed log processing method provided by the embodiment of the present invention is described below.
Fig. 1 is a flowchart illustrating a distributed log processing method according to an embodiment of the present invention. As shown in fig. 1, the distributed log processing method includes:
s110, acquiring an operation log and identification information of the operation log from an application host through message middleware;
s120, selecting a log to be analyzed from the running logs according to the identification information;
and S130, determining index data of the service processing process corresponding to the log to be analyzed according to the log to be analyzed.
The embodiment of the invention can acquire the running log from the application host in real time through the message middleware, thereby improving the real-time property of data acquisition, and the application host does not need to write and read the running log, thereby reducing the load of the application host. Meanwhile, the embodiment of the invention can select the log to be analyzed from the running log, and then determine the index data of the service processing process corresponding to the log to be analyzed according to the log to be analyzed, thereby determining the successful transaction service and the failed transaction service in the service processing process by monitoring the index data of the service processing process, and determining the failure reason of the failed transaction service according to the index data to determine the fault position. In addition, in the embodiment of the invention, the problem of consistency of the application host environment and the problem of newly expanding an application main body do not need to be considered, the deployment complexity can be reduced, and the monitoring intelligence and the monitoring automation are improved.
Fig. 2 shows a schematic diagram of an application side interface structure of the BOMC in the embodiment of the present invention. The embodiment of the invention can be applied to a method for processing a distributed operation log in the process of monitoring the operation state of an application host by using the BOMC with the application side interface structure shown in figure 2. The application host 210 may be a terminal that performs mobile service processing by using, for example, a Web client, an application program, and the like, when the application host 210 performs mobile service processing, an operation log may be generated, the terminal may call a client API interface 221 in the application side interface structure 220, and directly output the operation log to the message middleware 230 of the BOMC through an agent 222 in the application side interface structure, and the message middleware 230 may format the operation log, so that the BOMC acquires the formatted operation log and the identification information of the operation log through the message middleware 230, so that the BOMC performs subsequent processing on the operation log.
In the embodiment of the invention, the message middleware is preferably kafka message middleware which has higher throughput and is specially used for log collection and transmission. However, it should be noted that any message middleware having a function of collecting and transmitting the operation log may be applied to the embodiment of the present invention.
Fig. 3 is a flowchart illustrating a distributed log processing method according to another embodiment of the present invention. As shown in fig. 3, before the step S340 of the distributed log processing method selects a log to be analyzed from the running logs according to the identification information, the method further includes:
and S320, screening the running logs conforming to the format information according to the preset format information.
In the embodiment of the invention, the storm program can be used for receiving the running logs transmitted by the message middleware in real time, and format verification is carried out on the received running logs according to the preset format information, so that the running logs which accord with the preset format information are screened out for subsequent analysis and processing.
After the operation logs which accord with the format information are screened out, the part of the operation logs can be stored in the HBase database, so that the data corresponding to the operation logs can be searched and extracted when the operation logs are analyzed and processed. The running logs can be stored in respective detail tables according to different log types. For example, the table name of the detail table may be a log type, and data corresponding to each execution log and identification information corresponding to the execution log may be stored in the detail table.
As shown in fig. 3, the distributed log processing method further includes:
and S330, storing the running log which does not conform to the format information.
Specifically, in the embodiment of the present invention, the running log that does not conform to the format information may be saved in the ES database as an exception log, so as to search for the exception log in the following and analyze the exception reason of the exception log.
In the embodiment of the present invention, the identification information may include a service ID and an interface ID. In the service processing process, when a background interface is called through a foreground in each step, the background interface corresponds to the same interface ID, and each running log has a unique interface ID so as to associate the running logs of the same service, thereby facilitating the analysis and processing of the running logs in the follow-up process. In the process of service processing, when the same service is processed, the same service ID is only provided, and each operation log is only provided with the unique service ID, so that the operation logs of the same service are associated, and the subsequent analysis and processing of the operation logs are facilitated.
Therefore, in an example, in step S120, according to the identification information, a specific method for selecting a log to be analyzed from the running logs may be:
and selecting a log to be analyzed corresponding to the service ID or the interface ID from the running logs according to the service ID or the interface ID.
That is, in the embodiment of the present invention, a time interval for selecting a log to be analyzed corresponding to the service ID or the interface ID from the running log may be preset, for example, the time interval may be set to select the log to be analyzed from the running log every minute, so that the log to be analyzed may be analyzed subsequently.
Fig. 4 is a detailed flowchart illustrating an example of step S120 of the distributed log processing method illustrated in fig. 1. In the example shown in fig. 4, the step S120 of selecting the log to be analyzed from the running logs according to the identification information may further include:
s121, acquiring relation data between the operation log and the acquisition time of the operation log;
and S122, selecting a log to be analyzed from the running logs according to the relation data and the identification information.
Specifically, the relationship data may be parsed from the log to be analyzed. The relationship data may specifically be a relationship between the acquisition time and the interface ID: < time, List < traceId > >. The relational data can be stored in a Redis database for retrieval and use in subsequent analysis processes.
When the preset time interval is reached, whether the mobile service which is processed is ended or not can be determined according to the relation data, if so, all the running logs related to the mobile service are acquired according to the interface ID and the acquiring time sequence to be used as the logs to be analyzed.
Fig. 5 shows a specific flowchart of step S130 of the distributed log processing method shown in fig. 1. As shown in fig. 5, the step S130 of determining, according to the log to be analyzed, the index data of the service processing process corresponding to the log to be analyzed includes:
s131, acquiring field information related to a service processing process in a log to be analyzed;
and S132, determining index data of the service processing process according to the field information.
Firstly, the log to be analyzed is disassembled through the step S131, the field information related to the performance statistics in the single service processing process is analyzed), then, multiple lines of log information for the same call in the field information are combined into one call log information through the step S132, so that index data related to the performance statistics in the single service processing process are obtained, and finally, the index data can be sent to a cache region corresponding to the message middleware to determine the successful service transaction and the failed service transaction in the service processing process and determine the failure reason of the failed service transaction.
Specifically, in the embodiment of the present invention, the index data at least includes correspondence data between a service ID and an interface ID, service index data corresponding to a service processing procedure, and service index data corresponding to the service processing procedure.
The service index data comprises a successful or failed service transaction result and the time from the foreground to the background to finish the service. The service index data comprises the background response time of different services and response results of response success or failure in a service processing process.
The failure reason for handling the failure service can be determined according to the service index data and the service index data, and then the fault position can be determined according to the failure reason through the corresponding relation data of the service ID and the interface ID.
In summary, the embodiment of the present invention directly obtains the operation log from the application host through the message middleware, so that the pressure of reading and writing the operation log when the application host generates a large amount of operation logs can be reduced, the acquisition process of the operation log in the cloud environment is simplified, a log acquisition agent is not required to be deployed in each application host, and the management cost is saved. Meanwhile, the storm program is adopted to process the running logs, the data processing capacity of the cluster environment is fully utilized, a large amount of logs to be analyzed and processed which need to be analyzed and processed can be distributed to a plurality of application hosts to be processed respectively, namely the logs to be analyzed and processed in a distributed mode through a plurality of processes, wherein each application host analyzes and processes all the logs to be analyzed corresponding to the same service processing process, the analysis and processing efficiency is improved, and the capacity of data processing is easy to expand.
Fig. 6 is a schematic structural diagram illustrating a distributed log processing apparatus according to an embodiment of the present invention. As shown in fig. 6, the distributed log processing apparatus includes:
an obtaining unit 410 configured to obtain the operation log and the identification information of the operation log from the application host through the message middleware;
a processing unit 420 configured to select a log to be analyzed from the running logs according to the identification information;
and the analysis unit 430 is configured to determine index data of the service processing process corresponding to the log to be analyzed according to the log to be analyzed.
The embodiment of the invention can acquire the running log from the application host in real time through the message middleware, thereby improving the real-time property of data acquisition, and the application host does not need to write and read the running log, thereby reducing the load of the application host. Meanwhile, the embodiment of the invention can select the log to be analyzed from the running log, and then determine the index data of the service processing process corresponding to the log to be analyzed according to the log to be analyzed, thereby determining the successful transaction service and the failed transaction service in the service processing process by monitoring the index data of the service processing process, and determining the failure reason of the failed transaction service according to the index data to determine the fault position. In addition, in the embodiment of the invention, the problem of consistency of the application host environment and the problem of newly expanding an application main body do not need to be considered, the deployment complexity can be reduced, and the monitoring intelligence and the monitoring automation are improved. In this embodiment of the present invention, a cleaning unit may further be included, which is configured to filter the operation logs that conform to the preset format information, so that the processing unit 420 selects the log to be analyzed from the operation logs filtered by the cleaning unit.
In an example of the embodiment of the present invention, the processing unit 420 may be further configured to select a log to be analyzed corresponding to the service ID or the interface ID from the running logs according to the service ID or the interface ID.
In another example of the embodiment of the present invention, the processing unit 420 may also be further configured to obtain relationship data between the operation log and obtaining time of the operation log, and select a log to be analyzed from the operation log according to the relationship data and the identification information.
In the embodiment of the present invention, a storage unit is further included, and the storage unit may be formed by different databases, such as an ES database, an HBase database, and a Redis database. The HBase database is used for storing the operation logs into respective detail tables according to different log types, the ES database is used for storing the operation logs which do not conform to the format information, and the Redis database is used for storing the relational data.
In this embodiment of the present invention, the parsing unit 430 may further be configured to obtain field information related to a service processing procedure in the log to be analyzed; and determining index data of the service processing process according to the field information.
Specifically, the index data may include at least correspondence data between a service ID and an interface ID, service index data corresponding to a service processing procedure, and service index data corresponding to the service processing procedure.
Fig. 7 is a schematic diagram illustrating a hardware structure of a distributed log processing apparatus according to an embodiment of the present invention.
The distributed log processing device may comprise a processor 501 and a memory 502 in which computer program instructions are stored.
Specifically, the processor 501 may include a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or may be configured as one or more Integrated circuits implementing embodiments of the present invention.
The processor 501 reads and executes the computer program instructions stored in the memory 502 to implement any one of the distributed log processing methods in the above embodiments.
In one example, the distributed log processing device may also include a communication interface 503 and a bus 510. As shown in fig. 7, the processor 501, the memory 502, and the communication interface 503 are connected via a bus 510 to complete communication therebetween.
The communication interface 503 is mainly used for implementing communication between modules, apparatuses, units and/or devices in the embodiments of the present invention.
The distributed log processing device can acquire the running log from the application host in real time based on the message middleware and analyze and process the running log, so that the distributed log processing method and the distributed log processing device are realized.
In addition, in combination with the distributed log processing method in the foregoing embodiment, the embodiment of the present invention may provide a computer storage medium to implement. The computer storage medium having computer program instructions stored thereon; the computer program instructions, when executed by a processor, implement any of the distributed log processing methods in the above embodiments.
It is to be understood that the invention is not limited to the specific arrangements and instrumentality described above and shown in the drawings. A detailed description of known methods is omitted herein for the sake of brevity. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present invention are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications and additions or change the order between the steps after comprehending the spirit of the present invention.
The functional blocks shown in the above-described structural block diagrams may be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, plug-in, function card, or the like. When implemented in software, the elements of the invention are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine-readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" may include any medium that can store or transfer information. Examples of a machine-readable medium include electronic circuits, semiconductor memory devices, ROM, flash memory, Erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, Radio Frequency (RF) links, and so forth. The code segments may be downloaded via computer networks such as the internet, intranet, etc.
It should also be noted that the exemplary embodiments mentioned in this patent describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be performed in an order different from the order in the embodiments, or may be performed simultaneously.
As described above, only the specific embodiments of the present invention are provided, and it can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system, the module and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. It should be understood that the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present invention, and these modifications or substitutions should be covered within the scope of the present invention.
Claims (10)
1. A distributed log processing method, comprising:
acquiring an operation log and identification information of the operation log from an application host through message middleware;
selecting a log to be analyzed from the running logs according to the identification information;
and determining index data of the service processing process corresponding to the log to be analyzed according to the log to be analyzed.
2. The distributed log processing method according to claim 1, wherein before selecting a log to be analyzed from the running logs according to the identification information, the method further comprises:
and screening the running logs conforming to the format information according to the preset format information.
3. The distributed log processing method of claim 2, further comprising:
and storing the running log which does not conform to the format information.
4. The distributed log processing method according to claim 1, wherein the identification information includes a service ID and an interface ID, and selecting the log to be analyzed from the running log according to the identification information includes:
and selecting a log to be analyzed corresponding to the service ID or the interface ID from the running logs according to the service ID or the interface ID.
5. The distributed log processing method according to claim 1, wherein selecting a log to be analyzed from the running logs according to the identification information further comprises:
acquiring relation data of the running log and the acquisition time of the running log;
and selecting a log to be analyzed from the running logs according to the relationship data and the identification information.
6. The distributed log processing method according to claim 1, wherein determining, according to the log to be analyzed, index data of a service processing process corresponding to the log to be analyzed includes:
acquiring field information related to the service processing process in the log to be analyzed;
and determining the index data of the service processing process according to the field information.
7. The distributed log processing method according to claim 6, wherein the index data at least includes correspondence data of a service ID and an interface ID, service index data corresponding to the service processing procedure, and service index data corresponding to the service processing procedure.
8. A distributed log processing apparatus, the apparatus comprising:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is configured to acquire an operation log and identification information of the operation log from an application host through message middleware;
the processing unit is configured to select a log to be analyzed from the running logs according to the identification information;
and the analysis unit is configured to determine index data of the service processing process corresponding to the log to be analyzed according to the log to be analyzed.
9. A distributed log processing apparatus, characterized in that the apparatus comprises: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements the distributed log processing method of any of claims 1-7.
10. A computer storage medium having computer program instructions stored thereon, which when executed by a processor implement the distributed log processing method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811427214.1A CN111224807B (en) | 2018-11-27 | 2018-11-27 | Distributed log processing method, device, equipment and computer storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811427214.1A CN111224807B (en) | 2018-11-27 | 2018-11-27 | Distributed log processing method, device, equipment and computer storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111224807A true CN111224807A (en) | 2020-06-02 |
| CN111224807B CN111224807B (en) | 2023-08-01 |
Family
ID=70828830
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811427214.1A Active CN111224807B (en) | 2018-11-27 | 2018-11-27 | Distributed log processing method, device, equipment and computer storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111224807B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111639138A (en) * | 2020-06-03 | 2020-09-08 | 中国联合网络通信集团有限公司 | Data processing method, device, equipment and storage medium |
| CN111897834A (en) * | 2020-08-12 | 2020-11-06 | 网易（杭州）网络有限公司 | Log searching method and device and server |
| CN112416887A (en) * | 2020-11-18 | 2021-02-26 | 脸萌有限公司 | Information interaction method and device and electronic equipment |
| CN112965815A (en) * | 2021-02-19 | 2021-06-15 | 中国工商银行股份有限公司 | Host deployment method and device, electronic equipment and computer storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101938373A (en) * | 2010-08-16 | 2011-01-05 | 北京全路通信信号研究设计院 | Analysis and treatment method of RBC (Recording Buffer Controller) log record and RBC record analyzer |
| CN102385549A (en) * | 2010-09-02 | 2012-03-21 | 北京无限立通通讯技术有限责任公司 | Log processing system, log processing method and log storage sub-system |
| CN103023693A (en) * | 2012-11-27 | 2013-04-03 | 北京小米科技有限责任公司 | Behaviour log data management system and behaviour log data management method |
| CN103178982A (en) * | 2011-12-23 | 2013-06-26 | 阿里巴巴集团控股有限公司 | Method and device for analyzing log |
| US20170177861A1 (en) * | 2014-02-12 | 2017-06-22 | Mitsubishi Electric Corporation | Log analysis device, unauthorized access auditing system, computer readable medium storing log analysis program, and log analysis method |
| KR20180065400A (en) * | 2016-12-07 | 2018-06-18 | 동국대학교 산학협력단 | System and method for analyzing log data |
-
2018
- 2018-11-27 CN CN201811427214.1A patent/CN111224807B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101938373A (en) * | 2010-08-16 | 2011-01-05 | 北京全路通信信号研究设计院 | Analysis and treatment method of RBC (Recording Buffer Controller) log record and RBC record analyzer |
| CN102385549A (en) * | 2010-09-02 | 2012-03-21 | 北京无限立通通讯技术有限责任公司 | Log processing system, log processing method and log storage sub-system |
| CN103178982A (en) * | 2011-12-23 | 2013-06-26 | 阿里巴巴集团控股有限公司 | Method and device for analyzing log |
| CN103023693A (en) * | 2012-11-27 | 2013-04-03 | 北京小米科技有限责任公司 | Behaviour log data management system and behaviour log data management method |
| US20170177861A1 (en) * | 2014-02-12 | 2017-06-22 | Mitsubishi Electric Corporation | Log analysis device, unauthorized access auditing system, computer readable medium storing log analysis program, and log analysis method |
| KR20180065400A (en) * | 2016-12-07 | 2018-06-18 | 동국대학교 산학협력단 | System and method for analyzing log data |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111639138A (en) * | 2020-06-03 | 2020-09-08 | 中国联合网络通信集团有限公司 | Data processing method, device, equipment and storage medium |
| CN111639138B (en) * | 2020-06-03 | 2023-04-25 | 中国联合网络通信集团有限公司 | Data processing method, device, equipment and storage medium |
| CN111897834A (en) * | 2020-08-12 | 2020-11-06 | 网易（杭州）网络有限公司 | Log searching method and device and server |
| CN112416887A (en) * | 2020-11-18 | 2021-02-26 | 脸萌有限公司 | Information interaction method and device and electronic equipment |
| CN112416887B (en) * | 2020-11-18 | 2024-01-30 | 脸萌有限公司 | Information interaction method and device and electronic equipment |
| CN112965815A (en) * | 2021-02-19 | 2021-06-15 | 中国工商银行股份有限公司 | Host deployment method and device, electronic equipment and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111224807B (en) | 2023-08-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109995555B (en) | Monitoring method, device, equipment and medium | |
| CN111224807B (en) | Distributed log processing method, device, equipment and computer storage medium | |
| CN109992427B (en) | DPI association rule backfill processing method, device, equipment and medium | |
| US20200341868A1 (en) | System and Method for Reactive Log Spooling | |
| CN111367760B (en) | Log collection method and device, computer equipment and storage medium | |
| CN114077525A (en) | Abnormal log processing method and device, terminal equipment, cloud server and system | |
| CN111382146A (en) | Data storage control method, device, equipment and storage medium | |
| CN111193664A (en) | Link backup method and device for computer network | |
| CN111372209A (en) | Signaling data processing method, device, equipment and medium | |
| CN110083575A (en) | Fulfilling monitoring method, device, equipment and computer readable storage medium | |
| CN109992448A (en) | File change incremental backup method, device, equipment and medium | |
| WO2023125053A1 (en) | Data acquisition method and apparatus, electronic device and storage medium | |
| CN111177193A (en) | Flink-based log streaming processing method and system | |
| CN106951360B (en) | Data statistical integrity calculation method and system | |
| CN116302989A (en) | Pressure testing method and system, storage medium and computer equipment | |
| CN108829563B (en) | Alarm method and alarm device | |
| CN106648985A (en) | Disaster-tolerant repair method and device of text database | |
| JPH0865302A (en) | Network operation information collecting system | |
| CN113407428B (en) | Reliability evaluation method and device for artificial intelligence system and computer equipment | |
| CN113590047B (en) | Database screening method and device, electronic equipment and storage medium | |
| CN113342861B (en) | Data management method and device in service scene | |
| CN111294071B (en) | State data management method, device, equipment and medium for radio remote unit | |
| CN116094908A (en) | Log file storage method, device, equipment and storage medium | |
| CN116028362A (en) | Data acquisition method, device, equipment and storage medium | |
| CN114327967A (en) | Equipment repairing method and device, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |