CN111740868A - Alarm data processing method and device and storage medium - Google Patents

Alarm data processing method and device and storage medium Download PDF

Info

Publication number
CN111740868A
CN111740868A CN202010647837.0A CN202010647837A CN111740868A CN 111740868 A CN111740868 A CN 111740868A CN 202010647837 A CN202010647837 A CN 202010647837A CN 111740868 A CN111740868 A CN 111740868A
Authority
CN
China
Prior art keywords
target
time
data
log
alarm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010647837.0A
Other languages
Chinese (zh)
Other versions
CN111740868B (en
Inventor
李首正
孟冉
黄小华
雷蔡芳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202010647837.0A priority Critical patent/CN111740868B/en
Publication of CN111740868A publication Critical patent/CN111740868A/en
Application granted granted Critical
Publication of CN111740868B publication Critical patent/CN111740868B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/0604Management of faults, events, alarms or notifications using filtering, e.g. reduction of information by using priority, element types, position or time
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06Management of faults, events, alarms or notifications
    • H04L41/069Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications

Abstract

The invention discloses a method and a device for processing alarm data in a cloud technology scene and a storage medium, and particularly relates to a verification technology in the field of cloud security, an inquiry technology in the field of databases and the like. Wherein, the method comprises the following steps: acquiring alarm indicating data reported by each device in a target content distribution network, wherein the alarm indicating data is used for indicating whether the device is in a fault state or not; determining target equipment from a target content distribution network according to the alarm indication data, wherein the target equipment is equipment with the most times of reporting the alarm indication data; acquiring a target log of a target device; filtering the target log according to the first target filtering field to obtain first target fault data; and carrying out statistical processing on the first target fault data to obtain a first processing result, and displaying the first processing result. The invention solves the technical problem of low processing efficiency of alarm data.

Description

Alarm data processing method and device and storage medium
Technical Field
The invention relates to the technical field of cloud, in particular to a method and a device for processing alarm data and a storage medium.
Background
In recent years, a Content Delivery Network (CND) has become widely used as a Network application service model due to its high efficiency and high quality of service. In this context, the maintenance of CND networks is also becoming an important issue. In the prior art, the alarm information of the CND network device is mostly displayed in a curve view manner, after receiving a telephone alarm, an operation and maintenance worker checks the displayed curve view to obtain preliminary alarm information, then logs in a service monitoring view to find a view, and locates a problem through an operation and maintenance tool on an autonomous operation system. In other words, in the prior art, only the alarm data is processed into a curve view and displayed to the operation and maintenance personnel, and the operation and maintenance personnel need to distinguish which are invalid alarm data and which are valid alarm data in the massive curve view, and further process and analyze the valid alarm data, thereby resulting in lower maintenance efficiency of the CND network. Therefore, there is a problem that the processing efficiency of the alarm data is low.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a method and a device for processing alarm data and a storage medium, which are used for at least solving the technical problem of low processing efficiency of the alarm data.
According to an aspect of the embodiments of the present invention, a method for processing alarm data is provided, including: acquiring alarm indication data reported by each device in a target content distribution network, wherein the alarm indication data is used for indicating whether the device is in a fault state; determining target equipment from the target content distribution network according to the alarm indication data, wherein the target equipment is the equipment with the most times of reporting the alarm indication data; acquiring a target log of the target device, wherein the target log is used for recording the running data of the target device in a preset time period; filtering the target log according to a first target filtering field to obtain first target fault data; and performing statistical processing on the first target fault data to obtain a first processing result, and displaying the first processing result.
According to an aspect of the embodiments of the present invention, there is provided an apparatus for processing alarm data, including: a first obtaining unit, configured to obtain alarm indication data reported by each device in a target content distribution network, where the alarm indication data is used to indicate whether the device is in a fault state; a first determining unit, configured to determine a target device from the target content distribution network according to the alarm indication data, where the target device is the device that reports the alarm indication data the most times; a second obtaining unit, configured to obtain a target log of the target device, where the target log is used to record operation data of the target device within a predetermined time period; the first processing unit is used for filtering the target log according to a first target filtering field to obtain first target fault data; and the first display unit is used for carrying out statistical processing on the first target fault data to obtain a first processing result and displaying the first processing result.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, where the computer program is configured to execute the above processing method of alarm data when running.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for processing the alarm data through the computer program.
In the embodiment of the invention, alarm indication data reported by each device in a target content distribution network is obtained, wherein the alarm indication data is used for indicating whether the device is in a fault state or not; determining target equipment from the target content distribution network according to the alarm indication data, wherein the target equipment is the equipment with the most times of reporting the alarm indication data; acquiring a target log of the target device, wherein the target log is used for recording the running data of the target device in a preset time period; filtering the target log according to a first target filtering field to obtain first target fault data; the method comprises the steps of counting the first target fault data to obtain a first processing result, displaying the first processing result, determining target equipment (single-point equipment) with the maximum fault probability in all CDN equipment according to fault data reported by all CDN equipment, calling and processing target logs of the target equipment aiming at the target equipment, and filtering a large number of target logs to ensure that the processed alarm data have high efficiency, so that the purpose of effectively processing massive alarm data is achieved, the effect of improving the processing efficiency of the alarm data is achieved, and the technical problem of low processing efficiency of the alarm data is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a schematic diagram of an application environment of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a flow chart of an alternative method of alarm data processing according to an embodiment of the invention;
FIG. 3 is a diagram illustrating an alternative method for processing alarm data according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 5 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 6 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 7 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 8 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 9 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 10 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 11 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 12 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 13 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 14 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 15 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 16 is a schematic diagram of an alternative alarm data processing method according to an embodiment of the invention;
FIG. 17 is a schematic diagram of an alternative alert data processing apparatus according to an embodiment of the present invention;
FIG. 18 is a schematic diagram of an alternative alert data processing apparatus according to an embodiment of the present invention;
FIG. 19 is a schematic diagram of an alternative alert data processing apparatus according to an embodiment of the present invention;
FIG. 20 is a schematic diagram of an alternative alert data processing apparatus according to an embodiment of the present invention;
fig. 21 is a schematic structural diagram of an alternative electronic device according to an embodiment of the invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Cloud technology refers to a hosting technology for unifying serial resources such as hardware, software, network and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data.
Cloud technology (Cloud technology) is based on a general term of network technology, information technology, integration technology, management platform technology, application technology and the like applied in a Cloud computing business model, can form a resource pool, is used as required, and is flexible and convenient. Cloud computing technology will become an important support. Background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture-like websites and more web portals. With the high development and application of the internet industry, each article may have its own identification mark and needs to be transmitted to a background system for logic processing, data in different levels are processed separately, and various industrial data need strong system background support and can only be realized through cloud computing.
Database (Database), which can be regarded as an electronic file cabinet in short, a place for storing electronic files, a user can add, query, update, delete, etc. to data in files. A "database" is a collection of data that is stored together in a manner that can be shared by multiple users, has as little redundancy as possible, and is independent of the application.
A Database Management System (DBMS) is a computer software System designed for managing a Database, and generally has basic functions of storage, interception, security assurance, backup, and the like. The database management system may classify the database according to the database model it supports, such as relational, XML (Extensible Markup Language); or classified according to the type of computer supported, e.g., server cluster, mobile phone; or sorted according to the Query Language used, such as SQL (Structured Query Language), XQuery, or sorted according to performance impulse emphasis, such as max size, maximum operating speed, or other sorting.
Cloud Security (Cloud Security) refers to a generic term for Security software, hardware, users, organizations, secure Cloud platforms for Cloud-based business model applications. The cloud security integrates emerging technologies and concepts such as parallel processing, grid computing and unknown virus behavior judgment, abnormal monitoring of software behaviors in the network is achieved through a large number of meshed clients, the latest information of trojans and malicious programs in the internet is obtained and sent to the server for automatic analysis and processing, and then the virus and trojan solution is distributed to each client.
The main research directions of cloud security include: 1. the cloud computing security mainly researches how to guarantee the security of the cloud and various applications on the cloud, including the security of a cloud computer system, the secure storage and isolation of user data, user access authentication, information transmission security, network attack protection, compliance audit and the like; 2. the cloud of the security infrastructure mainly researches how to adopt cloud computing to newly build and integrate security infrastructure resources and optimize a security protection mechanism, and comprises the steps of constructing a super-large-scale security event and an information acquisition and processing platform through a cloud computing technology, realizing the acquisition and correlation analysis of mass information, and improving the handling control capability and the risk control capability of the security event of the whole network; 3. the cloud security service mainly researches various security services, such as anti-virus services and the like, provided for users based on a cloud computing platform.
According to an aspect of the embodiment of the present invention, a method for processing alarm data is provided, and optionally, as an optional implementation manner, the method for processing alarm data may be applied to, but is not limited to, an environment as shown in fig. 1. The system may include, but is not limited to, a user device 102, a network 110, and a server 112, wherein the user device 102 may include, but is not limited to, a display 108, a processor 106, and a memory 104, wherein the display 108 may be used, but is not limited to, displaying a target log 1022 and a processing result 1024 of a target device (not shown), and a "query" location on the display 108 may be used, but is not limited to, triggering a processing instruction.
The specific process comprises the following steps:
step S102, the user equipment 102 obtains a processing instruction through an "inquiry" location on the display 108, where the processing instruction is used to instruct to process a target log 1022 of a target device (not shown in the figure) according to a preset policy, where the preset policy may include, but is not limited to, filtering the target log 1022 according to key fields, and the preset policy may also include, but is not limited to, counting the target log 1022 filtered according to key fields;
step S104-S106, the user equipment 102 sends the processing instruction to the server 112 through the network 110;
step S108, the server 112 processes the target log 1022 through the processing engine 116 according to the processing instruction, so as to generate a processing result 1024;
steps S110-S112, the server 112 sends the processing result to the user device 102 through the network 110, and the processor 106 in the user device 102 displays the processing result 1024 on the display 108 and stores the processing result 1024 in the memory 104, wherein the processing result 1024 may include, but is not limited to, the processed target log 1022.
Optionally, as an optional implementation manner, as shown in fig. 2, the method for processing alarm data includes:
s202, acquiring alarm indicating data reported by each device in the target content distribution network, wherein the alarm indicating data is used for indicating whether the device is in a fault state;
s204, determining target equipment from the target content distribution network according to the alarm indication data, wherein the target equipment is equipment with the most times of reporting the alarm indication data;
s206, acquiring a target log of the target equipment, wherein the target log is used for recording the running data of the target equipment in a preset time period;
s208, filtering the target log according to the first target filtering field to obtain first target fault data;
s210, statistical processing is carried out on the first target fault data to obtain a first processing result, and the first processing result is displayed.
Optionally, the method for processing the alarm data may be but not limited to be applied to a maintenance scenario of the CDN network device, and specifically, the method for processing the alarm data may be but not limited to automatically obtain an existing network device that sends an alarm, and perform automatic analysis quickly before a call notifies an operation and maintenance, and determine whether to intercept, thereby reducing disturbance of an invalid alarm to the operation and maintenance, helping the operation and maintenance to locate a fault faster, and solving a problem; the alarm data processing method can also but not limited to the effect of full link analysis by searching back the back source type alarm through iteration; the alarm data processing method can also be used for but not limited to simultaneously finding out a group of nodes of different regions and different operators as analysis tools, and then performing statistical accumulation, so that the obtained data is more real and more accords with the state of the existing network. Optionally, the CDN network may be, but not limited to, an intelligent virtual network constructed on the basis of an existing network, and by means of edge servers deployed in various places and through functional modules of load balancing, content distribution, scheduling, and the like of the central platform, a user can obtain required content nearby, network congestion is reduced, and the access response speed and hit rate of the user are improved.
Optionally, the alarm data processing method may enable the CDN operation and maintenance to obtain the access request status and the source return link status of the current network device in the alarm time period only by using a mobile phone without starting a computer, and these pieces of information may reflect the problems that occur, so that the automatic analysis of the alarm greatly accelerates the speed of the operation and maintenance to troubleshoot the problems.
It should be noted that, the alarm indication data reported by each device in the target content distribution network is obtained, where the alarm indication data is used to indicate whether the device is in a fault state; determining target equipment from a target content distribution network according to the alarm indication data, wherein the target equipment is equipment with the most times of reporting the alarm indication data; acquiring a target log of target equipment, wherein the target log is used for recording operation data of the target equipment in a preset time period; filtering the target log according to the first target filtering field to obtain first target fault data; and carrying out statistical processing on the first target fault data to obtain a first processing result, and displaying the first processing result. Optionally, the alarm indication data may be, but is not limited to, shown in a curve view, a table view, and the like, the alarm indication data may be, but is not limited to, summarized by reporting data of all or part of device nodes in the CDN network, and the alarm indication data may be, but is not limited to, used for showing an index change trend of all or part of devices in the CDN network and all or part of reasons for an alarm. The first destination filter field may include, but is not limited to, fields that may be used to filter out invalid data, such as a time of day field, a domain name field, a status field, and the like. The statistical process may be, but not limited to, used to count the fault data that meets the preset policy, for example, count a most dominant client Internet Protocol (IP) in the filtered fault data.
For further example, optionally, for example, as shown in fig. 3, the method includes the alarm indication data 302, where the alarm indication data 302 includes alarm data reported by each device in the target content distribution network 314, specifically, the target content distribution network 314 includes a target device 310 and a device 312, where the first alarm indication data 304 and the second alarm indication data 306 are reported by the target device 310, and the third alarm indication data 308 is reported by the device 312; in the alarm indication data 302, the first alarm indication data 304 and the second alarm indication data 306 are reported by the target device 310, and the third alarm indication data 308 is reported by the device 312, in other words, in the alarm indication data 302, the target device 310 reports the alarm data more times than the device 312, or in other words, the number of times the target device 310 reports the alarm data is the most in each device in the target content distribution network 314; accordingly, the target device 310 is determined to be the device that reports the alarm data the most times.
For further example, optionally, as shown in fig. 4, the method includes the following steps that: s402, filtering the target log 402 according to a first target filtering field (not shown in the figure); s404, performing statistical processing on the first target fault data 404 to obtain a first processing result 406.
According to the embodiment provided by the application, the alarm indication data reported by each device in the target content distribution network is obtained, wherein the alarm indication data is used for indicating whether the device is in a fault state; determining target equipment from a target content distribution network according to the alarm indication data, wherein the target equipment is equipment with the most times of reporting the alarm indication data; acquiring a target log of target equipment, wherein the target log is used for recording operation data of the target equipment in a preset time period; filtering the target log according to the first target filtering field to obtain first target fault data; the method comprises the steps of carrying out statistical processing on first target fault data to obtain a first processing result, displaying the first processing result, determining target equipment (single-point equipment) with the maximum fault probability in all CDN equipment through fault data reported by all CDN equipment, calling and processing target logs of the target equipment aiming at the target equipment, and filtering a large number of target logs to ensure that processed alarm data have high efficiency, so that the purpose of effectively processing massive alarm data is achieved, and the effect of improving the processing efficiency of the alarm data is achieved.
As an optional scheme, filtering the target log according to the first target filtering field, and obtaining the first target fault data includes at least one of:
s1, filtering the target log according to the first filtering field to obtain first fault data in a target preset time period, wherein the first filtering field is used for indicating to filter the running data in a non-target preset time period;
s2, filtering the target log according to a second filtering field to obtain second fault data of the target domain name, wherein the second filtering field is used for indicating to filter the operating data of the non-target domain name;
and S3, filtering the target log according to a third filtering field to obtain third fault data of the target state code, wherein the third filtering field is used for indicating to filter the running data of the non-target state code.
It should be noted that, the target log is filtered according to the first filtering field to obtain first fault data in a target predetermined time period, where the first filtering field is used to indicate to filter operating data in a non-target predetermined time period; filtering the target log according to a second filtering field to obtain second fault data of the target domain name, wherein the second filtering field is used for indicating to filter the operating data of the non-target domain name; and filtering the target log according to a third filtering field to obtain third fault data of the target state code, wherein the third filtering field is used for indicating the filtering of the running data of the non-target state code.
For further example, optionally, as shown in fig. 5, for example, the target log 502 is filtered according to the first filtering field 504 to obtain first fault data 506 within a target predetermined time period, specifically, the first filtering field 504 may be, but is not limited to, a target predetermined time period, and then all fault data in the first fault data 506 outside the target predetermined time period are filtered out, and only fault data in the first fault data 506 within the target predetermined time period is left.
For further example, optionally, if it is desired to obtain fault data with a domain name status code of xxxx.qq.com of 404 in the target log at a time of 12:06 to 12:10, it may be, but is not limited to, filter out fault data that does not satisfy the condition by setting a first filtering field to be "12: 06 to 12: 10", a second filtering field to be "xxxx.qq.com", and a third filtering field to be "404".
According to the embodiment provided by the application, the target log is filtered according to the first filtering field to obtain first fault data in the target preset time period, wherein the first filtering field is used for indicating to filter the running data in the non-target preset time period; filtering the target log according to a second filtering field to obtain second fault data of the target domain name, wherein the second filtering field is used for indicating to filter the operating data of the non-target domain name; and filtering the target log according to a third filtering field to obtain third fault data of the target state code, wherein the third filtering field is used for indicating and filtering the running data of the non-target state code, so that the purpose of increasing the filtering type of the fault data is achieved, and the effect of improving the flexibility of data filtering is realized.
As an optional scheme, performing statistical processing on the first target fault data to obtain a first processing result, and displaying the first processing result includes:
and counting target fault data meeting preset conditions, obtaining second target fault data of a target type, displaying the second target fault data, and displaying target alarm indication data corresponding to the second target fault data, wherein the preset conditions are used for indicating to obtain the target fault data of the target type, and the target alarm indication data are used for indicating that the target equipment is in a target fault state.
It should be noted that, the target fault data meeting the preset condition is counted to obtain second target fault data of the target type, the second target fault data is displayed, and target alarm indication data corresponding to the second target fault data is displayed, where the preset condition is used to indicate that the target fault data of the target type is obtained, and the target alarm indication data is used to indicate that the target device is in the target fault state.
For further example, optionally, as shown in fig. 6, according to a preset condition 604, statistically processing the first target fault data 602, obtaining statistically processed second target fault data 606, and obtaining target alarm indication data 608 corresponding to the second target fault data 606.
For further example, if it is optional to obtain a most-occupied Uniform Resource Locator (URL) and a client IP, the occupation ratios of all URLs and client IPs may be counted in the target fault data, and the most-occupied URL and client IP may be determined.
According to the embodiment provided by the application, the target fault data meeting the preset conditions are counted to obtain the second target fault data of the target type, the second target fault data are displayed, and the target alarm indication data corresponding to the second target fault data are displayed, wherein the preset conditions are used for indicating to obtain the target fault data of the target type, and the target alarm indication data are used for indicating that the target equipment is in the target fault state, so that the purpose of quickly counting the fault data meeting the preset conditions is achieved, and the effect of improving the processing efficiency of the fault data is achieved.
As an optional scheme, after performing statistical processing on the first target fault data to obtain a first processing result and displaying the first processing result, the method includes:
s1, filtering the target log according to the second target filtering field to obtain second target fault data;
and S2, performing statistical processing on the second target fault data to obtain a second statistical result, and displaying the second statistical result.
It should be noted that, the target log is filtered according to the second target filtering field to obtain second target fault data; and carrying out statistical processing on the second target fault data to obtain a second statistical result, and displaying the second statistical result.
For further illustration, optionally, for example, as shown in fig. 7, the system includes a filter 704 and a filter 710 for performing filtering processing on the target log 702, a statistician 706 and a statistician 714 for performing statistical processing on the target log 702; inputting the target log 702 into the filter 104, inputting the result output by the filter 104 into the statistics device 706, and obtaining a first processing result 708; the target log 702 is input to the filter 710, the result output by the filter 710 is input to the filter 712, the result output by the filter 712 is input to the statistics device 714, and the second processing result 716 output by the statistics device 714 is obtained, wherein the optional filter 704, the filter 710, the filter 712, the statistics device 706, and the statistics device 714 can be flexibly combined in different business scenarios, and are not limited to the combination relationship shown in fig. 7.
For further example, optionally, a function chain is formed by combining a filter and a statistics device to improve flexibility of data processing, and thus, the operation and maintenance personnel only need to specify the used parsing rule to perform customized aggregation statistics on the log. During the processing, a temporary index of fields is established for each row of logs needing to be processed, and then the analysis mode specified by the operation and maintenance personnel is used for processing the index. The design ensures that even a long analysis rule does not influence the overall analysis performance too much.
According to the embodiment provided by the application, the target log is filtered according to the second target filtering field to obtain second target fault data; and carrying out statistical processing on the second target fault data to obtain a second statistical result, and displaying the second statistical result, so that the aim of flexibly filtering and counting the target logs is fulfilled, and the effect of improving the processing flexibility of the target logs is realized.
As an optional scheme, obtaining the target log of the target device includes:
s1, writing the running data of the target log into a cache region of a kernel space in a log analysis system, wherein the log analysis system is used for analyzing the target log;
s2, mapping the operation data written into the cache region to a mapping region of a process space in the log analysis system according to a target mapping relationship, wherein the target mapping relationship is a mapping relationship between the cache region and the mapping region which is established in advance.
Optionally, in order to improve the efficiency of reading and processing a large log file, the running efficiency is greatly improved by directly reading the page cache of the log file instead of copying the page cache to the user process space. Each CDN log is a gz compressed file of 1G or more, and if the file is read into the process space by simply using a standard IO, the time consumption is very large and unacceptable. In order to solve the problem of reading in large files, memory mapping is adopted to map the file privately of the log into the process memory space of the analysis tool, so that not only is the time reduced, but also the CPU consumption and the memory consumption of the integral copying of the file are saved. Alternatively, the memory mapping may be, but is not limited to, mapping from a file to a memory, where the memory mapping file may be, but is not limited to, reserving an address space region and giving physical storage to the region, where the physical storage of the memory file mapping is from a file already existing on the disk, and the file must be mapped first before the file is operated. When the memory mapped file is used for processing the file stored on the disk, I/O operation on the file is not needed, so that the memory mapped file can play a very important role in processing the file with large data volume.
It should be noted that, the running data of the target log is written into a cache area of a kernel space in the log analysis system, where the log analysis system is used to analyze the target log; and mapping the running data written into the cache region to a mapping region of a process space in the log analysis system according to a target mapping relationship, wherein the target mapping relationship is a mapping relationship between the cache region and the mapping region which is established in advance.
For further example, an optional example is shown in fig. 8, which includes a log analysis system 802 for processing and analyzing logs, where the log analysis system 802 includes a kernel space 804 and a process space 808, the kernel space 804 includes a cache region 806, and the process space 808 includes a mapping region 810; pre-establishing a target mapping relation 812 between the cache region 806 and the mapping region 810; target logs (not shown) written into the log analysis system 802 are cached in the cache area 806, and the target logs cached in the cache area 806 are mapped into a mapping area 810 in the process space 808 according to a target mapping relation 812.
By the embodiment provided by the application, the running data of the target log is written into the cache region of the kernel space in the log analysis system, wherein the log analysis system is used for analyzing the target log; and mapping the running data written into the cache region to a mapping region of a process space in the log analysis system according to a target mapping relationship, wherein the target mapping relationship is a mapping relationship between the cache region and the mapping region which is established in advance, so that the aim of directly mapping the file of the log to the process space by adopting memory mapping is fulfilled, and the effect of improving the processing efficiency of the log is realized.
As an optional scheme, before obtaining the target log of the target device, the method includes:
s1, acquiring all logs of the target device, wherein all logs comprise a plurality of compressed logs, the compressed logs are logs in a compressed state, and the compressed logs comprise time marks, and the time marks are used for indicating the compression time of each log in all logs;
s2, acquiring a plurality of time identifications respectively matched with each of the plurality of compressed logs;
s3, determining N target time identifiers in the time identifiers according to a binary search algorithm, wherein the N target time identifiers are used for representing N target times of a compressed target log, the N target times are used for representing a preset time period, and N is an integer greater than or equal to 0;
s4, determining N compressed logs matched with the N target time identifications, wherein the N compressed logs are used for representing target logs in a compressed state;
and S5, decompressing the N compressed logs to obtain the target log.
Optionally, the Binary Search method may be, but not limited to, a Binary Search method, and may be, but not limited to, a Search method with higher efficiency, where conditional Search is performed based on a sequential storage structure, where elements in the sequential storage structure need to be effectively sorted according to keywords, specifically, for example, the elements in the sequential storage structure are arranged in an ascending order, the keywords recorded in the middle position of the sequential storage structure are compared with the Search elements, and if the keywords recorded in the middle position of the sequential storage structure are equal to each other, the Search is successful, otherwise, the sequential storage structure is divided into a front sub-sequential storage structure and a rear sub-sequential storage structure by using a middle position record, and if the keywords recorded in the middle position are greater than the Search keywords, the front sub-sequential storage structure is further searched, and otherwise, the rear sub-sequential storage structure is searched; and repeating the processes until the records meeting the conditions are found, so that the results are found, or until other elements do not exist in the previous sub-sequence storage structure or the next sub-sequence storage structure, and the finding is unsuccessful.
It should be noted that all logs of the target device are obtained, where all logs include a plurality of compressed logs, a compressed log is a log in a compressed state, and the compressed log includes a time identifier, where the time identifier is used to indicate a compression time of each log in all logs; acquiring a plurality of time identifications respectively matched with each of a plurality of compressed logs; determining N target time identifications in the time identifications according to a binary search algorithm, wherein the N target time identifications are used for representing N target times of a compressed target log, the N target times are used for representing a preset time period, and N is an integer greater than or equal to 0; determining N compressed logs matched with the N target time identifications, wherein the N compressed logs are used for representing target logs in a compressed state; and decompressing the N compressed logs to obtain the target log. Optionally, the predetermined time period is a time period for compressing the target log.
Further by way of example, as shown in fig. 9, optionally, the compressed log 902 and the compressed log 904 are included, and a time identifier 906 and a time identifier 908 corresponding to the compressed log 902 and the compressed log 904, where the time identifier 906 and the time identifier 908 are in ascending order, and specifically, a time corresponding to the time identifier 906 is before a time corresponding to the time identifier 908;
for further example, optionally, as shown in fig. 9, if it is determined that the time corresponding to the time identifier 906 can represent a predetermined time period by a binary search method, it is determined that the compressed log 902 corresponding to the time identifier 906 is a compressed log of a target log (not shown in the figure).
According to the embodiment provided by the application, all logs of the target device are obtained, wherein all logs comprise a plurality of compressed logs, the compressed logs are logs in a compressed state, and the compressed logs comprise time marks, and the time marks are used for indicating the compression time of each log in all logs; acquiring a plurality of time identifications respectively matched with each of a plurality of compressed logs; determining N target time identifications in the time identifications according to a binary search algorithm, wherein the N target time identifications are used for representing N target times of a compressed target log, the N target times are used for representing a preset time period, and N is an integer greater than or equal to 0; determining N compressed logs matched with the N target time identifications, wherein the N compressed logs are used for representing target logs in a compressed state; and decompressing the N compressed logs to obtain the target log, so that the aim of selectively decompressing the compressed file of the target log without decompressing all the compressed logs is fulfilled, and the effect of improving the processing efficiency of decompressing the compressed logs is realized.
As an optional scheme, determining N target time identifiers from a plurality of time identifiers according to a binary search algorithm includes:
s1, determining a first time mark in the time marks, wherein the first time mark is used for marking the first time for compressing the first log;
s2, determining a second time mark in the plurality of time marks when the first time is smaller than a first target time, wherein the first target time is used for representing the starting time of a preset time period, the second time mark is used for representing a second time of compressing a second log, and the second time is larger than the first time;
s3, determining a third time mark in the plurality of time marks when the first time is larger than a second target time, wherein the first target time is used for representing the end time of a preset time period, the third time mark is used for representing a third time of compressing a third log, and the third time is smaller than the first time;
and S4, when the first time is smaller than the second target time and the first time is larger than the first target time, determining that the N target times include the first time and determining that the N target time identifications include the first time identification.
It should be noted that a first time identifier is determined among the multiple time identifiers, where the first time identifier is used to identify a first time at which the first log is compressed; under the condition that the first time is smaller than a first target time, determining a second time identifier in the plurality of time identifiers, wherein the first target time is used for representing the starting time of a preset time period, the second time identifier is used for representing the second time of compressing a second log, and the second time is larger than the first time; determining a third time mark in the plurality of time marks under the condition that the first time is larger than a second target time, wherein the first target time is used for representing the end time of a preset time period, the third time mark is used for representing a third time for compressing a third log, and the third time is smaller than the first time; and under the condition that the first time is less than the second target time and the first time is greater than the first target time, determining that the N target times comprise the first time, and determining that the N target time identifications comprise the first time identifications.
Further for example, as shown in fig. 10, optionally, a plurality of compression logs are included, specifically, a compression log 1002, a compression log 1004, a compression log 1006, a compression log 1008, a time identifier 1012, a time identifier 1014, a time identifier 1016, a time identifier 1018, a time identifier 1020 corresponding to the compression log 1002, the compression log 1004, the compression log 1006, the compression log 1008, and the compression log 1010, and a predetermined time period 1026 for representing a compression target log time, where the predetermined time period 1026 includes a first time 1022 for representing a compression inspiration time and a second time 1024 for representing a compression end time;
for further example, optionally, as shown in fig. 10, the time identifier 1016 of the middle position in the plurality of compressed logs is selected, and compared with the first time 1022 and the second time 1024 in the predetermined time period 1026, specifically as follows:
when the time mark 1016 is greater than or equal to the first time 1022 and the time mark 1016 is less than or equal to the second time 1024, determining that the compressed log 1006 corresponding to the time mark 1016 is a part or all of the compressed target log;
in the event that the time of day indicator 1016 is less than the first time of day 1022, continuing to look for a time of day indicator in the previous region 1028 that matches the predetermined time period 1026;
in the event that the time of day identifier 1016 is greater than the second time of day 1024, the search for a time of day identifier matching the predetermined time period 1026 continues in the latter region 1030.
According to the embodiment provided by the application, a first moment identifier is determined in a plurality of moment identifiers, wherein the first moment identifier is used for identifying a first moment for compressing a first log; under the condition that the first time is smaller than a first target time, determining a second time identifier in the plurality of time identifiers, wherein the first target time is used for representing the starting time of a preset time period, the second time identifier is used for representing the second time of compressing a second log, and the second time is larger than the first time; determining a third time mark in the plurality of time marks under the condition that the first time is larger than a second target time, wherein the first target time is used for representing the end time of a preset time period, the third time mark is used for representing a third time for compressing a third log, and the third time is smaller than the first time; under the condition that the first time is less than the second target time and the first time is greater than the first target time, the N target times are determined to comprise the first time, and the N target time identifications are determined to comprise the first time identifications, so that the purpose of selectively decompressing the compressed files of the target logs without decompressing all compressed logs is achieved, and the effect of improving the processing efficiency of decompressing the compressed logs is achieved.
As an optional scheme, a scene embodiment of the method for processing alarm data is provided, which includes the following specific contents:
optionally, as shown in fig. 11, for example, the method includes a query interface 1102, where the query interface 1102 is configured to enter an alarm analysis platform according to query information (e.g., a service name, a software name, etc.) and search an alarm analysis result (e.g., a characteristic ID, a monitoring full name, a responsible person, an access state, etc.), and optionally, the alarm analysis platform is configured to implement a method for processing alarm data;
further, after the optional access of the alarm information, the corresponding alarm information in the service space is matched to add a binding relationship, for example, as shown in fig. 12, the binding relationship includes a query interface 1202, where all alarms, whose alarm contents include p2.xxx.xx.cn, in the service "CDN-quality dial test" and software "single dial test" space displayed on the query interface 1202 are bound to a self-service operation and maintenance platform analysis atom with a code of XXXX;
optionally, for example, as shown in fig. 13, the accessed alarm information may be shielded in the alarm analysis platform 1302 by shielding the alarm 1304, claimed by claiming the alarm 1306, and analyzed by analyzing the alarm 1308;
further, optionally, after analyzing the alarm information, the alarm received on the operation and maintenance terminal may have an identifier for analyzing the alarm, and the content of the analysis of the alarm information is shown in fig. 14, so that the operation and maintenance personnel can check the current network status when the alarm occurs at any time after receiving the alarm information (the content in fig. 14 is merely used as an example), and can make different analysis strategies for different alarm types;
alternatively, for example, as shown in fig. 15, according to the business space 1502, the responsible person may query the generated alarms according to time, so as to analyze and query the historical alarms.
Furthermore, the overall logic of the optional alarm data processing method implemented according to the alarm analysis platform may be, but is not limited to, as shown in fig. 16, and includes the following specific steps:
the alarm analysis platform 1604-1 is responsible for accessing the service monitoring view uploaded by the monitoring platform 1602, and matching and convergence statistics, wherein in the alarm analysis preprocessing stage, the alarm analysis platform 1604-1 screens out the devices with the most reporting points on the curve and then transmits the devices to the designated alarm analysis atom 1606;
the alarm analysis atom 1606 analyzes the alarm and returns the identifier of whether the alarm analysis platform 1602-2 intercepts and the analysis result;
the alarm analysis platform 1604-2 determines whether to report the result to the alarm platform 1608 by intercepting the id after receiving the result.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
According to another aspect of the embodiment of the present invention, there is also provided an alarm data processing apparatus for implementing the alarm data processing method. As shown in fig. 17, the apparatus includes:
a first obtaining unit 1702, configured to obtain alarm indication data reported by each device in a target content distribution network, where the alarm indication data is used to indicate whether the device is in a fault state;
a first determining unit 1704, configured to determine a target device from the target content distribution network according to the alarm indication data, where the target device is a device that reports the alarm indication data most frequently;
a second obtaining unit 1706, configured to obtain a target log of the target device, where the target log is used to record operation data of the target device within a predetermined time period;
a first processing unit 1708, configured to filter the target log according to the first target filtering field to obtain first target fault data;
the first display unit 1710 is configured to perform statistical processing on the first target fault data to obtain a first processing result, and display the first processing result.
Optionally, the processing device of the alarm data may be applied to, but not limited to, a maintenance scene of the CDN network device, and specifically, the processing device of the alarm data may be applied to, but not limited to, automatically acquiring an existing network device that sends an alarm, and performing automatic analysis quickly before the operation and maintenance is notified by a telephone, and determining whether to intercept, thereby reducing disturbance of invalid alarms to the operation and maintenance, helping the operation and maintenance to locate a fault faster, and solving a problem; the alarm data processing device can also but not limited to search the back source type alarm through iteration, so as to achieve the effect of full link analysis; the alarm data processing device can also be used for finding out a group of nodes of different regions and different operators as analysis tools at the same time, and then performing statistical accumulation, so that the obtained data is more real and more accords with the state of the existing network. Optionally, the CDN network may be, but not limited to, an intelligent virtual network constructed on the basis of an existing network, and by means of edge servers deployed in various places and through functional modules of load balancing, content distribution, scheduling, and the like of the central platform, a user can obtain required content nearby, network congestion is reduced, and the access response speed and hit rate of the user are improved.
Optionally, the processing device of the alarm data may enable the CDN operation and maintenance to obtain the access request status and the source return link status of the current network device in the alarm time period only by using the mobile phone without starting a computer, and these pieces of information may reflect the occurrence of the problem, so that the automatic analysis of the alarm greatly accelerates the speed of the operation and maintenance to troubleshoot the problem.
It should be noted that, the alarm indication data reported by each device in the target content distribution network is obtained, where the alarm indication data is used to indicate whether the device is in a fault state; determining target equipment from a target content distribution network according to the alarm indication data, wherein the target equipment is equipment with the most times of reporting the alarm indication data; acquiring a target log of target equipment, wherein the target log is used for recording operation data of the target equipment in a preset time period; filtering the target log according to the first target filtering field to obtain first target fault data; and carrying out statistical processing on the first target fault data to obtain a first processing result, and displaying the first processing result. Optionally, the alarm indication data may be, but is not limited to, shown in a curve view, a table view, and the like, the alarm indication data may be, but is not limited to, summarized by reporting data of all or part of device nodes in the CDN network, and the alarm indication data may be, but is not limited to, used for showing an index change trend of all or part of devices in the CDN network and all or part of reasons for an alarm. The first destination filter field may include, but is not limited to, fields that may be used to filter out invalid data, such as a time of day field, a domain name field, a status field, and the like. The statistical process may be, but not limited to, used to count the fault data that meets the preset policy, for example, count a most dominant client Internet Protocol (IP) in the filtered fault data.
For a specific embodiment, reference may be made to an example shown in the above alarm data processing method, which is not described herein again in this example.
According to the embodiment provided by the application, the alarm indication data reported by each device in the target content distribution network is obtained, wherein the alarm indication data is used for indicating whether the device is in a fault state; determining target equipment from a target content distribution network according to the alarm indication data, wherein the target equipment is equipment with the most times of reporting the alarm indication data; acquiring a target log of target equipment, wherein the target log is used for recording operation data of the target equipment in a preset time period; filtering the target log according to the first target filtering field to obtain first target fault data; the method comprises the steps of carrying out statistical processing on first target fault data to obtain a first processing result, displaying the first processing result, determining target equipment (single-point equipment) with the maximum fault probability in all CDN equipment through fault data reported by all CDN equipment, calling and processing target logs of the target equipment aiming at the target equipment, and filtering a large number of target logs to ensure that processed alarm data have high efficiency, so that the purpose of effectively processing massive alarm data is achieved, and the effect of improving the processing efficiency of the alarm data is achieved.
As an alternative, the first processing unit 1708 includes at least one of:
the first filtering module is used for filtering the target log according to a first filtering field to obtain first fault data in a target preset time period, wherein the first filtering field is used for indicating to filter operation data in a non-target preset time period;
the second filtering module is used for filtering the target log according to a second filtering field to obtain second fault data of the target domain name, wherein the second filtering field is used for indicating to filter the operating data of the non-target domain name;
and the third filtering module is used for filtering the target log according to a third filtering field to obtain third fault data of the target state code, wherein the third filtering field is used for indicating the filtering of the running data of the non-target state code.
For a specific embodiment, reference may be made to an example shown in the above alarm data processing method, which is not described herein again in this example.
As an alternative, as shown in fig. 18, the first display unit 1710 includes:
the counting module 1802 is configured to count target fault data meeting preset conditions, obtain second target fault data of a target type, display the second target fault data, and display target alarm indication data corresponding to the second target fault data, where the preset conditions are used to indicate that the target fault data of the target type is obtained, and the target alarm indication data is used to indicate that the target device is in a target fault state.
For a specific embodiment, reference may be made to an example shown in the above alarm data processing method, which is not described herein again in this example.
As an alternative, the method comprises the following steps:
the second processing unit is used for performing statistical processing on the first target fault data to obtain a first processing result, displaying the first processing result, and filtering the target log according to a second target filtering field to obtain second target fault data;
and the second display unit is used for performing statistical processing on the first target fault data to obtain a first processing result, displaying the first processing result, performing statistical processing on the second target fault data to obtain a second statistical result, and displaying the second statistical result.
For a specific embodiment, reference may be made to an example shown in the above alarm data processing method, which is not described herein again in this example.
As an alternative, as shown in fig. 19, the second obtaining unit 1706 includes:
a writing module 1902, configured to write the running data of the target log into a cache area of a kernel space in a log analysis system, where the log analysis system is configured to analyze the target log;
the mapping module 1904 is configured to map the running data written into the cache region to a mapping region of a process space in the log analysis system according to a target mapping relationship, where the target mapping relationship is a mapping relationship between the cache region and the mapping region that is established in advance.
For a specific embodiment, reference may be made to an example shown in the above alarm data processing method, which is not described herein again in this example.
As an alternative, as shown in fig. 20, the method includes:
a third obtaining unit 2002, configured to obtain all logs of the target device before obtaining a target log of the target device, where all logs include a plurality of compressed logs, a compressed log is a log in a compressed state, and the compressed log includes a time identifier, where the time identifier is used to indicate a compression time of each log in all logs;
a fourth obtaining unit 2004 configured to obtain, before obtaining the target log of the target device, a plurality of time identifiers that respectively match with each of the plurality of compression logs;
a second determining unit 2006, configured to determine, before obtaining a target log of a target device, N target time identifiers in a plurality of time identifiers according to a binary search algorithm, where the N target time identifiers are used to represent N target times of a compressed target log, and the N target times are used to represent a predetermined time period, where N is an integer greater than or equal to 0;
a third determining unit 2008, configured to determine, before obtaining the target log of the target device, N compressed logs that are matched with the N target time identifiers, where the N compressed logs are used to represent the target log in a compressed state;
the decompressing unit 2010 is configured to decompress the N compressed logs to obtain the target log before obtaining the target log of the target device.
For a specific embodiment, reference may be made to an example shown in the above alarm data processing method, which is not described herein again in this example.
As an alternative, the second determining unit 2006 includes:
the first determining module is used for determining a first moment identifier in the plurality of moment identifiers, wherein the first moment identifier is used for identifying a first moment for compressing the first log;
the second determining module is used for determining a second moment identifier in the plurality of moment identifiers under the condition that the first moment is smaller than a first target moment, wherein the first target moment is used for representing the starting moment of a preset time period, the second moment identifier is used for representing the second moment of compressing a second log, and the second moment is larger than the first moment;
a third determining module, configured to determine a third time identifier from the multiple time identifiers when the first time is greater than a second target time, where the first target time is used to indicate an end time of a predetermined time period, the third time identifier is used to indicate a third time for compressing a third log, and the third time is less than the first time;
and the fourth determining module is used for determining that the N target moments comprise the first moments and determining that the N target moment identifications comprise the first moment identifications under the condition that the first moment is smaller than the second target moment and the first moment is larger than the first target moment.
For a specific embodiment, reference may be made to an example shown in the above alarm data processing method, which is not described herein again in this example.
According to another aspect of the embodiments of the present invention, there is also provided an electronic apparatus for implementing the method for processing alarm data, as shown in fig. 21, the electronic apparatus includes a memory 2102 and a processor 2104, the memory 2102 stores a computer program, and the processor 2104 is configured to execute the steps in any one of the method embodiments through the computer program.
Optionally, in this embodiment, the electronic apparatus may be located in at least one network device of a plurality of network devices of a computer network.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, acquiring alarm indication data reported by each device in the target content distribution network, wherein the alarm indication data is used for indicating whether the device is in a fault state;
s2, determining target equipment from the target content distribution network according to the alarm indication data, wherein the target equipment is equipment with the most times of reporting the alarm indication data;
s3, acquiring a target log of the target device, wherein the target log is used for recording the running data of the target device in a preset time period;
s4, filtering the target log according to the first target filtering field to obtain first target fault data;
and S5, performing statistical processing on the first target fault data to obtain a first processing result, and displaying the first processing result.
Alternatively, it can be understood by those skilled in the art that the structure shown in fig. 21 is only an illustration, and the electronic device may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 21 is a diagram illustrating a structure of the electronic device. For example, the electronic device may also include more or fewer components (e.g., network interfaces, etc.) than shown in FIG. 21, or have a different configuration than shown in FIG. 21.
The memory 2102 may be configured to store software programs and modules, such as program instructions/modules corresponding to the method and apparatus for processing alarm data in the embodiment of the present invention, and the processor 2104 executes various functional applications and data processing by running the software programs and modules stored in the memory 2102, so as to implement the above-mentioned method for processing alarm data. The memory 2102 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some instances, the memory 2102 may further include memory located remotely from the processor 2104, which may be connected to a terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 2102 may be specifically, but not limited to, configured to store information such as alarm indication data, a target log, first target failure data, and a first processing result. As an example, as shown in fig. 21, the memory 2102 may include, but is not limited to, a first obtaining unit 1702, a first determining unit 1704, a second obtaining unit 1706, a first processing unit 1708, and a first display unit 1710 in the processing apparatus that includes the alarm data. In addition, the module unit may further include, but is not limited to, other module units in the processing apparatus of the alarm data, which is not described in this example again.
Optionally, the transmission device 2106 is used for receiving or transmitting data via a network. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 2106 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmission device 2106 is a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
In addition, the electronic device further includes: a display 2108 for displaying the alarm indication data, the target log, the first target failure data, the first processing result and other information; and a connection bus 2110 for connecting the respective module parts in the electronic apparatus.
According to a further aspect of an embodiment of the present invention, there is also provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
Alternatively, in the present embodiment, the above-mentioned computer-readable storage medium may be configured to store a computer program for executing the steps of:
s1, acquiring alarm indication data reported by each device in the target content distribution network, wherein the alarm indication data is used for indicating whether the device is in a fault state;
s2, determining target equipment from the target content distribution network according to the alarm indication data, wherein the target equipment is equipment with the most times of reporting the alarm indication data;
s3, acquiring a target log of the target device, wherein the target log is used for recording the running data of the target device in a preset time period;
s4, filtering the target log according to the first target filtering field to obtain first target fault data;
and S5, performing statistical processing on the first target fault data to obtain a first processing result, and displaying the first processing result.
Alternatively, in this embodiment, a person skilled in the art may understand that all or part of the steps in the methods of the foregoing embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be substantially or partially implemented in the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, and including instructions for causing one or more computer devices (which may be personal computers, servers, or network devices) to execute all or part of the steps of the method according to the embodiments of the present invention.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, a division of a unit is merely a division of a logic function, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims (10)

1. A method for processing alarm data is characterized by comprising the following steps:
acquiring alarm indicating data reported by each device in a target content distribution network, wherein the alarm indicating data is used for indicating whether the device is in a fault state or not;
determining target equipment from the target content distribution network according to the alarm indication data, wherein the target equipment is the equipment with the most times of reporting the alarm indication data;
acquiring a target log of the target device, wherein the target log is used for recording the running data of the target device in a preset time period;
filtering the target log according to a first target filtering field to obtain first target fault data;
and carrying out statistical processing on the first target fault data to obtain a first processing result, and displaying the first processing result.
2. The method of claim 1, wherein filtering the target log according to the first target filtering field to obtain the first target fault data comprises at least one of:
filtering the target log according to a first filtering field to obtain first fault data in a target preset time period, wherein the first filtering field is used for indicating that the running data in the target preset time period is not filtered;
filtering the target log according to a second filtering field to obtain second fault data of a target domain name, wherein the second filtering field is used for indicating that the operating data which is not the target domain name is filtered;
and filtering the target log according to a third filtering field to obtain third fault data of the target state code, wherein the third filtering field is used for indicating to filter the running data which is not the target state code.
3. The method of claim 1, wherein the statistically processing the first target fault data to obtain a first processing result, and the displaying the first processing result comprises:
counting the target fault data meeting preset conditions to obtain second target fault data of a target type, displaying the second target fault data, and displaying target alarm indication data corresponding to the second target fault data, wherein the preset conditions are used for indicating to obtain the target fault data of the target type, and the target alarm indication data are used for indicating that the target equipment is in a target fault state.
4. The method according to claim 1, wherein after the statistical processing of the first target fault data to obtain a first processing result and displaying the first processing result, the method comprises:
filtering the target log according to a second target filtering field to obtain second target fault data;
and carrying out statistical processing on the second target fault data to obtain a second statistical result, and displaying the second statistical result.
5. The method of claim 1, wherein obtaining the target log of the target device comprises:
writing the running data of the target log into a cache region of a kernel space in a log analysis system, wherein the log analysis system is used for analyzing the target log;
and mapping the running data written into the cache region to a mapping region of a process space in the log analysis system according to a target mapping relationship, wherein the target mapping relationship is a mapping relationship between the cache region and the mapping region which is established in advance.
6. The method of claim 1, prior to said obtaining a target log for the target device, comprising:
acquiring all logs of the target device, wherein the all logs comprise a plurality of compressed logs, the compressed logs are logs in a compressed state, and the compressed logs comprise time marks, and the time marks are used for representing the compression time of each log in the all logs;
acquiring a plurality of time identifications respectively matched with each of the plurality of compressed logs;
determining N target time identifications in the time identifications according to a binary search algorithm, wherein the N target time identifications are used for representing N target times for compressing the target log, the N target times are used for representing the preset time period, and N is an integer greater than or equal to 0;
determining N compressed logs matched with the N target time identifications, wherein the N compressed logs are used for representing the target logs in the compressed state;
and decompressing the N compressed logs to obtain the target log.
7. The method of claim 6, wherein said determining N target time instances among said plurality of said time instances according to a binary search algorithm comprises
Determining a first time mark in the plurality of time marks, wherein the first time mark is used for marking a first time for compressing a first log;
determining a second time mark in the plurality of time marks under the condition that the first time is smaller than a first target time, wherein the first target time is used for representing the starting time of the preset time period, the second time mark is used for representing a second time of compressing a second log, and the second time is larger than the first time;
determining a third time mark in the plurality of time marks when the first time is larger than a second target time, wherein the first target time is used for representing the end time of the predetermined time period, the third time mark is used for representing a third time for compressing a third log, and the third time is smaller than the first time;
and under the condition that the first time is smaller than the second target time and the first time is larger than the first target time, determining that the N target times comprise the first time, and determining that the N target time identifications comprise the first time identifications.
8. An apparatus for processing alarm data, comprising:
the device comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring alarm indication data reported by each device in a target content distribution network, and the alarm indication data is used for indicating whether the device is in a fault state or not;
a first determining unit, configured to determine a target device from the target content distribution network according to the alarm indication data, where the target device is the device that reports the alarm indication data the most times;
a second obtaining unit, configured to obtain a target log of the target device, where the target log is used to record operation data of the target device within a predetermined time period;
the first processing unit is used for filtering the target log according to a first target filtering field to obtain first target fault data;
and the first display unit is used for carrying out statistical processing on the first target fault data to obtain a first processing result and displaying the first processing result.
9. A computer-readable storage medium, comprising a stored program, wherein the program is operable to perform the method of any one of claims 1 to 7.
10. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method of any of claims 1 to 7 by means of the computer program.
CN202010647837.0A 2020-07-07 2020-07-07 Alarm data processing method and device and storage medium Active CN111740868B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010647837.0A CN111740868B (en) 2020-07-07 2020-07-07 Alarm data processing method and device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010647837.0A CN111740868B (en) 2020-07-07 2020-07-07 Alarm data processing method and device and storage medium

Publications (2)

Publication Number Publication Date
CN111740868A true CN111740868A (en) 2020-10-02
CN111740868B CN111740868B (en) 2023-12-15

Family

ID=72655507

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010647837.0A Active CN111740868B (en) 2020-07-07 2020-07-07 Alarm data processing method and device and storage medium

Country Status (1)

Country Link
CN (1) CN111740868B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112416909A (en) * 2020-12-11 2021-02-26 深圳昂楷科技有限公司 Cloud database auditing method and device and server
CN112559811A (en) * 2020-12-23 2021-03-26 唐山松下产业机器有限公司 Storage method and storage system of welding data
CN112714013A (en) * 2020-12-22 2021-04-27 浪潮云信息技术股份公司 Application fault positioning method in cloud environment
CN113297041A (en) * 2021-05-28 2021-08-24 青岛海尔科技有限公司 Method and apparatus for determining device failure, storage medium, and electronic apparatus
CN114844766A (en) * 2022-03-25 2022-08-02 烽台科技(北京)有限公司 Method and device for building industrial information security guarantee system

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110058806A1 (en) * 2009-09-10 2011-03-10 Tyco Telecommunications (Us) Inc. System and Method for Distributed Fault Sensing and Recovery
JP2015060501A (en) * 2013-09-20 2015-03-30 日本電気株式会社 Alert output device, alert output method and alert output program
CN104796273A (en) * 2014-01-20 2015-07-22 中国移动通信集团山西有限公司 Method and device for diagnosing root of network faults
US20160253254A1 (en) * 2015-02-27 2016-09-01 Commvault Systems, Inc. Diagnosing errors in data storage and archiving in a cloud or networking environment
WO2016183967A1 (en) * 2015-05-19 2016-11-24 中兴通讯股份有限公司 Failure alarm method and apparatus for key component, and big data management system
CN106713049A (en) * 2017-02-04 2017-05-24 杭州迪普科技股份有限公司 Alarm method and device of monitor
WO2017124704A1 (en) * 2016-01-18 2017-07-27 中兴通讯股份有限公司 Method and apparatus for displaying log content
CN107104840A (en) * 2017-05-21 2017-08-29 郑州云海信息技术有限公司 A kind of daily record monitoring method, apparatus and system
CN109951345A (en) * 2019-04-16 2019-06-28 新华三信息安全技术有限公司 A kind of alert processing method and device
CN110149223A (en) * 2019-05-10 2019-08-20 中国联合网络通信集团有限公司 Fault Locating Method and equipment
CN110224858A (en) * 2019-05-16 2019-09-10 平安科技(深圳)有限公司 Alarm method and relevant apparatus based on log
CN110545195A (en) * 2018-05-29 2019-12-06 华为技术有限公司 network fault analysis method and device
WO2019233047A1 (en) * 2018-06-07 2019-12-12 国电南瑞科技股份有限公司 Power grid dispatching-based operation and maintenance method
CN110716832A (en) * 2019-09-24 2020-01-21 腾讯科技(深圳)有限公司 Service operation monitoring and alarming method, system, electronic equipment and storage medium
CN111106944A (en) * 2018-10-26 2020-05-05 中国移动通信有限公司研究院 Fault alarm information processing method and equipment

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110058806A1 (en) * 2009-09-10 2011-03-10 Tyco Telecommunications (Us) Inc. System and Method for Distributed Fault Sensing and Recovery
JP2015060501A (en) * 2013-09-20 2015-03-30 日本電気株式会社 Alert output device, alert output method and alert output program
CN104796273A (en) * 2014-01-20 2015-07-22 中国移动通信集团山西有限公司 Method and device for diagnosing root of network faults
US20160253254A1 (en) * 2015-02-27 2016-09-01 Commvault Systems, Inc. Diagnosing errors in data storage and archiving in a cloud or networking environment
WO2016183967A1 (en) * 2015-05-19 2016-11-24 中兴通讯股份有限公司 Failure alarm method and apparatus for key component, and big data management system
WO2017124704A1 (en) * 2016-01-18 2017-07-27 中兴通讯股份有限公司 Method and apparatus for displaying log content
CN106713049A (en) * 2017-02-04 2017-05-24 杭州迪普科技股份有限公司 Alarm method and device of monitor
CN107104840A (en) * 2017-05-21 2017-08-29 郑州云海信息技术有限公司 A kind of daily record monitoring method, apparatus and system
CN110545195A (en) * 2018-05-29 2019-12-06 华为技术有限公司 network fault analysis method and device
WO2019233047A1 (en) * 2018-06-07 2019-12-12 国电南瑞科技股份有限公司 Power grid dispatching-based operation and maintenance method
CN111106944A (en) * 2018-10-26 2020-05-05 中国移动通信有限公司研究院 Fault alarm information processing method and equipment
CN109951345A (en) * 2019-04-16 2019-06-28 新华三信息安全技术有限公司 A kind of alert processing method and device
CN110149223A (en) * 2019-05-10 2019-08-20 中国联合网络通信集团有限公司 Fault Locating Method and equipment
CN110224858A (en) * 2019-05-16 2019-09-10 平安科技(深圳)有限公司 Alarm method and relevant apparatus based on log
CN110716832A (en) * 2019-09-24 2020-01-21 腾讯科技(深圳)有限公司 Service operation monitoring and alarming method, system, electronic equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
吴凯兴;: "集群通信告警监控系统设计与实现", 信息通信, no. 04 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112416909A (en) * 2020-12-11 2021-02-26 深圳昂楷科技有限公司 Cloud database auditing method and device and server
CN112714013A (en) * 2020-12-22 2021-04-27 浪潮云信息技术股份公司 Application fault positioning method in cloud environment
CN112714013B (en) * 2020-12-22 2023-02-03 浪潮云信息技术股份公司 Application fault positioning method in cloud environment
CN112559811A (en) * 2020-12-23 2021-03-26 唐山松下产业机器有限公司 Storage method and storage system of welding data
CN113297041A (en) * 2021-05-28 2021-08-24 青岛海尔科技有限公司 Method and apparatus for determining device failure, storage medium, and electronic apparatus
CN114844766A (en) * 2022-03-25 2022-08-02 烽台科技(北京)有限公司 Method and device for building industrial information security guarantee system

Also Published As

Publication number Publication date
CN111740868B (en) 2023-12-15

Similar Documents

Publication Publication Date Title
CN111740868B (en) Alarm data processing method and device and storage medium
US11343268B2 (en) Detection of network anomalies based on relationship graphs
CN108460278B (en) Threat information processing method and device
CN110798472B (en) Data leakage detection method and device
US20190222603A1 (en) Method and apparatus for network forensics compression and storage
US8713682B2 (en) Dynamic learning method and adaptive normal behavior profile (NBP) architecture for providing fast protection of enterprise applications
US9569471B2 (en) Asset model import connector
CN109600258B (en) Industrial protocol message recording device and method
US20200372007A1 (en) Trace and span sampling and analysis for instrumented software
CN113489713B (en) Network attack detection method, device, equipment and storage medium
US10565372B1 (en) Subscription-based multi-tenant threat intelligence service
CN105743730A (en) Method and system used for providing real-time monitoring for webpage service of mobile terminal
CN113301012B (en) Network threat detection method and device, electronic equipment and storage medium
CN112040429B (en) Short message management system and method based on distributed storage
US11178160B2 (en) Detecting and mitigating leaked cloud authorization keys
US20150052147A1 (en) System And Method For Analyzing And Reporting Gateway Configurations And Rules
CN104871171A (en) Distributed pattern discovery
CN107885634B (en) Method and device for processing abnormal information in monitoring
CN112306700A (en) Abnormal RPC request diagnosis method and device
CN107332856B (en) Address information detection method and device, storage medium and electronic device
CN111368039B (en) Data management system
CN114969450A (en) User behavior analysis method, device, equipment and storage medium
CN115412359B (en) Web application security protection method and device, electronic equipment and storage medium
CN115604668B (en) Short message sending and pushing monitoring method, device, equipment and storage medium
CN116708024B (en) Threat information collision screening method, gateway system, electronic device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant