CN110865921A - Data monitoring method, apparatus, readable storage medium and electronic device - Google Patents

Data monitoring method, apparatus, readable storage medium and electronic device Download PDF

Info

Publication number
CN110865921A
CN110865921A CN201911086639.5A CN201911086639A CN110865921A CN 110865921 A CN110865921 A CN 110865921A CN 201911086639 A CN201911086639 A CN 201911086639A CN 110865921 A CN110865921 A CN 110865921A
Authority
CN
China
Prior art keywords
data
monitored
warning
rule
feedback information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911086639.5A
Other languages
Chinese (zh)
Inventor
肖政
黄杰
李刚
朱秋烨
夏天亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Lazas Network Technology Shanghai Co Ltd
Original Assignee
Lazas Network Technology Shanghai Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Lazas Network Technology Shanghai Co Ltd filed Critical Lazas Network Technology Shanghai Co Ltd
Priority to CN201911086639.5A priority Critical patent/CN110865921A/en
Publication of CN110865921A publication Critical patent/CN110865921A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/302Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3065Monitoring arrangements determined by the means or processing involved in reporting the monitored data

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Quality & Reliability (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

本发明实施例公开了一种数据监控方法、装置、可读存储介质和电子设备。本发明实施例在获取待监控数据后,将待监控数据与预定规则匹配以确定待监控数据的事件类型,并在待监控数据满足事件类型对应的触发规则时,确定待监控数据对应的警告标签,进而根据待监控数据和警告标签确定警告反馈信息,从而将警告反馈信息分发至待监控数据对应的服务器。本发明实施例的方法使得服务器可以高效、准确地对待监控数据进行自动监控,并在数据产生异常时及时进行报警通知,有效提升了数据异常时警告的准确性和及时性。

Figure 201911086639

The embodiments of the present invention disclose a data monitoring method, an apparatus, a readable storage medium and an electronic device. In this embodiment of the present invention, after acquiring the data to be monitored, the data to be monitored is matched with a predetermined rule to determine the event type of the data to be monitored, and when the data to be monitored satisfies the triggering rule corresponding to the event type, the warning label corresponding to the data to be monitored is determined , and then determine the warning feedback information according to the data to be monitored and the warning label, so as to distribute the warning feedback information to the server corresponding to the data to be monitored. The method of the embodiment of the present invention enables the server to efficiently and accurately monitor the monitoring data automatically, and timely alarm notification when abnormal data occurs, effectively improving the accuracy and timeliness of warning when data is abnormal.

Figure 201911086639

Description

数据监控方法、装置、可读存储介质和电子设备Data monitoring method, apparatus, readable storage medium and electronic device

技术领域technical field

本发明公开涉及计算机技术领域,具体涉及一种数据监控方法、装置、可读存储介质和电子设备。The disclosure of the present invention relates to the field of computer technology, in particular to a data monitoring method, device, readable storage medium and electronic device.

背景技术Background technique

随着计算机技术的不断发展,应用软件的功能和种类变得越来越丰富。应用软件在运行的过程中,需要通过网络对服务进行调用。任一应用软件对服务的至少一次调用均可以看作是一次事件。随着应用软件的功能、种类和服务的数量的不断增长,如何对每个事件进行准确监控,并在事件异常时及时发出警告是亟需解决的问题。With the continuous development of computer technology, the functions and types of application software become more and more abundant. When the application software is running, it needs to call the service through the network. At least one invocation of a service by any application software can be regarded as an event. With the increasing number of functions, types and services of application software, how to accurately monitor each event and issue a warning when an event is abnormal is an urgent problem to be solved.

发明内容SUMMARY OF THE INVENTION

有鉴于此,本发明实施例的目的在于提供一种数据监控方法、装置、可读存储介质和电子设备,用于对应用程序在运行过程中产生的各种事件数据进行准确监控,并在事件数据异常时及时发出警告。In view of this, the purpose of the embodiments of the present invention is to provide a data monitoring method, device, readable storage medium and electronic device for accurately monitoring various event data generated during the running of an application, and Issue a warning in a timely manner when the data is abnormal.

第一方面,本发明实施例提供了一种数据监控方法,所述方法包括:In a first aspect, an embodiment of the present invention provides a data monitoring method, the method comprising:

由具有至少一个处理器的计算机系统获取待监控数据;Obtain the data to be monitored by a computer system having at least one processor;

由所述计算机系统将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;Matching the data to be monitored with a predetermined rule by the computer system to determine the event type of the data to be monitored;

响应于所述待监控数据满足所述事件类型对应的触发规则,由所述计算机系统确定所述待监控数据对应的警告标签;In response to the data to be monitored satisfying the triggering rule corresponding to the event type, the computer system determines the warning label corresponding to the data to be monitored;

由所述计算机系统根据所述待监控数据和所述警告标签确定警告反馈信息;determining warning feedback information by the computer system according to the data to be monitored and the warning label;

由所述计算机系统将所述警告反馈信息分发至所述待监控数据对应的服务器。The computer system distributes the warning feedback information to a server corresponding to the data to be monitored.

优选地,所述获取待监控数据包括:Preferably, the obtaining the data to be monitored includes:

由预定的分布式发布订阅消息系统订阅预定服务的日志;The log of the predetermined service is subscribed by the predetermined distributed publish-subscribe messaging system;

由数据采集层中的数据采集器获取上传至所述分布式发布订阅消息系统的所述日志作为所述待监控数据。The log uploaded to the distributed publish-subscribe messaging system is acquired by a data collector in the data collection layer as the data to be monitored.

优选地,所述将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型包括:Preferably, the matching of the data to be monitored with a predetermined rule, and determining the event type of the data to be monitored includes:

由事件路由层中的事件路由器将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;The event router in the event routing layer matches the to-be-monitored data with a predetermined rule to determine the event type of the to-be-monitored data;

所述方法还包括:The method also includes:

由所述事件路由器将所述待监控数据发送至所述事件类型对应的规则处理节点。The event router sends the to-be-monitored data to a rule processing node corresponding to the event type.

优选地,所述事件路由层中的各所述事件路由器由规则管理器进行管理。Preferably, each of the event routers in the event routing layer is managed by a rule manager.

优选地,所述确定所述待监控数据对应的警告标签包括:Preferably, the determining of the warning label corresponding to the data to be monitored includes:

由所述规则处理节点根据预定的报警规则确定所述警告标签。The warning label is determined by the rule processing node according to a predetermined warning rule.

优选地,所述将所述警告反馈信息分发至所述待监控数据对应的服务器包括:Preferably, the distributing the warning feedback information to the server corresponding to the data to be monitored includes:

由规则处理层中的规则处理节点将所述警告反馈信息发送至预定的消息队列以对所述警告反馈信息进行存放;sending the warning feedback information to a predetermined message queue by the rule processing node in the rule processing layer to store the warning feedback information;

由预定的分布式发布订阅消息系统将所述消息队列中的警告反馈信息分发至对应的所述服务器。The warning feedback information in the message queue is distributed to the corresponding server by a predetermined distributed publish-subscribe message system.

第二方面,本发明实施例提供了一种数据监控装置,所述装置包括:In a second aspect, an embodiment of the present invention provides a data monitoring device, the device comprising:

数据获取单元,用于获取待监控数据;A data acquisition unit for acquiring data to be monitored;

类型确定单元,用于将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;a type determination unit, configured to match the data to be monitored with a predetermined rule, and determine the event type of the data to be monitored;

标签确定单元,用于响应于所述待监控数据满足所述事件类型对应的触发规则,确定所述待监控数据对应的警告标签;A label determination unit, configured to determine a warning label corresponding to the data to be monitored in response to the data to be monitored satisfying the triggering rule corresponding to the event type;

信息确定单元,用于根据所述待监控数据和所述警告标签确定警告反馈信息;an information determination unit, configured to determine warning feedback information according to the data to be monitored and the warning label;

信息分发单元,用于将所述警告反馈信息分发至所述待监控数据对应的服务器。An information distribution unit, configured to distribute the warning feedback information to the server corresponding to the data to be monitored.

第三方面,本发明实施例提供了一种计算机可读存储介质,其上存储计算机程序指令,其中,所述计算机程序指令在被处理器执行时实现如第一方面中任一项所述的方法。In a third aspect, an embodiment of the present invention provides a computer-readable storage medium on which computer program instructions are stored, wherein, when executed by a processor, the computer program instructions implement any one of the first aspects. method.

第四方面,本发明实施例提供了一种电子设备,包括存储器和处理器,其中,所述存储器用于存储一条或多条计算机程序指令,其中,所述一条或多条计算机程序指令被所述处理器执行以实现如下步骤:In a fourth aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are The processor described above executes the following steps:

获取待监控数据;Obtain the data to be monitored;

将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;Matching the data to be monitored with a predetermined rule to determine the event type of the data to be monitored;

响应于所述待监控数据满足所述事件类型对应的触发规则,确定所述待监控数据对应的警告标签;In response to the to-be-monitored data satisfying the triggering rule corresponding to the event type, determining a warning label corresponding to the to-be-monitored data;

根据所述待监控数据和所述警告标签确定警告反馈信息;Determine warning feedback information according to the data to be monitored and the warning label;

将所述警告反馈信息分发至所述待监控数据对应的服务器。Distributing the warning feedback information to the server corresponding to the data to be monitored.

优选地,所述获取待监控数据包括:Preferably, the obtaining the data to be monitored includes:

由预定的分布式发布订阅消息系统订阅预定服务的日志;The log of the predetermined service is subscribed by the predetermined distributed publish-subscribe messaging system;

由数据采集层中的数据采集器获取上传至所述分布式发布订阅消息系统的所述日志作为所述待监控数据。The log uploaded to the distributed publish-subscribe messaging system is acquired by a data collector in the data collection layer as the data to be monitored.

本发明实施例在获取待监控数据后,将待监控数据与预定规则匹配以确定待监控数据的事件类型,并在待监控数据满足事件类型对应的触发规则时,确定待监控数据对应的警告标签,进而根据待监控数据和警告标签确定警告反馈信息,从而将警告反馈信息分发至待监控数据对应的服务器。本发明实施例的方法使得服务器可以高效、准确地对待监控数据进行自动监控,并在数据产生异常时及时进行报警通知,有效提升了数据异常时警告的准确性和及时性。In this embodiment of the present invention, after acquiring the data to be monitored, the data to be monitored is matched with a predetermined rule to determine the event type of the data to be monitored, and when the data to be monitored satisfies the triggering rule corresponding to the event type, a warning label corresponding to the data to be monitored is determined , and then determine the warning feedback information according to the data to be monitored and the warning label, so as to distribute the warning feedback information to the server corresponding to the data to be monitored. The method of the embodiment of the present invention enables the server to efficiently and accurately monitor the monitoring data automatically, and timely alarm notification when abnormal data occurs, effectively improving the accuracy and timeliness of warning when data is abnormal.

附图说明Description of drawings

通过以下参照附图对本发明实施例的描述,本发明的上述以及其它目的、特征和优点将更为清楚,在附图中:The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the present invention with reference to the accompanying drawings, in which:

图1是本发明第一实施例的数据监控方法的流程图;1 is a flowchart of a data monitoring method according to a first embodiment of the present invention;

图2是本发明第一实施例的数据监控方法的数据流程图;Fig. 2 is the data flow chart of the data monitoring method of the first embodiment of the present invention;

图3是本发明第二实施例的数据监控装置的示意图;3 is a schematic diagram of a data monitoring device according to a second embodiment of the present invention;

图4是本发明第三实施例的电子设备的示意图。FIG. 4 is a schematic diagram of an electronic device according to a third embodiment of the present invention.

具体实施方式Detailed ways

以下基于实施例对本发明公开进行描述,但是本发明公开并不仅仅限于这些实施例。在下文对本发明公开的细节描述中,详尽描述了一些特定的细节部分。对本领域技术人员来说没有这些细节部分的描述也可以完全理解本发明公开。为了避免混淆本发明公开的实质,公知的方法、过程、流程、元件和电路并没有详细叙述。The present disclosure is described below based on examples, but the present disclosure is not limited only to these examples. In the following detailed description of the present disclosure, some specific details are described in detail. The present disclosure may be fully understood by those skilled in the art without these detailed descriptions. Well-known methods, procedures, procedures, components and circuits have not been described in detail in order to avoid obscuring the essence of the present disclosure.

此外,本领域普通技术人员应当理解,在此提供的附图都是为了说明的目的,并且附图不一定是按比例绘制的。Furthermore, those of ordinary skill in the art will appreciate that the drawings provided herein are for illustrative purposes and are not necessarily drawn to scale.

除非上下文明确要求,否则整个申请文件中的“包括”、“包含”等类似词语应当解释为包含的含义而不是排他或穷举的含义;也就是说,是“包括但不限于”的含义。Unless clearly required by the context, words such as "including", "comprising" and the like throughout this application should be construed in an inclusive rather than an exclusive or exhaustive sense; that is, in the sense of "including but not limited to".

在本发明公开的描述中,需要理解的是,术语“第一”、“第二”等仅用于描述目的,而不能理解为指示或暗示相对重要性。此外,在本发明公开的描述中,除非另有说明,“多个”的含义是两个或两个以上。In the description of the present disclosure, it should be understood that the terms "first", "second" and the like are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of the present disclosure, unless stated otherwise, "plurality" means two or more.

应用软件在运行的过程中,需要通过网络对服务进行调用。在计算机领域,“服务”为应用程序类型的一种,运行在后台。服务通常可以在本地(也即与用户直接连接的设备)或通过网络为用户提供不同功能,例如终端对文件进行远程上传/下载、联网等。任一应用软件对服务的至少一次调用均可以看作是一次事件。随着应用软件的功能、种类和服务的数量的不断增长,通过人工的方式已无法对全部事件进行有效监控。因此如何对每个事件进行准确监控,并在事件异常时及时发出警告是亟需解决的问题。When the application software is running, it needs to call the service through the network. In the computer world, a "service" is a type of application that runs in the background. Services can generally provide users with different functions locally (that is, devices directly connected to users) or through a network, such as remote uploading/downloading of files by terminals, networking, and the like. At least one invocation of a service by any application software can be regarded as an event. With the increasing number of functions, types and services of application software, it is impossible to effectively monitor all events by manual methods. Therefore, how to accurately monitor each event and issue a warning in time when the event is abnormal is an urgent problem to be solved.

本发明实施例提供的数据监控方法的执行主体包括但不限于服务器。The execution body of the data monitoring method provided by the embodiment of the present invention includes, but is not limited to, a server.

图1是本发明第一实施例的数据监控方法的流程图。如图1所示,本实施例的方法包括如下步骤:FIG. 1 is a flowchart of a data monitoring method according to a first embodiment of the present invention. As shown in Figure 1, the method of this embodiment includes the following steps:

步骤S100,获取待监控数据。Step S100, acquiring data to be monitored.

在本实施例中,待监控数据为在应用软件的运行过程中记录的与服务调用相关的数据,例如数据在通过网络传输过程中的网络传输速度、预定网页的访问频率或访问数量等。In this embodiment, the data to be monitored is the data related to service invocation recorded during the running of the application software, such as the network transmission speed of the data during transmission through the network, the access frequency or the number of accesses to the predetermined web page, and the like.

在一种可选的实现方式中,服务器可以通过预定的分布式发布订阅消息系统订阅预定服务的日志,并由data fetcher(也即,数据采集器)以预定周期或根据实际需求获取上传至分布式发布订阅消息系统的日志作为待监控数据。容易理解,周期长度可以根据实际需求进行设定。在本步骤中,分布式发布订阅消息系统可以为Kafka。In an optional implementation manner, the server may subscribe to the logs of the predetermined service through a predetermined distributed publish-subscribe message system, and the data fetcher (that is, the data collector) obtains and uploads the logs to the distribution at a predetermined period or according to actual needs. The log of the publish-subscribe messaging system is used as the data to be monitored. It is easy to understand that the cycle length can be set according to actual needs. In this step, the distributed publish-subscribe messaging system can be Kafka.

Kafka是一种高吞吐量的分布式发布订阅消息系统,具有高性能、持久化、多副本备份和横向扩展的特点,可以对来自不同数据库(例如,MySQL、ORACLE等)的全部动作流数据、服务调用数据等进行快速整合、处理。在生产(product)-消费(consume)模式下,Kafka可以通过productor接收各应用软件在调用服务产生并上报的数据(也即,预定服务的日志),并通过consumer(或consumer groups)(也即,消费者/组)进行处理。在将数据采集层中的data fetcher(数据采集器)作为Kafka的consumer时,Kafka将创建消费请求、获取响应等操作交给data fetcher完成。由此,当应用软件在运行过程中对服务进行调用时,Kafka可以通过data fetcher自动向存储预定服务的日志的数据库发送数据获取请求,并采集待监控数据。Kafka is a high-throughput distributed publish-subscribe messaging system with high performance, persistence, multi-copy backup and horizontal expansion. Service call data, etc. are quickly integrated and processed. In the production (product)-consume (consume) mode, Kafka can receive the data (that is, the log of the scheduled service) generated and reported by each application software in the calling service through the productor, and pass the consumer (or consumer groups) (that is, , consumer/group) for processing. When the data fetcher (data collector) in the data collection layer is used as the consumer of Kafka, Kafka will create a consumption request, get the response and other operations to the data fetcher to complete. Therefore, when the application software calls the service during the running process, Kafka can automatically send a data acquisition request to the database storing the logs of the predetermined service through the data fetcher, and collect the data to be monitored.

可选地,也可以通过其他方式,例如直接从数据库获取待监控数据,或者通过日志收集系统(例如flume等)、中间件等从数据库获取待监控数据等,本实施例不做限定。Optionally, the data to be monitored may also be obtained in other ways, such as directly obtaining the data to be monitored from the database, or obtaining the data to be monitored from the database through a log collection system (eg, flume, etc.), middleware, etc., which is not limited in this embodiment.

容易理解,待监控数据的数量可以为一个,也可以为多个。It is easy to understand that the number of data to be monitored may be one or multiple.

步骤S200,将待监控数据与预定规则匹配,确定待监控数据的事件类型。In step S200, the data to be monitored is matched with a predetermined rule, and the event type of the data to be monitored is determined.

在一种可选的实现方式中,服务器可以将待监控数据由数据采集层传输至事件路由层,并由事件路由层中对应的event router(事件路由器)将待监控数据与根据实际需求预先设定的预定规则匹配,从而确定待监控数据的事件类型。其中,预定规则可以为待监控数据中的预定字段等,本实施例不做限定。事件可以分为系统事件和用户事件,系统事件由系统激发,例如每经过24小时,系统日期向后更新一天;用户事件为可以被控件识别的操作,例如按下确定按钮、选择某个单选按钮或复选框等。各待监控数据均用于描述与服务调用相关的事件,因此可以根据预定规则匹配确定各待监控数据中的关键字段,从而根据关键字段对应的值确定各待监控数据的事件类型。事件类型可以包括load(加载)、unload(卸载)、abort(中止)、select(选择)等,本实施例不做限定。In an optional implementation manner, the server can transmit the data to be monitored from the data collection layer to the event routing layer, and the corresponding event router (event router) in the event routing layer compares the data to be monitored with preset data according to actual needs. It matches the predetermined rules to determine the event type of the data to be monitored. The predetermined rule may be a predetermined field in the data to be monitored, etc., which is not limited in this embodiment. Events can be divided into system events and user events. System events are triggered by the system. For example, every 24 hours, the system date is updated one day backward. User events are operations that can be recognized by controls, such as pressing the OK button, selecting a radio option. button or checkbox, etc. Each data to be monitored is used to describe events related to service invocation. Therefore, key fields in each data to be monitored can be determined by matching predetermined rules, so as to determine the event type of each data to be monitored according to the values corresponding to the key fields. The event type may include load (loading), unload (unloading), abort (abort), select (selection), etc., which is not limited in this embodiment.

例如,待监控数据描述的事件为“调用服务1产生的加载延迟为200ms”,预定规则包括“load”字段,则该待监控数据对应的事件类型可以为load。For example, if the event described by the data to be monitored is "the loading delay caused by calling service 1 is 200ms", and the predetermined rule includes the "load" field, the event type corresponding to the data to be monitored may be load.

其中,事件路由层中的event router可以由policy management(规则管理器)进行管理。不同时段待监控数据的数量不同,因此为了使得在待监控数据数量较多或较少时各待监控数据均能得到高效处理,可以通过policy management对event router的数量进行管理。可选地,policy management还可以用于对event router进行更新等。Among them, the event router in the event routing layer can be managed by policy management (rule manager). The amount of data to be monitored varies in different periods. Therefore, in order to efficiently process the data to be monitored when the amount of data to be monitored is large or small, the number of event routers can be managed through policy management. Optionally, policy management can also be used to update the event router, etc.

在确定待监控数据的事件类型后,服务器可以由event router将待监控数据发送至各待监控数据的事件类型对应的worker policy(规则处理节点)以对各待监控数据进行报警处理。After determining the event type of the data to be monitored, the server can send the data to be monitored to the worker policy (rule processing node) corresponding to the event type of the data to be monitored by the event router to perform alarm processing on the data to be monitored.

可选地,也可以不通过事件路由层的event router确定待监控数据的事件类型,由服务器直接确定待监控数据的事件类型。容易理解,若不通过事件路由层的eventrouter确定待监控数据的事件类型,则可以无需将待监控数据发送至事件类型对应的worker policy。Optionally, the event type of the data to be monitored may not be determined by the event router of the event routing layer, and the server may directly determine the event type of the data to be monitored. It is easy to understand that if the event type of the data to be monitored is not determined by the eventrouter of the event routing layer, it is not necessary to send the data to be monitored to the worker policy corresponding to the event type.

步骤S300,确定待监控数据对应的警告标签。Step S300, determining a warning label corresponding to the data to be monitored.

具体地,服务器可以在待监控数据满足事件类型对应的触发规则时,确定待监控数据对应的警告标签。其中,触发规则可以根据实际需求确定。例如,在事件类型为load时,对应的触发规则可以为加载延迟超过预定阈值、加载延迟的趋势为递减等;在事件类型为select时,对应的触发规则可以为选择频率超过预定阈值、相邻周期内选择频率的同环比(也即同比与环比)超过预定阈值等。Specifically, the server may determine a warning label corresponding to the data to be monitored when the data to be monitored satisfies the triggering rule corresponding to the event type. The triggering rules can be determined according to actual needs. For example, when the event type is load, the corresponding triggering rule may be that the loading delay exceeds a predetermined threshold, the trend of loading delay is decreasing, etc.; when the event type is select, the corresponding triggering rule may be that the selection frequency exceeds a predetermined threshold, adjacent The year-on-year ratio (ie, the year-on-year to the ring ratio) of the selected frequency in the cycle exceeds a predetermined threshold and the like.

在一种可选的实现方式中,可以通过规则处理层的worker policy根据预定的报警规则确定待监控数据的警告标签。worker policy是worker节点的一种,在本实施例中,可以用于对待监控数据进行触发规则和报警规则的判定。在待监控数据满足事件类型对应的触发规则时,通常表示对应的应用软件的性能产生了问题,因此可以通过worker policy对待监控数据进行规则判定,从而确定各待监控数据对应的应用软件的性能。In an optional implementation manner, the warning label of the data to be monitored may be determined according to a predetermined warning rule through the worker policy of the rule processing layer. The worker policy is a type of worker node, and in this embodiment, it can be used to determine the triggering rule and the alarming rule for the data to be monitored. When the data to be monitored satisfies the triggering rule corresponding to the event type, it usually means that there is a problem with the performance of the corresponding application software. Therefore, the worker policy can be used to determine the rules of the data to be monitored, so as to determine the performance of the application software corresponding to the data to be monitored.

报警规则可以根据实际需求进行设定,例如包括报警通知标签、待监控数据对应的服务器的静默开关标识等。在待监控数据异常时,报警通知标签可以用于对待监控数据的异常类型、等进行报警通知;静默开关标识分为两种情况,在标识为开启时,可以用于对待监控数据的异常类型进行报警通知,在标识为关闭时,可以不对异常数据进行报警通知,并由服务器自动进行异常处理。例如,待监控数据描述的事件为“调用服务1产生的加载延迟为200ms”,对应的事件类型为load,load对应的触发规则包括加载延迟超过100ms,报警规则为静默开关标识为开启、报警通知标签为加载异常的报警通知标签,待监控数据满足load对应的触发规则,可以将警告标签确定为静默开关标识-开启,报警通知标签-加载异常。容易理解,对于不同事件类型的待监控数据,其对应的报警规则也可以不同。The alarm rules can be set according to actual needs, for example, including the alarm notification label, the silent switch identifier of the server corresponding to the data to be monitored, and so on. When the data to be monitored is abnormal, the alarm notification label can be used to notify the abnormal type of the data to be monitored, etc.; the silent switch flag is divided into two cases. Alarm notification, when it is marked as closed, the abnormal data can not be notified of alarm, and the server will automatically handle the abnormality. For example, the event described by the data to be monitored is "The loading delay caused by calling service 1 is 200ms", the corresponding event type is load, the trigger rule corresponding to load includes that the loading delay exceeds 100ms, and the alarm rule is that the silent switch is marked as ON and the alarm notification The label is the alarm notification label with abnormal loading. The data to be monitored satisfies the triggering rule corresponding to the load, and the warning label can be determined as the silent switch flag-on, and the alarm notification label-loading abnormality. It is easy to understand that for data to be monitored of different event types, the corresponding alarm rules may also be different.

可选地,也可以不通过规则处理层的worker policy确定待监控数据对应的警告标签,由服务器直接确定待监控数据对应的警告标签。Optionally, the warning label corresponding to the data to be monitored may be determined without using the worker policy of the rule processing layer, and the server may directly determine the warning label corresponding to the data to be monitored.

步骤S400,根据待监控数据和警告标签确定警告反馈信息。In step S400, the warning feedback information is determined according to the data to be monitored and the warning label.

可选地,在待监控数据存在异常时,服务器可以将待监控数据和对应的用于表征待监控数据存在异常的警告标签确定为警告反馈信息;在待监控数据不存在异常时,服务器可以将待监控数据和对应的用于表征待监控数据不存在异常的警告标签确定为警告反馈信息。Optionally, when the data to be monitored is abnormal, the server may determine the data to be monitored and the corresponding warning label used to represent the abnormality of the data to be monitored as warning feedback information; when there is no abnormality in the data to be monitored, the server may The data to be monitored and the corresponding warning label used to indicate that the data to be monitored is not abnormal are determined as warning feedback information.

可选地,服务器也可以仅在待监控数据存在异常时,将待监控数据和对应的用于表征待监控数据存在异常的警告标签确定为警告反馈信息。Optionally, the server may also determine the data to be monitored and the corresponding warning label used to represent the abnormality of the data to be monitored as the warning feedback information only when the data to be monitored is abnormal.

容易理解,在存在至少一个存在异常和/或不存在异常的待监控数据时,服务器可以分别将各待监控数据和对应的警告标签分别确定为警告反馈信息。It is easy to understand that when there is at least one abnormal data to be monitored and/or no abnormal data to be monitored, the server can respectively determine each data to be monitored and the corresponding warning label as warning feedback information.

步骤S500,将警告反馈信息分发至待监控数据对应的服务器。Step S500: Distribute the warning feedback information to the server corresponding to the data to be monitored.

在一种可选的实现方式中,可以由规则处理层中的worker policy将警告反馈信息发送至预定的消息队列以对警告反馈信息进行存放,并通过预定的分布式发布订阅消息系统将消息队列中的警告反馈信息分发至对应的服务器。在本步骤中,分布式发布订阅消息系统同样可以为Kafka。In an optional implementation manner, the worker policy in the rule processing layer can send the warning feedback information to a predetermined message queue to store the warning feedback information, and the message queue is sent to the message queue through a predetermined distributed publish-subscribe message system. The warning feedback information in is distributed to the corresponding server. In this step, the distributed publish-subscribe message system can also be Kafka.

在发布(publish)-订阅(Subscribe)模式下,Kafka可以将规则处理层中的各worker policy看作发布者,通过不同的消息队列接收以及存放worker policy发布的各消息(在本实施例中,也即警告反馈信息),并将各消息分发至订阅各消息队列的订阅方(在本实施例中,也即待监控数据对应的服务器)。可选地,可以根据待监控数据的事件类型等确定待监控数据对应的服务器。In the publish-subscribe mode, Kafka can regard each worker policy in the rule processing layer as a publisher, and receive and store each message published by the worker policy through different message queues (in this embodiment, That is, the warning feedback information), and each message is distributed to the subscribers who subscribe to each message queue (in this embodiment, that is, the server corresponding to the data to be monitored). Optionally, the server corresponding to the data to be monitored may be determined according to the event type of the data to be monitored.

可选地,还可以通过其他的消息分发系统,例如《陈洁.基于内容分发的信息分发管理系统.软件开发与测评,2008年9月第34卷》中记载的信息分发管理系统对警告反馈信息进行分发。Optionally, other message distribution systems can also be used, such as the information distribution management system recorded in "Chen Jie. Information Distribution Management System Based on Content Distribution. Software Development and Evaluation, Volume 34, September 2008". information is distributed.

在本实施例中,具有数据采集层、事件路由层和规则处理层三层结构的框架可以为Storm。Storm具有低延迟、分布式、可扩展性以及较高的容错性,通过对于各节点的设定,高效、准确地对待监控数据进行自动监控,并在数据产生异常时及时进行报警通知。可选地,具有上述三层结构的框架还可以为其他框架,本实施例不做限定。In this embodiment, a framework having a three-layer structure of a data collection layer, an event routing layer, and a rule processing layer may be Storm. Storm has low latency, distribution, scalability and high fault tolerance. Through the setting of each node, it can automatically monitor the monitoring data efficiently and accurately, and timely alarm notification when data is abnormal. Optionally, the frame having the above three-layer structure may also be another frame, which is not limited in this embodiment.

本实施例在获取待监控数据后,将待监控数据与预定规则匹配以确定待监控数据的事件类型,并在待监控数据满足事件类型对应的触发规则时,确定待监控数据对应的警告标签,进而根据待监控数据和警告标签确定警告反馈信息,从而将警告反馈信息分发至待监控数据对应的服务器。本实施例的方法使得服务器可以高效、准确地对待监控数据进行自动监控,并在数据产生异常时及时进行报警通知,有效提升了数据异常时警告的准确性和及时性。In this embodiment, after acquiring the data to be monitored, the data to be monitored is matched with a predetermined rule to determine the event type of the data to be monitored, and when the data to be monitored satisfies the triggering rule corresponding to the event type, a warning label corresponding to the data to be monitored is determined, Further, the warning feedback information is determined according to the data to be monitored and the warning label, so that the warning feedback information is distributed to the server corresponding to the data to be monitored. The method of this embodiment enables the server to efficiently and accurately perform automatic monitoring on the monitoring data, and timely alarm and notify when the data is abnormal, which effectively improves the accuracy and timeliness of the warning when the data is abnormal.

图2是本发明第一实施例的数据监控方法的数据流程图。如图2所示,结合图1,本实施例的数据流向如下:FIG. 2 is a data flow chart of the data monitoring method according to the first embodiment of the present invention. As shown in FIG. 2 , in conjunction with FIG. 1 , the data flow in this embodiment is as follows:

步骤S100,获取待监控数据。Step S100, acquiring data to be monitored.

被监控的应用软件包括应用软件1-应用软件j(j≥1且为整数),此处以应用软件1为例。应用软件1调用某一服务后,将调用该服务产生的数据进行上报。服务器通过数据采集层的数据采集器(也即,数据采集器11)作为Kafka的consumer接收上报的数据作为待监控数据。The monitored application software includes application software 1-application software j (j≧1 and is an integer), and application software 1 is taken as an example here. After the application software 1 invokes a service, it will report the data generated by invoking the service. The server receives the reported data as the data to be monitored through the data collector of the data collection layer (that is, the data collector 11 ) as the consumer of Kafka.

步骤S200,将待监控数据与预定规则匹配,确定待监控数据的事件类型。In step S200, the data to be monitored is matched with a predetermined rule, and the event type of the data to be monitored is determined.

数据采集器11将应用软件1上传的待监控数据传输到事件路由层,并由事件路由层的事件路由器2m(m≥1且为整数)将待监控数据与预定规则匹配,从而确定待监控数据的事件类型。The data collector 11 transmits the data to be monitored uploaded by the application software 1 to the event routing layer, and the event router 2m (m ≥ 1 and an integer) of the event routing layer matches the data to be monitored with the predetermined rules, thereby determining the data to be monitored. event type.

进一步,在确定待监控数据的事件类型后,事件路由器2m将待监控数据发送至规则处理层,由事件类型对应的规则处理节点(也即,规则处理节点32)接收待监控数据。Further, after determining the event type of the data to be monitored, the event router 2m sends the data to be monitored to the rule processing layer, and the rule processing node corresponding to the event type (ie, the rule processing node 32 ) receives the data to be monitored.

步骤S300,确定待监控数据对应的警告标签。Step S300, determining a warning label corresponding to the data to be monitored.

规则处理层中的规则处理节点31-规则处理节点3k(k≥1且为整数)用于对待监控数据进行触发规则和报警规则的判断。在应用软件1上传的待监控数据满足事件类型对应的触发规则时,规则处理节点32根据报警规则确定待监控数据对应的警告标签。The rule processing node 31 - the rule processing node 3k (k≧1 and an integer) in the rule processing layer are used for judging the trigger rule and the alarm rule for the data to be monitored. When the data to be monitored uploaded by the application software 1 satisfies the triggering rule corresponding to the event type, the rule processing node 32 determines the warning label corresponding to the data to be monitored according to the alarm rule.

步骤S400,根据待监控数据和警告标签确定警告反馈信息。In step S400, the warning feedback information is determined according to the data to be monitored and the warning label.

步骤S500,将警告反馈信息分发至待监控数据对应的服务器。Step S500: Distribute the warning feedback information to the server corresponding to the data to be monitored.

服务器可以通过规则处理层中的worker policy将警告反馈信息发送至预定的消息队列以对警告反馈信息进行存放,并通过预定的分布式发布订阅消息系统将消息队列中的警告反馈信息分发至订阅该消息队列的订阅方,也即待监控数据对应的服务器。The server can send the warning feedback information to the predetermined message queue through the worker policy in the rule processing layer to store the warning feedback information, and distribute the warning feedback information in the message queue to the subscribers through the predetermined distributed publish-subscribe message system. The subscriber of the message queue, that is, the server corresponding to the data to be monitored.

图3是本发明第二实施例的数据监控装置的示意图。如图3所示,本实施例的装置包括数据获取单元301、类型确定单元302、标签确定单元303、信息确定单元304和信息分发单元305。FIG. 3 is a schematic diagram of a data monitoring device according to a second embodiment of the present invention. As shown in FIG. 3 , the apparatus of this embodiment includes a data acquisition unit 301 , a type determination unit 302 , a label determination unit 303 , an information determination unit 304 , and an information distribution unit 305 .

其中,数据获取单元301用于获取待监控数据。类型确定单元302用于将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型。标签确定单元303用于响应于所述待监控数据满足所述事件类型对应的触发规则,确定所述待监控数据对应的警告标签。信息确定单元304用于根据所述待监控数据和所述警告标签确定警告反馈信息。信息分发单元305用于将所述警告反馈信息分发至所述待监控数据对应的服务器。The data acquisition unit 301 is used for acquiring data to be monitored. The type determining unit 302 is configured to match the data to be monitored with a predetermined rule, and determine the event type of the data to be monitored. The label determining unit 303 is configured to determine a warning label corresponding to the data to be monitored in response to the data to be monitored satisfying the triggering rule corresponding to the event type. The information determining unit 304 is configured to determine warning feedback information according to the data to be monitored and the warning label. The information distribution unit 305 is configured to distribute the warning feedback information to the server corresponding to the data to be monitored.

本实施例在获取待监控数据后,将待监控数据与预定规则匹配以确定待监控数据的事件类型,并在待监控数据满足事件类型对应的触发规则时,确定待监控数据对应的警告标签,进而根据待监控数据和警告标签确定警告反馈信息,从而将警告反馈信息分发至待监控数据对应的服务器。本实施例的装置使得服务器可以高效、准确地对待监控数据进行自动监控,并在数据产生异常时及时进行报警通知,有效提升了数据异常时警告的准确性和及时性。In this embodiment, after acquiring the data to be monitored, the data to be monitored is matched with a predetermined rule to determine the event type of the data to be monitored, and when the data to be monitored satisfies the triggering rule corresponding to the event type, a warning label corresponding to the data to be monitored is determined, Further, the warning feedback information is determined according to the data to be monitored and the warning label, so that the warning feedback information is distributed to the server corresponding to the data to be monitored. The device of this embodiment enables the server to efficiently and accurately monitor the monitoring data automatically, and timely alarm notification when abnormal data occurs, effectively improving the accuracy and timeliness of warning when data is abnormal.

图4是本发明第三实施例的电子设备的示意图。如图4所示,该电子设备:至少包括一个处理器401;以及,与至少一个处理器401通信连接的存储器402;以及,与扫描装置通信连接的通信组件403,通信组件403在处理器401的控制下接收和发送数据;其中,存储器402存储有可被至少一个处理器401执行的指令,指令被至少一个处理器401执行以实现:FIG. 4 is a schematic diagram of an electronic device according to a third embodiment of the present invention. As shown in FIG. 4 , the electronic device: includes at least one processor 401 ; and a memory 402 that is communicatively connected to the at least one processor 401 ; Receive and send data under the control of the memory 402; wherein, the memory 402 stores instructions that can be executed by at least one processor 401, and the instructions are executed by at least one processor 401 to achieve:

获取待监控数据;Obtain the data to be monitored;

将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;Matching the data to be monitored with a predetermined rule to determine the event type of the data to be monitored;

响应于所述待监控数据满足所述事件类型对应的触发规则,确定所述待监控数据对应的警告标签;In response to the to-be-monitored data satisfying the triggering rule corresponding to the event type, determining a warning label corresponding to the to-be-monitored data;

根据所述待监控数据和所述警告标签确定警告反馈信息;Determine warning feedback information according to the data to be monitored and the warning label;

将所述警告反馈信息分发至所述待监控数据对应的服务器。Distributing the warning feedback information to the server corresponding to the data to be monitored.

进一步地,所述获取待监控数据包括:Further, the obtaining the data to be monitored includes:

由预定的分布式发布订阅消息系统订阅预定服务的日志;The log of the predetermined service is subscribed by the predetermined distributed publish-subscribe messaging system;

由数据采集层中的数据采集器获取上传至所述分布式发布订阅消息系统的所述日志作为所述待监控数据。The log uploaded to the distributed publish-subscribe messaging system is acquired by a data collector in the data collection layer as the data to be monitored.

进一步地,所述将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型包括:Further, the matching of the data to be monitored with a predetermined rule, and determining the event type of the data to be monitored includes:

由事件路由层中的事件路由器将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;The event router in the event routing layer matches the to-be-monitored data with a predetermined rule to determine the event type of the to-be-monitored data;

所述电子设备还用于执行如下步骤:The electronic device is also used to perform the following steps:

由所述事件路由器将所述待监控数据发送至所述事件类型对应的规则处理节点。The event router sends the to-be-monitored data to a rule processing node corresponding to the event type.

进一步地,所述事件路由层中的各所述事件路由器由规则管理器进行管理。Further, each of the event routers in the event routing layer is managed by a rule manager.

进一步地,所述确定所述待监控数据对应的警告标签包括:Further, the determining of the warning label corresponding to the data to be monitored includes:

由所述规则处理节点根据预定的报警规则确定所述警告标签。The warning label is determined by the rule processing node according to a predetermined warning rule.

进一步地,所述将所述警告反馈信息分发至所述待监控数据对应的服务器包括:Further, the distributing the warning feedback information to the server corresponding to the data to be monitored includes:

由规则处理层中的规则处理节点将所述警告反馈信息发送至预定的消息队列以对所述警告反馈信息进行存放;sending the warning feedback information to a predetermined message queue by the rule processing node in the rule processing layer to store the warning feedback information;

由预定的分布式发布订阅消息系统将所述消息队列中的警告反馈信息分发至对应的所述服务器。The warning feedback information in the message queue is distributed to the corresponding server by a predetermined distributed publish-subscribe message system.

具体地,该电子设备包括:一个或多个处理器401以及存储器402,图4中以一个处理器401为例。处理器401、存储器402可以通过总线或者其他方式连接,图4中以通过总线连接为例。存储器402作为一种非易失性计算机可读存储介质,可用于存储非易失性软件程序、非易失性计算机可执行程序以及模块。处理器401通过运行存储在存储器402中的非易失性软件程序、指令以及模块,从而执行设备的各种功能应用以及数据处理,即实现上述数据监控方法。Specifically, the electronic device includes: one or more processors 401 and a memory 402, and one processor 401 is taken as an example in FIG. 4 . The processor 401 and the memory 402 may be connected through a bus or in other ways, and the connection through a bus is taken as an example in FIG. 4 . As a non-volatile computer-readable storage medium, the memory 402 can be used to store non-volatile software programs, non-volatile computer-executable programs and modules. The processor 401 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory 402, that is, the above-mentioned data monitoring method is implemented.

存储器402可以包括存储程序区和存储数据区,其中,存储程序区可存储操作系统、至少一个功能所需要的应用程序;存储数据区可存储选项列表等。此外,存储器402可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实施例中,存储器402可选包括相对于处理器401远程设置的存储器,这些远程存储器可以通过网络连接至外接设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。The memory 402 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function; the storage data area may store an option list and the like. Additionally, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 402 may optionally include memory located remotely relative to processor 401, which may be connected to external devices via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

一个或者多个模块存储在存储器402中,当被一个或者多个处理器401执行时,执行上述任意方法实施例中的数据监控方法。One or more modules are stored in the memory 402, and when executed by one or more processors 401, perform the data monitoring method in any of the above method embodiments.

上述产品可执行本申请实施例所提供的方法,具备执行方法相应的功能模块和有益效果,未在本实施例中详尽描述的技术细节,可参见本申请实施例所提供的方法。The above product can execute the methods provided by the embodiments of the present application, and have functional modules and beneficial effects corresponding to the execution methods. For technical details not described in detail in the present embodiments, reference may be made to the methods provided by the embodiments of the present application.

本实施例在获取待监控数据后,将待监控数据与预定规则匹配以确定待监控数据的事件类型,并在待监控数据满足事件类型对应的触发规则时,确定待监控数据对应的警告标签,进而根据待监控数据和警告标签确定警告反馈信息,从而将警告反馈信息分发至待监控数据对应的服务器。本实施例使得服务器可以高效、准确地对待监控数据进行自动监控,并在数据产生异常时及时进行报警通知,有效提升了数据异常时警告的准确性和及时性。In this embodiment, after acquiring the data to be monitored, the data to be monitored is matched with a predetermined rule to determine the event type of the data to be monitored, and when the data to be monitored satisfies the triggering rule corresponding to the event type, a warning label corresponding to the data to be monitored is determined, Further, the warning feedback information is determined according to the data to be monitored and the warning label, so that the warning feedback information is distributed to the server corresponding to the data to be monitored. This embodiment enables the server to efficiently and accurately perform automatic monitoring on the monitoring data, and timely alarm notification when abnormal data occurs, effectively improving the accuracy and timeliness of warning when data is abnormal.

本发明的第四实施例涉及一种非易失性存储介质,用于存储计算机可读程序,所述计算机可读程序用于供计算机执行上述部分或全部的方法实施例。The fourth embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, and the computer-readable program is used for a computer to execute some or all of the above method embodiments.

即,本领域技术人员可以理解,实现上述实施例方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成,该程序存储在一个存储介质中,包括若干指令用以使得一个设备(可以是单片机,芯片等)或处理器(processor)执行本申请各实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(ROM,Read-OnlyMemory)、随机存取存储器(RAM,Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。That is, those skilled in the art can understand that all or part of the steps in the method for implementing the above embodiments can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium and includes several instructions to make a device ( It may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: U disk, removable hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes.

本发明实施例公开了A1、一种数据监控方法,所述方法包括:The embodiment of the present invention discloses A1, a data monitoring method, the method includes:

由具有至少一个处理器的计算机系统获取待监控数据;Obtain the data to be monitored by a computer system having at least one processor;

由所述计算机系统将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;Matching the data to be monitored with a predetermined rule by the computer system to determine the event type of the data to be monitored;

响应于所述待监控数据满足所述事件类型对应的触发规则,由所述计算机系统确定所述待监控数据对应的警告标签;In response to the data to be monitored satisfying the triggering rule corresponding to the event type, the computer system determines the warning label corresponding to the data to be monitored;

由所述计算机系统根据所述待监控数据和所述警告标签确定警告反馈信息;determining warning feedback information by the computer system according to the data to be monitored and the warning label;

由所述计算机系统将所述警告反馈信息分发至所述待监控数据对应的服务器。The computer system distributes the warning feedback information to a server corresponding to the data to be monitored.

A2、如A1所述的方法中,所述获取待监控数据包括:A2. In the method according to A1, the obtaining the data to be monitored includes:

由预定的分布式发布订阅消息系统订阅预定服务的日志;The log of the predetermined service is subscribed by the predetermined distributed publish-subscribe messaging system;

由数据采集层中的数据采集器获取上传至所述分布式发布订阅消息系统的所述日志作为所述待监控数据。The log uploaded to the distributed publish-subscribe messaging system is acquired by a data collector in the data collection layer as the data to be monitored.

A3、如A1所述的方法中,所述将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型包括:A3. In the method according to A1, the matching of the data to be monitored with a predetermined rule, and determining the event type of the data to be monitored includes:

由事件路由层中的事件路由器将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;The event router in the event routing layer matches the to-be-monitored data with a predetermined rule to determine the event type of the to-be-monitored data;

所述方法还包括:The method also includes:

由所述事件路由器将所述待监控数据发送至所述事件类型对应的规则处理节点。The event router sends the to-be-monitored data to a rule processing node corresponding to the event type.

A4、如A3所述的方法中,所述事件路由层中的各所述事件路由器由规则管理器进行管理。A4. In the method according to A3, each of the event routers in the event routing layer is managed by a rule manager.

A5、如A3所述的方法中,所述确定所述待监控数据对应的警告标签包括:A5. In the method according to A3, the determining the warning label corresponding to the data to be monitored includes:

由所述规则处理节点根据预定的报警规则确定所述警告标签。The warning label is determined by the rule processing node according to a predetermined warning rule.

A6、如A1所述的方法中,所述将所述警告反馈信息分发至所述待监控数据对应的服务器包括:A6. In the method according to A1, the distributing the warning feedback information to the server corresponding to the data to be monitored includes:

由规则处理层中的规则处理节点将所述警告反馈信息发送至预定的消息队列以对所述警告反馈信息进行存放;sending the warning feedback information to a predetermined message queue by the rule processing node in the rule processing layer to store the warning feedback information;

由预定的分布式发布订阅消息系统将所述消息队列中的警告反馈信息分发至对应的所述服务器。The warning feedback information in the message queue is distributed to the corresponding server by a predetermined distributed publish-subscribe message system.

本发明实施例还公开了B1、一种数据监控装置,所述装置包括:The embodiment of the present invention also discloses B1, a data monitoring device, the device includes:

数据获取单元,用于获取待监控数据;A data acquisition unit for acquiring data to be monitored;

类型确定单元,用于将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;a type determination unit, configured to match the data to be monitored with a predetermined rule, and determine the event type of the data to be monitored;

标签确定单元,用于响应于所述待监控数据满足所述事件类型对应的触发规则,确定所述待监控数据对应的警告标签;A label determination unit, configured to determine a warning label corresponding to the data to be monitored in response to the data to be monitored satisfying the triggering rule corresponding to the event type;

信息确定单元,用于根据所述待监控数据和所述警告标签确定警告反馈信息;an information determination unit, configured to determine warning feedback information according to the data to be monitored and the warning label;

信息分发单元,用于将所述警告反馈信息分发至所述待监控数据对应的服务器。An information distribution unit, configured to distribute the warning feedback information to the server corresponding to the data to be monitored.

本发明实施例还公开了C1、一种计算机可读存储介质,其上存储计算机程序指令,其中,所述计算机程序指令在被处理器执行时实现如A1-A6中任一项所述的方法。The embodiment of the present invention also discloses C1, a computer-readable storage medium on which computer program instructions are stored, wherein the computer program instructions implement the method according to any one of A1-A6 when executed by a processor .

本发明实施例还公开了D1、一种电子设备,包括存储器和处理器,其中,所述存储器用于存储一条或多条计算机程序指令,其中,所述一条或多条计算机程序指令被所述处理器执行以实现如下步骤:The embodiment of the present invention also discloses D1, an electronic device including a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are The processor performs the following steps:

获取待监控数据;Obtain the data to be monitored;

将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;Matching the data to be monitored with a predetermined rule to determine the event type of the data to be monitored;

响应于所述待监控数据满足所述事件类型对应的触发规则,确定所述待监控数据对应的警告标签;In response to the to-be-monitored data satisfying the triggering rule corresponding to the event type, determining a warning label corresponding to the to-be-monitored data;

根据所述待监控数据和所述警告标签确定警告反馈信息;Determine warning feedback information according to the data to be monitored and the warning label;

将所述警告反馈信息分发至所述待监控数据对应的服务器。Distributing the warning feedback information to the server corresponding to the data to be monitored.

D2、如D1所述的电子设备中,所述获取待监控数据包括:D2. In the electronic device according to D1, the acquiring the data to be monitored includes:

由预定的分布式发布订阅消息系统订阅预定服务的日志;The log of the predetermined service is subscribed by the predetermined distributed publish-subscribe messaging system;

由数据采集层中的数据采集器获取上传至所述分布式发布订阅消息系统的所述日志作为所述待监控数据。The log uploaded to the distributed publish-subscribe messaging system is acquired by a data collector in the data collection layer as the data to be monitored.

D3、如D1所述的电子设备中,所述将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型包括:D3. In the electronic device according to D1, the matching of the data to be monitored with a predetermined rule, and determining the event type of the data to be monitored includes:

由事件路由层中的事件路由器将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;The event router in the event routing layer matches the to-be-monitored data with a predetermined rule to determine the event type of the to-be-monitored data;

所述电子设备还用于执行如下步骤:The electronic device is also used to perform the following steps:

由所述事件路由器将所述待监控数据发送至所述事件类型对应的规则处理节点。The event router sends the to-be-monitored data to a rule processing node corresponding to the event type.

D4、如D3所述的电子设备中,所述事件路由层中的各所述事件路由器由规则管理器进行管理。D4. In the electronic device according to D3, each event router in the event routing layer is managed by a rule manager.

D5、如D3所述的电子设备中,所述确定所述待监控数据对应的警告标签包括:D5. In the electronic device according to D3, the determining of the warning label corresponding to the data to be monitored includes:

由所述规则处理节点根据预定的报警规则确定所述警告标签。The warning label is determined by the rule processing node according to a predetermined warning rule.

D6、如D1所述的电子设备中,所述将所述警告反馈信息分发至所述待监控数据对应的服务器包括:D6. In the electronic device according to D1, the distributing the warning feedback information to the server corresponding to the data to be monitored includes:

由规则处理层中的规则处理节点将所述警告反馈信息发送至预定的消息队列以对所述警告反馈信息进行存放;sending the warning feedback information to a predetermined message queue by the rule processing node in the rule processing layer to store the warning feedback information;

由预定的分布式发布订阅消息系统将所述消息队列中的警告反馈信息分发至对应的所述服务器。The warning feedback information in the message queue is distributed to the corresponding server by a predetermined distributed publish-subscribe message system.

本领域的普通技术人员可以理解,上述各实施例是实现本发明的具体实施例,而在实际应用中,可以在形式上和细节上对其作各种改变,而不偏离本发明的精神和范围。Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific embodiments for realizing the present invention, and in practical applications, various changes can be made in form and details without departing from the spirit and the spirit of the present invention. scope.

Claims (10)

1.一种数据监控方法,其特征在于,所述方法包括:1. a data monitoring method, is characterized in that, described method comprises: 由具有至少一个处理器的计算机系统获取待监控数据;Obtain the data to be monitored by a computer system having at least one processor; 由所述计算机系统将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;Matching the data to be monitored with a predetermined rule by the computer system to determine the event type of the data to be monitored; 响应于所述待监控数据满足所述事件类型对应的触发规则,由所述计算机系统确定所述待监控数据对应的警告标签;In response to the data to be monitored satisfying the triggering rule corresponding to the event type, the computer system determines the warning label corresponding to the data to be monitored; 由所述计算机系统根据所述待监控数据和所述警告标签确定警告反馈信息;determining warning feedback information by the computer system according to the data to be monitored and the warning label; 由所述计算机系统将所述警告反馈信息分发至所述待监控数据对应的服务器。The computer system distributes the warning feedback information to a server corresponding to the data to be monitored. 2.根据权利要求1所述的方法,其特征在于,所述获取待监控数据包括:2. The method according to claim 1, wherein the acquiring the data to be monitored comprises: 由预定的分布式发布订阅消息系统订阅预定服务的日志;The log of the predetermined service is subscribed by the predetermined distributed publish-subscribe messaging system; 由数据采集层中的数据采集器获取上传至所述分布式发布订阅消息系统的所述日志作为所述待监控数据。The log uploaded to the distributed publish-subscribe messaging system is acquired by a data collector in the data collection layer as the data to be monitored. 3.根据权利要求1所述的方法,其特征在于,所述将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型包括:3. The method according to claim 1, wherein the matching the data to be monitored with a predetermined rule, and determining the event type of the data to be monitored comprises: 由事件路由层中的事件路由器将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;The event router in the event routing layer matches the to-be-monitored data with a predetermined rule to determine the event type of the to-be-monitored data; 所述方法还包括:The method also includes: 由所述事件路由器将所述待监控数据发送至所述事件类型对应的规则处理节点。The event router sends the to-be-monitored data to a rule processing node corresponding to the event type. 4.根据权利要求3所述的方法,其特征在于,所述事件路由层中的各所述事件路由器由规则管理器进行管理。4. The method according to claim 3, wherein each of the event routers in the event routing layer is managed by a rule manager. 5.根据权利要求3所述的方法,其特征在于,所述确定所述待监控数据对应的警告标签包括:5. The method according to claim 3, wherein the determining the warning label corresponding to the data to be monitored comprises: 由所述规则处理节点根据预定的报警规则确定所述警告标签。The warning label is determined by the rule processing node according to a predetermined warning rule. 6.根据权利要求1所述的方法,其特征在于,所述将所述警告反馈信息分发至所述待监控数据对应的服务器包括:6. The method according to claim 1, wherein the distributing the warning feedback information to the server corresponding to the data to be monitored comprises: 由规则处理层中的规则处理节点将所述警告反馈信息发送至预定的消息队列以对所述警告反馈信息进行存放;sending the warning feedback information to a predetermined message queue by the rule processing node in the rule processing layer to store the warning feedback information; 由预定的分布式发布订阅消息系统将所述消息队列中的警告反馈信息分发至对应的所述服务器。The warning feedback information in the message queue is distributed to the corresponding server by a predetermined distributed publish-subscribe message system. 7.一种数据监控装置,其特征在于,所述装置包括:7. A data monitoring device, wherein the device comprises: 数据获取单元,用于获取待监控数据;A data acquisition unit for acquiring data to be monitored; 类型确定单元,用于将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;a type determination unit, configured to match the data to be monitored with a predetermined rule, and determine the event type of the data to be monitored; 标签确定单元,用于响应于所述待监控数据满足所述事件类型对应的触发规则,确定所述待监控数据对应的警告标签;A label determination unit, configured to determine a warning label corresponding to the data to be monitored in response to the data to be monitored satisfying the triggering rule corresponding to the event type; 信息确定单元,用于根据所述待监控数据和所述警告标签确定警告反馈信息;an information determination unit, configured to determine warning feedback information according to the data to be monitored and the warning label; 信息分发单元,用于将所述警告反馈信息分发至所述待监控数据对应的服务器。An information distribution unit, configured to distribute the warning feedback information to the server corresponding to the data to be monitored. 8.一种计算机可读存储介质,其上存储计算机程序指令,其特征在于,所述计算机程序指令在被处理器执行时实现如权利要求1-6中任一项所述的方法。8. A computer-readable storage medium on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the method according to any one of claims 1-6. 9.一种电子设备,包括存储器和处理器,其特征在于,所述存储器用于存储一条或多条计算机程序指令,其中,所述一条或多条计算机程序指令被所述处理器执行以实现如下步骤:9. An electronic device comprising a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to achieve Follow the steps below: 获取待监控数据;Obtain the data to be monitored; 将所述待监控数据与预定规则匹配,确定所述待监控数据的事件类型;Matching the data to be monitored with a predetermined rule to determine the event type of the data to be monitored; 响应于所述待监控数据满足所述事件类型对应的触发规则,确定所述待监控数据对应的警告标签;In response to the to-be-monitored data satisfying the triggering rule corresponding to the event type, determining a warning label corresponding to the to-be-monitored data; 根据所述待监控数据和所述警告标签确定警告反馈信息;Determine warning feedback information according to the data to be monitored and the warning label; 将所述警告反馈信息分发至所述待监控数据对应的服务器。Distributing the warning feedback information to the server corresponding to the data to be monitored. 10.根据权利要求9所述的电子设备,其特征在于,所述获取待监控数据包括:10. The electronic device according to claim 9, wherein the acquiring the data to be monitored comprises: 由预定的分布式发布订阅消息系统订阅预定服务的日志;The log of the predetermined service is subscribed by the predetermined distributed publish-subscribe messaging system; 由数据采集层中的数据采集器获取上传至所述分布式发布订阅消息系统的所述日志作为所述待监控数据。The log uploaded to the distributed publish-subscribe messaging system is acquired by a data collector in the data collection layer as the data to be monitored.
CN201911086639.5A 2019-11-08 2019-11-08 Data monitoring method, apparatus, readable storage medium and electronic device Pending CN110865921A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911086639.5A CN110865921A (en) 2019-11-08 2019-11-08 Data monitoring method, apparatus, readable storage medium and electronic device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911086639.5A CN110865921A (en) 2019-11-08 2019-11-08 Data monitoring method, apparatus, readable storage medium and electronic device

Publications (1)

Publication Number Publication Date
CN110865921A true CN110865921A (en) 2020-03-06

Family

ID=69653208

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911086639.5A Pending CN110865921A (en) 2019-11-08 2019-11-08 Data monitoring method, apparatus, readable storage medium and electronic device

Country Status (1)

Country Link
CN (1) CN110865921A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111641524A (en) * 2020-05-25 2020-09-08 北京青云科技股份有限公司 Monitoring data processing method, device, equipment and storage medium
CN114327991A (en) * 2021-12-29 2022-04-12 浙江时空道宇科技有限公司 Data processing method and device, electronic equipment and storage medium
CN114461500A (en) * 2022-02-14 2022-05-10 常熟市阳澄湖特种水产有限公司 Data monitoring method and system based on smart fishery
CN115858309A (en) * 2022-12-30 2023-03-28 北京百度网讯科技有限公司 Data monitoring method and device for distributed system and electronic equipment
CN115879104A (en) * 2023-02-14 2023-03-31 北京圣芯诺科技有限公司 Data processing system, electronic device and program product for monitoring data security

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105183609A (en) * 2015-09-16 2015-12-23 焦点科技股份有限公司 Real-time monitoring system and method applied to software system
CN106484595A (en) * 2016-10-09 2017-03-08 华青融天(北京)技术股份有限公司 A kind of event-handling method and device
CN107147639A (en) * 2017-05-08 2017-09-08 国家电网公司 A real-time security early warning method based on complex event processing
CN107547238A (en) * 2016-06-29 2018-01-05 阿里巴巴集团控股有限公司 Event monitoring system, method and device
CN107832200A (en) * 2017-10-24 2018-03-23 平安科技(深圳)有限公司 Alert processing method, device, computer equipment and storage medium
CN108170580A (en) * 2017-11-22 2018-06-15 链家网(北京)科技有限公司 A kind of rule-based log alarming method, apparatus and system
CN108365989A (en) * 2018-01-26 2018-08-03 北京奇虎科技有限公司 Event-handling method and device
CN109144817A (en) * 2018-08-03 2019-01-04 江苏满运软件科技有限公司 A kind of daily record data monitoring system and method
CN109254982A (en) * 2018-08-31 2019-01-22 杭州安恒信息技术股份有限公司 A kind of stream data processing method, system, device and computer readable storage medium
CN109783310A (en) * 2018-12-14 2019-05-21 深圳壹账通智能科技有限公司 The Dynamic and Multi dimensional method for safety monitoring and its monitoring device of information technoloy equipment
CN110362455A (en) * 2019-07-15 2019-10-22 北京奇艺世纪科技有限公司 A kind of data processing method and data processing equipment

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105183609A (en) * 2015-09-16 2015-12-23 焦点科技股份有限公司 Real-time monitoring system and method applied to software system
CN107547238A (en) * 2016-06-29 2018-01-05 阿里巴巴集团控股有限公司 Event monitoring system, method and device
CN106484595A (en) * 2016-10-09 2017-03-08 华青融天(北京)技术股份有限公司 A kind of event-handling method and device
CN107147639A (en) * 2017-05-08 2017-09-08 国家电网公司 A real-time security early warning method based on complex event processing
CN107832200A (en) * 2017-10-24 2018-03-23 平安科技(深圳)有限公司 Alert processing method, device, computer equipment and storage medium
CN108170580A (en) * 2017-11-22 2018-06-15 链家网(北京)科技有限公司 A kind of rule-based log alarming method, apparatus and system
CN108365989A (en) * 2018-01-26 2018-08-03 北京奇虎科技有限公司 Event-handling method and device
CN109144817A (en) * 2018-08-03 2019-01-04 江苏满运软件科技有限公司 A kind of daily record data monitoring system and method
CN109254982A (en) * 2018-08-31 2019-01-22 杭州安恒信息技术股份有限公司 A kind of stream data processing method, system, device and computer readable storage medium
CN109783310A (en) * 2018-12-14 2019-05-21 深圳壹账通智能科技有限公司 The Dynamic and Multi dimensional method for safety monitoring and its monitoring device of information technoloy equipment
CN110362455A (en) * 2019-07-15 2019-10-22 北京奇艺世纪科技有限公司 A kind of data processing method and data processing equipment

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111641524A (en) * 2020-05-25 2020-09-08 北京青云科技股份有限公司 Monitoring data processing method, device, equipment and storage medium
CN114327991A (en) * 2021-12-29 2022-04-12 浙江时空道宇科技有限公司 Data processing method and device, electronic equipment and storage medium
CN114461500A (en) * 2022-02-14 2022-05-10 常熟市阳澄湖特种水产有限公司 Data monitoring method and system based on smart fishery
CN114461500B (en) * 2022-02-14 2024-12-20 常熟市阳澄湖特种水产有限公司 Data monitoring method and system based on smart fishery
CN115858309A (en) * 2022-12-30 2023-03-28 北京百度网讯科技有限公司 Data monitoring method and device for distributed system and electronic equipment
CN115858309B (en) * 2022-12-30 2023-12-22 北京百度网讯科技有限公司 Data monitoring method and device for distributed system and electronic equipment
CN115879104A (en) * 2023-02-14 2023-03-31 北京圣芯诺科技有限公司 Data processing system, electronic device and program product for monitoring data security

Similar Documents

Publication Publication Date Title
CN110865921A (en) Data monitoring method, apparatus, readable storage medium and electronic device
US10904303B2 (en) Control message from streaming source to facilitate scaling
CN109257200B (en) Method and device for monitoring big data platform
US11321139B2 (en) Streaming traffic pattern for public cloud auto scaling
EP2656218A1 (en) Load shedding in a data stream management system
CN106301919A (en) Alarm system of privatized cloud platform and implementation method thereof
CN111124609B (en) Data acquisition method and device, data acquisition equipment and storage medium
CN106970696B (en) Electronic equipment management method and electronic equipment
CN110688277A (en) Data monitoring method and device for micro-service framework
CN111026606A (en) Alarm method and device based on hystrix fuse monitoring and computer equipment
CN112448987A (en) Fusing degradation triggering method and system and storage medium
CN108809720A (en) The management method and device of alarming assignment in cloud data system
CN110275815A (en) A system abnormal alarm processing method and device
CN111130867B (en) Intelligent household equipment alarm method and device based on Internet of things
CN102970162B (en) Processor and method used for processing data to be alarmed
US11057273B2 (en) Decentralized auto-scaling of network architectures
CN111949421B (en) SDK calling method, device, electronic equipment and computer readable storage medium
CN113835905A (en) Message queue load balancing method and device, electronic equipment and medium
CN115033445A (en) A real-time alarm system based on Siddhi
JP7623364B2 (en) Event notification method, system, server device and computer storage medium
WO2015180285A1 (en) Alarm reporting method and device
WO2012102727A1 (en) Distributing information
CN116827748A (en) Configuration information management, update methods, devices, electronic equipment and storage media
CN112749204B (en) Method and device for reading data
CN112437146A (en) Equipment state synchronization method, device and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200306