CN112131074A - Distributed monitoring method and system - Google Patents
Distributed monitoring method and system Download PDFInfo
- Publication number
- CN112131074A CN112131074A CN202010927750.9A CN202010927750A CN112131074A CN 112131074 A CN112131074 A CN 112131074A CN 202010927750 A CN202010927750 A CN 202010927750A CN 112131074 A CN112131074 A CN 112131074A
- Authority
- CN
- China
- Prior art keywords
- monitoring
- task
- distribution
- forwarder
- processor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 223
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 29
- 230000009545 invasion Effects 0.000 abstract description 2
- 238000002955 isolation Methods 0.000 abstract description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 235000008375 Decussocarpus nagi Nutrition 0.000 description 1
- 244000309456 Decussocarpus nagi Species 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a distributed monitoring method and a system, comprising the following steps: the monitoring method is realized based on a monitoring system, the monitoring system comprises a configuration center and an execution center, and the execution center comprises a distribution transponder and a plurality of processors; the monitoring method comprises the following steps: the distribution forwarder receives monitoring information and monitoring tasks of the monitored nodes; wherein, the monitoring information and the monitoring task are configured by a configuration center; the distribution transponder establishes a corresponding monitoring object and a corresponding monitoring task on the distribution transponder according to the monitoring information so as to establish a corresponding monitoring task in the distribution transponder; the allocation forwarder acquires state parameters of a plurality of processors and selects an idle processor according to the state parameters; the allocation forwarder allocates the monitoring task to the idle processor for processing. The invention has simple deployment, high efficiency of operation, no invasion to the system or application and good isolation; and the system supports various monitoring tasks, supports various alarm information and supports report display by distributing repeater bridging.
Description
Technical Field
The invention belongs to the technical field of telecommunication bank monitoring, and particularly relates to a distributed monitoring method and system.
Background
With the popularization of the telecommunication industry service and the acceleration of the service expansion speed, the service data volume has become an explosive growth mode, so that the difficulty in data processing is increased, the monitoring requirement on the data processing process is higher and higher, and the capability of post analysis is also required, thereby providing a basis for the optimization and maintenance of the program.
At present, there are more sophisticated distributed monitoring systems on the market, such as zabb ix and nagi os, but the distributed monitoring system has the following defects:
1. monitoring products in the market are biased to monitoring resources, such as a CPU, a network and the like;
2. the monitoring products on the market have more technical stack requirements, more dependence and heavier system.
Disclosure of Invention
Aiming at the problem that most of distributed monitoring only monitors one monitoring task, the invention provides a distributed monitoring method and a system, wherein the monitoring method is realized based on a monitoring system, the monitoring system comprises a configuration center and an execution center, and the execution center comprises a distribution transponder and a plurality of processors; the monitoring method comprises the following steps:
the distribution forwarder receives monitoring information and monitoring tasks of the monitored nodes; wherein the monitoring information and the monitoring task are configured by the configuration center;
the distribution forwarder establishes a corresponding monitoring object and a corresponding monitoring task on the distribution forwarder according to the monitoring information so as to establish the corresponding monitoring task in the distribution forwarder;
the distribution forwarder acquires state parameters of a plurality of processors and selects an idle processor according to the state parameters;
and the distribution forwarder distributes the monitoring task to an idle processor for processing.
Preferably, the monitoring system further includes an alarm center, the monitoring method further includes that the configuration center configures alarm rule information, and the alarm center acquires corresponding monitoring data from the monitored node according to the monitoring task processed by the processor, and performs alarm display according to the monitoring data and the alarm rule information.
Preferably, the monitoring system further includes an operation center, and the monitoring method further includes that the operation center obtains corresponding monitoring data from the monitored node according to the monitoring task processed by the processor to perform report display.
Preferably, the step of acquiring the state parameter of each processor by the allocation forwarder and distributing the monitoring task in the allocation forwarder to the idle processor according to the state parameter includes:
the state parameters comprise the current predicted processing completion time, the current number of tasks to be processed and the number of tasks being processed;
calculating the task processing speed of the processor according to the current predicted processing completion time and the number of the tasks being processed;
calculating waiting time according to the number of the current tasks to be processed and the task processing speed;
the processor with the lowest latency is selected as the most idle processor.
Preferably, the kind of the processor includes a specific monitoring executor and a general monitoring executor; wherein the specific monitoring executor is used for processing a certain monitoring task preferentially.
Preferably, the step of the specific monitoring executor for preferentially processing a certain monitoring task is:
a monitoring task queue and other monitoring task queues corresponding to the specific monitoring actuator are arranged in the specific monitoring actuator; wherein the monitoring task queue corresponding to the specific monitoring executor is located before the other monitoring task queues.
An embodiment of the present invention provides a distributed monitoring system, including:
the configuration center is used for configuring the monitoring information, the monitoring task and the alarm rule information of each monitored node and sending the monitoring information, the monitoring task and the alarm rule information to the distribution forwarder;
the execution center comprises a distribution repeater and a plurality of processors, each processor is connected with the distribution repeater, and the distribution repeater establishes a corresponding monitoring object and a corresponding monitoring task on the distribution repeater according to the monitoring information so as to establish the corresponding monitoring task in the distribution repeater; the distribution forwarder acquires state parameters of a plurality of processors and selects an idle processor according to the state parameters; the distribution forwarder distributes the monitoring task to an idle processor for processing;
the alarm center acquires corresponding monitoring data from the monitored node according to the monitoring task processed by the processor and performs alarm display according to the monitoring data and alarm rule information;
and the operation center acquires corresponding monitoring data from the monitored node according to the monitoring task processed by the processor and displays a report.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention has simple deployment, high efficiency of operation, no invasion to the system or application and good isolation;
2. and the system supports various monitoring tasks, supports various alarm information and supports report display by distributing repeater bridging.
Drawings
Fig. 1 is a schematic flow chart of a distributed monitoring method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a distributed monitoring system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should also be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
Fig. 1 is a flow chart of a distributed monitoring method according to the present invention.
Referring to fig. 1, the monitoring method is implemented based on a monitoring system, the monitoring system includes a configuration center and an execution center, the execution center includes a distribution repeater and a plurality of processors; the monitoring method comprises the following steps:
the distribution forwarder receives monitoring information, monitoring tasks and alarm rule information of the monitored nodes; wherein, the monitoring information and the monitoring task are configured by a configuration center;
the monitoring information includes a monitoring ID, a monitoring type, a monitoring command, a monitoring frequency, a monitoring threshold, an execution group I D, an execution type, an alarm group role, a service number, a notification template ID, short message content corresponding to the notification template ID, and the like of each monitored node.
Further, the monitoring tasks are monitoring indexes corresponding to the monitored nodes, and the monitoring tasks include task monitoring, process monitoring and resource monitoring.
Furthermore, the alarm rule information is information for alarming when the monitoring data meets the alarm rule, and the alarm rule information is different for different monitoring tasks.
The distribution transponder establishes a corresponding monitoring object and a corresponding monitoring task on the distribution transponder according to the monitoring information so as to establish a corresponding monitoring task in the distribution transponder;
specifically, the allocation forwarder acquires information such as I D address of the monitored node according to the configuration monitoring information, so that a corresponding monitoring task is established in the allocation forwarder, and the alarm rule is added to the monitoring task. And establishing a monitoring acquisition processing queue in the distribution forwarder, and caching the task to be monitored. The allocation forwarder establishes an independent data acquisition thread to acquire the information to be monitored, so that the situation that the main thread is blocked by the thread possibly generated by monitoring operation or the task processing of the main thread is influenced by the monitoring error is avoided.
The allocation forwarder acquires state parameters of a plurality of processors and selects an idle processor according to the state parameters;
specifically, the state parameters include the current estimated processing completion time, the current number of tasks to be processed, and the number of tasks being processed;
calculating the task processing speed of the processor according to the current predicted processing completion time and the number of the tasks being processed;
calculating waiting time according to the number of the current tasks to be processed and the task processing speed;
the processor with the lowest latency is selected as the most idle processor.
The allocation forwarder allocates the monitoring task to the idle processor for processing.
Through the operation, the monitoring tasks can be distributed to all the processing servers in a balanced manner, so that the impact of a large amount of data operation on the overall performance of the system is avoided, and load balance is achieved. And when the number of tasks to be processed or the total time of the predicted completed tasks of all the processing servers reaches a threshold value, the allocation forwarder can automatically give an alarm, so that operation and maintenance personnel can find the tasks in time conveniently and the problems are solved.
In this embodiment, the monitoring system further includes an alarm center, the monitoring method further includes that the configuration center configures alarm rule information, and the alarm center acquires corresponding monitoring data from the monitored node according to the monitoring task processed by the processor, and performs alarm display according to the monitoring data and the alarm rule information.
Specifically, the alarm center acquires the monitoring data on the corresponding monitoring node, analyzes the monitoring data, judges whether the monitoring data meets the alarm condition according to the alarm rule information, and performs corresponding monitoring operations such as alarm display if the monitoring data meets the alarm condition.
In this embodiment, the monitoring system further includes an operation center, and the monitoring method further includes that the operation center obtains corresponding monitoring data from the monitored node according to the monitoring task processed by the processor, and displays the report.
In this embodiment, the monitoring task includes service monitoring, process monitoring, and resource monitoring for the monitored node. The information configured for different monitoring tasks is different, and the specific information is as follows.
Through the configured information, the distributed forwarders establish different pairs of monitoring tasks through monitoring IDs, and the monitoring tasks have information such as alarm rule information and execution commands, so that different monitoring tasks of the monitored nodes are monitored simultaneously.
In this example, the class of processors may include a specific monitoring executor and a general monitoring executor; wherein a specific monitoring executor is used for processing a certain monitoring task in priority.
Specifically, a monitoring task queue and other monitoring task queues corresponding to the specific monitoring actuator are arranged in the specific monitoring actuator; and the monitoring task queue corresponding to the specific monitoring executor is positioned in front of other monitoring task queues.
An embodiment of the present invention further provides a distributed monitoring system, including:
the configuration center is used for configuring the monitoring information, the monitoring task and the alarm rule information of each monitored node and sending the monitoring information, the monitoring task and the alarm rule information to the distribution forwarder;
the execution center comprises a distribution transponder and a plurality of processors, each processor is connected with the distribution transponder, and the distribution transponder establishes a corresponding monitoring object and a corresponding monitoring task on the distribution transponder according to the monitoring information so as to establish the corresponding monitoring task in the distribution transponder; the allocation forwarder acquires state parameters of a plurality of processors and selects an idle processor according to the state parameters; the allocation forwarder allocates the monitoring task to the idle processor for processing;
the alarm center acquires corresponding monitoring data from the monitored node according to the monitoring task processed by the processor and performs alarm display according to the monitoring data and the alarm rule information;
and the operation center acquires corresponding monitoring data from the monitored node according to the monitoring task processed by the processor and displays the report.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes will occur to those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (7)
1. A distributed monitoring method is realized based on a monitoring system, the monitoring system comprises a configuration center and an execution center, and the execution center comprises a distribution transponder and a plurality of processors; the monitoring method is characterized by comprising the following steps:
the distribution forwarder receives monitoring information and monitoring tasks of the monitored nodes; wherein the monitoring information and the monitoring task are configured by the configuration center;
the distribution forwarder establishes a corresponding monitoring object and a corresponding monitoring task on the distribution forwarder according to the monitoring information so as to establish the corresponding monitoring task in the distribution forwarder;
the distribution forwarder acquires state parameters of a plurality of processors and selects an idle processor according to the state parameters;
and the distribution forwarder distributes the monitoring task to an idle processor for processing.
2. The distributed monitoring method according to claim 1, wherein the monitoring system further includes an alarm center, the monitoring method further includes the configuration center configuring alarm rule information, the alarm center obtaining corresponding monitoring data from the monitored node according to the monitoring task processed by the processor, and performing alarm display according to the monitoring data and the alarm rule information.
3. The distributed monitoring method according to claim 1, wherein the monitoring system further comprises an operation center, and the monitoring method further comprises the operation center obtaining corresponding monitoring data from the monitored node according to the monitoring task processed by the processor to display a report.
4. The distributed monitoring method according to claim 1, wherein the step of acquiring the status parameter of each processor by the allocation forwarder and distributing the monitoring task in the allocation forwarder to the idle processor according to the status parameter comprises:
the state parameters comprise the current predicted processing completion time, the current number of tasks to be processed and the number of tasks being processed;
calculating the task processing speed of the processor according to the current predicted processing completion time and the number of the tasks being processed;
calculating waiting time according to the number of the current tasks to be processed and the task processing speed;
the processor with the lowest latency is selected as the most idle processor.
5. The distributed monitoring method of claim 4, wherein the classes of processors include specific monitoring executors and general monitoring executors; wherein the specific monitoring executor is used for processing a certain monitoring task preferentially.
6. The distributed monitoring method of claim 5, wherein the step of the specific monitoring executor for prioritizing certain of the monitoring tasks is:
a monitoring task queue and other monitoring task queues corresponding to the specific monitoring actuator are arranged in the specific monitoring actuator; wherein the monitoring task queue corresponding to the specific monitoring executor is located before the other monitoring task queues.
7. A distributed monitoring system, comprising:
the configuration center is used for configuring the monitoring information, the monitoring task and the alarm rule information of each monitored node and sending the monitoring information, the monitoring task and the alarm rule information to the distribution forwarder;
the execution center comprises a distribution repeater and a plurality of processors, each processor is connected with the distribution repeater, and the distribution repeater establishes a corresponding monitoring object and a corresponding monitoring task on the distribution repeater according to the monitoring information so as to establish the corresponding monitoring task in the distribution repeater; the distribution forwarder acquires state parameters of a plurality of processors and selects an idle processor according to the state parameters; the distribution forwarder distributes the monitoring task to an idle processor for processing;
the alarm center acquires corresponding monitoring data from the monitored node according to the monitoring task processed by the processor and performs alarm display according to the monitoring data and alarm rule information;
and the operation center acquires corresponding monitoring data from the monitored node according to the monitoring task processed by the processor and displays a report.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010927750.9A CN112131074A (en) | 2020-09-07 | 2020-09-07 | Distributed monitoring method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010927750.9A CN112131074A (en) | 2020-09-07 | 2020-09-07 | Distributed monitoring method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112131074A true CN112131074A (en) | 2020-12-25 |
Family
ID=73848185
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010927750.9A Pending CN112131074A (en) | 2020-09-07 | 2020-09-07 | Distributed monitoring method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112131074A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140379561A1 (en) * | 2013-06-25 | 2014-12-25 | Quisk, Inc. | Fraud monitoring system with distributed cache |
| CN109873858A (en) * | 2018-12-27 | 2019-06-11 | 中科曙光南京研究院有限公司 | A kind of business datum distributed monitoring method and distributed monitoring cluster |
| CN110048888A (en) * | 2019-04-16 | 2019-07-23 | 深圳市致宸信息科技有限公司 | A kind of method based on zabbix monitoring alarm, server, equipment and storage medium |
| CN110297742A (en) * | 2019-07-04 | 2019-10-01 | 北京百佑科技有限公司 | Data monitoring system, method and server |
| CN111476539A (en) * | 2020-04-07 | 2020-07-31 | 深圳市分期乐网络科技有限公司 | Task monitoring method and device, computer equipment and storage medium |
-
2020
- 2020-09-07 CN CN202010927750.9A patent/CN112131074A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140379561A1 (en) * | 2013-06-25 | 2014-12-25 | Quisk, Inc. | Fraud monitoring system with distributed cache |
| CN109873858A (en) * | 2018-12-27 | 2019-06-11 | 中科曙光南京研究院有限公司 | A kind of business datum distributed monitoring method and distributed monitoring cluster |
| CN110048888A (en) * | 2019-04-16 | 2019-07-23 | 深圳市致宸信息科技有限公司 | A kind of method based on zabbix monitoring alarm, server, equipment and storage medium |
| CN110297742A (en) * | 2019-07-04 | 2019-10-01 | 北京百佑科技有限公司 | Data monitoring system, method and server |
| CN111476539A (en) * | 2020-04-07 | 2020-07-31 | 深圳市分期乐网络科技有限公司 | Task monitoring method and device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021008031A1 (en) | Processing method for implementing monitoring intellectualization on the basis of micro-services, and electronic device | |
| EP3072260B1 (en) | Methods, systems, and computer readable media for a network function virtualization information concentrator | |
| CN106656682B (en) | Cluster heartbeat detecting method, system and device | |
| CN111258737B (en) | Resource scheduling method and device and filter scheduler | |
| US7340654B2 (en) | Autonomic monitoring in a grid environment | |
| US20130268678A1 (en) | Method and Apparatus for Facilitating Fulfillment of Requests on a Communication Network | |
| CN109450693B (en) | Hybrid cloud monitoring system and monitoring method using same | |
| CN109271243B (en) | Cluster task management system | |
| CN109245966A (en) | The monitoring method and device of the service state of cloud platform | |
| CN113742066A (en) | Load balancing system and method for server cluster | |
| CN113873006A (en) | Baseline detection method based on priority | |
| CN111901204B (en) | Cloud network inspection method, device and system | |
| CN112671813B (en) | Server determination method, device, equipment and storage medium | |
| CN102404160B (en) | Method and system for realizing intelligent monitoring | |
| CN112333188A (en) | Data service interface, log supervision method, system, storage medium and terminal | |
| CN111209112A (en) | Exception handling method and device | |
| CN114666335A (en) | DDS-based distributed system load balancing device | |
| CN106899659B (en) | Distributed system and management method and management device thereof | |
| CN112131074A (en) | Distributed monitoring method and system | |
| CN112822245A (en) | Agricultural Internet of things big data access system and method | |
| CN105991305B (en) | A kind of method and device identifying link exception | |
| CN114979250A (en) | Message pushing method, device and equipment | |
| CN109286610A (en) | Service blocking apparatus, method, equipment and storage medium | |
| CN110636013B (en) | Dynamic scheduling method and device for message queue | |
| CN113986516A (en) | Distributed task scheduling system based on Hongming system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |