CN112732514A - Zabbix monitoring system based on distributed relational database - Google Patents

Zabbix monitoring system based on distributed relational database Download PDF

Info

Publication number
CN112732514A
CN112732514A CN202011531590.2A CN202011531590A CN112732514A CN 112732514 A CN112732514 A CN 112732514A CN 202011531590 A CN202011531590 A CN 202011531590A CN 112732514 A CN112732514 A CN 112732514A
Authority
CN
China
Prior art keywords
zabbix
server
relational database
monitoring system
distributed relational
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011531590.2A
Other languages
Chinese (zh)
Inventor
王煜
张�浩
白雪珂
王杰斌
张皓
李瑞祥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aisino Corp
Original Assignee
Aisino Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aisino Corp filed Critical Aisino Corp
Priority to CN202011531590.2A priority Critical patent/CN112732514A/en
Publication of CN112732514A publication Critical patent/CN112732514A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3003Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/302Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/0703Error or fault processing not based on redundancy, i.e. by taking additional measures to deal with the error or fault not making use of redundancy in operation, in hardware, or in data representation
    • G06F11/079Root cause analysis, i.e. error or fault diagnosis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/3051Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/32Monitoring with visual or acoustical indication of the functioning of the machine
    • G06F11/324Display of status information
    • G06F11/327Alarm or error message display
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases

Abstract

The invention discloses a distributed relational database-based zabbix monitoring system, which comprises: the system comprises a preset number of zabbix proxy servers, wherein the zabbix proxy servers are used for acquiring target data corresponding to monitored ends; a predetermined number of web ends; a distributed relational database; and the zabbix server is used for receiving the acquired target data of the monitored end, displaying the received data through a corresponding web end, and writing the received data into the distributed relational database. According to the invention, the problem that the existing Zabbix monitoring system cannot give consideration to both the monitoring object data and the monitoring data storage time because the system adopts a single database can be solved.

Description

Zabbix monitoring system based on distributed relational database
Technical Field
The invention belongs to the technical field of network monitoring, and particularly relates to a Zabbix monitoring system based on a distributed relational database.
Background
To know the operation status of the server in real time and to solve the problem in time, the monitoring software is a good choice. Zabbix is enterprise-level open source monitoring software based on a web interface, and a Zabbix server needs an LAMP (linux + apache + Mysql + php) environment or an LNMP (linux + nginx + Mysql + php) environment to provide functions of distributed system monitoring, data collection, network monitoring and the like.
Zabbix is mainly divided into four parts: the Agent is a data acquisition Agent and is deployed on the monitored server; server, Zabbix Server end, used for data collection and alarm sending; web, which is used for configuring data acquisition and alarm rules, etc., front-end data display and API call; the database, official recommendations and traditional usage are Mysql for the storage of configuration information and monitoring data.
The conventional Zabbix monitoring scheme using Mysql as a database mainly has the following problems, which are all caused by the standalone database Mysql: the database Mysql is stand-alone and therefore cannot support data of T magnitude; for this reason, the number of monitoring objects and the data storage time cannot be compatible; zabbix officials have given proxy's solution, want to promote the ability of monitoring and managing the monitored object through proxy, but proxy also will send data to rearmost main Mysql finally, and the pressure of database is in fact not alleviateed.
Disclosure of Invention
The invention aims to solve the problem that the existing Zabbix monitoring system cannot give consideration to both the monitoring object data and the monitoring data storage time because the system adopts a single-machine database.
In order to achieve the purpose, the invention provides a zabbix monitoring system based on a distributed relational database.
The invention discloses a distributed relational database-based zabbix monitoring system, which comprises:
the system comprises a preset number of zabbix proxy servers, wherein the zabbix proxy servers are used for acquiring target data corresponding to monitored ends;
a predetermined number of web ends;
a distributed relational database;
and the zabbix server is used for receiving the acquired target data of the monitored end, displaying the received data through a corresponding web end, and writing the received data into the distributed relational database.
Preferably, the zabbix proxy server is configured in an active mode.
Preferably, the zabbix monitoring system further comprises:
the system comprises a preset number of zabbix server agents, wherein the zabbix server agents are used for receiving target data of a monitored end collected by the corresponding zabbix proxy servers and sending the received data to the zabbix servers.
Preferably, the data received by the zabbix server is from the zabbix server proxy;
alternatively, the data received by the zabbix server comes from the zabbix server proxy and the zabbix proxy server.
Preferably, the zabbix server proxy is configured in an active mode.
Preferably, the history cache entry of the zabbix server is configured as a maximum value.
Preferably, the zabbix server assigns a unique identity to each user.
Preferably, the zabbix monitoring system further comprises:
and the alarm trigger is used for sending alarm information to the corresponding alarm information receiving end when the preset alarm mechanism is triggered.
Preferably, the alarm mode of the alarm trigger comprises WeChat, short message and mail.
Preferably, the alarm trigger is configured in a false alarm prevention mode.
The invention has the beneficial effects that:
according to the Zabbix monitoring system based on the distributed relational database, the stand-alone database of the existing Zabbix monitoring system is replaced by the distributed relational database, and the problem that the existing Zabbix monitoring system cannot give consideration to monitoring object data and monitoring data storage time due to the fact that the stand-alone database is adopted is solved.
Additional features and advantages of the invention will be set forth in the detailed description which follows.
Drawings
The above and other objects, features and advantages of the present invention will become more apparent by describing in more detail exemplary embodiments thereof with reference to the attached drawings, in which like reference numerals generally represent like parts throughout.
Fig. 1 shows an architecture diagram of a distributed relational database based zabbix monitoring system according to an embodiment of the present invention.
Detailed Description
Preferred embodiments of the present invention will be described in more detail below. While the following describes preferred embodiments of the present invention, it should be understood that the present invention may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Example (b): fig. 1 shows an architecture diagram of a distributed relational database-based zabbix monitoring system of the present embodiment. Referring to fig. 1, the zabbix monitoring system based on the distributed relational database of the present embodiment includes:
the four zabbix proxy servers are used for acquiring target data corresponding to the monitored end;
two web ends;
a distributed relational database;
and the zabbix server is used for receiving the acquired target data of the monitored end, displaying the received data through a corresponding web end, and writing the received data into the distributed relational database.
In this embodiment, the zabbix proxy server is configured in an active mode.
The zabbix monitoring system based on the distributed relational database of the embodiment further includes:
the zabbix server agent is used for receiving target data of the monitored end acquired by the corresponding zabbix proxy server and sending the received data to the zabbix server.
In this embodiment, the data received by the zabbix server comes from the zabbix server proxy and the zabbix proxy server.
In this embodiment, the zabbix server proxy is configured in an active mode.
In this embodiment, the history cache entry of the zabbix server is configured as a maximum value.
In this embodiment, the zabbix server allocates a unique identity to each user.
The zabbix monitoring system based on the distributed relational database of the embodiment further includes:
and the alarm trigger is used for sending alarm information to the corresponding alarm information receiving end when the preset alarm mechanism is triggered.
In this embodiment, the alarm mode of the alarm trigger includes WeChat, short message and email.
In this embodiment, the alarm trigger is configured to be in an anti-false alarm mode.
The zabbix monitoring system based on the distributed relational database of the embodiment is described in more detail as follows:
the distributed relational database combines the optimal characteristics of the traditional RDBMS and NoSQL, is compatible with Mysql, supports infinite horizontal extension, and has strong consistency and high availability. From the functionality and characteristics of the distributed relational database itself:
is highly compatible with Mysql. The Mysql can be easily migrated to the distributed relational database used by the method without modifying codes, and the Mysql cluster after being sorted and tabulated can also be migrated in real time through a distributed relational database tool used by the method.
Horizontal elastic expansion. Horizontal expansion of the distributed relational database can be realized by simply adding new nodes, throughput or storage is expanded as required, and high-concurrency and mass data scenes are easily coped with.
A distributed transaction. The distributed relational database used in the method supports standard ACID transactions, and guarantees atomicity, isolation, durability and consistency of data on the basis of a distributed architecture.
The true financial level is highly available. Compared with the traditional master-slave (M-S) replication scheme, the distributed relational database used in the method can provide strong data consistency guarantee of financial level based on the majority election protocol of the Raft, and can realize automatic recovery of faults without manual intervention on the premise of not losing most of replicas.
One-stop HTAP solutions. The distributed relational database used in the method is taken as a typical OLTP row storage database, has strong OLAP performance, can provide a one-stop HTAP solution, and simultaneously processes OLTP and OLAP by one part of storage without the traditional complicated ETL process.
A cloud native SQL database. The distributed relational database used in the method supports public cloud, private cloud and mixed cloud, and automatic operation and maintenance can be realized through the packet manager module, so that deployment, configuration and maintenance become very simple.
The distributed relational database used in the method has no invasion to services, and can elegantly replace fragmented sub-database sub-tables such as traditional database middleware, database sub-tables and the like. Meanwhile, development, operation and maintenance personnel do not need to pay attention to detailed problems such as database expansion and the like, and are concentrated on business development, so that the research and development productivity is greatly improved.
From the aspect of architectural design and using method:
proxy is not used for load balancing in the Zabbix monitoring system immediately after the replacement. Although Zabbix supports the use of proxy on the primary Server to achieve load balancing, its immediate use after replacement adds complexity. Under the condition of a single Zabbix Server, after a distributed relational database is deployed, the performance index of the Zabbix monitoring system is greatly improved, and proxy can be used subsequently along with data increase and service expansion.
By selecting the active mode, the pressure of the Server is reduced. The Agent end of the Zabbix system adopts an active mode, namely Zabbixagent active, which can effectively reduce the pressure of the ZabbixServer. Because in the passive mode, the performance of the Zabbix Server is consumed more, and in the active mode, the load pressure is pushed down to the acquisition end of the Zabbix Agent, and the pressure is very little for a single server; however, if the stack is pressurized to the Server, the Server side has a problem of an excessive load.
The instability of the hardware can be compensated by setting the HistoryCache Size of the configuration item in Zabbix to the maximum value of 2G. Compared with the monitoring data volume which grows exponentially, hardware resources are always limited, which can cause some unstable factors of the distributed structured database, sometimes, data writing is slow, and the data can be backlogged in a cache, and the cache is set to be as large as possible to ensure that the data is not lost to a certain extent. Although data backlog may occur, data may be rewritten as performance is restored, and only a certain delay is generated, but data is not lost.
Different users are identified using multiple Session IDs. Because the permission check in the API calling process needs to frequently check and write the same row data of the same table, serious transaction conflict can be generated, and particularly under optimistic transactions, the serious transaction conflict can bring huge influence on a database. The Session ID is the minimum unit for permission check, and this can be avoided to some extent by multiple read-write lock authentication mechanisms.
And the API is flexibly used. As the amount of data increases, some operations of the Web fail, requiring flexible use of the API. The failure is mainly due to the fact that the database transaction is large and the front end is overtime, so that when the data volume is large, the API needs to be used flexibly.
The zabbix monitoring system based on the distributed relational database of the embodiment realizes a monitoring data staging scheme. The technical use and the architecture design related in the method are a middle-platform scheme used for monitoring data, so that the problem of massive data acquisition, storage and query is solved, a larger amount of historical data monitored for a longer time can be provided, and data analysis, query and positioning of some complex problems are facilitated. The method realizes the calling of rich API, can flexibly call data through the API, and ensures the usability of Web. The distributed structured database is good in lateral expansion capability and extremely high in usability. By increasing the capacity and performance of the database through horizontal expansion, more data can be obtained and the storage time is longer.
The Zabbix monitoring system based on the distributed relational database of the embodiment is used for researching and realizing the problems of the Zabbix monitoring system in the aspects of use, management, performance and the like caused by the fact that monitoring data in the traditional Zabbix monitoring system continuously increase, monitoring items continuously increase, performance bottleneck of a database in the traditional monitoring system architecture and the like, and by means of high performance, flexible elastic expansion, convenient use and operation modes of the Zabbix monitoring system, the storable monitoring data amount of the Zabbix monitoring system is improved, the performance of reading and writing bottom layer data of the Zabbix monitoring system is improved, the usability of the Zabbix monitoring system in the management aspect is improved, and therefore large-scale, high-performance and operation Zabbix monitoring is achieved.
Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

Claims (10)

1. Zabbix monitored control system based on distributed relational database, its characterized in that includes:
the monitoring system comprises a predetermined number of Zabbix proxy servers, a monitoring server and a monitoring server, wherein the Zabbix proxy servers are used for acquiring target data corresponding to a monitored end;
a predetermined number of web ends;
a distributed relational database;
and the Zabbix server is used for receiving the acquired target data of the monitored end, displaying the received data through a corresponding web end and writing the received data into the distributed relational database.
2. The distributed relational database-based Zabbix monitoring system of claim 1, wherein the Zabbix proxy server is configured in an active mode.
3. The distributed relational database-based Zabbix monitoring system according to claim 1, further comprising:
the monitoring system comprises a predetermined number of Zabbix server agents, wherein the Zabbix server agents are used for receiving target data of a monitored end collected by a corresponding Zabbix proxy server and sending the received data to the Zabbix server.
4. The distributed relational database-based Zabbix monitoring system of claim 3, wherein the data received by the Zabbix server is from the Zabbix server agent;
alternatively, the data received by the Zabbix server is from the Zabbix server proxy and the Zabbix proxy server.
5. The distributed relational database-based Zabbix monitoring system of claim 3, wherein the Zabbix server proxy is configured in an active mode.
6. The distributed relational database-based Zabbix monitoring system of claim 1, wherein a history cache entry of the Zabbix server is configured to a maximum value.
7. The distributed relational database-based Zabbix monitoring system of claim 1, wherein the Zabbix server assigns each user a unique identity.
8. The distributed relational database-based Zabbix monitoring system according to claim 1, further comprising:
and the alarm trigger is used for sending alarm information to the corresponding alarm information receiving end when the preset alarm mechanism is triggered.
9. The Zabbix monitoring system based on the distributed relational database according to claim 8, wherein the alarm modes of the alarm trigger comprise WeChat, SMS and email.
10. The distributed relational database-based Zabbix monitoring system of claim 8, wherein the alarm trigger is configured in a false alarm prevention mode.
CN202011531590.2A 2020-12-22 2020-12-22 Zabbix monitoring system based on distributed relational database Pending CN112732514A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011531590.2A CN112732514A (en) 2020-12-22 2020-12-22 Zabbix monitoring system based on distributed relational database

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011531590.2A CN112732514A (en) 2020-12-22 2020-12-22 Zabbix monitoring system based on distributed relational database

Publications (1)

Publication Number Publication Date
CN112732514A true CN112732514A (en) 2021-04-30

Family

ID=75605780

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011531590.2A Pending CN112732514A (en) 2020-12-22 2020-12-22 Zabbix monitoring system based on distributed relational database

Country Status (1)

Country Link
CN (1) CN112732514A (en)

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140201356A1 (en) * 2013-01-16 2014-07-17 Delta Electronics, Inc. Monitoring system of managing cloud-based hosts and monitoring method using for the same
US20170034023A1 (en) * 2015-07-27 2017-02-02 Datagrid Systems, Inc. Techniques for evaluating server system reliability, vulnerability and component compatibility using crowdsourced server and vulnerability data
CN106549829A (en) * 2016-10-28 2017-03-29 北方工业大学 Big data calculating platform monitoring system and method
CN108268342A (en) * 2016-12-30 2018-07-10 上海华讯网络系统有限公司 The storage arithmetic system and method for extensive IT monitoring datas
CN108418710A (en) * 2018-02-09 2018-08-17 北京奇艺世纪科技有限公司 A kind of distributed monitoring system, method and device
CN109471778A (en) * 2018-11-19 2019-03-15 国网安徽省电力有限公司信息通信分公司 A kind of monitoring method of electric system, device and equipment
CN110413485A (en) * 2019-08-02 2019-11-05 上海数讯信息技术有限公司 A kind of one-stop Networked Control and Management System and method for based on Zabbix Open Source Platform
CN110445694A (en) * 2019-09-23 2019-11-12 成都长虹网络科技有限责任公司 A method of trigger notice is monitored based on Zabbix
CN110968470A (en) * 2018-09-28 2020-04-07 江苏赛融科技股份有限公司 Operation and maintenance monitoring and aggregation management system
CN111064609A (en) * 2019-12-11 2020-04-24 北京奇艺世纪科技有限公司 Master-slave switching method and device of message system, electronic equipment and storage medium
WO2020238415A1 (en) * 2019-05-29 2020-12-03 深圳前海微众银行股份有限公司 Method and apparatus for monitoring model training

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140201356A1 (en) * 2013-01-16 2014-07-17 Delta Electronics, Inc. Monitoring system of managing cloud-based hosts and monitoring method using for the same
US20170034023A1 (en) * 2015-07-27 2017-02-02 Datagrid Systems, Inc. Techniques for evaluating server system reliability, vulnerability and component compatibility using crowdsourced server and vulnerability data
CN106549829A (en) * 2016-10-28 2017-03-29 北方工业大学 Big data calculating platform monitoring system and method
CN108268342A (en) * 2016-12-30 2018-07-10 上海华讯网络系统有限公司 The storage arithmetic system and method for extensive IT monitoring datas
CN108418710A (en) * 2018-02-09 2018-08-17 北京奇艺世纪科技有限公司 A kind of distributed monitoring system, method and device
CN110968470A (en) * 2018-09-28 2020-04-07 江苏赛融科技股份有限公司 Operation and maintenance monitoring and aggregation management system
CN109471778A (en) * 2018-11-19 2019-03-15 国网安徽省电力有限公司信息通信分公司 A kind of monitoring method of electric system, device and equipment
WO2020238415A1 (en) * 2019-05-29 2020-12-03 深圳前海微众银行股份有限公司 Method and apparatus for monitoring model training
CN110413485A (en) * 2019-08-02 2019-11-05 上海数讯信息技术有限公司 A kind of one-stop Networked Control and Management System and method for based on Zabbix Open Source Platform
CN110445694A (en) * 2019-09-23 2019-11-12 成都长虹网络科技有限责任公司 A method of trigger notice is monitored based on Zabbix
CN111064609A (en) * 2019-12-11 2020-04-24 北京奇艺世纪科技有限公司 Master-slave switching method and device of message system, electronic equipment and storage medium

Similar Documents

Publication Publication Date Title
US20210279254A1 (en) Data replication method and apparatus, computer device, and storage medium
CN106662983B (en) The methods, devices and systems of data reconstruction in distributed memory system
US7076508B2 (en) Method, system, and program for merging log entries from multiple recovery log files
CN111723160A (en) Multi-source heterogeneous incremental data synchronization method and system
US20120072395A1 (en) System and method for variable block logging with log-ahead buffers
CN102073540A (en) Distributed affair submitting method and device thereof
US20170293632A1 (en) Reliable generation of a unique key in a distributed system
KR20060079078A (en) Server queuing system and method
Spirovska et al. Wren: Nonblocking reads in a partitioned transactional causally consistent data store
US20120278429A1 (en) Cluster system, synchronization controlling method, server, and synchronization controlling program
US10255341B2 (en) Mode switching in high availability disaster recovery (HADR) systems
CN104468274A (en) Cluster monitor and management method and system
CN102708158A (en) PostgreSQL (postgres structured query language) cloud storage filing and scheduling system
CN108255621A (en) A kind of MySQL incremental message analytic methods based on binlog
CN111966677A (en) Data report processing method and device, electronic equipment and storage medium
WO2017181430A1 (en) Method and device for duplicating database in distributed system
CN108090056B (en) Data query method, device and system
CN108833451B (en) Domestic safety control platform-based multistage control system and control method
CN110161930B (en) Data monitoring system and data monitoring method thereof
US9201685B2 (en) Transactional cache versioning and storage in a distributed data grid
CN112732514A (en) Zabbix monitoring system based on distributed relational database
US8230444B2 (en) Global attribute uniqueness (GAU) using an ordered message service (OMS)
CN116186082A (en) Data summarizing method based on distribution, first server and electronic equipment
CN114567633A (en) Cloud platform system supporting full life cycle of multi-stack database and management method
US11296870B2 (en) Key management configurations

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination