WO2022134919A1 - Distributed database monitoring method and apparatus, electronic device, and storage medium - Google Patents

Abstract

Embodiments of the present application relate to the technical field of communications. Disclosed are a distributed database monitoring method and apparatus, an electronic device, and a storage medium. The distributed database monitoring method in the present application comprises: delivering a data acquisition instruction to a data forwarding device in a distributed database server cluster, the data acquisition instruction being used for instructing the data forwarding device to intercept and copy communication data of the server cluster; acquiring the communication data copied by the data forwarding device; and monitoring and analyzing a distributed database according to the copied communication data.

Description

Distributed database monitoring method, device, electronic device and storage medium

cross reference

This application is based on the Chinese patent application with the application number "202011537140.4" and the application date is December 23, 2020, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated by reference. Application.

technical field

The embodiments of the present application relate to the field of communication technologies, and in particular, to a distributed database monitoring method, apparatus, electronic device, and storage medium.

Background technique

In recent years, with the development of the Internet industry, user consumption patterns have changed, banking business has grown, and financial-level databases have faced great challenges. Distributed database can well meet the high reliability and high availability requirements of banks for data storage, and is the mainstream direction of financial-level database development in the future. For banks, due to the particularity of their business, strict requirements are placed on data, so the operation and maintenance monitoring of the database is particularly important, and even more so for distributed databases. The financial-level database requires its operation and maintenance monitoring device to continuously monitor the system in real time, and to feedback the operating status of the system in real time, so that the fault can be detected in time before the transaction is issued, and the detailed data can be provided for tracking and positioning after the transaction is issued. problems, so as to ensure the continuous and stable operation of the business and provide customers with safe and reliable services.

However, in order to maximize the performance of the distributed database in the traditional operation and maintenance monitoring device, all monitoring device nodes need to be installed and deployed on different machines in principle, but deploying the acquisition end on each server has high coupling and high intrusion. The writing of the collection-end program is highly dependent on the format of the log and other information, which greatly reduces the reusability, portability, and independence of the program module, and increases the difficulty of code maintenance for program developers.

SUMMARY OF THE INVENTION

The embodiment of the present application provides a distributed database monitoring method, including: issuing a data collection instruction to a data forwarding device in a distributed database server cluster; the data collection instruction is used to instruct the data forwarding device to monitor and copy the communication data of the server cluster ; Obtain the communication data copied by the data forwarding device; monitor and analyze the distributed database according to the copied communication data.

The embodiment of the present application also provides a distributed database monitoring method, including: acquiring a data collection instruction issued by a distributed database monitoring device; monitoring and copying communication data between distributed database server clusters, and forwarding the communication data to a distributed database database monitoring device.

The embodiment of the present application also provides a distributed database monitoring device, including: an instruction issuing module, configured to issue a data collection instruction to a data forwarding device in a distributed database server cluster; the data collection instruction is used to instruct the data forwarding device Monitor and copy the communication data of the server cluster; the data acquisition module is used to obtain the communication data copied by the data forwarding equipment; the monitoring and analysis module is used to monitor and analyze the distributed database according to the copied communication data.

The embodiment of the present application also provides a data forwarding device, including: an instruction acquisition module for acquiring a data acquisition instruction issued by a distributed database monitoring device; a data monitoring module for monitoring and copying between clusters of distributed database servers The communication data is forwarded to the distributed database monitoring device.

Description of drawings

1 is a flowchart of a distributed database monitoring method according to the first embodiment of the present application;

2 is a schematic structural diagram of a distributed database network according to the first embodiment of the present application;

3 is a flowchart of a distributed database monitoring method according to a second embodiment of the present application;

4 is a schematic structural diagram of a distributed database monitoring device according to a third embodiment of the present application;

5 is a schematic structural diagram of a data forwarding device according to a fourth embodiment of the present application;

FIG. 6 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present application.

Detailed ways

In order to make the objectives, technical solutions and advantages of the embodiments of the present application more clear, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that, in each embodiment of the present application, many technical details are provided for readers to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be realized. The following divisions of the various embodiments are for the convenience of description, and should not constitute any limitation on the specific implementation of the present application, and the various embodiments may be combined with each other and referred to each other on the premise of not contradicting each other.

The main purpose of the embodiments of the present application is to provide a distributed database monitoring method, apparatus, electronic device, and storage medium.

The distributed database monitoring method proposed in the present application realizes monitoring of the distributed database by collecting the communication data of each database server node on the data forwarding device to obtain business data, and the monitoring device obtains the communication data collected and copied by the data forwarding device to The business data is obtained by analysis, thereby reducing the intrusion of server nodes, reducing the difficulty of programming the data collection end, and the program has good versatility, reducing the difficulty of code development by operation and maintenance personnel and maintenance of the monitoring system.

The first embodiment of the present application relates to a distributed database monitoring method. The specific process is shown in FIG. 1 , which includes: issuing a data collection instruction to a data forwarding device in a distributed database server cluster; the data collection instruction is used to instruct data forwarding The device monitors and replicates the communication data of the server cluster; obtains the communication data replicated by the data forwarding device; monitors and analyzes the distributed database according to the replicated communication data.

The execution body of the distributed database monitoring method in this embodiment is a distributed database monitoring device, which can be deployed on any terminal or server that is communicatively connected to the data forwarding device. The network structure composed of the distributed database server cluster, the distributed database monitoring device, and the data forwarding device in this embodiment is shown in FIG. 2 .

The implementation details of the distributed database monitoring method in this embodiment will be specifically described below, and the following contents are only provided for the convenience of understanding, and are not necessary for implementing this solution.

The distributed database monitoring method in this embodiment is shown in Figure 1, and specifically includes:

Step 101: Issue a data collection instruction to the data forwarding device in the distributed database server cluster.

Specifically, the distributed database server cluster in this embodiment is a server cluster composed of multiple server nodes, and each server node carries the data of the distributed database. The data forwarding device is in communication connection with each server node, and at the same time, the data forwarding device is in communication connection with the distributed database monitoring device in this embodiment, that is, the monitoring device, receives the data collection instruction issued by the monitoring device and starts to collect the data between each server node. communication data.

Step 102: Obtain the communication data copied by the data forwarding device.

Specifically, the data forwarding device configures its own port in advance, and sets the port connected to each server node as the source port, and the port connected with the monitoring device as the destination port. In this configuration process, the data forwarding device needs to ensure that the bandwidth of the source port and the destination port match, and at the same time, configure only the business of forwarding the copied communication data on the destination port, so as to avoid other data from affecting the forwarding of communication data, or the bandwidth When insufficient, all duplicated packets cannot be forwarded in time. When the data forwarding device receives the data collection instruction from the monitoring device, it starts to monitor the communication data on the source port and forwards the communication data to the monitoring device through the destination port for analysis by the monitoring device after copying the communication data.

Step 103: Monitor and analyze the distributed database according to the copied communication data.

Specifically, the monitoring device receives the communication data forwarded by the data forwarding device through the destination port, and analyzes the communication data to mine service data, so as to monitor, analyze and count the distributed database.

In an example, after the monitoring device obtains the communication data, it parses the communication data according to the packet structure of the communication protocol between each server to obtain the service data, that is, the network information in each data packet in the communication data, such as IP address, MAC address, etc. After removing addresses, etc., the actual service data is obtained, and further according to the service data, that is, the system log of the distributed database and the operation log of each component, the analysis of the log helps to locate and troubleshoot problems, and provide database language optimization suggestions.

In practical applications, the data forwarding device in this embodiment is not only implemented by physical data forwarding devices, such as switches, routers, etc.; it can also be implemented in a virtual local area network (VLAN) or VxLAN, for example, by using several The port forwards mirrored data to a destination port connected to the monitoring device, or forwards mirrored data from several VLANs to a destination port.

Compared with the related art in the field, in this embodiment, the communication data between each server node is copied through port mirroring, which will not affect the normal communication between server nodes; The communication data reduces the coupling and the degree of intrusion of the data between the servers, and at the same time makes the program module of the monitoring device have good versatility, which reduces the difficulty of development and the difficulty of code maintenance.

It should be noted that the above examples in this embodiment are all examples for easy understanding, and do not limit the technical solutions of the present application.

The second embodiment of the present application relates to a distributed database monitoring method. The distributed database monitoring method in this embodiment is applied to a data forwarding device, and the data forwarding device is connected to each server node and the monitoring device in the first embodiment. The process is shown in Figure 3, including:

Step 301: Obtain a data collection instruction issued by a distributed database monitoring device.

Step 302, monitor and copy the communication data in the distributed database cluster, and forward the communication data to the distributed database monitoring device.

Specifically, the data forwarding device configures each port in advance, sets the port connected with each server node as the source port, and sets the port connected with the monitoring device as the destination port. In this configuration process, the data forwarding device needs to ensure that the bandwidth of the source port and the destination port match, and configure only the business of forwarding and replicated communication data on the destination port to avoid other data from affecting the forwarding of communication data. When the data forwarding device receives the data collection instruction from the monitoring device, it starts to monitor the communication data on the source port and copies all the communication data flowing through the source port and forwards it to the monitoring device through the destination port for analysis by the monitoring device.

In an example, the data forwarding device in this embodiment may be a virtual switch or a virtual router. Taking a virtual local area network (VLAN) as an example, the communication data flowing through multiple virtual ports in one VLAN can be mirrored and copied, and then sent to a The destination port connected to the monitoring device performs mirroring data forwarding, or forwards mirroring data from several VLANs to one destination port. The VLAN mentioned in this embodiment may be formed by multiple distributed database virtual nodes carried on one server, or may be formed by multiple distributed database virtual nodes in a cluster formed by multiple servers.

It can be found that this embodiment is a distributed database monitoring method applied to a data forwarding device that is implemented in cooperation with the first embodiment. The relevant implementation details mentioned in the first embodiment can be applied in this embodiment, and the corresponding Yes, the relevant implementation details mentioned in this embodiment can also be applied to the first embodiment.

The steps of the above various methods are divided only for the purpose of describing clearly. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as the same logical relationship is included, they are all within the protection scope of this patent. ;Adding insignificant modifications to the algorithm or process or introducing insignificant designs, but not changing the core design of the algorithm and process are all within the scope of protection of this patent.

The third embodiment of the present application relates to a distributed database monitoring device, and the specific structure is shown in FIG. 4 , including:

The instruction issuing module 401 is configured to issue a data collection instruction to the data forwarding device in the distributed database server cluster; the data collection instruction is used to instruct the data forwarding device to monitor and copy the communication data of the server cluster.

The data acquisition module 402 is configured to acquire the communication data copied by the data forwarding device.

The monitoring and analysis module 403 is configured to monitor and analyze the distributed database according to the replicated communication data.

In one example, the monitoring and analysis module 403 is specifically configured to analyze the communication protocol packet structure of the distributed database, and obtain business data by analyzing the data packets in the copied communication data according to the packet structure.

It is worth mentioning that all modules involved in this implementation are logical modules. In practical applications, a logical unit may be a physical unit, a part of a physical unit, or multiple physical units. combination implementation. In addition, in order to highlight the innovative part of the present application, this embodiment does not introduce units that are not closely related to solving the technical problem raised by the present application, but this does not mean that there are no other units in this embodiment.

The fourth embodiment of the present application relates to a data forwarding device. The specific structure is shown in FIG. 5 , including:

The instruction acquisition module 501 is configured to acquire the data acquisition instruction issued by the distributed database monitoring device.

The data monitoring module 502 is used for monitoring and copying the communication data between the distributed database server clusters, and forwarding the communication data to the distributed database monitoring device.

In one example, the data monitoring module 502 is specifically configured to monitor the communication data of the server cluster through the source port; and forward the replicated communication data to the distributed database monitoring apparatus through the destination port.

It is worth mentioning that all modules involved in this implementation are logical modules. In practical applications, a logical unit may be a physical unit, a part of a physical unit, or multiple physical units. combination implementation. In addition, in order to highlight the innovative part of the present application, this embodiment does not introduce units that are not closely related to solving the technical problem raised by the present application, but this does not mean that there are no other units in this embodiment.

It can be found that the third embodiment and the fourth embodiment of the present application are respectively virtual device embodiments that cooperate with the first embodiment and the second embodiment in real time, and the related implementations mentioned in the first and second embodiments The details can be applied to the third and fourth embodiments, respectively, and similarly, the relevant implementation details mentioned in the third and fourth embodiments can be applied to the first and second embodiments, respectively.

The fifth embodiment of the present application relates to a server, as shown in FIG. 6 , comprising: at least one processor 601; and a memory 602 connected in communication with the at least one processor 601; Instructions executed by the processor 601, the instructions are executed by at least one processor 601, so that the at least one processor 601 can execute the distributed database monitoring method in the first or second embodiment. The memory 602 and the processor 601 are connected by a bus, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more processors 601 and various circuits of the memory 602 together. The bus may also connect together various other circuits, such as peripherals, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides the interface between the bus and the transceiver. A transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a means for communicating with various other devices over a transmission medium. The data processed by the processor 601 is transmitted on the wireless medium through the antenna, and further, the antenna also receives the data and transmits the data to the processor 601 . Processor 601 is responsible for managing the bus and general processing, and may also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. The memory 602 may be used to store data used by the processor 601 when performing operations.

The sixth embodiment of the present application relates to a computer-readable storage medium storing a computer program. The above method embodiments are implemented when the computer program is executed by the processor. That is, those skilled in the art can understand that all or part of the steps in the method for implementing the above embodiments can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium and includes several instructions to make a device ( It may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, removable hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific embodiments for realizing the present application, and in practical applications, various changes in form and details can be made without departing from the spirit and the spirit of the present application. scope.

Claims (10)

1. A distributed database monitoring method, comprising:

   Issue a data collection instruction to the data forwarding equipment in the distributed database server cluster; the data collection instruction is used to instruct the data forwarding equipment to monitor and copy the communication data of the server cluster;

   obtaining the communication data copied by the data forwarding device;

   The distributed database is monitored and analyzed according to the replicated communication data.
2. The distributed database monitoring method according to claim 1, wherein the performing database monitoring and analysis according to the communication data comprises:

   Analyzing the copied communication data to obtain business data;

   The distributed database is monitored and analyzed according to the business data.
3. The distributed database monitoring method according to claim 2, wherein before the analyzing the copied communication data to obtain the business data, the method further comprises:

   Analyze the communication protocol packet structure of the distributed database;

   The analyzing the copied communication data to obtain business data, including:

   Business data is obtained by analyzing the data packets in the copied communication data according to the packet structure.
4. The distributed database monitoring method according to any one of claims 1 to 3, wherein the data forwarding device comprises: a switch and/or a routing device.
5. A distributed database monitoring method, comprising:

   Obtain the data collection instruction issued by the distributed database monitoring device;

   Monitor and replicate the communication data in the distributed database cluster, and forward the communication data to the distributed database monitoring device.
6. The distributed database monitoring method according to claim 5, wherein the communication data between the replication server clusters comprises:

   Monitor the communication data of the server cluster through the source port;

   The replicated communication data is forwarded to the distributed database monitoring device through the destination port.
7. A distributed database monitoring device, comprising:

   an instruction issuing module, configured to issue a data collection instruction to the data forwarding equipment in the distributed database server cluster; the data collection instruction is used to instruct the data forwarding equipment to monitor and copy the communication data of the server cluster;

   a data acquisition module for acquiring the communication data copied by the data forwarding device;

   The monitoring and analysis module is used for monitoring and analyzing the distributed database according to the copied communication data.
8. A data forwarding device, comprising:

   an instruction acquisition module, used for acquiring the data acquisition instruction issued by the distributed database monitoring device;

   The data monitoring module is used for monitoring and copying the communication data between the distributed database server clusters, and forwarding the communication data to the distributed database monitoring device.
9. An electronic device comprising:

   at least one processor; and,

   a memory communicatively coupled to the at least one processor; wherein,

   The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the execution of any one of claims 1 to 4 The distributed database monitoring method described above, or the distributed database monitoring method according to claim 5 or 6.
10. A computer-readable storage medium storing a computer program, when the computer program is executed by a processor, the distributed database monitoring method according to any one of claims 1 to 4 is realized, or, as claimed in claim 5 or 6 The distributed database monitoring method.

Images