CN108696400A - Network monitoring method and device - Google Patents

Abstract

The application discloses a network monitoring method and device. A specific implementation of the method includes: receiving a network monitoring request sent by a user for execution by a specified proxy server group; obtaining status information of each proxy server in the proxy server group, the status information including: CPU utilization, memory utilization and execution Quantity and location information of the monitoring operation; According to CPU utilization rate, memory utilization rate and the quantity of the monitoring operation of execution, select predetermined number of proxy servers from the proxy server group, and determine the geographic location of the selected proxy server; Send a request to the selected proxy server to perform a network monitoring operation; receive the monitoring result returned by the selected proxy server, and determine whether the network in the geographical location is abnormal according to the monitoring result and the preset network quality index. This implementation mode realizes all-round real-time monitoring of network quality.

Description

Network monitoring method and device

technical field

The present application relates to the field of computer technology, specifically to the field of Internet technology, especially to a network monitoring method and device.

Background technique

In this information age, any online data transmission is inseparable from the network. With the rapid development of cloud computing technology, many industries in traditional fields are increasingly relying on cloud computing services provided by Internet companies. Therefore, the stability of the network especially important. However, due to many force majeure or human factors, the network cannot always be in a stable state. Therefore, real-time monitoring of the network quality of the Internet, finding out that the network is unstable and switching services in time has become an urgent problem to be solved.

In the prior art, the following two methods are generally adopted to monitor the network quality of the Internet in real time. One method is to select several servers in the core computer room of the business, and reversely monitor the network quality from the computer room to operators in various provinces and cities. This method can only monitor the network quality from the core computer room to the operators in various provinces and cities, and the network quality from the operators in various provinces and cities across the country to the core computer room is unknown. Another method is to rent a third-party keynote network monitoring to realize real-time monitoring of some services. Renting a third-party keynote network monitoring cannot conduct targeted monitoring according to its own business needs, and it cannot simulate real users' access to services.

Contents of the invention

The purpose of this application is to propose an improved network monitoring method and device to solve the technical problems mentioned in the above background technology section.

In the first aspect, the embodiment of the present application provides a network monitoring method, the method includes: receiving a network monitoring request sent by a user for execution by a specified proxy server group; obtaining status information of each proxy server in the proxy server group, the status information includes : CPU utilization rate, memory utilization rate and the number and location information of the monitoring operations performed; according to the CPU utilization rate, memory utilization rate and the number of monitoring operations performed, select a predetermined number of proxy servers from the proxy server group, and determine the The geographical location of the selected proxy server; sending a request to the selected proxy server to perform network monitoring operations; receiving the monitoring result returned by the selected proxy server, and determining whether the network in the geographical location is based on the monitoring result and the preset network quality index abnormal.

In some embodiments, the network monitoring operation includes at least one of the following: the operation of monitoring the transmission delay and/or packet loss rate of the data packet, the operation of monitoring the return status of the HTTP request, the operation of monitoring whether domain name hijacking occurs, and monitoring the network Whether the routing jump information is abnormal.

In some embodiments, the request to execute the network monitoring operation includes an execution mode identifier, and the execution mode identifier is used to instruct the selected proxy server to execute the network monitoring operation in parallel/serially.

In some embodiments, selecting a predetermined number of proxy servers from the proxy server group according to the CPU utilization rate, the memory utilization rate, and the number of monitoring operations performed, including: selecting the proxy server group from the proxy server group in ascending order of the CPU utilization rate Select a predetermined number of proxy servers from among; and/or select a predetermined number of proxy servers from the proxy server group in descending order of memory utilization; and/or select a predetermined number of proxy servers in descending order of the number of monitoring operations performed A predetermined number of proxy servers are selected from the proxy server group.

In some embodiments, the method further includes: buffering the monitoring results in a message queue, and creating different topics in the message queue for different types of network monitoring operations.

In some embodiments, the method further includes: in response to receiving the network monitoring request including the target location area, aggregating the monitoring results in the message queue to obtain the aggregated monitoring results, and posting the aggregated monitoring results to In the data persistence cluster; determine whether the network in the target location area is abnormal according to the aggregated monitoring results and the preset network quality indicators.

In some embodiments, the request for performing the network monitoring operation includes at least one of the following: a start time for instructing the selected proxy server to start performing the network monitoring operation, an end time for instructing the end of performing the network monitoring operation, and an end time for instructing the selected proxy server to start performing the network monitoring operation. The execution period of the network monitoring operation is periodically executed; and the method further includes: storing the start time, the end time and the execution period in the database.

In some embodiments, the method further includes: sending an update request to the proxy server, so that the proxy server performs a grayscale upgrade; receiving update information fed back by the proxy server and storing it in the database.

In some embodiments, the method further includes: storing the status information sent by the proxy server in the database; and/or storing the user information and the network monitoring request of the user in the database.

In some embodiments, sending a request to the selected proxy server to perform a network monitoring operation includes: in response to receiving a monitoring operation request periodically sent by the selected proxy server, sending a request to the selected proxy server to perform a network monitoring operation ask.

In the second aspect, the embodiment of the present application provides a network monitoring device, which includes: a receiving unit, configured to receive a network monitoring request sent by a user to be executed by a designated proxy server group; an obtaining unit, configured to obtain each The status information of a proxy server, the status information includes: CPU utilization, memory utilization and the number and location information of the monitoring operations performed; the selection unit is used for according to the CPU utilization, memory utilization and the number of monitoring operations performed, Select a predetermined number of proxy servers from the proxy server group, and determine the geographic location of the selected proxy server; the sending unit is used to send a request to the selected proxy server to perform network monitoring operations; the determination unit is used to receive the selected proxy server The monitoring results returned by the proxy server, and determine whether the network in the geographical location is abnormal according to the monitoring results and the preset network quality indicators.

In some embodiments, the network monitoring operation includes at least one of the following: the operation of monitoring the transmission delay and/or packet loss rate of the data packet, the operation of monitoring the return status of the HTTP request, the operation of monitoring whether domain name hijacking occurs, and monitoring the network Whether the routing jump information is abnormal.

In some embodiments, the request to execute the network monitoring operation includes an execution mode identifier, and the execution mode identifier is used to instruct the selected proxy server to execute the network monitoring operation in parallel/serially.

In some embodiments, the sending unit is configured to further: select a predetermined number of proxy servers from the proxy server group in ascending order of CPU utilization; and/or select a predetermined number of proxy servers from the proxy server in ascending order of memory utilization Select a predetermined number of proxy servers from the group; and/or select a predetermined number of proxy servers from the proxy server group in ascending order of the number of monitoring operations performed.

In some embodiments, the device further includes: a cache unit, configured to cache the monitoring result in a message queue, and create different topics for different types of network monitoring operations in the message queue.

In some embodiments, the device further includes: an aggregation unit, configured to aggregate the monitoring results in the message queue in response to receiving the network monitoring request including the target location area, obtain the aggregated monitoring results, and aggregate the The monitoring results are delivered to the data persistence cluster; and the determination unit is further used to determine whether the network in the target location area is abnormal according to the aggregated monitoring results and the preset network quality indicators.

In some embodiments, the request for performing the network monitoring operation includes at least one of the following: a start time for instructing the selected proxy server to start performing the network monitoring operation, an end time for instructing the end of performing the network monitoring operation, and an end time for instructing the selected proxy server to start performing the network monitoring operation. The execution cycle of periodically executing the network monitoring operation; and the apparatus further includes: storing the start time, the end time and the execution cycle in a database.

In some embodiments, the device further includes: an upgrade unit, configured to send an update request to the proxy server, so that the proxy server performs a grayscale upgrade; receive update information fed back by the proxy server and store it in the database.

In some embodiments, the storage unit is further configured to: store the status information sent by the proxy server in the database; and/or store the user information and network monitoring request of the user in the database.

In some embodiments, the sending unit is further configured to: send a request to the selected proxy server to perform a network monitoring operation in response to receiving a monitoring operation request periodically sent by the selected proxy server.

In a third aspect, the embodiment of the present application provides a server, including: one or more processors; a storage device for storing one or more programs, when one or more programs are executed by one or more processors, One or more processors are caused to implement the method according to any one of the first aspect.

In a fourth aspect, the embodiment of the present application provides that when the program is executed by a processor, any one of the methods in the first aspect is implemented.

In the network monitoring method and device provided in the embodiments of the present application, an agent server whose current load is suitable for monitoring the network is selected from the agent server (Agent) group specified by the user and distributed in different location areas, and sends an execution network monitoring message to the selected agent server. According to the monitoring result returned by the proxy server and the preset network quality index, it is determined whether the network in the geographic location where the selected proxy server is located is abnormal. This application implements different types of monitoring tasks issued by users through proxy servers deployed on different operators in various provinces and cities across the country, fully simulating users' access to core services to monitor the network quality of operators in various provinces and cities across the country in real time, and Generate early warnings to realize timely switching of services and reduce business losses caused by network problems. Compared with the existing implementation schemes, this technology implementation scheme is closer to the real scene of the user, and the types of monitoring are diversified, and it can be flexibly expanded according to business needs, greatly reducing operating costs.

Description of drawings

Other characteristics, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

FIG. 1 is an exemplary system architecture diagram to which the present application can be applied;

Fig. 2 is a flow chart according to an embodiment of the network monitoring method of the present application;

FIG. 3 is a schematic diagram of an application scenario of a network monitoring method according to the present application;

FIG. 4 is a flowchart of another embodiment of the network monitoring method according to the present application;

Fig. 5 is a flow chart of message interaction between the server and the proxy server according to the network monitoring method of the present application;

FIG. 6 is a schematic structural diagram of an embodiment of a network monitoring device according to the present application;

FIG. 7 is a schematic structural diagram of a computer system suitable for implementing the server of the embodiment of the present application.

Detailed ways

The application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain related inventions, rather than to limit the invention. It should also be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings.

It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

FIG. 1 shows an exemplary system architecture 100 to which embodiments of the network monitoring method or network monitoring device of the present application can be applied.

As shown in FIG. 1 , the system architecture 100 may include terminal devices 101 , 102 , 103 , a server 104 , proxy servers 105 , 106 , 107 and a target server 108 . Terminal devices 101, 102, 103, server 104, proxy servers 105, 106, 107, and target server 108 are interconnected through communication links, such as wired, wireless communication links or optical fiber cables.

Users can use terminal devices 101 , 102 , 103 to monitor the network connection quality of target server 108 through server 104 and proxy servers 105 , 106 , 107 . Various communication client applications such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social platform software can be installed on the terminal devices 101, 102, and 103.

Terminal devices 101, 102, 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, moving picture Expert Compression Standard Audio Layer 3), MP4 (Moving PictureExperts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) Players, Laptop Portable Computers and Desktop Computers, etc.

The server 104 may be a server that provides various services, for example, a network monitoring server that supports the network quality information web pages displayed on the terminal devices 101 , 102 , and 103 . The network monitoring server can analyze and process received data such as network monitoring requests, and allocate proxy servers 105, 106, and 107 for network monitoring according to the processing results. The proxy servers 105, 106, 107 perform network monitoring operations, and feed back monitoring results (such as network abnormal data) to terminal devices.

It should be noted that the network monitoring method provided by the embodiment of the present application is generally executed by the server 104 and the proxy servers 105, 106, 107, and correspondingly, the network monitoring device is generally set in the server 104 and the proxy servers 105, 106, 107.

It should be understood that the numbers of terminal devices, servers, proxy servers, and target servers in FIG. 1 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

Continuing to refer to FIG. 2 , a flow 200 of an embodiment of the network monitoring method according to the present application is shown. The network monitoring method includes the following steps:

Step 201, receiving a network monitoring request to be executed by a designated proxy server group sent by a user.

In this embodiment, the electronic device on which the network method runs (for example, the server 104 shown in FIG. 1 ) can receive the command executed by the designated proxy server group from the terminal that the user uses to monitor the network through a wired connection or a wireless connection. Network monitoring request. The server has received location information of multiple proxy servers, and the location information includes geographic location information and operator information. Multiple proxy servers may be grouped according to location area in advance. For example, if the location information sent by proxy server A is Beijing Unicom, the location information sent by proxy server B is Beijing Telecom, the location information sent by proxy server C is Beijing Unicom, and the location information sent by proxy server D is Shanghai Unicom, then the Proxy servers A and B are divided into one group, proxy server C is in one group, and proxy server D is in one group. Users can specify groups to perform network monitoring requests for Beijing Unicom's proxy server group.

Step 202, acquire status information of each proxy server in the proxy server group.

In this embodiment, the status information of each proxy server may include information such as CPU utilization, memory utilization, and the number and location information of the monitoring operations performed. The status information can be regularly reported by the proxy server. The location information includes geographic location information and operator information. For example, Beijing Unicom, Guangdong Mobile and so on. The number of monitoring operations is the number of operations (for example, sending ICMP (Internet Control Message Protocol, Internet Control Message Protocol) packets, HTTP requests, etc.) when the proxy server performs network monitoring.

Step 203: Select a predetermined number of proxy servers from the proxy server group according to CPU utilization, memory utilization and the number of monitoring operations performed, and determine the geographic location of the selected proxy servers.

In this embodiment, a proxy server with a smaller load may be selected from multiple candidate proxy servers in the proxy server group to perform the monitoring operation. The load profile can refer to CPU utilization, memory utilization, and the number of monitoring operations performed. The higher the CPU utilization rate and memory utilization rate and the greater the number of monitoring operations performed, it means that the load is heavy and it is not suitable to perform monitoring operations. The geographic location of the selected proxy server can be determined according to the location information obtained in step 202 .

In some optional implementation manners of this embodiment, the status information sent by the proxy server is stored in the database, and/or the user information and network monitoring request of the user are stored in the database. The database can be a MySQL database.

Step 204, sending a request for performing a network monitoring operation to the selected proxy server.

In this embodiment, based on the server selected in step 203, a request for performing a network monitoring operation is sent to the selected proxy server. There are many ways of network monitoring operation, and the request for network monitoring operation can be sent in a targeted manner according to the load conditions of different proxy servers. For example, a heavily loaded proxy server may send requests for network monitoring operations in a single mode, for example, sending only ICMP packets. A proxy server with a light load can send requests for network monitoring operations in various ways, for example, perform operations such as sending ICMP packets, HTTP requests, and DNS detection in parallel.

In some optional implementations of this embodiment, the network monitoring operation includes at least one of the following: the operation of monitoring the transmission delay and/or packet loss rate of the data packet, the operation of monitoring the return status of the HTTP request, and monitoring whether The operation of domain name hijacking, the operation of monitoring whether the routing information of the network is abnormal. An example of a specific monitoring operation could be as follows:

a) ICMP-PING: Through the monitoring method of sending ICMP packets at the bottom layer, you can know the basic network status of the operator in the province and city where the proxy server is located to a certain IP or domain name, which is reflected by the delay and packet loss rate.

b) TCP-PING: The bottom layer sends a TCP packet to the destination, and the destination directly returns a Reset packet, calculates the delay by recording the sending and returning time of the packet, calculates the packet loss rate by counting the number of returned packets, and increases the TCP-PING The task type is mainly due to the suppression of ICMP packets by some operators, but generally there is no restriction on TCP packets.

c) HTTP: By initiating an HTTP request to a website on the Internet inside the proxy server, we can fully simulate a user browsing a certain webpage, which is also a monitoring task method that is closest to user behavior, and the monitoring result returns the status through HTTP code and various other result parameters.

d) DNS: By analyzing a domain name inside the proxy server, and then comparing it with the standard domain name resolution, it is possible to monitor whether DNS hijacking for a domain name has occurred under the network of the operator in the area (in the hijacked network Intercept domain name resolution requests within the scope, analyze the requested domain name, and release requests outside the scope of review, otherwise return a false IP address or do nothing to make the request unresponsive, and the effect is that it cannot respond to or access a specific network. fake URL).

e) Traceroute: Traceroute is a tool used to pass through the gateway between the host sending the data packet and the target host. By performing Traceroute monitoring on a certain IP or domain name inside the proxy server, you can know the entire routing jump situation of accessing a certain IP or domain name from the network where the proxy server is located.

In some optional implementations of this embodiment, the request to execute the network monitoring operation includes an execution mode identifier, and the execution mode identifier is used to instruct the selected proxy server to execute the network monitoring operation in parallel/serially. For example, if the execution mode is identified as parallel, the above-mentioned multiple monitoring operations are executed simultaneously. If the execution mode is identified as serial, only one of the above-mentioned monitoring operations is performed at a time, and another one is performed after one monitoring operation is completed. The server can adjust the execution mode according to the current load condition of the proxy server. Different implementations can be sent to the same set of proxy servers. For example, 8 monitoring tasks have been running on proxy server A, and proxy server B is currently idle, then the execution mode sent by proxy server A can be marked as serial, and the execution mode sent by proxy server B can be marked as parallel.

In some optional implementations of this embodiment, a predetermined number of proxy servers are selected from the proxy server group according to CPU utilization, memory utilization and the number of monitoring operations performed, including: Select a predetermined number of proxy servers from the proxy server group in order of increasing memory utilization; and/or select a predetermined number of proxy servers from the proxy server group in ascending order of memory utilization; and/or according to the monitoring performed Select a predetermined number of proxy servers from the proxy server group in ascending order of the number of operations. For example, if the predetermined number is 1, and the CPU utilization rate of the proxy server A is 80%, and the CPU utilization rate of the proxy server B is 20%, then the proxy server B is selected to perform the monitoring operation. When the CPU utilization is the same, you can refer to the size of the memory utilization to select a proxy server with a low memory utilization. If the CPU utilization rate and memory utilization rate are the same, the number of monitoring operations performed may be referred to, and a proxy server with a smaller number of monitoring operations performed may be selected.

In some optional implementations of this embodiment, the request for performing the network monitoring operation includes at least one of the following: a start time for instructing the selected proxy server to start performing the network monitoring operation, an indication for ending the execution of the network monitoring operation The end time and the execution period used to indicate the periodic execution of the network monitoring operation; and the method further includes: storing the start time, the end time and the execution period in a database, such as a MySQL database. The server can control the working status of the proxy server through the start time, end time and execution cycle. And the status information of the proxy server can be determined. Even if the status information sent by the proxy server is not received, the status information of the proxy server can be calculated according to the start time, end time and execution cycle sent before.

In some optional implementations of this embodiment, sending a request to the selected proxy server to perform a network monitoring operation includes: in response to receiving the monitoring operation request periodically sent by the selected proxy server, sending the selected proxy server The server sends a request to perform a network monitoring operation. In this case, the server does not actively send a monitoring request, but waits to receive the monitoring request from the proxy server before sending it to the proxy server.

Step 205, receiving the monitoring result returned by the selected proxy server, and determining whether the network in the above geographic location is abnormal according to the monitoring result and the preset network quality index.

In this embodiment, the monitoring result may include information obtained from monitoring operations such as time delay and packet loss rate. Determine whether the network at the geographic location where the proxy server is located is abnormal according to preset network quality indicators (eg, packet loss rate threshold, delay threshold, HTTP request success rate threshold, etc.). For example, if the packet loss rate exceeds 10% of the packet loss rate threshold, it may be considered that the network in the geographic location is abnormal.

In some optional implementation manners of this embodiment, an update request is sent to the proxy server, so that the proxy server performs a grayscale upgrade. Receive the updated information fed back by the proxy server and store it in the database. Grayscale upgrade is a smooth switch during upgrade. When some proxy servers need to be upgraded, one of the proxy servers must be upgraded first as a test to ensure that the upgrade is correct. That is to say, all proxy servers are not updated and upgraded synchronously (for example, this time it is only Guangdong update). After the upgrade is complete, upgrade servers in other regions.

Continue to refer to FIG. 3 , which is a schematic diagram of an application scenario of the network monitoring method according to this embodiment. In the application scenario of FIG. 3 , the server 300 includes: a task scheduling server 302, a DB (Database, database) server 303, a state management server 304, a Server server 305, a data queue 308, a real-time computing cluster 309 and a distributed database cluster 310. The proxy server includes a downloader 306 and an Agent (proxy server) 307 . The user sends a network monitoring request to the server 300 through the user terminal 301, and the server 300 chooses to replace the Agent 307 to perform the network monitoring operation. Agent 307 returns the monitoring result to server 300 . The specific work is as follows:

User terminal 301: the user sends a network monitoring request to the server 300 through the user terminal 301, and can receive the monitoring result returned by the server 300.

Task scheduling server 302: responsible for generating monitoring operations and tasks that can be performed inside Agent307 according to monitoring requests issued by users. The main functions include: receiving monitoring tasks issued by users and transforming them into monitoring operations and tasks performed internally by Agent307. When the task scheduling server 302 generates monitoring operations and tasks, it needs to combine the Agent grouping, the CPU and memory status of the Agent 307, and the number of monitoring tasks that the Agent is running to select the best Agent for this monitoring task. .

DB server 303: responsible for persistent storage of business information, implemented by MySQL, with the following main functions: persistent storage of user information and monitoring task information issued by users. persistently store the status information of the uploader and downloader 306 and the Agent307;

Status management server 304: responsible for maintaining the status information of each resource in the system, the main functions are as follows: regularly update the status of the uploader and downloader 306 and Agent 307, mainly including online and offline status. According to the monitoring task, that is, the start and end running time of the monitoring operation, its running status is regularly updated, mainly including execution and stop.

Server 305: responsible for receiving inquiries from the Agent, and receiving the task running results reported by the Agent. The main functions include: receiving regular inquiries from the Agent 307 whether there are new monitoring tasks, and if so, returning the specific information of the monitoring tasks to the Agent. Receive the task running results reported by the Agent, perform simple data preprocessing according to the task type, and then deliver them to the data queue.

The downloader 306 is responsible for installing and updating the Agent 307 deployed on operators in various provinces and cities across the country. The main functions include: according to the system version and the public network IP assigned to it by the operator, grayscale update of the Agent 307;

Agent307: Collect running status indicators and execute network monitoring requests issued by users. The main functions are as follows: collected running status indicators, including CPU and memory usage, and the number of Agent running tasks. When Agent307 runs to generate a large load , The normal business that can be guaranteed by its own suicide mechanism will not be affected. Regularly ask whether the Server server 305 has a new monitoring operation assigned to itself, if so, pull the monitoring operation information from the Server server 305, and pull the task information contained in the monitoring operation from the Server server according to the unique monitoring operation ID , execute locally according to the execution timeline included in the task, and report the running results to the Server 305 in batches. The monitoring operations acquired by Agent 307 on Server 305 may include multiple different types of tasks, mainly including ICMP-PING, TCP-PING, HTTP, DNS, Traceroute and other monitoring types, and Agent 307 supports serial and parallel execution modes.

Data queue 308: responsible for caching the execution results reported by Agent307, the main functions are as follows: caching the monitoring results reported by Agent307 to prevent data loss. The open source message queue Kafka is used in this application. For different task types running in Agent307, it needs Create different topics (Topic) in Kafka.

Real-time computing cluster 309: responsible for real-time aggregation of data reported by Agent 307 according to business requirements, and delivery of aggregation results and original data to distributed data cluster 310, for example, data persistence cluster. In this application, an open-source streaming real-time computing cluster Storm (a distributed, fault-tolerant real-time computing system) is used. Perform aggregation and deliver the aggregation result to the data persistence cluster. In order to avoid the loss of the original data, the real-time computing cluster 309 will also deliver a copy of the original data to the data persistence cluster.

Distributed data cluster 310: persistently store the monitoring data reported by Agent307, the main functions are as follows: persistently store the original data and aggregated data, Druid (a real-time query and analysis for big data) is used in this application Highly fault-tolerant, high-performance open source distributed system, designed to quickly process large-scale data, and enable fast query and analysis).

The method provided by the above-mentioned embodiments of the present application realizes the forward detection from the operators in various provinces and cities to the core computer room through the proxy servers distributed in various places, which makes up for the deficiency that only reverse detection can be done before. The monitoring operation can fully simulate the user's online behavior, the monitoring results are closer to the real scene, and the data is more reliable. Effective use of ubiquitous x86 hardware resources and open source solutions reduces research and development costs. It can be deeply customized in combination with its own related business, and has good scalability. Covering most of the country's prefecture-level cities and different operators, the data is comprehensive. Replaced the third tone monitoring, saving the company operating costs.

Further referring to FIG. 4 , it shows a flow 400 of another embodiment of the network monitoring method. The process 400 of the network monitoring method includes the following steps:

Step 401, receiving a network monitoring request to be executed by a designated proxy server group sent by a user.

Step 402, acquiring status information of each proxy server in the proxy server group.

Step 403: Select a predetermined number of proxy servers from the proxy server group according to CPU utilization, memory utilization and the number of monitoring operations performed, and determine the geographic location of the selected proxy servers.

Step 404, sending a request for performing a network monitoring operation to the selected proxy server.

Step 405, receiving the monitoring result returned by the selected proxy server, and determining whether the network in the geographic location is abnormal according to the monitoring result and the preset network quality index.

Steps 401-405 are basically the same as steps 201-205, so details are not repeated here.

In step 406, the monitoring result is cached in the message queue, and different topics are created in the message queue for different types of network monitoring operations.

In this embodiment, the Kafka message queue can be used to cache according to the monitoring operation type (for example, monitoring packet loss rate, HTTP request success rate, etc.). Kafka is a high-throughput distributed publish-subscribe message system with the following characteristics: Provide message persistence through O(1) disk data structure, which can maintain long-term storage even for messages with several terabytes stable performance. High Throughput: Even very modest hardware Kafka can support millions of messages per second. Support for partitioning messages through Kafka server and consumer machine clusters. Support Hadoop parallel data loading. Each message published to the Kafka cluster has a category. This category is called a topic (Topic). Physically, messages of different Topics are stored separately. Logically, a Topic message is stored in one or more Brokers (the Kafka cluster contains One or more servers, this kind of server is called Broker), but users only need to specify the Topic of the message to produce or consume data without caring about where the data is stored.

Step 407, in response to receiving the network monitoring request including the target location area, aggregate the monitoring results in the message queue to obtain the aggregated monitoring results, and post the aggregated monitoring results to the data persistence cluster.

In this embodiment, the real-time computing cluster Storm will continuously consume the result data from Kafka, aggregate the data according to business requirements, and deliver the aggregated results to the back-end data persistence cluster. At the same time, Storm can also upload a copy of original data to the data persistence cluster for persistence to prevent loss of original data.

Step 408: Determine whether the network in the target location area is abnormal according to the aggregated monitoring results and the preset network quality index.

In this embodiment, the monitoring results may include network quality information of multiple URLs belonging to the same target location area obtained through monitoring operations such as time delay and packet loss rate. Determine whether the target location area is abnormal according to preset network quality indicators (eg, packet loss rate threshold, delay threshold, HTTP request success rate, etc.) and the average value of network quality information of multiple URLs. For example, if the average packet loss rate of multiple URLs exceeds 10%, it can be considered that the network in the target location area is abnormal.

It can be seen from FIG. 4 that, compared with the embodiment corresponding to FIG. 2 , the process 400 of the network monitoring method in this embodiment highlights the step of monitoring result aggregation. Therefore, the solution described in this embodiment can introduce more monitoring results to evaluate the network quality of a certain location area, thereby improving the coverage of monitoring, and can independently customize the monitoring of the target area for aggregation analysis, which has good scalability .

Further referring to FIG. 5 , it shows a flow chart of message interaction between the server and the proxy server according to the network monitoring method of the present application, including the following messages:

Message 501, register to server: the proxy server sends a registration request to the server, and the registration request may include hardware information of the proxy server, and may also include status information, such as CPU utilization, memory utilization, and other information.

Message 502, registration successful: the server verifies the validity of the proxy server after receiving the registration request, and then sends a message of successful registration.

Message 503, asking whether to assign a new monitoring operation: the proxy server periodically asks the server whether to assign a new monitoring operation.

Message 504, No new monitoring operation assigned: If the server does not receive a monitoring request from the user, there is no monitoring operation to be assigned, so it replies to the proxy server with a message that no new monitoring operation is assigned.

Message 505, obtain the monitoring operation information including task 1 and task 2: if the server receives the monitoring request from the user, it needs to assign the monitoring operation, so it notifies the proxy server to obtain the monitoring operation information of task 1 and task 2.

Message 506, requesting task 1 information and task 2 information: After receiving message 505, the proxy server sends a message to the server requesting to obtain task 1 information and task 2 information.

Message 507, return task 1 information and task 2 information: the server sends task 1 information and task 2 information to the proxy server.

Message 508, return the monitoring result of task 1: the proxy server returns the monitoring result of task 1 to the server after executing task 1.

Message 509, return the monitoring result of task 2: the proxy server returns the monitoring result of task 2 to the server after executing task 2.

Message 510, return the monitoring result of task 1: the server returns the monitoring result of task 1 to the message queue.

Message 511, return the monitoring result of task 2: the server returns the monitoring result of task 2 to the message queue.

Further referring to FIG. 6 , as an implementation of the methods shown in the above figures, the present application provides an embodiment of a network monitoring device. This device embodiment corresponds to the method embodiment shown in FIG. 2 , and the device can specifically Used in various electronic equipment.

As shown in FIG. 6 , the network monitoring device 600 of this embodiment includes: a receiving unit 601 , an acquiring unit 602 , a selecting unit 603 , a sending unit 604 and a determining unit 605 . Wherein, the receiving unit 601 is used to receive the network monitoring request that the specified proxy server group is sent by the user; the obtaining unit 602 is used to obtain the status information of each proxy server in the proxy server group, and the status information includes: CPU utilization rate, memory utilization rate and the number and location information of the monitoring operations performed; the selection unit 603 is used to select a predetermined number of proxy servers from the proxy server group according to the CPU utilization, memory utilization and the number of monitoring operations performed, and determine the selected proxy The geographic location of the server; the sending unit 604 is used to send a request to the selected proxy server to perform a network monitoring operation; the determining unit 605 is used to receive the monitoring result returned by the selected proxy server, and according to the monitoring result and the preset network quality Metrics to determine whether a geographic location's network is anomalous.

In this embodiment, the specific processing of the receiving unit 601, the acquiring unit 602, the selecting unit 603, the sending unit 604 and the determining unit 605 of the network monitoring device 600 can refer to the steps 201, 202 and 203 in the corresponding embodiment in FIG. 2 , step 204 and step 205.

In some optional implementations of this embodiment, the network monitoring operation includes at least one of the following: the operation of monitoring the transmission delay and/or packet loss rate of the data packet, the operation of monitoring the return status of the HTTP request, and monitoring whether The operation of domain name hijacking, the operation of monitoring whether the routing information of the network is abnormal.

In some optional implementations of this embodiment, the request to execute the network monitoring operation includes an execution mode identifier, and the execution mode identifier is used to instruct the selected proxy server to execute the network monitoring operation in parallel/serially.

In some optional implementations of this embodiment, the sending unit 604 is further configured to: select a predetermined number of proxy servers from the proxy server group in ascending order of CPU utilization; and/or select a predetermined number of proxy servers according to memory utilization Selecting a predetermined number of proxy servers from the proxy server group in ascending order; and/or selecting a predetermined number of proxy servers from the proxy server group in ascending order of the number of monitoring operations performed.

In some optional implementations of this embodiment, the apparatus 600 further includes a cache unit (not shown), configured to cache the monitoring results in a message queue, and create different themes.

In some optional implementations of this embodiment, the apparatus 600 further includes an aggregation unit (not shown), configured to aggregate the monitoring results in the message queue in response to receiving a network monitoring request including the target location area , to obtain the aggregated monitoring results, and deliver the aggregated monitoring results to the data persistence cluster; and the determination unit 605 is further used to determine whether the network in the target location area is abnormal.

In some optional implementations of this embodiment, the request for performing the network monitoring operation includes at least one of the following: a start time for instructing the selected proxy server to start performing the network monitoring operation, an indication for ending the execution of the network monitoring operation The end time and the execution period used to indicate the periodic execution of the network monitoring operation; the apparatus 600 also includes a storage unit (not shown) for storing the start time, end time and execution period in the database.

In some optional implementations of this embodiment, the device 600 also includes an upgrade unit (not shown), configured to send an update request to the proxy server, so that the proxy server performs a grayscale upgrade; receive an update fed back by the proxy server information and stored in the database.

In some optional implementations of this embodiment, the storage unit is further configured to: store the status information sent by the proxy server in the database; and/or store the user information and network monitoring request of the user in the database.

In some optional implementations of this embodiment, the sending unit 604 is further configured to, in response to receiving a monitoring operation request periodically sent by the selected proxy server, send a request for performing a network monitoring operation to the selected proxy server.

Referring now to FIG. 7 , it shows a schematic structural diagram of a computer system 700 suitable for implementing the server of the embodiment of the present application. The server shown in FIG. 7 is only an example, and should not limit the functions and scope of use of this embodiment of the present application.

As shown in FIG. 7 , a computer system 700 includes a central processing unit (CPU) 701 that can operate according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage section 708 into a random-access memory (RAM) 703 Instead, various appropriate actions and processes are performed. In the RAM 703, various programs and data necessary for the operation of the system 700 are also stored. The CPU 701 , ROM 702 , and RAM 703 are connected to each other via a bus 704 . An input/output (I/O) interface 705 is also connected to the bus 704 .

The following components are connected to the I/O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 708 including a hard disk, etc. and a communication section 709 including a network interface card such as a LAN card, a modem, or the like. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I/O interface 705 as needed. A removable medium 711 such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc. is mounted on the drive 710 as necessary so that a computer program read therefrom is installed into the storage section 708 as necessary.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication portion 709 and/or installed from removable media 711 . When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the method of the present application are performed. It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which computer-readable program codes are carried. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. . Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present application may be implemented by means of software or by means of hardware. The described units may also be set in a processor. For example, it may be described as: a processor includes a receiving unit, an acquiring unit, a selecting unit, a sending unit, and a determining unit. Wherein, the names of these units do not limit the unit itself under certain circumstances, for example, the receiving unit may also be described as "receiving a network monitoring request performed by a designated proxy server group sent by a user".

As another aspect, the present application also provides a computer-readable medium. The computer-readable medium may be included in the device described in the above embodiments, or it may exist independently without being assembled into the device. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the device, the device: receives a network monitoring request sent by a user to be executed by a designated proxy server group; The status information of each proxy server, the status information includes: CPU utilization, memory utilization, and the number and location information of the monitoring operations performed; according to the CPU utilization, memory utilization and the number of monitoring operations performed, from the proxy server group Select a predetermined number of proxy servers, and determine the geographical location of the selected proxy servers; send a request to the selected proxy servers to perform network monitoring operations; receive the monitoring results returned by the selected proxy servers, and according to the monitoring results and preset The network quality indicator of determines whether the network of the geographic location is anomalous.

The above description is only a preferred embodiment of the present application and an illustration of the applied technical principle. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, but should also cover the technical solution formed by the above-mentioned technical features without departing from the inventive concept. Other technical solutions formed by any combination of or equivalent features thereof. For example, a technical solution formed by replacing the above-mentioned features with technical features with similar functions disclosed in (but not limited to) this application.

Claims (22)

1. A network monitoring method, characterized in that the method comprises: Receive the network monitoring request performed by the specified proxy server group sent by the user; Obtaining status information of each proxy server in the proxy server group, the status information including: CPU utilization rate, memory utilization rate, and quantity and location information of monitoring operations performed; Select a predetermined number of proxy servers from the proxy server group according to CPU utilization, memory utilization and the number of monitoring operations performed, and determine the geographic location of the selected proxy servers; Send a request to perform a network monitoring operation to the selected proxy server; Receive the monitoring result returned by the selected proxy server, and determine whether the network at the geographic location is abnormal according to the monitoring result and a preset network quality index. 2. The method according to claim 1, wherein the network monitoring operation comprises at least one of the following: The operation of monitoring the transmission delay and/or packet loss rate of data packets, the operation of monitoring the return status of HTTP requests, the operation of monitoring whether domain name hijacking occurs, and the operation of monitoring whether the routing information of the network is abnormal. 3. The method according to claim 2, wherein the request for executing the network monitoring operation includes an execution mode identifier, and the execution mode identifier is used to instruct the selected proxy server to execute the network monitoring in parallel/serially operate. 4. The method according to claim 1, characterized in that, selecting a predetermined number of proxy servers from the proxy server group according to the CPU utilization rate, memory utilization rate and the number of monitoring operations performed, comprising: Selecting a predetermined number of proxy servers from the proxy server group in ascending order of CPU utilization; and/or Selecting a predetermined number of proxy servers from the proxy server group in ascending order of memory utilization; and/or A predetermined number of proxy servers are selected from the proxy server group in ascending order of the number of monitoring operations performed. 5. The method according to any one of claims 1-4, wherein the method further comprises: The monitoring result is cached in a message queue, and different topics are created in the message queue for different types of network monitoring operations. 6. The method according to claim 5, further comprising: In response to receiving the network monitoring request including the target location area, aggregate the monitoring results in the message queue to obtain the aggregated monitoring results, and deliver the aggregated monitoring results to the data persistence cluster; Whether the network in the target location area is abnormal is determined according to the aggregated monitoring result and a preset network quality index. 7. The method according to claim 1, wherein the request for performing the network monitoring operation includes at least one of the following: a start time for instructing the selected proxy server to start performing the network monitoring operation, Indicating an end time for ending execution of the network monitoring operation and an execution period for indicating periodic execution of the network monitoring operation; and The method also includes: The start time, the end time and the execution period are stored in a database. 8. The method according to claim 1, further comprising: Send an update request to the proxy server, so that the proxy server performs a grayscale upgrade; The updated information fed back by the proxy server is received and stored in the database. 9. The method according to claim 1, further comprising: storing the status information sent by the proxy server in a database; and/or storing the user information of the user and the network monitoring request in a database. 10. The method according to claim 1, wherein the sending a request for performing a network monitoring operation to the selected proxy server comprises: In response to receiving the monitoring operation request periodically sent by the selected proxy server, a request to perform a network monitoring operation is sent to the selected proxy server. 11. A network monitoring device, characterized in that the device comprises: The receiving unit is used to receive the network monitoring request executed by the specified proxy server group sent by the user; An acquisition unit, configured to acquire status information of each proxy server in the proxy server group, the status information including: CPU utilization, memory utilization, and the number and location information of the monitoring operations performed; The selection unit is used to select a predetermined number of proxy servers from the proxy server group according to CPU utilization, memory utilization and the number of monitoring operations performed, and determine the geographic location of the selected proxy servers; a sending unit, configured to send a request for performing a network monitoring operation to the selected proxy server; The determining unit is configured to receive the monitoring result returned by the selected proxy server, and determine whether the network at the geographic location is abnormal according to the monitoring result and a preset network quality index. 12. The device according to claim 11, wherein the network monitoring operation comprises at least one of the following: The operation of monitoring the transmission delay and/or packet loss rate of data packets, the operation of monitoring the return status of HTTP requests, the operation of monitoring whether domain name hijacking occurs, and the operation of monitoring whether the routing information of the network is abnormal. 13. The device according to claim 12, wherein the request for executing the network monitoring operation includes an execution mode identifier, and the execution mode identifier is used to instruct the selected proxy server to execute the network monitoring in parallel/serially operate. 14. The device according to claim 11, wherein the sending unit is used for further: Selecting a predetermined number of proxy servers from the proxy server group in ascending order of CPU utilization; and/or Selecting a predetermined number of proxy servers from the proxy server group in ascending order of memory utilization; and/or A predetermined number of proxy servers are selected from the proxy server group in ascending order of the number of monitoring operations performed. 15. The device according to any one of claims 11-14, wherein the device further comprises: The cache unit is configured to cache the monitoring results in a message queue, and create different topics for different types of network monitoring operations in the message queue. 16. The device according to claim 15, further comprising: The aggregation unit is configured to aggregate the monitoring results in the message queue in response to receiving the network monitoring request including the target location area, obtain the aggregated monitoring results, and deliver the aggregated monitoring results to the data persistence cluster; and The determining unit is further configured to determine whether the network of the target location area is abnormal according to the aggregated monitoring result and a preset network quality index. 17. The device according to claim 15, wherein the request for performing the network monitoring operation includes at least one of the following: a start time for instructing the selected proxy server to start performing the network monitoring operation, Indicating an end time for ending execution of the network monitoring operation and an execution period for indicating periodic execution of the network monitoring operation; and The device also includes: The start time, the end time and the execution period are stored in a database. 18. The device according to claim 11, further comprising: The upgrade unit is configured to send an update request to the proxy server, so that the proxy server performs grayscale upgrade; receive the update information fed back by the proxy server and store it in the database. 19. The device according to claim 11, wherein the storage unit is further used for: store state information sent by the proxy server in a database; and/or storing the user information of the user and the network monitoring request in a database. 20. The device according to claim 11, wherein the sending unit is further used for: In response to receiving the monitoring operation request periodically sent by the selected proxy server, a request to perform a network monitoring operation is sent to the selected proxy server. 21. A server comprising: one or more processors; storage means for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method according to any one of claims 1-10. 22. A computer-readable storage medium, on which a computer program is stored, wherein, when the program is executed by a processor, the method according to any one of claims 1-10 is implemented.