CN113542424A - Data processing method, device, equipment and computer program product - Google Patents
Data processing method, device, equipment and computer program product Download PDFInfo
- Publication number
- CN113542424A CN113542424A CN202110816540.7A CN202110816540A CN113542424A CN 113542424 A CN113542424 A CN 113542424A CN 202110816540 A CN202110816540 A CN 202110816540A CN 113542424 A CN113542424 A CN 113542424A
- Authority
- CN
- China
- Prior art keywords
- salt
- client
- data processing
- preset
- system information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/50—Testing arrangements
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer And Data Communications (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a data processing method, which comprises the following steps: the method comprises the following steps that a salt-master server side obtains system information sent by each salt-minus client side, wherein a monitoring analysis unit of the salt-minus client side obtains the system information in a timing mode and feeds the system information back to the salt-master server side; a monitoring analysis module of the salt-master server side determines whether the salt-termination client side meets a preset alarm condition or not based on the system information; and if the salt-mining client side is determined to have a target client side which reaches the preset alarm condition, the monitoring analysis module outputs alarm prompt information based on the target client side. The invention also discloses a data processing device, equipment and a computer program product. The invention analyzes the state of each salt-termination client in time through the monitoring analysis module of the salt-master server so as to inform maintenance personnel in time when the salt-termination client is down or the network is disconnected, thereby improving the reliability of the system.
Description
Technical Field
The present invention relates to the field of internet technologies, and in particular, to a data processing method, apparatus, device, and computer program product.
Background
A CDN (Content Delivery Network) distributes Content in an origin server to service nodes deployed in a distributed manner, and by applying a traffic scheduling technique, a user request can be automatically diverted to a globally optimal service node according to a specified policy, and the node provides a service for a user. In the service process, cache contents in the edge node of the actual service often need to be operated, such as cache refreshing (deleting cached expired contents), cache preheating (caching contents of the source station in advance, and reducing pressure of the source station on the return source when the user accesses concurrently), and the like.
In a content distribution network, thousands of servers are often deployed and used as edge nodes, and the servers need to be accurately managed and each node needs to be scheduled for content management (including cache refreshing, cache preheating, and the like). At present, task servers are often scheduled in a unified manner through a deployment center to achieve content management operation, however, the state of a machine cannot be monitored in this manner, so that operation and maintenance personnel cannot sense the machine in time when the server is down or a network is disconnected, and the system reliability is low.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide a data processing method, a data processing device, data processing equipment and a computer program product, and aims to solve the problem that the existing content distribution network cannot timely monitor the state of a server so as to cause low system reliability.
In order to achieve the above object, the present invention provides a data processing method, including the steps of:
the method comprises the steps that a salt-master server side obtains system information sent by each salt-termination client side, wherein the system information comprises CPU utilization rate, memory utilization rate, disk utilization rate and network card bandwidth, and a monitoring analysis unit of the salt-termination client side obtains the system information at regular time and feeds the system information back to the salt-master server side;
a monitoring analysis module of the salt-master server side determines whether the salt-termination client side meets a preset alarm condition or not based on the system information;
and if the salt-mining client side is determined to have a target client side which reaches the preset alarm condition, the monitoring analysis module outputs alarm prompt information based on the target client side.
Further, the step of determining whether the salt-termination client side meets the preset alarm condition by the monitoring analysis module of the salt-master server side based on the system information includes:
the monitoring analysis module of the salt-master server side determines whether the CPU utilization rate reaches a preset CPU utilization rate, whether the memory utilization rate reaches a preset memory utilization rate, whether the disk utilization rate reaches a preset disk utilization rate or whether the network card bandwidth reaches a preset bandwidth;
if the CPU utilization rate reaches a preset CPU utilization rate, the memory utilization rate reaches a preset memory utilization rate, the disk utilization rate reaches a preset disk utilization rate or the network card bandwidth reaches a preset bandwidth, the monitoring analysis module of the salt-master server side determines that the salt-termination client side reaches a preset alarm condition.
Further, the data processing method further comprises:
if the system information sent by the first salt-minus client side is not received within the first preset duration, the salt-master server side sets the state of the first salt-minus client side to be an unavailable state.
4. The data processing method of claim 1, wherein the step of determining whether the salt-termination client side reaches a preset alarm condition based on the system information by the monitoring analysis module of the salt-master server side comprises:
if a second salt-emergence client in an unavailable state exists in the salt-emergence clients, the monitoring analysis module sets the state of the second salt-emergence client to be an online state;
and the monitoring analysis module determines whether the salt-mining client side reaches a preset alarm condition or not based on the system information.
Further, the data processing method further comprises:
a content management module of a salt-master server side receives a task request sent by a management platform of the salt-master server side, wherein the task request comprises a content management type and a resource address;
the content management module determines a target salt-emergence client corresponding to the task request in each salt-emergence client based on the state information of the salt-emergence client;
the content management module sends a target task to the target salt-mining client based on the content management type and the resource address, wherein when the target task is received, the target salt-mining client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server;
and the management platform displays the operation result.
Further, after the step of sending the target task to the target salt-termination client by the content management module based on the content management type and the resource address, the method further includes:
if the operation result fed back by the target salt-mining client is not received within a second preset time length, the content management module updates the retry times;
if the retry times do not reach the preset retry times, returning to the step of executing the content management module to send the target task to the target salt-termination client based on the content management type and the resource address;
and if the retry times reach the preset retry times, the content management module outputs alarm information.
Further, after the step of outputting the alarm prompt information by the monitoring analysis module if it is determined that the salt-termination client side meets the preset alarm condition, the method further includes:
the monitoring analysis module acquires state information of the salt-mining client, wherein the state information comprises an online state and an unavailable state;
the monitoring analysis module sends the system information and the state information to a management platform of a salt-master server;
and the management platform displays the system information and the state information of each salt-termination client.
In addition, in order to achieve the above object, the present invention further provides a data processing apparatus applied to a salt-master server, the data processing apparatus including:
the system comprises an acquisition module, a monitoring analysis unit and a feedback module, wherein the acquisition module is used for acquiring system information sent by each salt-emergence client, the system information comprises CPU (Central processing Unit) utilization rate, memory utilization rate, disk utilization rate and network card bandwidth, and the monitoring analysis unit of the salt-emergence client acquires the system information at regular time and feeds back the system information;
the monitoring analysis module is used for determining whether the salt-termination client side reaches a preset alarm condition based on the system information;
and the output module is used for outputting alarm prompt information based on the target client if the target client which reaches the preset alarm condition exists in the salt-termination client.
Further, to achieve the above object, the present invention also provides a data processing apparatus comprising: a memory, a processor and a data processing program stored on the memory and executable on the processor, the data processing program, when executed by the processor, implementing the steps of the data processing method as described above.
Furthermore, to achieve the above object, the present invention also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the aforementioned data processing method.
The system information sent by each salt-termination client is obtained through a salt-master server, wherein the system information comprises CPU utilization rate, memory utilization rate, disk utilization rate and network card bandwidth, and a monitoring analysis unit of the salt-termination client obtains the system information at regular time and feeds back the system information to the salt-master server; then, a monitoring analysis module of the salt-master server side determines whether the salt-termination client side reaches a preset alarm condition or not based on the system information; and if it is determined that a target client which meets a preset alarm condition exists in the salt-termination clients, the monitoring analysis module outputs alarm prompt information based on the target client, and the monitoring analysis module of the salt-master server side analyzes the state of each salt-termination client in time so as to inform maintenance personnel in time when the salt-termination clients are down or the network is disconnected, thereby improving the reliability of the system.
Drawings
FIG. 1 is a block diagram of a data processing device in a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a data processing method according to a first embodiment of the present invention;
FIG. 3 is a functional block diagram of a data processing apparatus according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a data processing device in a hardware operating environment according to an embodiment of the present invention.
The data processing device in the embodiment of the present invention may be a PC, or may be a mobile terminal device having a display function, such as a smart phone, a tablet computer, an electronic book reader, an MP3(Moving Picture Experts Group Audio Layer III, motion video Experts compression standard Audio Layer 3) player, an MP4(Moving Picture Experts Group Audio Layer IV, motion video Experts compression standard Audio Layer 4) player, a portable computer, or the like.
As shown in fig. 1, the data processing apparatus may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Optionally, the data processing device may further include a camera, a Radio Frequency (RF) circuit, a sensor, an audio circuit, a WiFi module, and the like. Such as light sensors, motion sensors, and other sensors. Of course, the data processing device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which are not described herein again.
Those skilled in the art will appreciate that the terminal architecture shown in fig. 1 does not constitute a limitation of the data processing apparatus and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a data processing program.
In the data processing apparatus shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and communicating with the backend server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be used to invoke a data processing program stored in the memory 1005.
In this embodiment, the data processing apparatus includes: the system comprises a memory 1005, a processor 1001 and a data processing program which is stored on the memory 1005 and can run on the processor 1001, wherein when the processor 1001 calls the data processing program stored in the memory 1005, the steps of the data processing method in each embodiment are executed.
Referring to fig. 2, fig. 2 is a schematic flow chart of a first embodiment of the data processing method according to the present invention.
In this embodiment, the data processing method includes the following steps:
step S101, a salt-master server side obtains system information sent by each salt-termination client side, wherein the system information comprises CPU utilization rate, memory utilization rate, disk utilization rate and network card bandwidth, and a monitoring analysis unit of the salt-termination client side obtains the system information at regular time and feeds back the system information to the salt-master server side;
in this embodiment, the SaltStack is applied to the CDN, and is a centralized management platform of a server infrastructure, which is a configuration management system capable of maintaining remote nodes in a defined state (e.g., ensuring that a particular software package is installed and a particular service is run). The SaltStack is also a distributed remote execution system for executing commands and querying data on remote nodes, either individually or by any choice. The SaltStack is divided into a server and a client, all managed servers are provided with Salt-minus and run, in addition, one or more hosts are used as the server and provided with Salt-master, and instructions can be sent to the Salt-minus through a command line. The Salt-termination and the Salt-master establish a TCP connection through a message queue component ZeroMQ. The Salt-master can publish tasks through a publish/subscribe mode of the message queue, and the Salt-minus can send information and feed back an execution result to the Salt-master through a request/corresponding mode of the message queue. That is, the salt-master server is a host (main server) in the CDN, and each salt-termination client is a managed server in the CDN.
In this embodiment, each salt-mining client is provided with a monitoring analysis unit, and the monitoring analysis unit acquires the system information at regular time, where the system information includes a CPU usage rate, a memory usage rate, a disk usage rate, and a network card bandwidth, that is, each monitoring analysis unit acquires the CPU usage rate, the memory usage rate, the disk usage rate, and the network card bandwidth of the salt-mining client to which it belongs at regular time, and feeds back the system information to the salt-master server. And the salt-master server side acquires the system information sent by each salt-termination client side.
The time interval for acquiring the system information at regular time can be set reasonably to realize the regular monitoring of the salt-minus client, for example, the time interval can be 1 minute.
Step S102, a monitoring analysis module of the salt-master server side determines whether the salt-termination client side reaches a preset alarm condition or not based on the system information;
in this embodiment, the salt-master server is provided with a monitoring analysis module, and the monitoring analysis module is configured to analyze received system information of each salt-termination client, that is, the monitoring analysis module determines whether the salt-termination client meets a preset alarm condition based on the system information, so as to monitor a state of the machine.
Step S103, if it is determined that a target client which reaches a preset alarm condition exists in the salt-termination client, the monitoring analysis module outputs alarm prompt information based on the target client.
In this embodiment, whether a preset alarm condition is achieved in each salt-mining client is judged through system information, and if it is determined that a target client achieving the preset alarm condition exists in the salt-mining client, the monitoring analysis module acquires client information corresponding to the target client and outputs alarm prompt information based on the client information.
Further, in an embodiment, the step S102 includes:
the monitoring analysis module of the salt-master server side determines whether the CPU utilization rate reaches a preset CPU utilization rate, whether the memory utilization rate reaches a preset memory utilization rate, whether the disk utilization rate reaches a preset disk utilization rate or whether the network card bandwidth reaches a preset bandwidth;
if the CPU utilization rate reaches a preset CPU utilization rate, the memory utilization rate reaches a preset memory utilization rate, the disk utilization rate reaches a preset disk utilization rate or the network card bandwidth reaches a preset bandwidth, the monitoring analysis module of the salt-master server side determines that the salt-termination client side reaches a preset alarm condition.
In this embodiment, when the system information of the salt-mining client is acquired, the monitoring analysis module respectively determines whether the CPU utilization rate reaches a preset CPU utilization rate, whether the memory utilization rate reaches a preset memory utilization rate, whether the disk utilization rate reaches a preset disk utilization rate or whether the network card bandwidth reaches a preset bandwidth, if the CPU utilization rate reaches the preset CPU utilization rate, the memory utilization rate reaches the preset memory utilization rate, the disk utilization rate reaches the preset disk utilization rate or the network card bandwidth reaches the preset bandwidth, that is, any one of the CPU utilization rate, the memory utilization rate, the disk utilization rate and the network card bandwidth reaches the corresponding preset condition, the monitoring analysis module determines that the salt-termination client terminal reaches a preset alarm condition, and then whether the salt-termination client side reaches the preset alarm condition can be accurately judged according to the system information, and accurate monitoring of each salt-termination client side is realized.
The preset CPU utilization rate, the preset memory utilization rate, the preset disk utilization rate, the preset bandwidth, and the like may be reasonably set, for example, the preset CPU utilization rate is 80%, and the preset memory utilization rate is 80%.
Further, in an embodiment, the data processing method further includes:
step a, if the system information sent by the first salt-departure client is not received within a first preset time length, the salt-master server side sets the state of the first salt-departure client to be an unavailable state.
In this embodiment, for each salt-termination client, the salt-master server sets a timer with a first preset duration when receiving system information of the salt-termination client, so as to determine whether the system information of the salt-termination client is received within the first preset duration, and if the system information sent by the first salt-termination client is not received within the first preset duration, the first salt-termination client may have a downtime or a network interruption, and the salt-master server sets the state of the first salt-termination client to an unavailable state, and prompts a maintainer to process the unavailable state. By setting the state of the first salt-termination client to be the unavailable state, the problem that a task is still tried to be sent to the first salt-termination client when the first salt-termination client is down or the network is interrupted is avoided, the pressure of the salt-master server is reduced, the transmission of invalid flow in the network is reduced, and the network bandwidth resource is saved.
It should be noted that the first preset time period is longer than a time interval for acquiring system information at regular time, for example, the first preset time period is 5 minutes.
In the data processing method provided by this embodiment, system information sent by each salt-minus client is acquired through a salt-master server, where the system information includes a CPU usage rate, a memory usage rate, a disk usage amount, and a network card bandwidth, and a monitoring analysis unit of the salt-minus client acquires the system information at regular time and feeds back the system information to the salt-master server; then, a monitoring analysis module of the salt-master server side determines whether the salt-termination client side reaches a preset alarm condition or not based on the system information; and if it is determined that a target client which meets a preset alarm condition exists in the salt-termination clients, the monitoring analysis module outputs alarm prompt information based on the target client, and the monitoring analysis module of the salt-master server side analyzes the state of each salt-termination client in time so as to inform maintenance personnel in time when the salt-termination clients are down or the network is disconnected, thereby improving the reliability of the system.
A second embodiment of the data processing method of the present invention is proposed based on the first embodiment, and in this embodiment, step S102 includes:
step S201, if a second salt-emergence client terminal in an unavailable state exists in the salt-emergence client terminals, the monitoring analysis module sets the state of the second salt-emergence client terminal to be an online state;
step S202, the monitoring analysis module determines whether the salt-termination client side reaches a preset alarm condition based on the system information.
In this embodiment, after the salt-master server obtains the system information sent by each salt-sub client, the monitoring analysis module determines whether the system information includes system information of a second salt-sub client in an unavailable state, and further determines whether the second salt-sub client in the unavailable state exists in the salt-sub client, and if the second salt-sub client in the unavailable state exists in the salt-sub client, the monitoring analysis module sets the state of the second salt-sub client to an online state, so that the second salt-sub client is in a normal state capable of receiving the task obtaining request.
And then, the monitoring analysis module determines whether the salt-mining client side reaches a preset alarm condition based on the system information.
In the data processing method provided in this embodiment, if a second salt-termination client in an unavailable state exists in the salt-termination clients, the monitoring analysis module sets the state of the second salt-termination client to an online state; and then, the monitoring analysis module of the salt-master server side determines whether the salt-termination client side meets a preset alarm condition or not based on the system information, and the state of the second salt-termination client side is set to be an online state, so that the second salt-termination client side can normally execute the task distributed by the salt-master server side, and the resource utilization rate of the CDN is improved.
Based on the first embodiment, a third embodiment of the data processing method of the present invention is provided, and in this embodiment, the data processing method further includes:
step S301, a content management module of a salt-master server receives a task request sent by a management platform of the salt-master server, wherein the task request comprises a client name, a content management type and a resource address;
step S302, the content management module determines a target salt-emergence client corresponding to the task request in each salt-emergence client based on the state information of the salt-emergence client;
step S303, the content management module sends a target task to the target salt-mining client based on the content management type and the resource address, wherein when receiving the target task, the target salt-mining client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server;
step S304, the management platform displays the operation result.
The system comprises a salt-master server, a salt-minion client and a content management unit, wherein the salt-master server is provided with a content management module and a management platform, and the salt-minion client is provided with the content management unit.
In this embodiment, the management platform performs task scheduling, specifically, the management platform obtains a client name, a content management type, and a resource address of a current task, generates and sends a task request to the content management module, and the content management module receives the task request sent by the management platform of the salt-master server, where the task request includes the content management type and the resource address. The content management module determines a target salt-emergence client corresponding to the task request in each salt-emergence client based on the state information of the salt-emergence client, specifically, the content management module acquires the state information of each salt-emergence client, acquires the salt-emergence client in an online state based on the state information, and determines the target client corresponding to the task request in the salt-emergence client in the online state.
Then, the content management module sends a target task to the target salt-mining client based on the content management type and the resource address, wherein when the target task is received, the target salt-mining client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server; and the management platform displays the operation result.
In the data processing method provided by the embodiment, a task request sent by a management platform of a salt-master server is received through a content management module of the salt-master server, wherein the task request comprises a content management type and a resource address; then the content management module determines a target salt-emergence client corresponding to the task request in each salt-emergence client based on the state information of the salt-emergence client; then the content management module sends a target task to the target salt-mining client based on the content management type and the resource address, wherein when the target task is received, the target salt-mining client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server; and then the management platform displays the operation result, so that the monitoring and management of the state of the server are completed while the requirement of content management in the content network system is timely responded, a plurality of software systems are not required to be deployed on the server, and the daily management of operation and maintenance personnel is facilitated. The target salt-termination client is determined through the state information, the target task is prevented from being sent to the salt-termination client with downtime or network interruption, the pressure of the salt-master server is reduced, the transmission of invalid traffic in the network is reduced, and the network bandwidth resources are saved.
A fourth embodiment of the data processing method of the present invention is proposed based on the third embodiment, and in this embodiment, after step S303, the data processing method further includes:
step S401, if the operation result fed back by the target salt-termination client is not received within a second preset time after the target task is sent, the content management module updates the retry times;
step S402, if the retry times do not reach the preset retry times, returning to the step of executing the content management module to send the target task to the target salt-termination client based on the content management type and the resource address;
step S403, if the retry number reaches a preset retry number, the content management module outputs an alarm message.
In this embodiment, after the target task is sent, the duration of the target task after the target task is sent is accumulated, and if the running result fed back by the target salt-mining client is not received within a second preset duration, the content management module updates the retry number.
Then, if the retry number does not reach the preset retry number, returning to execute step S303; and if the retry times reach the preset retry times, the content management module outputs alarm information and resets the retry times.
The second preset time and the preset retry number can be reasonably set, for example, the second preset time can be set to 5 minutes, and the preset retry number can be set to 3.
In the data processing method provided in this embodiment, if the operation result fed back by the target salt-termination client is not received within a second preset time period, the content management module updates the retry number; then if the retry times do not reach the preset retry times, returning to the step of executing the content management module to send the target task to the target salt-termination client based on the content management type and the resource address; and if the retry times reach the preset retry times, the content management module outputs alarm information, and ensures that the target task can be completed in time through an overtime retry mechanism, thereby ensuring the operating efficiency of the CDN.
On the basis of the above-described respective embodiments, a fifth embodiment of the data processing method of the present invention is proposed, in the present embodiment, after step S103, the data processing method further comprising:
step S501, the monitoring analysis module acquires state information of the salt-mining client, wherein the state information comprises an online state and an unavailable state;
step S502, the monitoring analysis module sends the system information and the state information to a management platform of a salt-master server;
step S503, the management platform displays the system information and the state information of each salt-termination client.
In this embodiment, the monitoring analysis module obtains state information of the salt-sub client, where the state information includes an online state and an unavailable state, and sends the state information and the state information of each salt-sub client to the management platform of the salt-master server, and after receiving the state information and the state information of each salt-sub client, the management platform displays system information and the state information of each salt-sub client on a display interface, so that maintenance personnel can view the state of each salt-sub client.
It should be noted that, when the display interface displays the system information and the state information of each salt-minus ion client, if there is a salt-minus ion client in a non-possible state, the management platform may also display the non-possible time of the salt-minus ion client in the non-possible state.
In the data processing method provided in this embodiment, the monitoring analysis module obtains the state information of the salt-mining client, where the state information includes an online state and an unavailable state; then the monitoring analysis module sends the system information and the state information to a management platform of a salt-master server; and then the management platform displays the system information and the state information of each salt-minus client, so that maintenance personnel can conveniently check the information of each salt-minus client, the management of the salt-minus clients is further facilitated, and the management and maintenance efficiency of the salt-minus clients in the CDN is improved.
The present invention also provides a data processing apparatus, referring to fig. 3, the data processing apparatus including:
an obtaining module 10, configured to obtain system information sent by each salt-anion client, where the system information includes a CPU utilization rate, a memory utilization rate, a disk utilization rate, and a network card bandwidth, and a monitoring analysis unit of the salt-anion client obtains the system information at regular time and feeds back the system information;
a monitoring analysis module 20, configured to determine whether the salt-termination client meets a preset alarm condition based on the system information;
and the output module 30 is configured to output an alarm prompt message based on a target client if it is determined that the target client meeting a preset alarm condition exists in the salt-termination client.
The methods executed by the program units can refer to the embodiments of the data processing method of the present invention, and are not described herein again.
The invention also provides a computer readable storage medium.
The computer-readable storage medium of the present invention has stored thereon a data processing program which, when executed by a processor, implements the steps of the data processing method as described above.
The method implemented when the data processing program running on the processor is executed may refer to each embodiment of the data processing method of the present invention, and details are not described here.
Furthermore, an embodiment of the present invention further provides a computer program product, which includes a data processing program, and when the data processing program is executed by a processor, the data processing program implements the steps of the data processing method described above.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A data processing method, characterized in that the data processing method comprises the steps of:
the method comprises the steps that a salt-master server side obtains system information sent by each salt-termination client side, wherein the system information comprises CPU utilization rate, memory utilization rate, disk utilization rate and network card bandwidth, and a monitoring analysis unit of the salt-termination client side obtains the system information at regular time and feeds the system information back to the salt-master server side;
a monitoring analysis module of the salt-master server side determines whether the salt-termination client side meets a preset alarm condition or not based on the system information;
and if the salt-mining client side is determined to have a target client side which reaches the preset alarm condition, the monitoring analysis module outputs alarm prompt information based on the target client side.
2. The data processing method of claim 1, wherein the step of determining whether the salt-termination client side reaches a preset alarm condition based on the system information by the monitoring analysis module of the salt-master server side comprises:
the monitoring analysis module of the salt-master server side determines whether the CPU utilization rate reaches a preset CPU utilization rate, whether the memory utilization rate reaches a preset memory utilization rate, whether the disk utilization rate reaches a preset disk utilization rate or whether the network card bandwidth reaches a preset bandwidth;
if the CPU utilization rate reaches a preset CPU utilization rate, the memory utilization rate reaches a preset memory utilization rate, the disk utilization rate reaches a preset disk utilization rate or the network card bandwidth reaches a preset bandwidth, the monitoring analysis module of the salt-master server side determines that the salt-termination client side reaches a preset alarm condition.
3. The data processing method of claim 1, wherein the data processing method further comprises:
if the system information sent by the first salt-minus client side is not received within the first preset duration, the salt-master server side sets the state of the first salt-minus client side to be an unavailable state.
4. The data processing method of claim 1, wherein the step of determining whether the salt-termination client side reaches a preset alarm condition based on the system information by the monitoring analysis module of the salt-master server side comprises:
if a second salt-emergence client in an unavailable state exists in the salt-emergence clients, the monitoring analysis module sets the state of the second salt-emergence client to be an online state;
and the monitoring analysis module determines whether the salt-mining client side reaches a preset alarm condition or not based on the system information.
5. The data processing method of claim 1, wherein the data processing method further comprises:
a content management module of a salt-master server side receives a task request sent by a management platform of the salt-master server side, wherein the task request comprises a content management type and a resource address;
the content management module determines a target salt-emergence client corresponding to the task request in each salt-emergence client based on the state information of the salt-emergence client;
the content management module sends a target task to the target salt-mining client based on the content management type and the resource address, wherein when the target task is received, the target salt-mining client executes the target task through a content management unit and feeds back an operation result corresponding to the target task to a management platform of a salt-master server;
and the management platform displays the operation result.
6. The data processing method of claim 5, wherein after the step of the content management module sending a target task to the target salt-mining client based on the content management type and the resource address, the method further comprises:
if the operation result fed back by the target salt-mining client is not received within a second preset time length, the content management module updates the retry times;
if the retry times do not reach the preset retry times, returning to the step of executing the content management module to send the target task to the target salt-termination client based on the content management type and the resource address;
and if the retry times reach the preset retry times, the content management module outputs alarm information.
7. The data processing method according to any one of claims 1 to 6, wherein after the step of outputting an alarm prompt message by the monitoring analysis module if it is determined that the salt-termination client has reached a preset alarm condition, the method further comprises:
the monitoring analysis module acquires state information of the salt-mining client, wherein the state information comprises an online state and an unavailable state;
the monitoring analysis module sends the system information and the state information to a management platform of a salt-master server;
and the management platform displays the system information and the state information of each salt-termination client.
8. A data processing apparatus, applied to a salt-master server, the data processing apparatus comprising:
the system comprises an acquisition module, a monitoring analysis unit and a feedback module, wherein the acquisition module is used for acquiring system information sent by each salt-emergence client, the system information comprises CPU (Central processing Unit) utilization rate, memory utilization rate, disk utilization rate and network card bandwidth, and the monitoring analysis unit of the salt-emergence client acquires the system information at regular time and feeds back the system information;
the monitoring analysis module is used for determining whether the salt-termination client side reaches a preset alarm condition based on the system information;
and the output module is used for outputting alarm prompt information based on the target client if the target client which reaches the preset alarm condition exists in the salt-termination client.
9. A data processing apparatus, characterized in that the data processing apparatus comprises: memory, processor and data processing program stored on the memory and executable on the processor, which when executed by the processor implements the steps of the data processing method according to any one of claims 1 to 7.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the data processing method according to any one of claims 1 to 7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110816540.7A CN113542424B (en) | 2021-07-19 | 2021-07-19 | Data processing method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110816540.7A CN113542424B (en) | 2021-07-19 | 2021-07-19 | Data processing method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113542424A true CN113542424A (en) | 2021-10-22 |
| CN113542424B CN113542424B (en) | 2023-04-28 |
Family
ID=78100287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110816540.7A Active CN113542424B (en) | 2021-07-19 | 2021-07-19 | Data processing method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113542424B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114584600A (en) * | 2022-01-20 | 2022-06-03 | 国网青海省电力公司 | Data audit monitoring system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101741885A (en) * | 2008-11-19 | 2010-06-16 | 珠海市西山居软件有限公司 | Distributed system and method for processing task flow thereof |
| US20190179726A1 (en) * | 2016-12-08 | 2019-06-13 | Tencent Technology (Shenzhen) Company Limited | Monitoring method and apparatus of server, and storage medium |
| CN110389822A (en) * | 2019-07-29 | 2019-10-29 | 北京金山云网络技术有限公司 | The node scheduling method, apparatus and server of execution task |
| CN110636088A (en) * | 2018-06-21 | 2019-12-31 | 视联动力信息技术股份有限公司 | Data monitoring method, data transmission method, device and equipment based on video link |
| CN111611131A (en) * | 2020-05-18 | 2020-09-01 | 中移（杭州）信息技术有限公司 | Saltstack-based operation and maintenance method, device, system and storage medium |
| US20210014819A1 (en) * | 2019-07-12 | 2021-01-14 | Charter Communications Operating, Llc | Wi-fi access point coordinated transmission of data |
-
2021
- 2021-07-19 CN CN202110816540.7A patent/CN113542424B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101741885A (en) * | 2008-11-19 | 2010-06-16 | 珠海市西山居软件有限公司 | Distributed system and method for processing task flow thereof |
| US20190179726A1 (en) * | 2016-12-08 | 2019-06-13 | Tencent Technology (Shenzhen) Company Limited | Monitoring method and apparatus of server, and storage medium |
| CN110636088A (en) * | 2018-06-21 | 2019-12-31 | 视联动力信息技术股份有限公司 | Data monitoring method, data transmission method, device and equipment based on video link |
| US20210014819A1 (en) * | 2019-07-12 | 2021-01-14 | Charter Communications Operating, Llc | Wi-fi access point coordinated transmission of data |
| CN110389822A (en) * | 2019-07-29 | 2019-10-29 | 北京金山云网络技术有限公司 | The node scheduling method, apparatus and server of execution task |
| CN111611131A (en) * | 2020-05-18 | 2020-09-01 | 中移（杭州）信息技术有限公司 | Saltstack-based operation and maintenance method, device, system and storage medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114584600A (en) * | 2022-01-20 | 2022-06-03 | 国网青海省电力公司 | Data audit monitoring system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113542424B (en) | 2023-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112272239B (en) | Edge calculation method and device, intelligent terminal and computer readable storage medium | |
| CN107645529B (en) | Heartbeat packet sending method and device | |
| CN110311983B (en) | Service request processing method, device and system, electronic equipment and storage medium | |
| CN111800443B (en) | Data processing system and method, device and electronic equipment | |
| CN103516690A (en) | Service processing state information query method and apparatus thereof | |
| CN112118315A (en) | Data processing system, method, device, electronic equipment and storage medium | |
| EP3076290B1 (en) | Method and device for remotely updating application program | |
| CN108337127B (en) | Application performance monitoring method, system, terminal and computer readable storage medium | |
| CN107908487B (en) | Task control management method, device and equipment and computer readable storage medium | |
| CN108200132B (en) | Resource acquisition method, device, equipment and computer readable storage medium | |
| US10404568B2 (en) | Agent manager for distributed transaction monitoring system | |
| CN109428926B (en) | Method and device for scheduling task nodes | |
| CN113162848A (en) | Method, device, gateway and medium for realizing block chain gateway | |
| CN104750544A (en) | Process management system and process management method applied to distributed system | |
| CN109788251B (en) | Video processing method, device and storage medium | |
| CN103716230A (en) | Message sending method, device and server | |
| CN113542424B (en) | Data processing method, device, equipment and readable storage medium | |
| EP3373608B1 (en) | Method and apparatus for acquiring resources | |
| CN106657195B (en) | Task processing method and relay device | |
| CN109413180B (en) | Data acquisition method, system and equipment and storage medium | |
| CN103685491A (en) | Application service providing method, system and related equipment | |
| CN113094131A (en) | Prompt resource display method, device, terminal, server and storage medium | |
| CN114138895A (en) | Data synchronization method and device for multiple data sources, computer equipment and storage medium | |
| CN110247808B (en) | Information transmission method, device, equipment and readable storage medium | |
| CN113271228A (en) | Bandwidth resource scheduling method, device, equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |