CN109560949B

CN109560949B - Data processing method, management server and business equipment

Info

Publication number: CN109560949B
Application number: CN201710883017.XA
Authority: CN
Inventors: 彭放
Original assignee: Hangzhou Hikvision System Technology Co Ltd
Current assignee: Hangzhou Hikvision System Technology Co Ltd
Priority date: 2017-09-26
Filing date: 2017-09-26
Publication date: 2023-01-03
Anticipated expiration: 2037-09-26
Also published as: CN109560949A

Abstract

The embodiment of the application provides a data processing method, a management server and service equipment. The method comprises the following steps: determining the data volume to be acquired according to the remaining resource volume of the message middleware, acquiring the data volume from the message middleware, wherein the data volume is the data volume of the data to be acquired, and processing the acquired data, and the message middleware is used for storing the data received from each service device. By applying the scheme provided by the embodiment of the application, the accumulation of the data to be processed in the management server can be avoided, and further abnormal conditions such as data loss or system faults are avoided.

Description

Data processing method, management server and business equipment

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a data processing method, a management server, and a service device.

Background

The operation and maintenance management of the service equipment refers to a management method for acquiring data related to the conditions of a CPU, a memory, a disk, a network and the like during service operation from the service equipment during service operation, monitoring, evaluating and the like the data, and timely dealing with abnormal conditions according to processing results to avoid service faults. In the service devices providing various service services, the data related to the service devices are dynamically changed, and thus the data can be acquired at different times.

In general, the management server may directly obtain data from the service device and process the data, so as to perform operation and maintenance management on the service device according to a processing result. However, when the number of the service devices is large and the data amount of the service devices is large, the management server may have abnormal situations such as data loss or system failure caused by accumulation of data to be processed.

Disclosure of Invention

An object of the embodiments of the present application is to provide a data processing method, a management server, and a service device, so as to avoid accumulation of data to be processed in the management server, and further avoid abnormal situations such as data loss or system failure. The specific technical scheme is as follows.

The embodiment of the application provides a data processing method, which comprises the following steps:

determining the data volume to be acquired according to the self residual resource volume; wherein the resource amount is a resource amount for processing data;

acquiring data with the data volume to be acquired from the message middleware; the message middleware is used for storing data received from each service device;

and processing the acquired data.

Optionally, the step of obtaining the data volume from the message middleware as the data of the data volume to be obtained includes:

acquiring data with the data volume determined by the message middleware according to the priority corresponding to the data type as the data of the data volume to be acquired from the message middleware; alternatively, the first and second electrodes may be,

and acquiring the data volume determined by the message middleware according to the time information of the received data from the message middleware as the data of the data volume to be acquired.

Optionally, before acquiring the data with the data size to be acquired from the message middleware, the method further includes:

and sending a data acquisition instruction to each service device, wherein the data acquisition instruction is used for instructing each service device to send the data of each service device to the message middleware.

Optionally, the data obtaining instruction is configured to instruct each service device to send data of each service device to the message middleware when receiving the data obtaining instruction; alternatively, the first and second electrodes may be,

and the data acquisition instruction is used for indicating each service device to send the data of each service device to the message middleware when the preset sending condition is met.

Optionally, the data acquisition instruction carries a command line instructing each service device to call by using a preset program; different command lines are used to obtain data of different data types.

Optionally, the command line includes a command line of the operating system and a command line of the non-operating system;

the operating system is provided with a command line for acquiring data representing equipment information of the service equipment;

and the command line of the non-operating system is used for acquiring data of client information representing the operation of the service equipment.

receiving a data acquisition instruction sent by a management server;

acquiring own data according to the data acquisition instruction;

sending the acquired data to a message middleware to enable the management server to acquire data with data volume to be acquired from the message middleware, wherein the data volume to be acquired is as follows: and the management server determines the data volume according to the residual resource volume of the management server, wherein the resource volume is the resource volume for processing data.

Optionally, the step of acquiring the data of the user according to the data acquisition instruction includes:

when the data acquisition instruction is received, acquiring own data; alternatively, the first and second electrodes may be,

the data acquisition instruction carries a preset sending condition, and when the preset sending condition is met, the data acquisition instruction acquires data of the data acquisition instruction.

Optionally, the data acquisition instruction carries a target command line corresponding to the type of the data to be acquired, and different command lines are used for acquiring data of different data types; the step of acquiring the data of the user according to the data acquisition instruction comprises the following steps:

and calling the target command line by adopting a preset program to obtain the data of the data type to be acquired.

Optionally, the step of calling the target command line by using a preset program to obtain data of a data type to be acquired includes:

when the target command line is a command line of the operating system, calling the target command line by adopting a preset program to obtain data representing equipment information of the service equipment;

and when the target command line is not a command line of the operating system, calling the target command line by adopting a preset program to obtain data representing client information operated by the service equipment.

The embodiment of the application provides a management server, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for finishing mutual communication through the communication bus by the memory;

the processor is used for determining the data volume to be acquired according to the residual resource volume of the processor, acquiring data with the data volume as the data volume to be acquired from the message middleware, and processing the acquired data; wherein the resource amount is a resource amount for processing data; the message middleware is used for storing data received from each service device;

the memory is used for storing the acquired data.

Optionally, the processor is specifically configured to obtain, from a message middleware, data whose data volume determined by the message middleware according to a priority corresponding to a data type is the data of the data volume to be obtained; alternatively, the first and second electrodes may be,

the processor is specifically configured to acquire, from a message middleware, data whose data volume determined by the message middleware according to time information of received data is the data of the data volume to be acquired.

Optionally, the processor is further configured to send a data obtaining instruction to each service device before obtaining the data with the data amount to be obtained from the message middleware, where the data obtaining instruction is used to instruct each service device to send the data of each service device to the message middleware.

Optionally, the command line includes a command line carried by the operating system and a command line carried by the non-operating system;

The embodiment of the application provides business equipment, which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory finish mutual communication through the communication bus;

the processor is configured to receive a data acquisition instruction sent by the management server, acquire data of the management server according to the data acquisition instruction, and send the acquired data to the message middleware, so that the management server acquires data with a data volume to be acquired from the message middleware, where the data volume to be acquired is: the management server determines the data volume according to the residual resource volume of the management server, wherein the resource volume is used for processing data;

the memory is used for storing the acquired data.

Optionally, the processor is specifically configured to obtain data of the processor when the data obtaining instruction is received; alternatively, the first and second electrodes may be,

the processor is specifically configured to obtain the data of the processor when the data obtaining instruction carries a preset sending condition and the preset sending condition is met.

Optionally, the data acquisition instruction carries a target command line corresponding to the type of the data to be acquired, and different command lines are used for acquiring data of different data types;

the processor is specifically configured to call the target command line by using a preset program to obtain data of a data type to be acquired.

Optionally, when the target command line is a command line of the operating system, the processor is specifically configured to call the target command line by using a preset program to obtain data indicating device information of the service device;

when the target command line is not a command line of the operating system, the processor is specifically configured to call the target command line by using a preset program to obtain data indicating client information of service equipment running.

The embodiment of the application provides a computer-readable storage medium, a computer program is stored in the computer-readable storage medium, and the computer program is executed by a processor to realize the data processing method provided by the embodiment of the application. The data processing method comprises the following steps:

and processing the acquired data.

An embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements a data processing method provided in an embodiment of the present application. The data processing method comprises the following steps:

receiving a data acquisition instruction sent by a management server;

acquiring own data according to the data acquisition instruction;

The data processing method, the management server and the business equipment provided by the embodiment of the application can enable each business equipment to send data to the message middleware, and the message middleware stores the data of each business equipment. The management server obtains a certain data volume from the message middleware according to the remaining resource volume for processing data. When the data volume of the service equipment is large, the data is stored in the message middleware, and the management server does not generate data accumulation. Therefore, the embodiment of the application can avoid data accumulation in the management server, and further avoid abnormal conditions such as data loss or system faults. Of course, not all advantages described above need to be achieved at the same time in the practice of any one product or method of the present application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.

Fig. 1 is a schematic flowchart of a data processing method according to an embodiment of the present application;

fig. 2a and fig. 2b are schematic diagrams of two application scenarios provided in the embodiment of the present application;

FIG. 3a is a schematic diagram of an interaction between a management server, a service device, and message middleware;

fig. 3b and fig. 3c are schematic diagrams of two kinds of flows respectively for instructing, by the management client, the agent to send data to the message middleware, according to an embodiment of the present application;

fig. 4 is a schematic flowchart of another data processing method according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a management server according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a service device according to an embodiment of the present application.

Detailed Description

The technical solution in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the described embodiments are merely a few embodiments of the present application and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making creative efforts shall fall within the protection scope of the present application.

The embodiment of the application provides a data processing method, a management server and service equipment, which can avoid data accumulation in the management server and further avoid abnormal conditions such as data loss or system faults. The present application will be described in detail below with reference to specific examples.

Fig. 1 is a schematic flowchart of a data processing method according to an embodiment of the present application. The method is applied to the management server. The management server may be a stand-alone server or a distributed server, which is not specifically limited in this application. The management server may have a management client installed therein, and the method provided in this embodiment may be specifically executed by the management client.

The data in this embodiment may be used for operation and maintenance management, and may also be used for other management, which is not specifically limited in this application. The operation and maintenance management of the service equipment refers to a management method for acquiring data related to the conditions of a CPU, a memory, a disk, a network and the like during service operation from the service equipment during service operation, monitoring, evaluating and the like the data, and timely dealing with abnormal conditions according to processing results to avoid service faults. When the data is used for operation and maintenance management, the management client in the management server may be an operation and maintenance management client. The method comprises the following steps S101 to S103:

step S101: and determining the data volume to be acquired according to the self residual resource volume.

Wherein the resource amount is an amount of resources used for processing data. The amount of remaining resources may be understood as the amount of processable data. The processable data amount may be a data amount determined according to a remaining CPU utilization.

In this step, the management server may detect the remaining CPU utilization of itself in real time, and determine the remaining resource amount corresponding to the remaining CPU utilization according to the correspondence between the preset remaining resource amount and the remaining CPU utilization. Wherein, the corresponding relation between the residual resource amount and the residual CPU utilization rate can be predetermined according to the empirical value. For example, when the remaining CPU utilization is 20%, the corresponding remaining resource amount is 50M data processable.

In this step, before determining the amount of data to be acquired, the management server may further determine whether the amount of remaining resources is greater than a preset minimum resource amount threshold, and if so, perform the step of determining the amount of data to be acquired according to the amount of remaining resources of the management server.

For example, the current remaining resource amount is 6MB of data that can be processed, and when the preset minimum resource amount threshold is 10MB of data, the remaining resource amount is not greater than the preset minimum resource amount threshold, and the step of determining the data amount to be acquired according to the remaining resource amount may not be performed. When the preset minimum resource amount threshold is 5MB of data, the remaining resource amount is greater than the preset minimum resource amount threshold, and at this time, the step of determining the data amount to be acquired according to the remaining resource amount of the step may be executed. In this way, it is possible to avoid determining the amount of data to be acquired too frequently.

When determining the data volume to be acquired according to the remaining resource volume of the user, determining the data volume to be acquired by multiplying the remaining resource volume of the user by a preset ratio may be performed. Wherein the preset proportion is a numerical value which is larger than 0 and smaller than 1. For example, if the remaining resource amount is 50MB of data that can be processed, and the preset proportion is 80%, it may be determined that the amount of data to be acquired is 40MB. Therefore, the full-load work of the management server can be avoided, and the probability of system faults of the management server is reduced.

Step S102: and acquiring data with the data volume to be acquired from the message middleware.

The message middleware is used for storing the data received from each service device. The message middleware can be understood as a client. The message middleware can be open-source message middleware such as ActiveMQ, rabbitMQ, kafka and the like, and can also be non-open-source message middleware.

The service device may be a device such as a general computer or a server, or any other terminal device that can provide a service. The business equipment can provide various business services, for example, cloud storage services, intelligent analysis and other businesses can be provided. The data transmitted by the service device to the message middleware may be various data, for example, data representing device information of the service device, or data representing client information operated by the service device.

The data representing the device information of the service device may include data related to a CPU, a memory, a disk, a network status, an Internet Protocol (IP) address, and the like, for example, a CPU utilization rate, a memory utilization rate, a disk remaining space, a system operating state, a system operating log, a network bandwidth, a network delay, and the like. The data representing the client information of the service device may include monitoring data, alarm data, process data, client operation state, client internal operation data, and the like.

In this step, when the data with the data amount to be acquired is acquired from the message middleware, specifically, the acquisition request is sent to the message middleware, and the data with the data amount to be acquired sent by the message middleware is received. The obtaining request is used for obtaining the data with the data volume to be obtained. When receiving the acquisition request, the message middleware may acquire data with a data amount equal to the data amount to be acquired from the message middleware, and send the acquired data to the management server.

The message middleware in this embodiment may be installed in the management server, or may be installed in an electronic device other than the management server. Fig. 2a and fig. 2b are schematic diagrams of two application scenarios of the present embodiment.

Fig. 2a includes a plurality of service devices, a management server, and a message middleware, where the message middleware is installed in the management server, the management server further includes a management client, and the service devices include a cloud server, a common server, and a terminal device. When the data with the data volume to be acquired is acquired from the message middleware, the management client can send an acquisition request to the message middleware; the message middleware receives an acquisition request sent by a management client, acquires data with data volume to be acquired, and sends the acquired data to the management client; and the management client receives the data sent by the message middleware.

Fig. 2b includes a plurality of service devices, a management server, and an electronic device including a message middleware, where the management server includes a management client, and the service devices include a cloud server, a general server, and a terminal device. When the data volume of which is the data volume to be acquired is acquired from the message middleware, the management server may send an acquisition request to the message middleware; the message middleware receives an acquisition request sent by a management client, acquires data with data volume to be acquired, and sends the acquired data to a management server; the management server receives the data sent by the message middleware.

Step S103: and processing the acquired data.

In this embodiment, when the acquired data is processed, the acquired data may be analyzed, detected, and evaluated to obtain a processing result, and the processing result is displayed by using an image interface, so that a user can see the processing result. The processing result may include information such as an alarm. The above processing manner is only a part of the embodiments, and the specific manner of processing the data in the present application is not limited.

As can be seen from the above, in this embodiment, each service device can send data to the message middleware, and the message middleware stores the data of each service device. The management server obtains a certain data volume from the message middleware according to the remaining resource volume for processing data. When the data volume of the service equipment is large, the data is stored in the message middleware, and the management server does not generate data accumulation. Therefore, the embodiment can avoid data accumulation in the management server, and further avoid abnormal conditions such as data loss or system faults.

In another embodiment of the present application, in step S102 in the embodiment shown in fig. 1, the step of acquiring the data with the data size equal to the data size to be acquired from the message middleware may specifically include the following implementation manners:

in a first mode, the data volume determined by the message middleware according to the priority corresponding to the data type is acquired from the message middleware and is the data of the data volume to be acquired.

In this embodiment, the message middleware determines the data volume as the data of the data volume to be acquired according to the priority corresponding to the data type. Specifically, the message middleware may determine, according to the order from the high priority to the low priority, the data amount as the data amount to be acquired from the data corresponding to each data type stored in the message middleware. The method can ensure that the data with high priority is processed preferentially, and improve the rationality in data processing.

For example, the message middleware stores data of multiple data types, such as basic data, monitoring data, disk data, network condition data, alarm data, and the like, and the priority of the data types is, from high to low, respectively: alarm data, basic data, disk data, monitoring data and network condition data. When the message middleware determines that the data volume is the data of the data volume to be acquired, judging whether the data volume of the alarm data is larger than the data volume to be acquired or not, if not, determining partial data from the basic data for supplement, if the acquired data volume does not reach the data volume to be acquired, acquiring partial data from the disk data, and acquiring data from the data corresponding to the data type in sequence according to the rule until the acquired data volume reaches the data volume to be acquired.

The data type may also be a data type divided according to a service type, for example, the data type may include data of a cloud server, data analyzed intelligently, and the like, and different data types correspond to different priorities. The data type may also be a data type divided according to the service device identifier, for example, the data type may include data of the service device 1, data of the service device 2, data of the service device 3, and the like, and different data types correspond to different priorities.

The priority corresponding to each data type may be preset.

And acquiring data with the data volume determined by the message middleware according to the time information of the received data from the message middleware as the data of the data volume to be acquired.

In this embodiment, the message middleware determines, according to the time information of the received data, that the data amount is the data of the data amount to be acquired. Each service device can send data to the message middleware for multiple times according to a certain rule, and the message middleware can record time information of receiving data when receiving the data sent by each service device. Specifically, the message middleware may determine, from data corresponding to each data type stored in the message middleware, the data size as data of the data size to be acquired, in the order from morning to evening of the time of receiving the data. The data can be processed according to the time sequence in this way, and the situation that part of the data is not processed for a long time is avoided.

Therefore, in this embodiment, data determined according to the priority corresponding to the data type may be obtained, and the purpose of preferentially processing the corresponding data according to the corresponding priority rule is achieved. The embodiment can also acquire data determined according to the time information of the received data, and achieve the purpose of processing the corresponding data according to the corresponding time rule.

In another embodiment of the present application, in the embodiment shown in fig. 1, before step S102, that is, before acquiring data with a data size equal to the data size to be acquired from the message middleware, the following steps may be further included:

In an embodiment, the data obtaining instruction may be configured to instruct each service device to send data of each service device to the message middleware when receiving the data obtaining instruction.

In this embodiment, the management server may send the data acquisition instruction to each service device according to a preset sending cycle. And when the service equipment receives the data acquisition instruction sent by the management server, immediately sending the data of the service equipment to the message middleware.

Therefore, in this embodiment, the management server may send the data acquisition instruction to the service device as needed, and the service device reports the data when receiving the data acquisition instruction, thereby improving the accuracy of reporting the data.

In another embodiment, the data obtaining instruction may be configured to instruct each service device to send data of each service device to the message middleware when a preset sending condition is met.

In this embodiment, the preset transmission condition may be at least one of the following conditions: when a preset sending period comes; when the designated numerical value reaches a preset numerical value threshold value; and when the data volume reaches a preset data volume threshold value. The data acquisition instruction may carry at least one of the following information: presetting a sending condition; presetting a numerical threshold; and presetting a data quantity threshold.

The preset sending period, the preset value threshold and the preset data quantity threshold can be values predetermined according to actual conditions.

Therefore, in this embodiment, after the management server can send the data acquisition instruction once, the service device continuously sends data to the message middleware according to the preset sending condition, so that the service device actively reports the data, and the operation frequency of the management server is reduced.

In another embodiment of the present application, the data obtaining instruction may carry a command line instructing each service device to call by using a preset program. Wherein different command lines are used to obtain data of different data types.

In particular, the command lines may include command lines native to the operating system and command lines not native to the operating system. The command line of the operating system is used for acquiring data representing equipment information of the service equipment; and the command line of the non-operating system is used for acquiring data representing the client information operated by the service equipment. The operating system installed in the service device may be a Unix, linux, windows, or other operating system.

The data representing the device information of the service device may include data related to a CPU, a memory, a disk, a network status, an IP address, and the like, for example, a CPU utilization rate, a memory utilization rate, a disk remaining space, a system operating state, a system operating log, a network bandwidth, a network delay, and the like. The data representing the client information of the service device may include monitoring data, alarm data, process data, client operation state, client internal operation data, and the like.

When a client of a large project runs, a large amount of data related to the client, namely client information, is generated. The type of data that can be obtained by the command line of the operating system is very limited, and the client information cannot be obtained. In order to maintain the client, more client information needs to be acquired.

In this embodiment, the command line of the non-operating system may be a user-defined command line, that is, an executable program written in a preset programming language. The preset programming language can be a programming language such as shell, python, C or C + +. The command line of the non-operating system is self-defined, and the corresponding command line can be compiled according to the data to be acquired. Thus, the non-operating system's own command line may also be used to obtain data representing device information for the business device. Therefore, more data about the client can be acquired by adopting the custom command line, the running condition of the client can be known in more detail, and the operation and maintenance effect on the client is optimized. For example, when the processing pressure of the client is too high or abnormal, the management server can determine the result from the acquired data in time and display the result to the technician, so that the technician can process the problem in time and avoid hanging the project.

The management server may transmit the respective command lines to the service device in advance so that the service device stores the respective command lines.

When the service device acquires data, the command line may be called by using a preset program to acquire the data. The preset program may be an agent program pre-installed in each service device. The preset program may be an agent program written in C, C + +, java, shell, or python programming languages, may be resident in a memory of the service device to run, and may send data acquired when the command line is run to the message middleware. The preset program can also receive a data acquisition instruction sent by the management server, operate a specified command line according to the data acquisition instruction, and send the acquired data to the message middleware.

Therefore, in this embodiment, the data acquisition instruction may carry a command line instructing each service device to use a preset program to call, so as to acquire data of different data types, and this way may prevent the service device from sending data that is not needed by the management server to the message middleware, and improve accuracy of the management server in acquiring data.

As an example, fig. 3a provides a schematic diagram of interaction among a management server, a service device, and message middleware. The management server can send a data acquisition instruction to the service equipment when data needs to be acquired, and the service equipment acquires data of the service equipment and sends the acquired data to the message middleware when receiving the data acquisition instruction sent by the management server. The management server sends an acquisition request which is used for acquiring the data of the business equipment and carries the data volume to be acquired to the message middleware, and when the message middleware receives the acquisition request, the message middleware determines the data volume as the data of the data volume to be acquired and sends the determined data to the management server. And the management server receives the data sent by the message middleware and processes the data.

As another example, FIG. 3b provides a flow diagram for a management client instructing an agent to send data to messaging middleware. The management client is an operation and maintenance system, the agent program is a preset program, and the agent program and the command line are both located in the business equipment. Step S1, the management client sends a data acquisition instruction to the agent program, wherein the data acquisition instruction carries a command line required to be called by the agent program and indicates the agent program to call the command line immediately after receiving the data acquisition instruction. And S2, when receiving the data acquisition instruction sent by the management client, the agent program immediately calls a command line carried in the data acquisition instruction. And step S3, the agent program immediately sends the data returned by the command line to the message middleware. And S4, the management client acquires data from the message middleware, namely the message middleware sends the data to the management client. And S5, the management client displays the acquired data to the user. In this example, the management client actively sends a command line to be called by the agent to the service device, and the agent passively calls the command line and sends the acquired data to the message middleware.

As another example, FIG. 3c provides a flow diagram for a management client instructing an agent to send data to messaging middleware. The management client is an operation and maintenance system, the agent program is a preset program, and the agent program and the command line are both located in the business equipment. Step S1, the management client sends a data acquisition instruction to the agent program, and the data acquisition instruction indicates the agent program to call a command line regularly. And S2, the agent program calls the command line at regular time when receiving the data acquisition instruction sent by the management client. And step S3, the agent program sends the data returned by the command line to the message middleware at regular time. And S4, the management client acquires data from the message middleware, namely the message middleware sends the data to the management client. And S5, the management client displays the acquired data to the user. In this example, the management client actively sends a command line that needs to be called by the agent program at regular time to the service device, and the agent program actively calls the command line at regular time and sends the acquired data to the message middleware.

Fig. 4 is another data processing method according to an embodiment of the present disclosure. The method is applied to business equipment. The service device may be a device such as a general computer or a server, or any other terminal device that can provide a service. The business equipment can provide various business services, for example, cloud storage services, intelligent analysis and other businesses can be provided. The business equipment can be provided with a client, and the business equipment can provide various business services by operating the client. The method may include the following steps S401 to S403:

step S401: and receiving a data acquisition instruction sent by the management server.

Step S402: and acquiring the data of the user according to the data acquisition instruction.

The data of the service device may include various data, for example, data representing device information of the service device, or data representing client information of the service device.

The data representing the device information of the service device may include data related to a CPU, a memory, a disk, a network status, an IP address, and the like, for example, CPU utilization, memory utilization, disk remaining space, a system operating state, a system operating log, a network bandwidth, a network delay, and the like. The data representing the client information of the service device may include monitoring data, alarm data, process data, client operation state, client internal operation data, and the like.

Step S403: sending the acquired data to a message middleware to enable a management server to acquire data with data volume to be acquired from the message middleware, wherein the data volume to be acquired is as follows: the management server determines the data amount according to the remaining resource amount of the management server, wherein the resource amount is the resource amount used for processing the data.

When sending the acquired data to the message middleware, the data type of the data may be sent to the message middleware together. The data type can be basic data, monitoring data, disk data, network condition data, alarm data and other data types. The data type may also be a data type divided according to a service type, for example, the data type may include data of a cloud server, data of intelligent analysis, and the like. The data type may also be a data type divided according to the service device identifier, for example, the data type may include data of the service device 1, data of the service device 2, data of the service device 3, and the like.

When the data type of the data is sent to the message middleware, the message middleware may determine the data volume as the data of the data volume to be acquired according to the priority corresponding to the data type.

In a specific embodiment, when sending the acquired data to the Message middleware, the acquired data may be sent to the Message middleware by using an Advanced Message Queuing Protocol (AMQP), a Message Queuing Telemetry Transport (MQTT) Protocol, an open connection (OpenWire) Protocol, a Simple (Streaming) Text oriented Messaging Protocol (stmp), and the like.

As can be seen, in this embodiment, the data of the management server can be acquired according to the data acquisition instruction sent by the management server, and the data is sent to the message middleware, so that the management server acquires data with the data amount to be acquired from the message middleware. When the data volume of the service equipment is large, the data is stored in the message middleware, and the management server does not generate data accumulation. Therefore, the embodiment can avoid data accumulation in the management server, and further avoid abnormal conditions such as data loss or system faults.

In another embodiment of the present application, step S402 in the embodiment shown in fig. 4, that is, the step of acquiring the own data according to the data acquisition instruction, may specifically include the following implementation manners:

in the first mode, when a data acquisition instruction is received, own data is acquired.

In this embodiment, the data acquisition instruction may be a data acquisition instruction that is sent by the management server according to a preset sending cycle. And when the service equipment receives the data acquisition instruction sent by the management server, the service equipment immediately acquires the data of the service equipment, and when the data acquisition instruction is not received, the step of acquiring the data of the service equipment is not executed. The implementation mode can determine the time for acquiring the data more simply and accurately and improve the accuracy of the time for reporting the data.

And in the second mode, under the condition that the data acquisition instruction carries the preset sending condition, acquiring the data of the data acquisition instruction when the preset sending condition is met.

In this embodiment, after the management server can send the data acquisition instruction once, the service device continuously sends data to the message middleware according to the preset sending condition, so that the service device actively reports the data, and the operation frequency of the management server is reduced.

In order to enable the management server to obtain specific data more accurately, in another embodiment of the present application, in an embodiment shown in fig. 4, the data obtaining instruction may carry a target command line corresponding to a data type to be obtained, and different command lines are used to obtain data of different data types. Correspondingly, step S402 is a step of acquiring data of the mobile terminal according to the data acquisition instruction, and specifically may be:

In particular, the command lines may include command lines native to the operating system and command lines not native to the operating system. The operating system is provided with a command line for acquiring data representing equipment information of the service equipment; and the command line of the non-operating system is used for acquiring data representing the client information operated by the service equipment.

In this step, when the target command is a command line of the operating system, the preset program is used to call the target command line, and when the data of the data type to be acquired is obtained, the preset program may be used to call the target command line, so as to obtain data representing device information of the service device.

In the second mode, when the target command line is not a command line of the operating system, the preset program is used to call the target command line, and when the data of the data type to be acquired is obtained, the preset program may be used to call the target command line, so as to obtain data representing client information of the service device.

In this embodiment, the command line of the non-operating system may be a user-defined command line, that is, an executable program written in a preset programming language. The preset programming language can be a programming language such as Shell, python, C or C + +. The command line of the non-operating system is self-defined, and the corresponding command line can be compiled according to the data to be acquired. Thus, the command line of the non-operating system itself can also be used to obtain data representing device information of the service device. Therefore, more data about the client can be acquired by adopting the custom command line, the running condition of the client can be known in more detail, and the operation and maintenance effect on the client is optimized. For example, when the processing pressure of the client is too high or abnormal, the management server can determine the result from the acquired data in time and display the result to the technician, so that the technician can process the problem in time and avoid hanging the project.

The service device may receive each command line sent by the management server in advance, and store each command line.

The target command line may be a designated command line in a command line of the operating system itself, may also be a designated command line in a command line of the non-operating system itself, may also be a part of a command line of the operating system itself, and may also be a part of a command line of the non-operating system itself. This is not particularly limited in this application.

When the service device acquires data, a preset program can be used to call a target command line to acquire the data. The preset program may be an agent program pre-installed in each service device. The preset program may be an agent program written in C, C + +, java, shell, or python programming languages, may be resident in a memory of the service device to run, and may send data acquired when the command line is run to the message middleware. The preset program can also receive a data acquisition instruction sent by the management server, run a specified command line according to the data acquisition instruction, and send the acquired data to the message middleware.

As can be seen, in this embodiment, the data acquisition instruction may carry a target command line corresponding to the type of the data to be acquired, and when receiving the data acquisition instruction, the service device may call the target command line by using a preset program to obtain the data of the type of the data to be acquired. The method can avoid that the service equipment sends the data which is not needed by the management server to the message middleware, and improve the accuracy of the management server when acquiring the data.

Fig. 5 is a schematic structural diagram of a management server according to an embodiment of the present application. This embodiment corresponds to the embodiment of the method shown in fig. 1. The management server comprises a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 are communicated with each other through the communication bus 504;

the processor 501 is configured to determine a data amount to be acquired according to a remaining resource amount of the processor, acquire data of which the data amount is the data amount to be acquired from the message middleware, and process the acquired data; wherein the resource amount is a resource amount for processing data; the message middleware is used for storing data received from each service device;

and a memory 503 for storing the acquired data.

In another embodiment of the present application, the processor 501 in the embodiment shown in fig. 5 may be specifically configured to obtain, from a message middleware, data whose data volume determined by the message middleware according to a priority corresponding to a data type is the data of the data volume to be obtained; alternatively, the first and second electrodes may be,

the processor 501 may be specifically configured to acquire, from the message middleware, data whose data amount determined by the message middleware according to the time information of the received data is the data of the data amount to be acquired.

In another embodiment of the present application, the processor 501 in the embodiment shown in fig. 5 may be further configured to send a data obtaining instruction to each service device before obtaining, from the message middleware, data whose data amount is the data amount to be obtained, where the data obtaining instruction is used to instruct each service device to send data of each service device to the message middleware.

In another embodiment of the present application, in the embodiment shown in fig. 5, the data obtaining instruction may be used to instruct each service device to send data of each service device to the message middleware when receiving the data obtaining instruction; alternatively, the first and second electrodes may be,

the data acquisition instruction may be configured to instruct each service device to send data of each service device to the message middleware when a preset sending condition is met.

In another embodiment of the present application, in the embodiment shown in fig. 5, the data obtaining instruction carries a command line instructing each service device to call by using a preset program; different command lines are used to obtain data of different data types.

In another embodiment of the present application, the embodiment shown in FIG. 5, the command lines include command lines native to the operating system and command lines not native to the operating system;

Since the embodiment of the management server is obtained based on the embodiment of the method, and has the same technical effect as the method, the technical effect of the embodiment of the apparatus is not described herein again. For the embodiment of the management server, since it is basically similar to the embodiment of the method, it is described relatively simply, and the relevant points can be referred to the partial description of the embodiment of the method.

Fig. 6 is a schematic structural diagram of a service device according to an embodiment of the present application. This embodiment corresponds to the method embodiment shown in fig. 4. The service equipment comprises a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602 and the memory 603 complete mutual communication through the communication bus 604;

the processor 601 is configured to receive a data acquisition instruction sent by a management server, acquire data of the management server according to the data acquisition instruction, and send the acquired data to a message middleware, so that the management server acquires data with a data volume to be acquired from the message middleware, where the data volume to be acquired is: the management server determines the data volume according to the residual resource volume of the management server, wherein the resource volume is used for processing data;

the memory 603 is used for storing the acquired data.

In another embodiment of the present application, the processor 601 in the embodiment shown in fig. 6 may be specifically configured to obtain data of itself when receiving the data obtaining instruction; alternatively, the first and second electrodes may be,

the processor 601 may be specifically configured to enable the data obtaining instruction to carry a preset sending condition, and obtain data of the processor when the preset sending condition is met.

In another embodiment of the present application, in the embodiment shown in fig. 6, the data obtaining instruction carries a target command line corresponding to a data type to be obtained, and different command lines are used for obtaining data of different data types;

the processor 601 is specifically configured to call the target command line by using a preset program to obtain data of a data type to be acquired.

In another embodiment of the present application, in the embodiment shown in fig. 6, when the target command line is a command line of an operating system, the processor 601 may be specifically configured to call the target command line by using a preset program, to obtain data representing device information of a service device;

when the target command line is not a command line of the os, the processor 601 may be specifically configured to call the target command line by using a preset program to obtain data indicating client information of service device operation.

Since the device embodiment is obtained based on the method embodiment and has the same technical effect as the method, the technical effect of the device embodiment is not described herein again. For the apparatus embodiment, since it is substantially similar to the method embodiment, it is described relatively simply, and reference may be made to some descriptions of the method embodiment for relevant points.

The communication bus mentioned in the embodiments of the management server and the service device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the management server or the service equipment and other equipment.

The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

The embodiment of the application provides a computer readable storage medium. The computer readable storage medium stores therein a computer program, and when executed by a processor, the computer program implements a data processing method provided in an embodiment of the present application. The data processing method comprises the following steps:

and processing the acquired data.

Therefore, in this embodiment, each service device may send data to the message middleware, and the message middleware stores the data of each service device. The management server obtains a certain data volume from the message middleware according to the remaining resource volume for processing data. When the data volume of the service equipment is large, the data is stored in the message middleware, and the management server does not generate data accumulation. Therefore, the embodiment can avoid data accumulation in the management server, and further avoid abnormal conditions such as data loss or system faults.

The embodiment of the application provides another computer readable storage medium. The computer readable storage medium stores therein a computer program, which when executed by a processor implements another data processing method provided by an embodiment of the present application. The data processing method comprises the following steps:

receiving a data acquisition instruction sent by a management server;

acquiring own data according to the data acquisition instruction;

Therefore, in this embodiment, the data of the management server can be acquired according to the data acquisition instruction sent by the management server, and the data is sent to the message middleware, so that the management server acquires the data with the data volume to be acquired from the message middleware. When the data volume of the service equipment is large, the data is stored in the message middleware, and the management server does not generate data accumulation. Therefore, the embodiment can avoid data accumulation in the management server, and further avoid abnormal conditions such as data loss or system faults.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on differences from other embodiments.

The above description is only for the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method of data processing, the method comprising:

judging whether the self residual resource amount is larger than a preset minimum resource amount threshold value or not, and if so, determining the data amount to be acquired according to the self residual resource amount; wherein the resource amount is a resource amount for processing data;

sending a data acquisition instruction to each service device, wherein the data acquisition instruction is used for instructing each service device to send data of each service device to a message middleware when a preset sending condition is met, and the data comprises: the data acquisition instruction carries a command line which indicates each service device to call by adopting a preset program; different command lines are used for acquiring data of different data types;

and processing the acquired data.

2. The method according to claim 1, wherein the step of obtaining the data with the data amount to be obtained from the message middleware comprises:

3. The method of claim 1, wherein the command lines include command lines native to the operating system and command lines not native to the operating system;

4. A method of data processing, the method comprising:

receiving a data acquisition instruction sent by a management server;

the data acquisition instruction carries a preset sending condition, and when the preset sending condition is met, the data acquisition instruction acquires data of the data acquisition instruction, wherein the data includes: the data acquisition instruction carries a target command line corresponding to the type of the data to be acquired, and different command lines are used for acquiring data of different data types;

sending the acquired data to a message middleware to enable the management server to acquire data with data volume to be acquired from the message middleware, wherein the data volume to be acquired is as follows: and the management server determines the data volume according to the residual resource volume of the management server under the condition that the residual resource volume of the management server is larger than a preset minimum resource volume threshold, wherein the resource volume is the resource volume for processing data.

5. The method according to claim 4, wherein the step of acquiring the own data according to the data acquisition instruction comprises:

6. The method according to claim 5, wherein the step of calling the target command line by using a preset program to obtain the data of the data type to be acquired comprises:

7. A management server is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing the communication between the processor and the memory through the communication bus;

the processor is used for judging whether the self residual resource amount is larger than a preset minimum resource amount threshold value or not, if so, determining the data amount to be acquired according to the self residual resource amount, and sending a data acquisition instruction to each service device, wherein the data acquisition instruction is used for indicating each service device to send the self data of each service device to the message middleware when a preset sending condition is met; acquiring data with the data volume to be acquired from the message middleware, and processing the acquired data; wherein the resource amount is a resource amount for processing data; the message middleware is configured to store data received from each service device, where the data includes: the data acquisition instruction carries a command line which indicates each service device to call by adopting a preset program; different command lines are used for acquiring data of different data types;

the memory is used for storing the acquired data.

8. The business equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;

the processor is used for receiving a data acquisition instruction sent by the management server, the data acquisition instruction carries a preset sending condition, when the preset sending condition is met, self data is acquired, the acquired data is sent to the message middleware, so that the management server acquires data volume from the message middleware as data with the data volume to be acquired, and the data volume to be acquired is as follows: the management server determines a data volume according to the remaining resource volume of the management server when the remaining resource volume of the management server is greater than a preset minimum resource volume threshold, where the resource volume is a resource volume for processing data, and the data includes: the data acquisition instruction carries a target command line corresponding to the type of the data to be acquired, and different command lines are used for acquiring data of different data types;

the memory is used for storing the acquired data.