CN113688149A - Monitoring method and device - Google Patents
Monitoring method and device Download PDFInfo
- Publication number
- CN113688149A CN113688149A CN202110820270.7A CN202110820270A CN113688149A CN 113688149 A CN113688149 A CN 113688149A CN 202110820270 A CN202110820270 A CN 202110820270A CN 113688149 A CN113688149 A CN 113688149A
- Authority
- CN
- China
- Prior art keywords
- data
- monitoring
- storage space
- monitoring data
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/54—Presence management, e.g. monitoring or registration for receipt of user log-on information, or the connection status of the users
Abstract
The invention discloses a monitoring method and a monitoring device. Wherein, the method comprises the following steps: collecting monitoring data of server-side equipment, wherein the monitoring data comprises at least one of the following data: service data, state data of hardware devices and network traffic data; transmitting the monitoring data to a storage space in an asynchronous transmission mode, wherein the storage space stores the monitoring data in an aggregation storage mode; under the condition that a request instruction is received, retrieving monitoring data in a storage space based on the request instruction, and acquiring query data required by a predetermined function, wherein the predetermined function comprises at least one of the following functions: an alarm function, an analysis function, and a retrieval function. The invention solves the technical problem that the error information of the server is difficult to know in time in the related technology, thereby improving the user experience.
Description
Technical Field
The invention relates to the field of monitoring, in particular to a monitoring method and a monitoring device.
Background
The current software system can be divided into networking application and single machine version (desktop version) application according to whether the software system is connected with a network or not, wherein the single machine version application does not need to be connected with the network and is only used in the physical machine, such as a calculator carried by a Windows system; however, the current era is more application programs of the networking type, and the applications of the networking type can be divided into C-C (Client-Client) network applications and C-S (Client-Server) network applications.
As shown in fig. 1, in a Server, an application is generally deployed on a physical machine or a virtual machine, and then a port is opened to be called by the Client, where the Client may be a browser or an APP in a mobile phone. However, as the server cannot know whether the application program normally runs, the existing method is to check log records manually and periodically, check whether an error log exists, and then perform centralized processing and repair, or to query a code or a log to repair a related problem through a bug returned by a user, but the error information cannot be obtained in time no matter what method is used, so that the user experience is reduced.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a monitoring method and a monitoring device, which are used for at least solving the technical problem that error information of a server is difficult to know in time in the related technology, so that the user experience is improved.
According to an aspect of an embodiment of the present invention, there is provided a monitoring method, including: collecting monitoring data of server-side equipment, wherein the monitoring data comprises at least one of the following data: service data, state data of hardware devices and network traffic data; transmitting the monitoring data to a storage space in an asynchronous transmission mode, wherein the storage space stores the monitoring data in an aggregation storage mode; under the condition that a request instruction is received, retrieving monitoring data in a storage space based on the request instruction, and acquiring query data required by a predetermined function, wherein the predetermined function comprises at least one of the following functions: an alarm function, an analysis function, and a retrieval function.
Optionally, the collecting monitoring data of the server device includes: the method comprises the following steps of adopting a point burying tool to collect service data of a server device, wherein the service data is log data generated in the running process of an application program on the server device, and the point burying mode of the point burying tool comprises the following steps: active buried points and passive centralized buried points.
Optionally, the log data further comprises: delay data, error data, response time, SLA performance data, user access data for the application.
Optionally, the collecting monitoring data of the server device includes: an acquisition module that starts at least one hardware device, wherein the hardware device comprises: a processor and a memory; asynchronously collecting working data of at least one hardware device of a server device, wherein the working data comprises: state data of the processor and state data of the memory.
Optionally, the collecting monitoring data of the server device includes: and acquiring network flow data generated when the server equipment performs network transmission under a TCP (transmission control protocol) network protocol by monitoring task network signals in the network data channel.
Optionally, retrieving the monitoring data in the storage space based on the request instruction, and obtaining query data required by the predetermined function, includes: acquiring a configuration file carried in the request instruction, wherein at least one type of information as follows is configured in the configuration file: mail service address information, control interface information and access information of an application program; acquiring matched query data from the monitoring data information query in the storage space based on the configuration file, wherein the query data comprises at least one of the following data: mail, charts, security control instructions.
According to another aspect of the embodiments of the present invention, there is also provided a monitoring apparatus, including: the acquisition module is used for acquiring monitoring data of the server-side equipment, wherein the monitoring data comprises at least one of the following data: service data, state data of hardware devices and network traffic data; the transmission module is used for transmitting the monitoring data to a storage space in an asynchronous transmission mode, wherein the storage space stores the monitoring data in an aggregation storage mode; the processing module is used for retrieving the monitoring data in the storage space based on the request instruction and acquiring query data required by a predetermined function under the condition that the request instruction is received, wherein the predetermined function comprises at least one of the following functions: an alarm function, an analysis function, and a retrieval function.
Optionally, the acquisition module comprises: the first classification acquisition module is used for acquiring service data of the server equipment by adopting a point burying tool, wherein the service data is log data generated in the running process of an application program on the server equipment, and the point burying mode of the point burying tool comprises the following steps: active buried points and passive centralized buried points.
Optionally, the log data further comprises: delay data, error data, response time, SLA performance data, user access data for the application.
Optionally, the acquisition module comprises: a starting module for starting the acquisition module of at least one hardware device, wherein the hardware device comprises: a processor and a memory; the second classification acquisition module is used for asynchronously acquiring the working data of at least one hardware device of the server side equipment, wherein the working data comprises: state data of the processor and state data of the memory.
Optionally, the acquisition module comprises: and the third classification acquisition module is used for acquiring network flow data generated when the server equipment performs network transmission under a TCP network protocol by monitoring task network signals in the network data channel.
Optionally, the processing module comprises: an obtaining module, configured to obtain a configuration file carried in the request instruction, where the configuration file is configured with at least one of the following types of information: mail service address information, control interface information and access information of an application program; the query module is used for querying from the monitoring data information in the storage space based on the configuration file and acquiring matched query data, wherein the query data comprises at least one of the following data: mail, charts, security control instructions.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, which includes a stored program, wherein when the program runs, the apparatus where the computer-readable storage medium is located is controlled to execute the monitoring method.
According to another aspect of the embodiments of the present invention, there is also provided a processor, configured to execute a program, where the program executes the monitoring method described above.
In the embodiment of the present invention, monitoring data of a server device is first collected, and then the monitoring data is transmitted to a storage space in an asynchronous transmission manner, and in a case that a request instruction is received, the monitoring data in the storage space may be retrieved based on the request instruction to obtain query data required by a predetermined function, where the predetermined function includes at least one of: the warning function, analysis function and retrieval function, realized can monitor the data of server equipment through the data of gathering server equipment, and when the data of control goes wrong, can retrieve the monitoring data in the storage space through sending the request instruction, in order to obtain the required inquiry data of warning function, so that send out through this inquiry data and report an emergency and ask for help or increased vigilance and inform, can make the user in time know the mistake that the server equipment appears, and then solved and be difficult to in time to know the technical problem of the wrong information that the server appears among the correlation technique, thereby user experience has been promoted.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
FIG. 1 is a diagram illustrating the relationship between an application and a server according to the prior art;
FIG. 2 is a flow chart of a monitoring method according to an embodiment of the invention;
FIG. 3 is an overall architecture diagram of a monitoring method according to an embodiment of the invention;
FIG. 4 is a diagram illustrating a user requesting an application-dependent path according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a HAproxy (load balancing service software) and a carbon-relay node for load balancing according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a data presentation according to an embodiment of the present invention;
FIG. 7 is a schematic illustration of a monitoring device according to an embodiment of the present invention;
fig. 8 is a block diagram of a hardware configuration of a computer terminal of a data request processing method according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
In accordance with an embodiment of the present invention, there is provided an embodiment of a monitoring method, it should be noted that the steps illustrated in the flowchart of the accompanying drawings may be performed in a computer system such as a set of computer-executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
Fig. 2 is a flow chart of a monitoring method according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step S202, collecting monitoring data of the server side equipment.
Wherein the monitoring data comprises at least one of: traffic data, status data of hardware devices, and network traffic data.
The server device may be an electronic device such as a computer or a mobile terminal that can provide services for the client.
The service data may be data generated by a client calling a server, and may be a call volume and a failure volume of the client calling the server, where the failure volume may be login failure, registration failure, call failure, and the like in the client interface, and the state data of the hardware device may be CPU (central processing unit) state acquisition and hardware memory usage acquisition.
In an alternative embodiment, in order to fully grasp the system operation state, as many indexes as possible may be included in the collection range, including but not limited to CPU usage, CPU load, remaining amount of memory usage, partial indexes of network entry/exit flow and program operation, and the like.
In an alternative embodiment, the monitoring data of the server device may be collected through the entire bucket of Beats (a collection of lightweight and open source log transmitters).
And step S204, transmitting the monitoring data to a storage space in an asynchronous transmission mode.
The storage space stores monitoring data in an aggregation storage mode.
The asynchronous transmission mode means that each character independently forms a frame for transmission, a continuous character string is also packaged into a continuous independent frame for transmission, and the interval between the characters can be any.
In an alternative embodiment, the monitoring data may be fed into the graph (open source monitoring drawing tool) timing database through a logstack (open source data collection engine).
The aggregate storage model described above is a database storage model that supports large, sparsely distributed data, where the data can be categorized into many large dimensions, aggregating and storing selected data values, often improving in aggregation time.
In an alternative embodiment, Metricbeat (lightweight index collector) can deliver various system and service statistics in a lightweight manner, from CPU to memory, from Redis (remote dictionary service) to Nginx (lightweight reverse proxy server), without missing task network signals, and delivers the index to disk via asynchronous transfer, so that the data pipeline does not miss any data point, and when a problem is encountered downstream, Packetbeat (network packet analyzer) will retain the data that was taken, and deliver all the content to logstack (open source data collection engine) after it is back to normal.
Packetbeat is a lightweight network packet parser that can send the data in hosts and containers to a logstack or Elasticsearch engine. The Packetbeat is similar to the above Metricbeat acquisition method, and network traffic information is also transmitted to the disk in an asynchronous transmission manner and then transmitted to the logstack.
In step S206, when the request instruction is received, the monitoring data in the storage space is retrieved based on the request instruction, and the query data required by the predetermined function is acquired.
Wherein the predetermined function comprises at least one of: an alarm function, an analysis function, and a retrieval function.
In an alternative embodiment, the request instruction may be sent through the graph.
In another optional embodiment, when the monitored index data reaches the threshold set in the graph, the daemon of the graph may detect and send the request instruction, where the monitoring data in the storage space may be retrieved according to the request instruction, and query data required by the alarm function is queried from the monitoring data.
In another alternative embodiment, a request instruction of the analysis function may be sent according to a user requirement, and the monitoring data in the storage space is retrieved based on the request instruction of the analysis function, so as to obtain query data required by the analysis function.
In another alternative embodiment, a request instruction of the retrieval function may be sent according to a user requirement, and the monitoring data in the storage space is retrieved based on the request instruction of the retrieval function, so as to obtain the query data required by the retrieval function.
Through the steps, monitoring data of the server-side equipment are collected firstly, then the monitoring data are transmitted to the storage space in an asynchronous transmission mode, and under the condition that a request instruction is received, the monitoring data in the storage space can be retrieved based on the request instruction so as to obtain query data required by a preset function, wherein the preset function comprises at least one of the following functions: the warning function, analysis function and retrieval function, realized can monitor the data of server equipment through the data of gathering server equipment, and when the data of control goes wrong, can retrieve the monitoring data in the storage space through sending the request instruction, in order to obtain the required inquiry data of warning function, so that send out through this inquiry data and report an emergency and ask for help or increased vigilance and inform, can make the user in time know the mistake that the server equipment appears, and then solved and be difficult to in time to know the technical problem of the wrong information that the server appears among the correlation technique, thereby the technical problem of user experience has been promoted.
Optionally, the collecting monitoring data of the server device includes: the method comprises the following steps of adopting a point burying tool to collect service data of a server device, wherein the service data is log data generated in the running process of an application program on the server device, and the point burying mode of the point burying tool comprises the following steps: active buried points and passive centralized buried points.
In an optional embodiment, a point-embedded tool is used for collecting service data of the server device, and state data of the running program can be covered as fine as possible.
The embedded points can be divided into an active embedded point mode and a passive centralized embedded point mode, the active embedded points are embedded points in a mode of adding codes at positions where the embedded points need to be embedded manually by developers, the embedded points can be embedded more flexibly and changeably, and data really concerned by the developers or operators can be collected.
The other point burying mode is passive centralized point burying, which is mainly characterized in that a section such as an interceptor or a filter is added in a project to perform uniform point burying, and the centralized point burying can quickly acquire important points, so that the point burying time and the acquisition time are saved.
In an alternative embodiment, the service data may be printed in the form of a log and a jar package may be provided, which may be introduced in an application on the server device that needs to be monitored. The embedded point tool is encapsulated in the jar packet, and a developer can record a value needing embedded point into a log by using the tool in the jar packet, and then asynchronously acquire and store service data.
The method for introducing the monitoring function has almost zero cost, only one jar (java archive file) package needs to be introduced, the monitoring data is more accurate, more flexible and traceable, and fewer system resources are occupied by acquiring the service data in an asynchronous mode, and the operation is simple.
In an alternative embodiment, the service data can be directly collected by using the Logstash; it is also possible to use an open source filebear (lightweight journal collector) to collect the service data, where the filebear occupies very little CPU and memory of the system. And since Filebeat is part of the Elastic Stack, it can seamlessly collaborate with logstack, Elastic search, and Kibana (open source data analysis and visualization platform).
Optionally, the service data further includes: delay data, error data, response time, SLA (service level agreement) performance data, and user access data of the application.
The user access data may include a user access pattern.
In an alternative embodiment, network traffic is done under the TCP network protocol, so application delays and errors, response times, SLA performance, user access patterns and trends, etc. can be monitored. Among other things, monitoring network traffic is critical to achieving observability and protecting the environment, which helps ensure a high level of performance and security.
Optionally, the collecting monitoring data of the server device includes: an acquisition module that starts at least one hardware device, wherein the hardware device comprises: a processor and a memory; asynchronously collecting working data of at least one hardware device of a server device, wherein the working data comprises: state data of the processor and state data of the memory.
The configuration file may be a pre-configured file.
The state data of the processor can be the operation speed and the operation efficiency of the processor; the state data of the memory can be the remaining amount of the memory and the occupied amount of the memory.
In an alternative embodiment, the application service is operated depending on a good system environment, so that whether the system environment is healthy or not needs to be discovered and solved in time. Metricbeat is able to convey various system and service statistics, from CPU to memory, from Redis (Remote dictionary Server) to Nginx (lightweight reverse proxy Server), in a lightweight manner.
In an alternative embodiment, Metricbeat provides a variety of internal modules that can collect metrics from a variety of services, such as Apache (web server software), Jolokia (open source project managed jmx remotely by json http), Nginx, MongoDB (open source database system based on distributed file storage), MySQL (database management system), PostgreSQL (object-relational database server), Prometheus (open source system monitoring and alert toolkit), and the like. The installation is simple, and the dependency is completely zero. It is only necessary to enable the required modules in the configuration file. Moreover, if the needed module is not found, the module can be built by a user, and the Metricbed module can be written in the Go language, so that the process is very simple.
In addition, MetricBeat does not miss a task network signal, and the index can be transmitted to a disk in a spooling transmission mode, so that any data point cannot be missed in a data pipeline, and even if a downstream problem occurs, the MetricBeat does not need to worry about. Packetbeat will retain the data that has been retrieved and deliver all content to Logstash after returning to normal.
Optionally, the collecting monitoring data of the server device includes: and acquiring network flow data generated when the server equipment performs network transmission under a TCP (transmission control protocol) network protocol by monitoring task network signals in the network data channel.
The monitoring network data channel can be used for transmitting task network signals and acquiring network traffic data.
Since the applications of the server device are generally transmitted in the network under the TCP network protocol, application delay and error, response time, SLA performance, user access pattern and trend, etc. need to be monitored. Monitoring network traffic is critical to, among other things, achieving observability and protecting the environment, helping to ensure a high level of performance and security. Packetbeat is a lightweight network packet analyzer that can send the data in the host and container to a logstack or Elasticsearch. The Packetbeat is similar to the above Metricbeat acquisition method, and it also transmits the network traffic information to the disk by asynchronous transmission, and then to the logstack.
Optionally, retrieving the monitoring data in the storage space based on the request instruction, and obtaining query data required by the predetermined function, includes: acquiring a configuration file carried in the request instruction, wherein at least one type of information as follows is configured in the configuration file: mail service address information, control interface information and access information of an application program; acquiring matched query data from the monitoring data information query in the storage space based on the configuration file, wherein the query data comprises at least one of the following data: mail, charts, security control instructions.
In an alternative embodiment, the alarm notification is one of the monitoring purposes, and when the monitored index data reaches the threshold value set in Grafana (open source data visualization tool developed by using Go language), the daemon of Grafana detects and invokes the alarm mechanism. The warning modes of Grafana are also diverse: mail, nail, HTTP call, and kafka (open source streaming platform) message, etc.
Configuring mail service address information in a configuration file, wherein an SMTP server can be directly configured in a Grafana configuration file; the system's own mail service/etc/mail. rc can be configured, the SMTP server configuration is set behind this configuration file, and Grafana returns to send through it when it needs to send mail.
Configuring control interface information and access information of an application program in a configuration file can be to create group chat in a nail, add robots, for example, enter group settings in a computer-version nail, select an intelligent group assistant, add robots, select custom robots, add, finally set robot names, configure security settings, set IP, fill in Grafana host outlet IP, select agreement terms, and complete configuration. And creating a nail alarm channel in Grafana, filling in a robot url, and adding the nail channel into alarm configuration, so that the configuration of the control interface information and the access information of the application program is completed.
A preferred embodiment of the present invention will be described in detail with reference to fig. 3 to 6. As shown in fig. 3, the method includes:
as shown in fig. 3, which is a schematic flow diagram of data, in order to comprehensively grasp the system operation state, as many indicators as possible may be included in the collection range, including but not limited to CPU usage, CPU load, remaining amount of memory usage, partial indicators of network entry traffic and program operation, and the like. The whole process comprises the collection of data, the display of charts and the notification of warnings. Data acquisition mainly comprises the steps of adopting a Beats family bucket, sending the data into a graph time sequence database by combining kafka and Logstash, displaying pictures according to the statistical query result of the characteristics of the time sequence database, and informing an alarm mainly according to the implementation mechanism of the graph.
As shown in fig. 4, which is a schematic diagram of a user requesting an application-related path, a logging system actively buries a point for access of the user, prints the Metric in the form of a log into a file through a logger, pushes the file content into Kafka by Filebeat, goes into Kafka by Logstash for consumption, and pushes a Metric (a "path value Unix timestamp", for example, "com. haier. business 601616378521", for the graph, each piece of this structured data is called the Metric) structure into the graph for storage, and finally, the graph by the graph is used as a data source to point the application to be presented to the user in a rich graph library.
Wherein the composition part of the Graphite is as follows: carbon-relay: data can be copied and fragmented; carbon-agglomerator (graphite polymerizer): aggregating the data according to rules; carbon-cache (graphite cache): and the core component can receive and cache data according to different strategies, efficiently persist the data to the hard disk, and in order to improve the capability of the graph for receiving the data, the core component can use a HAproxy (load balancing service software) and a carbon-relay to perform load balancing.
Fig. 5 is a schematic diagram illustrating the HAProxy and the carbon-relay nodes as load balancing. Since the transmission frequency of the traffic metric (index) is not determined, a carbon-aggregator is used for aggregation. For example, after the metric data per minute are aggregated, the aggregated data are forwarded to the carbon-cache node and finally stored in the whisper (storage engine) timing database, and then the user can call the aggregated data from the open-source monitoring drawing tool network application.
Fig. 6 is a schematic diagram of data presentation, which may add a data source to Grafana, select a web component access address of grafpit, obtain data using an API in grapite-web (network presentation system), and add a Grafana-rich graph library to perform dynamic presentation.
Example 2
According to the embodiment of the present invention, there is also provided a monitoring apparatus, which can execute the monitoring method in the foregoing embodiment, and the specific implementation manner and the preferred application scenario are the same as those in the foregoing embodiment, and are not described herein again.
Fig. 7 is a schematic diagram of a monitoring device according to an embodiment of the present invention, as shown in fig. 7, the device including:
the acquisition module 72 is configured to acquire monitoring data of the server device, where the monitoring data includes at least one of the following: service data, state data of hardware devices and network traffic data;
the transmission module 74 is configured to transmit the monitoring data to a storage space in an asynchronous transmission manner, where the storage space stores the monitoring data in an aggregation storage mode;
the processing module 76 is configured to, in a case that a request instruction is received, retrieve the monitoring data in the storage space based on the request instruction, and obtain query data required by a predetermined function, where the predetermined function includes at least one of: an alarm function, an analysis function, and a retrieval function.
Optionally, the acquisition module comprises: the first classification acquisition module is used for acquiring service data of the server equipment by adopting a point burying tool, wherein the service data is log data generated in the running process of an application program on the server equipment, and the point burying mode of the point burying tool comprises the following steps: active buried points and passive centralized buried points.
Optionally, the log data further comprises: delay data, error data, response time, SLA performance data, user access data for the application.
Optionally, the acquisition module comprises: a starting module for starting the acquisition module of at least one hardware device, wherein the hardware device comprises: a processor and a memory; the second classification acquisition module is used for asynchronously acquiring the working data of at least one hardware device of the server side equipment, wherein the working data comprises: state data of the processor and state data of the memory.
Optionally, the acquisition module comprises: and the third classification acquisition module is used for acquiring network flow data generated when the server equipment performs network transmission under a TCP network protocol by monitoring task network signals in the network data channel.
Optionally, the processing module comprises: an obtaining module, configured to obtain a configuration file carried in the request instruction, where the configuration file is configured with at least one of the following types of information: mail service address information, control interface information and access information of an application program; the query module is used for querying from the monitoring data information in the storage space based on the configuration file and acquiring matched query data, wherein the query data comprises at least one of the following data: mail, charts, security control instructions.
An embodiment of the present invention also provides a storage medium including a stored program that, when executed, performs any of the methods described above.
Alternatively, in the present embodiment, the storage medium may be configured to store program codes for performing the following steps:
s1, collecting monitoring data of the server side equipment, wherein the monitoring data comprises at least one of the following data: service data, state data of hardware devices and network traffic data;
s2, transmitting the monitoring data to a storage space in an asynchronous transmission mode, wherein the storage space stores the monitoring data in an aggregation storage mode;
s3, in case of receiving the request instruction, retrieving the monitoring data in the storage space based on the request instruction, and obtaining the query data required by the predetermined function, wherein the predetermined function includes at least one of the following: an alarm function, an analysis function, and a retrieval function.
Embodiments of the present invention further provide an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, collecting monitoring data of the server side equipment, wherein the monitoring data comprises at least one of the following data: service data, state data of hardware devices and network traffic data;
s2, transmitting the monitoring data to a storage space in an asynchronous transmission mode, wherein the storage space stores the monitoring data in an aggregation storage mode;
s3, in case of receiving the request instruction, retrieving the monitoring data in the storage space based on the request instruction, and obtaining the query data required by the predetermined function, wherein the predetermined function includes at least one of the following: an alarm function, an analysis function, and a retrieval function.
Optionally, in this embodiment, the storage medium may include, but is not limited to: various media capable of storing program codes, such as a usb disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.
Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation manners, and this embodiment is not described herein again.
The method provided by the embodiment of the application can be executed in a computer terminal, a computer terminal or a similar operation device. Taking the example of being operated on a computer terminal, fig. 8 is a hardware structure block diagram of the computer terminal of a data request processing method according to an embodiment of the present invention. As shown in fig. 8, the computer terminal may include one or more processors 802 (only one is shown in fig. 8) (the processor 802 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 804 for storing data, and in an exemplary embodiment, may further include a transmission device 806 for communication functions and an input-output device 808. It will be understood by those skilled in the art that the structure shown in fig. 8 is only an illustration, and is not intended to limit the structure of the computer terminal. For example, the computer terminal may also include more or fewer components than shown in FIG. 8, or have a different configuration with equivalent functionality to that shown in FIG. 8 or with more functionality than that shown in FIG. 8.
The memory 804 may be used for storing computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the data request processing method in the embodiment of the present invention, and the processor 802 executes various functional applications and data processing by running the computer programs stored in the memory 804, so as to implement the above-mentioned method. The memory 804 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 804 can further include memory located remotely from the processor 802, which can be connected to a computer terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission means 806 is used for receiving or sending data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal. In one example, the transmission device 806 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 806 can be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method of monitoring, comprising:
collecting monitoring data of server-side equipment, wherein the monitoring data comprises at least one of the following data: service data, state data of hardware devices and network traffic data;
transmitting the monitoring data to a storage space in an asynchronous transmission mode, wherein the storage space stores the monitoring data in an aggregation storage mode;
under the condition that a request instruction is received, retrieving monitoring data in the storage space based on the request instruction, and acquiring query data required by a predetermined function, wherein the predetermined function comprises at least one of the following functions: an alarm function, an analysis function, and a retrieval function.
2. The method of claim 1, wherein collecting monitoring data of the server device comprises:
acquiring service data of the server device by using a point burying tool, wherein the service data is log data generated in the running process of an application program on the server device, and the point burying mode of the point burying tool comprises the following steps: active buried points and passive centralized buried points.
3. The method of claim 2, wherein the log data further comprises: delay data, error data, response time, SLA performance data, user access data for the application.
4. The method of claim 1, wherein collecting monitoring data of the server device comprises:
an acquisition module that starts at least one hardware device, wherein the hardware device comprises: a processor and a memory;
asynchronously collecting working data of at least one hardware device of the server device, wherein the working data comprises: state data of the processor and state data of the memory.
5. The method of claim 1, wherein collecting monitoring data of the server device comprises:
and acquiring the network flow data generated when the server equipment performs network transmission under a TCP (transmission control protocol) network protocol by monitoring task network signals in a network data channel.
6. The method of claim 5, wherein retrieving the monitoring data in the storage space based on the request instruction to obtain query data required by the predetermined function comprises:
acquiring a configuration file carried in the request instruction, wherein at least one type of information is configured in the configuration file: mail service address information, control interface information and access information of an application program;
acquiring matched query data from the monitoring data information query in the storage space based on the configuration file, wherein the query data comprises at least one of the following data: mail, charts, security control instructions.
7. A monitoring device, comprising:
the acquisition module is used for acquiring monitoring data of the server-side equipment, wherein the monitoring data comprises at least one of the following data: service data, state data of hardware devices and network traffic data;
the transmission module is used for transmitting the monitoring data to a storage space in an asynchronous transmission mode, wherein the storage space stores the monitoring data in an aggregation storage mode;
the processing module is used for retrieving the monitoring data in the storage space based on the request instruction and acquiring query data required by a predetermined function under the condition that the request instruction is received, wherein the predetermined function comprises at least one of the following functions: an alarm function, an analysis function, and a retrieval function.
8. The apparatus of claim 7, wherein the acquisition module comprises:
a first classification acquisition module, configured to perform acquisition on service data of the server device by using a point burying tool, where the service data is log data generated in an operation process of an application program on the server device, and a point burying manner of the point burying tool includes: active buried points and passive centralized buried points.
9. A computer-readable storage medium, comprising a stored program, wherein the program, when executed, controls an apparatus in which the computer-readable storage medium is located to perform the monitoring method of any one of claims 1 to 6.
10. A processor, characterized in that the processor is configured to run a program, wherein the program is configured to perform the monitoring method according to any one of claims 1 to 6 when running.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110820270.7A CN113688149A (en) | 2021-07-20 | 2021-07-20 | Monitoring method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110820270.7A CN113688149A (en) | 2021-07-20 | 2021-07-20 | Monitoring method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113688149A true CN113688149A (en) | 2021-11-23 |
Family
ID=78577521
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110820270.7A Pending CN113688149A (en) | 2021-07-20 | 2021-07-20 | Monitoring method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113688149A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105911219A (en) * | 2016-04-08 | 2016-08-31 | 北京盈盛恒泰科技有限责任公司 | Monitoring and early warning system and method for pollution gas |
| CN107528870A (en) * | 2016-06-22 | 2017-12-29 | 腾讯科技（深圳）有限公司 | A kind of collecting method and its equipment |
| CN109088785A (en) * | 2018-06-14 | 2018-12-25 | 苏州新研联信息科技有限公司 | A kind of IDC data center apparatus management system |
| CN109800133A (en) * | 2019-01-28 | 2019-05-24 | 浪潮软件集团有限公司 | A kind of method, one-stop monitoring alarm platform and the system of unified monitoring alarm |
| CN111049705A (en) * | 2019-12-23 | 2020-04-21 | 深圳前海微众银行股份有限公司 | Method and device for monitoring distributed storage system |
| CN111698127A (en) * | 2020-04-29 | 2020-09-22 | 新华三技术有限公司 | System, method and device for monitoring state of equipment in network |
| CN111694708A (en) * | 2020-05-28 | 2020-09-22 | 新浪网技术(中国)有限公司 | Data query method and device, electronic equipment and storage medium |
| CN112069049A (en) * | 2020-09-09 | 2020-12-11 | 阳光保险集团股份有限公司 | Data monitoring management method and device, server and readable storage medium |
| CN112131073A (en) * | 2020-08-25 | 2020-12-25 | 新浪网技术(中国)有限公司 | Server monitoring method and system |
| CN112579390A (en) * | 2020-12-04 | 2021-03-30 | 麒麟软件有限公司 | Monitoring data storage method and system based on real-time memory TSDB alarm |
-
2021
- 2021-07-20 CN CN202110820270.7A patent/CN113688149A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105911219A (en) * | 2016-04-08 | 2016-08-31 | 北京盈盛恒泰科技有限责任公司 | Monitoring and early warning system and method for pollution gas |
| CN107528870A (en) * | 2016-06-22 | 2017-12-29 | 腾讯科技（深圳）有限公司 | A kind of collecting method and its equipment |
| CN109088785A (en) * | 2018-06-14 | 2018-12-25 | 苏州新研联信息科技有限公司 | A kind of IDC data center apparatus management system |
| CN109800133A (en) * | 2019-01-28 | 2019-05-24 | 浪潮软件集团有限公司 | A kind of method, one-stop monitoring alarm platform and the system of unified monitoring alarm |
| CN111049705A (en) * | 2019-12-23 | 2020-04-21 | 深圳前海微众银行股份有限公司 | Method and device for monitoring distributed storage system |
| CN111698127A (en) * | 2020-04-29 | 2020-09-22 | 新华三技术有限公司 | System, method and device for monitoring state of equipment in network |
| CN111694708A (en) * | 2020-05-28 | 2020-09-22 | 新浪网技术(中国)有限公司 | Data query method and device, electronic equipment and storage medium |
| CN112131073A (en) * | 2020-08-25 | 2020-12-25 | 新浪网技术(中国)有限公司 | Server monitoring method and system |
| CN112069049A (en) * | 2020-09-09 | 2020-12-11 | 阳光保险集团股份有限公司 | Data monitoring management method and device, server and readable storage medium |
| CN112579390A (en) * | 2020-12-04 | 2021-03-30 | 麒麟软件有限公司 | Monitoring data storage method and system based on real-time memory TSDB alarm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10097433B2 (en) | Dynamic configuration of entity polling using network topology and entity status | |
| US7461369B2 (en) | Java application response time analyzer | |
| CN111367760B (en) | Log collection method and device, computer equipment and storage medium | |
| CN109039817B (en) | Information processing method, device, equipment and medium for flow monitoring | |
| US20080162690A1 (en) | Application Management System | |
| CN111400127B (en) | Service log monitoring method and device, storage medium and computer equipment | |
| US10931513B2 (en) | Event-triggered distributed data collection in a distributed transaction monitoring system | |
| US9935853B2 (en) | Application centric network experience monitoring | |
| CN111176941A (en) | Data processing method, device and storage medium | |
| CN114090366A (en) | Method, device and system for monitoring data | |
| GB2553784A (en) | Management of log data in electronic devices | |
| US10394531B2 (en) | Hyper dynamic Java management extension | |
| CN117176802B (en) | Full-link monitoring method and device for service request, electronic equipment and medium | |
| US20210227351A1 (en) | Out of box user performance journey monitoring | |
| CN110309028B (en) | Monitoring information acquisition method, service monitoring method, device and system | |
| CN113688149A (en) | Monitoring method and device | |
| US20170223136A1 (en) | Any Web Page Reporting and Capture | |
| EP3002910B1 (en) | Connecting computer management systems via cellular digital telecommunication networks | |
| CN113032054B (en) | Service execution method and device, storage medium and electronic device | |
| CN106933718B (en) | Method for monitoring performance and device | |
| CN112491601B (en) | Traffic topology generation method and device, storage medium and electronic equipment | |
| CN114461501A (en) | Resource abnormality detection method and device, storage medium, and electronic device | |
| CN112181929A (en) | Cloud management platform log processing method and device, electronic device and storage medium | |
| Uzun et al. | End-to-end internet speed analysis of mobile networks with mapReduce | |
| CN110852537A (en) | Service quality detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |