CN107579864B

CN107579864B - Request monitoring method and device and server

Info

Publication number: CN107579864B
Application number: CN201710970240.8A
Authority: CN
Inventors: 李睿
Original assignee: Shenzhen Dayu Wuxian Technology Co ltd
Current assignee: Shenzhen Dayu Wuxian Technology Co ltd
Priority date: 2017-10-16
Filing date: 2017-10-16
Publication date: 2021-01-08
Anticipated expiration: 2037-10-16
Also published as: CN107579864A

Abstract

The invention provides a request monitoring method, a request monitoring device and a server, and the method and the device are applied to the server. The method comprises the following steps: receiving a network request sent by a client, and judging whether the network request is a preset network request needing monitoring; if the network request is a network request needing monitoring, before the monitoring data of the network request is recorded to a log, acquiring index data corresponding to the configured monitoring index from the monitoring data; and storing the index data into a shared memory of the server. Therefore, comprehensive monitoring data can be obtained, and the monitoring data can be rapidly stored and read.

Description

Request monitoring method and device and server

Technical Field

The invention relates to the technical field of monitoring, in particular to a request monitoring method, a request monitoring device and a server.

Background

As the size of services increases, the monitoring of client requests and their results by servers becomes especially important. Currently, monitoring is generally performed in the following manner: the method comprises the steps that firstly, monitoring data requested by a network are counted through an access log; and in the second mode, the log file is sampled periodically, and monitoring is carried out according to the sampled data.

However, since the reading and writing of the log file are actually the reading and writing of the disk on the server, when the number of network requests is large, the above method has the following disadvantages: on one hand, frequent reading and writing of the disk can affect the performance of the server, and on the other hand, the speed is slow when the monitoring data is counted or accessed. The second method cannot completely cover all network requests, and has a vulnerability.

Disclosure of Invention

In view of the above, the present invention provides a request monitoring method, a request monitoring device and a server to improve the above problem.

In order to achieve the above object, an embodiment of the present invention provides a request monitoring method, which is applied to a server, and the method includes:

receiving a network request sent by a client, and judging whether the network request is a preset network request needing monitoring;

if the network request is a network request needing monitoring, before the monitoring data of the network request is recorded to a log, acquiring index data corresponding to the configured monitoring index from the monitoring data;

and storing the index data into a shared memory of the server.

Optionally, in the method, the step of obtaining, from the monitoring data, index data corresponding to the configured monitoring index includes:

searching for a statistical mode of the obtained configured monitoring indexes and at least one data dimension which needs to be counted;

and counting the monitoring indexes from the at least one data dimension by adopting the statistical mode to obtain index data corresponding to the monitoring indexes.

Optionally, in the above method, the method further comprises:

registering a URI address for the index data in the server;

and when the URI address input by the user is received, acquiring the index data from the shared memory through a monitoring data acquisition interface configured in advance by the server.

Optionally, in the method, the monitoring data obtaining interface is configured in a response content generation phase in which the server processes a network request.

The embodiment of the invention also provides a request monitoring device, which is applied to the server, and the device comprises:

the judging module is used for receiving a network request sent by a client and judging whether the network request is a preset network request needing monitoring;

the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring index data corresponding to a configured monitoring index from monitoring data before the monitoring data of the network request is recorded into a log when the network request is the network request needing monitoring;

and the storage module is used for storing the index data into a shared memory of the server.

Optionally, in the above apparatus, a manner of acquiring, by the acquiring module, index data corresponding to the configured monitoring index from the monitoring data includes:

Optionally, in the above apparatus, the apparatus further comprises:

the registration module is used for registering a URI address for the index data in the server;

and the second acquisition module is used for acquiring the index data from the shared memory through a monitoring data acquisition interface configured in advance by the server when the URI address input by the user is received.

Optionally, in the above apparatus, the monitoring data obtaining interface is configured in a response content generation phase in which the server processes a network request.

The embodiment of the present invention further provides a server, which includes a memory, a processor, and computer instructions stored in the memory and executed by the processor, where the computer instructions, when executed, implement the request monitoring method provided by the embodiment of the present invention.

Embodiments of the present invention also provide a computer-readable storage medium, on which computer instructions are stored, and when the computer instructions are executed, the method for monitoring a request provided by an embodiment of the present invention is implemented.

The embodiment of the invention provides a request monitoring method, a request monitoring device and a server. If so, before the monitoring data of the network request is recorded to the log, acquiring index data corresponding to the configured monitoring index from the monitoring data, and storing the index data into a shared memory of the server. Therefore, comprehensive monitoring data can be obtained, and the monitoring data can be rapidly stored and read.

In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.

Fig. 1 is a schematic diagram of interaction between a server and a user terminal according to an embodiment of the present invention;

fig. 2 is a schematic block diagram of a server according to an embodiment of the present invention;

fig. 3 is a schematic flowchart of a request monitoring method according to an embodiment of the present invention;

FIG. 4 is a schematic diagram illustrating the sub-steps of step S120 shown in FIG. 3;

fig. 5 is a schematic flowchart of a request monitoring method according to an embodiment of the present invention;

fig. 6 is a functional block diagram of a request monitoring apparatus according to an embodiment of the present invention.

Icon: 100-a server; 110-a memory; 120-a processor; 130-a communication unit; 200-a user terminal; 300-a client; 400-request monitoring device; 410-a judgment module; 420-a first acquisition module; 430-a storage module; 440-a registration module; 450-second acquisition module.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

Fig. 1 is a schematic diagram illustrating an interaction between a server 100 and a user terminal 200 according to an embodiment of the present invention. The server 100 may communicate with the user terminal 200 through a network to implement data communication or interaction between the user terminal 200 and the server 100.

In this embodiment, the user terminal 200 is installed with a client 300 corresponding to the server 100, and a user can obtain a service provided by the server 100 through the client 300. In this embodiment, the user terminal 200 may be, but is not limited to, a Personal Computer (PC), a smart phone, a tablet Computer, a Personal Digital Assistant (PDA), a Mobile Internet Device (MID), and the like, and the server 100 may be an Nginx server.

In this embodiment, the client 300 is an Application program (APP) that can be installed and run in the user terminal 200, for example, a browser.

Fig. 2 is a block diagram of a server 100 according to an embodiment of the present invention. The server 100 comprises a request monitoring device 400, a memory 110, a processor 120 and a communication unit 130.

The elements of the memory 110, the processor 120 and the communication unit 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. The request monitoring apparatus 400 includes at least one software function module that can be stored in the memory 110 or solidified in an Operating System (OS) of the user terminal 200 in the form of firmware.

In this embodiment, the Memory 110 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.

The processor 120 may be an integrated circuit chip having signal processing capabilities. The Processor 120 may also be a general-purpose Processor, such as a Central Processing Unit (CPU), a Network Processor (NP), etc.; but also digital signal processors (CPUs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components; the processor 120 may also be a microprocessor or any conventional processor. The processor 120 may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present invention.

The communication unit 130 is configured to establish a communication connection between the server 100 and an external device (e.g., the user terminal 200) to implement data interaction. The processor 120 is configured to execute the software functional modules or the computer programs stored in the memory 110 when receiving the execution instructions.

It should be understood that the illustration of fig. 2 is merely an illustration of server 100, and that server 100 may have more or fewer components than those shown in fig. 2, or may have a configuration that is completely different than that shown in fig. 2. The components shown in fig. 2 may be implemented in software, hardware, or a combination thereof.

Fig. 3 shows a request monitoring method according to an embodiment of the present invention, which is applied to the server 100 shown in fig. 2. The specific process and steps shown in fig. 3 will be described in detail below.

Step S110, receiving a network request sent by the client 300, and determining whether the network request is a preset network request that needs to be monitored.

In this embodiment, the configuration file of the server 100 is configured with a network request type to be monitored, for example, a POST request, a GET request, and the like. When the server 100 is started, the configuration file is read, and the network request type to be monitored is obtained.

In implementation, when receiving a network request sent by the client 300, the server 100 determines the type of the network request, and determines whether the type of the network request to be monitored includes the type of the network request, if so, determines that the network request is the network request to be monitored, and if not, determines that the network request is not the network request to be monitored.

Step S120, if the network request is a network request that needs to be monitored, before the monitoring data of the network request is recorded in the log, obtaining index data corresponding to the configured monitoring index from the monitoring data.

Generally, for each network request received, after generating the response content of the network request, the server 100 records the monitoring data of the network request into a log file. Wherein the monitoring data includes all relevant information of the network request (e.g., the network request, response content of the network request, response time, etc.).

However, the inventor has found that the access to the log file is substantially the read/write of the disk, and when the network request amount is large, the performance of the server 100 is affected, and the efficiency of request monitoring is affected.

In order to solve the problem, some existing methods detect whether monitoring data requested by a network has a problem by adopting a periodic sampling mode so as to reduce the data access amount. However, the regular sampling method cannot monitor all network requests, and a security hole exists.

The inventors have intensively studied and found that the above-mentioned problem can be solved if the monitoring data is acquired and recorded to a place other than the disk before the server 100 records the monitoring data to the log file.

In this embodiment, the server 100 is further configured with monitoring indicators that need to be monitored, such as request amount, response time, error rate, and the like. In implementation, when the network request received by the server 100 is a network request that needs to be monitored, the server 100 may obtain index data corresponding to each monitoring index from the monitoring data of the network request before the monitoring data of the network request is recorded in the log file.

Optionally, as shown in fig. 4, in this embodiment, the step of acquiring index data corresponding to the configured monitoring index from the monitoring data in the step S120 may include two substeps, i.e., step S121 and step S122.

Step S121, searching for a statistical manner of the obtained configured monitoring index and at least one data dimension that needs to be counted.

And step S122, counting the monitoring indexes from the at least one data dimension by adopting the counting mode to obtain index data corresponding to the monitoring indexes.

Optionally, in this embodiment, different monitoring indexes may have different statistical modes. For example, when the monitoring index is the requested amount, the statistical manner is accumulation. Every time a network request of a target type is detected, the request amount is increased by 1. For another example, when the monitoring index is the response content, the statistical manner is to directly obtain the corresponding response content.

Optionally, in this embodiment, each monitoring index has at least one data dimension that needs to be counted, and the at least one data dimension is preset in the configuration file. Still taking the example that the monitoring index is the request amount, statistics can be performed from two data dimensions of the HOST field of the request header and the URL address of the network request. That is, a network request type is uniquely distinguished based on the HOST field of the request header and the URL address of the network request, and the request amount of the network request of the type is counted.

It should be noted that, the request amount of the network request is counted based on two data dimensions, that is, the HOST field of the request header and the URL address of the network request, and a determined value is finally obtained, that is, for each monitoring index, a result is obtained from the at least one data dimension of the monitoring index, and the result is index data of the monitoring index.

Optionally, in this embodiment, the server 100 may be an Nginx server, in which case the network request refers to an HTTP request. The processing of network requests by the Nginx server is typically divided into 9 processing stages, each of which can be pipelined by any number of HTTP modules.

The 9 processing stages are POST _ READ, SERVER _ REWRITE, CONFIG _ FIND, REWRITE, PREACCESS, ACCESS, TRY _ FILES, CONTENT and LOG respectively, and through the stage division, developers can define the development module in the corresponding stage according to requirements.

Wherein POST _ READ is a POST-processing stage after a Nginx server receives a complete HTTP header; SERVER _ REWRITE is a URI (Uniform Resource identifier) REWRITE phase at the SERVER 100 level, the execution of which is within the SERVER block of the Nginx SERVER; CONFIG _ FIND is a stage of finding a location configuration, which uses the URI after the rewrite to FIND the corresponding location; REWRITE is URI REWRITE phase of location level; PREACCESS is the previous stage of access rights control, which is also typically used for access control; ACCESS is an ACCESS authority control stage, and TRY _ FILES is a processing stage of TRY _ FILES instructions; the LOG is the logging phase.

In this embodiment, the step of determining whether the network request is the preset network request that needs to be monitored in step S110 and the step S120 are both executed in the LOG phase, and the corresponding software function module is registered in the LOG phase.

Alternatively, the method provided by the embodiment of the present invention may be implemented by a Lua instruction. Wherein, Lua is an embedded script language.

Step S130, storing the index data in the shared memory of the server 100.

When the server 100 is an Nginx server, the index data may be stored in the dictionary of the shared memory.

Optionally, as shown in fig. 5, in this embodiment, the request monitoring method may further include two steps, i.e., step S140 and step S150.

In step S140, a URI address is registered in the server 100 for the index data.

Wherein the URI address is registered at the start-up of the server 100.

Step S150, when the URI address input by the user is received, acquiring the index data from the shared memory through a monitoring data acquisition interface configured in advance by the server 100.

Optionally, the monitoring data obtaining interface may be configured in a response CONTENT generating phase, that is, the CONTENT phase, in which the server 100 processes a network request.

Through the above design, a user (e.g., an administrator of the server 100, etc.) may input the URI address at the client 300 or the server 100 to trigger the monitoring data obtaining interface to obtain the index data from the shared memory.

As shown in fig. 6, an embodiment of the present invention further provides a request monitoring apparatus 400, which is applied to the server 100 shown in fig. 2, and the apparatus includes a determining module 410, a first obtaining module 420, and a storing module 430.

The determining module 410 is configured to receive a network request sent by the client 300, and determine whether the network request is a preset network request that needs to be monitored.

In this embodiment, the description of the determining module 410 may specifically refer to the detailed description of the step S110 shown in fig. 3, that is, the step S110 may be executed by the determining module 410.

The first obtaining module 420 is configured to, when the network request is a network request requiring monitoring, obtain index data corresponding to a configured monitoring index from the monitoring data before monitoring data of the network request is recorded in a log.

In this embodiment, the description of the first obtaining module 420 may specifically refer to the detailed description of the step S120 shown in fig. 3, that is, the step S120 may be executed by the first obtaining module 420.

Optionally, in this embodiment, a manner that the first obtaining module 420 obtains the index data corresponding to the configured monitoring index from the monitoring data may include:

The storage module 430 is configured to store the index data in a shared memory of the server 100.

In this embodiment, the description of the storage module 430 may specifically refer to the detailed description of step S130 shown in fig. 3, that is, step S130 may be executed by the storage module 430.

Optionally, the apparatus may further include a registration module 440 and a second acquisition module 450.

Wherein, the registering module 440 is configured to register a URI address for the metric data in the server 100.

The second obtaining module 450 is configured to obtain the index data from the shared memory through a monitoring data obtaining interface preconfigured in the server 100 when the URI address input by the user is received.

Optionally, the monitoring data obtaining interface is configured in a response content generating stage of processing the network request by the server 100.

In summary, the embodiments of the present invention provide a request monitoring method, an apparatus, and a server 100, where after receiving a network request sent by a client 300, the server 100 determines whether the network request is a preset network request that needs to be monitored. If yes, before the monitoring data of the network request is recorded in the log, index data corresponding to the configured monitoring index is obtained from the monitoring data, and the index data is stored in the shared memory of the server 100. Therefore, comprehensive monitoring data can be obtained, and the monitoring data can be rapidly stored and read.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, the functional modules in the embodiments of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A request monitoring method is applied to a server, and comprises the following steps:

if the network request is the network request needing monitoring, before the monitoring data of the network request is recorded to a log, searching for a statistical mode for obtaining a configured monitoring index and at least one data dimension needing to be counted;

counting the monitoring indexes from the at least one data dimension by adopting the statistical mode to obtain index data corresponding to the monitoring indexes;

storing the index data into a shared memory of the server;

registering a URI address for the index data in the server;

2. The method of claim 1, wherein the monitoring data acquisition interface is configured during a response content generation phase of the server processing network requests.

3. A request monitoring apparatus, applied to a server, the apparatus comprising:

the first acquisition module is used for searching and acquiring a statistical mode of a configured monitoring index and at least one data dimension which needs to be counted before monitoring data of the network request is recorded into a log when the network request is the network request which needs to be monitored;

the first obtaining module is further configured to perform statistics on the monitoring index from the at least one data dimension by using the statistical manner to obtain index data corresponding to the monitoring index; the storage module is used for storing the index data into a shared memory of the server;

4. The apparatus of claim 3, wherein the monitoring data acquisition interface is configured during a response content generation phase of the server processing the network request.

5. A server comprising a memory, a processor, and computer instructions stored in the memory and executed by the processor that, when executed, implement the method of any of claims 1-2.

6. A computer readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed, implement the method of any of claims 1-2.