CN115426253B - Web server monitoring method and system based on log file - Google Patents
Web server monitoring method and system based on log file Download PDFInfo
- Publication number
- CN115426253B CN115426253B CN202211012977.6A CN202211012977A CN115426253B CN 115426253 B CN115426253 B CN 115426253B CN 202211012977 A CN202211012977 A CN 202211012977A CN 115426253 B CN115426253 B CN 115426253B
- Authority
- CN
- China
- Prior art keywords
- log
- web server
- log file
- information
- apache
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000012544 monitoring process Methods 0.000 title claims abstract description 28
- 238000007619 statistical method Methods 0.000 claims abstract description 35
- 230000007246 mechanism Effects 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 description 15
- 241001522296 Erithacus rubecula Species 0.000 description 9
- 238000007726 management method Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000013439 planning Methods 0.000 description 2
- 230000002194 synthesizing effect Effects 0.000 description 2
- 238000012550 audit Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012384 transportation and delivery Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0803—Configuration setting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/2866—Architectures; Arrangements
- H04L67/30—Profiles
Abstract
The invention discloses a web server monitoring method and a web server monitoring system based on log files, belongs to the technical field of web servers, and aims to solve the technical problem of how to monitor the operation condition of a web server by means of statistical analysis of the log files of the web server. The method comprises the following steps: configuring an Apache configuration file; setting log_format attributes of the log file and custom attributes through the log_format attributes of the nginx; configuring a log polling mode; encrypting the information to be counted by an encryption method of a preset value, and adding the encrypted information into a header customized in the request information; polling the log file and reading the log file through Apache software, and transmitting the read log file and the encryption information into a database; decrypting the encrypted information by a decryption method configured in the database to obtain information needing statistics, and carrying out statistical analysis on the log file.
Description
Technical Field
The invention relates to the technical field of web servers, in particular to a web server monitoring method and system based on log files.
Background
With the development of Web services on the Internet, almost every government department, company, college, scientific research institution, etc. is building or is building its own website. Meanwhile, various problems are encountered by each unit in the construction of the website, so that detailed and comprehensive analysis on the operation and access conditions of the web server is self-evident to know the operation conditions of the website and find the defects of the website, and the importance of promoting the better development of the website is self-evident.
The management of Web sites not only monitors the speed of Web and the content delivery of Web, but also requires attention to the daily throughput of servers, knowledge of external accesses to these Web sites, knowledge of the access situation of each page of the Web sites, improvement of the content and quality of Web pages, improvement of the readability of the content according to the click frequency of each page, tracking of the steps including commercial transactions, management of the data of the Web sites "behind the scenes", and the like.
In order to better provide WWW services, it is increasingly important and urgent to monitor the operation of WEB servers and to know the detailed access status of WEB site contents. These requirements can be achieved by statistics and analysis of the log files of the web server.
How to monitor the operation condition of the web server by means of statistical analysis of the log files of the web server is a technical problem to be solved.
Disclosure of Invention
The technical task of the invention is to provide a web server monitoring method and a web server monitoring system based on log files, which aim at the defects, so as to solve the technical problem of how to monitor the operation condition of a web server by carrying out statistical analysis on the log files of the web server.
In a first aspect, the present invention provides a web server monitoring method based on log files, applied to a service model including a client and a web server, in which Apache software is running, the method includes the following steps:
configuring an Apache configuration file, wherein the configuration file comprises a storage path for configuring log files accessed by the Apache;
setting log_format attributes of the log file and custom attributes through the log_format attributes of the nginx;
configuring a log polling mode, and polling a log file by Apache based on the configured log polling mode;
when a client sends request information to a web server, encrypting information to be counted by an encryption method of a pre-configured value to obtain encrypted information, and adding the encrypted information into a header self-defined in the request information;
after receiving the request information, the web server calls Apache software, polls the log file and reads the log file through the Apache software based on a configured log polling mode, and transmits the read log file and the encrypted information into a database;
decrypting the encrypted information through a decryption method configured in the database to obtain information to be counted, carrying out statistical analysis on the log file based on the information to be counted to obtain a statistical analysis result, and returning the statistical analysis result to the client.
Preferably, the log file is read by Apache software based on logstack.
Preferably, a TCP connection is connected between the client and the web server, and the request information comprises the IP address of the client, the type of the browser and the URL of the request;
after receiving the request information added with the encryption information, the web server returns the page content required by the client to the client based on the request information and records the corresponding access information into a log file; if an error occurs, an error is returned to the client and error information is recorded in the log file.
Preferably, if a virtual host is configured in the web server, for each virtual host, configuring a storage path of a log file accessed by Apache in httpd.conf, including configuring a storage path and a format of a log file accessed by Apache corresponding to the virtual host, where error log is used to indicate a storage path of an error information log file accessed by Apache;
if no virtual host is configured in the web server, the CustomLog configuration is looked up in httpd.conf and modified.
Preferably, the log polling mode includes a log file polling mechanism logrotate of the Linux system, a log polling program rotalogs carried by Apache, and a log polling tool cronolog supported in the FAQ of Apache.
In a second aspect, the present invention provides a log file-based web server monitoring system, applied to a service model including a client and a web server, where the web server runs Apache software, configured to perform statistical analysis on log files of the web server by using the log file-based web server monitoring method according to any one of the first aspects, where the system includes:
the log access configuration module is used for configuring Apache configuration files, including configuring a storage path of log files accessed by Apache, setting log_format attributes of log files and custom attributes through the log_format attributes of the nginx, and configuring a log polling mode;
the client side sends request information to the web server, and calls the request information configuration module, wherein the request information configuration module is used for encrypting information to be counted through an encryption method of a preset value to obtain encryption information and adding the encryption information into a customized header in the request information;
a database, the database interacting with a web server and a client;
the system comprises a log file acquisition module, a web server and a database, wherein the log file acquisition module is interacted with the web server, the web server receives request information from a client and then invokes the log file acquisition module, the Apache software is invoked through the log file acquisition module, and the log file is polled and read through the Apache software based on a configured log polling mode and is used for transmitting the read log file and encryption information into the database;
the log file statistical analysis module is interacted with the database and used for decrypting the encrypted information through a decryption method configured in the database to obtain information to be counted, carrying out statistical analysis on the log file based on the information to be counted to obtain a statistical analysis result, and returning the statistical analysis result to the client through the web server.
Preferably, the log file collection module calls Apache software to read the log file based on logstack.
Preferably, a TCP connection is connected between the client and the web server, and the request information comprises the IP address of the client, the type of the browser and the URL of the request;
after receiving the request information added with the encryption information, the web server returns the page content required by the client to the client based on the request information and records the corresponding access information into a log file; if an error occurs, an error is returned to the client and error information is recorded in the log file.
Preferably, if a virtual host is configured in the web server, for each virtual host, the log access configuration module is configured to configure a storage path of a log file accessed by Apache in httpd.conf, including configuring a storage path and a format of a log file accessed by Apache corresponding to the virtual host, where error log is used to indicate a storage path of an error information log file accessed by Apache;
if no virtual host is configured in the web server, the log access configuration module is configured to find the customLog configuration in httpd.conf and modify it.
Preferably, the log polling mode includes a log file polling mechanism logrotate of the Linux system, a log polling program rotalogs carried by Apache, and a log polling tool cronolog supported in the FAQ of Apache.
The web server monitoring method and system based on the log file have the following advantages:
1. configuring a log file path and a log polling mode, adding information to be counted into request information when a client sends a request to a web server, reading a log file by Apache and carrying out statistical analysis on the log file based on the information to be counted to obtain a statistical analysis result, thereby realizing the monitoring of the web server through the log file;
2. after encrypting information to be counted, adding the encrypted information into a custom header, sending request information to a web server, returning page request content to a client by the web server based on the request information, storing access information or error information corresponding to the process into a log file, storing the encrypted information into the log file, apache polls the log file and reads the log file, storing the encrypted information and the log file into a database, decrypting the encrypted information in the database, and then carrying out statistical analysis on the log file based on the information to be counted to obtain an analysis result, thereby realizing decoupling of a service system and a data statistical service without affecting service processing of the service system;
3. the method has flexible data statistics service, can be quickly adapted to different platforms, and gives consideration to the safety of data statistics.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
The invention is further described below with reference to the accompanying drawings.
Fig. 1 is a flow chart of a method and a system for monitoring a web server based on a log file in embodiment 1.
Detailed Description
The invention will be further described with reference to the accompanying drawings and specific examples, so that those skilled in the art can better understand the invention and implement it, but the examples are not meant to limit the invention, and the technical features of the embodiments of the invention and the examples can be combined with each other without conflict.
The embodiment of the invention provides a web server monitoring method and a web server monitoring system based on log files, which are used for solving the technical problem of how to monitor the running condition of a web server by means of carrying out statistical analysis on the log files of the web server.
Example 1:
the invention relates to a web server monitoring method based on log files, which is applied to a service model comprising a client and a web server, wherein Apache software is operated in the web server, and the method comprises the following steps:
s100, configuring an Apache configuration file, wherein the configuration of the Apache configuration file comprises configuring a storage path of a log file accessed by the Apache;
configuring a log polling mode, and polling a log file by Apache based on the configured log polling mode;
setting log_format attributes of the log file and custom attributes through the log_format attributes of the nginx;
s200, when a client sends request information to a web server, encrypting information to be counted through an encryption method of a preset value to obtain encryption information, and adding the encryption information into a header head customized in the request information;
s300, after receiving the request information, the web server calls Apache software, polls a log file and reads the log file through the Apache software based on a configured log polling mode, and transmits the read log file and encrypted information into a database;
s400, decrypting the encrypted information through a decryption method configured in the database to obtain information to be counted, performing statistical analysis on the log file based on the information to be counted to obtain a statistical analysis result, and returning the statistical analysis result to the client.
Step S100 involves configuration and management of the Apache log. If a virtual host is configured in the web server, for each virtual host, configuring a storage path of a log file accessed by Apache in httpd.conf, wherein the configuration includes that a CustomLog and ErrorLog, customLog corresponding to the virtual host are configured to indicate a storage path and a format of the log file accessed by Apache, and an ErrorLog is configured to indicate a storage path of an error information log file accessed by Apache; if no virtual host is configured in the web server, the CustomLog configuration is looked up in httpd.conf and modified.
Suppose there are two virtual hosts running by Apache: www.secfocus.com and www.tomorrowtel.com. Access log analysis and statistics need to be performed on the two virtual hosts respectively.
In the Apache profile, there are two concerns about log-related configurations:
$CustomLog/www/logs/access_log common
$ErrorLog/www/logs/error_log
the CustomLog is used to indicate the location (here saved in/www/logs/access_log) and format (here common) of the Apache's access log deposit; errorLog is used to indicate where the Apache error information log is stored.
For a server without a virtual host, the configuration of the CustomLog is only required to be directly searched in httpd.conf for modification. For a Web server having a plurality of virtual servers, access logs of each virtual server need to be separated so as to perform access statistics and analysis on each virtual server. Thus, a separate log configuration in the virtual server configuration is required.
Each virtual host definition has a custom log command to specify the storage file of the virtual host access log; and the Alias command is used for enabling the report generated by log analysis to be accessed in a www.secfocus.com/user/mode. The saving of the log file is completed by the above configuration.
Step S100 also relates to configuration of web server log-round robin. There are three good ways for web server log to round robin: the first method is to utilize a log file round robin mechanism of a Linux system itself; the second method is to utilize the Apache self-contained log round robin program, rotate; the third is to use a log round robin tool cronolog recommended to develop already mature in the FAQ of Apache.
For large-scale web services, a practical load balancing technology is often used for improving the service capacity of the web site, so that a plurality of servers are arranged in the background to provide the web services, and the distribution planning and expansibility of the services are greatly facilitated. If the distribution of a plurality of servers needs to be combined, the logs are uniformly subjected to statistical analysis. Thus, in order to ensure statistical accuracy, it is necessary to automatically generate logs strictly in terms of time-of-day.
The embodiment adopts a method of log file round-robin mechanism logrotate of a Linux system. The log is a log round-robin program carried by the Linux system, and is a program for specially round-robin various system logs (syslogd, mail). The program is run by the service crond running the program 4:02 a day in the morning. The logrotate file can be seen under the per etc/cron.
The Web server log records various original information such as processing requests received by the Web server and runtime errors. By counting, analyzing and synthesizing the logs, the running condition of the server can be effectively mastered, error reasons can be found and removed, client access distribution can be known, and the maintenance and management of the system can be better enhanced.
In the service model in this embodiment, a client (browser) and a Web server establish a TCP connection, and after the connection is established, an access request (e.g., get) is sent to the Web server. According to the HTTP protocol, the request contains a series of information such as the IP address of the client, the type of browser, the URL of the request, etc.
And after receiving the request, the Web server returns the page content required by the client to the client. If an error occurs, an error code is returned. The server side records the access information and the error information into a log file.
Based on the above, in step S200 of the present embodiment, when the client sends a request to the web server, the data to be counted is encrypted by RSA and then added to the custom header, and log format and custom attribute are set in advance by log_format attribute of the nginx.
In step S300, after receiving the request information added with the encryption information, the web server returns the page content required by the client to the client based on the request information, and records the encryption information and the corresponding access information into the log file; if errors occur, the errors are returned to the client, and the encryption information and the corresponding error information are recorded in the log file.
In this embodiment, the web server runs Apache, and Apache monitors log files, polls the log files in a pre-configured polling mode, reads the log files, and stores the log files in the database.
In step S400, the encryption information is RSA decrypted through the function configured in the database to obtain information to be counted, and then the log file is statistically analyzed according to the information to be counted to obtain a statistical analysis result, and the statistical analysis result is returned to the client.
According to the method, log statistics analysis is performed through the Web server, custom attributes are added on the basis of the original logs, personalized statistics is performed, and data statistics analysis can be performed rapidly and safely.
Example 2:
the invention discloses a web server monitoring system based on log files, which is applied to a service model comprising a client and a web server.
The log access configuration module is used for configuring Apache configuration files, including configuring storage paths of log files accessed by Apache, setting log files and custom attributes through log_format attributes of the nginx, and configuring log polling modes.
As a specific implementation, the log access configuration module realizes configuration and management of Apache logs and configuration and management of log polling modes.
If a virtual host is configured in the web server, for each virtual host, configuring a storage path of a log file accessed by Apache in httpd.conf, wherein the configuration includes that a CustomLog and ErrorLog, customLog corresponding to the virtual host are configured to indicate a storage path and a format of the log file accessed by Apache, and an ErrorLog is configured to indicate a storage path of an error information log file accessed by Apache; if no virtual host is configured in the web server, the CustomLog configuration is looked up in httpd.conf and modified.
Suppose there are two virtual hosts running by Apache: www.secfocus.com and www.tomorrowtel.com. Access log analysis and statistics need to be performed on the two virtual hosts respectively.
In the Apache profile, there are two concerns about log-related configurations:
$CustomLog/www/logs/access_log common
$ErrorLog/www/logs/error_log
the CustomLog is used to indicate the location (here saved in/www/logs/access_log) and format (here common) of the Apache's access log deposit; errorLog is used to indicate where the Apache error information log is stored.
For a server without a virtual host, the configuration of the CustomLog is only required to be directly searched in httpd.conf for modification. For a Web server having a plurality of virtual servers, access logs of each virtual server need to be separated so as to perform access statistics and analysis on each virtual server. Thus, a separate log configuration in the virtual server configuration is required.
Each virtual host definition has a custom log command to specify the storage file of the virtual host access log; and the Alias command is used for enabling the report generated by log analysis to be accessed in a www.secfocus.com/user/mode. The saving of the log file is completed by the above configuration.
There are three good ways for web server log to round robin: the first method is to utilize a log file round robin mechanism of a Linux system itself; the second method is to utilize the Apache self-contained log round robin program, rotate; the third is to use a log round robin tool cronolog recommended to develop already mature in the FAQ of Apache.
For large-scale web services, a practical load balancing technology is often used for improving the service capacity of the web site, so that a plurality of servers are arranged in the background to provide the web services, and the distribution planning and expansibility of the services are greatly facilitated. If the distribution of a plurality of servers needs to be combined, the logs are uniformly subjected to statistical analysis. Thus, in order to ensure statistical accuracy, it is necessary to automatically generate logs strictly in terms of time-of-day.
The embodiment adopts a method of log file round-robin mechanism logrotate of a Linux system. The log is a log round-robin program carried by the Linux system, and is a program for specially round-robin various system logs (syslogd, mail). The program is run by the service crond running the program 4:02 a day in the morning. The logrotate file can be seen under the per etc/cron.
The Web server log records various original information such as processing requests received by the Web server and runtime errors. By counting, analyzing and synthesizing the logs, the running condition of the server can be effectively mastered, error reasons can be found and removed, client access distribution can be known, and the maintenance and management of the system can be better enhanced.
In the service model in this embodiment, a client (browser) and a Web server establish a TCP connection, and after the connection is established, an access request (e.g., get) is sent to the Web server. According to the HTTP protocol, the request contains a series of information such as the IP address of the client, the type of browser, the URL of the request, etc.
When a client sends request information to a web server, a request information configuration module is called, wherein the request information configuration module is used for encrypting information to be counted through an encryption method of a preset value to obtain encryption information, and is used for adding the encryption information into a header customized in the request information.
When a client sends a request to a web server, the data to be counted is encrypted by RSA and then added into a custom header, and log format and custom attributes are set in advance through log_format attributes of the nginx.
The database interacts with the web server and the client, and an algorithm for decrypting the encrypted information is configured in the database, and the encrypted information is decrypted by an RSA decryption method in the embodiment.
The method comprises the steps that a log file acquisition module interacts with a web server, the web server receives request information from a client and then invokes the log file acquisition module, apache software is invoked through the log file acquisition module, and the log file is polled and read through the Apache software based on a configured log polling mode and used for transmitting the read log file and encryption information into a database.
Specifically, after receiving the request information added with the encryption information, the web server returns page contents required by the client to the client based on the request information, and records the encryption information and the corresponding access information into a log file; if errors occur, the errors are returned to the client, and the encryption information and the corresponding error information are recorded in the log file.
In this embodiment, the web server is running with Apache, and the log file collection module is configured to monitor the log file by Apache, and poll the log file in a pre-configured polling manner, read the log file, and store the log file in the database.
The log file statistical analysis module is interacted with the database and used for decrypting the encrypted information through a decryption method configured in the database to obtain information to be counted, carrying out statistical analysis on the log file based on the information to be counted to obtain a statistical analysis result, and returning the statistical analysis result to the client through the web server.
As a specific implementation, the log file statistics analysis module performs RSA decryption on the encrypted information through a function configured in the database to obtain information to be counted, then performs statistics analysis on the log file according to the information to be counted to obtain a statistics analysis result, and returns the statistics analysis result to the client.
While the invention has been illustrated and described in detail in the drawings and in the preferred embodiments, the invention is not limited to the disclosed embodiments, and it will be appreciated by those skilled in the art that the code audits of the various embodiments described above may be combined to produce further embodiments of the invention, which are also within the scope of the invention.
Claims (10)
1. The web server monitoring method based on the log file is characterized by being applied to a service model comprising a client and a web server, wherein Apache software is operated in the web server, and the method comprises the following steps of:
configuring an Apache configuration file, wherein the configuration file comprises a storage path for configuring log files accessed by the Apache;
setting custom attributes of the log file through log_format attributes of the nginx;
configuring a log polling mode, and polling a log file by Apache based on the configured log polling mode;
when a client sends request information to a web server, encrypting information to be counted by a pre-configured encryption method to obtain encrypted information, and adding the encrypted information into a header self-defined in the request information;
after receiving the request information, the web server calls Apache software, polls the log file and reads the log file through the Apache software based on a configured log polling mode, and transmits the read log file and the encrypted information into a database;
decrypting the encrypted information through a decryption method configured in the database to obtain information to be counted, carrying out statistical analysis on the log file based on the information to be counted to obtain a statistical analysis result, and returning the statistical analysis result to the client.
2. The log file based web server monitoring method of claim 1, wherein the log file is read by Apache software based on logstack.
3. The web server monitoring method based on log files according to claim 1, wherein a TCP connection is connected between the client and the web server, and the request information includes an IP address of the client, a type of browser, and a URL of the request;
after receiving the request information added with the encryption information, the web server returns the page content required by the client to the client based on the request information, and records the encryption information and the corresponding access information into a log file; if errors occur, the errors are returned to the client, and the encryption information and the corresponding error information are recorded in the log file.
4. A method for monitoring a web server based on log files according to any one of claims 1-3, wherein if a virtual host is configured in the web server, for each virtual host, configuring a storage path of the log file accessed by Apache in httpd.conf, including configuring CustomLog and ErrorLog, customLog corresponding to the virtual host to indicate a storage path and a format of the log file accessed by Apache, and error log to indicate a storage path of the log file accessed by Apache;
if no virtual host is configured in the web server, the CustomLog configuration is looked up in httpd.conf and modified.
5. A web server monitoring method based on log files according to any one of claims 1-3, wherein the log polling mode comprises a log file round-robin mechanism logrotate of a Linux system, a log round-robin program rotalogs carried by Apache, and a log round-robin tool cronolog supported in a FAQ of Apache.
6. A log file based web server monitoring system for a service model comprising a client and a web server, the web server having Apache software running therein for statistical analysis of log files of the web server by a log file based web server monitoring method according to any of claims 1-5, the system comprising:
the log access configuration module is used for configuring Apache configuration files, including configuring a storage path of log files accessed by Apache, setting custom attributes of the log files through log_format attributes of the nginx, and configuring a log polling mode;
the client side sends request information to the web server, and calls the request information configuration module, wherein the request information configuration module is used for encrypting information to be counted through a pre-configured encryption method to obtain encryption information and adding the encryption information into a customized header in the request information;
a database, the database interacting with a web server and a client;
the system comprises a log file acquisition module, a web server and a database, wherein the log file acquisition module is interacted with the web server, the web server receives request information from a client and then invokes the log file acquisition module, the Apache software is invoked through the log file acquisition module, and the log file is polled and read through the Apache software based on a configured log polling mode and is used for transmitting the read log file and encryption information into the database;
the log file statistical analysis module is interacted with the database and used for decrypting the encrypted information through a decryption method configured in the database to obtain information to be counted, carrying out statistical analysis on the log file based on the information to be counted to obtain a statistical analysis result, and returning the statistical analysis result to the client through the web server.
7. The log file based web server monitoring system of claim 6 wherein the log file collection module invokes Apache software to read log files based on logstack.
8. The log file based web server monitoring system of claim 6 wherein the TCP connection is connected between the client and the web server, the request information including the IP address of the client, the type of browser, and the URL of the request;
after receiving the request information added with the encryption information, the web server returns the page content required by the client to the client based on the request information, and records the encryption information and the corresponding access information into a log file; if errors occur, the errors are returned to the client, and the encryption information and the corresponding error information are recorded in the log file.
9. The web server monitoring system based on log files according to any one of claims 6-8, wherein if a virtual host is configured in the web server, for each virtual host, the log access configuration module is configured to configure a storage path of the log file accessed by Apache in httpd.conf, including configuring custom log and ErrorLog, customLog corresponding to the virtual host to indicate a storage path and format of the log file accessed by Apache, and error log to indicate a storage path of the log file accessed by Apache;
if no virtual host is configured in the web server, the log access configuration module is configured to find the customLog configuration in httpd.conf and modify it.
10. The web server monitoring system based on log files according to any one of claims 6-8, wherein the log polling mode comprises a log file polling mechanism logrotate of a Linux system, a log polling program rotalogs carried by Apache, and a log polling tool cronolog supported in FAQ of Apache.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211012977.6A CN115426253B (en) | 2022-08-23 | 2022-08-23 | Web server monitoring method and system based on log file |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211012977.6A CN115426253B (en) | 2022-08-23 | 2022-08-23 | Web server monitoring method and system based on log file |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115426253A CN115426253A (en) | 2022-12-02 |
| CN115426253B true CN115426253B (en) | 2024-01-26 |
Family
ID=84198865
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211012977.6A Active CN115426253B (en) | 2022-08-23 | 2022-08-23 | Web server monitoring method and system based on log file |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115426253B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20010035379A (en) * | 2001-02-08 | 2001-05-07 | 오충용 | An web log analyzing service method, the system thereof and the recording medium thereof |
| CN105933268A (en) * | 2015-11-27 | 2016-09-07 | 中国银联股份有限公司 | Webshell detection method and apparatus based on total access log analysis |
| CN108509326A (en) * | 2018-04-09 | 2018-09-07 | 四川长虹电器股份有限公司 | A kind of service state statistical method and system based on nginx daily records |
| CN108833091A (en) * | 2018-05-28 | 2018-11-16 | 武汉斗鱼网络科技有限公司 | A kind of encryption method of journal file, decryption method and device |
| CN109039749A (en) * | 2018-08-10 | 2018-12-18 | 广州天予智能科技有限公司 | A kind of acquisition of remote journal and encryption transmission system and method |
| CN114640567A (en) * | 2022-02-23 | 2022-06-17 | 中银金融科技有限公司 | Apache log analysis method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8935382B2 (en) * | 2009-03-16 | 2015-01-13 | Microsoft Corporation | Flexible logging, such as for a web server |
-
2022
- 2022-08-23 CN CN202211012977.6A patent/CN115426253B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20010035379A (en) * | 2001-02-08 | 2001-05-07 | 오충용 | An web log analyzing service method, the system thereof and the recording medium thereof |
| CN105933268A (en) * | 2015-11-27 | 2016-09-07 | 中国银联股份有限公司 | Webshell detection method and apparatus based on total access log analysis |
| CN108509326A (en) * | 2018-04-09 | 2018-09-07 | 四川长虹电器股份有限公司 | A kind of service state statistical method and system based on nginx daily records |
| CN108833091A (en) * | 2018-05-28 | 2018-11-16 | 武汉斗鱼网络科技有限公司 | A kind of encryption method of journal file, decryption method and device |
| CN109039749A (en) * | 2018-08-10 | 2018-12-18 | 广州天予智能科技有限公司 | A kind of acquisition of remote journal and encryption transmission system and method |
| CN114640567A (en) * | 2022-02-23 | 2022-06-17 | 中银金融科技有限公司 | Apache log analysis method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115426253A (en) | 2022-12-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8626908B2 (en) | Distributed capture and aggregation of dynamic application usage information | |
| US7360251B2 (en) | Method and system for monitoring online behavior at a remote site and creating online behavior profiles | |
| EP0983581B1 (en) | System and method for analyzing remote traffic data in a distributed computing environment | |
| EP1386240B1 (en) | Synthetic transaction monitor | |
| US7853579B2 (en) | Methods, systems and software for identifying and managing database work | |
| WO2003017055A2 (en) | Method and system for delivering multiple services electronically to customers via a centralized portal architecture | |
| CN112039701B (en) | Interface call monitoring method, device, equipment and storage medium | |
| WO2002079909A2 (en) | Synthetic transaction monitor | |
| US7171464B1 (en) | Method of tracing data traffic on a network | |
| JP2002108824A (en) | System/method for auditing electronic business and recording medium with electronic business auditing program recorded thereon | |
| US20030055951A1 (en) | Products, apparatus and methods for handling computer software/hardware messages | |
| CN111241104A (en) | Operation auditing method and device, electronic equipment and computer-readable storage medium | |
| CN113449339A (en) | Log collection method, system, computer device and computer readable storage medium | |
| CN115426253B (en) | Web server monitoring method and system based on log file | |
| CN112187509A (en) | Multi-architecture cloud platform execution log management method, system, terminal and storage medium | |
| JP2004246747A (en) | Wrapping method and system of existing service | |
| CN112835863A (en) | Processing method and processing device of operation log | |
| CN112783920A (en) | Industrial Internet of things data real-time computing method and system based on data arrangement | |
| CN111885177A (en) | Biological information analysis cloud computing method and system based on cloud computing technology | |
| US20240070037A1 (en) | Multi-Computer System for Maintaining Application Programming Interface Stability with Shared Computing Infrastructure | |
| KR20100072515A (en) | Remote server log analysis system and the method thereof | |
| CN111861828A (en) | Method for accessing internal and external network resource sharing pre-subsystem in big data environment | |
| CN116186427A (en) | Time sequence data query method, time sequence data query device, time sequence data storage method, time sequence data query device, and time sequence data storage medium | |
| JP4184169B2 (en) | Service order information management control client device, service order information management control method for client device, service order information management control client program, and storage medium storing the client program | |
| US20020188647A1 (en) | Method and apparatus for optimizing data transfers between processes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |