CN114598589A - Usability analysis method and system based on log data and storage medium - Google Patents

Usability analysis method and system based on log data and storage medium Download PDF

Info

Publication number
CN114598589A
CN114598589A CN202210270005.0A CN202210270005A CN114598589A CN 114598589 A CN114598589 A CN 114598589A CN 202210270005 A CN202210270005 A CN 202210270005A CN 114598589 A CN114598589 A CN 114598589A
Authority
CN
China
Prior art keywords
request
requests
alarm
unavailability
alarm information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
CN202210270005.0A
Other languages
Chinese (zh)
Inventor
胡庆银
谭光柱
李英
华中生
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Pig Bajie Co ltd
Original Assignee
Pig Bajie Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Pig Bajie Co ltd filed Critical Pig Bajie Co ltd
Priority to CN202210270005.0A priority Critical patent/CN114598589A/en
Publication of CN114598589A publication Critical patent/CN114598589A/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/008—Reliability or availability analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/32—Monitoring with visual or acoustical indication of the functioning of the machine
    • G06F11/324—Display of status information
    • G06F11/327—Alarm or error message display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466—Performance evaluation by tracing or monitoring
    • G06F11/3476—Data logging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L61/00—Network arrangements, protocols or services for addressing or naming
    • H04L61/09—Mapping addresses
    • H04L61/10—Mapping addresses of different types
    • H04L61/103—Mapping addresses of different types across network layers, e.g. resolution of network layer into physical layer addresses or address resolution protocol [ARP]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/80—Database-specific techniques
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Quality & Reliability (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Hardware Design (AREA)
  • Computational Linguistics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses an availability analysis method, a system and a storage medium based on log data, wherein the method comprises the following steps: configuring a Nginx log directory; collecting request logs in the Nginx log catalog and transferring the request logs to a MongoDB database; screening and reading data in the MongoDB database, and analyzing requests of domain names according to the data to obtain request types of the domain names and the request quantity corresponding to each request type; acquiring an unavailability rate according to the request quantity, performing unavailability decision judgment according to the unavailability rate to acquire alarm information, and storing the alarm information into a Mysql database; and reading alarm information from the Mysql database and sending the alarm information to a related person in charge. The invention analyzes the unavailable alarm in time and pushes the unavailable alarm to the related responsible person, helps the operation and maintenance personnel to investigate the reason of the unavailability, reduces the difficulty of problem investigation and improves the availability of the whole online service.

Description

Usability analysis method and system based on log data and storage medium
Technical Field
The invention relates to the technical field of data availability analysis, in particular to an availability analysis method, an availability analysis system and a storage medium based on log data.
Background
With the rapid development of information technology in China, the internet is widely applied to life and work of people, but various faults, such as dead halt, blue screen and unavailable project, are frequently encountered in the daily use process.
However, there is no project availability warning in the existing online service system, and when a single project is unavailable, problems cannot be found in time, and related responsible persons cannot be notified in time. When a problem occurs in dependence on a certain bottom layer of a machine room, the corresponding machine room cannot be quickly positioned by judging whether the project is available or not, so that the time for the project to be unavailable is increased, the difficulty of problem troubleshooting is increased, and the usability of the whole online service is seriously influenced.
Disclosure of Invention
Aiming at the defects in the prior art, the invention provides a log data-based usability analysis method, a log data-based usability analysis system and a storage medium.
In a first aspect, a method for availability analysis based on log data includes the steps of:
configuring a Nginx log directory;
collecting request logs in the Nginx log catalog and transferring the request logs to a MongoDB database;
screening and reading data in the MongoDB database, and analyzing requests of domain names according to the data to obtain request types of the domain names and the request quantity corresponding to each request type;
acquiring an unavailability rate according to the request quantity, performing unavailability decision judgment according to the unavailability rate to acquire alarm information, and storing the alarm information into a Mysql database;
and reading alarm information from the Mysql database and sending the alarm information to a related person in charge.
Further, the screening and reading the data in the MongoDB database, and analyzing the request of the domain name according to the data to obtain the request type of the domain name and the number of requests corresponding to each request type specifically includes:
reading data in the MongoDB database according to screening conditions set by a user, and analyzing the request response time and the request response state of the domain name according to the data;
dividing the request into the following parts according to the request response time and the request response state: timeout requests, 5xx status requests, and normal requests;
and acquiring the number of the timeout requests, the 5xx requests and the normal requests.
Further, the acquiring of the unavailability rate according to the number of the requests, and performing unavailability decision-making judgment according to the unavailability rate to acquire alarm information, and storing the alarm information in a Mysql database specifically includes:
according to the formula: calculating the unavailability of the domain name (the number of overtime requests +5xx requests in minutes)/the total number of requests in minutes;
performing unavailability grade division according to the unavailability rate;
carrying out unavailable decision judgment according to the overtime request quantity, the 5xx request quantity and the unavailable grade, and sending an alarm;
and acquiring alarm information according to the alarm and storing the alarm information into a Mysql database.
Further, the making of unavailable decision judgment according to the number of timeout requests, the number of 5xx requests and the unavailable level and sending an alarm are specifically:
judging whether the total station is unavailable or not according to the number of the domain names of the unavailable grades, and if so, sending an alarm;
judging whether a chain reaction exists according to the domain name overtime request quantity and the 5xx request quantity, and if so, sending an alarm;
and judging whether to continuously alarm or not according to the number of the unavailable levels continuously appearing in the domain name, and if so, sending an alarm.
Further, still include: and displaying the timeout request quantity and the 5xx request quantity in a classified manner according to the machine room and the type.
In a second aspect, a log data-based availability analysis system includes:
a Nginx module: for configuring a Nginx log directory;
a log collection module: the system is used for collecting the request logs in the Nginx log catalog and transferring the request logs to a MongoDB database;
an analysis module: the MongoDB database is used for screening and reading data in the MongoDB database, and analyzing requests of domain names according to the data to obtain request types of the domain names and request quantity corresponding to each request type;
a decision module: the system comprises a Mysql database, a request quantity management server and a user terminal, wherein the Mysql database is used for storing the request quantity and the request quantity, and is used for acquiring the unavailability rate according to the request quantity, performing unavailability decision judgment according to the unavailability rate to acquire alarm information and storing the alarm information into the Mysql database;
an alarm module: and the Mysql database is used for reading alarm information from the Mysql database and sending the alarm information to a relevant responsible person.
Further, the analysis module is specifically configured to:
reading data in the MongoDB database according to screening conditions set by a user, and analyzing the request response time and the request response state of the domain name according to the data;
dividing the request into the following parts according to the request response time and the request response state: timeout requests, 5xx status requests, and normal requests;
and acquiring the number of the timeout requests, the 5xx requests and the normal requests.
Further, the decision module is specifically configured to:
according to the formula: calculating the unavailability of the domain name (the number of overtime requests +5xx requests in minutes)/the total number of requests in minutes;
performing unavailability grade division according to the unavailability rate;
carrying out unavailable decision judgment according to the overtime request quantity, the 5xx request quantity and the unavailable grade, and sending an alarm;
and acquiring alarm information according to the alarm and storing the alarm information into a Mysql database.
Further, the unavailable decision making judgment is performed according to the number of timeout requests, the number of 5xx requests and the unavailable level, and an alarm is sent, specifically:
judging whether the total station is unavailable or not according to the number of the domain names of the unavailable grades, and if so, sending an alarm;
judging whether a chain reaction exists according to the domain name overtime request quantity and the 5xx request quantity, and if so, sending an alarm;
and judging whether to continuously alarm or not according to the number of the unavailable levels continuously appearing in the domain name, and if so, sending an alarm.
In a third aspect, a computer-readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of the first aspect described above.
The invention has the beneficial effects that: the method and the system realize timely analysis of the unavailable alarm and push the alarm information to the relevant responsible persons, help the operation and maintenance personnel to investigate the reason of the unavailable alarm, enable the operation and maintenance personnel to visually know whether a single project is unavailable or a large number of projects are unavailable, reduce the difficulty in problem investigation and improve the availability of the whole online service.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below. Throughout the drawings, like elements or portions are generally identified by like reference numerals. In the drawings, elements or portions are not necessarily drawn to scale.
Fig. 1 is a flowchart of a method for availability analysis based on log data according to an embodiment of the present invention;
fig. 2 is a block diagram of a system for analyzing availability based on log data according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby.
It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.
As shown in fig. 1, a method for analyzing availability based on log data includes the steps of:
s1: configuring a Nginx log directory;
specifically, the request log of the application is written into the corresponding log directory, the Nginx log directory is configured, and the format of the Nginx log is specified, so that the subsequent log can be read conveniently.
S2: collecting request logs in the Nginx log catalog and transferring the request logs to a MongoDB database;
specifically, the request log in the Nginx log directory is read out and is transferred to the MongoDB database, and preferably, the MongoDB database is written in the C + + language and is an open source database system based on distributed file storage.
S3: screening and reading data in the MongoDB database, and analyzing requests of domain names according to the data to obtain request types of the domain names and the request quantity corresponding to each request type;
specifically, corresponding data in the MongoDB database are read according to screening conditions set by a user, a request of a domain name is analyzed according to the data, the request response time and the request response state of the domain name are obtained, and the request is divided into three types according to the request response time and the request response state, wherein the three types include an overtime request, a 5xx state request and a normal request. Taking the request of domain name www.xxx.com as an example, the request is classified into the above three types: when the request response time exceeds a certain set response time, assuming that the set response time is 2.5s, if the request response time exceeds 2.5s, the request is an overtime request; when the request response status is 5xx, if 502, the request response status is 5 xx; when the request response state is not 5xx and the request response time does not exceed the set response time, the request is a normal request. And after the classification is finished, acquiring the number of the overtime requests, the number of the 5xx requests and the number of the normal requests.
S4: acquiring an unavailability rate according to the request quantity, performing unavailability decision judgment according to the unavailability rate to acquire alarm information, and storing the alarm information into a Mysql database;
specifically, according to the request types analyzed in step S3 and the number of requests corresponding to each request type, the unavailability of the domain name is calculated, and the calculation formula is: (number of timeout requests within minute +5xx requests number)/total number of requests within minute. And performing unavailability grade division according to the acquired unavailability rate, wherein the division rule is as follows:
if the unavailability rate is greater than 5%, the unavailability grade is P0;
if the unavailability is more than 3% and less than or equal to 5%, the unavailability grade is P1;
if the unavailability is more than 1% and less than or equal to 3%, the unavailability grade is P2;
if the unavailability is more than 0.5% and less than or equal to 1%, the unavailability grade is P3;
further, after the unavailable grade division is completed, unavailable decision judgment is carried out according to the overtime request quantity, the 5xx request quantity and the unavailable grade, and the decision judgment comprises total-station unavailable judgment, chain reaction judgment and continuous alarm judgment. The total station unavailability judgment is to judge whether total station unavailability is caused according to the number of domain names of the unavailability levels, and specifically includes: and when the number of the domain names at a certain unavailable level exceeds a first judgment threshold value, judging that the total station is unavailable, and sending an alarm at the moment. For example, the first determination threshold is set to 5, if the unavailability rate is greater than 5% in a minute, that is, the unavailability level is P0, when the number of domain names with the unavailability level P0 exceeds 5, it is determined that the total station is caused to be unavailable, and an alarm is issued.
Further, the chain reaction is judged to be whether the chain reaction is judged according to the domain name timeout request quantity and the 5xx request quantity, and specifically, the method comprises the following steps: the setting condition "the sum of the timeout request quantity and the 5xx request quantity exceeds the setting request quantity value", and when the domain name quantity meeting the setting condition exceeds the second judgment threshold, the chain reaction is judged. For example, if the request quantity value is set to 10 and the second determination threshold is 5, the setting condition is that the sum of the timeout request quantity and the 5xx request quantity exceeds 10 ", and when the domain name satisfying the setting condition exceeds the second determination threshold 5, it is determined to be a chain reaction, and an alarm is issued.
Further, the continuous alarm judgment is to judge whether to continuously alarm according to the number of the unavailable levels continuously appearing in the domain name, and if so, the alarm is sent out, specifically: and when the number of the domain names with the continuously unavailable levels exceeds a third judgment threshold value, judging that the continuous alarm is performed. For example, the third determination threshold is set to 7, and when four levels of P0 to P3 continue to appear in 7 domain names, it is determined that the alarm continues, and the alarm is issued.
Preferably, in the practical application process, the first set threshold, the second set threshold and the third set threshold may be set according to practical situations, and this embodiment is only an example.
Further, after the judgment is completed, corresponding alarm information is obtained according to the sent alarm, and the alarm information is stored in the Mysql database.
S5: reading alarm information from the Mysql database and sending the alarm information to a related person in charge;
specifically, the alarm information is obtained from the Mysql database in an active reading mode and sent to the corresponding related responsible person. After the project responsible person, the OA and the operation and maintenance receive the corresponding alarm, the problem of a single project or the problem of a batch project can be quickly positioned, whether the availability of the whole station is influenced or not can be judged, and whether the problem of a single machine room or not can be judged from the alarm information.
Preferably, in the analysis process, after the timeout request number and the 5xx request number are obtained, the timeout request number and the 5xx request number of each machine room can be displayed in a machine room and classified mode, so that a relevant responsible person can conveniently and accurately find which machine room has a problem in time, and can visually find that only the request is slow or an error occurs in application.
By implementing the method, the unavailable alarm is analyzed in time and the alarm information is pushed to the relevant responsible person, so that the operation and maintenance personnel are helped to investigate the reason of the unavailability, the operation and maintenance personnel can visually know whether a single project is unavailable or a large batch of projects are unavailable, the difficulty in problem investigation is reduced, and the availability of the whole online service is improved.
Based on the same inventive concept, an embodiment of the present invention provides a system for availability analysis based on log data, as shown in fig. 2, including:
a Nginx module: for configuring a Nginx log directory;
a log collection module: the system is used for collecting the request logs in the Nginx log catalog and transferring the request logs to a MongoDB database;
an analysis module: the system is used for screening and reading request logs in the MongoDB database, analyzing request types according to the request logs and acquiring the request quantity corresponding to each request type according to the request types;
a decision module: the system comprises a Mysql database, a request quantity management server and a user terminal, wherein the Mysql database is used for storing the request quantity and the request quantity, and is used for acquiring the unavailability rate according to the request quantity, performing unavailability decision judgment according to the unavailability rate to acquire alarm information and storing the alarm information into the Mysql database;
an alarm module: and the Mysql database is used for reading alarm information from the Mysql database and sending the alarm information to a related responsible person.
Further, the analysis module is specifically configured to:
reading data in the MongoDB database according to screening conditions set by a user, and analyzing the request response time and the request response state of the domain name according to the data;
dividing the request into the following parts according to the request response time and the request response state: timeout requests, 5xx status requests, and normal requests;
and acquiring the timeout request quantity, the 5xx request quantity and the normal request quantity.
Further, the decision module is specifically configured to:
according to the formula: calculating the unavailability of the domain name (the number of overtime requests +5xx requests in minutes)/the total number of requests in minutes;
performing unavailability grade division according to the unavailability rate;
carrying out unavailable decision judgment according to the overtime request quantity, the 5xx request quantity and the unavailable grade, and sending an alarm;
and acquiring alarm information according to the alarm and storing the alarm information into a Mysql database.
Further, the unavailable decision making judgment is performed according to the number of timeout requests, the number of 5xx requests and the unavailable level, and an alarm is sent, specifically:
judging whether the total station is unavailable or not according to the number of the domain names of the unavailable level, and if so, sending an alarm;
judging whether a chain reaction exists according to the domain name overtime request quantity and the 5xx request quantity, and if so, sending an alarm;
and judging whether to continuously alarm or not according to the number of the unavailable levels continuously appearing in the domain name, and if so, sending an alarm.
Further, corresponding to the foregoing method and system, an embodiment of the present invention further provides a readable storage medium storing a computer program, where the computer program includes program instructions, and when the program instructions are executed by a processor, the computer program instructions implement: the usability analysis method based on the log data is described.
The computer readable storage medium may be an internal storage unit of the system according to any of the foregoing embodiments, for example, a hard disk or a memory of the system. The computer readable storage medium may also be an external storage device of the system, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) Card, Flash memory Card (Flash Card), etc. provided on the system. Further, the computer readable storage medium may also include both an internal storage unit and an external storage device of the system. The computer-readable storage medium is used for storing the computer program and other programs and data required by the system. The computer readable storage medium may also be used to temporarily store data that has been output or is to be output.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, the division of the units is only one logical functional division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. A log data-based usability analysis method is characterized by comprising the following steps:
configuring a Nginx log directory;
collecting request logs in the Nginx log catalog and transferring the request logs to a MongoDB database;
screening and reading data in the MongoDB database, and analyzing requests of domain names according to the data to obtain request types of the domain names and the number of requests corresponding to each request type;
acquiring an unavailability rate according to the request quantity, performing unavailability decision judgment according to the unavailability rate to acquire alarm information, and storing the alarm information into a Mysql database;
and reading alarm information from the Mysql database and sending the alarm information to a related person in charge.
2. The method according to claim 1, wherein the screening and reading of the data in the MongoDB database and the analysis of the domain name request according to the data are performed to obtain the request type of the domain name and the number of requests corresponding to each request type, specifically:
reading data in the MongoDB database according to screening conditions set by a user, and analyzing the request response time and the request response state of the domain name according to the data;
dividing the request into the following parts according to the request response time and the request response state: timeout requests, 5xx status requests, and normal requests;
and acquiring the number of the timeout requests, the 5xx requests and the normal requests.
3. The method according to claim 2, wherein the unavailability rate is obtained according to the number of requests, the unavailability decision judgment is performed according to the unavailability rate to obtain alarm information, and the alarm information is stored in a Mysql database, specifically:
according to the formula: calculating the unavailability of the domain name (the number of overtime requests +5xx requests in minutes)/the total number of requests in minutes;
performing unavailability grade division according to the unavailability rate;
carrying out unavailable decision judgment according to the overtime request quantity, the 5xx request quantity and the unavailable grade, and sending an alarm;
and acquiring alarm information according to the alarm and storing the alarm information into a Mysql database.
4. The method according to claim 3, wherein the unavailability decision is determined according to the number of timeout requests, the number of 5xx requests, and the unavailability level, and an alarm is issued, specifically:
judging whether the total station is unavailable or not according to the number of the domain names of the unavailable grades, and if so, sending an alarm;
judging whether a chain reaction exists according to the domain name overtime request quantity and the 5xx request quantity, and if so, sending an alarm;
and judging whether to continuously alarm or not according to the number of the unavailable levels continuously appearing in the domain name, and if so, sending an alarm.
5. The log data-based availability analysis method of claim 4, further comprising: and displaying the timeout request quantity and the 5xx request quantity in a classified manner according to the machine room and the type.
6. A log data-based availability analysis system, comprising:
a Nginx module: for configuring a Nginx log directory;
a log collection module: the system is used for collecting the request logs in the Nginx log catalog and transferring the request logs to a MongoDB database;
an analysis module: the MongoDB database is used for screening and reading data in the MongoDB database, and analyzing requests of domain names according to the data to obtain request types of the domain names and request quantity corresponding to each request type;
a decision module: the system comprises a Mysql database, a request quantity management server and a user terminal, wherein the Mysql database is used for storing the request quantity and the request quantity, and is used for acquiring the unavailability rate according to the request quantity, performing unavailability decision judgment according to the unavailability rate to acquire alarm information and storing the alarm information into the Mysql database;
an alarm module: and the Mysql database is used for reading alarm information from the Mysql database and sending the alarm information to a related responsible person.
7. The system of claim 6, wherein the analysis module is specifically configured to:
reading data in the MongoDB database according to screening conditions set by a user, and analyzing the request response time and the request response state of the domain name according to the data;
dividing the request into the following parts according to the request response time and the request response state: timeout requests, 5xx status requests, and normal requests;
and acquiring the timeout request quantity, the 5xx request quantity and the normal request quantity.
8. The system according to claim 7, wherein the decision module is specifically configured to:
according to the formula: calculating the unavailability of the domain name (the number of overtime requests +5xx requests in minutes)/the total number of requests in minutes;
performing unavailability grade division according to the unavailability rate;
carrying out unavailable decision judgment according to the overtime request quantity, the 5xx request quantity and the unavailable grade, and sending an alarm;
and acquiring alarm information according to the alarm and storing the alarm information into a Mysql database.
9. The system according to claim 8, wherein the system performs the unavailability decision-making judgment according to the number of timeout requests, the number of 5xx requests, and the unavailability level, and issues an alarm, specifically:
judging whether the total station is unavailable or not according to the number of the domain names of the unavailable level, and if so, sending an alarm;
judging whether a chain reaction exists according to the domain name overtime request quantity and the 5xx request quantity, and if so, sending an alarm;
and judging whether to continuously alarm or not according to the number of the unavailable levels continuously appearing in the domain name, and if so, sending an alarm.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program comprising program instructions that, when executed by a processor, cause the processor to carry out the method according to any one of claims 1-5.
CN202210270005.0A 2022-03-18 2022-03-18 Usability analysis method and system based on log data and storage medium Withdrawn CN114598589A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210270005.0A CN114598589A (en) 2022-03-18 2022-03-18 Usability analysis method and system based on log data and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210270005.0A CN114598589A (en) 2022-03-18 2022-03-18 Usability analysis method and system based on log data and storage medium

Publications (1)

Publication Number Publication Date
CN114598589A true CN114598589A (en) 2022-06-07

Family

ID=81810316

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210270005.0A Withdrawn CN114598589A (en) 2022-03-18 2022-03-18 Usability analysis method and system based on log data and storage medium

Country Status (1)

Country Link
CN (1) CN114598589A (en)

Similar Documents

Publication Publication Date Title
CN109345417B (en) Online assessment method and terminal equipment for business personnel based on identity authentication
CN110610431A (en) Intelligent claim settlement method and intelligent claim settlement system based on big data
CN111340584A (en) Method, device, equipment and storage medium for determining fund side
CN111984495A (en) Big data monitoring method and device and storage medium
CN113949652B (en) User abnormal behavior detection method and device based on artificial intelligence and related equipment
CN113342939B (en) Data quality monitoring method and device and related equipment
CN114757639A (en) Data processing method, device, equipment and storage medium
CN113780986A (en) Measurement method, system, equipment and medium for software development process
CN116074183B (en) C3 timeout analysis method, device and equipment based on rule engine
CN114598589A (en) Usability analysis method and system based on log data and storage medium
CN115766793A (en) Based on data center computer lab basis environmental monitoring alarm device
CN112751976B (en) Agent association method, system, equipment and storage medium based on authentication log
CN116062009A (en) Fault analysis method, device, electronic equipment and storage medium
CN115222549A (en) Risk assessment processing method and device, computer equipment and storage medium
CN113327341A (en) Equipment early warning system, method and storage medium based on network technology
CN112580089A (en) Information leakage early warning method, device and system, storage medium and electronic device
CN117709911B (en) Attendance management system and attendance management method
CN108776935A (en) A kind of audit platform suitable for medical system
CN113986698A (en) Communication log quantity diagnosis method, device and storage medium
CN114356928A (en) Risk analysis method and device, electronic equipment and storage medium
CN116230174A (en) Hospital intelligent registration management system based on big data
CN113191688A (en) Commercial data diagnosis and analysis method based on Internet of things and big data
CN116385155A (en) Risk list processing method, apparatus, device and storage medium
CN112463801A (en) Report form pushing method and device, terminal equipment and storage medium
CN118053125A (en) Project progress visualization image supervision method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WW01 Invention patent application withdrawn after publication
WW01 Invention patent application withdrawn after publication

Application publication date: 20220607