CN114218299A

CN114218299A - Method, device, equipment and storage medium for monitoring interface response condition

Info

Publication number: CN114218299A
Application number: CN202111541707.XA
Authority: CN
Inventors: 张洪姣
Original assignee: Xinao Shuneng Technology Co Ltd
Current assignee: Xinao Shuneng Technology Co Ltd
Priority date: 2021-12-16
Filing date: 2021-12-16
Publication date: 2022-03-22

Abstract

The disclosure provides a monitoring method, a monitoring device, equipment and a storage medium for interface response conditions. The method comprises the following steps: acquiring interface request data generated by a user access interface in a preset time period from a distributed file system, and synchronizing the interface request data into a data warehouse tool; performing data cleaning on the interface request data by using a data warehouse tool, and integrating the interface request data after the data cleaning according to domain name information of the interface to obtain preset table data; and inquiring the table data by using a preset data inquiry script to obtain target data in the table data, and generating an interface response report based on the target data and the report configuration created in advance so as to monitor the response condition of the interface according to the interface response report. The method and the device realize automatic monitoring of the interface response condition, improve the efficiency of troubleshooting the interface problem and improve the stability and reliability of interface service.

Description

Method, device, equipment and storage medium for monitoring interface response condition

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a method, an apparatus, a device, and a storage medium for monitoring an interface response condition.

Background

With the development and popularization of the internet and big data technology, program developers have higher and higher requirements on the reliability and stability of application programs. Because the operation and maintenance personnel need to develop an interface to provide the interface for the external user to access, when the external user accesses the interface every time, if the response time of the interface is too slow, on one hand, the user experience is very poor, and on the other hand, the operation and maintenance personnel are added to record and track the fault condition, so that the monitoring of the response condition of the application program interface is of great significance.

In the prior art, data generated when an external user requests an interface is JSON format data, the JSON data cannot directly check the problem of the interface, difficulty is caused to operation and maintenance personnel to record and troubleshoot the problem of the interface, and the operation and maintenance personnel can not only have high labor cost but also have long flow period according to a mode of feeding back the problem of the interface by the user. Because the operation and maintenance personnel cannot intuitively acquire the response condition of the interface, the monitoring of the application program interface cannot be realized, and the stability and reliability of the interface service cannot be ensured.

In view of the above problems in the prior art, it is desirable to provide a monitoring method for monitoring interface response conditions, which can automatically monitor the interface response conditions, improve the efficiency of interface examination, and ensure the stability and reliability of interface services.

Disclosure of Invention

In view of this, embodiments of the present disclosure provide a method, an apparatus, a device, and a storage medium for monitoring an interface response condition, so as to solve the problem in the prior art that the efficiency of troubleshooting on an interface response problem is low, which results in the reduction of stability and reliability of an interface service.

In a first aspect of the embodiments of the present disclosure, a method for monitoring an interface response condition is provided, including: acquiring interface request data generated by a user access interface in a preset time period from a distributed file system, and synchronizing the interface request data into a data warehouse tool; performing data cleaning on the interface request data by using a data warehouse tool, and integrating the interface request data after the data cleaning according to domain name information of the interface to obtain preset table data; and inquiring the table data by using a preset data inquiry script to obtain target data in the table data, and generating an interface response report based on the target data and the report configuration created in advance so as to monitor the response condition of the interface according to the interface response report.

In a second aspect of the embodiments of the present disclosure, a device for monitoring interface response conditions is provided, including: the acquisition module is configured to acquire interface request data generated by a user accessing an interface within a preset time period from the distributed file system and synchronize the interface request data to the data warehouse tool; the integration module is configured to perform data cleaning on the interface request data by using a data warehouse tool and integrate the interface request data after the data cleaning according to domain name information of the interface to obtain preset table data; the generating module is configured to query the table data by using a preset data query script to obtain target data in the table data, and generate an interface response report based on the target data and a report configuration created in advance so as to monitor a response condition of an interface according to the interface response report.

The embodiment of the present disclosure adopts at least one technical scheme that can achieve the following beneficial effects:

interface request data generated by a user accessing an interface within a preset time period are obtained from a distributed file system, and the interface request data are synchronized to a data warehouse tool; performing data cleaning on the interface request data by using a data warehouse tool, and integrating the interface request data after the data cleaning according to domain name information of the interface to obtain preset table data; and inquiring the table data by using a preset data inquiry script to obtain target data in the table data, and generating an interface response report based on the target data and the report configuration created in advance so as to monitor the response condition of the interface according to the interface response report. The method and the device can realize automatic interface response monitoring and improve the troubleshooting efficiency of interface response problems, thereby further improving the stability and reliability of interface service and finally improving the experience of users.

Drawings

To more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without inventive efforts.

Fig. 1 is a schematic flowchart of a method for monitoring interface response conditions according to an embodiment of the present disclosure;

FIG. 2 is a schematic flow chart of ETL data processing of log data based on a Hive data warehouse tool provided by the embodiment of the present disclosure;

FIG. 3 is a schematic flow chart illustrating the development of an interface response report based on a report platform according to an embodiment of the present disclosure;

FIG. 4 is a schematic structural diagram of a monitoring apparatus for interface response provided by an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the disclosed embodiments. However, it will be apparent to one skilled in the art that the present disclosure may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present disclosure with unnecessary detail.

As described above, with the development of the era of mobile internet, enterprises have increasingly high requirements for reliability and stability of applications developed by operation and maintenance staff of companies. The operation and maintenance personnel need to develop an interface to provide access for an external user, but when the external user accesses the interface each time, if the response time of the interface is too slow, on one hand, the user experience is very poor; on the other hand, for the problems existing in interface response, operation and maintenance personnel need to record and track fault conditions, so that the workload of the operation and maintenance personnel is increased to a great extent, project progress is influenced, and the whole development of a company is not facilitated.

As is well known, the monitoring of the operation and maintenance to the service all the time is one of the most important links in the whole operation and maintenance and even the whole product life cycle, problems are found by early warning in time in advance, and detailed data are provided for tracing and positioning the problems afterwards. However, after the interface is accessed, the data structure for counting the high failure rate and the fast and slow response time of the interface is data in a JSON format, and which interfaces have problems cannot be directly found based on the JSON data, so that operation and maintenance personnel can intuitively find which interfaces have problems, and the monitoring of the interface response condition is of great significance. In addition, related reasons are also checked and optimized for interfaces with high failure rate and interfaces with slow response time, namely corresponding improvement and optimization are performed for different reasons, and customer experience is improved by improving the loading speed of the report.

Taking the field of energy data platforms as an example, the following details are provided to solve the problems in the conventional interface troubleshooting and troubleshooting, and may specifically include the following contents:

the application program of the enterprise operation and maintenance personnel records log information of each time when an external user requests an interface, and after the interface is accessed by the external user, the operation and maintenance personnel cannot know relevant information such as high interface failure rate, slow interface response time and the like due to the fact that the data format for counting the interface failure rate and the response time is the JSON format. When an external user accesses the interface, if the interface response fails or the response time of the interface is long, the experience of the user is very poor, the loss of the user is easily caused, and the popularization and the application of the product are not facilitated.

When the interface fails to be accessed or the response time is too long, the user can feed back the situation, and then the requirement for solving the problems is provided for operation and maintenance personnel of the enterprise through a certain flow, and the operation and maintenance personnel need to track and record the problems according to the feedback of the user and solve the problems according to the result of the interface investigation. The whole process of interface troubleshooting and solution needs manual full-flow participation, so that the cost is high, the period is long, and when other development requirements of an enterprise are increased, scheduling is needed, so that the waiting time of a user is too long, the project is delayed, and the enterprise development is not facilitated.

In view of the above problems in the prior art, the embodiment of the present disclosure synchronizes log data generated when a user accesses an interface in a distributed file system to a data warehouse tool, cleans and processes interface request data (i.e., log data) of the user by using the data warehouse tool, summarizes the log data according to a domain name, then obtains target data for generating an interface response report by using a query engine in a report platform, and develops the interface response report by using the target data, so that an operation and maintenance worker can directly monitor an actual response condition of the interface based on the interface response report, thereby finding and solving problems existing in the interface in advance. Therefore, the cost for solving the interface problem is reduced, the time for checking the interface is saved, the service quality of the interface and the reliability and stability of the interface service are improved, and the long-term development of enterprises is facilitated.

It should be emphasized that, in order to ensure the stability and reliability of the interface service and improve the user experience, the actual situation that the interface is accessed and then responded needs to be comprehensively grasped, the embodiment of the present disclosure counts data from the following three dimensions, that is, the application health report, the interface failure rate, and the interface response time, respectively. By developing the interface response report forms corresponding to the three dimensions, operation and maintenance personnel can directly know which interfaces have high failure rate and slow response time, and then carry out targeted recording and tracking, check related reasons and carry out corresponding optimization, so that the interface response speed is improved, and the interface access amount is increased.

Fig. 1 is a schematic flowchart of a method for monitoring an interface response condition according to an embodiment of the present disclosure. The method of monitoring the interface response condition of fig. 1 may be performed by a server. As shown in fig. 1, the method for monitoring the interface response condition may specifically include:

s101, acquiring interface request data generated by a user accessing an interface within a preset time period from a distributed file system, and synchronizing the interface request data into a data warehouse tool;

s102, performing data cleaning on the interface request data by using a data warehouse tool, and integrating the interface request data after the data cleaning according to domain name information of the interface to obtain preset table data;

s103, the table data is inquired by using a preset data inquiry script to obtain target data in the table data, and an interface response report is generated based on the target data and the report configuration established in advance so as to monitor the response condition of the interface according to the interface response report.

Specifically, the distributed file system of the embodiment of the present disclosure may adopt an HDFS file system, where the HDFS is a distributed file system, and stores oversized files in a streaming data access mode, and stores data in blocks to different machines in a business hardware cluster. The distributed storage of the log information is realized by storing the log information generated when a user in a background database of an application program accesses an interface into an HDFS file system.

Further, the background database of the application program can be a MySQL database, which is a relational database management system and belongs to Oracle products. In terms of WEB applications, MySQL is one of the best RDBMS (Relational Database Management System) application software, and the Relational Database stores data in different tables, thereby increasing speed and flexibility. The SQL language used by MySQL is the most common standardized language for accessing databases.

Further, a Data Warehouse (DW) is obtained by performing system processing, summarizing and arrangement on the basis of extracting and cleaning original dispersed database Data. The purpose of data warehouse construction is to provide functional analysis and decision support for front-end query and analysis as a basis. The embodiment of the present disclosure uses a Hive data warehouse tool, which is mainly used to process structured data, and the Hive data warehouse is generally divided into 4 levels, i.e., an ODS layer, a DWD layer, a DWS layer, and an ADS layer, and each level is used to store different types of tables.

Further, the report platform may adopt a universal report platform, and in the following embodiments, the report platform in the embodiments of the present disclosure may also be replaced by the universal report platform. And in the universal report platform, the widget is the diagram content of the report. In the embodiment of the disclosure, the query operation in the universal report platform uses Presto query engine, and Presto is used for querying data in the Hive data warehouse tool.

According to the technical scheme provided by the embodiment of the disclosure, interface request data generated by a user accessing an interface in a preset time period are acquired from a distributed file system, and the interface request data are synchronized to a data warehouse tool; performing data cleaning on the interface request data by using a data warehouse tool, and integrating the interface request data after the data cleaning according to domain name information of the interface to obtain preset table data; and inquiring the table data by using a preset data inquiry script to obtain target data in the table data, and generating an interface response report based on the target data and the report configuration created in advance so as to monitor the response condition of the interface according to the interface response report. The method and the device can realize automatic interface response monitoring and improve the troubleshooting efficiency of interface response problems, thereby further improving the stability and reliability of interface service and finally improving the experience of users.

In some embodiments, before obtaining the interface request data generated by the user accessing the interface within the preset time period from the distributed file system, the method further includes: and sending interface request data stored in a background database of the application program to the distributed file system, wherein the interface request data comprises log data generated by the interface responding to the access request of the user when the user accesses the interface each time, and the log data is data in a JSON format.

Specifically, the application program of the embodiment of the present disclosure may be an application program or a client corresponding to the energy data platform, and an external user accesses the energy data platform through the client each time corresponding log information is generated, where the log information is stored in a database of the energy data platform. In practical applications, the interface request data stored in the background database may be sent to the distributed file system for saving, and thus the log data synchronized to the data warehouse tool is the data from the distributed file system.

Further, JSON (JSON) is a lightweight data exchange format. It stores and represents data in a text format that is completely independent of the programming language, based on a subset of ECMAScript (js specification set by the european computer association). The simple and clear hierarchical structure enables JSON to become an ideal data exchange language, so that the JSON is easy to read and write, is easy to analyze and generate by a machine, and effectively improves the network transmission efficiency.

Further, the interface of the embodiment of the present disclosure may be a URL interface, where the URL is a character string used to describe information resources on the Internet, and is mainly used in various client programs and server programs. The URL can be used for describing various information resources in a uniform format, including files, addresses and directories of servers and the like. A URL generally consists of three parts: protocol (or called service mode), host IP address (sometimes including port number) where the resource is stored, and specific address of host resource.

In some embodiments, synchronizing the interface request data into the data warehouse tool comprises: the interface request data generated in the preset time period are synchronized to a data warehouse tool from a distributed file system by using a data scheduling platform, and the interface request data are stored in an original data layer of the data warehouse tool, so that the structured interface request data are mapped to a database table in the original data layer, wherein the data warehouse tool adopts a Hive data warehouse tool.

In particular, the ETL data processing method can be used to synchronize interface request data from the distributed file system to the Hive data warehouse tool. And in the synchronous process, processing and processing the interface request data by utilizing ETL data processing to obtain table data. ETL refers to a process of extracting, cleaning, converting, and loading data from a source (e.g., a business system, log information, internet of things data, etc.) to a destination (typically, a data warehouse). ETL is equivalent to a bridge that transfers data from a distributed file system into a data warehouse. In actual practice, the databases, data warehouse tools, or programming types employed by the different systems may vary.

Furthermore, Hive is a data warehouse tool based on Hadoop, and can map the structured data file into a database table and provide a query function similar to SQL. The UDF function adopted in Hive is a Hive self-defined function, the Hive self-contained function cannot completely meet the service requirement, the self-defined function is needed to be used, and different processing logics can be used for processing different requirements through the self-defined function.

Further, the disclosed embodiments synchronize interface request data in the distributed file system directly into the ODS layer in the Hive data warehouse tool through ETL. The ODS (operation Data store) of Hive is an original Data layer used for storing original Data, original log Data can be directly loaded from a distributed file system, and the log Data keeps an original appearance and is not processed. In other words, the ODS layer can directly obtain the original log data in the distributed file system, and can modify the table name of the structured original log data, or does not perform any processing.

In some embodiments, the data warehouse tool is used to perform data washing on the interface request data, and the interface request data after data washing is integrated according to domain name information of the interface to obtain predetermined table data, including: performing data cleaning operation on a database table corresponding to interface request data in an original data layer, and performing desensitization treatment on the database table; and summarizing the database tables subjected to data cleaning and desensitization treatment according to the domain name information corresponding to the interface so as to integrate the database tables into table data.

Specifically, after the interface request data (i.e. the log data) is synchronized from the distributed file system to the Hive data warehouse tool, ETL data processing needs to be performed on the log data, and a detailed description will be given below of a process of performing ETL data processing on the log data in the Hive data warehouse tool with reference to the accompanying drawings and specific embodiments. Fig. 2 is a schematic flow chart of ETL data processing on log data based on a Hive data warehouse tool according to an embodiment of the present disclosure. As shown in fig. 2, the specific process of ETL data processing of log data by the Hive-based data warehouse tool mainly includes the following steps:

s201, cleaning original log data in an ODS layer (such as removing null values, dirty data and the like) by using a DWD (data Warehouse detail) layer in a Hive data Warehouse tool, and performing desensitization processing on the original log data;

s202, a DWS layer (Data Warehouse Service) in the Hive Data Warehouse tool collects the information in the ODS layer table;

s203, the obtained table Data after the summary is transmitted to an ADS (application Data store) layer to generate an ADS layer table, and the ADS layer table is an object of the report platform query.

Here, log data synchronized from the distributed file system to the ODS layer may correspond to three ODS layer tables, that is, a first ODS layer table, a second ODS layer table, and a third ODS layer table, where the first ODS layer table is a table obtained by filtering data that does not meet a condition, the second ODS layer table includes information such as a person in charge of a URL interface and a service domain, the third ODS layer table is mainly used for storing log data in a JSON format, and the log data includes response data generated when a user requests access to an interface each time.

In some embodiments, querying the table data by using a preset data query script to obtain target data in the table data includes: based on a preset report platform, the data query script in the report platform is used for executing calling operation on the data warehouse tool so as to obtain target data from table data stored in the data warehouse tool, wherein the target data is used for generating an interface response report.

Specifically, the table data obtained after ETL processing is data stored in Hive, and the development stage of the report is completed in the universal report platform, that is, Hive is only used for storing the table data and is not responsible for developing a final report, and therefore the data in Hive needs to be called by the universal report platform to develop the report. The data set of the universal report platform comprises SQL scripts (data query scripts) for data query, and the universal report platform queries the table data obtained after processing in the Hive data warehouse tool through a Presto query engine to obtain target data for developing reports, so that the universal report platform can develop reports based on the target data.

It should be noted that data of related reports are stored in the Hive, but the data are only data in a column and are not shown in the reports, and the reports do not necessarily use all data in all fields in the Hive table; therefore, the required report data can be inquired by adding an SQL script in the data set by using the report platform.

In some embodiments, generating an interface response report based on the target data and the pre-created report configuration includes: in the report platform, report configuration is created based on the report style, and the interface response report is developed by utilizing the report configuration and the target data so as to generate an interface response report corresponding to the interface, wherein the interface response report comprises a first report, a second report and a third report; the first report is used for representing the health condition of the application program, the second report is used for representing the failure rate of the interface, and the third report is used for representing the response time of the interface.

Specifically, after the target data is obtained by the Presto query engine, the report platform performs report development operation based on the target data by using preset report configuration, so as to obtain an interface response report for monitoring interface response conditions by operation and maintenance personnel. Fig. 3 is a schematic flowchart of report development based on interface response in the report platform according to the embodiment of the present disclosure. As shown in fig. 3, the report development process based on the interface response report platform mainly includes the following steps:

s301, inquiring table data in the Hive data warehouse tool by using an inquiry engine to obtain target data;

s302, the report platform develops the report based on preset report configuration and target data;

s303, generating an interface response report meeting the format requirement of a preset report style according to the report development result;

and finally, sending the interface response report to a decision layer or an operation and maintenance management department so as to finish the release of the report. In practical application, the statistical report finally developed by the report platform can be three, namely an application health report, an interface failure rate report and an interface response time report. The following describes the contents of the three interface response reports in detail with reference to a specific embodiment, which may specifically include the following contents:

in the application health report, the following may be included: the method comprises the steps of domain name, the number of interfaces with high failure rate (the response state type is that the ratio of the number of http5xx to the total number of interfaces is more than 5%, namely the interfaces with high failure rate), the number of interfaces with slow response time (the response time is more than 3 seconds, namely the interfaces with slow response time), report generation time, responsible persons and business domains.

In the interface failure rate report, the following may be included: domain name, url _ pattern, http5xx request total, error rate (http 5xx duty), request total, http4xx request total, http4xx duty, http3xx request total, http3xx duty, http2xx request total, http2xx duty.

In the interface response time report, the following may be included: the method comprises the following steps of domain name, url _ pattern, report generation time, request total number, response time total number, average response time, response time t (t <1s) total number, response time t (t <1s) ratio, response time t (1s < ═ t <3s) total number, response time t (1s < ═ t <3s) ratio, response time t (3s < ═ t <5s) total number, response time t (3s < t <5s) ratio, response time t (t > -5 s) total number and response time t (t > -5 s) ratio.

In some embodiments, monitoring the response condition of the interface according to the interface response report includes: monitoring the response condition of the interfaces according to the number of the interfaces with the failure rate larger than the threshold value and the number of the interfaces with the response time larger than the threshold value in the first report; and/or monitoring the response condition of the interface according to the total number of the requests of each type of protocol and the request proportion of each type of protocol in the second report; and/or monitoring the response condition of the interface according to the total response time of each segment in the third report and the proportion of the response time of each segment.

Specifically, after the three interface response reports are obtained, the interface response reports are issued to the operation and maintenance platform, and operation and maintenance personnel perform problem troubleshooting and monitoring on the interface of the application program according to the interface response reports. For example, the operation and maintenance staff may perform troubleshooting and resolution on the problem of the domain name interface according to the abnormal data presented in the interface response report, for example, when the number of interfaces in the first report (i.e., the application health report) for which the failure rate of a domain name interface is greater than the threshold is large, or when the number of interfaces for which the response time is greater than the threshold is large, the interface is regarded as an abnormal interface, so as to perform further troubleshooting on the interface; for another example, when the total number of requests of the http5xx protocol of a certain domain name interface in the second report (i.e., the interface failure rate report) exceeds a set value, the interface is used as an abnormal interface to perform problem troubleshooting.

Furthermore, operation and maintenance personnel can perform problem troubleshooting and corresponding optimization according to the interface response conditions presented by the interface response report, the decision layer can also perform analysis and decision according to the actual conditions of interface loading, and the decision layer can also perform analysis and prediction on the response capability of each application program, so that the operation and maintenance are promoted to perform service management.

Based on the foregoing embodiment, taking the development process of the interface response report in the actual application scenario as an example, it can be seen that the main purpose of the technical solution of the present disclosure includes:

when the interfaces are accessed by an external user, the method can count which interfaces have high failure rate and slow response time, and carry out deep analysis on JSON format data, thereby realizing real-time monitoring on interface response conditions, ensuring reliability and safety of services and realizing stable operation of service continuity.

After the interface is accessed by an external user, the method can count the interfaces with high failure rate and the interfaces with slow response time, and then optimize in time and solve corresponding problems aiming at different conditions, so that the situation that the user loses patience and leaves due to interface failure or too slow response time of the interface can be avoided as much as possible, and the popularization and application of products are facilitated.

After the interface is optimized in a targeted manner, the situation that a user finds the interface failed or the interface response time is too slow can be avoided in time, once an enterprise operation and maintenance personnel finds a problem, the problem can be positioned in advance and solved without waiting for the user to find the problem, so that the interface management cost is reduced, the interface management period is shortened, and the time is saved.

Because enterprise operation and maintenance personnel need carry out concrete optimization and the daily operation and maintenance control of interface according to the loading condition of interface, based on this disclosed technical scheme, when carrying out daily operation and maintenance inspection, operation and maintenance personnel in case find that the failure rate of interface is high or the response time of interface is slow, can timely investigation reason and solve relevant problem, service quality has not only been ensured, and when taking place like the problem, can solve relevant problem fast according to experience before, the degree of difficulty and the time of investigation problem have been reduced for service quality promotes greatly, interface service's reliability, stability have been promoted.

According to the technical scheme provided by the embodiment of the disclosure, the method and the device can be used for counting the interfaces with high failure rate and the interfaces with slow response time by analyzing the related JSON format data, and not only can realize the stable operation of the service continuity, but also can ensure the reliability and the safety of the service by continuously monitoring the response condition of the interfaces. In addition, the actual condition of interface response can be obtained by monitoring the loading condition of the interface, and then the corresponding problems can be optimized in time according to different conditions, so that the condition that the user loses patience and leaves due to interface failure or too slow response time of the interface can be avoided as much as possible, and the user experience is further improved.

Furthermore, aiming at the interfaces with high failure rate and the interfaces with too low response time, the enterprise operation and maintenance personnel can solve the problems in time after finding the interfaces, but not solve the problems after finding the interfaces by the user, so that the time is saved, and the risk of project delay is effectively reduced. Based on the interface response report generated by the method, operation and maintenance personnel can find and solve problems in time and perform corresponding improvement and optimization aiming at different reasons, so that the loading condition of the interface is favorably optimized specifically, the interface is more convenient to perform daily operation and maintenance, the data quality is guaranteed, related problems can be solved quickly according to previous experience, the data quality is improved, and the difficulty and time for troubleshooting the interface problems are reduced.

The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods. For details not disclosed in the embodiments of the apparatus of the present disclosure, refer to the embodiments of the method of the present disclosure.

Fig. 4 is a schematic structural diagram of a monitoring apparatus for interface response conditions according to an embodiment of the present disclosure. As shown in fig. 4, the monitoring device for interface response condition includes:

the acquisition module 401 is configured to acquire interface request data generated by a user accessing an interface within a preset time period from a distributed file system, and synchronize the interface request data into a data warehouse tool;

an integration module 402, configured to perform data cleaning on the interface request data by using a data warehouse tool, and integrate the interface request data after data cleaning according to domain name information of the interface, so as to obtain predetermined table data;

the generating module 403 is configured to query the table data by using a preset data query script to obtain target data in the table data, and generate an interface response report based on the target data and a report configuration created in advance, so as to monitor a response condition of the interface according to the interface response report.

In some embodiments, before the obtaining module 401 in fig. 4 obtains, from the distributed file system, interface request data generated by a user accessing the interface within a preset time period, the interface request data stored in the background database of the application program is sent to the distributed file system, where the interface request data includes log data generated by the interface responding to an access request of the user each time the user accesses the interface, and the log data is data in JSON format.

In some embodiments, the obtaining module 401 of fig. 4 utilizes the data scheduling platform to synchronize the interface request data generated within the preset time period from the distributed file system to the data warehouse tool, and store the interface request data into the original data layer of the data warehouse tool, so as to map the structured interface request data into a database table in the original data layer, where the data warehouse tool employs a Hive data warehouse tool.

In some embodiments, the integration module 402 in fig. 4 performs a data cleansing operation on a database table corresponding to interface request data in the original data layer, and performs desensitization processing on the database table; and summarizing the database tables subjected to data cleaning and desensitization treatment according to the domain name information corresponding to the interface so as to integrate the database tables into table data.

In some embodiments, the generating module 403 in fig. 4 performs a call operation on the data warehouse tool by using the data query script in the report platform based on a preset report platform, so as to obtain target data from table data stored in the data warehouse tool, where the target data is data used for generating an interface response report.

In some embodiments, the generating module 403 in fig. 4 creates a report configuration based on the report style in the report platform, and develops an interface response report using the report configuration and the target data so as to generate an interface response report corresponding to the interface, where the interface response report includes a first report, a second report, and a third report; the first report is used for representing the health condition of the application program, the second report is used for representing the failure rate of the interface, and the third report is used for representing the response time of the interface.

In some embodiments, the monitoring module 404 in fig. 4 monitors the response condition of the interface according to the number of interfaces with failure rates greater than the threshold and the number of interfaces with response times greater than the threshold in the first report; and/or monitoring the response condition of the interface according to the total number of the requests of each type of protocol and the request proportion of each type of protocol in the second report; and/or monitoring the response condition of the interface according to the total response time of each segment in the third report and the proportion of the response time of each segment.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.

Fig. 5 is a schematic structural diagram of an electronic device 5 provided in the embodiment of the present disclosure. As shown in fig. 5, the electronic apparatus 5 of this embodiment includes: a processor 501, a memory 502 and a computer program 503 stored in the memory 502 and operable on the processor 501. The steps in the various method embodiments described above are implemented when the processor 501 executes the computer program 503. Alternatively, the processor 501 implements the functions of the respective modules/units in the above-described respective apparatus embodiments when executing the computer program 503.

Illustratively, the computer program 503 may be partitioned into one or more modules/units, which are stored in the memory 502 and executed by the processor 501 to accomplish the present disclosure. One or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program 503 in the electronic device 5.

The electronic device 5 may be a desktop computer, a notebook, a palm computer, a cloud server, or other electronic devices. The electronic device 5 may include, but is not limited to, a processor 501 and a memory 502. Those skilled in the art will appreciate that fig. 5 is merely an example of the electronic device 5, and does not constitute a limitation of the electronic device 5, and may include more or less components than those shown, or combine certain components, or be different components, e.g., the electronic device may also include input-output devices, network access devices, buses, etc.

The Processor 501 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The storage 502 may be an internal storage unit of the electronic device 5, for example, a hard disk or a memory of the electronic device 5. The memory 502 may also be an external storage device of the electronic device 5, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the electronic device 5. Further, the memory 502 may also include both internal storage units and external storage devices of the electronic device 5. The memory 502 is used for storing computer programs and other programs and data required by the electronic device. The memory 502 may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules, so as to perform all or part of the functions described above. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

In the embodiments provided in the present disclosure, it should be understood that the disclosed apparatus/computer device and method may be implemented in other ways. For example, the above-described apparatus/computer device embodiments are merely illustrative, and for example, a division of modules or units, a division of logical functions only, an additional division may be made in actual implementation, multiple units or components may be combined or integrated with another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, the present disclosure may implement all or part of the flow of the method in the above embodiments, and may also be implemented by a computer program to instruct related hardware, where the computer program may be stored in a computer readable storage medium, and when the computer program is executed by a processor, the computer program may implement the steps of the above methods and embodiments. The computer program may comprise computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain suitable additions or additions that may be required in accordance with legislative and patent practices within the jurisdiction, for example, in some jurisdictions, computer readable media may not include electrical carrier signals or telecommunications signals in accordance with legislative and patent practices.

The above examples are only intended to illustrate the technical solutions of the present disclosure, not to limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present disclosure, and are intended to be included within the scope of the present disclosure.

Claims

1. A method for monitoring interface response condition is characterized by comprising the following steps:

acquiring interface request data generated by a user access interface in a preset time period from a distributed file system, and synchronizing the interface request data to a data warehouse tool;

performing data cleaning on the interface request data by using the data warehouse tool, and integrating the interface request data subjected to the data cleaning according to domain name information of an interface to obtain preset table data;

and querying the table data by using a preset data query script to obtain target data in the table data, and generating an interface response report based on the target data and pre-established report configuration so as to monitor the response condition of the interface according to the interface response report.

2. The method of claim 1, wherein before the obtaining of the interface request data generated by the user accessing the interface within the preset time period from the distributed file system, the method further comprises:

and sending the interface request data stored in a background database of an application program to the distributed file system, wherein the interface request data comprises log data generated by the interface responding to the access request of the user when the user accesses the interface every time, and the log data is data in a JSON format.

3. The method of claim 2, wherein synchronizing the interface request data into a data warehouse tool comprises:

synchronizing the interface request data generated in the preset time period from the distributed file system to the data warehouse tool by using a data scheduling platform, and storing the interface request data to an original data layer of the data warehouse tool so as to map the structured interface request data to a database table in the original data layer, wherein the data warehouse tool adopts a Hive data warehouse tool.

4. The method of claim 3, wherein the performing data washing on the interface request data by using the data warehouse tool and integrating the interface request data after the data washing according to domain name information of the interface to obtain predetermined table data comprises:

performing data cleaning operation on a database table corresponding to the interface request data in the original data layer, and performing desensitization processing on the database table; and summarizing the database tables subjected to the data cleaning and desensitization treatment according to the domain name information corresponding to the interface so as to integrate the database tables into the table data.

5. The method according to claim 1, wherein the querying the tabular data by using a preset data query script to obtain target data in the tabular data comprises:

based on a preset report platform, utilizing a data query script in the report platform to execute a calling operation on the data warehouse tool so as to obtain the target data from the table data stored in the data warehouse tool, wherein the target data is used for generating an interface response report.

6. The method of claim 5, wherein generating an interface response report based on the target data and a pre-created report configuration comprises:

in the report platform, the report configuration is created based on the report style, the interface response report is developed by using the report configuration and the target data so as to generate an interface response report corresponding to the interface, and the interface response report comprises a first report, a second report and a third report;

the first report is used for representing the health condition of an application program, the second report is used for representing the failure rate of the interface, and the third report is used for representing the response time of the interface.

7. The method according to claim 6, wherein the monitoring the response of the interface according to the interface response report includes:

monitoring the response condition of the interfaces according to the number of the interfaces with the failure rate larger than the threshold value and the number of the interfaces with the response time larger than the threshold value in the first report; and/or the presence of a gas in the gas,

monitoring the response condition of the interface according to the total number of the requests of each type of protocol and the request proportion of each type of protocol in the second report; and/or the presence of a gas in the gas,

and monitoring the response condition of the interface according to the total response time of each segment in the third report and the ratio of the response time of each segment.

8. An interface response condition monitoring apparatus, comprising:

the acquisition module is configured to acquire interface request data generated by a user accessing an interface within a preset time period from a distributed file system and synchronize the interface request data into the data warehouse tool;

the integration module is configured to perform data cleaning on the interface request data by using the data warehouse tool and integrate the interface request data after the data cleaning according to domain name information of an interface to obtain preset table data;

and the generating module is configured to query the table data by using a preset data query script to obtain target data in the table data, and generate an interface response report based on the target data and a pre-established report configuration so as to monitor the response condition of the interface according to the interface response report.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.