CN114629768A - Application topology processing method and device, electronic equipment, storage medium and product - Google Patents
Application topology processing method and device, electronic equipment, storage medium and product Download PDFInfo
- Publication number
- CN114629768A CN114629768A CN202210147211.2A CN202210147211A CN114629768A CN 114629768 A CN114629768 A CN 114629768A CN 202210147211 A CN202210147211 A CN 202210147211A CN 114629768 A CN114629768 A CN 114629768A
- Authority
- CN
- China
- Prior art keywords
- application
- data
- topology
- application server
- calling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/02—Standardisation; Integration
- H04L41/024—Standardisation; Integration using relational databases for representation of network management data, e.g. managing via structured query language [SQL]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/02—Standardisation; Integration
- H04L41/0246—Exchanging or transporting network management information using the Internet; Embedding network management web servers in network elements; Web-services-based protocols
- H04L41/0273—Exchanging or transporting network management information using the Internet; Embedding network management web servers in network elements; Web-services-based protocols using web services for network management, e.g. simple object access protocol [SOAP]
- H04L41/028—Exchanging or transporting network management information using the Internet; Embedding network management web servers in network elements; Web-services-based protocols using web services for network management, e.g. simple object access protocol [SOAP] for synchronisation between service call and response
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/12—Discovery or management of network topologies
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Debugging And Monitoring (AREA)
Abstract
The embodiment of the application provides a processing method and device of an application topology, electronic equipment, a storage medium and a product, and relates to the field of computers. The method comprises the following steps: acquiring a call chain data stream sent by at least one application server, and determining topology field data from the call chain data stream; distributing the topology field data to corresponding time windows according to the time stamps of the topology field data; preprocessing the topology field data in each time window to obtain application state data among the application servers; and determining the application topology among the application servers according to the application state data. According to the embodiment of the application, a large amount of call chain data can be processed in time, and the accuracy of data processing is improved.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for processing an application topology, an electronic device, a storage medium, and a product.
Background
The application topology is used for reflecting calling relations among a plurality of services in the application. At present, billions of call chain data need to be called from each application in the processing process of application topology, the data processing process is complex, so that response cannot be timely obtained, and the processing accuracy of the application topology data is influenced.
Disclosure of Invention
The embodiment of the application provides a processing method and device of application topology, electronic equipment, a storage medium and a product, which can process a large amount of call chain data in time and improve the accuracy of data processing.
According to an aspect of an embodiment of the present application, there is provided a method for processing an application topology, including:
acquiring a call chain data stream sent by at least one application server, and determining topology field data from the call chain data stream;
distributing the topology field data to corresponding time windows according to the time stamps of the topology field data;
preprocessing the topology field data in each time window to obtain application state data among the application servers;
and determining the application topology among the application servers according to the application state data.
In a possible implementation manner, the preprocessing the topology field data in each time window to obtain application state data between each application server includes:
determining a delay of the time window;
determining the topology field data of the timestamp in the delay as delay data;
and preprocessing the topology field data in the time window again based on the delay data so as to update the application state data.
In one possible implementation, the topology field data includes at least one of service provider data related to the invoked application server and service consumer data related to the invoking application server;
the preprocessing the topology field data in each time window to obtain the application state data between each application server includes:
dividing the topology field data into a plurality of data groups according to a calling relation formed by the same called application server and a calling application server;
and counting the topology field data in each data group, and determining application state data including application calling relationship data between the application servers based on the counting result of each data group.
In a possible implementation manner, the preprocessing the topology field data in each time window to obtain application state data between the application servers further includes:
determining the operation times of each application server and corresponding calling results and service calling time consumption according to the topology field data in the time window, wherein the operation times comprise calling times and/or called times;
determining the corresponding failure operation times when the calling result is calling failure from the operation times;
comparing the service calling time consumption of each operation with a preset time threshold, and determining the running speed of the application server according to the comparison result;
and determining the application state data of the application server including the application health data according to the failure operation times and the running speed of the application server.
In one possible implementation, the application state data includes application call relationship data and application health data; determining the application topology among the application servers according to the application state data includes:
acquiring an application topology display condition, wherein the application topology display condition is used for indicating a central application server to be displayed, a time period and a topology display level;
querying an application list of a next topology level corresponding to the central application server based on the application call relation data in the time window corresponding to the time period, wherein the application list is used for indicating the application call relation data between the application server of the current topology level and the application server of the next topology level;
sequentially traversing the application list of each topology level according to the topology display levels to determine application calling relationship data of the application server to be displayed;
screening out application health data corresponding to the determined application calling relation data, and calculating the screened application health data to obtain application health degrees among application servers to be displayed;
and constructing an application topology between the application servers to be displayed according to the application calling relation data and the application health degree.
In one possible implementation, the determining topology field data from the call chain data stream includes:
cleaning the call chain data stream;
and extracting topology field data related to the application topology from the cleaned call chain data stream.
According to another aspect of the embodiments of the present application, there is provided a processing apparatus for applying a topology, the apparatus including:
the data acquisition module is used for acquiring a call chain data stream sent by at least one application server and determining topology field data from the call chain data stream;
the data distribution module is used for distributing the topology field data to corresponding time windows according to the time stamps of the topology field data;
the preprocessing module is used for preprocessing the topology field data in each time window to obtain application state data among the application servers;
and the application topology generation module is used for determining the application topology among the application servers according to the application state data.
According to another aspect of the embodiments of the present application, there is provided an electronic device, which includes a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the processing method of the application topology described in the above embodiments.
According to still another aspect of embodiments of the present application, there is provided a computer-readable storage medium, wherein the computer program, when executed by a processor, implements the steps of the processing method of the application topology described in the above embodiments.
According to an aspect of the embodiments of the present application, there is provided a computer program product, which when executed by a processor, implements the steps of the processing method of the application topology described in the above embodiments.
The technical scheme provided by the embodiment of the application has the following beneficial effects:
the processing method of the application topology includes the steps of obtaining a call chain data stream sent by at least one application server, determining topology field data from the call chain data stream, distributing the topology field data to corresponding time windows according to time stamps of the topology field data, preprocessing the topology field data in the time windows to obtain application state data between the application servers, and determining the application topology between the application servers according to the application state data.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.
FIG. 1 is a block diagram of a computer system according to an embodiment of the present disclosure;
fig. 2 is a schematic architecture diagram of a server according to an embodiment of the present application;
fig. 3 is a schematic flowchart of a processing method for an application topology according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a time window preprocessing process provided by an exemplary embodiment of the present application;
fig. 5 is a schematic flowchart of application topology generation provided in an embodiment of the present application;
FIG. 6 is a schematic diagram of an application topology provided by an embodiment of the present application;
fig. 7 is a schematic structural diagram of a processing apparatus applying a topology according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the embodiments set forth below in connection with the drawings are exemplary descriptions for explaining technical solutions of the embodiments of the present application, and do not limit the technical solutions of the embodiments of the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the terms "comprises" and/or "comprising," when used in this specification in connection with embodiments of the present application, specify the presence of stated features, information, data, steps, operations, elements, and/or components, but do not preclude the presence or addition of other features, information, data, steps, operations, elements, components, and/or groups thereof, as embodied in the art. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates at least one of the items defined by the term, e.g., "a and/or B" may be implemented as "a", or as "B", or as "a and B".
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Fig. 1 is a schematic diagram of a computer system architecture provided in an embodiment of the present application, where the computer system 1 includes a terminal 20 and a server 10. The terminal 20 and the server 10 are connected through a communication network, and the terminal 20 and the server 10 may be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.
The terminal 20 may be any terminal device installed with an application program, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, an intelligent wearable device, and the like, which is not limited in this embodiment. Regarding the hardware structure, the terminal 20 includes a display, a memory, a processor and an input device, but is not limited thereto. Illustratively, the application is a terminal-side application of the multimedia platform. By way of example, the multimedia platform includes a social application platform, a game application platform, a life service platform, and the like, and the type of the multimedia platform is not limited in the present application.
The server 10 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a content distribution network, and a big data and artificial intelligence platform.
A multimedia platform is loaded on the server 10, and the server 10 provides background services for the application programs developed and running in the terminal 20. Specifically, fig. 2 is a schematic diagram of a server architecture provided in the embodiment of the present application, where the server 10 includes at least one application server 101 and a management server 102, and the management server 102 is connected to each application server 101 through a communication network. It should be noted that an application is implemented by interaction of multiple services, where a service refers to different functional units in an application, and different services have different service logics, for example, an application includes three services, namely a service a, a service B, and a service C, and when the application is executed, a call link for implementing an account login service calls the service B for the service a, and the service B calls the service C, and thus a call link is formed. The application topology is used for indicating the calling relationship among the services in the application program. Therefore, each application server 101 forms a one-to-one mapping relationship with each service, a code program for running the service logic of each service is configured in each application server 101, and each application server 101 can call each other according to the service function requirement. The management server 102 is configured to run a code program to implement management of the plurality of application servers 101.
Fig. 3 is a schematic flowchart of a processing method for an application topology according to an embodiment of the present application. In the embodiment of the present application, a processing method of an application topology is provided for explaining the management server described in fig. 2 as an example, and the method includes steps S301 to S303.
S301, obtaining a call chain data stream sent by at least one application server, and determining topology field data from the call chain data stream.
It should be noted that, since one application has a plurality of services, the service is triggered to perform service processing of a corresponding service by calling a relevant application server. In the process of one service call, call information (such as call time, service interface, hierarchical relation, call result and the like) among called services is dotted into a log, so that all dotted data are connected into a tree chain to form a call chain. The management server thus obtains from at least one application server a call chain data stream, which is a data sequence of bytes for a call chain having a start point and an end point.
Optionally, monitoring the application server and collecting the corresponding call chain data stream are usually achieved by setting an application probe or sending a probe data packet to the relevant application server. Because a large amount of call chain data streams need to be collected from the application server, the process of processing the call chain data streams is complicated, and the call chain data streams cannot be responded in time. According to the method and the device, the call chain data flow is used as basic data, and the topology field data for applying topology processing is extracted from the call chain data flow, so that the complexity of data processing can be reduced, and the processing flow is simplified.
Wherein the topology field data includes at least one of service provider data associated with the invoked application server, service consumer data associated with the invoked application server, invocation result, service invocation time, and timestamp.
Illustratively, the topology field data is shown in the following cell format: { provider, consumer, flag, elaspTime, logTime }, where provider is service provider data, specifically, related data of a called application server, such as a name and an address of the called application server; the consumer is service consumer data, specifically related data for calling the application server, such as the name and address of the calling application server; flag is a calling result, namely calling is successful or calling is failed; elaspTime is time consuming for service calls; logTime is the timestamp of the data, i.e., the data issue time.
In some embodiments, the determining topology field data from the call chain data stream comprises:
cleaning the call chain data stream;
and extracting topology field data related to the application topology from the cleaned call chain data stream.
In the embodiment, the call chain data streams acquired from the multiple application servers are cleaned, and links and key fields related to the application topology are extracted to form topology field data, so that the complexity of the data can be reduced, the usability of the data is improved, and the efficiency of subsequent data processing is improved.
S302, distributing the topology field data to corresponding time windows according to the time stamps of the topology field data.
S303, preprocessing the topology field data in each time window to obtain application state data among the application servers.
In the application, the topology field data is divided into a plurality of time windows according to the timestamp carried by the topology field data. Each time window is provided with a processing time interval, the lower limit of the processing time interval is data occurrence time, the upper limit of the processing time interval is data processing time, and therefore when topology field data with the time stamp exceeding the data occurrence time appears, the corresponding time window is created, and then the topology field data with the time stamp in the processing time interval are distributed to the time window. Further, when the time stamp reaches the data processing time, the calculation of the topology field data in the current time window is triggered. Therefore, according to the method and the device, a large amount of topology field data are divided into a plurality of time windows, and the data in each time window are preprocessed, so that the complexity of subsequent data processing can be reduced, and the processing efficiency is improved.
Optionally, the application state data includes application call relation data. Specifically, after the data processing time of the time window is reached, the topology field data in the current time window is obtained, and the topology field data in the time window is processed according to the preset calling relation processing rule in the time window. The calling relation processing rule may classify data according to a calling relation formed by the same called application server and the calling application server, or classify data according to the same calling result, which is not limited in this application.
Optionally, the application state data comprises application health data. Specifically, after the data processing time of the time window is reached, the topology field data in the current time window is obtained, and the topology field data in the time window is processed according to the application health degree pre-statistical rule preset in the time window. The application health degree pre-statistical rule may be used to analyze the service invocation execution condition according to the invoked service and/or the invocation result of the invoked service and the service invocation time, which is not limited in the present application. Therefore, by preprocessing the application health data, the execution condition of the application service under the related calling relationship is rapidly displayed while the application calling relationship data is displayed.
S304, determining the application topology among the application servers according to the application state data.
In the application, the execution condition of the application service under the corresponding calling relation is displayed while the application topology is displayed by preprocessing the application calling relation data and the application health data.
According to the processing method of the application topology, the call chain data stream sent by at least one application server is obtained, the topology field data are determined from the call chain data stream, the topology field data are distributed to the corresponding time windows according to the time stamps of the topology field data, then the topology field data in each time window are preprocessed, the application state data between the application servers are obtained, the application topology between the application servers is determined according to the application state data, and therefore a large amount of topology field data are divided into a plurality of time windows, the data in each time window are preprocessed, timely response facing a large amount of data is achieved, and accuracy of data processing is improved.
In some embodiments, the preprocessing the topology field data in each time window to obtain application state data between each application server includes:
determining a delay of the time window;
determining the topology field data of the timestamp in the delay as delay data;
and preprocessing the topology field data in the time window again based on the delay data so as to update the application state data.
In this embodiment, when the data processing time of the time window is reached, the calculation of the topology field data in the current time window is triggered, and the current calculation result is saved. And dividing the topology field data of the timestamp in the time delay into the time window by setting the time delay of the time window, taking the topology field data of the timestamp in the time delay as time delay data, and updating the topology field data in the current time window based on the time delay data, so that the calculation result is updated based on the updated topology field data of the time window, thereby updating the application state data. In addition, topology field data with a timestamp exceeding the delay is discarded.
Illustratively, the processing time of the set time window is 30 minutes, the delay time is 3 minutes, and the processing time interval of the previous time window is 11: 30-12: 00, time stamping at 11: 30-12: the topology field data of 00 is written into the time window. At 12: and 00, triggering to calculate the topology field data in the time window, and storing the calculation result in a cache. If at 12: 00-12: if topology field information (in this case, delay data) is also received in 03, the saved calculation result is recalculated based on the delay data to update the calculation result.
Therefore, in the embodiment, by setting the delay of the time window, the application state data can be updated based on the delay data, the processing capability of the delay data is provided, and the accuracy of data statistics is improved.
In some embodiments, the preprocessing the topology field data in each time window to obtain application state data between each application server includes:
dividing the topology field data into a plurality of data groups according to a calling relation formed by the same called application server and a calling application server;
and counting the topology field data in each data group, and determining application state data including application calling relation data between the application servers based on the counting result of each data group.
In this embodiment, the topology field data in the time window are data-classified according to the same call relation. Specifically, the topology field data having the calling relationship formed by the same called application server and the calling application server are organized into a data group, so that the topology field data in the time window is divided into a plurality of data groups, and then the topology field data in each data group is counted to obtain the preliminarily processed application calling relationship data between the application servers. The content of the data set includes, but is not limited to, the service provider data, the service consumer data, and the number of occurrences of the call relation. Further, the application calling relation data obtained by preprocessing is stored in the elastic search cluster, so that corresponding data can be called from the cluster in the subsequent calculation process. Therefore, in the embodiment, the application call relation data with the same specific characteristic is obtained by grouping the topology field data in the time window, so that the efficiency and the accuracy of the preprocessing can be improved.
For example, referring to fig. 4, a schematic diagram of a time window preprocessing process provided in an exemplary embodiment of the present application is illustrated, where the topology field data has a data unit format of { provide, consumer, flag, elastime }, and if there are 10 topology field data in a current time window, the topology field data specifically includes (a, B, True,1), (a, B, True,4), (a, B, False,7), (a, C, True,10), (a, C, True,15), (a, C, True,8), (B, D, True,10), (B, D, False,15), (C, D, True,10), (C, D, False, 10). Wherein, A is the name of the application server A, B is the name of the application server B, C is the name of the application server C, D is the name of the application server D, True indicates that the call is successful, and False indicates that the call is failed. Then, topology field data related to the call relation of the called application server as application server a and the calling server as application server B, such as (a, B, True,1), (a, B, True,4), (a, B, False,7), is obtained to organize into a data group having the call relation. Then, based on the topology field data in the data group, application call relation window calculation is performed, and it is determined that the number of times of occurrence of the above-described call relation composed of the called application server as the application server a and the call server as the application server B is 3 times, so that the application call relation data between the application server a and the application server B is represented as (a, B, 3). By analogy, the topology field data of the call relation constituted by the called application server a as the application server a and the call server as the application server C is organized into application call relation data (a, C,3), the topology field data of the call relation constituted by the called application server B as the application server B and the call server as the application server D as the application call relation data (B, D,2), the topology field data of the call relation constituted by the called application server C as the application server C and the call server as the application server D as the application call relation data (C, D,2), so that the topology field data within the current time window is computed to obtain the application call relation data between the four application servers, specifically represented as (a, B,3), (a, C,3) (B, D,2), (C, D, 2).
In some embodiments, the preprocessing the topology field data in each time window to obtain application state data between each application server further includes:
determining the operation times of each application server and corresponding calling results and service calling time consumption according to the topology field data in the time window, wherein the operation times comprise calling times and/or called times;
determining the corresponding failure operation times when the calling result is calling failure from the operation times;
comparing the service calling time consumption of each operation with a preset time threshold, and determining the running speed of the application server according to the comparison result;
and determining the application state data of the application server including the application health data according to the failure operation times and the running speed of the application server.
In this embodiment, the application health data is used to reflect the execution of the application service over a period of time. Specifically, the topology field data in the time window is obtained to determine the operation times of each application server and the operation condition of each operation, where the operation times include calling times and/or called times, the operation condition includes calling conditions and/or called conditions, the calling conditions include calling results and service calling time consumption generated by the application server in the calling process, and the called conditions include calling results and service calling time consumption generated by the application server in the called process. Then, the number of failed operations of which the calling result is a call failure is determined based on the number of operations of the application server. Meanwhile, comparing the service calling consumed time of each operation with a plurality of preset time thresholds respectively to obtain the times of the service calling consumed time falling in each preset time threshold, and determining the running speed of the application server. Therefore, the application health data of the application server is determined according to the operation times, the failure operation times, the running speed and the like of the application server. Further, the application health data obtained by preprocessing is stored in the elastic search cluster, so that corresponding data can be called from the cluster in the subsequent calculation process. Therefore, the execution condition of the application service under the relevant calling relation is rapidly displayed while the application calling relation data is displayed by preprocessing the application health data at the same time.
Illustratively, referring to fig. 4, if there are 10 pieces of topology field data in the current time window, specifically including (a, B, True,1), (a, B, True,4), (a, B, False,7), (a, C, True,10), (a, C, True,15), (a, C, True,8), (B, D, True,10), (B, D, False,15), (C, D, True,10), (C, D, False, 10). Therefore, for the application server B, the number of calls is 3, wherein the number of failed operations is 1, the number of calls whose service call time is less than 1 second is 1, the number of calls whose service call time is greater than or equal to 3 seconds and less than 5 seconds is 1, and the number of calls whose service call time is greater than 5 seconds is 1. Meanwhile, the data unit format of the application health data is set as { X, pos1, pos2, pos3, pos4, pos5}, wherein X is the name of the calling application server; pos1 indicates a number of calls that take less than 1 second for service calls; pos2 represents the number of calls that take a service call of 1 second or more and less than 3 seconds; pos3 represents the number of calls that take 3 seconds or more and less than 5 seconds for service calls; pos4 represents the number of calls that take more than 5 seconds for a service call; pos5 indicates the corresponding number of failed operations when the call result is a call failure. Thus, an application health data window calculation is performed, with application health data for application server B represented as (B,1,0,1,1, 1). Similarly, for the application server C, the number of calls is 3, where the number of failed operations is 0, and the number of calls that take more than 5 seconds for service call is 3, which is denoted as (C,0,0,0,3, 0). For the application server D, the number of calls is 4, where the number of failed operations is 2, and the number of calls that take more than 5 seconds for service call is 4, which is denoted as (D,0,0,0,4, 2).
In some embodiments, the application state data includes application invocation relationship data and application health data; determining the application topology among the application servers according to the application state data includes:
acquiring an application topology display condition, wherein the application topology display condition is used for indicating a central application server to be displayed, a time period and a topology display level;
querying an application list of a next topology level corresponding to the central application server based on the application call relation data in the time window corresponding to the time period, wherein the application list is used for indicating the application call relation data between the application server of the current topology level and the application server of the next topology level;
sequentially traversing the application list of each topology level according to the topology display levels to determine application calling relationship data of the application server to be displayed;
screening out application health data corresponding to the determined application calling relation data, and calculating the screened application health data to obtain application health degrees among application servers to be displayed;
and constructing an application topology between the application servers to be displayed according to the application calling relation data and the application health degree.
In this embodiment, referring to fig. 5, which is a schematic flowchart of generating an application topology according to an embodiment of the present application, after the application topology is started, an application topology display condition is obtained, where the application topology display condition includes central application server data to be displayed, a time period, and a topology display hierarchy to be displayed, and the central application server data is a name, an address, and the like of a central application server. And then, determining a time window corresponding to the time period, and acquiring topology field data in the time window and application calling relation data and application health data obtained by preprocessing. Therefore, according to the application calling relation data in the time window, the application list of the next topological level is inquired according to the name of the central application server and is used as the application list of the first topological level, and the application list is used for indicating the application calling relation data between the central application server and the application server of the next topological level. And traversing the application list of the first topology level, and acquiring the application server data of the first topology level. And then, judging whether the currently traversed topology level is the topology display level, namely judging whether an application list of the next topology level needs to be inquired, if so, repeating the steps, inquiring the application list of the next topology level according to the name of the application server of the first topology level to serve as the application list of the second topology level, wherein the application list is used for indicating application calling relation data between the application server of the first topology level and the application server of the next topology level. And repeating the steps until the currently traversed topology level is the topology display level, and summarizing the application list obtained from the first topology level to the topology display level so as to determine the application servers to be displayed and the application calling relationship data of each application server.
Further, the application health data related to the application call relation data of the application server to be displayed is screened out from the application health data in the time window, so that the application health degree marking is carried out according to the screened application health data. Specifically, an application monitoring degree rule is set, and the application health degree of each application server is determined according to the application monitoring degree rule.
Illustratively, the application of the monitoring degree rule includes a normal service (i.e., the service call takes less than 5 seconds) duty ratio, a service call exception duty ratio, and a slow service call (i.e., the service call takes more than 5 seconds) duty ratio. Taking the central application server as the application server a for explanation, for the application server B calling the central application server a, according to the application health data (B,1,0,1,1,1) of the application server B in the above exemplary embodiment, the normal service occupancy ratio is calculated to be 66%, the service calling exception occupancy ratio is calculated to be 33%, and the service calling slow occupancy ratio is calculated to be 33%, that is, the application health degree of the application server B is expressed as { normal: 66%, operation slow: 33%, exception: 33% }. For the application server C calling the central application server a, according to the application health data (C,0,0,0,3,0) of the application server C in the above exemplary embodiment, it is calculated that the normal service occupancy is 0, the service call exception occupancy is 0, and the service call slow occupancy is 100%, that is, the application health of the application server C is expressed as { normal: 0%, slow operation: 100%, exception: 0% }. And calling the application server B and the application server C respectively for the application server D, calculating that the normal service duty ratio is 0, the service call exception duty ratio is 50%, and the service call slow duty ratio is 100% according to the application health data (D,0,0,0,4,2) of the application server D in the above exemplary embodiment, that is, the application health of the application server D is expressed as { normal: 0%, slow operation: 100%, exception: 50% }.
Further, an application topological graph between the application servers to be displayed is formed according to the application calling relationship data and the application health degree between the application servers to be displayed, and the application topological graph is used for displaying the calling relationship between the application servers and the execution condition of the application service. For example, referring to fig. 6, which is a schematic diagram of an application topology diagram provided in an embodiment of the present application, on the basis of the above exemplary embodiment, a central application server is taken as an application server a for explanation, a topology display hierarchy is set as two levels, an application list of a next topology hierarchy corresponding to the application server a, that is, an application list of a first topology hierarchy, is queried from data of a specified time window, where the application list shows that an application server B calls the application server a and the occurrence number of the call relationship is 3, and an application server C calls the application server a and the occurrence number of the call relationship is 3. Meanwhile, the application health degree of the application server B { normal: 66%, slow running: 33%, abnormal: 33% } and the application health degree of the application server C { normal: 0%, slow running: 100%, abnormal: 0% } are displayed. Further, an application list of a next topology level corresponding to the application server B and the application server C, that is, an application list of a second topology level is queried, where the application list shows that the application server D calls the application server B and the occurrence number of the call relationship is 2, and the application server D calls the application server C and the occurrence number of the call relationship is 2. Meanwhile, aiming at the calling condition of the application server D, the health degree of the application is displayed as { normal: 0%, slow operation: 100%, and abnormal: 50% }.
Fig. 7 is a schematic structural diagram of a processing apparatus applying a topology according to an embodiment of the present application, where the processing apparatus 400 applying a topology includes:
a data obtaining module 401, configured to obtain a call chain data stream sent by at least one application server, and determine topology field data from the call chain data stream;
a data allocating module 402, configured to allocate the topology field data to a corresponding time window according to a timestamp of the topology field data;
a preprocessing module 403, configured to preprocess the topology field data in each time window to obtain application state data between each application server;
an application topology generating module 404, configured to determine an application topology between the application servers according to the application state data.
In some embodiments, the pre-processing module 403 includes:
a delay determining unit, configured to determine a delay of the time window;
the time delay data acquisition unit is used for determining that the topology field data of the time stamp in the time delay is time delay data;
and the delay data processing unit is used for preprocessing the topology field data in the time window again based on the delay data so as to update the application state data.
In some embodiments, the topology field data includes at least one of service provider data related to the invoked application server and service consumer data related to the invoking application server; the preprocessing module 403 includes:
the data group dividing unit is used for dividing the topology field data into a plurality of data groups according to a calling relationship formed by the same called application server and the calling application server;
and the grouping counting unit is used for counting the topology field data in each data group and determining the application state data including the application calling relation data between the application servers based on the counting result of each data group.
In some embodiments, the pre-processing module 403 includes:
an operation condition obtaining unit, configured to determine, according to the topology field data in the time window, operation times of each application server and corresponding call results and service call time consumption, where the operation times include call times and/or called times;
a failure operation data obtaining unit, configured to determine, from the operation times, a corresponding failure operation time when the call result is a call failure;
the running speed determining unit is used for comparing the service calling consumed time of each operation with a preset time threshold value and determining the running speed of the application server according to a comparison result;
and the application health data determining unit is used for determining the application state data of the application server including the application health data according to the failure operation times and the running speed of the application server.
In some embodiments, the application state data includes application invocation relationship data and application health data; the application topology generation module 404 includes:
the device comprises an application topology display condition acquisition unit, a topology display unit and a topology display unit, wherein the application topology display condition acquisition unit is used for acquiring an application topology display condition which is used for indicating a central application server to be displayed, a time period and a topology display level;
an application list query unit, configured to query, based on application call relationship data in a time window corresponding to the time period, an application list of a next topology level corresponding to the central application server, where the application list is used to indicate application call relationship data between an application server of a current topology level and an application server of the next topology level;
the traversal unit is used for sequentially traversing the application list of each topology level according to the topology display levels so as to determine application calling relation data of the application server to be displayed;
the application health degree calculation unit is used for screening out application health data corresponding to the determined application calling relation data and calculating the screened application health data to obtain the application health degree between the application servers to be displayed;
and the application topology construction unit is used for constructing the application topology between the application servers to be displayed according to the application calling relation data and the application health degree.
In some embodiments, the data acquisition module 401 includes:
the data cleaning unit is used for cleaning the call chain data stream;
and the data extraction unit is used for extracting topology field data related to the application topology from the cleaned call chain data stream.
The apparatus of the embodiment of the present application may execute the method provided by the embodiment of the present application, and the implementation principle is similar, the actions executed by the modules in the apparatus of the embodiments of the present application correspond to the steps in the method of the embodiments of the present application, and for the detailed functional description of the modules of the apparatus, reference may be specifically made to the description in the corresponding method shown in the foregoing, and details are not repeated here.
The embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory, where the processor executes the computer program to implement the steps of the processing method of the application topology, and compared with the related art, the electronic device can implement: a large amount of topology field data are divided into a plurality of time windows, so that the data in each time window are preprocessed, timely response facing a large amount of data is realized, and the accuracy of data processing is improved.
In an alternative embodiment, an electronic device is provided, as shown in fig. 8, the electronic device 500 shown in fig. 8 comprising: a processor 501 and a memory 503. Wherein the processor 501 is coupled to the memory 503, such as via the bus 502. Optionally, the electronic device 500 may further include a transceiver 504, and the transceiver 504 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data. It should be noted that the transceiver 504 is not limited to one in practical applications, and the structure of the electronic device 500 is not limited to the embodiment of the present application.
The Processor 501 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other Programmable logic device, transistor logic device, hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 501 may also be a combination of implementing computing functionality, e.g., comprising one or more microprocessors, a combination of DSPs and microprocessors, and the like.
The Memory 503 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic Disc storage medium, other magnetic storage devices, or any other medium that can be used to carry or store a computer program and that can be Read by a computer, without limitation.
The memory 503 is used for storing computer programs for executing the embodiments of the present application, and is controlled by the processor 501 for execution. The processor 501 is adapted to execute a computer program stored in the memory 503 to implement the steps shown in the aforementioned method embodiments.
Embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, and when being executed by a processor, the computer program may implement the steps and corresponding contents of the foregoing method embodiments.
Embodiments of the present application further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.
The terms "first," "second," "third," "fourth," "1," "2," and the like in the description and in the claims of the present application and in the above-described drawings (if any) are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in other sequences than illustrated or otherwise described herein.
It should be understood that, although each operation step is indicated by an arrow in the flowchart of the embodiment of the present application, the implementation order of the steps is not limited to the order indicated by the arrow. In some implementation scenarios of the embodiments of the present application, the implementation steps in the flowcharts may be performed in other sequences as desired, unless explicitly stated otherwise herein. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on an actual implementation scenario. Some or all of these sub-steps or stages may be performed at the same time, or each of these sub-steps or stages may be performed at different times, respectively. Under the scenario that the execution time is different, the execution sequence of the sub-steps or phases may be flexibly configured according to the requirement, which is not limited in the embodiment of the present application.
The foregoing is only an optional implementation manner of a part of implementation scenarios in this application, and it should be noted that, for those skilled in the art, other similar implementation means based on the technical idea of this application are also within the protection scope of the embodiments of this application without departing from the technical idea of this application.
Claims (10)
1. A processing method of application topology is characterized by comprising the following steps:
acquiring a call chain data stream sent by at least one application server, and determining topology field data from the call chain data stream;
distributing the topology field data to corresponding time windows according to the time stamps of the topology field data;
preprocessing the topology field data in each time window to obtain application state data among the application servers;
and determining the application topology among the application servers according to the application state data.
2. The method for processing application topologies according to claim 1, wherein the preprocessing the topology field data in each of the time windows to obtain the application state data between each of the application servers includes:
determining a delay of the time window;
determining the topology field data of the timestamp in the delay as delay data;
and preprocessing the topology field data in the time window again based on the delay data so as to update the application state data.
3. The method of processing an application topology of claim 1, wherein the topology field data comprises at least one of service provider data related to a invoked application server and service consumer data related to an invoking application server;
the preprocessing the topology field data in each time window to obtain the application state data between each application server includes:
dividing the topology field data into a plurality of data groups according to a calling relation formed by the same called application server and a calling application server;
and counting the topology field data in each data group, and determining application state data including application calling relation data between the application servers based on the counting result of each data group.
4. The method for processing application topology according to claim 1, wherein the preprocessing the topology field data in each time window to obtain the application state data between each application server further comprises:
determining the operation times of each application server and corresponding calling results and service calling time consumption according to the topology field data in the time window, wherein the operation times comprise calling times and/or called times;
determining the corresponding failure operation times when the calling result is calling failure from the operation times;
comparing the service calling time consumption of each operation with a preset time threshold, and determining the running speed of the application server according to the comparison result;
and determining the application state data of the application server including the application health data according to the failure operation times and the running speed of the application server.
5. The processing method of application topology according to claim 1, wherein the application state data comprises application invocation relationship data and application health data; determining the application topology among the application servers according to the application state data includes:
acquiring an application topology display condition, wherein the application topology display condition is used for indicating a central application server to be displayed, a time period and a topology display level;
querying an application list of a next topology level corresponding to the central application server based on the application call relation data in the time window corresponding to the time period, wherein the application list is used for indicating the application call relation data between the application server of the current topology level and the application server of the next topology level;
sequentially traversing the application list of each topology level according to the topology display levels to determine application calling relationship data of the application server to be displayed;
screening out application health data corresponding to the determined application calling relation data, and calculating the screened application health data to obtain application health degrees among application servers to be displayed;
and constructing an application topology between the application servers to be displayed according to the application calling relation data and the application health degree.
6. The method for processing application topology according to claim 1, wherein the determining topology field data from the call chain data stream comprises:
cleaning the call chain data stream;
and extracting topology field data related to the application topology from the cleaned call chain data stream.
7. A processing apparatus for application topology, comprising:
the data acquisition module is used for acquiring a call chain data stream sent by at least one application server and determining topology field data from the call chain data stream;
the data distribution module is used for distributing the topology field data to corresponding time windows according to the time stamps of the topology field data;
the preprocessing module is used for preprocessing the topology field data in each time window to obtain application state data among the application servers;
and the application topology generation module is used for determining the application topology among the application servers according to the application state data.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the steps of the processing method of an application topology according to any one of claims 1 to 6.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method for processing an application topology according to any one of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, realizes the steps of the processing method of an application topology of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210147211.2A CN114629768A (en) | 2022-02-17 | 2022-02-17 | Application topology processing method and device, electronic equipment, storage medium and product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210147211.2A CN114629768A (en) | 2022-02-17 | 2022-02-17 | Application topology processing method and device, electronic equipment, storage medium and product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114629768A true CN114629768A (en) | 2022-06-14 |
Family
ID=81900711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210147211.2A Pending CN114629768A (en) | 2022-02-17 | 2022-02-17 | Application topology processing method and device, electronic equipment, storage medium and product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114629768A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106034047A (en) * | 2015-04-16 | 2016-10-19 | 腾讯科技（深圳）有限公司 | Data processing method and device |
| WO2017076188A1 (en) * | 2015-11-03 | 2017-05-11 | 阿里巴巴集团控股有限公司 | Method and device for processing service calling information |
| CN112783720A (en) * | 2021-01-05 | 2021-05-11 | 广州品唯软件有限公司 | Topological structure diagram generation method and device, computer equipment and display system |
| CN113051446A (en) * | 2019-12-26 | 2021-06-29 | 百度国际科技（深圳）有限公司 | Topological relation query method, device, electronic equipment and medium |
| CN113094157A (en) * | 2021-02-25 | 2021-07-09 | 青岛海尔科技有限公司 | Generation method and device for calling topological graph, storage medium and electronic device |
| CN113114533A (en) * | 2021-04-08 | 2021-07-13 | 中国工商银行股份有限公司 | Network time consumption display method and device for distributed service invocation |
| CN113254341A (en) * | 2021-05-31 | 2021-08-13 | 康键信息技术（深圳）有限公司 | Link data tracking method, device, equipment and storage medium |
| US20210328877A1 (en) * | 2020-04-16 | 2021-10-21 | Vmware, Inc. | Application topology visualization |
| CN113987074A (en) * | 2021-10-27 | 2022-01-28 | 中国工商银行股份有限公司 | Distributed service full-link monitoring method and device, electronic equipment and storage medium |
-
2022
- 2022-02-17 CN CN202210147211.2A patent/CN114629768A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106034047A (en) * | 2015-04-16 | 2016-10-19 | 腾讯科技（深圳）有限公司 | Data processing method and device |
| WO2017076188A1 (en) * | 2015-11-03 | 2017-05-11 | 阿里巴巴集团控股有限公司 | Method and device for processing service calling information |
| CN113051446A (en) * | 2019-12-26 | 2021-06-29 | 百度国际科技（深圳）有限公司 | Topological relation query method, device, electronic equipment and medium |
| US20210328877A1 (en) * | 2020-04-16 | 2021-10-21 | Vmware, Inc. | Application topology visualization |
| CN112783720A (en) * | 2021-01-05 | 2021-05-11 | 广州品唯软件有限公司 | Topological structure diagram generation method and device, computer equipment and display system |
| CN113094157A (en) * | 2021-02-25 | 2021-07-09 | 青岛海尔科技有限公司 | Generation method and device for calling topological graph, storage medium and electronic device |
| CN113114533A (en) * | 2021-04-08 | 2021-07-13 | 中国工商银行股份有限公司 | Network time consumption display method and device for distributed service invocation |
| CN113254341A (en) * | 2021-05-31 | 2021-08-13 | 康键信息技术（深圳）有限公司 | Link data tracking method, device, equipment and storage medium |
| CN113987074A (en) * | 2021-10-27 | 2022-01-28 | 中国工商银行股份有限公司 | Distributed service full-link monitoring method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10394693B2 (en) | Quantization of data streams of instrumented software | |
| CN109587008B (en) | Method, device and storage medium for detecting abnormal flow data | |
| CN103220173A (en) | Alarm monitoring method and alarm monitoring system | |
| CN109684052A (en) | Transaction analysis method, apparatus, equipment and storage medium | |
| CN111552570B (en) | Self-adaptive distribution method of data processing resources of Internet of things and cloud computing server | |
| CN109992473A (en) | Monitoring method, device, equipment and the storage medium of application system | |
| CN111181799A (en) | Network traffic monitoring method and equipment | |
| CN114091704B (en) | Alarm suppression method and device | |
| CN111240876A (en) | Fault positioning method and device for microservice, storage medium and terminal | |
| CN110807050B (en) | Performance analysis method, device, computer equipment and storage medium | |
| CN111258798A (en) | Fault positioning method and device for monitoring data, computer equipment and storage medium | |
| Solmaz et al. | ALACA: A platform for dynamic alarm collection and alert notification in network management systems | |
| CN116418653A (en) | Fault positioning method and device based on multi-index root cause positioning algorithm | |
| CN111800292A (en) | Early warning method and device based on historical flow, computer equipment and storage medium | |
| CN114443437A (en) | Alarm root cause output method, apparatus, device, medium, and program product | |
| CN112948223A (en) | Method and device for monitoring operation condition | |
| CN117931589A (en) | Operation and maintenance fault identification method and device | |
| CN113656369A (en) | Log distributed streaming acquisition and calculation method in big data scene | |
| CN108595685A (en) | A kind of data processing method and device | |
| CN117093627A (en) | Information mining method, device, electronic equipment and storage medium | |
| CN116662127A (en) | Method, system, equipment and medium for classifying and early warning equipment alarm information | |
| CN113835961B (en) | Alarm information monitoring method, device, server and storage medium | |
| CN114629768A (en) | Application topology processing method and device, electronic equipment, storage medium and product | |
| CN115269519A (en) | Log detection method and device and electronic equipment | |
| CN112994978B (en) | Network traffic monitoring method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |