CN113987074A - Distributed service full-link monitoring method and device, electronic equipment and storage medium - Google Patents
Distributed service full-link monitoring method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113987074A CN113987074A CN202111259247.1A CN202111259247A CN113987074A CN 113987074 A CN113987074 A CN 113987074A CN 202111259247 A CN202111259247 A CN 202111259247A CN 113987074 A CN113987074 A CN 113987074A
- Authority
- CN
- China
- Prior art keywords
- service
- link data
- call
- calling
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The present disclosure provides a distributed service full link monitoring method, apparatus, electronic device, storage medium, and computer program product, which may be applied to the financial field, the computer field, or other fields. The distributed service full link monitoring method comprises the following steps: acquiring a plurality of link data related to a service according to a call request of a user, wherein each link data comprises a tracking tag, and the tracking tag is used for marking the link data; routing the link data with the same tracking label to the same aggregation service node, and generating a topological structure according to the routing; and storing the topological structure and the link data in a graph database according to a preset rule to form a service call topological graph.
Description
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a method and an apparatus for monitoring a distributed service full link, an electronic device, and a storage medium.
Background
With the increasing popularization of the cloud native technology and the maturity of the distributed architecture, the application architecture gradually changes to the service, the calling level between the service system and the service is deeper and deeper, and the deployment situation is more and more complex. However, currently, fault root cause positioning for application is mainly performed in a manual analysis mode, and after a fault occurs, a user often needs to perform troubleshooting analysis through massive discrete logs, so that efficiency is low. Therefore, how to realize efficient and accurate fault root cause positioning is an urgent technical problem to be solved.
Disclosure of Invention
In view of the above, the present disclosure provides a distributed service full link monitoring method, apparatus, electronic device, storage medium, and computer program product.
According to one aspect of the present disclosure, a distributed service full link monitoring method is provided, including:
acquiring a plurality of link data related to a service according to a call request of a user, wherein each link data comprises a tracking tag, and the tracking tag is used for marking the link data;
routing the link data with the same tracking label to the same aggregation service node, and generating a topological structure according to the routing;
and storing the topological structure and the link data in a graph database according to a preset rule to form a service call topological graph.
According to an embodiment of the present disclosure, wherein routing link data having the same trace label onto the same aggregation service node and generating a topology according to the link data comprises:
routing link data with the same tracking label to the same aggregation service node;
and interpreting the link data with the same tracking label into a character string set of a call service name of the service name, and generating a topological structure according to the character string set.
According to an embodiment of the present disclosure, wherein generating the topology from the set of strings comprises:
generating a unique identification string by adopting an MD5 algorithm on the character string set;
and generating a topological structure by taking the service name as a point, the calling relation as an edge and the MD5 value corresponding to the tracking label and the calling chain as the attributes of the edge.
According to the embodiment of the present disclosure, the preset rule includes a service name call service name.
According to the embodiment of the disclosure, collecting a plurality of link data related to a service according to a call request of a user includes:
aiming at a call request of a user, identifying a service call chain entrance and generating a tracking label so that the tracking label is transmitted in a full link along with a call chain when the call request passes through each node in a call process;
collecting a plurality of link data related to the service in a buried point mode;
and sending a plurality of link data related to the service to the distributed log collection system based on an asynchronous mode.
According to an embodiment of the present disclosure, the method further includes:
generating tracking data based on the tracking label when the calling request passes through each node in the calling process;
the trace data includes at least one of: the starting time of calling, the protocol type, the calling party address, the calling party port, the service name of the request, the calling time, the calling result and the exception information.
According to an embodiment of the present disclosure, the method further includes: based on the tracking tag, topological data associated with the tracking tag is read from the graph database and presented graphically.
According to another aspect of the present disclosure, there is provided a distributed service full link monitoring apparatus, including:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a plurality of link data related to a service aiming at a call request of a user, each link data comprises a tracking label, and the tracking label is used for marking the link data;
the generating module is used for routing the link data with the same tracking label to the same aggregation service node and generating a topological structure according to the link data;
and the storage module is used for storing the topological structure and the plurality of link data into the graph database according to a preset rule to form a service call topological graph.
According to an embodiment of the present disclosure, wherein the generating module includes:
the first generation module is used for routing the link data with the same tracking label to the same aggregation service node;
and the second generation module is used for interpreting the link data with the same tracking label as a character string set of the service name call service name and generating a topological structure according to the character string set.
According to an embodiment of the present disclosure, the second generating module is further configured to:
generating a unique identification string by adopting an MD5 algorithm on the character string set;
and generating a topological structure by taking the service name as a point, the calling relation as an edge and the MD5 value corresponding to the tracking label and the calling chain as the attributes of the edge.
According to the embodiment of the present disclosure, the preset rule includes a service name call service name.
According to an embodiment of the present disclosure, wherein the acquisition module includes:
the first acquisition module is used for identifying an entrance of a service call chain aiming at a call request of a user and generating a tracking label so that the tracking label is transmitted in a full link along with the call chain when the call request passes through each node in a call process;
the second acquisition module is used for acquiring a plurality of link data related to the service in a point-buried mode;
and the third acquisition module is used for sending the link data related to the service to the distributed log collection system based on an asynchronous mode.
According to an embodiment of the present disclosure, wherein the second acquisition module is further configured to:
generating tracking data based on the tracking label when the calling request passes through each node in the calling process;
the trace data includes at least one of: the starting time of calling, the protocol type, the calling party address, the calling party port, the service name of the request, the calling time, the calling result and the exception information.
According to an embodiment of the present disclosure, the apparatus further includes: and the display module is used for reading the topological data related to the tracking label from the graph database based on the tracking label and displaying the topological data in a graphical form.
According to another aspect of the present disclosure, there is provided an electronic device including: one or more processors; memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform operations that implement the methods described above.
According to another aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform a method implementing the above.
According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method as described above.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:
fig. 1 schematically illustrates an application scenario of a distributed service full link monitoring method, apparatus, electronic device, storage medium and computer program product according to an embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow diagram of a distributed service full link monitoring method according to an embodiment of the disclosure;
FIG. 3 schematically illustrates a flow diagram of a distributed service full link monitoring method according to another embodiment of the present disclosure;
FIG. 4 schematically illustrates a flow diagram of a distributed service full link monitoring method according to another embodiment of the present disclosure;
FIG. 5 schematically illustrates a flow diagram of a distributed service full link monitoring method according to another embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of a distributed service full link monitoring apparatus according to an embodiment of the present disclosure;
FIG. 7 schematically illustrates a block diagram of a distributed service full link monitoring apparatus according to another embodiment of the present disclosure;
FIG. 8 schematically illustrates a block diagram of a distributed service full link monitoring apparatus according to another embodiment of the present disclosure;
FIG. 9 schematically illustrates a block diagram of a distributed service full link monitoring apparatus according to another embodiment of the present disclosure; and
fig. 10 schematically illustrates a block diagram of an electronic device suitable for implementing a distributed service full link monitoring method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
Some block diagrams and/or flow diagrams are shown in the figures. It will be understood that some blocks of the block diagrams and/or flowchart illustrations, or combinations thereof, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the instructions, which execute via the processor, create means for implementing the functions/acts specified in the block diagrams and/or flowchart block or blocks. The techniques of this disclosure may be implemented in hardware and/or software (including firmware, microcode, etc.). In addition, the techniques of this disclosure may take the form of a computer program product on a computer-readable storage medium having instructions stored thereon for use by or in connection with an instruction execution system.
Embodiments of the present disclosure provide a distributed service full link monitoring method, apparatus, electronic device, storage medium, and computer program product, which may be used in the financial field, the computer field, or other fields, and are not limited herein. The distributed service full link monitoring method comprises the following steps: acquiring a plurality of link data related to a service according to a call request of a user, wherein each link data comprises a tracking tag, and the tracking tag is used for marking the link data; routing the link data with the same tracking label to the same aggregation service node, and generating a topological structure according to the routing; and storing the topological structure and the link data in a graph database according to a preset rule to form a service call topological graph.
Fig. 1 schematically illustrates an application scenario diagram of a distributed service full link monitoring method, apparatus, electronic device, storage medium, and computer program product according to an embodiment of the present disclosure. It should be noted that fig. 1 is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.
As shown in fig. 1, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 may have installed thereon various communication client applications, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.
The server 105 may be a server providing various services, such as a background management server (for example only) providing support for websites browsed by users using the terminal devices 101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that the distributed service full link monitoring method provided by the embodiment of the present disclosure may be generally executed by the server 105. Accordingly, the distributed service full link monitoring apparatus provided by the embodiments of the present disclosure may be generally disposed in the server 105. The distributed service full link monitoring method provided by the embodiment of the present disclosure may also be executed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Correspondingly, the distributed service full-link monitoring apparatus provided by the embodiment of the present disclosure may also be disposed in a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The distributed service full link monitoring method according to the embodiment of the present disclosure will be described in detail below with reference to fig. 2 to 5.
Fig. 2 schematically shows a flow chart of a distributed service full link monitoring method according to an embodiment of the present disclosure.
As shown in fig. 2, the distributed service full link monitoring method specifically includes operations S210 to S230.
In operation S210, a plurality of link data related to a service are collected for a call request of a user, where each link data includes a tracking tag, and the tracking tag is used for marking the link data.
Full link monitoring is a kind of distributed service tracking for one user request process in a distributed system. The monitoring object of the distributed service tracking technology is each request (or called transaction) of a user, and the distributed service tracking specifically comprises recording or determining the call chain of the distributed service in the process of one user request and information of all nodes involved on the call chain. A distributed system generally includes a plurality of service modules, wherein each service module is deployed on a different server, and each service module can perform data interaction through an interface. In this embodiment, a service module may also be referred to as an application, and if an application is called in a request process, the application is a node that passes through in a user request process, and each node has a corresponding node label for representation. If the service call chain is identified at the request entrance when the user request is received, a tracking label is generated at the moment, the tracking label is transmitted in the whole link along with the whole call chain, so that the distributed nodes can be connected in series, and the call condition of the whole link can be monitored.
In the embodiment of the present disclosure, the multiple link data specifically refer to link data information generated by calling from each node of the distributed system for a call request of a user, where the link data information includes a tracking tag, where the tracking tag is used to mark link data when the call request passes through each node in a calling process, and thus the tracking tag can be transparently transmitted in a full link along with a call chain, thereby ensuring relevance between data and realizing monitoring of a call condition of the full link.
In operation S220, link data having the same trace label is routed to the same aggregation service node, and a topology is generated accordingly.
Specifically, after the link data containing the tracking label is obtained, a consistent hashing algorithm is applied to the scattered link data to route the link data with the same tracking label to the same aggregation service node. And then, carrying out topology calculation on the data belonging to the same link according to the service calling sequence to generate a topology structure.
In operation S230, the topology structure and the plurality of link data are stored in the graph database according to a preset rule, so as to form a service invocation topology graph.
The preset rule here includes "service name call service name". In the operation, the obtained topological structure and the plurality of link data are stored in a graph database according to the service name call service name, and a service calling topological graph is formed.
Because a series of related systems are mutually called aiming at a certain service calling scene (for example, the credit card is opened), which may involve a plurality of services, applications and clusters, when the topological structure of the service dimension is obtained by the method, the calling topological structures of the cluster dimension and the application dimension can be formed, and then the topological structures of the three dimensions can be stored in the graph database.
According to the distributed service full-link monitoring method in the embodiment of the disclosure, the same call chain data are collected based on the tracking label in the call chain, and the relevance between the data is ensured, so that the abnormal data is accurately positioned in an efficient manner, and the abnormal data is captured more quickly.
Fig. 3 schematically illustrates a flow chart of a distributed service full link monitoring method according to another embodiment of the present disclosure.
As shown in fig. 3, the distributed service full link monitoring method specifically includes operations S310 to S330. Here, operations S320 and S330 are implemented in the same manner as operations S220 and S230, respectively, and repeated parts will not be described in detail. In the embodiment of the present disclosure, operation S310 specifically includes operations S311 to S313.
In operation S311, for the call request of the user, the service call chain entry is identified, and a tracking tag is generated, so that when the call request passes through each node in the call process, the tracking tag is transparently transmitted along with the call chain in the full link.
In the embodiment of the disclosure, a Java annotation mode or a Setter method may be adopted to identify the service calling chain entry service and generate the tracking tag, so that when data related to the service is called, the tracking tag may be transparently transmitted over the whole link, giving business meaning to the link, and thereby concatenating all link information.
In operation S312, a plurality of link data related to the service are collected in a buried point manner.
In this embodiment, in a call link formed by one request, a node through which a call request initiated by a user passes is referred to as a local node, a next node of the local node through which the call request passes is a child node of the local node, and a previous node of the local node through which the call request passes is a parent node of the local node. When the calling request passes through the child node of the node, the node is the father node of the child node.
The transmission mode of the tracking label in the link is as follows: when the call request passes through the node (for example, the node is the entrance of the call chain where the node is), the tracking label of the node is generated based on the tracking label of the parent node of the node and the call request of the parent node of the node to the node, and the tracking label generated at the entrance of the service call chain is sequentially transmitted to the full link according to the above manner.
One calling chain represents one calling request, one calling chain comprises one or more nodes through which the calling request passes, each node on the same calling chain comprises the same tracking label, namely the tracking labels indicate the correlation among the nodes, therefore, in the face of a complex calling link, data on the same link can be found out only through the tracking labels of the calling link, and once an exception occurs in the calling process, the exception data or the fault can be quickly and accurately positioned.
In some embodiments, trace data is generated based on the trace tag as the invocation request passes through the nodes during the invocation. Wherein the tracking data includes at least one of: the starting time of calling, the protocol type, the calling party address, the calling party port, the service name of the request, the calling time, the calling result and the exception information. The tracking data of each node is generated by depending on the tracking label of the parent node, so that the uniqueness of the tracking data of each node can be ensured, and the tracking data can be completely recorded.
In operation S313, a plurality of link data related to the traffic is transmitted to the distributed log collection system based on the asynchronous manner.
In the embodiment of the disclosure, the link data generated by each call is sent to the distributed log collection system for temporary storage in an asynchronous manner, so that stable operation of services in the distributed system can be effectively ensured. The asynchronous mode can be used for sending the link data to the kafka tracking and collecting server for temporary storage in an http mode, for example. It should be noted that, the above-mentioned sending the link data generated by the call to the distributed log collection system may select other suitable sending manners and storage manners according to actual needs, which is not limited herein.
In operation S320, the link data having the same trace label is routed to the same aggregation service node, and a topology is generated accordingly.
In operation S330, the topology structure and the plurality of link data are stored in the graph database according to the preset rule, so as to form a service invocation topology graph.
Fig. 4 schematically illustrates a flow chart of a distributed service full link monitoring method according to another embodiment of the present disclosure.
As shown in fig. 4, the distributed service full link monitoring method specifically includes operations S410 to S430. Here, operations S410 and S430 are implemented in the same manner as operations S210 and S230, respectively, and repeated parts will not be described in detail. In the embodiment of the present disclosure, operation S420 specifically includes operations S421 to S423.
In operation S410, a plurality of link data related to a service are collected for a call request of a user, where each link data includes a trace tag, and the trace tag is used for marking the link data.
In operation S421, the link data having the same trace label is routed to the same aggregation service node.
In operation S422, link data having the same trace tag is interpreted as a character string set of a service name call service name.
In operation S423, a topology is generated according to the set of character strings.
Specifically, for example, an MD5 algorithm is adopted to generate a unique identification string from the character string set, and a topology is generated by using a service name as a point, a call relation as an edge, a tracking tag and an MD5 value corresponding to a call chain as attributes of the edge. Where one MD5 value represents a certain calling scenario, the service and calling scenarios may be in a one-to-many relationship.
It should be noted that, the generating of the unique identification string by combining the character string sets may also be implemented in other manners, and is selected according to actual needs, and is not limited herein.
In operation S430, the topology structure and the plurality of link data are stored in the graph database according to the preset rule, so as to form a service invocation topology graph.
Fig. 5 schematically illustrates a flow chart of a distributed service full link monitoring method according to another embodiment of the present disclosure.
As shown in fig. 5, the distributed service full link monitoring method specifically includes operations S510 to S540. Operation S510, operation S520, and operation S530 are implemented in the same manner as operation S210, operation S220, and operation S230, respectively, and repeated details will not be repeated.
In operation S510, a plurality of link data related to a service are collected for a call request of a user, where each link data includes a trace tag, and the trace tag is used to mark the link data.
In operation S520, link data having the same trace label is routed to the same aggregation service node, and a topology is generated accordingly.
In operation S530, the topology structure and the plurality of link data are stored in the graph database according to the preset rule, so as to form a service invocation topology graph.
In operation S540, based on the tracking tag, topology data associated with the tracking tag is read from the graph database and graphically displayed.
Specifically, the tracking tag can be directly input as a parameter, and topology data (such as the topology structure of the service dimension, the application dimension and the cluster dimension) related to the tracking tag and related tracking data can be screened and read from the graph database and displayed in a graphical form. The user can analyze and evaluate the overall indexes of the service scene according to the graphically displayed content and locate the fault problem by utilizing the tracking label. By adopting the method, the data required by the user can be quickly screened and inquired from the mass data, and the data analysis efficiency is improved.
Based on the distributed service full link monitoring method, the disclosure also provides a distributed service full link monitoring device. The apparatus will be described in detail below with reference to fig. 6, 7, 8 and 9.
Fig. 6 schematically illustrates a block diagram of a distributed service full link monitoring apparatus according to an embodiment of the present disclosure.
As shown in fig. 6, the distributed service full link monitoring apparatus 600 in the embodiment of the present disclosure includes: an acquisition module 610, a generation module 620 and a storage module 630.
The collecting module 610 is configured to collect, for a call request of a user, a plurality of link data related to a service, where each link data includes a tracking tag, and the tracking tag is used to mark the link data.
The generating module 620 is configured to route the link data with the same trace tag to the same aggregation service node, and generate the topology accordingly.
The storage module 630 is configured to store the topology structure and the plurality of link data in the graph database according to a preset rule, so as to form a service invocation topology graph.
Fig. 7 schematically illustrates a block diagram of a distributed service full link monitoring apparatus according to another embodiment of the present disclosure.
As shown in fig. 7, the distributed service full link monitoring apparatus 700 in the embodiment of the present disclosure includes: an acquisition module 710, a generation module 720 and a storage module 730. Wherein the acquisition module 710 comprises a first acquisition module 711, a second acquisition module 712 and a third acquisition module 713.
The first acquisition module 711 is configured to identify an entry of a service invocation chain for a call request of a user, and generate a tracking label, so that when the call request passes through each node in a call process, the tracking label is transparently transmitted along with the invocation chain in a full link.
The second collecting module 712 is used for collecting a plurality of link data related to the service in a buried point manner.
A third collecting module 713, configured to send, based on an asynchronous manner, a plurality of link data related to the service to the distributed log collection system.
The generating module 720 is used for routing the link data with the same tracing label to the same aggregation service node, and generating the topology structure accordingly.
The storage module 730 is configured to store the topology structure and the plurality of link data in the graph database according to a preset rule, so as to form a service invocation topology graph. Specifically, the preset rule here includes a service name call service name.
In some embodiments of the present disclosure, the second collecting module 712 is further configured to generate trace data based on the trace tag when a call request passes through each node in a call process. Wherein the trace data includes at least one of: the starting time of calling, the protocol type, the calling party address, the calling party port, the service name of the request, the calling time, the calling result and the exception information.
As shown in fig. 8, the distributed service full link monitoring apparatus 800 in the embodiment of the present disclosure includes: an acquisition module 810, a generation module 820 and a storage module 830. Wherein the generating module 820 comprises a first generating module 821 and a second generating module 822.
The collecting module 810 is configured to collect, for a call request of a user, a plurality of link data related to a service, where each link data includes a tracking tag, and the tracking tag is used to mark the link data.
The first generation module 821 is used to route the link data with the same trace label to the same aggregation service node.
The second generating module 822 is configured to interpret link data with the same tracking tag as a string set of the service name call service name, and generate a topology structure according to the string set.
The storage module 830 is configured to store the topology structure and the plurality of link data in the graph database according to a preset rule, so as to form a service invocation topology graph. Specifically, the preset rule here includes a service name call service name.
In some embodiments of the present disclosure, the second generating module 822 is further configured to generate a unique identification string by using an MD5 algorithm on the string set, and generate the topology by using the service name as a point, the call relation as an edge, and the MD5 value corresponding to the tracking label and the call chain as attributes of the edge.
As shown in fig. 9, the distributed service full link monitoring apparatus 900 in the embodiment of the present disclosure includes: an acquisition module 910, a generation module 920, a storage module 930, and a presentation module 940.
The collecting module 910 is configured to collect, for a call request of a user, a plurality of link data related to a service, where each link data includes a tracking tag, and the tracking tag is used for marking the link data.
The generating module 920 is configured to route the link data with the same tracking label to the same aggregation service node, and generate a topology accordingly.
The storage module 930 is configured to store the topology structure and the plurality of link data in the graph database according to a preset rule, so as to form a service invocation topology graph.
The presentation module 940 is configured to read topology data associated with the tracking tag from the graph database based on the tracking tag, and present the topology data in a graphical form.
It should be noted that the implementation, solved technical problems, implemented functions, and achieved technical effects of each module/unit/subunit and the like in the apparatus part embodiment are respectively the same as or similar to the implementation, solved technical problems, implemented functions, and achieved technical effects of each corresponding step in the method part embodiment, and are not described herein again.
According to the embodiment of the present disclosure, any plurality of the receiving module acquiring module 910, the generating module 920, the storing module 930, and the displaying module 940 may be combined into one module to be implemented, or any one of the modules may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 910, the generation module 920, the storage module 930, and the presentation module 940 may be implemented at least partially as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented by any one of three implementations of software, hardware, and firmware, or in a suitable combination of any of them. Alternatively, at least one of the acquisition module 910, the generation module 920, the storage module 930 and the presentation module 940 may be at least partially implemented as a computer program module, which when executed may perform the respective functions.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and do not violate the good custom of the public order.
Fig. 10 schematically illustrates a block diagram of an electronic device suitable for implementing a distributed service full link monitoring method according to an embodiment of the present disclosure.
As shown in fig. 10, an electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. Processor 1001 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 1001 may also include onboard memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the present disclosure.
In the RAM 1003, various programs and data necessary for the operation of the electronic apparatus 1000 are stored. The processor 1001, ROM 1002, and RAM 1003 are connected to each other by a bus 1004. The processor 1001 performs various operations of the method flow according to the embodiments of the present disclosure by executing programs in the ROM 1002 and/or the RAM 1003. Note that the programs may also be stored in one or more memories other than the ROM 1002 and the RAM 1003. The processor 1001 may also perform various operations of the method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include the ROM 1002 and/or the RAM 1003 described above and/or one or more memories other than the ROM 1002 and the RAM 1003.
Embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for performing the method illustrated in the flow chart. When the computer program product runs in a computer system, the program code is used for causing the computer system to realize the distributed service full link monitoring method provided by the embodiment of the disclosure.
The computer program performs the above-described functions defined in the system/apparatus of the embodiments of the present disclosure when executed by the processor 1001. The systems, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
In one embodiment, the computer program may be hosted on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted in the form of a signal on a network medium, distributed, downloaded and installed via the communication part 1009, and/or installed from the removable medium 1011. The computer program containing program code may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
In such an embodiment, the computer program may be downloaded and installed from a network through the communication part 1009 and/or installed from the removable medium 1011. The computer program performs the above-described functions defined in the system of the embodiment of the present disclosure when executed by the processor 1001. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
In accordance with embodiments of the present disclosure, program code for executing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, these computer programs may be implemented using high level procedural and/or object oriented programming languages, and/or assembly/machine languages. The programming language includes, but is not limited to, programming languages such as Java, C + +, python, the "C" language, or the like. The program code may execute entirely on the user computing device, partly on the user device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.
Claims (17)
1. A distributed service full link monitoring method comprises the following steps:
acquiring a plurality of link data related to a service aiming at a call request of a user, wherein each link data comprises a tracking label, and the tracking label is used for marking the link data;
routing the link data with the same tracking label to the same aggregation service node, and generating a topological structure according to the routing;
and storing the topological structure and the plurality of link data in a graph database according to a preset rule to form a service call topological graph.
2. The distributed service full link monitoring method according to claim 1, wherein the routing the link data with the same trace label onto the same aggregation service node and generating the topology according to the link data comprises:
routing link data with the same tracking label to the same aggregation service node;
and interpreting the link data with the same tracking label into a character string set of a call service name of the service name, and generating a topological structure according to the character string set.
3. The distributed service full link monitoring method of claim 2, wherein the generating a topology from the set of strings comprises:
generating a unique identification string by adopting an MD5 algorithm for the character string set;
and generating the topological structure by taking the service name as a point, the calling relation as an edge and the MD5 value corresponding to the tracking label and the calling chain as the attributes of the edge.
4. The distributed service full link monitoring method as claimed in claim 1, wherein the preset rule includes a service name call service name.
5. The distributed service full link monitoring method according to claim 1, wherein the collecting a plurality of link data related to the service according to the call request of the user comprises:
identifying a service call chain entrance aiming at a call request of a user, and generating the tracking label so that the tracking label is transmitted in a full link along with a call chain when the call request passes through each node in a call process;
collecting a plurality of link data related to the service in a buried point mode;
and sending the plurality of link data related to the service to a distributed log collection system based on an asynchronous mode.
6. The distributed service full link monitoring method of claim 5, further comprising:
generating tracking data based on the tracking label when a calling request passes through each node in the calling process;
the tracking data includes at least one of: the starting time of calling, the protocol type, the calling party address, the calling party port, the service name of the request, the calling time, the calling result and the exception information.
7. The distributed service full link monitoring method of claim 1, further comprising:
based on the tracking tag, reading topological data related to the tracking tag from the graph database, and displaying the topological data in a graphical form.
8. A distributed service full link monitoring apparatus, comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a plurality of link data related to a service aiming at a call request of a user, each link data comprises a tracking label, and the tracking label is used for marking the link data;
the generating module is used for routing the link data with the same tracking label to the same aggregation service node and generating a topological structure according to the link data;
and the storage module is used for storing the topological structure and the plurality of link data into a graph database according to a preset rule to form a service call topological graph.
9. The distributed service full link monitoring apparatus of claim 8, wherein the generating module comprises:
the first generation module is used for routing the link data with the same tracking label to the same aggregation service node;
and the second generation module is used for interpreting the link data with the same tracking label as a character string set of a service name call service name and generating a topological structure according to the character string set.
10. The distributed service full link monitoring apparatus of claim 9, wherein the second generating module is further configured to:
generating a unique identification string by adopting an MD5 algorithm for the character string set;
and generating the topological structure by taking the service name as a point, the calling relation as an edge and the MD5 value corresponding to the tracking label and the calling chain as the attributes of the edge.
11. The distributed service full link monitoring apparatus as claimed in claim 8, wherein the preset rule includes a service name call service name.
12. The distributed service full link monitoring apparatus of claim 8, wherein the acquisition module comprises:
the first acquisition module is used for identifying an entrance of a service call chain aiming at a call request of a user and generating the tracking label so that the tracking label is transmitted in a full link along with the call chain when the call request passes through each node in a call process;
the second acquisition module is used for acquiring a plurality of link data related to the service in a point-buried mode;
and the third acquisition module is used for sending the plurality of link data related to the service to the distributed log collection system based on an asynchronous mode.
13. The distributed service full link monitoring apparatus according to claim 12, wherein the second acquisition module is further configured to:
generating tracking data based on the tracking label when a calling request passes through each node in the calling process;
the tracking data includes at least one of: the starting time of calling, the protocol type, the calling party address, the calling party port, the service name of the request, the calling time, the calling result and the exception information.
14. The distributed service full link monitoring apparatus of claim 8, further comprising:
and the display module is used for reading the topological data related to the tracking label from the graph database based on the tracking label and displaying the topological data in a graphical form.
15. An electronic device, comprising:
one or more processors;
a memory for storing one or more programs,
wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-7.
16. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 7.
17. A computer program product comprising a computer program which, when executed by a processor, implements a method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111259247.1A CN113987074A (en) | 2021-10-27 | 2021-10-27 | Distributed service full-link monitoring method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111259247.1A CN113987074A (en) | 2021-10-27 | 2021-10-27 | Distributed service full-link monitoring method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113987074A true CN113987074A (en) | 2022-01-28 |
Family
ID=79742982
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111259247.1A Pending CN113987074A (en) | 2021-10-27 | 2021-10-27 | Distributed service full-link monitoring method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113987074A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114598622A (en) * | 2022-03-10 | 2022-06-07 | 平安科技（深圳）有限公司 | Data monitoring method and device, storage medium and computer equipment |
| CN114629768A (en) * | 2022-02-17 | 2022-06-14 | 亚信科技（南京）有限公司 | Application topology processing method and device, electronic equipment, storage medium and product |
| CN114666238A (en) * | 2022-03-16 | 2022-06-24 | 平安科技（深圳）有限公司 | Visual monitoring method, system, equipment and medium for data link |
| CN114785709A (en) * | 2022-03-24 | 2022-07-22 | 中国农业银行股份有限公司 | Monitoring method and device for full-link production pressure test |
| CN114844795A (en) * | 2022-04-27 | 2022-08-02 | 工银科技有限公司 | End-to-end link monitoring method and device |
| CN115118574A (en) * | 2022-06-07 | 2022-09-27 | 马上消费金融股份有限公司 | Data processing method, device and storage medium |
| CN115509869A (en) * | 2022-08-26 | 2022-12-23 | 国科础石(重庆)软件有限公司 | Service call chain data visual display method and device and electronic equipment |
| CN116049263A (en) * | 2023-03-31 | 2023-05-02 | 北京比格大数据有限公司 | Data call link tracking method, device and system, equipment and storage medium |
| CN116962407A (en) * | 2023-07-31 | 2023-10-27 | 上海中汇亿达金融信息技术有限公司 | Processing method, device and application of distributed link label |
| CN117176802A (en) * | 2023-11-02 | 2023-12-05 | 太平金融科技服务（上海）有限公司 | Full-link monitoring method and device for service request, electronic equipment and medium |
| CN117453280A (en) * | 2023-09-12 | 2024-01-26 | 湖南长银五八消费金融股份有限公司 | Code topology and service topology generation method, device, equipment and medium |
| CN117808570A (en) * | 2024-02-29 | 2024-04-02 | 梅州客商银行股份有限公司 | Method and device for monitoring all links of bank distributed system and electronic equipment |
-
2021
- 2021-10-27 CN CN202111259247.1A patent/CN113987074A/en active Pending
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114629768A (en) * | 2022-02-17 | 2022-06-14 | 亚信科技（南京）有限公司 | Application topology processing method and device, electronic equipment, storage medium and product |
| CN114598622B (en) * | 2022-03-10 | 2023-04-25 | 平安科技（深圳）有限公司 | Data monitoring method and device, storage medium and computer equipment |
| CN114598622A (en) * | 2022-03-10 | 2022-06-07 | 平安科技（深圳）有限公司 | Data monitoring method and device, storage medium and computer equipment |
| CN114666238A (en) * | 2022-03-16 | 2022-06-24 | 平安科技（深圳）有限公司 | Visual monitoring method, system, equipment and medium for data link |
| CN114666238B (en) * | 2022-03-16 | 2023-07-25 | 平安科技（深圳）有限公司 | Visual monitoring method, system, equipment and medium for data link |
| CN114785709A (en) * | 2022-03-24 | 2022-07-22 | 中国农业银行股份有限公司 | Monitoring method and device for full-link production pressure test |
| CN114844795A (en) * | 2022-04-27 | 2022-08-02 | 工银科技有限公司 | End-to-end link monitoring method and device |
| CN115118574A (en) * | 2022-06-07 | 2022-09-27 | 马上消费金融股份有限公司 | Data processing method, device and storage medium |
| CN115118574B (en) * | 2022-06-07 | 2023-07-21 | 马上消费金融股份有限公司 | Data processing method, device and storage medium |
| CN115509869A (en) * | 2022-08-26 | 2022-12-23 | 国科础石(重庆)软件有限公司 | Service call chain data visual display method and device and electronic equipment |
| CN115509869B (en) * | 2022-08-26 | 2023-10-31 | 国科础石(重庆)软件有限公司 | Method and device for visual display of service call chain data and electronic equipment |
| CN116049263A (en) * | 2023-03-31 | 2023-05-02 | 北京比格大数据有限公司 | Data call link tracking method, device and system, equipment and storage medium |
| CN116962407A (en) * | 2023-07-31 | 2023-10-27 | 上海中汇亿达金融信息技术有限公司 | Processing method, device and application of distributed link label |
| CN116962407B (en) * | 2023-07-31 | 2024-02-13 | 上海中汇亿达金融信息技术有限公司 | Distributed link label processing method and device, distributed link tracking system and distributed system |
| CN117453280A (en) * | 2023-09-12 | 2024-01-26 | 湖南长银五八消费金融股份有限公司 | Code topology and service topology generation method, device, equipment and medium |
| CN117453280B (en) * | 2023-09-12 | 2024-05-14 | 湖南长银五八消费金融股份有限公司 | Code topology and service topology generation method, device, equipment and medium |
| CN117176802A (en) * | 2023-11-02 | 2023-12-05 | 太平金融科技服务（上海）有限公司 | Full-link monitoring method and device for service request, electronic equipment and medium |
| CN117176802B (en) * | 2023-11-02 | 2024-01-16 | 太平金融科技服务（上海）有限公司 | Full-link monitoring method and device for service request, electronic equipment and medium |
| CN117808570A (en) * | 2024-02-29 | 2024-04-02 | 梅州客商银行股份有限公司 | Method and device for monitoring all links of bank distributed system and electronic equipment |
| CN117808570B (en) * | 2024-02-29 | 2024-05-07 | 梅州客商银行股份有限公司 | Method and device for monitoring all links of bank distributed system and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113987074A (en) | Distributed service full-link monitoring method and device, electronic equipment and storage medium | |
| CN109344170B (en) | Stream data processing method, system, electronic device and readable storage medium | |
| CN111190888A (en) | Method and device for managing graph database cluster | |
| US10084637B2 (en) | Automatic task tracking | |
| US11263267B1 (en) | Apparatuses, methods, and computer program products for generating interaction vectors within a multi-component system | |
| CN115357761A (en) | Link tracking method and device, electronic equipment and storage medium | |
| CN113505302A (en) | Method, device and system for supporting dynamic acquisition of buried point data and electronic equipment | |
| US10775751B2 (en) | Automatic generation of regular expression based on log line data | |
| CN111444077A (en) | Method and device for generating flow node test data | |
| US20140052851A1 (en) | Systems and methods for discovering sources of online content | |
| US10445385B2 (en) | Social sharing path user interface insights | |
| CN114153703A (en) | Micro-service exception positioning method and device, electronic equipment and program product | |
| CN114840379A (en) | Log generation method, device, server and storage medium | |
| CN113535577A (en) | Application testing method and device based on knowledge graph, electronic equipment and medium | |
| CN112954056A (en) | Monitoring data processing method and device, electronic equipment and storage medium | |
| CN110489326B (en) | IDS-based HTTPAPI debugging method device, medium and equipment | |
| CN113553123B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN114490130A (en) | Message subscription method and device, electronic equipment and storage medium | |
| CN113762910A (en) | Document monitoring method and device | |
| CN111651330A (en) | Data acquisition method and device, electronic equipment and computer readable storage medium | |
| CN116796086B (en) | Page display method, device, equipment and storage medium | |
| CN111767447A (en) | Method and device for determining user traffic path | |
| CN114844957B (en) | Link message conversion method, device, equipment, storage medium and program product | |
| CN107609008A (en) | A kind of data importing device and method from relevant database to Kafka based on Apache Sqoop | |
| CN115185798A (en) | Data acquisition and analysis method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |