CN111464352A - Call link data processing method and device - Google Patents
Call link data processing method and device Download PDFInfo
- Publication number
- CN111464352A CN111464352A CN202010243064.XA CN202010243064A CN111464352A CN 111464352 A CN111464352 A CN 111464352A CN 202010243064 A CN202010243064 A CN 202010243064A CN 111464352 A CN111464352 A CN 111464352A
- Authority
- CN
- China
- Prior art keywords
- topological relation
- link
- link data
- relation
- topology
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims description 12
- 238000004220 aggregation Methods 0.000 claims abstract description 47
- 230000002776 aggregation Effects 0.000 claims abstract description 47
- 238000000034 method Methods 0.000 claims abstract description 40
- 238000010276 construction Methods 0.000 claims abstract description 29
- 238000004458 analytical method Methods 0.000 claims abstract description 22
- 238000004364 calculation method Methods 0.000 claims abstract description 16
- RYYVLZVUVIJVGH-UHFFFAOYSA-N caffeine Chemical compound CN1C(=O)N(C)C(=O)C2=C1N=CN2C RYYVLZVUVIJVGH-UHFFFAOYSA-N 0.000 claims description 33
- 238000003860 storage Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 15
- LPHGQDQBBGAPDZ-UHFFFAOYSA-N Isocaffeine Natural products CN1C(=O)N(C)C(=O)C2=C1N(C)C=N2 LPHGQDQBBGAPDZ-UHFFFAOYSA-N 0.000 claims description 12
- 229960001948 caffeine Drugs 0.000 claims description 12
- VJEONQKOZGKCAK-UHFFFAOYSA-N caffeine Natural products CN1C(=O)N(C)C(=O)C2=C1C=CN2C VJEONQKOZGKCAK-UHFFFAOYSA-N 0.000 claims description 12
- 238000012163 sequencing technique Methods 0.000 claims description 9
- 238000009826 distribution Methods 0.000 claims description 7
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000004891 communication Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 13
- 238000005192 partition Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 239000000872 buffer Substances 0.000 description 4
- 230000008030 elimination Effects 0.000 description 4
- 238000003379 elimination reaction Methods 0.000 description 4
- 230000036541 health Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/12—Discovery or management of network topologies
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/24—Multipath
- H04L45/245—Link aggregation, e.g. trunking
Abstract
The embodiment of the application provides a method and a device for processing call link data, wherein the method comprises the following steps: receiving link information sent by a link data acquisition end through first message middleware, and forwarding the link information to a corresponding routing node for analysis to obtain span information; routing the span information to a corresponding topological relation construction container for topological aggregation through second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value; sending the topological relation and the link MD5 value to batch processing nodes through third message middleware to write in a graph database in batch, and carrying out graph calculation on the topological relation in the graph database to obtain a topological relation calling map; according to the method and the device, the accurate topological relation can be obtained through aggregation, and therefore timeliness and accuracy in problem analysis from a service perspective are improved.
Description
Technical Field
The application relates to the field of data processing, in particular to a method and a device for processing call link data.
Background
In the micro-service era, service scenes are split according to a domain model, and a complete service flow is closed only by message communication among a plurality of services. Large-scale distributed clusters, while carrying larger-scale concurrency, are more involved in invoking links. Especially, the construction of large and medium-sized stations, the multiplexing and combination of services constitute various business transactions. Especially, a large amount of link data can be generated in a business peak period, the prior art cannot realize large-scale real-time topology aggregation, cannot analyze and process all call relations timely and efficiently, cannot accurately perform operations such as application management fault prejudgment and the like, and therefore, the health condition of the whole business process is difficult to monitor.
Disclosure of Invention
Aiming at the problems in the prior art, the application provides a method and a device for processing call link data, which can aggregate to obtain an accurate topological relation, and further improve timeliness and accuracy when analyzing problems from a service perspective.
In order to solve at least one of the above problems, the present application provides the following technical solutions:
in a first aspect, the present application provides a method for processing call link data, including:
receiving link information sent by a link data acquisition end through first message middleware, and forwarding the link information to a corresponding routing node for analysis to obtain span information;
routing the span information to a corresponding topological relation construction container for topological aggregation through second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value;
and sending the topological relation and the link MD5 value to batch processing nodes through third message middleware to write in a graph database in batch, and carrying out graph calculation on the topological relation in the graph database to obtain a topological relation calling map.
Further, the routing, according to the tracking code in the span information, the span information to the corresponding topology relation construction container through a second message middleware for topology aggregation to obtain the corresponding topology relation and a link MD5 value includes:
and carrying out topology aggregation on the span information with the same tracking codes through the topology relation construction container to obtain a corresponding topology relation and store a corresponding link MD5 value.
Further, before the topology aggregation, the method further includes:
and carrying out integrity check and sequencing on the span information through the topological relation construction container.
Further, the sending the topological relation and the link MD5 value to the batching node through the third message middleware for batch writing into the graph database includes:
and judging whether the topological relation exists in a preset map database or a preset caffeine cache according to the link MD5 value, if not, writing the topological relation into the preset map database and the preset caffeine cache, otherwise, judging that the topological relation is repeated.
In a second aspect, the present application provides a call link data processing apparatus, including:
the link data distribution module is used for receiving link information sent by a link data acquisition end through the first message middleware and forwarding the link information to a corresponding routing node for analysis to obtain span information;
the topological relation building module is used for routing the span information to a corresponding topological relation building container through second message middleware for topological aggregation according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value;
and the batch processing module is used for sending the topological relation and the link MD5 value to batch processing nodes through third message middleware to write in a graph database in batches, and carrying out graph calculation on the topological relation in the graph database to obtain a topological relation calling map.
Further, the topological relation building module comprises:
and the topology aggregation unit is used for performing topology aggregation on the span information with the same tracking codes through the topology relation construction container to obtain a corresponding topology relation and store a corresponding link MD5 value.
Further, still include:
and the topology aggregation preprocessing unit is used for carrying out integrity check and sequencing on the span information through the topology relation construction container.
Further, the batch processing module includes:
and the duplication removing unit is used for judging whether the topological relation exists in a preset map database or a preset caffeine cache according to the link MD5 value, if not, writing the topological relation into the preset map database and the preset caffeine cache, otherwise, judging that the topological relation is repeated.
In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of invoking the link data processing method when executing the program.
In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of invoking the link data processing method.
According to the technical scheme, the method and the device for processing the call link data are characterized in that link information sent by a link data acquisition end is received through a first message middleware and forwarded to a corresponding routing node for analysis, so that span information is obtained; routing the span information to a corresponding topological relation construction container for topological aggregation through second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value; the topological relation and the link MD5 value are sent to batch processing nodes through third message middleware to be written into a graph database in batch, graph calculation is carried out on the topological relation in the graph database, a topological relation calling map is obtained, the accurate topological relation can be obtained through aggregation, and timeliness and accuracy of problem analysis from a service perspective are improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flowchart of a method for processing call link data in an embodiment of the present application;
FIG. 2 is a block diagram of a call link data processing apparatus according to an embodiment of the present application;
FIG. 3 is a second block diagram of a call link data processing apparatus according to an embodiment of the present application;
FIG. 4 is a third block diagram of a call link data processing apparatus according to an embodiment of the present application;
FIG. 5 is a flowchart illustrating a method for invoking link data processing according to an embodiment of the present application
Fig. 6 is a schematic structural diagram of an electronic device in an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In consideration of the problems that the prior art cannot realize large-scale real-time topology aggregation, cannot analyze and process all calling relations timely and efficiently, cannot accurately perform operations such as application management fault prejudgment and the like, and therefore the health condition of the whole service flow is difficult to monitor, the application provides a method and a device for processing calling link data, link information sent by a link data acquisition end is received through a first message middleware and is forwarded to a corresponding routing node for analysis, and span information is obtained; routing the span information to a corresponding topological relation construction container for topological aggregation through second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value; the topological relation and the link MD5 value are sent to batch processing nodes through third message middleware to be written into a graph database in batch, graph calculation is carried out on the topological relation in the graph database, a topological relation calling map is obtained, the accurate topological relation can be obtained through aggregation, and timeliness and accuracy of problem analysis from a service perspective are improved.
In order to aggregate an accurate topological relation and further improve timeliness and accuracy when analyzing a problem from a service perspective, the present application provides an embodiment of a method for processing call link data, and referring to fig. 1, the method for processing call link data specifically includes the following contents:
step S101: and receiving link information sent by the link data acquisition end through the first message middleware, and forwarding the link information to the corresponding routing node for analysis to obtain span information.
Optionally, after the link data acquisition end sends the data to the first message middleware (e.g., kafka), the routing node of the link data distribution layer consumes the link data, and parses the original compressed packet into multiple pieces of span information (i.e., the span information). Obtaining the traceid (namely the tracking code) from the span information for carrying out hash routing, and sending the data under the same traceid to the partition of the same second message middleware to ensure that the span information under the same traceid can be sent to a server for aggregation.
Optionally, all the routing nodes are deployed in a containerization manner, and can be dynamically expanded and contracted according to pressure.
Step S102: and routing the span information to a corresponding topological relation construction container for topological aggregation through a second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value.
Optionally, the traceid (that is, the tracking code) is obtained from the span information to perform hash routing, and the data under the same traceid is sent to the partition of the same second message middleware, so that the span information under the same traceid can be sent to one server to be aggregated.
Optionally, each of the topology construction containers may consume 1 to n partitions (corresponding to the second message middleware topic). When the container is started, temporary nodes of the instance numbers are created on the zookeeper, the temporary nodes are sequentially increased, and the corresponding partitions are consumed according to the instance numbers, so that the situation that the temporary nodes are not overlapped and leaked is achieved. The consumed span is written into a redis cache, and the traceid in the span is written into a local cafefine cache. The cafeine cache can set a storage period, a cafeine removal monitor can be triggered when data is expired, and all span under one traceid is fetched from the redis according to the expired traceid to perform subsequent aggregation operation.
Optionally, after integrity check and sequencing are performed on the span, three topological relations applied to application, cluster to cluster, and service to service are analyzed from all span messages under the unified traceid, and an MD5 value for marking the link is obtained. Because the nodes for carrying out the aggregation topology are numerous and the aggregated relation is repeated, the information is sent to the next layer for duplication elimination through the message middleware, so that the resource exhaustion of a database connection pool is avoided, and the storage pressure of the database is reduced.
Step S103: and sending the topological relation and the link MD5 value to batch processing nodes through third message middleware to write in a graph database in batch, and carrying out graph calculation on the topological relation in the graph database to obtain a topological relation calling map.
Optionally, the topological relation is drawn from a preset map database and written into a local cache cafeine, the maximum number of storage strips is set for the cafeine, and elimination is performed through a L RU policy.
As can be seen from the above description, the method for processing call link data provided in the embodiment of the present application can receive link information sent by a link data acquisition end through a first message middleware, and forward the link information to a corresponding routing node for analysis to obtain span information; routing the span information to a corresponding topological relation construction container for topological aggregation through second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value; the topological relation and the link MD5 value are sent to batch processing nodes through third message middleware to be written into a graph database in batch, graph calculation is carried out on the topological relation in the graph database, a topological relation calling map is obtained, the accurate topological relation can be obtained through aggregation, and timeliness and accuracy of problem analysis from a service perspective are improved.
In order to obtain an accurate topological relation and a corresponding link MD5 value, in an embodiment of the method for invoking link data processing according to the present application, the step S102 may further include the following steps:
and carrying out topology aggregation on the span information with the same tracking codes through the topology relation construction container to obtain a corresponding topology relation and store a corresponding link MD5 value.
Optionally, after integrity check and sequencing are performed on the span, three topological relations applied to application, cluster to cluster, and service to service are analyzed from all span messages under the unified traceid, and an MD5 value for marking the link is obtained.
In order to improve the accuracy of the topological relation, in an embodiment of the method for invoking link data processing according to the present application, the following may be specifically included before the step S102:
and carrying out integrity check and sequencing on the span information through the topological relation construction container.
Optionally, before performing the aggregation topology, integrity check and sorting may be performed on the received span information by using an existing data checking method and an existing sorting method (time sorting, data amount sorting, and the like).
In order to obtain the accurate writing of the topological relation into the graph database and avoid the waste of system resources caused by the repeated data, in an embodiment of the method for processing call link data of the present application, the step S103 may further specifically include the following contents:
and judging whether the topological relation exists in a preset map database or a preset caffeine cache according to the link MD5 value, if not, writing the topological relation into the preset map database and the preset caffeine cache, otherwise, judging that the topological relation is repeated.
Optionally, the topological relation is drawn from a preset map database and written into a local cache cafeine, the maximum number of storage strips is set for the cafeine, and elimination is performed through a L RU policy.
In order to aggregate and obtain an accurate topological relation and further improve timeliness and accuracy when analyzing a problem from a service perspective, the present application provides an embodiment of a call link data processing apparatus for implementing all or part of contents of the call link data processing method, and referring to fig. 2, the call link data processing apparatus specifically includes the following contents:
the link data distribution module 10 is configured to receive link information sent by a link data acquisition end through a first message middleware, and forward the link information to a corresponding routing node for analysis to obtain span information.
And the topology relationship building module 20 is configured to route the span information to a corresponding topology relationship building container through a second message middleware for topology aggregation according to the tracking code in the span information, so as to obtain a corresponding topology relationship and a link MD5 value.
And the batch processing module 30 is configured to send the topological relation and the link MD5 value to batch processing nodes through third message middleware to write the topological relation and the link MD5 value into a graph database in batch, and perform graph calculation on the topological relation in the graph database to obtain a topological relation call map.
As can be seen from the above description, the call link data processing apparatus provided in the embodiment of the present application can receive link information sent by a link data acquisition end through the first message middleware, and forward the link information to the corresponding routing node for analysis to obtain span information; routing the span information to a corresponding topological relation construction container for topological aggregation through second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value; the topological relation and the link MD5 value are sent to batch processing nodes through third message middleware to be written into a graph database in batch, graph calculation is carried out on the topological relation in the graph database, a topological relation calling map is obtained, the accurate topological relation can be obtained through aggregation, and timeliness and accuracy of problem analysis from a service perspective are improved.
In order to obtain an accurate topological relation and a corresponding link MD5 value, in an embodiment of the present application, referring to fig. 3, where the link data processing apparatus is called, the topological relation building module 20 includes:
and the topology aggregation unit 21 is configured to perform topology aggregation on the span information with the same tracking code through the topology relation construction container to obtain a corresponding topology relation, and store a corresponding link MD5 value.
In order to improve the accuracy of the topological relation, in an embodiment of the present application, the following is further specifically included:
and the topology aggregation preprocessing unit is used for carrying out integrity check and sequencing on the span information through the topology relation construction container.
In order to obtain accurate writing of the topological relation into the graph database and avoid wasting system resources by repeated data, in an embodiment of the call link data processing apparatus of the present application, referring to fig. 4, the batch processing module 30 includes:
and the duplication removing unit 31 is configured to determine whether the topological relation exists in a preset map database or a preset cafeiine cache according to the link MD5 value, if not, write the topological relation into the preset map database and the preset cafeiine cache, and otherwise, determine that the topological relation is duplicated.
To further explain the present solution, the present application further provides a specific application example for implementing the method for processing call link data by using the call link data processing apparatus, and referring to fig. 5, the specific application example specifically includes the following contents:
the link data distribution layer, the topological relation construction layer and the batch processing layer are connected together through three different message middleware, and the original data is processed in multiple layers to finally generate the topological relation, which specifically comprises the following steps:
1. link data distribution layer-router
And after the acquisition end sends the data to the message middleware, the routing node of the link data distribution layer consumes the link data. The original compressed message is analyzed into a plurality of span information, the traceid is obtained from the span to carry out Hash routing, the data under the same traceid is sent to the partition of the same message middleware, and the span under the same traceid can be sent to a server to carry out aggregation. All routing nodes are deployed in a containerization mode and can be dynamically stretched according to pressure.
2. Topology build layer-process
A Process node is a container containing java engineering and redis cache, each container consuming 1 to n partitions (message middleware topic). When the container is started, temporary nodes of the instance numbers are created on the zookeeper, the temporary nodes are sequentially increased, and the corresponding partitions are consumed according to the instance numbers, so that the situation that the temporary nodes are not overlapped and leaked is achieved. The consumed span is written into a redis cache, and the traceid in the span is written into a local cafefine cache. The Caffeine cache can set a storage period, a Caffeine removal monitor can be triggered when data is expired, all span under one traceid is fished out from the redis according to the expired traceid, and subsequent aggregation operation is carried out.
After the Process node carries out integrity check and sequencing on the spans, three topological relations of application, cluster to cluster and service to service are analyzed from all span messages under the unified traceid, and an MD5 value for marking the link is obtained. Because the number of the processing nodes is large and the aggregated relation is repeated, the information is sent to the next layer for duplication elimination through the message middleware, so that the resource exhaustion of a database connection pool is avoided and the storage pressure of the database is reduced.
The maximum deployable number of the Process nodes is the same as the number of the partitions, and each node can consume several partitions according to pressure.
3. Batch processing layer-batch
The topology relationship analyzed by the Process node is written into the map database and the cafeine cache if the topology relationship does not appear in the cafeine cache, and no operation is performed if the topology relationship appears in the cafeine cache.
The Batch nodes are all deployed in a containerization mode, and dynamic telescopic adjustment can be performed according to pressure.
4. Link topology storage layer-neo 4j
Various calling relations are stored in neo4j for graph calculation, and maps of global service calling, cluster calling and application calling are drawn. And then by means of marking service labels and the like, the calling relations of all levels can be easily and quickly searched in the service view angle.
As can be seen from the above, the present application can also achieve the following technical effects:
based on message-driven microservice, from the perspective of business, the health condition of the whole business process is monitored, the overall situation and the upstream and downstream are visually seen, and the problems are conveniently and quickly found and positioned. An intelligent service map is provided, the bottleneck of business transaction is found visually, and related services can be limited in current, fused and the like according to expert experience or deep learning experience. The accurate flow guiding or flow limiting of the business transaction is achieved. The stable and healthy operation of the whole system is guaranteed.
In order to obtain an accurate topological relation in an aggregation manner from a hardware level and further improve timeliness and accuracy when analyzing a problem from a service perspective, the present application provides an embodiment of an electronic device for implementing all or part of the contents in the method for invoking link data processing, where the electronic device specifically includes the following contents:
a processor (processor), a memory (memory), a communication Interface (Communications Interface), and a bus; the processor, the memory and the communication interface complete mutual communication through the bus; the communication interface is used for realizing information transmission between the calling link data processing device and relevant equipment such as a core service system, a user terminal, a relevant database and the like; the logic controller may be a desktop computer, a tablet computer, a mobile terminal, and the like, but the embodiment is not limited thereto. In this embodiment, the logic controller may refer to the embodiment of the method for processing call link data and the embodiment of the apparatus for processing call link data in the embodiments for implementation, and the contents thereof are incorporated herein, and repeated descriptions are omitted.
It is understood that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a Personal Digital Assistant (PDA), an in-vehicle device, a smart wearable device, and the like. Wherein, intelligence wearing equipment can include intelligent glasses, intelligent wrist-watch, intelligent bracelet etc..
In practical applications, the part calling the link data processing method may be executed on the electronic device side as described above, or all operations may be completed in the client device. The selection may be specifically performed according to the processing capability of the client device, the limitation of the user usage scenario, and the like. This is not a limitation of the present application. The client device may further include a processor if all operations are performed in the client device.
The client device may have a communication module (i.e., a communication unit), and may be communicatively connected to a remote server to implement data transmission with the server. The server may include a server on the task scheduling center side, and in other implementation scenarios, the server may also include a server on an intermediate platform, for example, a server on a third-party server platform that is communicatively linked to the task scheduling center server. The server may include a single computer device, or may include a server cluster formed by a plurality of servers, or a server structure of a distributed apparatus.
Fig. 6 is a schematic block diagram of a system configuration of an electronic device 9600 according to an embodiment of the present application. As shown in fig. 6, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this FIG. 6 is exemplary; other types of structures may also be used in addition to or in place of the structure to implement telecommunications or other functions.
In one embodiment, the function of invoking the link data processing method may be integrated into central processor 9100. The central processor 9100 may be configured to control as follows:
step S101: and receiving link information sent by the link data acquisition end through the first message middleware, and forwarding the link information to the corresponding routing node for analysis to obtain span information.
Step S102: and routing the span information to a corresponding topological relation construction container for topological aggregation through a second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value.
Step S103: and sending the topological relation and the link MD5 value to batch processing nodes through third message middleware to write in a graph database in batch, and carrying out graph calculation on the topological relation in the graph database to obtain a topological relation calling map.
As can be seen from the above description, the electronic device provided in the embodiment of the present application receives link information sent by a link data acquisition end through a first message middleware, and forwards the link information to a corresponding routing node for analysis, so as to obtain span information; routing the span information to a corresponding topological relation construction container for topological aggregation through second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value; the topological relation and the link MD5 value are sent to batch processing nodes through third message middleware to be written into a graph database in batch, graph calculation is carried out on the topological relation in the graph database, a topological relation calling map is obtained, the accurate topological relation can be obtained through aggregation, and timeliness and accuracy of problem analysis from a service perspective are improved.
In another embodiment, the call link data processing apparatus may be configured separately from the central processor 9100, for example, the call link data processing apparatus may be configured as a chip connected to the central processor 9100, and the call link data processing method function is realized by the control of the central processor.
As shown in fig. 6, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 also does not necessarily include all of the components shown in fig. 6; further, the electronic device 9600 may further include components not shown in fig. 6, which may be referred to in the art.
As shown in fig. 6, a central processor 9100, sometimes referred to as a controller or operational control, can include a microprocessor or other processor device and/or logic device, which central processor 9100 receives input and controls the operation of the various components of the electronic device 9600.
The memory 9140 can be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 9100 can execute the program stored in the memory 9140 to realize information storage or processing, or the like.
An input unit 9120 provides input to the cpu 9100, the input unit 9120 is, for example, a key or a touch input device, a power supply 9170 supplies power to the electronic apparatus 9600, a display 9160 displays display objects such as images and characters, and the display may be, for example, an L CD display, but is not limited thereto.
The memory 9140 can be a solid state memory, e.g., Read Only Memory (ROM), Random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 9140 could also be some other type of device. Memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 being used for storing application programs and function programs or for executing a flow of operations of the electronic device 9600 by the central processor 9100.
The memory 9140 can also include a data store 9143, the data store 9143 being used to store data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, contact book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in the case of a conventional mobile communication terminal.
Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing ordinary telecommunications functions. The audio processor 9130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100, thereby enabling recording locally through the microphone 9132 and enabling locally stored sounds to be played through the speaker 9131.
An embodiment of the present application further provides a computer-readable storage medium capable of implementing all the steps in the call link data processing method with the execution subject being the server or the client in the foregoing embodiments, where the computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the computer program implements all the steps in the call link data processing method with the execution subject being the server or the client in the foregoing embodiments, for example, when the processor executes the computer program, the processor implements the following steps:
step S101: and receiving link information sent by the link data acquisition end through the first message middleware, and forwarding the link information to the corresponding routing node for analysis to obtain span information.
Step S102: and routing the span information to a corresponding topological relation construction container for topological aggregation through a second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value.
Step S103: and sending the topological relation and the link MD5 value to batch processing nodes through third message middleware to write in a graph database in batch, and carrying out graph calculation on the topological relation in the graph database to obtain a topological relation calling map.
As can be seen from the above description, the computer-readable storage medium provided in the embodiment of the present application receives, through the first message middleware, link information sent by a link data acquisition end, and forwards the link information to the corresponding routing node for analysis, so as to obtain span information; routing the span information to a corresponding topological relation construction container for topological aggregation through second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value; the topological relation and the link MD5 value are sent to batch processing nodes through third message middleware to be written into a graph database in batch, graph calculation is carried out on the topological relation in the graph database, a topological relation calling map is obtained, the accurate topological relation can be obtained through aggregation, and timeliness and accuracy of problem analysis from a service perspective are improved.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The principle and the implementation mode of the invention are explained by applying specific embodiments in the invention, and the description of the embodiments is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A method for call link data processing, the method comprising:
receiving link information sent by a link data acquisition end through first message middleware, and forwarding the link information to a corresponding routing node for analysis to obtain span information;
routing the span information to a corresponding topological relation construction container for topological aggregation through second message middleware according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value;
and sending the topological relation and the link MD5 value to batch processing nodes through third message middleware to write in a graph database in batch, and carrying out graph calculation on the topological relation in the graph database to obtain a topological relation calling map.
2. The method for call link data processing according to claim 1, wherein the routing the span information to a corresponding topology relation construction container through a second message middleware for topology aggregation according to the tracking code in the span information, to obtain a corresponding topology relation and a link MD5 value, includes:
and carrying out topology aggregation on the span information with the same tracking codes through the topology relation construction container to obtain a corresponding topology relation and store a corresponding link MD5 value.
3. The method for call link data processing according to claim 1, further comprising, before said performing topology aggregation:
and carrying out integrity check and sequencing on the span information through the topological relation construction container.
4. The method for call link data processing according to claim 1, wherein the sending the topological relation and the link MD5 value to a batching node through a third message middleware for batch writing to a graph database comprises:
and judging whether the topological relation exists in a preset map database or a preset caffeine cache according to the link MD5 value, if not, writing the topological relation into the preset map database and the preset caffeine cache, otherwise, judging that the topological relation is repeated.
5. A call link data processing apparatus, comprising:
the link data distribution module is used for receiving link information sent by a link data acquisition end through the first message middleware and forwarding the link information to a corresponding routing node for analysis to obtain span information;
the topological relation building module is used for routing the span information to a corresponding topological relation building container through second message middleware for topological aggregation according to the tracking codes in the span information to obtain a corresponding topological relation and a link MD5 value;
and the batch processing module is used for sending the topological relation and the link MD5 value to batch processing nodes through third message middleware to write in a graph database in batches, and carrying out graph calculation on the topological relation in the graph database to obtain a topological relation calling map.
6. The call link data processing apparatus according to claim 5, wherein the topological relation building module comprises:
and the topology aggregation unit is used for performing topology aggregation on the span information with the same tracking codes through the topology relation construction container to obtain a corresponding topology relation and store a corresponding link MD5 value.
7. The call link data processing apparatus according to claim 5, further comprising:
and the topology aggregation preprocessing unit is used for carrying out integrity check and sequencing on the span information through the topology relation construction container.
8. The call link data processing apparatus according to claim 5, wherein the batch processing module includes:
and the duplication removing unit is used for judging whether the topological relation exists in a preset map database or a preset caffeine cache according to the link MD5 value, if not, writing the topological relation into the preset map database and the preset caffeine cache, otherwise, judging that the topological relation is repeated.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of invoking the link data processing method according to any one of claims 1 to 4 are implemented by the processor when executing the program.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of invoking the link data processing method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010243064.XA CN111464352A (en) | 2020-03-31 | 2020-03-31 | Call link data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010243064.XA CN111464352A (en) | 2020-03-31 | 2020-03-31 | Call link data processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111464352A true CN111464352A (en) | 2020-07-28 |
Family
ID=71680912
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010243064.XA Pending CN111464352A (en) | 2020-03-31 | 2020-03-31 | Call link data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111464352A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112615786A (en) * | 2020-12-04 | 2021-04-06 | 北京神州泰岳软件股份有限公司 | Route determining method and device, electronic equipment and computer readable storage medium |
| CN113254341A (en) * | 2021-05-31 | 2021-08-13 | 康键信息技术（深圳）有限公司 | Link data tracking method, device, equipment and storage medium |
| CN114598622A (en) * | 2022-03-10 | 2022-06-07 | 平安科技（深圳）有限公司 | Data monitoring method and device, storage medium and computer equipment |
| CN114827639A (en) * | 2021-01-28 | 2022-07-29 | 华为技术有限公司 | Multi-application distributed implementation method, readable medium and electronic device thereof |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140032974A1 (en) * | 2012-07-25 | 2014-01-30 | Texas Instruments Incorporated | Method for generating descriptive trace gaps |
| CN110442641A (en) * | 2019-08-06 | 2019-11-12 | 中国工商银行股份有限公司 | A kind of link topology figure methods of exhibiting, device, storage medium and equipment |
| CN110730246A (en) * | 2019-10-23 | 2020-01-24 | 家乡互动(厦门)网络科技有限公司 | Distributed link tracking method under micro-service architecture |
-
2020
- 2020-03-31 CN CN202010243064.XA patent/CN111464352A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140032974A1 (en) * | 2012-07-25 | 2014-01-30 | Texas Instruments Incorporated | Method for generating descriptive trace gaps |
| CN110442641A (en) * | 2019-08-06 | 2019-11-12 | 中国工商银行股份有限公司 | A kind of link topology figure methods of exhibiting, device, storage medium and equipment |
| CN110730246A (en) * | 2019-10-23 | 2020-01-24 | 家乡互动(厦门)网络科技有限公司 | Distributed link tracking method under micro-service architecture |
Non-Patent Citations (1)
| Title |
|---|
| 杨帆: "基于zipkin协议的分布式调用跟踪方案", 《福建电脑》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112615786A (en) * | 2020-12-04 | 2021-04-06 | 北京神州泰岳软件股份有限公司 | Route determining method and device, electronic equipment and computer readable storage medium |
| CN114827639A (en) * | 2021-01-28 | 2022-07-29 | 华为技术有限公司 | Multi-application distributed implementation method, readable medium and electronic device thereof |
| CN114827639B (en) * | 2021-01-28 | 2023-07-11 | 华为技术有限公司 | Multi-application distributed implementation method, readable medium and electronic device thereof |
| CN113254341A (en) * | 2021-05-31 | 2021-08-13 | 康键信息技术（深圳）有限公司 | Link data tracking method, device, equipment and storage medium |
| CN113254341B (en) * | 2021-05-31 | 2023-10-20 | 康键信息技术（深圳）有限公司 | Link data tracking method, device, equipment and storage medium |
| CN114598622A (en) * | 2022-03-10 | 2022-06-07 | 平安科技（深圳）有限公司 | Data monitoring method and device, storage medium and computer equipment |
| CN114598622B (en) * | 2022-03-10 | 2023-04-25 | 平安科技（深圳）有限公司 | Data monitoring method and device, storage medium and computer equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111464352A (en) | Call link data processing method and device | |
| CN111818136B (en) | Data processing method, device, electronic equipment and computer readable medium | |
| CN111786885B (en) | Distributed full-link gray level routing method and device | |
| CN111782470A (en) | Distributed container log data processing method and device | |
| CN112615753A (en) | Link abnormity tracking method, first node, second node and link | |
| CN111143039A (en) | Virtual machine scheduling method and device and computer storage medium | |
| CN111782473A (en) | Distributed log data processing method, device and system | |
| CN114257532B (en) | Method and device for detecting state of server | |
| CN111445331A (en) | Transaction matching method and device | |
| CN112486825B (en) | Multi-lane environment architecture system, message consumption method, device, equipment and medium | |
| CN111190731A (en) | Cluster task scheduling system based on weight | |
| CN111767558A (en) | Data access monitoring method, device and system | |
| CN115914375A (en) | Disaster tolerance processing method and device for distributed message platform | |
| CN110022323A (en) | A kind of method and system of the cross-terminal real-time, interactive based on WebSocket and Redux | |
| CN114374614A (en) | Network topology configuration method and device | |
| CN113965563B (en) | Service processing method and device based on model and server | |
| CN111813529B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN114637531A (en) | Method and device for dynamically generating application program interface | |
| CN113645151A (en) | DUP equipment message management method and device | |
| CN111191103B (en) | Method, device and storage medium for identifying and analyzing enterprise subject information from internet | |
| CN114237896A (en) | Distributed node resource dynamic scheduling method and device | |
| CN113434423A (en) | Interface test method and device | |
| CN112764957A (en) | Application fault delimiting method and device | |
| CN113050985A (en) | Front-end engineering dependence automatic registration method and device | |
| CN111343172A (en) | Network access authority dynamic processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200728 |
|
| RJ01 | Rejection of invention patent application after publication |