CN116049263A - Data call link tracking method, device and system, equipment and storage medium - Google Patents
Data call link tracking method, device and system, equipment and storage medium Download PDFInfo
- Publication number
- CN116049263A CN116049263A CN202310337468.9A CN202310337468A CN116049263A CN 116049263 A CN116049263 A CN 116049263A CN 202310337468 A CN202310337468 A CN 202310337468A CN 116049263 A CN116049263 A CN 116049263A
- Authority
- CN
- China
- Prior art keywords
- data
- time sequence
- dimension
- call
- link
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The present disclosure relates to a method, an apparatus, a system, a device and a storage medium for tracking a data call link, where the method includes: performing rasterization processing on calling parameters called for multiple times under the same data request to obtain rasterized calling parameters; classifying the rasterized call parameters to generate multi-dimension call parameters, and time sequence data and unstructured data of each dimension call parameter; according to the time sequence data of each dimension calling parameter, constructing multi-dimension time sequence data link data; the unstructured data of each dimension call parameter is matched with the multi-dimension time sequence data link data to obtain multi-dimension call parameter link data, and the unstructured data and the time sequence data are processed separately, so that the processing capacity of the link data can be reduced and the processing efficiency of the link data can be improved on the premise of ensuring the integrity of the link data.
Description
Technical Field
The disclosure relates to the technical field of big data, and in particular relates to a method, a device, a system, equipment and a storage medium for tracking a data call link.
Background
In the cloud primordial age, microservice architecture has become the standard for many large and medium-sized enterprises, government agencies, splitting huge monolithic applications into multiple subsystems and common component units. This concept brings many benefits: such as simplification and isolation of the complex system, reusability improvement and more reasonable resource allocation of the public module, great improvement of the speed of system change iteration, more flexible expandability, applicability in cloud computing and the like.
However, the micro-service architecture also brings new problems: if the user requests are split, tens of interactions of subsystems can be required for final return of results, and if a certain request is wrong, positioning problems can be required to be checked subsystem by subsystem; or a certain request is high in time consumption and cannot be traced, so that the operation and maintenance difficulty is increased.
Disclosure of Invention
To solve or at least partially solve the above technical problems, embodiments of the present disclosure provide a method, apparatus, system, device, and storage medium for tracing a data call link.
In a first aspect, embodiments of the present disclosure provide a data call link tracking method, including:
performing rasterization processing on calling parameters called for multiple times under the same data request to obtain rasterized calling parameters;
classifying the rasterized call parameters to generate multi-dimension call parameters, and time sequence data and unstructured data of each dimension call parameter;
according to the time sequence data of each dimension calling parameter, constructing multi-dimension time sequence data link data;
and matching unstructured data of each dimension call parameter with the multi-dimension time sequence data link data to obtain multi-dimension call parameter link data.
In one possible implementation manner, the classifying the rasterized call parameters to generate the multi-dimension call parameters and the time sequence data and unstructured data of each dimension call parameter includes:
classifying the rasterized call parameters to generate multi-dimensional call parameters, wherein the multi-dimensional call parameters comprise at least two of clusters, namespaces, probes, instances, clients, services, containers, users and databases;
and dividing each dimension call parameter into time sequence data and unstructured data according to preset characteristic data.
In one possible implementation manner, the constructing multi-dimensional time sequence data link data according to the time sequence data of each dimension calling parameter includes:
classifying the time sequence data of each dimension respectively to obtain a plurality of time sequence data nodes;
according to at least one of time information and father node information in the time sequence data nodes, different time sequence data nodes are associated to form time sequence data node link data;
and respectively nesting time sequence data nodes with different dimensions in the time sequence data node links with different dimensions to obtain a plurality of multi-dimensional time sequence data areas and multi-dimensional time sequence data link data formed by the multi-dimensional time sequence data areas.
In a possible implementation manner, the matching the unstructured data of each dimension call parameter with the multi-dimension time sequence data link data to obtain multi-dimension call parameter link data includes:
determining dimension information of unstructured data to be matched;
extracting target time sequence data nodes corresponding to dimension information of unstructured data to be matched from a multi-dimensional time sequence data area of multi-dimensional time sequence data link data;
performing similarity matching on unstructured data to be matched and a target time sequence data node;
and supplementing unstructured data matched with the target time sequence data node into the target time sequence data to obtain multidimensional call parameter link data.
In one possible embodiment, before classifying the rasterized call parameters, the method further comprises:
and carrying out noise reduction treatment on the rasterized call parameters, thereby classifying the rasterized call parameters after noise reduction.
In one possible embodiment, the method further comprises:
responding to the received link data query request, and extracting a target service scene in the link data query request;
determining dimension information corresponding to a target service scene based on a corresponding relation between the preset target service scene and the dimension information;
and displaying the link data consistent with the dimension information in the multi-dimension call parameter link data.
In a second aspect, embodiments of the present disclosure provide a data call link tracking apparatus, comprising:
the rasterization module is used for carrying out rasterization processing on calling parameters called for multiple times under the same data request to obtain rasterized calling parameters;
the classification module is used for classifying the rasterized calling parameters and generating multi-dimensional calling parameters, time sequence data and unstructured data of each dimension calling parameter;
the construction module is used for constructing multi-dimensional time sequence data link data according to the time sequence data of each dimension calling parameter;
and the matching module is used for matching unstructured data of each dimension calling parameter with the multi-dimension time sequence data link data to obtain multi-dimension calling parameter link data.
In a third aspect, embodiments of the present disclosure provide a data call link tracking system comprising a first data center and at least one second data center, comprising:
the first data center performs rasterization processing on a first calling parameter which is called for multiple times under the same local data request to obtain the first rasterization calling parameter, classifies the first rasterization calling parameter, and generates a first multi-dimensional calling parameter, and first time sequence data and first unstructured data of each dimension calling parameter;
the second data center performs rasterization processing on a second calling parameter which is called for multiple times under the same local data request to obtain a second rasterization calling parameter, classifies the second rasterization calling parameter, generates second time sequence data and second unstructured data of the second multi-dimensional calling parameter and each dimension calling parameter, and sends the second time sequence data and the second unstructured data to the first data center respectively;
the first data center constructs multi-dimensional time sequence data link data according to the first time sequence data and the second time sequence data of each dimension calling parameter, and matches the first unstructured data and the second unstructured data of each dimension calling parameter with the multi-dimensional time sequence data link data to obtain multi-dimensional calling parameter link data.
In a fourth aspect, embodiments of the present disclosure provide an electronic device including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing the data calling link tracking method when executing the program stored in the memory.
In a fifth aspect, embodiments of the present disclosure provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the data call link tracking method described above.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has at least part or all of the following advantages:
according to the data call link tracking method, the call parameters which are called for multiple times under the same data request are subjected to rasterization processing, and the rasterized call parameters are obtained; classifying the rasterized call parameters to generate multi-dimension call parameters, and time sequence data and unstructured data of each dimension call parameter; according to the time sequence data of each dimension calling parameter, constructing multi-dimension time sequence data link data; the unstructured data of each dimension call parameter is matched with the multi-dimension time sequence data link data to obtain multi-dimension call parameter link data, and the unstructured data and the time sequence data are processed separately, so that the processing capacity of the link data can be reduced and the processing efficiency of the link data can be improved on the premise of ensuring the integrity of the link data.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings that are required to be used in the description of the embodiments or the related art will be briefly described below, and it will be apparent to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
FIG. 1 schematically illustrates a flow diagram of a data call link tracking method according to an embodiment of the disclosure;
FIG. 2 schematically illustrates a detailed flow diagram of steps S1 and S2 according to an embodiment of the present disclosure;
FIG. 3 schematically illustrates an application flow diagram of a data call link tracking method according to an embodiment of the disclosure;
fig. 4 schematically shows a detailed flow diagram of step S3 and step S4 according to an embodiment of the present disclosure;
FIG. 5 schematically illustrates a data transfer process diagram of a data call link tracking system according to an embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of a data call link tracking apparatus according to an embodiment of the disclosure; and
fig. 7 schematically shows a block diagram of an electronic device according to an embodiment of the disclosure.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are some, but not all, embodiments of the present disclosure. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the disclosure, are within the scope of the disclosure.
Referring to fig. 1, an embodiment of the present disclosure provides a data call link tracking method, including:
s1, rasterizing the calling parameters called for a plurality of times under the same data request to obtain the rasterized calling parameters.
In some embodiments, the call parameters are obtained by a probe that supports multiple development languages: java, golang, PHP, NET, python, ruby, node. Js, C++, erlang/Elixir, rust, swift, etc., program data written in different languages may be collected and uniformly reported to the corresponding receivers. By the probe, cross-language, cross-cluster and cross-hardware architecture data link acquisition tracking can be realized. After the probe is injected into the system, the probe uniformly reports Span data (which is the minimum unit of data collection in the link) for analysis and processing. The Span data is processed as follows: an external request needs to be completed through the mutual calling of a plurality of internal services and data models, and a Span is created for each calling in the process and is used as a minimum record unit for constructing Trace data (link data which is used for recording processing information in the single request range and comprises data such as service calling, processing time length and the like), wherein each Span encapsulates the following states: operation name, timestamp, key-value (database index), parent node, etc., and assembling the calling chain data fragment data to form a complete request calling Trace link through parent node aggregation, and finally storing the collected data finally through message queue technology.
In an actual use scenario, a user firstly creates a receiver, then performs probe injection according to probe injection instructions of different languages provided in a probe SDK (Software Development Kit ) document, acquires monitoring data collected by a probe, and pushes the monitoring data to a distributed full-link monitoring platform for analysis and processing, and specifically comprises the following steps:
creating a receiver and acquiring a receiver address collector1;
injection probes are injected according to different language types (Java, golang, python, etc.);
and configuring a receiver collector1 for the injection probe, uniformly reporting Trace information to the receiver collector1 by the probe when each program runs, and finally pushing and reporting received data to a message queue by the receiver to be preliminarily stored as c1_data.
In some embodiments, prior to classifying the rasterized call parameters, the method further comprises:
and carrying out noise reduction treatment on the rasterized call parameters, thereby classifying the rasterized call parameters after noise reduction.
S2, classifying the rasterized call parameters to generate multi-dimension call parameters, and time sequence data and unstructured data of each dimension call parameter.
In some embodiments, classifying the rasterized call parameters to generate the multi-dimensional call parameters and the time series data and unstructured data of each of the multi-dimensional call parameters includes:
classifying the rasterized call parameters to generate multi-dimensional call parameters, wherein the multi-dimensional call parameters comprise at least two of clusters, namespaces, probes, instances, clients, services, containers, users and databases;
and dividing each dimension call parameter into time sequence data and unstructured data according to preset characteristic data.
As shown in fig. 2, in some embodiments, after acquiring data in real time, the data may be stored through a message queue, and collected into a raster computation engine through calling the message queue, where the real-time data is raster processed first, and complex data is preprocessed to generate raster data, so that subsequent computation and analysis are facilitated. The method comprises the steps of carrying out noise reduction processing on raster data through a filtering algorithm, automatically filtering useless data generated in data acquisition, generating noise reduction data, carrying out calculation and analysis on the noise reduction data through a classification algorithm, carrying out batch classification processing on complex redundant data, finally generating multi-dimensional data such as clusters, namespaces, probes, examples, clients, services, containers, users and databases, and time sequence data and unstructured data in each dimension, storing the time sequence data into a time sequence database, storing the unstructured data into an execution log, and carrying out distributed storage on the time sequence database and the execution log.
In an actual application scenario, in a raster calculation engine, data c1_data stored in an acquisition message queue is called in real time, and first, the c1_data is rasterized to generate a plurality of raster data such as c1_data1, c1_data2, c1_data3 and the like. And filtering the raster data to remove useless data, and finally processing the data through a classification algorithm to generate multi-dimensional multi-state data such as clusters, namespaces, probes, users, http, span and the like.
S3, constructing multi-dimensional time sequence data link data according to the time sequence data of each dimension calling parameter.
In some embodiments, the constructing multi-dimensional time sequence data link data according to the time sequence data of each dimension calling parameter includes:
classifying the time sequence data of each dimension respectively to obtain a plurality of time sequence data nodes;
according to at least one of time information and father node information in the time sequence data nodes, different time sequence data nodes are associated to form time sequence data node link data;
and respectively nesting time sequence data nodes with different dimensions in the time sequence data node links with different dimensions to obtain a plurality of multi-dimensional time sequence data areas and multi-dimensional time sequence data link data formed by the multi-dimensional time sequence data areas.
And S4, matching unstructured data of each dimension call parameter with the multi-dimension time sequence data link data to obtain multi-dimension call parameter link data.
In some embodiments, the matching the unstructured data of each dimension call parameter with the multi-dimension time sequence data link data to obtain multi-dimension call parameter link data includes:
determining dimension information of unstructured data to be matched;
extracting target time sequence data nodes corresponding to dimension information of unstructured data to be matched from a multi-dimensional time sequence data area of multi-dimensional time sequence data link data;
performing similarity matching on unstructured data to be matched and a target time sequence data node;
and supplementing unstructured data matched with the target time sequence data node into the target time sequence data to obtain multidimensional call parameter link data.
Referring to fig. 3, in the case where the data call link tracking method of the present embodiment is applied to the data call link tracking apparatus, steps S1 and S2 may be performed at the back end of the data call link tracking apparatus, and steps S3 and S4 may be performed at the front end of the data call link tracking apparatus.
In some embodiments, the link trace topology is assembled by:
supporting a plurality of data collectors and data format types; processing the assembly call data by adopting a grid technology: grid technology is well suited for helping programs in cloud-native scenarios to reliably pass data and provide powerful observability functionality when service calls are intricate. In actual use, the request data of the service at the front end and the back end of the program can be processed and assembled into a topological graph style through the calling relation between the service and the data model which are called mutually in the request by the grid drawing engine and the grid computing engine, and the topological graph style is stored in a time sequence database, a distributed big data engine and a distributed data storage, so that the target object can be conveniently and directly called.
Referring to fig. 4, in the raster drawing engine, data obtained by calculation and analysis of the raster calculation engine are obtained in real time, after the calculated data are obtained, the data are firstly identified and classified according to a classification and identification algorithm built in the raster drawing engine, and then the data are processed and preprocessed through a data processing algorithm built in the raster drawing engine, and finally the data are assembled and calculated by combining a raster topology algorithm to generate a link tracking topology graph.
In an actual application scenario, a multi-type full-link tracking topological graph can be generated through the following steps:
invoking calculation data stored in the time sequence database and the distributed database by the grid calculation engine in real time through the grid drawing engine;
according to the call path and state of the recorded user behavior among the systems, the call and the dependence among macroscopic and microscopic control clients, services and databases are combined with time sequence data and father node data to carry out link tracking on all data;
the cluster, the name space, the probe, the service, the data model and the single Trace link data which are calculated and analyzed in real time through the grid drawing engine and the grid computing engine are combined to carry out aggregation calculation, the drawing of the full link topological graph is completed through the built-in drawing engine, the bottom layer architecture of each system is analyzed, abnormal call data is found, and the system faults are decomposed and accurately positioned.
In this embodiment, the method further includes:
responding to the received link data query request, and extracting a target service scene in the link data query request;
determining dimension information corresponding to a target service scene based on a corresponding relation between the preset target service scene and the dimension information;
and displaying the link data consistent with the dimension information in the multi-dimension call parameter link data.
In an actual application scene, the collected indexes are supported to be output to the corresponding outputter through the receiver, and the data can be expanded according to service requirements, such as providing different data outlets; and responding to a link information query request aiming at a target service scene, determining target data corresponding to the target service scene, and supporting the display and call of a link topological graph by taking a system, a cluster, a name space, a probe, a service, a data model and a single Trace as analysis objects. And screening Trace data associated with the selected target service scene from all Trace link data according to analysis requirements of different dimensions, assembling and combining by adopting a grid technology according to the service data carried by the Trace link, and displaying a monitoring call topological graph.
The data call link tracking method of the embodiment is based on a cross-architecture link tracking technology, adopts a grid technology, analyzes and processes data in real time through a grid computing engine and a grid drawing engine, and is assembled into various types of visual call link topological graphs according to different service scenes, so that cross-language, cross-cluster and cross-hardware architecture data link tracking can be realized, end-to-end full-link observability construction is realized, the data call and the dependence condition of the whole system are monitored, and the call link data is subjected to multi-type query, statistical analysis and management control to realize one-stop service, container and user architecture system monitoring.
Referring to fig. 5, an embodiment of the present disclosure provides a data call link tracking system including a first data center (data center a) and two second data centers (data centers B), including:
the first data center acquires a first calling parameter through a probe;
the first data center performs rasterization processing on a first calling parameter which is called for multiple times under the same local data request to obtain the first rasterization calling parameter, classifies the first rasterization calling parameter, and generates a first multi-dimensional calling parameter, and first time sequence data and first unstructured data of each dimension calling parameter;
the second data center performs rasterization processing on a second calling parameter which is called for multiple times under the same local data request to obtain a second rasterization calling parameter, classifies the second rasterization calling parameter, generates second time sequence data and second unstructured data of the second multi-dimensional calling parameter and each dimension calling parameter, and sends the second time sequence data and the second unstructured data to the first data center respectively;
the first data center constructs multi-dimensional time sequence data link data according to the first time sequence data and the second time sequence data of each dimension calling parameter, and matches the first unstructured data and the second unstructured data of each dimension calling parameter with the multi-dimensional time sequence data link data to obtain multi-dimensional calling parameter link data.
The data call link tracking system disclosed by the invention not only can track and collect call relations and data among different clusters; in addition, the grid technology is adopted, the data is analyzed and processed in real time through a grid computing engine and a grid drawing engine, and various types of visual call link topological diagrams are assembled in an aggregation mode, so that one-stop service, container and user architecture system monitoring is realized.
The data call link tracking method of the disclosure realizes the cross-architecture full-link tracking of the probe-service-database, adopts a grid technology, supports multi-type data analysis and multi-type call topology chart visual display, supports custom service information, combines service attribute information with technical information, realizes the service translation of the technical data, displays full-link monitoring information with service attribute, provides complete full-sample full-link call link tracking data, comprises detailed call link access paths and performance, code stack, SQL statement and other component access information, and related service data indexes such as various request parameters, and provides detailed reference data for fault location.
Referring to fig. 6, an embodiment of the present disclosure provides a data call link tracking apparatus, including:
the rasterizing module 61 is configured to perform rasterizing processing on a call parameter that is called for multiple times under the same data request, so as to obtain a rasterized call parameter;
the classification module 62 is configured to classify the rasterized call parameters, and generate a multi-dimensional call parameter, and time sequence data and unstructured data of each dimension call parameter;
a construction module 63, configured to construct multi-dimensional time-series data link data according to the time-series data of each dimension call parameter;
and the matching module 64 is configured to match unstructured data of each dimension call parameter with the multi-dimension time sequence data link data to obtain multi-dimension call parameter link data.
The implementation process of the functions and roles of each unit in the above device is specifically shown in the implementation process of the corresponding steps in the above method, and will not be described herein again.
For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present invention. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
In the second embodiment described above, any of the rasterizing module 61, the classifying module 62, the constructing module 63, and the matching module 64 may be incorporated in one module to be implemented, or any of them may be split into a plurality of modules. Alternatively, at least some of the functionality of one or more of the modules may be combined with at least some of the functionality of other modules and implemented in one module. At least one of the rasterizing module 61, the classifying module 62, the building module 63, and the matching module 64 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable way of integrating or packaging the circuits, or in any one of or a suitable combination of any of three implementations of software, hardware, and firmware. Alternatively, at least one of the rasterizing module 61, the classifying module 62, the constructing module 63 and the matching module 64 may be at least partially implemented as a computer program module which, when executed, may perform the corresponding functions.
Referring to fig. 7, an electronic device provided by an embodiment of the present disclosure includes a processor 1110, a communication interface 1120, a memory 1130, and a communication bus 1140, where the processor 1110, the communication interface 1120, and the memory 1130 perform communication with each other through the communication bus 1140;
a memory 1130 for storing a computer program;
performing rasterization processing on calling parameters called for multiple times under the same data request to obtain rasterized calling parameters;
classifying the rasterized call parameters to generate multi-dimension call parameters, and time sequence data and unstructured data of each dimension call parameter;
according to the time sequence data of each dimension calling parameter, constructing multi-dimension time sequence data link data;
and matching unstructured data of each dimension call parameter with the multi-dimension time sequence data link data to obtain multi-dimension call parameter link data.
The communication bus 1140 may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The communication bus 1140 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface 1120 is used for communication between the electronic device and other devices described above.
The memory 1130 may include random access memory (Random Access Memory, simply RAM) or may include non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. Optionally, the memory 1130 may also be at least one storage device located remotely from the processor 1110.
The processor 1110 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
Embodiments of the present disclosure also provide a computer-readable storage medium. The computer-readable storage medium has stored thereon a computer program which, when executed by a processor, implements the data retrieval method as described above.
The computer-readable storage medium may be embodied in the apparatus/means described in the above embodiments; or may exist alone without being assembled into the apparatus/device. The computer-readable storage medium described above carries one or more programs, which when executed, implement a data retrieval method according to an embodiment of the present disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example, but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method for data call link tracking, the method comprising:
performing rasterization processing on calling parameters called for multiple times under the same data request to obtain rasterized calling parameters;
classifying the rasterized call parameters to generate multi-dimension call parameters, and time sequence data and unstructured data of each dimension call parameter;
according to the time sequence data of each dimension calling parameter, constructing multi-dimension time sequence data link data;
and matching unstructured data of each dimension call parameter with the multi-dimension time sequence data link data to obtain multi-dimension call parameter link data.
2. The method of claim 1, wherein classifying the rasterized call parameters to generate the multi-dimensional call parameters and the time series data and unstructured data for each of the multi-dimensional call parameters comprises:
classifying the rasterized call parameters to generate multi-dimensional call parameters, wherein the multi-dimensional call parameters comprise at least two of clusters, namespaces, probes, instances, clients, services, containers, users and databases;
and dividing each dimension call parameter into time sequence data and unstructured data according to preset characteristic data.
3. The method of claim 1, wherein constructing multi-dimensional time series data link data from the time series data of each dimension call parameter comprises:
classifying the time sequence data of each dimension respectively to obtain a plurality of time sequence data nodes;
according to at least one of time information and father node information in the time sequence data nodes, different time sequence data nodes are associated to form time sequence data node link data;
and respectively nesting time sequence data nodes with different dimensions in the time sequence data node links with different dimensions to obtain a plurality of multi-dimensional time sequence data areas and multi-dimensional time sequence data link data formed by the multi-dimensional time sequence data areas.
4. A method according to claim 3, wherein said matching unstructured data of each dimension call parameter with said multi-dimension time series data link data to obtain multi-dimension call parameter link data comprises:
determining dimension information of unstructured data to be matched;
extracting target time sequence data nodes corresponding to dimension information of unstructured data to be matched from a multi-dimensional time sequence data area of multi-dimensional time sequence data link data;
performing similarity matching on unstructured data to be matched and a target time sequence data node;
and supplementing unstructured data matched with the target time sequence data node into the target time sequence data to obtain multidimensional call parameter link data.
5. The method of claim 1, wherein prior to classifying the rasterized call parameters, the method further comprises:
and carrying out noise reduction treatment on the rasterized call parameters, thereby classifying the rasterized call parameters after noise reduction.
6. The method according to claim 1, wherein the method further comprises:
responding to the received link data query request, and extracting a target service scene in the link data query request;
determining dimension information corresponding to a target service scene based on a corresponding relation between the preset target service scene and the dimension information;
and displaying the link data consistent with the dimension information in the multi-dimension call parameter link data.
7. A data call link tracking apparatus, comprising:
the rasterization module is used for carrying out rasterization processing on calling parameters called for multiple times under the same data request to obtain rasterized calling parameters;
the classification module is used for classifying the rasterized calling parameters and generating multi-dimensional calling parameters, time sequence data and unstructured data of each dimension calling parameter;
the construction module is used for constructing multi-dimensional time sequence data link data according to the time sequence data of each dimension calling parameter;
and the matching module is used for matching unstructured data of each dimension calling parameter with the multi-dimension time sequence data link data to obtain multi-dimension calling parameter link data.
8. A data call link tracking system comprising a first data center and at least one second data center, comprising:
the first data center performs rasterization processing on a first calling parameter which is called for multiple times under the same local data request to obtain the first rasterization calling parameter, classifies the first rasterization calling parameter, and generates a first multi-dimensional calling parameter, and first time sequence data and first unstructured data of each dimension calling parameter;
the second data center performs rasterization processing on a second calling parameter which is called for multiple times under the same local data request to obtain a second rasterization calling parameter, classifies the second rasterization calling parameter, generates second time sequence data and second unstructured data of the second multi-dimensional calling parameter and each dimension calling parameter, and sends the second time sequence data and the second unstructured data to the first data center respectively;
the first data center constructs multi-dimensional time sequence data link data according to the first time sequence data and the second time sequence data of each dimension calling parameter, and matches the first unstructured data and the second unstructured data of each dimension calling parameter with the multi-dimensional time sequence data link data to obtain multi-dimensional calling parameter link data.
9. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
a memory for storing a computer program;
a processor configured to implement the data call link tracking method of any one of claims 1-6 when executing a program stored on a memory.
10. A computer readable storage medium having stored thereon a computer program, which when executed by a processor implements the data call link tracking method of any of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310337468.9A CN116049263B (en) | 2023-03-31 | 2023-03-31 | Data call link tracking method, device and system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310337468.9A CN116049263B (en) | 2023-03-31 | 2023-03-31 | Data call link tracking method, device and system, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116049263A true CN116049263A (en) | 2023-05-02 |
| CN116049263B CN116049263B (en) | 2023-06-02 |
Family
ID=86127654
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310337468.9A Active CN116049263B (en) | 2023-03-31 | 2023-03-31 | Data call link tracking method, device and system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116049263B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106790718A (en) * | 2017-03-16 | 2017-05-31 | 北京搜狐新媒体信息技术有限公司 | Service call link analysis method and system |
| CN112422335A (en) * | 2020-11-10 | 2021-02-26 | 普元信息技术股份有限公司 | Method, system, device and storage medium for realizing service link analysis based on micro-service architecture in technical middle station |
| CN113987074A (en) * | 2021-10-27 | 2022-01-28 | 中国工商银行股份有限公司 | Distributed service full-link monitoring method and device, electronic equipment and storage medium |
| CN114429265A (en) * | 2021-12-07 | 2022-05-03 | 北京比格大数据有限公司 | Enterprise portrait service construction method, device and equipment based on grid technology |
| CA3141329A1 (en) * | 2020-12-08 | 2022-06-08 | 10353744 Canada Ltd. | Request link tracking method and service request processing method |
| CN112486786B (en) * | 2020-11-12 | 2022-08-09 | 贝壳技术有限公司 | Calling link tracking method and device |
-
2023
- 2023-03-31 CN CN202310337468.9A patent/CN116049263B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106790718A (en) * | 2017-03-16 | 2017-05-31 | 北京搜狐新媒体信息技术有限公司 | Service call link analysis method and system |
| CN112422335A (en) * | 2020-11-10 | 2021-02-26 | 普元信息技术股份有限公司 | Method, system, device and storage medium for realizing service link analysis based on micro-service architecture in technical middle station |
| CN112486786B (en) * | 2020-11-12 | 2022-08-09 | 贝壳技术有限公司 | Calling link tracking method and device |
| CA3141329A1 (en) * | 2020-12-08 | 2022-06-08 | 10353744 Canada Ltd. | Request link tracking method and service request processing method |
| CN113987074A (en) * | 2021-10-27 | 2022-01-28 | 中国工商银行股份有限公司 | Distributed service full-link monitoring method and device, electronic equipment and storage medium |
| CN114429265A (en) * | 2021-12-07 | 2022-05-03 | 北京比格大数据有限公司 | Enterprise portrait service construction method, device and equipment based on grid technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116049263B (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Johanson et al. | Big automotive data: Leveraging large volumes of data for knowledge-driven product development | |
| CN101945009A (en) | Positioning method and device of power communication network fault based on case and pattern matching | |
| CN111309550A (en) | Data acquisition method, system, equipment and storage medium of application program | |
| CN111143097A (en) | GNSS positioning service-oriented fault management system and method | |
| CN113254254B (en) | Root cause positioning method and device of system fault, storage medium and electronic device | |
| CN110648080A (en) | Information physical system based on intelligent points and construction method thereof | |
| CN108427709B (en) | Multi-source mass data processing system and method | |
| CN111159897B (en) | Target optimization method and device based on system modeling application | |
| CN114328277A (en) | Software defect prediction and quality analysis method, device, equipment and medium | |
| CN116049263B (en) | Data call link tracking method, device and system, equipment and storage medium | |
| CN113360581A (en) | Data processing method, device and storage medium | |
| CN111125199B (en) | Database access method and device and electronic equipment | |
| CN111324583B (en) | Service log classification method and device | |
| CN115220131A (en) | Meteorological data quality inspection method and system | |
| CN112784025B (en) | Method and device for determining target event | |
| WO2017205299A1 (en) | Data driven invocation of real time wind market forecasting analytics | |
| CN117194668A (en) | Knowledge graph construction method, device, equipment and storage medium | |
| CN110928938B (en) | Interface middleware system | |
| CN114445162A (en) | Method for reversely tracing enterprise invoice system configuration | |
| CN110955521A (en) | Power distribution network wide area distributed type sinking calculation system and method based on big data | |
| CN117389908B (en) | Dependency analysis method, system and medium for interface automation test case | |
| CN111324434B (en) | Configuration method, device and execution system of computing task | |
| CN116227585B (en) | Parallel execution method and device for cluster tasks, computer equipment and storage medium | |
| CN114490811A (en) | Moonlet life cycle data management system | |
| de Brum Müller et al. | GPipe: Using Adaptive Directed Acyclic Graphs to Run Data and Feature Pipelines with on-the-fly Transformations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |