CN115794808A

CN115794808A - Data processing method, system and device and data management center

Info

Publication number: CN115794808A
Application number: CN202111064056.XA
Authority: CN
Inventors: 李辉
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-09-10
Filing date: 2021-09-10
Publication date: 2023-03-14

Abstract

The embodiment of the application provides a data processing method, a system, a device and a data management center, relates to the technical field of computers, and can be applied to the scenes of cloud technology, artificial intelligence, intelligent transportation, internet of vehicles and the like. The method comprises the following steps: sending a log query request to a data management center to indicate the data management center to return at least one debugging log and corresponding debugging sequence information; displaying the received at least one debugging log and corresponding debugging sequence information; the debugging sequence information is generated according to the routing information of each data processing system participating in processing the debugging log, and the routing information comprises the actual debugging sequence of each data processing system, the system identification and the log identification of the debugging log. The embodiment of the application can help a debugger to accurately and quickly locate links with problems in the debugging process, and the cost of manual troubleshooting is reduced.

Description

Data processing method, system and device and data management center

Technical Field

The present application relates to the field of computer technologies, and in particular, to a data processing method, system, device, and data management center.

Background

Generally, a plurality of systems cooperate to process data in a company enterprise, and the data may not reach a destination system due to problems of a source data reporting format, a network, a processing loop subsystem and the like.

For the problem industry, schemes such as real-time joint debugging, packet capturing and the like are generally provided to verify the data reporting connectivity, but the verification methods exist in links such as data reporting and link processing in an isolated manner and cannot cover the whole system, so that a user cannot be well helped to check links with problems in the upstream when some link has no data, and meanwhile, the user cannot be helped to check links with problems in the downstream when the downstream has no data. The problem of data existence in each link is very difficult to be checked.

Disclosure of Invention

Embodiments of the present application provide a data processing method, system, device, and data management center that overcome the above problems or at least partially solve the above problems.

In a first aspect, a data processing method is provided, and the method includes:

acquiring a message, wherein the message comprises a debugging log sent by a device or debugging data and routing information of a previous data processing system;

generating debugging data and routing information of the current data processing system according to the message;

sending the routing information of the current data processing system to a data management center, packaging the debugging data and the routing information of the current data processing system into a new message, and sending the new message to the next data processing system;

the routing information includes an actual debugging sequence of the corresponding data processing system, a system identifier and a log identifier of the debugging log.

In a second aspect, a data processing method is provided, which includes:

sending a log query request to a data management center to indicate the data management center to return at least one debugging log and corresponding debugging sequence information;

displaying the received at least one debugging log and corresponding debugging sequence information;

the debugging sequence information is generated according to routing information of each data processing system participating in processing the debugging log, and the routing information comprises an actual debugging sequence of each data processing system, a system identifier and a log identifier of the debugging log.

In a third aspect, a data processing method is provided, which includes:

receiving a log query request sent by a device, wherein the log query request comprises a unique identifier of the device;

searching and returning at least one debugging log corresponding to the unique identifier of the device and debugging sequence information;

In a fourth aspect, there is provided an apparatus comprising

The log query module is used for sending a log query request to a data management center so as to indicate the data management center to return at least one debugging log and corresponding debugging sequence information;

the log display module is used for displaying the received at least one debugging log and the corresponding debugging sequence information;

In a fifth aspect, a data processing system is provided, comprising:

the message acquisition module is used for acquiring a message, wherein the message comprises a debugging log sent by the device or debugging data and routing information of a previous data processing system;

the message processing module is used for generating debugging data and routing information of the current data processing system according to the message;

the message sending module is used for sending the routing information of the current data processing system to a data management center, packaging the debugging data and the routing information of the current data processing system into a new message and sending the new message to the next data processing system;

In a sixth aspect, a data management center is provided, including:

the device comprises a log query request receiving module, a log query request receiving module and a log query processing module, wherein the log query request receiving module is used for receiving a log query request sent by a device, and the log query request comprises a unique identifier of the device;

the log returning module is used for searching and returning at least one debugging log corresponding to the unique identifier of the device and debugging sequence information;

In a seventh aspect, an embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the steps of the method as provided in the first, second, or third aspect.

In an eighth aspect, embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the steps of the method as provided in the first, second or third aspect.

In a ninth aspect, embodiments of the present application provide a computer program, where the computer program includes computer instructions stored in a computer-readable storage medium, and when the computer instructions are read from the computer-readable storage medium by a processor of a computer device, the processor executes the computer instructions to make the computer device execute steps implementing the method as provided in the first, second or third aspect.

According to the data processing method, the data processing system, the data processing device and the data management center, the log query request is sent to the data management center, the data management center is indicated to return the at least one debugging log and the corresponding debugging sequence information, the received at least one debugging log and the corresponding debugging sequence information are displayed, a debugger does not need to manually check upstream links one by one, the debugger can be helped to quickly obtain the debugging log and the debugging sequence information, and a foundation is laid for a follow-up user to quickly find an abnormal data processing system from the debugging sequence information.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.

Fig. 1 is a schematic diagram of a system architecture of an application environment of a data processing method according to an embodiment of the present application;

fig. 2 is a schematic flow chart of a data processing method provided in an embodiment of the present application on a device side;

fig. 3 is a schematic interface diagram of a device provided in the embodiment of the present application for data debugging;

FIG. 4 is a schematic structural diagram of a topology provided by an embodiment of the present application;

FIG. 5 is a schematic structural diagram of a topology provided by another embodiment of the present application;

FIG. 6 is a schematic flow chart of a data processing method provided by an embodiment of the present application on a data processing system side;

FIG. 7 is a schematic diagram of a data pipeline for data transmission according to an embodiment of the present disclosure;

FIG. 8 is a schematic structural diagram of a topology provided in accordance with yet another embodiment of the present application;

FIG. 9 is a schematic flowchart of a data processing system and a data trace SDK participating in processing a debug log according to an embodiment of the present application;

fig. 10 is a schematic flowchart of a data processing method at a data management center side according to an embodiment of the present application;

fig. 11 is a schematic diagram of a system architecture of an application environment of a data processing method according to another embodiment of the present application;

fig. 12 is a schematic diagram illustrating interaction among services of a data management center according to an embodiment of the present application;

FIG. 13 is a schematic diagram of an apparatus according to an embodiment of the present disclosure;

FIG. 14 is a block diagram of a data processing system according to an embodiment of the present application;

fig. 15 is a schematic structural diagram of a data management center according to an embodiment of the present application;

fig. 16 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.

As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The application provides a data processing method, a system, a device and a data management center, and aims to solve the above technical problems in the prior art.

The following describes the technical solutions of the present application and how to solve the above technical problems with specific embodiments. These several specific embodiments may be combined with each other below, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.

Referring to fig. 1, which schematically illustrates a system architecture of an application environment of the data processing method according to the embodiment of the present application, as shown in the figure, the system architecture 100 may include an apparatus 101, one or more of

data processing systems

201, 202, and 203, and a data management center 301. It should be understood that the number of devices, data processing systems in FIG. 1 is merely illustrative. There may be any number of devices and data processing systems, as desired for an implementation.

The apparatus 101 may be various electronic devices having a display screen, including but not limited to desktop computers, portable computers, smart phones, tablet computers, smart voice interaction devices, smart appliances, in-vehicle apparatuses, and the like. The device of the embodiment of the application can generate the debugging log to be sent to the data processing system when data scheduling is needed, the data processing system carries out debugging according to the debugging log, and can also send the log query request to the data management center 301 when the debugging result needs to be checked so as to instruct the data management center to return at least one debugging log and corresponding debugging sequence information.

The data processing system and the data management center in the embodiment of the application can be independent physical servers, can also be a server cluster or distributed system formed by a plurality of physical servers, and can also be cloud servers for providing basic cloud computing services such as cloud service, cloud databases, cloud computing, cloud functions, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN (content delivery network), big data and artificial intelligence platforms and the like.

The data processing system is a system for performing data processing, such as a log server, a real-time ETL (process of data extraction (Extract), cleaning (Cleaning), transformation (Transform), loading (Load)) task system, a data warehouse, a real-time online analysis system, and the like. Different data processing systems typically exist in isolation at different stages of data processing. The data processing system can obtain debugging data and routing information of the previous data processing system; the debug data and the routing information of the previous data processing system are processed to generate debug data and routing information of the current data processing system, and the data management center 301 generates debug sequence information according to the routing information of each data processing system.

Referring to fig. 2, a schematic flow diagram of the data processing method of the present application on the device side is exemplarily shown, and as shown, the data processing method includes:

s101, sending a log query request to a data management center to indicate the data management center to return at least one debugging log and corresponding debugging sequence information;

in the embodiment of the application, a debugger can send a log query request to a data management center through a device, and the log query request may include a user identifier of the debugger and/or an equipment identifier of the device, and the like, where the user identifier is identification information representing an identity of the debugger, and the equipment identifier is identification information representing equipment used by the debugger.

According to the data management method and device, the user identification of the debugger can be registered in the data management center in advance, so that when the debugger sends a log query request to the data management center through the device for the first time, the data management center records the equipment identification of the debugger when determining that the user identification is legal, the debugger using the device subsequently can perform the data processing task of the data management method and device, and convenience of data management is improved.

And the data management center checks the user identification and/or the equipment identification in the received log query request, and if the user identification and/or the equipment identification are legal, the debugging log and the debugging sequence information which are debugged before or currently by the debugger or the debugging equipment are returned.

It can be understood that, during the running process of the application program, the running result is often inconsistent with the expected result, and a developer needs to troubleshoot bugs in the program and fix the bugs to achieve the expected running result. Generally, a developer needs to call a program to run a debug log to locate a problem to be solved according to the debug log. The permission of the debug log needs to relate to the cooperation of a plurality of data processing systems, so that different data processing systems have different debug sequences during scheduling, and the data management center of the embodiment of the application records the actual debug sequence and system identification of each data processing system participating in processing the debug log and returns the debug log as debug sequence information to the device together with the debug log for displaying. The system identification can be generally implemented in a manner of < type, instance ID >, which may refer to a functional type of the system, such as belonging to a log server, a data warehouse, a data analysis system, or the like, and the instance ID is a unique identification of an instance (instance).

In one embodiment, the data management center may return the debug logs to the device in reverse chronological order, facilitating the debugger to preferentially see recent debug logs. For example, there are 4 debug logs, and the debug time is 2021 year, 1 month, 1 day, 2 days, 3 days, and 4 days, then the data management center sequentially returns the debug logs of 4 days, 3 days, 2 days, and 1 day to the device, so that when the device displays according to the received sequence, the debug logs can be displayed according to the sequence of 4 days, 3 days, 2 days, and 1 day, and the debugger preferentially sees the debug logs of 4 days.

The debugging sequence information of the embodiment of the application is generated according to the routing information of each data processing system participating in processing the debugging log, the data processing system transmits the generated routing information to the data management center by participating in processing the debugging log, and the data management center generates the debugging sequence information according to the routing information of each debugging log. The routing information of the embodiment of the application includes the actual debugging sequence of each data processing system, the system identifier and the log identifier of the debugging log.

The debug order information may include the actual debug order of the data processing systems participating in processing the debug log, as well as system identification. For example, the debug order information may be (1) xy32123 → (2) xDRF2199 → (3) xITc2895, where (1), (2) and (3) represent the actual debug order of the respective data processing systems, and xy32123, xDRF2199 and xITc2895 represent the system identification of the respective data processing systems. The debugger can clearly know which data processing systems participate in the processing of a certain debug log and the debugging sequence of each data processing system through the debugging sequence information.

S102, displaying the received at least one debugging log and corresponding debugging sequence information;

for example, a debugger may perform data debugging and query of a debug log through a preset application program, please refer to fig. 3, which exemplarily shows an interface schematic diagram of the apparatus of the embodiment of the present application for performing data debugging, as shown in the figure, a left diagram in the figure shows function entries for data debugging and debugging log viewing, a user enters a debug log viewing page shown in the figure through the function entries for debugging log viewing, and the debug log viewing page shows a debug log returned by the data management center and corresponding debug sequence information, as shown in the debug log viewing page, each line of data includes debug time, a unique identifier (device identifier) of the apparatus, a log identifier of the debug log, and debug sequence information.

It should be noted that, in the data processing method according to the embodiment of the present application, the log query request is sent to the data management center, the data management center is instructed to return the at least one debug log and the corresponding debug order information, and the received at least one debug log and the corresponding debug order information are displayed, so that a debugger can be helped to quickly obtain the debug log and the debug order information, and a foundation is laid for a subsequent user to quickly find an abnormal data processing system from the debug order information.

On the basis of the foregoing embodiments, as an optional embodiment, at least one debug log and corresponding debug order information are displayed, and then the method further includes:

and responding to the operation instruction of the debugging sequence information, and obtaining and displaying a topological graph corresponding to the debugging sequence information.

When a user operates the adjustment sequence information corresponding to a certain debug log, the device further displays a topological graph corresponding to the debugging sequence information. The topological graph can be generated by the device according to the debugging sequence information, or the device can further generate a topological graph acquisition request in response to the operation instruction, optionally, the topological graph acquisition request can include a unique identifier of the debugging log, and the data management center generates and returns the topological graph according to the topological graph acquisition request.

The topological graph of the embodiment of the application comprises a first node, a first directed edge and the routing information of the first node; wherein the first node is to represent a data processing system participating in processing the debug log, and the first directed edge is to connect two nodes having an adjacent relationship in an actual debug order. For example, the data processing system 1 generates debugging data 1 according to the debugging log, the data processing system 2 processes the debugging data 1 to generate debugging data 2, the data processing system 3 processes the debugging data 2 to generate debugging data 3, and so on until the debugging is stopped or the debugging is finished, the data systems participating in processing the debugging log are displayed in a topological graph in a node mode, and meanwhile, the topological graph is connected with two nodes with a vector relation in an actual debugging sequence in a directed edge mode.

For example, for a certain debug log, 4 data processing systems are involved in processing the debug log, specifically, the data processing system 1 processes the debug log to generate debug data 1, the data processing system 2 processes the debug data 1 to generate debug data 2, the data processing system 3 processes the debug data 2 to generate debug data 3, the data processing system 4 processes the debug data 3 to generate debug data 4, and the debugging is completed.

Please refer to fig. 4, which exemplarily shows a schematic structural diagram of a topology map according to an embodiment of the present application, where the topology map includes 4 nodes 1 to 4, the nodes 1 to 4 respectively correspond to the data processing systems 1 to 4, a directed edge 1 is connected between the node 1 and the node 2, a directed edge 2 is connected between the node 2 and the node 3, and a directed edge 3 is connected between the node 3 and the node 4. Obviously, the user can very intuitively determine the data processing systems related to the debugging logs and the debugging sequence of each data processing system through the topological graph, and guarantee is provided for the data processing systems for troubleshooting by the user.

On the basis of the foregoing embodiments, as an optional embodiment, the topology graph of the embodiment of the present application further includes at least one of a second node and a second directed edge.

Wherein the second node is to represent a data processing system that is not engaged in processing the debug log as planned; the data processing system pointed by the second node not only comprises the debugging logs which appear in the plan but do not actually participate in the processing, but also comprises the data processing system which is not debugged according to the preset debugging.

For example, a debug log originally schedules 4 data processing systems to participate, the scheduled debug order is that the data processing system 1 processes the debug log to generate debug data 1, the data processing system 2 and the data processing system 3 process the debug data 1 to generate debug data 2 and 3, respectively, the data processing system 4 processes the debug data 2 and 3 to generate data 4, and the debug is completed. However, during actual debugging, the data processing system 3 does not generate the debug data 3 according to the debug data 1, and the corresponding data processing system 4 does not correctly generate the debug data 4, so that both the data processing systems 3 and 4 belong to the second node because they do not participate in processing the debug log according to the plan.

The second directed edge is used for connecting two nodes with adjacent relation in the target debugging sequence; the target debugging sequence is a debugging sequence with an actual debugging sequence different from a preset debugging sequence.

Referring to fig. 5, which exemplarily shows a schematic structural diagram of a topology diagram according to another embodiment of the present application, a debug log originally plans 5 data processing systems to participate, the 5 data processing systems respectively correspond to nodes 1 to 5 in the topology diagram, a scheduled debug sequence is that the data processing system 1 processes the debug log to generate debug data 1, the data processing system 2 and the data processing system 3 respectively process the debug data 1 to generate debug data 2 and 3, the data processing system 4 processes the data 2 and 3 to generate debug data 4, the data processing system 5 generates data 5 according to the debug data 4, and the debug is completed. However, during actual debugging, the data processing system 3 does not generate the debug data 3 according to the data 1 and sends the debug data 3 to the data processing system 4, and the corresponding data processing system 4 does not correctly generate the debug data 4, so that the data processing system 5 does not send the data, and therefore the data processing systems 3, 4, and 5 all belong to the second node because they do not participate in processing the debug log according to the plan. Then a second directed edge is connected between data processing systems 3 and 4 and data processing systems 4 and 5 are directly connected to the second directed edge. As is clear from fig. 5, in the embodiment of the present application, there are obvious differences in the presentation of the first node and the second node, and the first directed edge and the second directed edge, and it should be understood that the embodiment of the present application may also distinguish the first node and the second node, and the first directed edge and the second directed edge in other presentation manners, for example, from a color, a shape, a font, and the like, and the embodiment of the present application is not limited in particular.

In the embodiment of the application, the first node and the second node, and the first directed edge and the second directed edge are distinguished in the display mode, so that a debugger can intuitively see which data processing systems have problems in which step, taking fig. 5 as an example, since the display mode of the node 3, the node 4 and the node 5 is the display mode of the second node, the debugger can intuitively see that the data processing systems 3 to 5 have an unplanned state in the debugging process, further, since the directed edge between the node 1 and the node 3 is the first directed edge, and the directed edge between the node 3 and the node 4 is the second directed edge, it can be known that the data processing system 3 has successfully received the debugging data of the data processing system 1, but the debugging data 3 is not generated according to the debugging data 1 and sent to the data processing system 4, the directed edge between the peer node 2 and the node 4 is the first directed edge, and the directed edge between the node 4 and the node 5 is the second directed edge, it can be known that the data processing system 4 has received the debugging data 2 of the data processing system, but has not received the debugging data processing system 2, and the data processing system 4 has not generated because the debugging data 4, and the data processing system has not sent to the debugging data. Therefore, the topological graph of the embodiment of the application can obviously improve the efficiency of a debugger for knowing the debugging process, accurately and quickly locate links with problems, and reduce the cost of manual troubleshooting.

According to the embodiment of the application, the data processing system which is planned to participate in the debugging log and the preset debugging sequence can be preset for the debugging log, so that after the debugging is stopped or finished, the data processing system which is not processed according to the plan is determined according to the data processing system which is actually participated in processing the debugging log and the debugging sequence, and a user can conveniently and visually determine the abnormal condition of the debugging. Taking the fifth graph as an example, a debugger can intuitively determine that the data processing systems 3 to 5 have the abnormality according to the two second directed edges, so that troubleshooting on the data processing systems 3 to 5 can be quickly initiated.

On the basis of the above embodiments, as an optional embodiment, the routing information further includes a debug start time and a debug end time of the data processing system; the debugging start time is the time when the data processing system determines the actual debugging sequence; the debug end time is a time at which the data processing system generates debug data.

Furthermore, the embodiment of the application can also calculate the difference between the debugging start time and the debugging end time, namely the debugging time consumption of the data processing system, and correspondingly mark the debugging start time, the debugging end time and/or the debugging time consumption in the topological graph by recording the debugging start time, the debugging end time and/or the debugging time consumption and other information in the routing information, so that a debugger can be helped to more intuitively see the processing time lengths of different data processing systems, and a foundation is laid for discovering a failed data processing system.

Furthermore, the embodiment of the application can also record the difference value between the debugging ending time of the previous node and the debugging starting time of the next node in the two nodes in front and back in the routing information and the topological graph, so that a debugger can be helped to intuitively know the processing delay between the two data processing systems, and a foundation is laid for finding out other faults outside the data processing systems.

On the basis of the foregoing embodiments, as an optional embodiment, the topology graph of the embodiment of the present application further labels at least one of a debugging duration of the first node and a transmission duration of the first directed edge;

the debugging time length of the first node is the time interval of the corresponding debugging start time and the debugging end time of the data processing system; the transmission duration of the first directed edge is the time interval between the debugging end time of the data processing system corresponding to the head node of the directed edge and the debugging start time of the data processing system corresponding to the tail node of the directed edge.

For example, the debug start time of a certain data processing system (data processing system a) is 13:20, debugging end time is 13:24, if the debugging duration of the first node is 4 minutes, and another data processing system (data processing system b) receives the debugging data of the data processing system a, a first directed edge exists between the data processing system a and the data processing system b, and if the debugging start time of the data processing system b is 13:25, debugging end time is 13:26, then the first directed edge has a legacy time duration of 1 minute (time interval between 13.

According to the method and the device, the debugging time of the nodes and the transmission time of the directed edges are recorded in the topological graph, so that a debugger can clearly know the debugging time consumption and the transmission time consumption of the debugging data of the data processing system.

On the basis of the above embodiments, the embodiment of the present application may further preset the preset index value of the debugging duration of each first node and the transmission duration of each first directed edge, and perform a striking display on the nodes and directed edges whose durations exceed or do not exceed the preset index value, so as to provide data support for the whole-process performance optimization.

On the basis of the foregoing embodiments, as an optional embodiment, the sending a log query request to a data management center in the embodiment of the present application further includes:

and sending a debugging log to a data processing system, wherein the debugging log comprises a log identifier of the debugging log and an equipment identifier of the device.

The debugger can open the debugging mode on the device, then the device interface displays the equipment identification of the device, the device reports the equipment identification to the data management center, and when the device is communicated with the data management center, the device can display a message for representing the successful registration of the equipment identification and prompt the debugger to start debugging. When the device is not in communication with the data management center, the device may prompt the user to check the network.

The method and the device for processing the debugging log have the advantages that the debugging log is sent to the data processing system to indicate the data processing system to process the debugging log, and it is noted that the debugging log further comprises the log identification of the debugging log and the equipment identification of the device, so that a foundation is laid for marking generated data after the debugging log is processed by a subsequent data processing system to track a data processing flow.

Referring to fig. 6, a schematic flow chart of the data processing method in the embodiment of the present application on the data processing system side is exemplarily shown, and as shown in the drawing, the method includes:

s201, obtaining a message, wherein the message comprises a debugging log sent by the device or debugging data and routing information of a previous data processing system.

In this embodiment, the apparatus may send the debug log to the data processing system in the form of a message through the data channel, and similarly, the data processing system may send the generated debug data and the routing information to the next data processing system in the form of a message through the data channel, and the data pipe is only used to transmit data and does not perform a dump operation on the data.

The routing information of the embodiment of the application comprises an actual debugging sequence of a corresponding data processing system, a system identifier and a log identifier of the debugging log. According to the routing information, the three pieces of information are recorded, so that a foundation can be laid for updating of an actual debugging sequence and generation of debugging sequence information, and when a ring exists in a topological graph due to the fact that the actual debugging sequence is added, different debugging sequences of the same data processing node are identified.

The embodiment of the application can send the preset debugging sequence of the debugging log to the input data pipeline, so that after the data processing system generates the scheduling data and the routing information, the data processing system only needs to send the scheduling data and the routing information to the data pipeline, and the data pipeline executes data transmission, so that the data processing system only needs to be concentrated in data processing (including debugging).

It should be noted that the data communication formats between the data processing systems of the embodiments of the present application are various, for example, the device reporting to the logging service system may be http protocol, and the flow from the message queue to the ETL may be message queue specific consumption protocols, which are required by technologies and services, and therefore cannot be limited to define a general protocol. But some information still needs to be known among the nodes to perform special processing on the debug data to construct routing information to be reported to the data management center. Optionally, the embodiment of the present application sends the routing data through the extension header of these specific protocols.

Referring to fig. 7, which exemplarily shows a schematic diagram of data transmission performed by a data pipeline according to an embodiment of the present application, as shown in the drawing, there are 3 data processing systems, namely, a data processing system A1, a data processing system B2, and a data processing system C3, and a predicted storage order of debug logs with log identifiers of rz321 is pre-stored in the data pipeline: data processing system A1 → data processing system C3.

When the data processing system A1 generates debug data 1 and routing information 1 for the debug log, the debug data 1 and the routing information 1 are sent to the data pipeline, wherein the actual debug sequence 1 of the data processing system A1, the system identifier A1 and the log identifier rz321 of the debug log are recorded in the routing information 1, and the data pipeline determines that the debug data and the routing information need to be further sent to the data processing system C3 according to the predicted storage sequence and the system identifier A1.

For example, there are 3 data processing systems participating in processing of a certain debug log, which are respectively data processing systems 1 to 3, where the data processing system 1 processes the debug log to generate data 1, the data processing system 2 processes the debug data 1 to generate debug data 2, the data processing system 3 processes the debug data 2 to generate debug data 3, the data processing system 1 processes the debug data 3 to generate debug data 4, and the processing is finished. The corresponding topological graph is shown in fig. 8, the directed edges between the nodes 1 to 3 corresponding to the data processing systems 1 to 3 form a ring, the actual debugging sequence of the node 1 is two, which are 1 and 4, respectively, and a debugger can clearly see the data flow through the directed edges and the nodes in the topological graph.

The data channel of the embodiment of the application is not a fixed path, the path of the data channel can be changed along with the change of the topological structure of the system, and the data channel can be routed to different data processing systems according to different contents of the data.

S202, debugging data and routing information of the current data processing system are generated according to the message.

After receiving the debug data of the previous data processing system, the data processing system can process the debug data according to a preset data processing method to generate the debug data of the current data processing system, and can also generate the routing information of the current data processing system according to the routing information of the previous data processing system. The data included in the routing information indicates that the actual debug order of the current data processing system in the newly generated routing information is the value obtained by adding 1 to the actual debug order of the previous data processing system. For example, if the routing information of the previous data processing system is (1, xt01, rz123) and the system identifier of the current data processing system is xt21, the routing information generated by the current data processing system is (2, xt21, rz123), where 2 represents that the actual debugging order of the current data processing system is 2, and the system identifier is updated for the system identifier of the current data processing system, but the log identifier of the debug log is not changed, so that the trace consistency of the debug log can be ensured.

It is understood that the debugging data of the first data processing system in the embodiment of the present application is generated according to the debugging log output by the apparatus, and the debugging data received by the subsequent data processing systems is generated according to the debugging data of the previous data processing system.

S203, sending the routing information of the current data processing system to a data management center, packaging the debugging data and the routing information of the current data processing system into a new message, and sending the new message to the next data processing system.

After the data processing system generates the debugging data and the routing information, the routing information needs to be sent to the data management center for recording, and the debugging data and the routing information are sent to the next data processing system through the data pipeline, so that not only can the debugging task be continuously circulated and executed in different data processing systems be ensured, but also the execution process of the debugging task can be ensured to be recorded, a foundation is laid for follow-up inquiry and debugging records, more importantly, the data management center does not receive the debugging data, and each system does not need to open complete data access authority to troubleshooting personnel in problems, and data safety is ensured.

On the basis of the above embodiments, as an optional embodiment, the routing information further includes a debug start time and a debug end time of the current data processing system;

wherein the debugging start time is the time when the current data processing system determines the actual debugging sequence; the debugging end time is the time when the current data processing system generates debugging data.

The data processing system of the embodiment of the application takes the time for obtaining the actual debugging sequence of the current data processing system as the debugging starting time of the current data processing system, and takes the time for obtaining the debugging data of the current data processing system as the debugging finishing time of the current data processing system and sends the debugging finishing time to the data management center.

According to the embodiment of the application, the debugging starting time and the debugging ending time are recorded, two times are recorded in the debugging sequence information of each node in the topological graph for the data management center, a debugger can conveniently and visually see the time consumed by each data processing system during debugging, the data processing system with overlong processing time can be found in time, and whether a fault occurs is checked.

On the basis of the foregoing embodiments, as an optional embodiment, the data processing system may report the debug start time, the debug end time, the debug data, and the routing information to the data management center by calling a preset data tracing Software Development Kit (SDK).

The data tracking SDK of the embodiment of the application is responsible for actual work required by data tracking so as to reduce the cost of accessing the data node to the data tracking system. The data trace SDK may provide the above-described functionality for different data pipe types, such as http, kafka, grpc, and also allows the user to extend the supported types for a particular data pipe.

The method for generating the debugging data and the routing information of the current data processing system according to the message comprises the following steps:

s301, calling a preset SDK, and generating an actual debugging sequence of the current data processing system and the debugging starting time according to the debugging log in the message or the routing information of the previous data processing system;

s302, carrying out data processing on the debugging log in the message or the debugging data of the previous data processing system to obtain the debugging data of the current data processing system;

s303, calling the SDK to generate debugging end time, and acquiring a system identifier of the data processing system and a log identifier of the debugging log to generate the routing information of the current data processing system.

Referring to fig. 9, it exemplarily shows a schematic flowchart of the data processing system and the data trace SDK participating in processing the debug log according to the embodiment of the present application, as shown in fig. 9:

1. the data pipeline sends a message to the data processing system, wherein the message comprises debugging data a and routing information b;

2. the data processing system unpacks the message to obtain debugging data a and routing information b, wherein the debugging data a can be debugging data obtained after a last data processing system carries out debugging or can be debugging logs sent by a device, the routing information comprises an actual debugging sequence of the last data processing system, when the routing information is empty, the message is marked to be sent from the device, and the current data processing system is the first data processing system for processing the debugging logs;

3. the data processing system calls the data tracking SDK, and generates an actual debugging sequence of the current data processing system according to the actual debugging sequence of the last data processing system recorded in the routing information b and adds the actual debugging sequence of the current data processing system into the routing information d of the current data processing system;

4. generating debugging starting time of the previous data processing system according to the time for generating the actual debugging sequence of the current data processing system;

5. the data tracking SDK returns the debugging data a to the data processing system;

6. the data processing system carries out data processing related to debugging on the debugging data a to obtain debugging data c;

7. the data processing system packs the debugging data c and sends the debugging data c to the data tracking SDK;

8. the data tracking SDK generates debugging ending time according to the time of receiving the debugging data, records the debugging ending time into the routing information d, packs the debugging data c and the routing information d into a new message and sends the new message to the data processing system;

9. the data processing system sends the new message to the data pipeline;

10. and the data tracking SDK sends the debugging ending time to the data management center to end the process.

As can be seen from the above examples, the SDK of the examples of the present application has the following effects:

a. collecting logic and real time of debugging data;

b. extracting information transmitted from an upstream, and judging whether debugging data exists in the message according to the logic of the debugging data;

c. constructing debugging start time and debugging end time according to the real time, and reporting to a data management center;

d. constructed as a downstream output message.

It should be noted that all messages received by the data processing system from the data pipeline are not necessarily messages with debug data, and therefore, before step 1, it is further necessary to determine whether the messages have debug data, and if so, perform the subsequent steps.

Referring to fig. 10, a schematic flow chart of a data processing method at a data management center side according to an embodiment of the present application is exemplarily shown, and as shown in the drawing, the method includes:

s401, receiving a log query request sent by a device, wherein the log query request comprises a unique identifier of the device;

s402, at least one debugging log corresponding to the unique identifier of the device and debugging sequence information are searched and returned;

The data management center of the embodiment of the application can correspondingly store the unique identifier of the device initiating the debugging, the log identifier of the debugging log and the corresponding debugging sequence information for each debugging, so that when a log query request is received, the log identifier of the debugging log and the corresponding debugging sequence information are searched and returned according to the unique identifier of the device, and a debugger can conveniently and quickly find an abnormal data processing system.

On the basis of the foregoing embodiments, as an optional embodiment, the receiving device further includes:

receiving the routing information sent by the data processing system;

and updating debugging sequence information corresponding to the debugging log according to the routing information.

The data structure of the routing information may be expressed as < MessageID, nodeID, LT >, where the MessageID field indicates a log identifier of the debug log, the NodeID field indicates a system identifier of the corresponding data processing system, and LT is an abbreviation of Logic Time, which refers to an actual debug order of the corresponding data processing system in the embodiment of the present application. In an optional embodiment, the routing information may further include a system identifier of the previous data processing system, which is represented by a prenoidd.

The routing information records the actual debugging sequence and system identification of each data processing system related to the processing of the debugging log, so the debugging sequence information can be continuously updated according to the information. Specifically, the debugging order information in the embodiment of the present application may include the following three tables:

a message table, the data structure of which may be represented as < MessageID, deviceID, log >, wherein the DeviceID field represents the unique identification of the device and the Log field is used to record the debug Log.

The data structure of the routing node table may be denoted < MessageID, nodeID, LT >; if the routing information further includes the debug start time and the debug end time, it may be expressed as < MessageID, nodeID, LT, startTime, endTime >. When some entries in the routing node table do not have EndTime, there are 3 possibilities that this node may be an end node, an error occurs, or the route is not finished.

The data structure of the routing edge table may be denoted as < MessageID, startNodeID, endnoded, LT >; the StartNodeID and EndNodeID respectively represent two data processing systems with precedence relationship in debugging sequence.

For example, tables 1 to 3 are a message table, a routing node table, and a routing edge table, respectively, in the embodiment of the present application:

MessageID	DeviceID	Log
			Rz031	827439	Log1
Rz412	811321	Log2

table 1 message table of the embodiment of the present application

Table 2 a routing node table according to an embodiment of the present application

MessageID	StartNodeID	EndNodeID	LT
				Rz031	Xt00031	Xt00012		1
Rz031	Xt00012	Xt00044	2
				Rz031	Xt00044	Xt00031	3
Rz412	Xt00015	Xt00001						1

Table 3 routing edge table according to the embodiment of the present application

The data processing center may generate two pieces of debugging sequence information according to the log identifiers of the two debugging logs recorded in tables 1 to 3 and the actual debugging sequence of the corresponding data processing system, where the debugging sequence information 1 records the debugging sequence information of the debugging log Rz 031: (1) Xt00031 → (2) Xt00012 → (3) Xt00044 → (4) Xt00031; debug order information of the debug log Rz412 is recorded in the debug order information 2: (1) Xt00015 → (2) Xt00001.

When a new debug log is generated, a new row of data is created in the message table, and the data records the log identifier of the debug log, the unique identifier of the device for generating the debug log and the specific content of the debug log respectively. When a data processing system generates new debugging data and routing information, the data management center updates the routing node table and the routing edge table according to the routing information.

On the basis of the foregoing embodiments, as an optional embodiment, the data management center of the embodiment of the present application is further configured to:

receiving a topological graph acquisition request generated by the device in response to an operation instruction of the debugging sequence information, wherein the topological graph acquisition request comprises a unique identifier of the debugging log;

generating a topological graph according to debugging sequence information corresponding to the unique identifier of the debugging log, wherein the topological graph comprises a first node and a first directed edge;

returning the topology map to the device.

Specifically, since the actual debugging sequence and the system identifier of each data processing system are recorded in the debugging sequence information, the data processing system participating in processing the debugging log can be used as a first node, and a first directed edge is connected between two nodes having an adjacent relationship in the actual debugging sequence, thereby obtaining the topological graph. In addition, the embodiment of the present application may also generate the topology map according to the routing node table or the routing edge table, that is, the node recorded in the routing node table is used as the first node in the topology map, and the first directed edge is added to the corresponding node in the topology map according to the StartNodeID and the endnoded recorded in the routing edge table. In addition, an incoming edge or an outgoing edge of a node may also be constructed according to the sequence of the debug start time and the debug end time of the routing information, for example, if the debug end time in the routing information a is the closest to the debug start time in the routing information b, the outgoing edge of the node corresponding to the data processing system in the routing information a points to the node corresponding to the data processing system in the routing information b.

Taking the above tables 1 to 3 as examples, if the device wants to view the topology map of the debug log with the file identifier Rz031, the topology map acquisition request includes the unique identifier Rz031 of the debug log. The data management center queries all the line data with MessageID Rz031 in Table 2, then generates a topology map according to NodeID and LT data in each line data, or queries all the line data with MessageID Rz031 in Table 3, and generates a topology map according to StartNodeID and EndNodeID.

On the basis of the above embodiments, the embodiment of the present application may further track a call chain triggered by data by using opentracking related technology, and then construct a topological graph by using the tracked data, which may be implemented when the opentracking technology is accessed to all the existing system participants in a micro-service manner.

On the basis of the above embodiments, as an optional embodiment, the topology graph further includes a second node and a second directed edge;

wherein the second node is used for representing a data processing system which does not participate in processing the debug log according to a plan; the second directed edge is used for connecting two nodes with adjacent relation in the target debugging sequence; the target debugging sequence is a debugging sequence of which the actual debugging sequence is different from the preset debugging sequence.

The data management center of the embodiment of the application can also obtain data processing systems with scheduled participation aiming at the debugging logs and a preset debugging sequence of each data processing system, and establish a predicted routing node table and a predicted routing edge table. Whether the second node exists can be quickly inquired by comparing whether the NodeID corresponding to the same LT in the expected routing node table and the routing node table is the same or not.

Referring to FIG. 11, a system architecture diagram of another embodiment of the present application is illustratively shown, as shown, the system comprising a device, a data management center front end, a data management center, a data pipeline, and a data processing system.

The data processing method is that the debugging log is reported to a data pipeline from the device, the data pipeline sends the debugging log to a data processing system, the data processing system sends the generated debugging data and routing information to the data pipeline to be further sent to a downstream data processing system, and the data processing system also sends the routing information to a routing reporting service of a data management center. The routing reporting service of the embodiment of the application can receive the routing information of the data processing system, the service can be an independent service or a log server, the key point is the reliability of service and data storage, and the received log can be generally placed in a data pipeline.

The data management center records the routing information of each data processing system through the routing calculation service to generate debugging sequence information corresponding to the debugging logs, and stores the debugging sequence information into the database.

The data management center also provides a routing query service, provides specific information of a topological graph of a debugging log for the front end of the data management center or other external systems, and mainly provides two query interfaces:

1. inquiring a debugging log according to conditions such as equipment number, field value and the like;

2. inquiring a topological graph according to the log identification of the debugging log;

the general interactive mode is that the front end provides a device number, a log query request is sent to the interface service, the interface service reads information in the database, and a log list is returned to the front end. And then the front end selects a certain log according to the user, continuously inquires the routing network of the selected log to the interface service, the interface service inquires the top point and the side information stored in the database according to the MessageID, returns the information to the front end, and renders the information by the directed graph component of the front end to be presented to the user.

On the basis of the foregoing embodiments, as an optional embodiment, the management center front end in the embodiment of the present application requests an equipment identifier list from the data management center, sends the equipment identifier list to the apparatus, and the apparatus presents the equipment identifier list to a debugger, who selects an equipment identifier, or according to other conditions, such as some field values in a debug log, the apparatus constructs a log query request in response to information selected by the debugger, and sends the log query request to the data management center to request a corresponding debug log (which may be a debug log sent by the apparatus corresponding to the equipment identifier in the past or in real time, or a debug log matched with the field values, where field values for which the debug log can be searched are not specifically limited, and can be set according to an actual application scenario);

and the data management center returns the log list according to the reverse time sequence. The device responds to a debugger to select a certain debugging log, requests debugging sequence information of the debugging log and a corresponding topological graph from a data management center through the front end of the management center, wherein the topological graph comprises a first node and a first directed edge; the first node is used for representing a data processing system participating in processing the debugging log, the first directed edge is used for connecting two nodes having an adjacent relation in an actual debugging sequence and representing a time sequence relation of data passing through the processing nodes, and routing information such as debugging starting time, debugging ending time and the like can be marked on the first node.

Referring to fig. 12, which schematically illustrates an interaction between services of the data management center according to the embodiment of the present application, as shown in the figure,

1. the routing reporting service receives routing information (which is regarded as routing information in the figure) sent by the data processing system and sends the routing information to the routing calculation service;

2. the routing calculation service updates the NodeID in the routing node table according to the system identification of the data processing system recorded in the routing information;

3. the routing calculation service updates StartNodeID and EndNodeID in the routing edge table, and records the updated routing node table and routing edge table into a database;

4. when the query service receives a log query request or a topological graph acquisition request sent by the device, relevant information is acquired from the database to generate debugging sequence information or a topological graph returning device.

An embodiment of the present application provides an apparatus, as shown in fig. 13, the apparatus may include: the log query module 201 and the log display module 202 specifically:

a log query module 201, configured to send a log query request to a data management center to instruct the data management center to return at least one debug log and corresponding debug sequence information;

a log display module 202, configured to display the received at least one debug log and corresponding debugging sequence information;

The device provided in this embodiment of the present application specifically executes the process of the foregoing method embodiment, and for details, please refer to the content of the embodiment of the foregoing data processing method on the device side, which is not described herein again. According to the device provided by the embodiment of the application, the log query request is sent to the data management center, the data management center is instructed to return at least one debugging log and corresponding debugging sequence information, the received at least one debugging log and the corresponding debugging sequence information are displayed, a debugger can be helped to quickly obtain the debugging log and the debugging sequence information, and a foundation is laid for a follow-up user to quickly find an abnormal data processing system from the debugging sequence information.

An embodiment of the present application provides a data processing system, as shown in fig. 14, the data processing system may include: the message acquiring module 301, the message processing module 302, and the message sending module 303 specifically:

a message obtaining module 301, configured to obtain a message, where the message includes a debug log sent by a device or debug data and routing information of a previous data processing system;

a message processing module 302, configured to generate debug data and routing information of a current data processing system according to the message;

a message sending module 303, configured to send the routing information of the current data processing system to a data management center, package debug data and routing information of the current data processing system into a new message, and send the new message to a next data processing system;

The data processing system provided in the embodiment of the present application specifically executes the processes of the foregoing method embodiments, and please refer to the contents of the data processing method in the data processing system, which are not described herein again. According to the data processing system, the message is received, the new message is generated according to the message and is sent to the downstream, the debugging task can be guaranteed to be continuously circulated and executed in different data processing systems, the execution process of the debugging task can be guaranteed to be recorded, a foundation is laid for follow-up inquiry of the debugging record, and more importantly, the data management center does not receive the debugging data, and data safety is guaranteed.

An embodiment of the present application provides a data management center, as shown in fig. 15, the data management center may include: the log query request receiving module 401 and the log returning module 402 specifically:

a log query request receiving module 401, configured to receive a log query request sent by a device, where the log query request includes a unique identifier of the device;

a log returning module 402, configured to search and return at least one debug log and debug sequence information corresponding to the unique identifier of the apparatus;

The data management center provided in the embodiment of the present application specifically executes the processes of the method embodiments, and please refer to the content of the data processing method in the data management center, which is not described herein again. The data management center provided by the embodiment of the application correspondingly stores the unique identifier of the device initiating the debugging, the log identifier of the debugging log and the corresponding debugging sequence information for each debugging, so that when a log query request is received, the log identifier of the debugging log and the corresponding debugging sequence information are searched and returned according to the unique identifier of the device, and a debugger can conveniently and quickly find an abnormal data processing system.

An embodiment of the present application provides an electronic device, including: a memory and a processor; at least one program stored in the memory for execution by the processor, which when executed by the processor, implements: the method comprises the steps of sending a log query request to a data management center, indicating the data management center to return at least one debugging log and corresponding debugging sequence information, and displaying the received at least one debugging log and the corresponding debugging sequence information, so that a debugger can be helped to quickly obtain the debugging log and the debugging sequence information, and a foundation is laid for a subsequent user to quickly find an abnormal data processing system from the debugging sequence information.

In an alternative embodiment, there is provided an electronic apparatus, as shown in fig. 16, an electronic apparatus 4000 shown in fig. 16 including: a processor 4001 and a memory 4003. Processor 4001 is coupled to memory 4003, such as via bus 4002. Optionally, the electronic device 4000 may further comprise a transceiver 4004. It should be noted that the transceiver 4004 is not limited to one in practical applications, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.

The Processor 4001 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (field programmable Gate Array) or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 4001 may also be a combination that performs a computational function, including, for example, a combination of one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

Bus 4002 may include a path that carries information between the aforementioned components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 4002 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 16, but this is not intended to represent only one bus or type of bus.

The Memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic Disc storage medium or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to these.

The memory 4003 is used for storing application codes for implementing the present scheme, and execution is controlled by the processor 4001. Processor 4001 is configured to execute application code stored in memory 4003 to implement what is shown in the foregoing method embodiments.

The embodiment of the present application provides a computer readable storage medium, on which a computer program is stored, and when the computer program runs on a computer, the computer is enabled to execute the corresponding content in the foregoing method embodiment. Compared with the prior art, the method has the advantages that the log query request is sent to the data management center, the data management center is instructed to return at least one debugging log and the corresponding debugging sequence information, the received at least one debugging log and the corresponding debugging sequence information are displayed, a debugger can be helped to quickly obtain the debugging log and the debugging sequence information, and a foundation is laid for a subsequent user to quickly find an abnormal data processing system from the debugging sequence information.

The embodiment of the present application provides a computer program, which includes computer instructions stored in a computer-readable storage medium, and when a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executes the computer instructions, so that the computer device executes the contents as shown in the foregoing method embodiment. Compared with the prior art, the method has the advantages that the log query request is sent to the data management center, the data management center is instructed to return at least one debugging log and the corresponding debugging sequence information, the received at least one debugging log and the corresponding debugging sequence information are displayed, a debugger can be helped to quickly obtain the debugging log and the debugging sequence information, and a foundation is laid for a subsequent user to quickly find an abnormal data processing system from the debugging sequence information.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and may be performed in other orders unless otherwise indicated herein. Moreover, at least a portion of the steps in the flow chart of the figure may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

The foregoing is only a few embodiments of the present application and it should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present application, and that these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. A data processing method, comprising:

2. The data processing method of claim 1, wherein the routing information further comprises a debug start time and a debug end time of a current data processing system;

wherein the debug start time is a time at which the current data processing system determines an actual debug order; the debugging end time is the time when the current data processing system generates debugging data.

3. The data processing method of claim 1, wherein generating debug data and routing information for the current data processing system from the message comprises:

calling a preset software development kit, and generating an actual debugging sequence of the current data processing system and the debugging starting time according to the debugging log in the message or the routing information of the previous data processing system;

processing the debugging log in the message or the debugging data of the previous data processing system to obtain the debugging data of the current data processing system;

and calling the software development kit to generate debugging ending time, and acquiring a system identifier of the data processing system and a log identifier of the debugging log so as to generate the routing information of the current data processing system.

4. A data processing method, comprising:

5. The data processing method of claim 4, wherein the exposing at least one debug log and corresponding debug sequence information further comprises:

responding to an operation instruction of the debugging sequence information, and obtaining and displaying a topological graph corresponding to the debugging sequence information;

wherein the topological graph comprises a first node, a first directed edge and the routing information of the first node; the first node is to represent a data processing system that participates in processing the debug log, and the first directed edge is to connect two nodes having an adjacent relationship in an actual debug order.

6. The data processing method of claim 5, further comprising at least one of a second node and a second directed edge in the topology graph;

wherein the second node is to represent a data processing system not participating in processing the debug log as planned;

the second directed edge is used for connecting two nodes with adjacent relation in the target debugging sequence; the target debugging sequence is a debugging sequence of which the actual debugging sequence is different from the preset debugging sequence.

7. The data processing method of claim 5, wherein the routing information further includes a debug start time and a debug end time of the data processing system;

wherein the debug start time is a time at which the data processing system determines an actual debug order; the debug end time is a time when the data processing system generates debug data.

8. The data processing method according to claim 5, wherein at least one of a debugging duration of the first node and a transmission duration of the first directed edge is further labeled in the topology graph;

the debugging time length of the first node is the time interval of the corresponding debugging starting time and the debugging ending time of the data processing system; the transmission duration of the first directed edge is the time interval between the debugging ending time of the data processing system corresponding to the head node of the directed edge and the debugging starting time of the data processing system corresponding to the tail node of the directed edge.

9. A method of data processing, comprising:

10. The data processing method of claim 9, wherein the receiving device sends a log query request, and the method further comprises:

receiving the routing information sent by the data processing system;

updating debugging sequence information corresponding to the debugging log according to the routing information; the debugging sequence information comprises a routing node table and a routing edge table; the routing node table comprises the system identification and the log identification of the debugging log; the routing edge table comprises a log identification of the debugging log and system identifications of two data processing systems which have adjacent relation in an actual debugging sequence.

11. An apparatus, comprising:

the log display module is used for displaying the received at least one debugging log and corresponding debugging sequence information;

12. A data processing system, comprising:

a message acquisition module, configured to acquire a message, where the message includes a debug log sent by a device or debug data and routing information of a previous data processing system;

13. A data management center, comprising:

14. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the data processing method according to any of claims 1 to 10 are implemented when the computer program is executed by the processor.

15. A computer-readable storage medium, characterized in that it stores computer instructions which cause the computer to perform the steps of the data processing method according to any one of claims 1 to 10.