CN117675545A - Flow processing fault detection method and device, storage medium and electronic equipment - Google Patents

Abstract

The embodiment of the application provides a flow processing fault detection method, a device, a storage medium and an electronic device, wherein the flow processing fault detection method is used for a main device coupled with a special data processor, the special data processor is used for deploying a flow processing agent and a log collector, the flow processing agent is used for carrying out flow processing on a message sent by the main device, and the log collector is used for collecting logs in the flow processing and sending the logs to a shared storage which can be accessed by both the main device and the special data processor. Sending a target log acquisition request of target flow to the shared storage; receiving a target log returned by the shared storage in response to the target log acquisition request; sending the target log to a log parser; and receiving a fault detection result of the target flow returned by the log analyzer. When receiving the log acquisition request, the target log can be directly acquired from the shared storage, and then the fault detection result is analyzed, so that the analysis difficulty of a user is reduced, and the fault detection efficiency of flow processing is improved.

Description

Flow processing fault detection method and device, storage medium and electronic equipment

Technical Field

The embodiment of the application relates to the technical field of data processing, in particular to a flow processing fault detection method, a device, a storage medium and electronic equipment.

Background

In a cloud native acceleration scenario in which a dedicated Data Processor (DPU) accelerates, the DPU connects with a host device, transferring communications between various services that the host device would otherwise perform to a CPU for execution, i.e., traffic handling. After the flow processing is carried out by the DPU, the difficulty of troubleshooting and debugging the flow processing in the main equipment is increased. This is because the DPU is two devices with the master device. If the master device needs to obtain the log of the flow processing at the DPU side, it needs to establish a TCP connection with the DPU, and send the log through the TCP connection, which causes a large overhead. And the content of the log is complex, so that the analysis difficulty for common users is high, and the burden is increased. Therefore, the efficiency of flow processing failure detection is low.

Disclosure of Invention

In view of the foregoing, embodiments of the present application provide a method, an apparatus, a computer storage medium, and an electronic device for detecting a traffic processing failure, so as to at least partially solve the foregoing problems.

According to a first aspect of an embodiment of the present application, there is provided a traffic processing fault detection method for a host device coupled to a dedicated data processor, the dedicated data processor having a traffic processing agent and a log collector disposed therein, the traffic processing agent being configured to perform traffic processing on a message sent from the host device, the log collector being configured to collect logs in the traffic processing and send the collected logs to a shared storage accessible to both the host device and the dedicated data processor, the traffic processing fault detection method comprising: sending a target log acquisition request of target flow to the shared storage; receiving a target log returned by the shared storage in response to the target log acquisition request; sending the target log to a log parser; and receiving a fault detection result of the target flow returned by the log analyzer.

According to a second aspect of the embodiments of the present application, there is provided a flow processing fault detection method for a dedicated data processor coupled to a host device, the dedicated data processor having a flow processing agent and a log collector deployed therein, the flow processing fault detection method comprising: the message sent by the main equipment is subjected to flow processing through a flow processing agent; collecting logs in flow processing through a log collector; and sending the collected logs to a shared storage which can be accessed by both the main equipment and the special data processor through a log collector, so that the main equipment can acquire target logs of target flow from the shared storage and analyze the target logs, thereby obtaining a fault detection result of the target flow.

According to a third aspect of the embodiments of the present application, there is provided a flow processing fault detection apparatus, which is configured to be used in a host device coupled to a dedicated data processor, where the dedicated data processor is configured to configure a flow processing agent and a log collector, where the flow processing agent is configured to perform flow processing on a message sent by the host device, and the log collector is configured to collect a log in the flow processing and send the collected log to a shared storage accessible to both the host device and the dedicated data processor, where the flow processing fault detection apparatus includes a request module, a target log receiving module, a log sending module, and a fault detection result receiving module, where the request module is configured to send a target log obtaining request of a target flow to the shared storage; the target log receiving module is used for receiving and sharing a target log returned by the storage response to the target log obtaining request; the log sending module is used for sending the target log to the log analyzer; the fault detection result receiving module is used for receiving the fault detection result of the target flow returned by the log analyzer.

According to a fourth aspect of embodiments of the present application, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the method according to the first or second aspect.

According to a fifth aspect of embodiments of the present application, there is provided an electronic device, including a processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; the memory is configured to store at least one executable instruction that causes the processor to perform operations corresponding to the method according to the first or second aspect.

In the embodiment of the application, a log collector is arranged in the DPU, logs generated in the DPU flow processing are collected, and the logs are sent to a shared storage shared by the master device and the DPU. Once the log acquisition request is received, the target log can be directly acquired from the shared storage without establishing TCP connection, and the target log is sent to a special log analyzer to analyze the fault detection result. The processing burden of establishing TCP connection is reduced by directly acquiring the target log from the shared storage, and the analysis difficulty of a common user is reduced by a special log analyzer, so that the failure detection efficiency of flow processing is greatly improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following description will briefly introduce the drawings that are required to be used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present application, and other drawings may also be obtained according to these drawings for a person having ordinary skill in the art.

Fig. 1 is a schematic diagram of an application scenario of a traffic processing fault detection method according to an exemplary embodiment of the present application;

FIG. 2 is a flow chart of steps of a method of flow processing fault detection according to an exemplary embodiment of the present application;

FIG. 3 is a flow chart of steps of a method of flow processing fault detection according to another exemplary embodiment of the present application;

FIG. 4 is a flow chart of steps of a method of flow processing fault detection according to another exemplary embodiment of the present application;

FIG. 5 is a schematic diagram of a master device according to an exemplary embodiment of the present application;

FIG. 6 is a block diagram of a flow processing fault detection device according to an exemplary embodiment of the present application;

FIG. 7 is a block diagram of a flow processing fault detection device according to another exemplary embodiment of the present application;

fig. 8 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application.

Detailed Description

In order to better understand the technical solutions in the embodiments of the present application, the following descriptions will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the embodiments of the present application shall fall within the scope of protection of the embodiments of the present application.

Before explaining the flow processing fault detection method in the embodiment of the present application, an application scenario of the flow processing fault detection method is briefly described for easy understanding.

Referring to fig. 1, fig. 1 is a schematic view of an application scenario of a flow processing fault detection method according to an exemplary embodiment of the present application. The method for detecting the traffic processing failure in this embodiment may be used for a master device coupled to a dedicated data processor, where the master device may be a server or other device, where a plurality of service containers may be deployed in the master device, and one or more services are deployed in the service containers, which is not limited in this embodiment. The special data processor (Data Processing Unit, DPU) can be plugged into the host device through a PCIE slot to couple the special data processor to the host device. PCIE refers to PCI-Express, which is a bus and interface standard, and a special data processor and a main device can perform data transmission through PCIE.

A dedicated data processor is deployed with a traffic handling agent, which may be a service grid agent such as envoy, and a log collector for traffic handling messages sent by the master device. The Service Mesh (Service Mesh) is an infrastructure layer that handles Service communication specifically, and is composed of a Service Mesh agent and a set of task management components for each Service, where each Service communicates through the Service Mesh agent. In the prior art, the service grid agent is deployed in a service container on the main device, so that services in each service container can communicate, and in this embodiment, the service grid agent can be deployed in a special data processor by coupling the special data processor on the main device, so as to achieve the purpose of unloading the service grid agent to the special data processor, thereby reducing the communication data traffic processing of the main device and improving the network performance of the main device. Envoy is a component capable of realizing service grid agent, and can generate log data with specific format, namely Envoy log, in the process of processing traffic. The Envoy log may include various information such as a traffic processing time, a sending IP address, and a receiving IP address of a message sent by the master device, and each item of information is listed in a space as a division.

The log collector is used for collecting logs in the flow processing and sending the collected logs to a shared storage which is accessible by both the host device and the special data processor. The log collector may be fluent bit, which is an open-source log processor and repeater. The log generated by the traffic handling agent may be collected by a log collector and forwarded to a specified destination. The designated destination may be a shared memory accessible to both the host device and the dedicated data processor, or the shared memory may be disposed on the host device. The shared storage can comprise local shared storage and cloud shared storage, the cloud shared storage can be a cloud disk, the local shared storage can be an elastiscearch, the elastiscearch is a Lucene-based search server, a distributed full-text search engine is provided, and the logs are sent to the elastiscearch for storage, so that the real-time query of users can be facilitated. The logs are sent to the cloud disk for storage, so that the user can file conveniently.

It should be noted that, the method for detecting a traffic processing fault in this embodiment may also be used for a processing device that is not coupled to a dedicated data processor, and the processing device may access a shared storage, or the shared storage may be provided on the processing device, so that a user may apply the method in this embodiment to the processing device to query and parse a log generated by performing traffic processing in the dedicated data processor, which is not limited in this embodiment.

Referring to fig. 5, fig. 5 is a schematic structural diagram of a main apparatus according to an exemplary embodiment of the present application, and may also be a schematic structural diagram of a processing apparatus. The main device or the processing device may include a query module for querying the log in the shared storage, for example, the query module may include Kibana, which is an open source analysis and visualization platform for the elastsearch and may be used to search and view the data stored in the elastsearch. Thus, the user may query and display the log stored on the elastsearch through kibana, or the user may query the log through a log-in cloud.

The master device or the processing device may include a log parsing module, where the log parsing module is configured to parse a log, and the log parsing module may obtain a target log from the shared storage or the query module.

The main device or the processing device may further include a visualization module and a rendering module, where the visualization module includes an interface, for example, a target log input interface, a log obtaining interface, a log parsing output interface, and the embodiment is not limited thereto. The rendering module is used for rendering according to the fault detection result and the target details so as to display the fault detection result and the target details through the visualization module, for example, the fault detection result and the target details are displayed on the log analysis output interface.

Referring to fig. 2, a flow chart of steps of a method of flow processing fault detection according to an exemplary embodiment of the present application is shown.

The flow processing fault detection method of the embodiment comprises the following steps:

s202, sending a target log acquisition request of target flow to the shared storage.

For example, the target log obtaining request may include keyword information of the target flow, for example, processing time of the target flow, etc., and the master device may display a log obtaining interface, where the user may input the keyword information in the log obtaining interface, send the target log obtaining request of the target flow to the shared storage through the log obtaining interface, so that the target log is queried according to the keyword information in the target log obtaining request in the shared storage, and return the target log. The shared storage may be a cloud disk or an elastic search, which is not limited in this embodiment.

In an alternative implementation, the shared storage includes a local shared storage located at the host device or the third party device, and a cloud shared storage located at the cloud server; the log collector sends the log to the cloud sharing storage when a transmission queue for transmitting to the local sharing storage is congested; sending a target log acquisition request of target traffic to the shared storage, comprising: and sending a target log acquisition request of the target flow to the local shared storage so as to acquire the target log from the local shared storage, and if the acquisition fails, acquiring the target log from the cloud shared storage.

For example, the local shared storage at the host device or the third party device may be an elastic search, and the cloud shared storage at the cloud server may be a cloud disk. The log collector may transmit the collected log to at least one of the local shared storage and the cloud shared storage, which is not limited in this embodiment. The rules for the log collector to transfer logs to the local and cloud-shared stores may be as follows: the log may be preferentially transmitted to the local shared storage, and when the log collector is congested in a transmission queue transmitted to the local shared storage, the log may be transmitted to the cloud shared storage. Further, due to limited space of the local shared storage, an archiving period can be set, and the logs in the local shared storage are periodically archived in the cloud shared storage, so that the space of the local shared storage is released, and the latest logs are stored conveniently.

Further, when the target log obtaining request of the target flow is sent to the shared storage, the target log obtaining request of the target flow may be sent to the local shared storage first, so as to obtain the target log from the local shared storage, and if the target log obtaining request of the target flow fails in the local shared storage, the target log obtaining request of the target flow is sent to the cloud shared storage, and the target log is obtained from the cloud shared storage.

In this implementation, first, the log collector transmits the log to the local shared storage, and when the log collector congests in a transmission queue for transmitting to the local shared storage, the log is transmitted to the cloud shared storage. When the target log is acquired, a target log acquisition request of target flow is sent to the local shared storage, so that the target log is acquired from the local shared storage, the log can be acquired from the local shared storage preferentially by storing the log in the local shared storage preferentially, and when the acquisition fails, the target log is acquired from the cloud shared storage, the frequency of acquiring the log from the cloud shared storage is reduced, and therefore the log acquisition efficiency can be improved.

S204, receiving a target log returned by the shared storage in response to the target log obtaining request.

The target log obtaining request is sent to the shared storage, the target log is queried in the shared storage according to the keyword information in the target log obtaining request, the target log can be returned to the log obtaining interface, and the target log is displayed on the log obtaining interface.

S206, sending the target log to a log parser.

The method of sending the target log to the log parser may be, for example, by inputting the obtained target log on the interface, or setting a parse button in the log obtaining interface, and after the target log is displayed in the log obtaining interface, triggering a log parse request by the parse button, so as to send the target log carried by the log parse request to the log parser. Specifically, the target log carried in the log analysis request can be transmitted to the log analyzer as an interface parameter by calling a log transmission interface on the log analyzer. For example, the log transfer interface may be an update interface.

In an alternative implementation, sending the target log to the log parser includes: displaying a target log input interface; receiving an input target log in a target log input interface; the target log is sent to a log parser.

For example, a target log input interface may be displayed on the host device, where the target log input interface includes a log input area and a trigger button, and the user may input a log to be parsed, for example, a target log, to the log input area in the target log input interface, and click the trigger button, then send a log parsing request, call a log transmission interface through the log parsing request, and send the target log input in the target log input interface to the log parser through the log transmission interface.

In the implementation manner, through setting and displaying the target log input interface, a user can input a log to be analyzed into the target log input interface so as to send the target log to the log analyzer for analysis. By means of analyzing the input logs in the target log input interface, the analysis process and the process of obtaining the target logs can be decoupled, so that the method is not limited to the process of only analyzing the obtained target logs, and a user can conveniently and independently select the logs needing to be analyzed to analyze.

S208, receiving a fault detection result of the target flow returned by the log analyzer.

The log analysis output interface may be displayed on the host device, and the log analyzer analyzes the target log to obtain a log analysis result, that is, a fault detection result, and displays the fault detection result on the log analysis output interface, so that a user may directly observe the fault detection result.

In an alternative implementation, the fault detection result is generated by the log parser by: extracting an original log field and an original log field value corresponding to the original log field from a target log based on a log regular expression; substituting each original log field and the original log field value corresponding to the original log field into an analysis function to obtain a target log analysis field and a target log analysis field value corresponding to the target log analysis field as a fault detection result.

For example, a log regular expression may be determined according to a format of the target log, and an original log field value corresponding to the original log field may be extracted from the target log based on the log regular expression, for example, the original log field may be a processing START TIME [%start_time% ], and the corresponding original log field value is 2023-08-04t05:50:29.404z. Substituting each original log field and the original log field value corresponding to the original log field into an analysis function, and analyzing each original log field and the original log field value corresponding to the original log field into data of a structure type through the analysis function, so as to obtain a target log analysis field and a target log analysis field value corresponding to the target log analysis field, for example, the log analysis field can be timestamp, and the corresponding target log analysis field value is 2023-08-04T05:50:29.404Z, and the target log analysis field value is taken as a fault detection result.

In the implementation mode, an original log field and an original log field value corresponding to the original log field are extracted from a target log through a log regular expression; substituting each original log field and the original log field value corresponding to the original log field into an analysis function to obtain a target log analysis field and a target log analysis field value corresponding to the target log analysis field as a fault detection result, so that the obtained fault detection result is simple and clear to display the content of the target log, and a user can directly obtain the content of the target log conveniently to perform fault analysis.

In the embodiment of the application, a log collector is arranged in the DPU, logs generated in the DPU flow processing are collected, and the logs are sent to a shared storage shared by the master device and the DPU. Once the log acquisition request is received, the target log can be directly acquired from the shared storage without establishing TCP connection, and the target log is sent to a special log analyzer to analyze the fault detection result. The processing burden of establishing TCP connection is reduced by directly acquiring the target log from the shared storage, and the analysis difficulty of a common user is reduced by a special log analyzer, so that the failure detection efficiency of flow processing is greatly improved.

Referring to fig. 3, a flow chart illustrating steps of a flow processing fault detection method according to another exemplary embodiment of the present application is shown, and as shown, this embodiment mainly illustrates a subsequent implementation of step S208 of the above-described embodiment, which mainly includes the following steps:

s302, sending a target log acquisition request of target flow to the shared storage.

S304, receiving a target log returned by the shared storage in response to the target log obtaining request.

S306, the target log is sent to a log parser.

In an alternative implementation, sending the target log to the log parser includes: sending the target log to a parser controller; acquiring a log analysis field detail comparison table from a flow processing agent through a parser controller, and storing the log analysis field detail comparison table in an intermediate storage; the target log is sent to a log parser.

For example, referring to fig. 5, the log parsing module may include a log parser, a parser controller, and an intermediate storage, and may send a target log to the parser controller through a target log input interface, the parser controller receives a log parsing request, obtains a log parsing field detail comparison table from a home network of the flow processing agent, where the log parsing field detail comparison table includes log parsing fields and corresponding field meaning interpretation details, and stores the log parsing field detail comparison table to the intermediate storage. Or, preprocessing the log analysis field detail comparison table to obtain a processing result, for example, the preprocessing mode may be reverse-serialization processing, the processing result is log analysis field detail comparison table data of a structure type, and then the processing result is stored in an intermediate storage. And then the target log is sent to a log analyzer through the analyzer controller.

In the implementation manner, the target log is sent to the parser controller, and the parser controller obtains the log parsing field detail comparison table from the flow processing agent and stores the log parsing field detail comparison table in the intermediate storage, so that the log parsing field detail comparison table can be quickly queried later, and the query efficiency of the log parsing field details is improved.

S308, receiving a fault detection result of the target flow returned by the log analyzer.

It should be noted that, the steps S302 to S308 can be implemented with reference to the specific embodiments of the steps S202 to S208, and are not described herein.

S310, displaying a fault detection result.

For example, the fault detection result may be rendered to a log parsing output interface on the host device by invoking a template rendering engine to display the fault detection result. For example, the stencil rendering engine may be a thymeleaf.

S312, in response to receiving the detail query trigger of the target log analysis field in the fault detection result, sending a detail query request to the analyzer controller.

Illustratively, after the fault detection result is displayed on the log resolution output interface on the host device, the user may trigger a detail query for the target log resolution field by clicking on the target log resolution field, e.g., timestamp, in the fault detection result, and send a detail query request to the resolver controller.

S314, responding to the detail query request through the analyzer controller, and querying a log analysis field detail comparison table in the intermediate storage to obtain target details corresponding to the target log analysis field.

Illustratively, the parser controller may call a javascript function, through which the/docinfo interface is called to obtain a log parsing field detail comparison table in the intermediate storage, and may query, from the log parsing field detail comparison table, target details corresponding to the target log parsing field according to the id of the field target log parsing field, where the target details may be meaning interpretations for the log parsing field.

S316, displaying the target details.

Illustratively, the target details may be rendered by a template rendering engine on a log parsing output interface on the host device, which may include two regions: after the fault detection result display area displays the fault detection result, the user can click the target log analysis field in the fault detection result display area to trigger the detail query of the target log analysis field, and then the target details corresponding to the target log analysis field can be displayed in the detail display area, so that the user can directly observe the fault detection result and the target details corresponding to the target log analysis field in the fault detection result.

In this embodiment, after receiving the fault detection result of the target flow returned by the log analyzer, the fault detection result is displayed, if a log analysis field unclear to the user exists in the fault detection result, the detail query request of the target analysis field may be further sent to the analyzer controller, so that the analyzer controller queries the log analysis field detail comparison table in the intermediate storage to obtain and display the target details corresponding to the target log analysis field, so that the user can timely obtain the detail information of the target log analysis field, so that the user can know the content of the fault detection result, and the efficiency of the user for performing the flow processing fault investigation is improved.

Referring to fig. 4, a flowchart of steps of a flow processing fault detection method according to another exemplary embodiment of the present application is shown. The flow processing fault detection method of the present embodiment is used for a dedicated data processor coupled to a master device, where a flow processing agent and a log collector are disposed.

The flow processing fault detection method of the embodiment comprises the following steps:

s402, the message sent by the main equipment is subjected to flow processing through the flow processing agent.

Illustratively, the traffic handling agent may be a service grid agent, such as envoy. The Service Mesh (Service Mesh) is an infrastructure layer that handles Service communication specifically, and is composed of a Service Mesh agent and a set of task management components for each Service, where each Service communicates through the Service Mesh agent. Envoy is a component capable of realizing service grid agent, and can generate log data with specific format, namely Envoy log, in the process of processing traffic. The Envoy log may include various information such as a traffic processing time, a sending IP address, and a receiving IP address of a message sent by the master device, and each item of information is listed in a space as a division.

S404, collecting logs in flow processing through a log collector.

Illustratively, the log collector may be fluent bit, which is an open source log processor and forwarder. The log collector can collect logs generated by the flow processing agent in real time in the flow processing process, and can be configured according to preset filtering conditions, so that the logs generated by the flow processing agent can be filtered in the collecting process, for example, incomplete logs and the like can be filtered.

S406, the collected logs are sent to a shared storage which can be accessed by both the main equipment and the special data processor through a log collector, so that the main equipment can acquire target logs of target flow from the shared storage, and analyze the target logs, thereby obtaining fault detection results of the target flow.

For example, the log collector may be configured according to a preset storage destination of the shared storage, and after collecting the log generated by the flow processing agent, the log collector may send the log to the shared storage in real time according to the storage destination.

In this embodiment, the dedicated data processor is coupled to the master device, and the flow processing agent is deployed in the dedicated data processor, so that the flow processing agent in the dedicated data processor is used to perform flow processing on the message sent by the master device, thereby reducing data flow processing of the master device and improving network performance of the master device. And moreover, the log collector is deployed, so that the log generated by the flow processing agent can be collected and sent to the shared storage, and therefore, when the fault investigation and debugging of flow processing are required, a user can conveniently and directly acquire the log from the shared storage to inquire and analyze, and the efficiency of flow processing fault detection is improved.

Referring to fig. 6, a block diagram of a flow processing fault detection device according to an exemplary embodiment of the present application is shown.

The flow processing fault detection device of the embodiment is used for a main device coupled with a special data processor, the special data processor is provided with a flow processing agent and a log collector, the flow processing agent is used for processing flow of messages sent by the main device, the log collector is used for collecting logs in the flow processing and sending the collected logs to a shared storage which can be accessed by the main device and the special data processor,

the flow processing fault detection device of the present embodiment includes a request module 602, a target log receiving module 604, a log transmitting module 606, and a fault detection result receiving module 608.

The request module 602 is configured to send a target log obtaining request of a target flow to the shared storage; the target log receiving module 604 is configured to receive a target log returned by the shared storage in response to the target log obtaining request; the log sending module 606 is configured to send the target log to a log parser; the fault detection result receiving module 608 is configured to receive a fault detection result of the target flow returned by the log parser.

In an alternative implementation, the fault detection result is generated by the log parser by: extracting an original log field and an original log field value corresponding to the original log field from a target log based on a log regular expression; substituting each original log field and the original log field value corresponding to the original log field into an analysis function to obtain a target log analysis field and a target log analysis field value corresponding to the target log analysis field as a fault detection result.

In an alternative implementation, the log sending module 606 is further configured to: sending the target log to a parser controller; acquiring a log analysis field detail comparison table from a flow processing agent through a parser controller, and storing the log analysis field detail comparison table in an intermediate storage; the target log is sent to a log parser.

In an optional implementation manner, the flow processing fault detection device further includes a detail parsing module, configured to: displaying a fault detection result; responding to receiving detail query trigger of a target log analysis field in a fault detection result, and sending a detail query request to a resolver controller; responding to the detail query request through the analyzer controller, and querying a log analysis field detail comparison table in the intermediate storage to obtain target details corresponding to the target log analysis field; and displaying the object details.

In an alternative implementation, the log sending module 606 is further configured to: displaying a target log input interface; receiving an input target log in a target log input interface; the target log is sent to a log parser.

In an alternative implementation, the shared storage includes a local shared storage located at the host device or the third party device, and a cloud shared storage located at the cloud server; the log collector sends the log to the cloud sharing storage when a transmission queue for transmitting to the local sharing storage is congested; the request module 602 is further configured to: and sending a target log acquisition request of the target flow to the local shared storage so as to acquire the target log from the local shared storage, and if the acquisition fails, acquiring the target log from the cloud shared storage.

The flow processing fault detection device of the present embodiment is configured to implement the corresponding flow processing fault detection method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein. In addition, the functional implementation of each module in the flow processing fault detection device of the present embodiment may refer to the description of the corresponding portion in the foregoing method embodiment, which is not repeated herein.

Referring to fig. 7, a block diagram of a flow processing fault detection device according to another exemplary embodiment of the present application is shown.

The flow processing fault detection device of the present embodiment is used for a dedicated data processor coupled to the master device, where a flow processing agent and a log collector are disposed.

The flow processing fault detection device of the present embodiment includes a proxy module 702, a log collection module 704, and a log forwarding module 706.

The proxy module 702 is configured to perform flow processing on a message sent by the master device through a flow processing proxy; the log collecting module 704 is configured to collect logs in the flow process through a log collector; the log forwarding module 706 is configured to send, through a log collector, the collected log to a shared storage that is accessible to both the host device and the dedicated data processor, so that the host device obtains a target log of the target traffic from the shared storage, and parses the target log, thereby obtaining a fault detection result of the target traffic.

The flow processing fault detection device of the present embodiment is configured to implement the corresponding flow processing fault detection method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein. In addition, the functional implementation of each module in the flow processing fault detection device of the present embodiment may refer to the description of the corresponding portion in the foregoing method embodiment, which is not repeated herein.

Referring to fig. 8, a schematic structural diagram of an electronic device according to an exemplary embodiment of the present application is shown. The specific embodiments of the present invention are not limited to specific implementations of electronic devices.

As shown in fig. 8, the electronic device may include: a processor 801, a memory 803, a communication bus 804, and a communication interface (Communications Interface) 805.

Wherein: the processor 801, the memory 803, and the communication interface 805 communicate with each other via the communication bus 804.

A communication interface 805 for communicating with other electronic devices or servers.

The processor 801 is configured to execute the program 802, and may specifically execute the steps of the traffic processing fault detection method in any of the foregoing embodiments.

In particular, program 802 may include program code including computer operating instructions.

The processor 801 may be a central processing unit CPU or a specific integrated circuit ASIC (Application Specific Integrated Circuit) or one or more integrated circuits configured to implement embodiments of the present application. The one or more processors comprised by the smart device may be the same type of processor, such as one or more CPUs; but may also be different types of processors such as one or more CPUs and one or more ASICs.

A memory 803 for storing the program 802. The memory 803 may comprise a high-speed RAM memory or may further comprise a non-volatile memory (non-volatile memory), such as at least one magnetic disk memory.

The program 802 is specifically operable to cause the processor 801 to execute to implement the steps of any of the flow handling failure detection methods described in the embodiments. The specific implementation of each step in the procedure 802 may refer to the corresponding descriptions in the steps and units executed by any one of the foregoing flow processing fault detection methods, which are not described herein. It will be apparent to those skilled in the art that for convenience and brevity of description, the specific operation of the apparatus and modules described above may be described with reference to corresponding processes in the foregoing method embodiments.

The present application also provides a computer storage medium having stored thereon a computer program which, when executed by a processor, implements a flow processing failure detection method as described in any of the above-described method embodiments.

Embodiments of the present application also provide a computer program product, including computer instructions that instruct a computing device to perform operations corresponding to the flow processing fault detection method described in any one of the above-described method embodiments.

It should be noted that, according to implementation requirements, each component/step described in the embodiments of the present application may be split into more components/steps, and two or more components/steps or part of operations of the components/steps may be combined into new components/steps, so as to achieve the purposes of the embodiments of the present application.

The above-described methods according to embodiments of the present application may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and to be stored in a local recording medium downloaded through a network, so that the methods described herein may be stored on such software processes on a recording medium using a general purpose computer, special purpose processor, or programmable or special purpose hardware such as an ASIC or FPGA. It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by a computer, processor, or hardware, performs the methods described herein. Furthermore, when a general purpose computer accesses code for implementing the methods illustrated herein, execution of the code converts the general purpose computer into a special purpose computer for performing the methods illustrated herein.

Those of ordinary skill in the art will appreciate that the elements and method steps of the examples described in connection with the embodiments disclosed herein can be implemented as electronic hardware, or as a combination of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.

The above embodiments are only for illustrating the embodiments of the present application, but not for limiting the embodiments of the present application, and various changes and modifications can be made by one skilled in the relevant art without departing from the spirit and scope of the embodiments of the present application, so that all equivalent technical solutions also fall within the scope of the embodiments of the present application, and the scope of the embodiments of the present application should be defined by the claims.

Claims (10)

1. A method of traffic handling fault detection for a host device coupled to a dedicated data processor, the dedicated data processor having disposed therein a traffic handling agent for traffic handling messages sent by the host device and a log collector for collecting logs in the traffic handling and sending the collected logs to a shared store accessible to both the host device and the dedicated data processor, the method comprising:

sending a target log acquisition request of target flow to the shared storage;

receiving the target log returned by the shared storage in response to the target log acquisition request;

sending the target log to a log parser;

and receiving a fault detection result of the target flow returned by the log analyzer.

2. The flow processing failure detection method according to claim 1, wherein the failure detection result is generated by the log parser by:

extracting an original log field and an original log field value corresponding to the original log field from the target log based on a log regular expression;

substituting each original log field and the original log field value corresponding to the original log field into an analysis function to obtain a target log analysis field and a target log analysis field value corresponding to the target log analysis field as the fault detection result.

3. The flow processing fault detection method of claim 2, wherein the sending the target log to a log parser comprises:

sending the target log to a resolver controller;

acquiring a log analysis field detail comparison table from the flow processing agent through the analyzer controller, and storing the log analysis field detail comparison table in an intermediate storage;

and sending the target log to the log parser.

4. The flow processing fault detection method according to claim 3, wherein after receiving a fault detection result of the target flow returned by the log parser, the flow processing fault detection method further comprises:

displaying the fault detection result;

responding to receiving a detail query trigger for a target log analysis field in the fault detection result, and sending a detail query request to the analyzer controller;

responding to the detail query request by the analyzer controller, and querying a log analysis field detail comparison table in the intermediate storage to obtain target details corresponding to the target log analysis field;

and displaying the target details.

5. The flow processing fault detection method of claim 1, wherein the sending the target log to a log parser comprises:

displaying a target log input interface;

receiving the input target log at the target log input interface;

and sending the target log to the log parser.

6. The traffic handling failure detection method according to claim 1, wherein the shared storage includes a local shared storage located at the master device or a third party device, and a cloud shared storage located at the cloud server; the log collector sends the log to the cloud sharing storage when a transmission queue for transmitting to the local sharing storage is congested;

the sending the target log obtaining request of the target flow to the shared storage includes: and sending a target log acquisition request of target flow to the local shared storage so as to acquire the target log from the local shared storage, and if acquisition fails, acquiring the target log from the cloud shared storage.

7. A method of traffic handling fault detection, characterized by a dedicated data processor for coupling with a host device, the dedicated data processor having a traffic handling agent and a log collector deployed, the method comprising:

the message sent by the main equipment is subjected to flow processing through the flow processing agent;

collecting, by the log collector, logs in the flow processing;

and sending the collected logs to a shared storage which can be accessed by both the master device and the special data processor through the log collector, so that the master device can acquire target logs of target flow from the shared storage and analyze the target logs, thereby obtaining a fault detection result of the target flow.

8. A traffic handling fault detection apparatus for a host device coupled to a dedicated data processor, the dedicated data processor having disposed therein a traffic handling agent for traffic handling messages sent by the host device and a log collector for collecting logs in the traffic handling and sending the collected logs to a shared store accessible to both the host device and the dedicated data processor, the traffic handling fault detection apparatus comprising:

the request module is used for sending a target log acquisition request of target flow to the shared storage;

the target log receiving module is used for receiving the target log returned by the shared storage in response to the target log obtaining request;

the log sending module is used for sending the target log to a log analyzer;

and the fault detection result receiving module is used for receiving the fault detection result of the target flow returned by the log analyzer.

9. An electronic device, comprising: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;

the memory is configured to store at least one executable instruction that causes the processor to perform operations corresponding to the method of any one of claims 1-7.

10. A computer storage medium having stored thereon a computer program, which when executed by a processor performs the method according to any of claims 1-7.