CN117478562A

CN117478562A - IPC link forwarding performance monitoring method and device

Info

Publication number: CN117478562A
Application number: CN202311204969.6A
Authority: CN
Inventors: 程基
Original assignee: New H3C Technologies Co Ltd
Current assignee: New H3C Technologies Co Ltd
Priority date: 2023-09-18
Filing date: 2023-09-18
Publication date: 2024-01-30

Abstract

The present invention relates to the field of network communications technologies, and in particular, to a method and an apparatus for monitoring forwarding performance of an IPC link. The method is applied to a first node of a distributed device, the method comprising: receiving an IPC message sent by a second node, wherein the IPC message carries a sending time stamp of the IPC message sent by the second node; determining a receiving time stamp for receiving the IPC message; based on the sending timestamp and the receiving timestamp, determining an IPC link transmission delay between the first node and the second node; if the IPC link transmission time delay is monitored to be larger than a preset threshold value, recording an abnormal transmission event corresponding to the IPC message.

Description

IPC link forwarding performance monitoring method and device

Technical Field

The present invention relates to the field of network communications technologies, and in particular, to a method and an apparatus for monitoring forwarding performance of an IPC link.

Background

IPC (Inter-Process Communication ) is a reliable communication mechanism between different nodes, typically one node corresponds to one CPU, and for a distributed device-independent operation mode, one distributed device is configured with multiple boards, each board has one CPU, and even one board has multiple CPUs (such as a service CPU, an OAM CPU, etc.), so the device is a set of multiple nodes.

And the link between any two nodes is of a full-connection structure. Meanwhile, each node can allocate a local unique local channel number to the upper layer application module to identify the link, and one IPC channel can have a plurality of links, and each link has a corresponding state. See fig. 1 for relationships between nodes, links and lanes.

In practical application, various uncertain factors of a link channel in a message transmission process often cause unstable IPC forwarding performance and even problems of IPC forwarding faults, and reasons of the problems cannot be traced, so that the communication efficiency, reliability and stability of the IPC are reduced.

Disclosure of Invention

The application provides a method and a device for monitoring IPC message forwarding performance, which are used for solving related problems in the prior art.

In a first aspect, the present application provides a method for monitoring IPC link forwarding performance, applied to a first node of a distributed device, where the method includes:

receiving an IPC message sent by a second node, wherein the IPC message carries a sending time stamp of the IPC message sent by the second node;

determining a receiving time stamp for receiving the IPC message;

based on the sending timestamp and the receiving timestamp, determining an IPC link transmission delay between the first node and the second node;

if the IPC link transmission time delay is monitored to be larger than a preset threshold value, recording an abnormal transmission event corresponding to the IPC message.

Optionally, the step of recording the abnormal transmission event corresponding to the IPC message includes:

and recording the IPC message, a receiving and transmitting node corresponding to the IPC message and the time for receiving and transmitting the IPC message.

Optionally, the method further comprises:

receiving an abnormal transmission event query instruction sent by a user, wherein the abnormal transmission event query instruction comprises at least one query condition;

and displaying the target abnormal transmission event matched with the at least one query condition to a user.

Optionally, the IPC message also carries a message sequence number; the method further comprises the steps of:

and judging whether the received message sequence numbers of the IPC messages sent by the second node are continuous or not so as to monitor whether the condition of IPC link transmission packet loss exists between the second node and the second node or not.

Optionally, the method further comprises:

and if the IPC link transmission time delay is monitored to be larger than a preset threshold value and/or if the IPC link transmission packet loss exists between the second node and the second node, generating an alarm of abnormal IPC link transmission.

In a second aspect, the present application provides an IPC link forwarding performance monitoring apparatus, applied to a first node of a distributed device, where the apparatus includes:

the receiving unit is used for receiving an IPC message sent by a second node, wherein the IPC message carries a sending time stamp of the IPC message sent by the second node;

the determining unit is used for determining a receiving time stamp for receiving the IPC message;

the determining unit is further configured to determine an IPC link transmission delay between the first node and the second node based on the transmission timestamp and the reception timestamp;

the monitoring unit is used for monitoring that the transmission delay of the IPC link is larger than a preset threshold value;

and the recording unit is used for recording the abnormal transmission event corresponding to the IPC message.

Optionally, when recording the abnormal transmission event corresponding to the IPC message, the recording unit is specifically configured to:

Optionally, the device further comprises a display unit:

the receiving unit is further configured to receive an abnormal transmission event query instruction sent by a user, where the abnormal transmission event query instruction includes at least one query condition;

the display unit is used for displaying the target abnormal transmission event matched with the at least one query condition to a user.

Optionally, the IPC message also carries a message sequence number;

the monitoring unit is further configured to monitor whether there is an IPC link transmission packet loss condition between the second node and the second node by determining whether the received message sequence numbers of the IPC messages sent by the second node are continuous.

Optionally, the device further comprises an alarm unit:

and if the monitoring unit monitors that the transmission delay of the IPC link is greater than a preset threshold value and/or monitors that the transmission packet loss of the IPC link exists between the second node and the second node, the alarm unit is used for generating an alarm of abnormal transmission of the IPC link.

In a third aspect, an embodiment of the present application provides an IPC link forwarding performance monitoring apparatus, where the IPC link forwarding performance monitoring apparatus includes:

a memory for storing program instructions;

a processor for invoking program instructions stored in said memory, performing the steps of the method according to any of the first aspects above in accordance with the obtained program instructions.

In a fourth aspect, embodiments of the present application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the steps of the method according to any one of the first aspects.

As can be seen from the foregoing, the IPC link forwarding performance monitoring method provided in the embodiments of the present application is applied to a first node of a distributed device, and the method includes: receiving an IPC message sent by a second node, wherein the IPC message carries a sending time stamp of the IPC message sent by the second node; determining a receiving time stamp for receiving the IPC message; based on the sending timestamp and the receiving timestamp, determining an IPC link transmission delay between the first node and the second node; if the IPC link transmission time delay is monitored to be larger than a preset threshold value, recording an abnormal transmission event corresponding to the IPC message.

The IPC provided by the embodiment of the application is applied to the first node of the distributed equipment, and the method comprises the following steps: the IPC message sent by the second node is received, and the residence time of the IPC message can be accurately measured by introducing a time stamp, so that the performance bottleneck in a communication link can be positioned, and the system performance can be optimized; the packet loss condition of the IPC message is detected and processed by using the sequence number of the sequence message, so that the reliability and stability of the IPC communication can be improved. The efficiency of location IPC problem has been promoted, location problem manpower input has been saved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following description will briefly describe the drawings that are required to be used in the embodiments of the present application or the description in the prior art, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings may also be obtained according to these drawings of the embodiments of the present application for a person having ordinary skill in the art.

FIG. 1 is a schematic diagram of the relationship between nodes, links and channels in a distributed device-independent mode of operation;

FIG. 2 is a schematic diagram of a device configuration in a distributed device-independent mode of operation;

FIG. 3 is a schematic diagram of an IPC message processing procedure;

fig. 4 is a detailed flowchart of an IPC link forwarding performance monitoring method provided in an embodiment of the present application;

fig. 5 is a schematic structural diagram of an IPC link forwarding performance monitoring apparatus provided in the embodiments of the present application;

fig. 6 is a schematic hardware architecture diagram of an IPC link forwarding performance monitoring apparatus provided in an embodiment of the present application.

Detailed Description

The terminology used in the embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to any or all possible combinations including one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in embodiments of the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, a first message may also be referred to as a second message, and similarly, a second message may also be referred to as a first message, without departing from the scope of the present application. Depending on the context, furthermore, the word "if" used may be interpreted as "at … …" or "at … …" or "in response to a determination".

Exemplary, referring to FIG. 2, a schematic device architecture in a distributed device-independent mode of operation is shown; the distributed devices include SLOT0, SLOT 1, SLOT2 and SLOT 3, and the devices in the distributed device-independent operation mode are generally configured with a plurality of single boards, that is, a plurality of nodes exist in the devices, and when a service module needs to collect a plurality of single board information, the service module passes through an internal IPC channel, so that the stability and reliability of the IPC performance have great influence on the operation of the devices. If the IPC message processing time of a certain node is too long, the processing of the IPC message is blocked. Or when the IPC module fails and the IPC message is discarded, the upper business module is caused to have a clamping and stopping condition in the related process of synchronously acquiring the data.

For example, a command for the running condition of each single board CPU in the device is obtained. After the command line is executed, the device processes the command in the SLOT0 master control logic, collects the CPU use condition of SLOT0, traverses the on-site single board, sends a message to the single board through the IPC to acquire the CPU operation condition of the single board, sends back to SLOT0 through the IPC, and outputs the collected single board information to the console for display. Log below shows that there is a 10 second delay from the last veneer at the time SLOT2 was collected.

[2023/08/03 22:03:28]Slot 0 CPU 0 CPU usage:

[2023/08/03 22:03:28] 4％in last 5 seconds

[2023/08/03 22:03:28] 3％in last 1 minute

[2023/08/03 22:03:28] 3％in last 5 minutes

[2023/08/03 22:03:28]

[2023/08/03 22:03:28]Slot 1 CPU 0 CPU usage:

[2023/08/03 22:03:28] 4％in last 5 seconds

[2023/08/03 22:03:28] 3％in last 1 minute

[2023/08/03 22:03:28] 3％in last 5 minutes

[2023/08/03 22:03:28]

[2023/08/03 22:03:39]Slot 2 CPU 0 CPU usage:

[2023/08/03 22:03:39] 10％in last 5 seconds

[2023/08/03 22:03:39] 9％in last 1 minute

[2023/08/03 22:03:39] 10％in last 5 minutes

[2023/08/03 22:03:39]

[2023/08/03 22:03:39]Slot 3 CPU 0 CPU usage:

[2023/08/03 22:03:39] 7％in last 5 seconds

[2023/08/03 22:03:39] 8％in last 1 minute

[2023/08/03 22:03:39] 8％in last 5 minutes

For this problem, it is necessary to trace back to the point in time to check whether the IPC channel has a forwarding failure, thereby causing a problem. The process of processing the IPC message is shown in FIG. 3, the current IPC process is in a user mode, the IPC message is constructed by the IPC process to be issued to the kernel, then forwarded to the corresponding node through the IPC link, then reported to the IPC process through the kernel, and then processed by the IPC process.

At present, there is no monitoring mechanism for the internal forwarding performance of the IPC, so that it cannot be judged whether there is an IPC forwarding problem when a fault occurs, and it cannot be monitored in real time whether there is time consumption caused by internal processing in the IPC forwarding flow and whether there is a packet loss condition at each stage.

The embodiment of the application provides a scheme for monitoring the forwarding performance of an IPC link in real time.

Exemplary, referring to fig. 4, a detailed flowchart of an IPC link forwarding performance monitoring method according to an embodiment of the present application is shown, where the method is applied to a first node of a distributed device, and the method includes the following steps:

step 100: receiving an IPC message sent by a second node, wherein the IPC message carries a sending time stamp of the IPC message sent by the second node;

it should be noted that, in the embodiment of the present application, the first node and the second node refer to any node (board) of the distributed device, and when an IPC process of the first node constructs and sends an IPC message (specific service message) to the second node, a sending timestamp is carried in the constructed IPC message (such as a message header). The IPC process of the first node sends the IPC message to the kernel of the first node to carry out IPC chip package; and after the inner core IPC chip of the second node receives the IPC message, forwarding the message to an IPC process of the second node for processing.

Step 110: determining a receiving time stamp for receiving the IPC message;

specifically, in the embodiment of the present application, after the IPC process of the second node receives the IPC packet, the receiving timestamp of the IPC packet is determined.

In practical applications, the time between nodes of the distributed device is already up to ns precision based on hardware implementation, and then the precision of sending the timestamp can be accurate to ns precision. It should be noted that, in the actual application scenario, the accuracy of the message filling timestamp may be determined based on different IPC performance requirements, which is not specifically limited in the embodiment of the present application.

Step 120: based on the sending timestamp and the receiving timestamp, determining an IPC link transmission delay between the first node and the second node;

specifically, based on the receiving timestamp and the sending timestamp, the transmission delay of the IPC link between the second node and the first node for forwarding the IPC message is calculated.

Step 130: if the IPC link transmission time delay is monitored to be larger than a preset threshold value, recording an abnormal transmission event corresponding to the IPC message.

In this embodiment of the present application, a preset threshold is set for each IPC link (IPC link between different nodes), where the preset threshold of each IPC link may be the same or different. And then, judging whether the calculated transmission delay of the IPC link between the first node and the second node is larger than a preset threshold value, if so, determining that the transmission delay is too large, and recording an abnormal transmission event when the transmission delay of the IPC link is abnormal.

Specifically, when recording the abnormal transmission event corresponding to the IPC message, a preferred implementation manner is as follows: and recording the IPC message, a receiving and transmitting node corresponding to the IPC message and the time for receiving and transmitting the IPC message.

That is, whether the transmission delay value of the IPC link is within a reasonable range is determined according to a preset transmission delay threshold, and if not, the IPC message, the node for transmitting and receiving the IPC message, and the timestamp for transmitting and receiving the IPC message are recorded.

Further, an abnormal transmission event query instruction sent by a user is received, wherein the abnormal transmission event query instruction comprises at least one query condition; and displaying the target abnormal transmission event matched with the at least one query condition to a user.

For example, the distributed device under the user sends an abnormal transmission event query instruction for querying between the first node and the second node (query condition 1) for a certain period of time (query condition 2), and then the first node/the second node can display the target abnormal transmission event which is locally recorded and matches the query condition to the user.

In practical application, the IPC link transmission delay information can be integrated based on different transmission link nodes to obtain the average transmission delay of the IPC link in one time period such as last 5s, last30 s, last 60s and last300 s. Then, for the problem that the CPU usage information of each node is slowly read by the command line, whether the congestion condition exists can be judged by checking whether the condition that the transmission delay of the IPC link from SLOT0 to SLOT2 is overlarge exists in the time and analyzing the average value of the transmission delay in the command knocking time.

Through the recorded information, whether the fault time point is overlapped with the abnormal time point or not can be judged according to the record of the abnormal time point exceeding the IPC transmission delay threshold value, so that fault analysis is carried out.

Further, in the embodiment of the present application, the IPC message also carries a message sequence number; the method further comprises the steps of:

For the IPC link between the node a (e.g., the first node) and the node B (e.g., the second node), for each IPC message sent to the first node by the second node, a message sequence number is carried in the IPC message header, specifically, a message sequence number statistics period may be set, and in the current period, several IPC messages continuously sent to the first message all carry continuous message sequence numbers, so after the first node receives each IPC message sent by the second node in the current period, whether there is an IPC message loss condition in the IPC link (i.e., the IPC link between the second node- > the first node) can be judged based on whether the message sequence numbers of each IPC message are continuous. The first node sends each IPC message to the second node, or may carry a message sequence number based on a preset rule, and then the second node may also determine whether an IPC link (i.e., an IPC link between the first node and the second node) has a condition that the IPC message is lost based on whether the message sequence number carried by each IPC message sent by the first node and received in the current period is continuous.

Further, in the embodiment of the present application, if it is monitored that the IPC link transmission delay is greater than a preset threshold, and/or it is monitored that there is a packet loss in IPC link transmission between the second node and the second node, an alarm of abnormal IPC link transmission is generated.

That is, the forwarding performance of each IPC link is monitored in real time, if the performance is found to be unable to meet the preset requirement/the link fails, an alarm message can be generated to inform the user/manager to timely conduct problem investigation, so as to improve the stability and reliability of IPC communication.

For example, referring to fig. 3, when determining that the IPC packet counts the packet sequence number in the stage (3), there is a packet sequence number interruption at the time point of the blocking of the command execution; then, if the message sequence number in the stage (1) to (2) is not intermittent, the problem of the IPC link can be rapidly checked, and corresponding alarm information is generated so as to prompt a user/manager to timely check the problem.

An exemplary embodiment, referring to fig. 5, is a schematic structural diagram of an IPC link forwarding performance monitoring apparatus provided in an embodiment of the present application, where the apparatus is applied to a first node of a distributed device, and the apparatus includes:

the receiving unit 50 is configured to receive an IPC packet sent by a second node, where the IPC packet carries a sending timestamp of the IPC packet sent by the second node;

a determining unit 51, configured to determine a reception timestamp of the IPC packet;

the determining unit 51 is further configured to determine an IPC link transmission delay between the first node and the second node based on the transmission timestamp and the reception timestamp;

the monitoring unit 52 is configured to monitor that the IPC link transmission delay is greater than a preset threshold;

and a recording unit 53, configured to record an abnormal transmission event corresponding to the IPC packet.

Optionally, when recording the abnormal transmission event corresponding to the IPC message, the recording unit 53 is specifically configured to:

Optionally, the device further comprises a display unit:

the receiving unit 50 is further configured to receive an abnormal transmission event query instruction sent by a user, where the abnormal transmission event query instruction includes at least one query condition;

Optionally, the IPC message also carries a message sequence number;

the monitoring unit 52 is further configured to monitor whether there is an IPC link transmission packet loss condition between the second node and the second node by determining whether the received message sequence numbers of the IPC messages sent by the second node are continuous.

Optionally, the device further comprises an alarm unit:

if the monitoring unit 52 monitors that the IPC link transmission delay is greater than a preset threshold, and/or monitors that there is a packet loss in IPC link transmission between the second node and the second node, the alarm unit is configured to generate an alarm of abnormal IPC link transmission.

The above units may be one or more integrated circuits configured to implement the above methods, for example: one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more microprocessors (digital singnal processor, abbreviated as DSP), or one or more field programmable gate arrays (Field Programmable Gate Array, abbreviated as FPGA), or the like. For another example, when a unit is implemented in the form of a processing element scheduler code, the processing element may be a general purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program code. For another example, the units may be integrated together and implemented in the form of a system-on-a-chip (SOC).

Further, in the IPC link forwarding performance monitoring device provided in the embodiments of the present application, from a hardware level, a hardware architecture schematic diagram of the IPC link forwarding performance monitoring device may be shown in fig. 6, and the IPC link forwarding performance monitoring device may include: a memory 60 and a processor 61,

memory 60 is used to store program instructions; the processor 61 invokes the program instructions stored in the memory 60 to perform the above-described method embodiments in accordance with the obtained program instructions. The specific implementation manner and the technical effect are similar, and are not repeated here.

Optionally, the present application also provides a distributed device comprising at least one processing element (or chip) for performing the above-described method embodiments.

Optionally, the present application also provides a program product, such as a computer readable storage medium, storing computer executable instructions for causing the computer to perform the above-described method embodiments.

Here, a machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, or the like. For example, a machine-readable storage medium may be: RAM (Radom Access Memory, random access memory), volatile memory, non-volatile memory, flash memory, a storage drive (e.g., hard drive), a solid state drive, any type of storage disk (e.g., optical disk, dvd, etc.), or a similar storage medium, or a combination thereof.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present application.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Moreover, these computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The foregoing description of the preferred embodiments of the present invention is not intended to limit the invention to the precise form disclosed, and any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention are intended to be included within the scope of the present invention.

Claims

1. An IPC link forwarding performance monitoring method, applied to a first node of a distributed device, the method comprising:

determining a receiving time stamp for receiving the IPC message;

2. The method of claim 1, wherein the step of recording the abnormal transmission event corresponding to the IPC message comprises:

3. The method of claim 1, wherein the method further comprises:

4. The method of any one of claims 1-3, wherein the IPC message further carries a message sequence number; the method further comprises the steps of:

5. The method of claim 4, wherein the method further comprises:

6. An IPC link forwarding performance monitoring apparatus, applied to a first node of a distributed device, the apparatus comprising:

7. The apparatus of claim 6, further comprising a display unit:

8. The apparatus according to any one of claims 6-7, wherein the IPC message further carries a message sequence number;

9. An IPC link forwarding performance monitoring apparatus, wherein the IPC link forwarding performance monitoring apparatus includes:

a memory for storing program instructions;

a processor for invoking program instructions stored in the memory and performing the steps of the method according to any of claims 1-5 in accordance with the obtained program instructions.

10. A computer readable storage medium storing computer executable instructions for causing a computer to perform the steps of the method according to any one of claims 1-5.