WO2023112174A1

WO2023112174A1 - Data processing device, data processing method, and program

Info

Publication number: WO2023112174A1
Application number: PCT/JP2021/046134
Authority: WO
Inventors: 悠介関原; 奈美子池田; 寛之鵜澤; 晶子大輝; 彩希八田; 周平吉田
Original assignee: 日本電信電話株式会社
Priority date: 2021-12-14
Filing date: 2021-12-14
Publication date: 2023-06-22

Abstract

This data processing device (10) comprises a process executing unit (14) that receives a packet from a reception unit (11), receives a timestamp from a timestamp unit (12), receives flow information from a packet identifying unit (13), and executes processing based on the received packet, timestamp, and flow information. The respective processing times of the reception processing by the reception unit, timestamp processing by the timestamp unit, and packet identification processing by the packet identifying unit are constant regardless of the contents of the data to be processed. The process executing unit starts processing using at least one of a packet and a timestamp for the packet before the flow information for the packet is input thereto.

Description

DATA PROCESSING DEVICE, DATA PROCESSING METHOD, AND PROGRAM

The present invention relates to a data processing device, a data processing method, and a program.

In order to understand the communication status in the network, it is common practice to acquire the packets flowing through the network and their information. In the past, it was common for packets on a network to have a unique field structure in their design concept, but as network virtualization progresses, multiple networks are configured on a single physical configuration. As a result, the field structure of packets on the network is no longer uniquely determined. Under such circumstances, a communication data identification device is required to be able to flexibly select an identification object with respect to the field structure of a packet. As a solution to such problems, there is generally a comparison and matching method using TCAM (Ternary Content Addressable Memory) (for example, Non-Patent Document 1). However, since the TCAM is mounted as a dedicated chip, there is a problem that it cannot be introduced unless it is an expensive dedicated device. On the other hand, for hardware and software that are not dedicated chips, a design method using a high-level language such as C or JAVA, which is easy to adapt to a new field with a small number of man-hours, is used.

A high-level language is a procedural description language, and is a description of processes that are performed sequentially. FIG. 9 shows a data processing device 910 as a comparative example that can be considered as a device employing sequential processing. In this comparative example, after all the output data from the receiving unit 911, the time stamp unit 912, and the packet identification unit 913 are stored in the buffer 919, the processing by the processing execution unit 914 is started. Therefore, the processing execution unit 914 has to wait until all the output data (processing results) from the units 911 to 913 are available, and the operation efficiency of the processing execution unit is poor. Such a problem can also be applied to general data processing devices other than the data processing device 910 .

An object of the present invention is to improve the operational efficiency of the processing execution unit.

In order to solve the above problems, a data processing apparatus according to the present invention includes a receiving section that executes reception processing for receiving and outputting data, and a first processing for the data output from the reception section. a first processing unit that outputs a first processing result of the first processing; and a second processing unit that executes a second processing on the data after the first processing and outputs a second processing result of the second processing. a second processing unit that receives the data from the receiving unit, receives the first processing result from the first processing unit, receives the second processing result from the second processing unit, and receives the received a processing execution unit that executes processing based on data, the first processing result, and the second processing result, wherein at least the first processing out of the reception processing, the first processing, and the second processing and the processing time of each of the second processing is constant irrespective of the content of the data to be processed, and the processing execution unit performs the processing of the data and the second processing of the data before receiving the second processing result. 1 Start processing using at least one of the processing results.

In order to solve the above problems, a data processing method according to the present invention includes a receiving step of executing a receiving process of receiving and outputting data, and executing a first process on the data output in the receiving step. a first processing step of outputting a first processing result of the first processing; and executing a second processing on the data after the first processing, and outputting a second processing result of the second processing. a second processing step for receiving the data output in the receiving step, receiving the first processing result output in the first processing step, and receiving the second processing result output in the second processing step; a processing execution step of receiving a processing result and executing processing based on the received data, the first processing result, and the second processing result; Of the two processes, the processing time for each of at least the first process and the second process is constant regardless of the content of data to be processed, and in the process execution step, before receiving the result of the second process Processing using at least one of the data and the first processing result of the data is started.

In order to solve the above problems, a computer according to the present invention includes a receiving step of executing a receiving process of receiving and outputting data in the computer, and performing a first process on the data output in the receiving step. a first processing step of executing and outputting a first processing result of the first processing; and executing a second processing on the data after the first processing and outputting a second processing result of the second processing. a second processing step of outputting; receiving the data output in the receiving step; receiving the first processing result output in the first processing step; and receiving the first processing result output in the second processing step. 2 receiving a processing result, causing a processing execution step of executing processing based on the received data, the first processing result, and the second processing result, and performing the reception processing, the first processing, and Each processing time of at least the first processing and the second processing of the second processing is constant regardless of the contents of data to be processed, and the processing execution step receives the result of the second processing. A process using at least one of the data and the first process result on the data is started.

According to the present invention, the operational efficiency of the processing execution unit is improved.

FIG. 1 is a configuration diagram of a data processing device according to one embodiment of the present invention. FIG. 2 is a diagram showing an example of the data structure of a packet. FIG. 3 is a diagram showing a configuration example of a rule table. FIG. 4 is a diagram for explaining a method for making the processing time constant. FIG. 5 is a diagram for explaining a method for making the processing time constant. FIG. 6 is a timing chart showing the operation of the data processing device. FIG. 7 is a timing chart showing the operation of the data processing device. FIG. 8 is a configuration diagram when the data processing apparatus of FIG. 1 is configured by a computer. 1 is a configuration diagram of a data processing device according to a comparative example; FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

As shown in FIG. 1, the data processing apparatus 10 according to the present embodiment includes a receiving section 11, a time stamp section 12 configured as a first processing section, and a packet identification section 13 configured as a second processing section. , a process execution unit 14 , an output unit 15 , and a storage unit 19 . The data processing device 10 having such a configuration is configured as a traffic monitoring device that monitors the traffic of the communication network NW for each flow.

At least part of each of the above units 11 to 14 is composed of a logic circuit written in, for example, FPGA (Field-Programmable Gate Array), ASIC (Application Specific Integrated Circuit), or the like. At least part of each unit 11 to 14 may be configured by a processor such as a CPU (Central Processing Unit) that executes programs. The storage unit 19 is composed of an appropriate non-volatile storage device such as a flash memory or SSD (Solid State Drive). The storage unit 19 may be composed of at least a part of memory such as FPGA (Field-Programmable Gate Array), ASIC (Application Specific Integrated Circuit), or the like. Each of the units 11 to 14 may include a memory that temporarily holds data being processed.

The receiving unit 11 receives a packet (more specifically, a mirroring packet) flowing through the communication network NW, and executes reception processing for outputting it to the time stamp unit 12 and the processing execution unit 14 . As shown in FIG. 2, a packet consists of data (also called payload) and a header placed before the data. Each data that constitutes the header has meaning for each certain number of bits. This meaningful bit-delimited data is called a header field value, and a group of packets with matching header field values is called a flow. Therefore, the header field value is also information that identifies the flow. Note that the header field value may include IP (Internet Protocol) addresses indicating the source and destination of the L3 header. A flow may be, for example, a set of packets having a common header field value indicating at least one of destination, source, and communication protocol.

Returning to FIG. 1, the time stamp unit 12 refers to a real-time clock or the like, generates a time stamp for the packet output from the reception unit 11, and executes time stamp processing for outputting the generated time stamp to the processing execution unit 14. The time stamp unit 12 outputs the packet from the reception unit 11 to the packet identification unit 13 . At this time, the packet may be given a time stamp.

The packet identification unit 13 identifies the flow to which the packet from the timestamp unit 12 for which the timestamp is generated belongs, and performs packet identification processing to output a flow ID, which is flow information identifying the identified flow. The packet identification unit 13 includes a header analysis unit 13A, a hash calculation unit 13B, and a rule matching unit 13C. These

units

13A and 13B perform the following processes to perform packet identification processing.

A packet input from the time stamp unit 12 to the packet identification unit 13 is first input to the header analysis unit 13A. The header analysis unit 13A analyzes the header of the input packet and executes analysis processing for extracting header field values from the header. The extracted header field value is input to hash calculator 13B.

The hash calculation unit 13B executes hash calculation processing for hashing the input header field value. The hash calculation unit 13B outputs the hashed header field value together with the time stamp to the rule matching unit 13C.

The rule matching unit 13C matches the input header field value (hashed header field value) with a predetermined packet belonging to the monitored flow registered in the rule table 19A stored in the storage unit 19. Executes a matching process that compares the rule with As shown in FIG. 3, rules registered in the rule table 19A are registered for each flow, and one rule consists of a header field value (hashed header field value) of the flow and and the associated flow ID. A flow ID may consist of a rule ID that identifies a rule. In the matching process, the rule matching unit 13C refers to each rule in the rule table 19A and acquires the flow ID corresponding to the input header field value. The rule matching unit 13</b>C outputs the acquired flow ID to the process execution unit 14 .

The processing execution unit 14 receives a packet from the reception unit 11, receives a time stamp from the time stamp unit 12, receives a flow ID from the packet identification unit 13, and performs processing based on the received packet, time stamp, and flow ID. (here, the processing required for traffic monitoring). The processing execution unit 14 includes an aggregation unit 14A, a packet capture unit 14B, and an anomaly detection unit 14C as calculation units that execute the above processing.

The aggregation unit 14A counts the number of times the flow ID is input from the packet identification unit 13 for each flow ID, that is, for each flow. The counted number of receptions can be said to be the number of packets received by the receiving unit 11, that is, the number of packets. The counting unit 14A measures the amount of data based on the packets from the receiving unit 11, and integrates the measured amount of data for each flow. The tallying unit 14A outputs the number of packets counted or integrated during the predetermined period and the amount of data for each flow to the output unit 15 at regular intervals as a tallying result. The processing result may be either the number of packets or the amount of data.

The packet capture unit 14B executes packet capture to store packets from the reception unit 11 in the storage unit 19. Furthermore, the packet capture unit 14B adds a time stamp from the time stamp unit 12. FIG.

The anomaly detection unit 14C detects a traffic anomaly (for example, , microbursts, etc.). When detecting traffic anomaly, the anomaly detection unit 14C outputs to the output unit 15 as anomaly occurrence information that the traffic anomaly has occurred. The anomaly occurrence information may include information on the type of traffic anomaly that has occurred.

The output unit 15 outputs the result of processing by the processing execution unit 14, here, the aggregated result from the aggregation unit 14A and the abnormality occurrence information from the abnormality detection unit 14C. The output unit 15 displays the tally result and/or the abnormality occurrence information on, for example, a display unit provided in the data processing device 10 . The output unit 15 outputs, for example, the abnormality occurrence information to a predetermined processing unit provided inside or outside the data processing device 10, and the processing unit outputs the traffic abnormality detected flow based on the abnormality occurrence information. You may perform the process for interrupting communication about. The output unit 15 may acquire packets captured by the packet capture unit 14B and output the acquired packets to a device external to the data processing device 10 .

In this embodiment, the receiving section 11 is configured such that the processing time for receiving processing (hereinafter also referred to as processing time A) is constant regardless of the data content of the packet input to the receiving section 11 . The time stamp unit 12 is also configured such that the processing time for time stamp processing (hereinafter also referred to as processing time B) is constant regardless of the data content of the packet input to the time stamp unit 12 . The packet identification unit 13 is also configured such that the processing time for packet identification processing (hereinafter also referred to as processing time C) is constant regardless of the data content of the packet input to the packet identification unit 13 . In other words, each of the units 11 to 13 completes necessary processing for any input data (data to be processed) within a certain processing time. Each start timing of the processing times A to C is, for example, the timing at which data input is started. The end timing of each of the processing times A to C is, for example, the timing at which the output of data (processing results, etc.) is started. The processing times A to C may be the same processing time, or may be different times.

The fact that the processing time A is the same for any packet processing means that the processing in the receiving unit 11 is completed within a certain processing time regardless of which processing branch is passed through. For example, as shown in the upper diagram of FIG. 4, if the processing time of the process A after branching is 3 clock cycles and the processing time of the process B is 2 clock cycles, then the process shown in the lower diagram A delay circuit for one clock cycle is inserted after B. This makes the processing time after branching constant. Alternatively, as shown in FIG. 5, part of the processing A after branching is parallelized to shorten the processing time to two clock cycles. This makes the processing time after branching constant. The same applies to the time stamp section 12 and the packet identification section 13 as well. Note that the adjustment of the processing time is not limited to this. Each unit 11 to 13 measures the time from the start of processing with a timer or the like, and outputs the data of the processing result until the time from the start of processing measured by the timer or the like reaches the processing times A to C even after the processing is completed. may wait.

As shown in FIG. 6, the receiving unit 11 executes the receiving process during the processing time A. A packet received in the reception process is input to the time stamp unit 12 and the process execution unit 14 (timing T2). The time stamp unit 12 executes time stamp processing during the processing time B, triggered by the packet input from the reception unit 11 . The time stamp added by the time stamp processing is input to the processing execution unit 14, and the packet to which the time stamp is to be added is input to the packet identification unit 13 (timing T3). The packet identification unit 13 executes packet identification processing in processing time C, triggered by the packet input from the time stamp unit 12 . The flow ID and the like acquired in the packet identification process are input to the process execution unit 14 (timing T4).

The processing execution unit 14 can identify which data is input to itself in relation to the processing times A to C. This is because the processing times A to C are constant as described above. For example, the reception unit 11 transmits a packet reception notification to the processing execution unit 14 at the start of packet reception processing (timing T1).

After sending the reception notification, the packet is input to the processing execution unit 14 without fail after the processing time A has elapsed (timing T2). Therefore, the processing execution unit 14 recognizes data input after the processing time A has elapsed after receiving the reception notification as a packet. The processing execution unit 14 performs packet capture, for example, by storing packets in the storage unit 19 using the packet capture unit 14B.

After the processing time B has elapsed from the input of the packet, the time stamp is always input to the processing execution unit 14 (timing T3). Therefore, the processing execution unit 14 recognizes the data input after the processing time B has elapsed after the packet is input as the time stamp given to the packet. For example, the processing execution unit 14 associates the time stamp input from the time stamp unit 12 with the packet stored in the storage unit 19 by the packet capture unit 14B, and stores the time stamp in the storage unit 19. Memorize. Note that the process execution unit 14 may hold the packet, add a time stamp to the packet, and store the packet in the storage unit 19 when the time stamp is input.

After the time stamp is input, the flow ID and the like obtained by the packet identification process are input to the process execution unit 14 after the processing time C has elapsed (timing T4). Therefore, the processing execution unit 14 recognizes the data input after the processing time C has elapsed after the time stamp is input as the flow ID of the flow to which the current packet belongs. The processing execution unit 14 executes the processing of the aggregation unit 14A and the processing of the abnormality detection unit 14C based on the flow ID and the like.

The processing execution unit 14 determines which data arrived at which timing based on the designed processing times A to C of the respective units 11 to 13 for each data input at different timings. can do. . Then, the processing execution unit 14 uses at least one of the packet and the time stamp of the packet (for example, packet capture and time stamping in FIG. 6) before the flow ID of the packet is input. Processing can begin. As a result, the process execution unit 14 does not wait until the flow ID or the like is input before starting the process as in the comparative example, thereby improving the operation efficiency.

Further, the processing execution unit 14 receives a packet from the reception unit 11, and processes a plurality of pieces of data, which are input for each predetermined processing time of the time stamp processing and the packet identification processing, as a time stamp for the packet, Then, the processing is executed as the flow ID for the packet. This prevents the processing execution unit 14 from confusing the time stamp or flow ID of one packet with the time stamp or flow ID of another packet.

Further, each of the units 11 to 13 in the preceding stage of the processing execution unit 14 can sequentially accept subsequent input data after data output has been completed. As shown in FIG. 7, for example, after the receiving unit 11 outputs the first packet (see “receiving process” indicated by the solid line), before the packet identification process by the packet identifying unit 13 is completed, the receiving unit 11 outputs the second packet (referred to by the dotted line). (See "Receiving Process"). As a result, the speed of data processing is increased.

As indicated by the dotted line in FIG. 1, at least one of the header analysis unit 13A and the hash calculation unit 13B of the packet identification unit 13 may output processing result data to the processing execution unit 14. The processing execution unit 14 may perform a predetermined processing based on the input processing result data. In such a case, the processing time of each section 13A to 13C of the packet identification section 13 should also be constant regardless of the contents of the data to be processed. The processing execution unit 14 may detect the input of a packet by its data content, etc. In this case, which data is the time stamp and flow ID is specified based on the processing times B and C after the packet is input. can.

FIG. 8 shows a hardware configuration diagram when the data processing device 10 is configured by a computer. The data processing device 10 includes a processor 101 such as a CPU, a main memory 102 of the processor 101, and a non-volatile storage device 103 that stores programs and various data and constitutes the storage unit 19 in FIG. Further, the data processing device 10 includes a NIC (Network Interface Card) 104 that is connected to the communication network NW and relays packets. The processor 101 executes or uses programs and data stored in the storage device 103 and read out to the main memory 102 to operate as the units 11 to 15 described above. Note that the receiving unit 11 and the output unit 15 may be implemented by a combination of the processor 101 executing the program and the NIC 104 .

The data processing device 10 is not limited to the above configuration. The data processing apparatus 10 includes, for example, a receiving unit (receiving unit 11 in the above configuration) that executes reception processing for receiving and outputting data (packets in the above configuration), and for the data output from the receiving unit: a first processing unit (time stamp unit 12 in the above configuration) that executes a first process (time stamp processing in the above configuration) and outputs a first processing result (time stamp in the above configuration) of the first process; be prepared. The data processing device 10 performs, for example, a second process (a packet identification process in the above configuration) on the data after the first process, and outputs a second process result of the second process. (packet identification unit 13 in the above description), the data is received from the receiving unit, the first processing result is received from the first processing unit, and the second processing result is received from the second processing unit. , a processing execution unit ( In the above, the processing execution unit 14) can be provided. The processing time of at least the first processing and the second processing out of the reception processing, the first processing, and the second processing may be constant regardless of the content of the data to be processed. Before receiving the second processing result, the processing execution unit often performs processing using at least one of the data and the first processing result of the data (in the above description, packet capture, time stamping in FIG. 6, ) may be started. With such a configuration, unlike the comparative example, the processing execution unit does not wait until the second processing result is input before starting the processing, and the operation efficiency of the processing execution unit is improved. It should be noted that specific examples of the above processes are not limited to the above embodiments. The data may be image data, the first processing may be image binarization processing, and the second processing may be image recognition for the binarized image. The processing execution unit may execute processing for storing the image data, the binarized image, and the result of image recognition in the storage unit.

The processing execution unit receives the data from the reception unit and processes the plurality of data input at predetermined processing time intervals for each of the first processing and the second processing. The processing may be performed as the first processing result and the second processing result, thereby preventing the first processing result and the second processing result for certain data from being mistaken for processing results for other data. .

After outputting the first data as the data, the receiving section is capable of receiving second data as the other data before the second processing ends. This improves the processing speed.

The present invention is not limited to the above embodiments and modifications. For example, the present invention includes various modifications to the above embodiments and modifications that can be understood by those skilled in the art within the scope of the technical idea of the present invention. The configurations described in the above embodiments and modified examples can be appropriately combined within a consistent range. It is also possible to delete any configuration among the above configurations. The program may be stored not only in the non-volatile storage device 103 but also in a non-temporary computer-readable storage medium. "Apparatus" and "unit" refer to an object whose configuration realizing its operation is housed in a plurality of housings, even if the configuration realizing its operation is housed in a single housing ( system).

DESCRIPTION OF SYMBOLS 10... Data processing apparatus 11... Receiving part 12... Timestamp part 13... Packet identification part 13A... Header analysis part 13B... Hash calculation part 13C... Rule matching part 14... Processing execution part 14A... Aggregation Unit 14B Packet capture unit 14C Abnormality detection unit 15 Output unit 19 Storage unit 19A Rule table 101 Processor 102 Main memory 103 Storage device NW Communication network.

Claims

a receiving unit that executes a receiving process for receiving and outputting data;
a first processing unit that performs a first process on the data output from the receiving unit and outputs a first processing result of the first process;
a second processing unit that executes a second process on the data after the first process and outputs a second process result of the second process;
receiving the data from the receiving unit, receiving the first processing result from the first processing unit, receiving the second processing result from the second processing unit, and receiving the received data and the first processing result and a processing execution unit that executes processing based on the second processing result,
each processing time of at least the first processing and the second processing out of the reception processing, the first processing, and the second processing is constant regardless of the content of the data to be processed;
The processing execution unit starts processing using at least one of the data and the first processing result of the data before receiving the second processing result.
Data processing equipment.
The data are packets flowing through a communication network,
The first processing unit, as the first processing, generates a time stamp for the packet output from the receiving unit, and executes time stamp processing for outputting the generated time stamp as the first processing result. is a time stamp part that
The second processing unit, as the second processing, identifies a flow to which the packet for which the timestamp is generated belongs, and outputs flow information identifying the identified flow as the result of the second processing. is a packet identifier that performs
2. A data processing apparatus according to claim 1.
The processing execution unit receives the data from the reception unit and processes the plurality of data input at predetermined processing time intervals for each of the first processing and the second processing. executing the process as a first process result and the second process result;
3. A data processing apparatus according to claim 1 or 2.
After outputting the first data as the data, the receiving unit is provided so as to be able to receive second data as the other data before the second processing ends.
4. A data processing apparatus according to any one of claims 1 to 3.
The data are packets flowing through a communication network,
The second processing unit includes a header analysis unit that analyzes the header of the packet and extracts a header field value, a hash calculation unit that hashes the extracted header field value, and a hashed header field value. a rule matching unit that acquires the flow information of the flow to which the packet belongs by comparing with a predetermined rule;
5. A data processing apparatus according to any one of claims 1 to 4.
a reception step for executing reception processing for receiving and outputting data;
a first processing step of performing a first process on the data output in the receiving step and outputting a first processing result of the first process;
a second processing step of performing a second processing on the data after the first processing and outputting a second processing result of the second processing;
receiving the data output in the receiving step, receiving the first processing result output in the first processing step, receiving the second processing result output in the second processing step, and receiving a processing execution step of executing processing based on the data, the first processing result, and the second processing result;
each processing time of at least the first processing and the second processing out of the reception processing, the first processing, and the second processing is constant regardless of the content of the data to be processed;
In the process execution step, before receiving the second process result, the process using at least one of the data and the first process result of the data is started.
Data processing method.
to the computer,
a reception step for executing reception processing for receiving and outputting data;
a first processing step of performing a first process on the data output in the receiving step and outputting a first processing result of the first process;
a second processing step of performing a second processing on the data after the first processing and outputting a second processing result of the second processing;
receiving the data output in the receiving step, receiving the first processing result output in the first processing step, receiving the second processing result output in the second processing step, and receiving a processing execution step of executing processing based on the data, the first processing result, and the second processing result;
each processing time of at least the first processing and the second processing out of the reception processing, the first processing, and the second processing is constant regardless of the content of the data to be processed;
In the process execution step, before receiving the second process result, the process using at least one of the data and the first process result of the data is started.
program.