JP2013179429A - Parallel packet processing method and device for switching distribution destination - Google Patents

Abstract

PROBLEM TO BE SOLVED: To change a distribution destination when processing packets by parallel processing by a plurality of processing cores, and to reduce influence of processing delay generated when changing the distribution destination.SOLUTION: In a distribution part 107 for distributing flow to a plurality of processing cores, utilization of two old and new distribution algorithms is made possible. The state of applying the old distribution algorithm and distributing the flow is switched to the state of applying the new distribution algorithm to the flow by temporally changing a threshold for a hash value calculated from the flow so as to increase the probability of applying the new distribution algorithm to the flow with the lapse of time for each flow.

Description

  The present invention relates to a method and apparatus for transmitting / receiving and relaying data in a network, and in particular, corrects transfer data with reference to a wide range of transfer data such as L7 (Layer 7) / L4 (Layer 4) switches. The present invention relates to a method for improving performance by parallel processing in processing for transferring data or processing for transferring or discarding data to a certain path based on information of reference data, and an apparatus configured based on such a method.

  The packet transfer is generally processed by a node device provided in the network. With the diversification and sophistication of packet transfer processing requirements, packet transfer processing can be implemented flexibly on a software basis, but multiple node devices can be installed, node devices can be configured as multiprocessors, and the processor within the node device It is important to improve performance by scaling out by parallel processing using a multi-core configuration in which a plurality of cores are integrated on one chip. Furthermore, in order to realize flexible processing allocation in accordance with virtualization, the parallel number for each processing program is dynamically changed, or the parallel number is changed in order to reduce power consumption. There is also an increasing demand to execute off control. Parallel processing is particularly effective when a large amount of simple processing must be executed in a short time, such as packet processing at a router or L7 switch.

  General parallel processing in packet transfer processing is realized by a plurality of packet processing units that actually perform processing and a distribution unit that distributes packets or data to these packet processing units in consideration of load distribution and the like. The If a processor having a multi-core configuration is used, the packet processing unit is realized by a core loaded with a packet processing program, and the distribution unit is realized by a core loaded with a distribution program. Therefore, the packet processing unit is also called a processing core, and the sorting unit is also called a sorting core.

  FIG. 1 shows a general configuration of a packet processing apparatus 101 that performs the parallel processing described above. Two interface (I / F) units 102 and 103 used for connection to a network are provided, and both interface units 102 and 103 are connected to a packet input unit 104 and a packet output unit 105. The packet input unit 104 is a reception interface circuit for receiving a packet to be subjected to packet processing from the outside, and the packet output unit 105 is a transmission interface circuit for transmitting the packet subjected to packet processing to the outside. . The packet input unit 104 and the packet output unit 105 are connected to an internal bus 106, and a distribution unit (distribution core) 107 and a plurality of packet processing units (processing cores) 108 are also connected to the internal bus 106. A data holding unit 109 is provided as an external memory to hold data to be necessary for performing packet processing, and the plurality of packet processing units 108 are connected to the data holding unit 109 via the memory bus 110. The data holding unit 109 functions as a shared memory for the plurality of packet processing units 108. In this configuration, the packet input unit 104, the packet output unit 105, the internal bus 106, the distribution unit 107, the packet processing unit 108, and the memory bus 110 are included in a multi-core processor (for example, a multi-core CPU (central processing unit)) 111. Is formed.

  In the following description, when the individual packet processing units 108 are described separately, the term “core” with a number such as core 1, core 2,... Is used.

  Examples of packet processing executed by each packet processing unit 108 include firewall (FW) processing, QoS (Quality of Service) processing, NAPT (Network Address Port Translation) processing, encryption / decoding processing, and the like. The processing unit 108 is installed with a program for executing these processes. In this configuration, a packet received from the network side is sent to the packet processing unit 108 designated by the distribution unit 107 via the packet input unit 104. The packet processing unit 108 that has received the packet performs packet processing on the packet, and the packet that has been subjected to the packet processing is transmitted to the network side via the packet output unit 105.

  Each packet processing unit 108 accesses the data holding unit 109 to refer to data such as a transfer table and an address conversion table when performing packet processing. The data holding unit 109 receives data from a plurality of packet processing units 108. The access to the data storage unit 109 is likely to become a bottleneck, and the time itself for accessing the data holding unit 109, which is an external memory, causes degradation in performance. In order to avoid this bottleneck, each packet processing unit 108 is provided with a cache memory that functions as a data copy unit, and high-speed processing is realized by holding a part of the data copy in this cache memory. doing. The cache memory here is provided as a cache memory for each core in the multi-core processor 111, for example. However, the capacity of the cache memory is limited, and a copy of all necessary data cannot be placed on the cache memory, causing a miss hit in the cache. When such a mishit occurs, the packet processing unit 108 needs to access the data holding unit 109, and the performance deteriorates. FIG. 2 shows a parallel processing operation in the packet processing apparatus described above. In the example shown in FIG. 2, it is assumed that each packet processing unit 108 (processing cores 1 to 4) can execute the same program, and the allocating unit 107 uses a simple algorithm using a hash function or the like to be described later. The allocation destination is determined in units. FIG. 2 shows that an inquiry to the data holding unit 109 occurs when a cache miss hit occurs.

  In order to reduce cache miss hits as much as possible, the distribution unit 107 may distribute packets to be subjected to the same processing to the same packet processing unit as much as possible. For this reason, the allocating unit 107 refers to a packet header (5-tuple or the like), identifies a series of packet groups (flows) to be subjected to the same packet processing, and distributes them in units of flows.

  Further, since the packets to be processed by the plurality of packet processing units 108 are distributed, the load tends to concentrate on the distribution unit 107. In order to prevent the sorting process from becoming a bottleneck, it is necessary to use a simple operation as the sorting process in the sorting unit 107. Distribution in which individual flows and IDs (identification numbers) of processing cores (assignment processing cores) assigned to those flows are managed in a table on the memory, and the assignment processing cores are specified by a search involving memory access The operation tends to be a bottleneck due to many memory accesses. Instead, as described in Non-Patent Document 1, a hash value is calculated from an area for specifying a flow in the packet header by a hash function or the like, and the ID of the allocation processing core is described based on the hash value. It is preferable to obtain an allocation core from an operation such as subtracting a memory address or searching for an allocation processing core ID by a flow group collected to some extent from hash values. By using these operations, the number of memory accesses And can be processed at high speed.

  In this way, when allocating using a hash function, if the number of parallel processing of the packet processing units 108 is increased or decreased for flexible processing allocation or power saving, the newly allocated packet processing units 108 are also allocated for processing allocation. The distribution algorithm (distribution condition) is changed so that processing allocation is not performed for the packet processing units that are performed or reduced. Specifically, the change of the hash function used to obtain the hash value, the change of the threshold value used when collecting flows from the hash value, the change of the correspondence between the group of flows and the allocation processing core ID, etc. There is a need to do. When such a distribution condition is changed, many flows are assigned to different packet processing units before and after the change in the number of parallel processes. As a result, cache misses are concentrated in a short time, and processing is performed. Delay increases. For example, when the number of parallel processes is increased by 1, a cache miss hit occurs in all of the flow processes assigned to the new packet processing unit 108, so that the time required for accessing the data holding unit 109 is long. As a result, a processing delay in the packet processing unit 108 occurs.

  FIG. 3 is a diagram for explaining the occurrence of a cache miss when one packet processing unit 108 is added. As shown in (A), three packet processing units (processing cores) 108 from core 1 to core 3 are provided, and the distribution unit 107 distributes flows A to D to these processing cores. It shall be. Here, if the core 4 is newly added as a processing core, the distribution unit 107 switches the distribution algorithm from one for three divisions to one for four divisions, and flows A to D with four cores 1. Reassign to ~ 4. As a result, as shown in (B), a flow different from the previous flow is assigned to each core, and a cache miss hit occurs in each core. As a result, as shown in (C), the processing delay increases rapidly immediately after the change timing.

  As another example, when the number of parallel processes is reduced by 1, the flow handled by the reduced packet processing unit is assigned to all of the remaining packet processing units. Processing delay in each packet processing unit due to a waiting time due to contention for access to the data holding unit. Although the amount of processing for each packet is small, the delay that occurs when changing the distribution destination is a major issue in packet transfer processing that requires processing with a low delay and no order inversion for many packets. is there.

JP-T-2002-538724

  When parallel processing is adopted in packet transfer processing, a large delay occurs when the distribution destination is changed in accordance with the change of the parallel number. The occurrence of such a delay may hinder the normal transfer of packets.

  An object of the present invention is to provide a parallel packet processing method in which a distribution destination can be changed when a packet is processed by parallel processing, and the influence of a processing delay generated when the distribution destination is changed is reduced. It is in.

  Another object of the present invention is to provide a parallel packet processing apparatus that can change the distribution destination when processing packets by parallel processing and that reduces the influence of processing delay that occurs when the distribution destination is changed. There is to do.

  A first parallel packet processing method of the present invention is a parallel packet processing method for distributing a flow to a plurality of packet processing means and executing packet processing on the flow in parallel between the plurality of packet processing means, From the state in which the first distribution algorithm different from each other and the first distribution algorithm different from each other are allocated to the packet processing means, and the first distribution algorithm is applied to the flow, The probability that the second distribution algorithm is applied is increased with the passage of time so that the second distribution algorithm is applied to the flow. In an exemplary aspect of the first parallel packet processing method, the number of packet processing means to be assigned by the first distribution algorithm and the number of packet processing means to be assigned by the second distribution algorithm And may be different.

  A second parallel packet processing method of the present invention is a parallel packet processing method for distributing a flow to a plurality of packet processing means and executing packet processing on the flow in parallel between the plurality of packet processing means, Prepare different first and second distribution algorithms that distribute flows to packet processing means, and distribute flows to a plurality of packet processing means using the first distribution algorithm. Among them, the second distribution algorithm is applied to the flow that has been distributed to the specific packet processing means, and redistribution is executed. In an exemplary aspect of the second parallel packet processing method, the second distribution algorithm may be a distribution algorithm that distributes a flow to a plurality of packet processing means excluding a specific packet processing means. .

  A third parallel packet processing method of the present invention is a parallel packet processing method for distributing a flow to a plurality of packet processing means and executing packet processing on the flow in parallel between the plurality of packet processing means, The distribution algorithm for distributing flows to packet processing means can be changed, and before changing the distribution algorithm, it is determined which of the multiple packet processing means will change each flow due to the change. Then, based on the determination result, the data necessary for the processing of each changed flow is held in advance in the corresponding packet processing means, and then the distribution algorithm is changed.

  The first parallel packet processing device according to the present invention includes a plurality of packet processing means for performing packet processing on a flow, and a plurality of packet processing means for distributing flows to the plurality of packet processing means using mutually different first and second distribution algorithms. From the state in which the distribution means and the first distribution algorithm are applied to the flow, the probability that the second distribution algorithm is applied to the flow for each flow is increased with the passage of time. Control means for controlling the distribution means so as to switch to a state in which the distribution algorithm is applied to the flow. In the exemplary aspect of the first parallel packet processing apparatus, the number of packet processing means that are the distribution destinations by the first distribution algorithm and the number of packet processing means that are the distribution destinations by the second distribution algorithm The control unit may instruct the distribution unit to change the state when the parallel number is changed by the plurality of packet processing units.

  A second parallel packet processing device of the present invention includes a plurality of packet processing means for performing packet processing on a flow, a distribution means for distributing the flow to the plurality of packet processing means by a first distribution algorithm, Control means for controlling the distribution means, and the distribution means determines whether or not the redistribution is necessary based on the distribution destination of the flow for the flow distributed by applying the first distribution algorithm. A redistribution determination unit that redistributes the flow to a plurality of packet processing means using a second distribution algorithm different from the first distribution algorithm when it is determined that redistribution is necessary. The control unit instructs the distribution unit to redistribute the flow that has been allocated to the specific packet processing unit by the first distribution algorithm. In an exemplary aspect of the second parallel packet processing device, the second distribution algorithm may be a distribution algorithm that distributes a flow to a plurality of packet processing means excluding a specific packet processing means. .

  A third parallel packet processing device of the present invention includes a plurality of packet processing means for performing packet processing on a flow, a distribution means for distributing the flow to the plurality of packet processing means by a changeable distribution algorithm, Stores the algorithm information, the number of packet processing means to be distributed and each flow information, and determines which packet processing means each flow changes after the distribution algorithm is changed. And control means for controlling the distribution means so as to change the distribution destination after instructing each packet processing means to hold necessary data in advance.

  In the present invention, a state in which the second distribution algorithm is applied from the state in which the first distribution algorithm is applied by incorporating the time element into the change of the distribution destination and temporally distributing it to a small number of flow units. By changing the distribution destination, it is possible to prevent the occurrence of cache miss hits and reduce the processing delay when changing the distribution destination. In addition, in the present invention, by performing redistribution on a flow allocated to a specific distribution destination, a cache miss hit occurs while reducing the burden of allocation management between the flow and the packet processing core in the distribution processing. It is possible to reduce the occurrence of processing delay when changing the distribution destination. In the present invention, it is predicted which packet processing means each flow will be processed by changing the distribution destination, and data necessary for packet processing of each flow is previously stored in each packet based on the prediction result. By causing the processing means to cache, it is possible to prevent the occurrence of a cache miss and reduce the occurrence of a processing delay when changing the distribution destination.

It is a block diagram which shows an example of the general structure of a packet processing apparatus. It is a figure explaining the parallel processing operation | movement in the packet processing apparatus of a multi-core structure. It is a figure explaining the redistribution of the flow when the number of parallel is increased, and generation | occurrence | production of a cache miss hit. It is a figure explaining the packet processing in the packet processing apparatus of one Embodiment of this invention. It is a figure explaining the judgment logic in a distribution part. It is a figure explaining the re-distribution of the flow when a failure occurs in a specific core. It is a figure explaining the re-distribution of the flow when the bias of a process generate | occur | produces in a specific core. It is a block diagram which shows the 1st structural example of a packet processing apparatus. It is a block diagram which shows the 2nd structural example of a packet processing apparatus. It is a block diagram which shows the 3rd structural example of a packet processing apparatus. It is a block diagram which shows the 4th structural example of a packet processing apparatus. It is a block diagram which shows the 5th structural example of a packet processing apparatus. It is a block diagram which shows the 6th structural example of a packet processing apparatus. It is a block diagram which shows an example of the packet processing apparatus made into the multiprocessor structure. It is a block diagram which shows an example of the packet processing system by a multinode structure.

  Next, a preferred embodiment of the present invention will be described with reference to the drawings.

  4A and 4B are diagrams showing packet processing in the packet processing apparatus according to the embodiment of the present invention. FIG. 4A is a block diagram showing the main part of the packet processing apparatus according to the embodiment, and FIG. The figure explaining minute processing, (C) is a graph which shows the time change of the delay in packet processing.

  The packet processing apparatus according to the present embodiment has a plurality of interface units used for connection to a network, a multi-core processor 111 that executes packet processing in parallel, and implementation of packet processing, as shown in FIG. The data holding unit 109 is provided as an external memory for holding data because it is necessary for the above. Since the difference from what is shown in FIG. 1 is how each processor core in the processor 111 is used, in FIG. 4A, among the packet processing devices of this embodiment, Only the processor 111 and the data holding unit (external memory) 109 are shown.

  The processor 111 includes a packet input unit 104, a packet output unit 105, an internal bus 106, a distribution unit (distribution core) 107, a plurality of packet processing units (processing cores) 108, as shown in FIG. And a memory bus 110, and a control unit 112 for controlling packet processing in each packet processing unit 108 and flow distribution in the distribution unit 107. The control unit 112 is realized by a core loaded with a control program in the processor 111 having a multi-core configuration, and is also called a control core. The control unit 112 is connected to each packet processing unit 108 and the distribution unit 107 via the internal bus 106, dynamically changes the processing program in each packet processing unit 108, and further to the distribution unit 107. By instructing the change of the distribution method, the number of parallel processes can be changed dynamically according to the program load status, etc., and when the number of parallel processes changes due to external factors, the change So that the packet processing apparatus can cope with this. The dynamic change of the processing program in each packet processing unit 108 is performed, for example, by loading or reloading the processing program. In particular, in the packet processing apparatus according to the present embodiment, when changing the distribution destination in the flow distribution to a plurality of packet processing units 108, a time element is taken in and dispersed in a small number of flow units. By changing the distribution destination, it is possible to prevent the occurrence of cache miss hits and reduce the processing delay when changing the distribution destination. Changing the distribution destination that incorporates the time element is called time variation switching. Further, in this packet processing device, by executing the distribution process again only for the flow allocated to the specific packet processing unit, the burden of allocation management of the flow and the packet processing unit in the distribution process is performed. This reduces the occurrence of cache miss hits and reduces processing delays when changing the distribution destination. Performing distribution processing again on a flow assigned to a specific packet processing unit is called redistribution. Further, in this packet processing apparatus, it is predicted which packet processing means each flow will be processed by changing the distribution destination, and data necessary for packet processing of each flow is preliminarily stored according to the prediction result. By causing each packet processing means to cache, the occurrence of a cache miss hit can be prevented, and the occurrence of processing delay when changing the distribution destination can be reduced. Hereinafter, the change of the distribution destination in the present embodiment will be described.

  First, switching of the distribution algorithm by time variation switching will be described.

  In the configuration shown in FIG. 4A, it is assumed that initially three packet processing units 108, specifically, only the cores 1 to 3 are used for packet processing. In this state, the distribution unit 107 distributes the flows to the cores 1 to 3 using an algorithm that distributes the input flow to the three cores. Here, the control unit 112 outputs a parallel number change instruction for the number of packet processing units (processing cores) participating in the parallel processing, thereby newly starting the core 4 or changing the allocation, or changing the parallel number. Is detected and it is recognized that the core 4 is newly added ([1] in FIG. 4). At this time, the control unit 112 instructs the distribution unit 107 to start distribution destination switching and a distribution algorithm to be used ([2] in FIG. 4). The instruction of the distribution algorithm is specifically an instruction of a hash function to be used.

  The distribution unit 107 is controlled by the control unit 112 to distribute the flow to the three cores (distribution algorithm for three divisions) and the distribution algorithm for distributing the flows to the four cores (distribution for four divisions). Algorithm) and a sorting algorithm (new and old sorting algorithm) that determines which of the two sorting algorithms to apply to each flow. In particular, the old and new distribution algorithms use the elapsed time from the switching start instruction as a threshold value, and the flow passed to the distribution algorithm corresponding to the new distribution (here, the four-segment distribution algorithm) gradually increases. In this manner, flows are assigned to two distribution algorithms (a three-segment distribution algorithm and a four-division distribution algorithm). Specifically, as shown in FIG. 4B, a hash value is calculated from the flow, and the flow is distributed to the (old) 3-division distribution algorithm according to the hash value (new). Whether to distribute to the minute algorithm is determined, and the threshold value of this determination is changed with the passage of time. The 3-division distribution algorithm and 4-division distribution algorithm distribute the flows distributed by the old and new distribution algorithms to the packet processing unit (processing core). In this way, by gradually increasing the number of flows to which the distribution algorithm corresponding to the new distribution is applied, the distribution destination is changed while preventing concentration of accesses from the cores 1 to 4 to the external memory. It can be performed.

  FIG. 4C is a graph showing a change over time in packet processing delay in the present embodiment. As can be seen from this graph, since new distribution is gradually applied to the flow, the processing delay does not increase immediately after the change timing, and the influence of changing the distribution destination on the packet processing performance is not affected. It is small and can sufficiently satisfy the time constraint required for packet processing.

FIG. 5 shows a specific example of determination logic in the distribution unit. When the time of the switching start instruction is t ₀ and the current time is t, the elapsed time Δt from the switching start is
Δt = t−t ₀
It is represented by Therefore, the threshold value is expressed as a function F (Δt) of the elapsed time Δt. Assuming that the hash value (output index value) obtained from the flow is between Index 0 and Index X in the hash space, the flow can be changed between old and new by the output index and the time-varying threshold value F (Δt). Judge whether to assign to the sorting algorithm. As a function F representing the threshold value, F (0) has a value of Index 0, a predetermined time is T, and F (T) has a value of Index X, and during that time, for example, a straight line according to the elapsed time Δt. Can be used.

  In the time variation switching, dynamically increasing or decreasing the parallel number of packet processing units for flexible process allocation and power saving can be implemented with less influence on packet processing performance.

  In addition, when the time variation switching is biased in the number of distribution flows to the packet processing unit, the distribution method is more evenly distributed while reducing the burden of allocation change between the flow and the packet processing unit in the distribution process. Can be used to change When a processing amount deviation occurs between the packet processing units, the deviation is detected by the control unit 112. Based on the detection, the control unit 112 changes the algorithm in the flow distribution (for example, changes in the hash function used to obtain the hash value, used when collecting the flows from the hash value so that the bias does not occur more. The distribution unit 107 is instructed. In response to this instruction, the distribution unit 107 distributes the flow between the old and new distribution algorithms so that the flow allocated to the new distribution algorithm gradually increases using a time-varying threshold value.

  In the configuration shown in FIG. 4, when switching the allocation according to the parallel number change instruction, the control unit 112 is configured to reload the processor 111 with a packet processing program for changing the parallel number of processing cores. The distribution unit 107 is instructed to change the distribution destination. If you want to reduce the number of processing cores, reload the program after the change of the allocation destination is completed, and for the bridge that increases the number of processing cores, start the allocation destination change after the completion of the program reload. .

  When switching distribution based on the occurrence of flow bias, the number of flows allocated to each processing core by the distribution unit 107 is fed back to the control unit 112, and the control unit 112 provides feedback information. For example, a bias is detected by applying a threshold value, and the distribution destination change is started when the bias is detected. Alternatively, the load status (for example, CPU usage rate) of each core may be collected by the control unit 112, and a bias may be detected by applying a threshold value to the load status, for example.

  Next, switching of the distribution algorithm by redistribution will be described.

  FIG. 6 is a diagram for explaining flow redistribution when a failure occurs in a specific packet processing device (processing core). FIG. 6A shows a state where four packet processing units (cores 1 to 4) are performing packet processing in the packet processing apparatus shown in FIG. At this time, as shown in FIG. 6B, the flow is distributed to the core 1, the core 2, and the core 4, for example. Here, it is assumed that a failure has occurred in the core 4. Then, the control unit 112 detects the occurrence of the failure ([1] in FIG. 6), and in response to the detection of the occurrence of the failure, the control unit 112 instructs the distribution unit 107 to start redistribution and to use a distribution algorithm. It instructs ([2] in FIG. 6). Specifically, the redistribution judgment is made as to whether redistribution is required for the flow allocated by the existing distribution algorithm, and the flow allocated to the packet processing unit (in this case, core 4) in which the failure has occurred. , The control unit 112 instructs the distribution unit 107 to distribute the packet processing units (cores 1 to 3 in this case) that have not failed. As a result, as shown in FIG. 6C, the distribution unit 107 determines that the flow that was supposed to go to the core 4 requires redistribution as a result of the redistribution determination. Try to distribute to another core. Assuming that the flow is distributed as shown in (B) of FIG. 6 before the failure occurs, the flow directed to the core 4 is distributed to the core 3 as shown in (D) of FIG. The occurrence of a failure can be easily detected by the control unit 112 constantly monitoring the normality of the processing core.

  In this way, when a failure occurs in a specific packet processing unit, by switching to a distribution that looks back at the flow except for the packet processing unit in which the failure has occurred, the flow in the distribution processing and the packet processing unit Packet processing for all flows can be continued while reducing the burden of allocation change and keeping the impact on packet processing performance low. As a distribution algorithm in the re-distribution, a distribution function is used in which distribution is performed in the packet processing unit except for one. The distribution function is configured by, for example, excluding one packet processing unit from all the packet processing units participating in the parallel processing and storing the remaining packet processing unit at an address address derived from the hash function. Prepare such distribution functions as many as the number of packet processing units, which differ only in which packet processing units have been removed, and select the appropriate distribution function from those depending on the packet processing unit in which a failure has occurred. By using it for redistribution, it is possible to redistribute the flow allocated to the packet processing unit in which a failure has occurred to a normal packet processing unit.

  In addition, switching of the distribution algorithm by redistribution should also be used to eliminate a bias that causes a flow to concentrate on a specific packet processing unit among a plurality of packet processing units. Can do. FIG. 7 shows an example of changing the distribution destination for eliminating the bias. FIG. 7A shows a state in which a large number of flows are assigned to the core 4 among the cores 1 to 4. When the control unit 112 detects such a bias, the control unit 112 instructs the distribution unit 107 to start redistribution and to use a distribution algorithm, as in the case of a failure. As a result, as shown in (B) of FIG. 7, the distribution unit 107 determines that the flow that was supposed to go to the core 4 requires redistribution as a result of the redistribution determination. Flow redistribution is performed so that the flow is distributed throughout the cores 1 to 4 so that the bias is eliminated. Examples of the redistribution algorithm include an algorithm for obtaining a hash value from a flow using different hash functions and performing redistribution. Since the failure does not occur in the core 4, a part of the flow is still allocated to the core 4 due to the redistribution. By switching the distribution algorithm by redistribution, when there is a bias in the number of distribution flows to the packet processing unit, while reducing the burden of changing the allocation of the flow and packet processing unit in the distribution processing, It is possible to change to a distribution method without bias while keeping the influence on the processing performance low.

  For example, detection of the occurrence of a flow deviation is performed by feeding back the number of flows distributed to each processing core by the distribution unit 107 to the control unit 112, and the control unit 112 applies, for example, a threshold value to the feedback information. Can be realized. Alternatively, the control unit 112 may collect the load status (for example, CPU usage rate) of each core and apply a threshold value to the load status, for example.

  Next, switching for predicting the packet processing unit 108 after the change of the distribution destination of each flow and holding each packet processing unit 108 in advance with data necessary for packet processing of each flow based on the prediction will be described. This does not depend on the distribution algorithm, but here, FIG. 4A will be described as an example.

  Based on the flow information of each flow in the data holding unit 109, the distribution algorithm, and the number of packet processing units 108 to be distributed, the control unit 112 determines the number of packet processing units 108 that are the distribution algorithm or the distribution destination. The packet processing unit 108 is predicted in advance to which packet processing unit 108 the flow is changed by the change, and before the distribution unit 107 is instructed to execute the distribution algorithm change. In this cache, data necessary for packet processing of each flow to be processed after the change is held. For example, when a hash is calculated from the 5-tuple of the packet header and the packet processing unit to be distributed is determined from the hash value, the 5-tuple of each flow in the data holding unit 109 and the hash calculation are used. If the allocation method for determining the packet processing unit 108 from the hash function and the hash value is known, it is possible to determine which packet processing unit 108 is to process each flow.

  Furthermore, by reducing the number of flows whose assignment destinations are changed at the same time due to time-variant distribution or redistribution, the amount of data that the packet processing unit 108 should hold in advance can be reduced. The required size of the cache of the unit 108 can be reduced.

  For data related to processing in which table information is updated for each packet, such as shaping of QoS processing, information synchronization is performed between the cache of each packet processing unit 108 and the data holding unit 109, and the distribution destination Prevent inconsistencies in information when changing.

  Next, a specific configuration of the packet processing device according to the present invention will be described. 8 to 13 each show a configuration example of the packet processing device. These packet processing apparatuses are the same as those shown in FIG. 4, and the specific configurations of the distribution unit 107, the plurality of packet processing units 108, and the control unit 112 are clarified. 8 to 12, the packet input unit 103, the packet output unit 104, the distribution unit 107, the plurality of packet processing units 108, and the control unit 112 are included in the packet processing apparatuses in FIGS. 8 to 12. Only the part is shown. In FIG. 13, in addition to the packet input unit 103, the packet output unit 104, the distribution unit 107, the plurality of packet processing units 108, and the control unit 112, a data holding unit 109 is also shown. A plurality of packet processing units 108 constitute a parallel processing unit 113. Further, it is assumed that each packet processing unit (processing core) 108 is assigned a core ID (identification number) that identifies the core.

  FIG. 8 shows an example of the configuration of a packet processing device that executes time variation switching.

  The allocating unit 107 performs a first allocating process on the packets input to the first allocating processing unit 122 and the second allocating processing unit 123 that distribute the flows by different allocating algorithms, and the packet input unit 104 in units of flows. A distribution switching unit 121 that distributes and transfers the data to the unit 122 and the second distribution processing unit 123. The distribution switching unit 121 delivers the flow only to one of the first distribution processing unit 122 and the second distribution processing unit 123 in a steady state where the distribution number is not changed. In particular, the distribution switching unit 121 corresponds to the distribution unit 107 shown in FIGS. 4 and 5 that performs old and new distribution, and in accordance with instructions from the control unit 112, the first distribution processing unit 122 and the second distribution unit. The state where the flow is delivered to one of the minute processing units 123 is gradually changed to the state where the flow is delivered to the other one of them.

  In the example shown here, it is assumed that the core ID is stored at an address address directly derived from the hash value for distribution of the flow to the packet processing unit. Each distribution processing unit 122, 123 obtains a hash value by applying a hash function to the flow, and sends the flow to the packet processing unit 108 corresponding to the core ID stored at the address address represented by the hash value. Is supposed to be distributed. For example, the first distribution processing unit 122 uses a hash function such that any one of three address addresses is derived according to the flow, and the second distribution processing unit 123 uses, for example, four address addresses. Use a hash function that either is derived according to the flow. In this case, the first distribution processing unit 122 performs flow distribution by the three-divided distribution algorithm, and the second distribution processing unit 123 performs flow distribution by the four-divided distribution algorithm. In the distribution unit 107, these hash functions, the number of distribution destinations, the address address to be used, and the core ID are instructed and set from the control unit 112.

  The control unit 112 includes an information collection unit 141 and a distribution instruction unit 142. The information collecting unit 141 collects and manages the information of each packet processing unit 108, extracts the switching trigger of distribution ([1] in FIG. 8), and distributes the switching trigger and the status of the packet processing unit 108. The distribution instruction unit 142 is notified as a switching opportunity notification ([2] in FIG. 8). Examples of the switching trigger of the distribution include a change in the number of parallel processes, a deviation in the number of processing flows in each packet processing unit, and the like. In addition, the control unit 112 instructs the parallel processing unit 113 to change the parallel number. It is also included. The distribution instruction unit 142 determines a distribution method (for example, a distribution destination, a hash function, etc.) from the distribution switching trigger notification of the information collection unit 141, and the first distribution processing unit 122 and the second distribution processing unit. A new distribution method (distribution algorithm) is instructed to the one not operating in 123 ([3] in FIG. 8), and the distribution switching unit 121 is instructed to start distribution switching (FIG. 8). [4]). At that time, the distribution instructing unit 142 refers to the timer and sequentially instructs the new distribution algorithm to change the judgment threshold so that a small number of flows are transferred at regular intervals.

  When receiving an instruction from the distribution instruction unit 142 in this way, the distribution switching unit 121, as shown in FIG. 5, from the hash value obtained from the packet data by the hash function and the determination threshold that varies with time, It is determined for each flow whether the flow is distributed to the packet processing unit 108 by the first distribution processing unit 122 or the second distribution processing unit 123, and the flow is transferred to the corresponding distribution processing unit. Deliver. As described above, the distribution switching unit 121 starts switching the distribution destination from the distribution destination switching start instruction from the control unit 112. The decision threshold fluctuates so that the core of the distribution destination is changed by a small number of flows at regular time intervals, so that the flow delivery destination is operating from the distribution processing section that was operating until then. The flow gradually changes to the distribution processing unit that did not exist, and finally, the entire flow is delivered to the distribution processing unit that has not been operating.

  When the first distribution processing unit 122 corresponds to the three-division distribution algorithm and the second distribution processing unit 123 corresponds to the four-division distribution algorithm, if the parallel number related to packet processing is 3, the first The flow is delivered only to the distribution processing unit 122. Here, for example, when the number of parallel processes related to packet processing increases from 3 to 4, the control unit 112 prepares a new packet processing unit and instructs the distribution unit 107 to start distribution switching. Thereby, switching by the distribution switching unit 121 is started, and only a small number of flows are first passed to the second distribution processing unit 123. The control unit 112 changes the threshold value so that the flow passed to the second distribution processing unit 123 increases after a certain period of time, and repeats this threshold value change. As a result, all the flows are finally passed to the second distribution processing unit 123, and the transition from 3 parallel to 4 parallel is completed.

  FIG. 9 shows another example of the configuration of the packet processing device that executes time variation switching. 9 is the same as that shown in FIG. 8, but the distribution instruction unit 142 of the control unit 112 does not sequentially instruct the determination threshold value to the distribution unit 107. Instead, the distribution changing unit 124 is provided in the distribution unit 107, which is different from that of FIG. When receiving a switching start instruction from the distribution instruction unit 142 ([4] in FIG. 9), the distribution changing unit 124 refers to the built-in timer and instructs the distribution switching unit 121 after a certain period of time has elapsed. An instruction to change the judgment threshold is given ([5] in FIG. 9). This decision threshold change is made by gradually changing the flow delivery destination from the distribution processing unit that was operating to the distribution processing unit that was not operating until then. It is performed so that the entire flow is delivered to the distribution processing unit that was not. In the example shown in FIG. 8, the judgment threshold value is changed by a single instruction from the control unit 112, whereas in the example shown in FIG. Will be.

  FIG. 10 shows a packet processing device based on time variation switching. Assuming a sufficiently large number of virtual nodes compared to the number of packet processing units, a core ID is assigned to each virtual node, and the flow allocation is performed. For the minutes, a virtual node is derived from the hash value of the flow, and the flow is distributed to the packet processing unit according to which core ID the derived virtual node corresponds to. In this configuration, the distribution unit 107 includes a virtual node distribution unit 125 and a real node distribution unit 126. The virtual node distribution unit 125 obtains a hash value by applying a hash function to the packet data, and derives an address address associated with the virtual node from the hash value. Although the number of virtual nodes is sufficiently larger than the number of packet processing units 108 to be distributed, a plurality of flows are still distributed to the same virtual node. When the packet data is distributed to the virtual node, the real node distribution unit 126 searches for the core ID stored at the address address associated with the virtual node, and the searched core ID for the flow of the packet data To the packet processing unit 108 (that is, the real node). The correspondence relationship between the virtual node and the core ID described above is recorded in the real node distribution unit 126.

  The control unit 112 is the same as that shown in FIG. 8, and the allocation changing unit 142 instructs the real node allocation unit 126 to change the allocation of the virtual node and the core ID. ([3] in FIG. 10). At this time, the allocation changing unit 142 refers to the timer and instructs the real node allocating unit 126 to change the processing core of the allocation destination by one virtual node at regular time intervals. As a result, the distribution unit 107 starts switching the distribution destination, and processes each packet of the flow so that the flow allocated based on the new distribution gradually increases as time elapses from the switching start instruction. The distribution to the unit 108 is executed.

  In the example shown in FIG. 10, the allocation change in the real node allocation unit 126 has been performed in accordance with sequential instructions from the control unit 112, but the allocation unit 107 itself performs the allocation change autonomously. May be. FIG. 11 shows a configuration in which the allocating unit 107 itself changes assignments autonomously. In the configuration shown in FIG. 11, the control unit 112 is not provided with an assignment changing unit, and instead, the distribution unit 107 is provided with an assignment changing unit 127. The distribution switching trigger notification is directly sent from the control unit 112 to the distribution unit 107 ([2] in FIG. 11), and the allocation changing unit 127 of the distribution unit 107 receives the distribution switching trigger notification. Then, the real node allocation unit 126 is instructed to change the allocation ([3] in FIG. 11).

  FIG. 12 shows an example of a configuration of a packet processing device configured to redistribute a flow assigned to a specific packet processing unit.

  In the apparatus shown in FIG. 12, the control unit 112 is provided with an information collection unit 141 and a distribution instruction unit 142, similar to those shown in FIGS. The distribution unit 107 includes a first distribution processing unit 128, a second distribution processing unit 129, and a redistribution determination unit 130.

  In the control unit 112, the information collection unit 141 collects and manages information of each packet processing unit 108, and extracts a trigger for performing redistribution. The information collecting unit 141 collects information from each packet processing unit 108, detects a change such as occurrence of a failure or a deviation in the number of flows ([1] in FIG. 12), and determines whether or not to redistribute the flow. It judges and notifies the redistribution opportunity notification to the distribution instruction section 142 together with information on the status of each packet processing section 108 ([2] in FIG. 12). When the distribution instruction unit 142 receives the redistribution opportunity notification, the distribution instruction unit 142 determines a distribution method (for example, distribution destination, hash function, etc.) from the redistribution opportunity notification, and the first distribution The distribution unit (distribution algorithm) is instructed to the processing unit 128 and the second distribution processing unit 129 ([3] in FIG. 12), and the redistribution determination unit 130 is a target of redistribution. An ID and re-distribution start instruction are sent ([4] in FIG. 12).

  The first distribution processing unit 128 distributes packet data to each packet processing function in units of flows using a distribution algorithm (distribution destination, hash function, etc.) based on an instruction from the distribution instruction unit 142, and the core ID of the flow However, the data itself is transferred to the redistribution determination unit 130. Based on the instruction from the distribution instruction unit 142, the second distribution processing unit 129 uses a different distribution algorithm (distribution destination, hash function, etc.) from that in the first distribution processing unit 128 to process the packet data. It has a function of distributing to each packet processing unit 108 in units of flows. When the redistribution determination unit 130 is in a state of performing redistribution in accordance with an instruction from the distribution instruction unit 142, the flow received from the first distribution processing unit 128 is associated with the flow. It is determined whether or not the flow is distributed to the packet processing unit 108 to be redistributed from the core ID. If the flow is not distributed to the packet processing unit to be redistributed, the redistribution determining unit 130 transfers the flow to the packet processing unit 108 indicated by the core ID associated with the flow. On the other hand, when the flow received from the first distribution processing unit 128 is to be redistributed, the redistribution determination unit 130 passes the flow to the second distribution processing unit 129.

  A specific operation of the packet processing apparatus shown in FIG. 12 will be described.

  First, a case where a failure occurs in any of the packet processing units (processing cores) 108 will be described. The second distribution processing unit 129 incorporates in advance a distribution function that distributes the flow to a packet processing unit other than one of the plurality of packet processing units 108 in the parallel processing unit 113. ing. When the control unit 112 detects the occurrence of a failure in any one of the packet processing units 108, an instruction to specify a processing core to be redistributed and start redistribution so as to remove the processing core is issued to the distribution instructing unit 142 is sent to the redistribution determination unit 130 and the second distribution processing unit 129. The flow distributed to the faulty core by the distribution in the first distribution processing unit 128 is reassigned to the processing core in which no failure has occurred by the redistribution determination unit 130 and the second distribution processing unit 129. Will be divided. Such redistribution can be realized, for example, by storing a pointer to an instruction for calling a redistribution function at the address where the core ID of the processing core in which the failure has occurred is stored.

  Next, redistribution when the flow is biased with respect to any one of the packet processing units 108 will be described. The second distribution processing unit holds a hash function different from the first distribution processing unit and a distribution function based on a reference range of packet data. When the redistribution instruction is sent from the distribution instruction unit 142 to the redistribution determination unit 130 and the second distribution processing unit 129, the flow allocated to the processing core in which the bias has occurred is redistributed determination unit 130. Is detected, and the second distribution processing unit 129 distributes the detected flow again, so that the distribution with the corrected flow deviation is performed.

  FIG. 13 is the same as the packet processing apparatus shown in FIG. 8, but when changing the distribution destination, predicts which packet processing unit 108 each flow is allocated to after the change, and the prediction result 1 shows an example of a configuration of a packet processing device configured to perform switching so that data necessary for packet processing of each flow after change is stored in advance in each packet processing unit 108.

  First, the control unit 112 refers to the flow information held in the data holding unit 109, and determines the allocation algorithm managed by the control unit 112 and the number of packet processing units 108 that are the allocation destinations. It is predicted to which packet processing unit 108 each flow will be distributed by changing the minute algorithm or changing the number of packet processing units 108 to be distributed. The control unit 112 is necessary for packet processing of each flow to be processed after the change to each packet processing unit 108 before instructing the distribution unit 112 to change the distribution algorithm. Instructs to hold data (specific data) in advance. As a result, each packet processing unit 108 acquires prior information from the data holding unit 109.

  In addition, when executing the time-variable distribution or the re-distribution, the control unit 112 performs the distribution instruction so that the number of flows whose distribution destinations are simultaneously changed by such distribution switching is not increased. And instructing each packet processing unit 108 to acquire specific data accordingly can reduce the amount of data that the packet processing unit 108 must hold in advance, and the packet processing unit 108 needs to be cached. The size can be reduced.

  For data related to processing in which table information is updated for each packet, such as shaping of QoS processing, information synchronization is performed between the cache of each packet processing unit 108 and the data holding unit 109, and the distribution destination Prevent inconsistencies in information when changing.

  As described above, the embodiment of the present invention has been described by taking as an example a case where packet processing is performed by a plurality of processing cores (packet processing units) in a single processor. However, the embodiment of the present invention is not limited to the above. For example, as shown in FIG. 14, the parallel packet processing method of the present invention is applied even when a plurality of processors are provided in the packet processing apparatus and the packet processing is executed in parallel by the plurality of processors. be able to. The packet processing device 151 shown in FIG. 14 replaces the processing core (packet processing unit 108) in the packet processing device shown in FIG. 4 with the packet processing processor 153, and the distribution unit 107 and the control unit 112 also. These are replaced with independent processors (distribution processor 152 and control processor 155). This configuration is of a multiprocessor configuration, and the internal bus 106 and the memory bus 110 are provided in the packet processing device 151 as interprocessor buses rather than so-called intraprocessor buses. The data holding unit 154 is provided as a common shared memory for the plurality of processors 153. Furthermore, the parallel packet processing method of the present invention can also be applied to packet processing distributed to a plurality of nodes provided in a network. FIG. 15 shows a packet processing system configured as a system including a network. In the system shown in FIG. 15, a distribution node 162, a plurality of packet processing nodes 163, a data server 164, and a control node 165 are provided for the network 161. Packets subject to packet processing are directly input / output from / to the network 161. Here, the distribution node 162 performs the same process as the distribution unit 107, the packet processing node 163 performs the same process as the packet processing unit 108, and the data server 164 has the same function as the data holding unit 109. The control node 165 has the same function as the control unit 112. A system as shown in FIG. 15 is also included in the category of the parallel packet processing apparatus of the present invention.

101, 151 Packet processing device 102, 103 Interface unit 104 Packet input unit 105 Packet output unit 106 Internal bus 107 Distribution unit 108 Packet processing unit 109, 154 Data holding unit 110 Memory bus 111 Processor 112 Control unit 113 Parallel processing unit 121 Minute switching unit 122, 128 First distribution processing unit 123, 129 Second distribution processing unit 124 Distribution change unit 125 Virtual node distribution unit 126 Real node distribution unit 127, 143 Allocation change unit 130 Redistribution determination unit 141 Information Collecting Unit 142 Distribution Instruction Unit 152 Distribution Processor 153 Packet Processing Processor 155 Control Processor 161 Network 162 Distribution Node 163 Packet Processing Node 164 Data Server 165 Control Node

Claims (12)

A parallel packet processing method that distributes flows to a plurality of packet processing means and executes packet processing on the flows in parallel between the plurality of packet processing means,
Preparing different first and second distribution algorithms for distributing the flows to the plurality of packet processing means;
From the state where the first distribution algorithm is applied to the flow, the probability that the second distribution algorithm is applied to the flow for each flow is increased with the passage of time, and the first 2. A parallel packet processing method for switching to a state in which the distribution algorithm of 2 is applied to the flow.   2. The number of the packet processing means that is a distribution destination by the first distribution algorithm is different from the number of the packet processing means that is a distribution destination by the second distribution algorithm. Parallel packet processing method. A parallel packet processing method that distributes flows to a plurality of packet processing means and executes packet processing on the flows in parallel between the plurality of packet processing means,
Preparing different first and second distribution algorithms for distributing the flows to the plurality of packet processing means;
The flow is distributed to the plurality of packet processing means using the first distribution algorithm, and the second flow is assigned to the flow that has been distributed to a specific packet processing means among the distributed flows. A parallel packet processing method that executes redistribution by applying a distribution algorithm.   The parallel packet processing method according to claim 3, wherein the second distribution algorithm is a distribution algorithm that distributes a flow to the plurality of packet processing units excluding the specific packet processing unit. Before changing in the distribution algorithm, determine which of the plurality of packet processing means each flow is changed by the change,
6. The information processing apparatus according to any one of claims 1 to 5, wherein data necessary for processing of each changed flow is held in advance in a corresponding packet processing unit based on a result of the determination, and then the change is performed. The parallel packet processing method described. A parallel packet processing method that distributes flows to a plurality of packet processing means and executes packet processing on the flows in parallel between the plurality of packet processing means,
The distribution algorithm for distributing the flow to the plurality of packet processing means can be changed,
Before changing the distribution algorithm, determine which of the plurality of packet processing means will change each flow by the change,
A parallel packet processing method in which data necessary for the processing of each changed flow is held in advance in a corresponding packet processing unit based on the determination result, and then the change is performed. A plurality of packet processing means for performing packet processing on the flow;
Distribution means for distributing flows to the plurality of packet processing means by different first and second distribution algorithms;
From the state in which the first distribution algorithm is applied to the flow, the probability that the second distribution algorithm is applied to the flow for each flow is increased with the passage of time. Control means for controlling the allocating means so as to switch to a state in which the allocating algorithm is applied to the flow;
A parallel packet processing apparatus.   The number of the packet processing means that is the distribution destination by the first distribution algorithm is different from the number of the packet processing means that is the distribution destination by the second distribution algorithm, and the control means is The parallel packet processing device according to claim 7, wherein the parallel packet processing device is configured to instruct the distribution unit to switch the state when the parallel number is changed in the plurality of packet processing units. A plurality of packet processing means for performing packet processing on the flow;
Distribution means for distributing the flow to the plurality of packet processing means by a first distribution algorithm;
Control means for controlling the allocating means;
With
The distribution means includes a redistribution determination unit that determines whether or not redistribution is necessary based on a distribution destination of the flow with respect to a flow distributed by applying the first distribution algorithm. When it is determined that distribution is necessary, the flow is redistributed to the plurality of packet processing means by a second distribution algorithm different from the first distribution algorithm,
The parallel packet processing device, wherein the control means instructs the distribution means to redistribute a flow that has been allocated to a specific packet processing means by the first distribution algorithm.   The parallel packet processing device according to claim 9, wherein the second distribution algorithm is a distribution algorithm that distributes a flow to the plurality of packet processing units excluding the specific packet processing unit.   The control means holds distribution algorithm information, the number of packet processing means to be distributed and each flow information, so that each flow is processed by any packet processing means after the distribution algorithm is changed. And determining whether to change, and instructing each of the packet processing means to hold necessary data in advance based on the result of the determination, and then controlling the distribution means to change the distribution destination. The parallel packet processing device according to any one of 7 to 10. A plurality of packet processing means for performing packet processing on the flow;
A distribution means for distributing the flow to the plurality of packet processing means by a changeable distribution algorithm;
Holds distribution algorithm information, the number of packet processing means to be distributed and each flow information, and determines which packet processing means each flow changes after the distribution algorithm is changed And, after instructing each packet processing means to hold necessary data in advance based on the result of the determination, a control means for controlling the distribution means to change the distribution destination;
A parallel packet processing apparatus.

Images