JP4041038B2 - Higher layer processing method and system - Google Patents

Description

  The present invention relates to a high-level layer processing method and system, and more specifically, processing is performed based mainly on high-level layer information of layer 4 or higher, such as a server load balancer (SLB) and a firewall (FireWall: FW). The present invention relates to a high layer processing method and system in the field of a high layer processing apparatus.

  Here, the higher layer processing apparatus is placed at the front stage of a boundary between different networks or various servers (Web server / application server) of a corporate data center, and is disposed based on the higher layer information, discarded, relayed, NAT (Network). A process such as a transfer accompanied by address translation such as Address Translation (network address translation) is performed.

  2. Description of the Related Art A load balancing control system that allocates access to a group of servers accessed by clients so that the load is balanced is known.

  As a system of this type, for n servers, the predetermined load measurement items are measured, the load values of the n servers are respectively obtained, and the average load values are calculated in advance. A technique is known in which a hash value for identification information of request data is calculated every time a request is received, a server corresponding to the hash value is determined, a predetermined server is found according to the load value of the server, and access is performed. (For example, refer to Patent Document 1).

  Also, session management is not performed in the previous stage of the high-order layer processing device group arranged in parallel, and the hash value obtained as the hash key is the IP address value of the packet at most, and the distribution table is referenced using the hash value as the search key. In this technique, a high-order layer processing apparatus as a distribution destination is determined, a packet distribution mechanism that accommodates a high-speed interface is arranged, and scalability (variable bandwidth) is ensured.

  FIG. 14 is a diagram for explaining the operation of the conventional system. When a packet is transmitted from the client 1, this packet is notified to the upstream packet distribution device 6 via the Internet 20. The upstream distribution unit 32 of the upstream packet distribution device 6 uses the IP address of the packet as an input of the hash function, calculates a hash value, and determines the ID of the distribution destination higher layer processing device corresponding to this hash value. To do. Reference numeral 21 in the figure is a distribution management table created in this way. The contents are composed of the hash value and the ID of the corresponding higher layer processing apparatus.

  The upstream packet distribution device 3 selects the distribution-destination high-layer processing device 5 based on the hash value generated in this way, and transmits the packet. For example, if the input hash value is 2, the higher layer processing device 5 of #a is selected as the higher layer processing device 5 and the packet is transmitted. The output from each higher layer processing device 5 is connected to the server 2.

  FIG. 15 is a block diagram showing a main table of the distribution management unit of the conventional packet distribution device. (1) is a diagram showing a configuration example of a higher layer processing apparatus status management table. As shown in the figure, it is composed of a higher layer processing device ID, a valid display, a normal / failure display, a session processing state, and a CPU usage rate. The session processing state is composed of the number of sessions and the session processing speed. (2) is a diagram showing a configuration example of a higher layer apparatus-hash value correspondence table. As shown in the figure, it is composed of a higher layer processing device ID, a hash value valid display, a hash value (normal time), and a failure distribution target higher layer processing device ID.

  Currently, a high-speed interface is required, for example, between an iDC (internet data center) and a carrier (service provider) assuming a higher layer service for the purpose of reducing the line cost that the iDC borrows from the carrier side, and flows there. The session data speed does not reach the high interface speed. That is, a number of session data streams are aggregated (aggregated) and flow on the high-speed interface.

  Therefore, there is no need to implement a high-speed interface speed for session management, and the aggregate granularity may be reduced to a speed smaller than the high-speed interface speed, and higher layer processing may be performed for that granularity.

As in the prior art described above, a configuration in which a high-speed interface is accommodated and a packet for distributing the high-speed interface and a higher layer processing block having a processing speed smaller than that of the high-speed interface are developed in parallel in the subsequent stage.
Japanese Patent Laid-Open No. 2001-297046 (pages 4 to 6, FIG. 1)

  In the prior art, in order to reduce the aggregate granularity, the IP address of the packet is used as a hash key, and the aggregate stream is divided based on the hash value generated according to the hash key. That is, if the length of a normal address is long, processing takes time. Therefore, for example, when a hash value is used to compress an IP address (for example, 32 bits) to about 8 bits using a hash function, the amount of data handled is less than that of the session management function, and a distribution destination table using the hash value as a search key The number of table accesses can be reduced. Although overhead of hash calculation processing occurs, it is compensated by the small number of table accesses, and high-speed processing is possible.

  Here, the distribution management table is, for example, a table in which the range of hash output values is equally divided by the number of high-order layer processing devices deployed in parallel, and the high-order layer processing devices are respectively associated with the divided ranges. Yes (see 21 in FIG. 14).

  However, the method for statically determining the distribution destination of the higher layer processing device as described above does not achieve uniformization of the number of distribution sessions, and the concentration and bias of the session processing load on a specific higher layer processing device occurs. Degradation (packet loss / delay) occurs.

  The reason for this is that active distribution is desired for uniform distribution, but in the static distribution rule described above, random addresses in the entire address space used for hash keys including idle (unused) sessions are used. This is because it is only uniform.

  When viewed in an active session, the distribution to a specific higher layer processing apparatus is biased, where service degradation (packet loss / delay) occurs.

  Here, a method is considered in which the activeness is considered for the hash output value to which the session is aggregated. That is, if a distribution-destination high-level layer processing apparatus is dynamically assigned to a hash value newly calculated by receiving an active session, for example, by round robin, an erroneous calculation order of the number of sessions to which the hash value is aggregated Thus, the distribution can be made uniform.

  However, this function, in addition to referring to the table as to whether or not a higher layer processing apparatus is allocated, performs a selection process of a higher layer processing apparatus when not assigned, and a registration process of a correspondence table entry of a selected higher layer processing apparatus and a hash value. The required number of table accesses is close to the session management function, and it is difficult to increase the speed.

An object of the present invention is to construct a scalable high layer processing system for a session stream multiplexed on a high-speed interface. The following factors can hinder scalability:
○ In high-layer processing, the session management function (session recognition / detection / registration / state recognition) is the bottleneck, making it difficult to increase capacity and support high-speed interfaces.
○ The session management function consists of the following processes.
Management table entry search expanded in the external memory / CAM in order to determine whether the session is a new session or an in-service session by referring to the information in the packet up to L4 such as the IP address and TCP / UDP port number. Here, TCP is an abbreviation for Transmission Control Protocol, UDP is an abbreviation for User Datagram Protocol, and CAM is an abbreviation for Content Addressable Memory.
-For a new session, add the corresponding entry to the management table.
In order to determine the end of the session, the TCP flag is monitored and the session state is stored in the management table.

  As described above, the amount of information extracted or manipulated from the packet and the large number of accesses to the table developed on the external memory / CAM are bottlenecks for increasing the processing speed.

  The present invention has been made in view of such a problem, and does not impair the high speed of the packet distribution unit, and evenly utilizes the resources of the group of layer processing apparatuses deployed in parallel to achieve high-speed interface aggregation. It is an object of the present invention to provide a higher layer processing method and system capable of processing a gated session stream.

(1) The invention described in claim 1 is as follows. FIG. 1 is a flowchart showing the principle of the method of the present invention. The present invention is a high-level layer processing system that is located at a boundary between different networks or upstream of various servers in an iDC or a corporate data center and performs processing based on high-level layer information of layer 4 or more of a received packet. In order to ensure scalability, a plurality of high-level layer processing devices are arranged in parallel, and in the high-level layer processing system including packet distribution devices that distribute packets to these high-level layer processing device groups based on information below L3 of received packets, The distribution device calculates from the information in the received packet, and notifies the higher layer processing device together with the transfer packet of the hash value used for the determination of the higher layer processing device that is the distribution destination (step 1). Monitors processing resources for each hash value, retains information about the results, and When a device anticipates congestion or congestion, it selects a session group that hands over processing to another higher layer processing device in units of hash values, and takes over the processing of the selected hash value and the session corresponding to the hash value. Information required for the high-level layer processing device group is notified to the operation management unit of the packet distribution device that monitors the resources of the high-level layer processing device unit (step 2), and the operation management unit takes over the take-over destination high-level layer processing device. And the session takeover information is notified to the higher layer processing device, the congestion state of the higher layer processing is determined, and at the time of congestion, the packet distribution device determines that the packet corresponding to the hash value is congestion The distribution destination is changed and transferred from the higher layer processing apparatus to the transfer destination higher layer processing apparatus (step 3).
(2) The invention described in claim 2 is as follows. FIG. 2 is a principle block diagram of the present invention. The system shown in the figure is a high-level layer processing system that is located at the boundary between different networks or upstream of various servers in an iDC or a corporate data center, and performs processing based on high-level layer information of layer 4 or higher of a received packet. In order to ensure the scalability of the system, a high-level layer processing system including a plurality of high-level layer processing devices arranged in parallel and distributing packets to these high-level layer processing device groups based on information below L3 of received packets is provided. It is composed. In the figure, reference numeral 1 denotes a plurality of clients (Web browsers) 20, which is the Internet connected to these clients 1.

  Reference numeral 10 denotes the system of the present invention connected to the Internet 20. Reference numeral 30 denotes an iDC / corporate data center including the system 10 of the present invention. Reference numeral 3 denotes a pre-stage packet distribution device that is connected to the Internet 20 and calculates a hash value used for determination of the higher layer processing device that is the distribution destination, and is passed to the higher layer processing device together with the transfer packet. . 6 is an L2 switch connected to the upstream packet distribution device 3, 5 is a monitor of processing resources for each hash value, holds information on the result, and the higher layer processing device predicts congestion or congestion In addition, a group of sessions that hand over the processing to another higher layer processing apparatus is selected in units of hash values, and the information necessary to take over the processing of the selected hash value and the session corresponding to the hash value is processed in the higher layer processing. This is a high-level layer processing device (HL processing device) that is handed over to an operation management unit of a packet distribution device that monitors resources in units of high-level layer processing devices. Each higher layer processing apparatus 5 is provided from # 0 to #n.

  Reference numeral 7 denotes an L2 switch connected to the higher layer processing apparatus 5. Reference numeral 4 denotes a latter-stage packet distribution device connected to the L2 switch 7. The operation of the subsequent packet distribution device 4 is basically the same as that of the previous packet distribution device 3. Reference numeral 8 denotes an L2 switch connected to the latter-stage packet distribution device 4, and 2 denotes a Web server connected to the L2 switch 8. A plurality of Web servers 2 exist, and a server farm group 40 is configured by the plurality of Web servers. The iDC / corporate data center 30 has a function of connecting the client 1 and the Web server 2.

  The upstream packet distribution device 3 includes an operation management unit 31 and a distribution unit 32. The operation management unit 31 selects a higher layer processing apparatus as a takeover destination, notifies the higher layer processing apparatus of session takeover information, determines the congestion state of the higher layer processing apparatus, and at the time of congestion, the packet distribution apparatus Has a function of changing the distribution destination and transferring the packet corresponding to the hash value from the higher layer processing apparatus that has determined the congestion to the higher layer processing apparatus that is the takeover destination.

  In the system configured as described above, a packet transmitted from the client 1 to the server 2 enters the upstream packet distribution device 3 of the system 10 of the present invention via the Internet 20. In the packet distribution device 3, the distribution unit 32 transmits a hash value used for determination of the higher layer processing device 5 that is a distribution destination to the predetermined higher layer processing device 5 together with the transfer packet via the L2 switch 6. The layer processing apparatus 5 manages processing resources for each hash value, and passes information necessary for taking over processing of a session corresponding to the hash value to the operation management unit 31.

Upon receipt of this information, the operation management unit 31 determines the congestion state of the higher layer processing device 5, and at the time of congestion, the packet distribution device 3 transmits the packet corresponding to the hash value to the higher layer processing of the takeover destination. The distribution destination is changed and transferred to the device 5.
(3) According to the invention described in claim 3, the information necessary for taking over the processing of the session corresponding to the selected hash value is not transmitted via the operation management unit of the packet distribution device, but by the packet distribution device and the higher layer processing. The high layer processing apparatus acquires information on the takeover destination high layer processing apparatus from the operation management unit via a switch mechanism provided between the apparatus groups, and the apparatus itself transfers the information to the takeover destination apparatus.
(4) The invention according to claim 4 does not transfer information necessary for taking over the processing of the session corresponding to the selected hash value according to the attribute of the selected hash value to the takeover destination higher layer processing apparatus, The packet distribution device judges the congestion state of the higher layer processing device, and at the time of congestion, the packet corresponding to the hash value is only distributed from the higher layer processing judged to be congested to the takeover higher layer processing device. To do.
(5) The invention according to claim 5 does not transfer information necessary for taking over the processing of the session corresponding to the selected hash value to the takeover destination higher layer processing apparatus, and the packet distribution apparatus is The congestion state is determined, the packet corresponding to the hash value is copied at the time of the congestion, and the packet is transferred to the takeover destination higher layer processing apparatus together with the higher layer processing apparatus of the current processing system determined to be the congestion state. To do.
(6) In this invention, L2 deployed between the packet distribution device and the higher layer processing device without transferring the information necessary for taking over the processing of the session corresponding to the selected hash value to the takeover destination higher layer processing device. Using a switch, a VLAN is set in advance between the packet distribution device and a possible pair of two higher layer processing devices, the packet distribution device determines the congestion state of the higher layer processing device, A packet corresponding to the hash value is broadcasted to the VLAN to which the determined higher layer processing apparatus of the current processing system and the takeover destination higher layer processing apparatus belong.
(7) Also, in the present invention, the higher layer processing system provides a VLAN configuration service to the outside, and between the packet distribution device and the higher layer processing device also supports an external VLAN. In the case of providing a VLAN, the VLAN tag value for the outside is assigned a value that is not used as the VLAN tag value for the distribution VLAN for activation at the time of congestion, and the VLAN tag value used at the time of distribution at the time of congestion corresponds to the VLAN service for the outside. The L2 switch converts the VLAN tag value and rewrites it.
(8) In the present invention, the higher layer processing apparatus converts / rewrites the VLAN tag value used at the time of start of congestion when it is congested into a VLAN tag value corresponding to an external VLAN service. .
(9) In the present invention, the higher layer processing system provides a VLAN configuration service to the outside, and a VLAN corresponding to the VLAN for the outside is also provided between the packet distribution device and the higher layer processing device. In the case of providing, a VLAN corresponding to an external VLAN and a VLAN used when allocating activation during congestion are provided hierarchically between the packet distribution device and the higher layer processing device.
(10) In the present invention, the packet distribution device assigns information indicating whether or not the distribution destination has been changed due to congestion of a certain higher layer processing device to the packet, and has received the distribution destination change packet. The processing apparatus is characterized in that it performs differentiation control of processing resource allocation for the distribution destination change session, the in-service session so far, and the new service target session so far.
(11) Further, in the present invention, as a hash value notification method for notifying each packet from the packet distribution device to the higher layer processing device, the hash value is embedded in the ToS field of the IP header.

Here, ToS is an abbreviation for Type of Service (service type).
(12) In the present invention, as a hash value notification method for notifying each packet from the packet distribution device to the higher layer processing device, VLAN is used for packet transfer and embedded in the Priority field of the VLAN header. To do.
(13) Further, in the present invention, as a hash value notification method for notifying each packet from the packet distribution device to the higher layer processing device, in the case of an IPv6 packet, it is embedded in the Flow ID field of the IP header. .

Here, IPv6 is an abbreviation for Internet Protocol version 6.
(14) Further, in the present invention, as a hash value notification method for notifying each packet from the packet distribution device to a higher layer processing device, a transfer packet is encapsulated and embedded in an encapsulation header.
(15) Also, in the present invention, when the higher layer processing apparatus provides a server load distribution service and the packet distribution apparatus performs packet distribution from the client to the server, a hash value notification method for notifying each packet And embedded in the destination IP address field of the packet.
(16) In the present invention, the higher layer processing apparatus uses a session processing speed value as processing resource information to be monitored for each hash value.
(17) Further, in the present invention, the session processing speed value is weighted corresponding to the type of service performed for the session, and used for selecting a hash value.
(18) In the present invention, the higher layer processing apparatus uses a CPU usage rate as processing resource information to be monitored for each hash value.
(19) In the present invention, the hash value selection method of the higher layer processing apparatus is characterized in that the smallest non-zero processing resource monitoring result held for each hash value is selected.
(20) Moreover, in this invention, about the hash value selection method of the said high layer processing apparatus, the thing close | similar to a predetermined value is selected among the processing resource monitoring results currently hold | maintained for every hash value, It is characterized by the above-mentioned. To do.
(21) In the present invention, the processing resource monitoring result held for each hash value is held for each quality of service provided by a predetermined higher layer processing apparatus, and the notification hash value is determined based on the service quality. It is characterized by selecting.
(22) Also, in the present invention, the notification hash value is passed as a plurality of lists instead of one at a time, and the packet distribution device is based on the monitoring resource value of the list and the load status to each higher layer processing device. The distribution destination high-order layer processing apparatus is selected.

(1) According to the first aspect of the present invention, in a system composed of a packet distribution device and a higher layer processing device, the speed of the packet distribution device that cannot be added as a high layer processing device and becomes a speed bottleneck is increased. can do. Further, session load concentration on a specific higher layer processing apparatus due to distribution without seeing a session can be avoided by a dynamic distribution destination changing mechanism, and the entire resources of the higher layer processing apparatus can be effectively utilized.
(2) According to the invention described in claim 2, in a system composed of a packet distribution device and a higher layer processing device, the speed of the packet distribution device that cannot be added as a high layer processing device and becomes a speed bottleneck is increased. can do. In addition, it is possible to avoid session load concentration on a specific higher layer processing apparatus due to distribution without seeing the session by a dynamic distribution destination change mechanism, and to effectively use the entire resources of the higher layer processing apparatus.
(3) According to the invention described in claim 3, in the packet distribution device, the session information transfer process becomes unnecessary and the function becomes light.
(4) According to the invention described in claim 4, when sessions having different service qualities are multiplexed, it is possible to differentiate services between a low priority session that does not require session synchronization and other sessions.
(5) According to the fifth aspect of the present invention, it is possible to omit simultaneous processing of session information between higher layer processes that can occur periodically in advance, and to reduce resource consumption of the L2 switch in session information transfer.
(6) Also, according to the present invention, high-speed processing can be performed as a result of no need for copy processing in the packet distribution unit.
(7) Further, according to the present invention, it is possible to accommodate a plurality of intranet networks in which address collision may occur.
(8) Further, according to the present invention, it is possible to prevent deterioration of the VLAN transfer performance when a general-purpose L2 switch is used.
(9) Further, according to the present invention, by making hierarchies, VLAN tag values can be freely used, the number of usable tags can be increased, and scalability can be maintained.
(10) Further, according to the present invention, it is possible to protect the processing resources of the processing session at the delivery destination.
(11) Further, according to the present invention, the transfer address information field can be used in any direction of the server → client and the client → server without damaging the address information field.
(12) Further, according to the present invention, the transfer address information field can be used in any direction of the server → client and the client → server without any loss.
(13) According to the present invention, the hash value range can be increased as compared with the above (11) / (12).
(14) Further, according to the present invention, the hash value range can be increased as compared with the above (11) / (12).
(15) Further, according to the present invention, the hash value range can be increased as compared with the above (11) / (12).
(16) According to the present invention, the session processing speed observation itself can be realized by hardware by using the session management table, and the load of software processing can be reduced.
(17) Further, according to the present invention, it is possible to accurately estimate the resource consumption status.
(18) Further, according to the present invention, it is possible to estimate the resource status more accurately.
(19) Further, according to the present invention, a hash value can be easily selected.
(20) Further, according to the present invention, it is possible to obtain a reliable control effect (processing amount reduction effect by delivery).
(21) Further, according to the present invention, it is possible to differentiate services.
(22) Further, according to the present invention, the packet distribution unit can perform a delivery process a plurality of times in accordance with the list, and resource allocation is possible while taking into consideration the effect of delivery and the influence on the delivery destination.

  As described above, according to the present invention, the session flow aggregated to the high-speed interface can be obtained by utilizing the resources of the higher-layer processing apparatus group deployed in parallel evenly without losing the high-speed performance of the packet distribution unit. Can be processed.

The present invention intends to effectively use all processing resources by linking a packet distribution device and a higher layer processing device. Here, in applying the solution method, the following contents expressed in the prior art are assumed as preconditions.
-Assumptions regarding the packet distribution unit The distribution method is based on the existing technology, that is, the hash processing base.
-Premises regarding higher layer processing apparatus L2-L4 processing and session management functions are performed by hardware.
・ L5 and above processing is basically performed by software.
-The session management function is used even in processing of L5 or higher. Mainly, software processing is performed in the session start phase, session management information is set in the session management function, and only the session management function is supported in the simple data transfer phase. This means that the (session / second) value can be used as a measure of the software processing load.

The outline | summary of the cooperation used for the solution of this invention and its process is as follows. FIG. 2 is used as the system configuration.
(A) The packet distribution device 3 embeds the hash output value in the packet and notifies the higher layer processing device 5 of it.
(B) The higher layer processing device 5 collects processing resource statistics for each hash value, and selects a hash value (session group) that it wants to request another higher layer processing device to process when it is congested. An entry related to session information related to the hash value is created in the higher layer processing apparatus that requests processing takeover via the operation management unit 31.
(C) The higher layer processing device 5 periodically notifies the packet distribution device 3 of the selected hash value. No notification is given when congestion occurs.
(D) The packet distribution device 3 changes the distribution-destination higher-layer processing device related to the recorded hash value when the notification is not made (not received).

  The hash value of (a) above is originally calculated to select a higher layer processing apparatus as a distribution destination, and the packet distribution apparatus 3 needs a table access for extracting new information during packet transfer. And not. In packet transfer, the hash value embedding process (a) is an overhead, but the processing amount is smaller than the active session management process, and the processing speed can be increased.

  Therefore, in the system composed of the packet distribution device 3 and the higher layer processing device 5, the packet distribution device 3 that cannot be added as the higher layer processing device 5 and becomes a speed bottleneck can be speeded up. In addition, it is possible to avoid session load concentration on the specific higher layer processing apparatus due to the distribution without seeing the session by the dynamic distribution destination changing mechanism of (b) to (d) above, and the entire resources of the higher layer processing apparatus 5 Can be used effectively.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 2 shows the positioning of the higher layer processing system to which the present invention is applied. Configuration examples of a packet distribution device and a higher layer processing device, which are devices to which the function of the present invention is applied, are shown in FIGS. 3 and 4, respectively. 5 to 8 are apparatus level process sequence diagrams. 9 to 12 are diagrams showing examples of the configuration of the main table of the apparatus to which the present invention is applied.

  FIG. 3 is a block diagram illustrating a configuration example of the packet distribution device. In the figure, solid arrows indicate packet data streams, and dashed arrows indicate internal control data streams. In the figure, 51 is a packet receiving unit that receives a packet from the Internet 20, and 52 is a packet identifying unit that receives a packet from the packet receiving unit 51 and identifies the packet. 53 is a distribution destination determination unit that receives the output of the packet identification unit 52 and determines a distribution destination, and 54 is a distribution table. The distribution destination determination unit 53 refers to the distribution table 54 to determine the distribution destination. A packet processing unit 55 receives the output of the distribution destination determination unit 53 and processes the packet. A packet transmission unit 56 receives the output of the packet processing unit 55 and transmits the packet to the outside.

  57 is a high-level layer processing device monitoring unit that monitors a high-level layer processing device based on data from the message type determination unit 60, and 58 is a distribution management unit that is connected to the high-level layer processing device 57 and manages distribution. . Reference numeral 59 denotes a higher layer processing device setting information management unit that is connected to the distribution management unit 58 and manages higher layer processing device setting information that manages setting information of the higher layer processing device. Reference numeral 60 denotes a message type determination unit that is connected to the packet identification unit 52 and determines the message type. Reference numeral 61 denotes a high layer processing apparatus cooperation unit that receives information from the message type determination unit 60 and cooperates with a high layer processing apparatus. is there. Outputs from the higher layer processing device monitoring unit 57 and the higher layer processing device cooperation unit 61 are sent to the packet transmission unit 56. The distribution management unit 58 and the distribution table 54 are interconnected.

  Then, the high-level layer processing device monitoring unit 57, the distribution management unit 58, the high-level layer processing device setting information management unit 59, the message type determination unit 60, and the high-level layer processing device 61 include the operation management unit 31 described with reference to FIG. It is composed. The operation management unit 31, the packet reception unit 51, the packet identification unit 52, the distribution destination determination unit 53, the distribution table 54, the packet processing unit 55, and the packet transmission unit 56 constitute the packet distribution device 3. ing. The operation of the apparatus configured as described above will be described as follows.

  Packets from the client 1 enter the packet receiving unit 51 via the Internet 20. The packet reception unit 51 passes the received packet to the packet identification unit 52. The packet identification unit 52 identifies the type of the input packet and passes it to the message type determination unit 60 or the distribution destination determination unit 53 according to the identification result. The distribution determination unit 53 refers to the distribution table 54 corresponding to the input packet and recognizes a higher layer processing apparatus as a distribution destination.

  Next, the distribution destination determination unit 53 passes the packet to the packet processing unit 55. The packet processing unit 55 inserts the hash value into the packet and notifies the packet transmission unit 56, and the packet transmission unit 56 transmits the packet to the higher layer processing device 5.

  The operation management unit 31 selects the takeover destination higher layer processing device, notifies the higher layer processing device 5 of the session takeover information, determines the congestion state of the higher layer processing device 5, and at the time of congestion the packet distribution The device 3 changes the distribution destination of the packet corresponding to the hash value from the higher layer processing device determined to be congested to the takeover destination higher layer processing device.

  Specifically, the packet identification unit 52 selects and passes the packet that is determined to be processed by the operation management unit 31 to the message type determination unit 60, and the message type determination unit 60 receives the packet from the packet identification unit 52. The packet is further classified into a packet for monitoring the state of the conventional higher layer processing device 5 and a packet related to takeover information, the former to the higher layer processing device monitoring unit 57 and the latter to the higher layer processing device cooperation unit. Pass to 61.

  On the other hand, the distribution management unit 58 is connected to the higher layer processing device monitoring unit 57, the higher layer processing device cooperation unit 61, the higher layer processing device setting information management unit 59, and the distribution table 54 through internal control lines. Based on the failure information of the higher layer processing device 5 detected by the higher layer processing device monitoring unit 57, the distribution table 54 is changed, and the information collected by the higher layer processing device cooperation unit 61 is changed to the higher layer processing device setting information management unit. 59, and also generates session takeover information based on the information of the higher layer processing apparatus setting information management unit 59, and transfers the takeover information to an appropriate higher layer processing apparatus 5 via the higher layer processing apparatus linkage unit 61. Control is performed.

  FIG. 4 is a block diagram illustrating a configuration example of the higher layer processing apparatus. The solid arrow indicates the packet data flow, and the wavy arrow indicates the internal control data flow. In the figure, 41 is a packet receiving unit that receives an output from the packet transmitting unit 56 of the packet distribution device 3, 72 is a packet identifying unit that receives the output of the packet receiving unit 71 and identifies a packet, and 73 is a packet identifying unit 72. Is a packet information extraction unit that receives packet output and extracts packet information.

  74 is a session management unit that receives the output of the packet information extraction unit 73, 75 is a session management table that is mutually connected to the session management unit 74, and 76 is connected to the packet information extraction unit 73 and the session management unit 74 to receive packets. A packet processing unit 77 for processing is a packet transmission unit that is connected to the packet processing unit 76 and transmits a packet to an external device. Reference numeral 78 denotes a higher layer information processing unit interconnected with the session management unit 74, and reference numeral 79 denotes an apparatus management unit interconnected with the higher level information processing unit 78. The device management unit 79 is a device management unit connected to the packet identification unit 72, session management unit 74, and packet transmission unit 77.

  Among these components, the higher layer information processing unit 78 and the device management unit 79 are configured by software. The packet reception unit 71, the packet identification unit 72, the packet information extraction unit 73, the session management unit 74, the session management table 75, the packet The processing unit 76 and the packet transmission unit 77 are configured by hardware. The operation of the apparatus configured as described above will be described as follows.

  The packet receiving unit 71 receives the output packet from the packet transmitting unit 56 of the packet sorting device 3 and passes it to the packet identifying unit 72. The packet identification unit 72 identifies whether the packet is to be processed by the device management unit 79 or the packet to be passed to the packet information extraction unit 73 based on the received packet information, and based on the identification result, the device management unit 79, the packet The packet is passed to one of the information extraction units 73. When passed to the packet information extraction unit 73, the packet information extraction unit 73 extracts the hash value included in the packet together with information necessary for conventional session processing, and notifies the session management unit 74 of it.

  The session management unit 74 acquires, updates, or generates session management information using the session management table 75, and at the same time, records the resource usage state related to session processing using the extracted hash value. Further, the session management unit 74 reads information such as a destination IP address necessary for packet transfer from the session management table 75 and notifies the packet processing unit 76 of the information. In addition, the session management unit 74 is connected to the internal control line with the higher layer processing information processing unit 78, and in response to an instruction based on the processing result of the higher layer processing information processing unit 78 such as determining the transfer destination of a new session, Perform 75 rewrites.

  On the other hand, the packet processing unit 76 processes the packet from the packet information extraction unit 73 based on the information from the session management unit 74. The processed packet is transmitted from the packet transmission unit 77 to the outside of the device.

  5 to 8 showing the flow of processing of the system of the present invention, respectively, the client 1, the packet distribution device (previous stage) 3, the higher layer processing device 5-1, the higher layer processing device 5-2, An exchange between the packet distribution device (second stage) 4 and the server 2 is shown. In the packet distribution device 3, 31 is an operation management unit, and 32 is a distribution unit. In the packet distribution device 4, 41 is an operation management unit, and 42 is a distribution unit.

  FIG. 9 and FIG. 10 are diagrams showing a configuration example of the main table of the distribution management unit of the packet distribution device. (1) shows a configuration example of the distribution destination change hash value table. As shown in the figure, it is composed of a higher layer processing apparatus ID, a hash value valid display, a control target hash value, a resource consumption index value, and a monitoring timer. Here, the resource consumption index values are the session processing speeds of supplementary notes 16 and 17 and the CPU usage rate of supplementary note 18.

  (2) shows a configuration example of a higher layer apparatus-hash value correspondence table. As shown in the figure, from the higher layer processing device ID, the hash value valid display, the control target hash value, the congestion target, higher layer processing device ID, and the used VLAN (virtual LAN) tag value at the time of congestion It is configured. (3) shows a configuration example of the session takeover information control table. As shown in the figure, it is composed of a higher layer processing apparatus ID, a valid display, and a takeover session information storage location pointer.

  FIG. 11 is a diagram illustrating a configuration example of a session management table of the higher layer processing apparatus. (1) shows a configuration example of a session management table. The session management table includes a transmission source IP address, a transmission source port, a protocol ID, a destination IP address, a destination port, a session state, a transfer route, conversion contents, and a hash value. . (2) is a session processing information table for each hash. The per-hash session processing information table includes a hash value and a session processing speed (session / second).

FIG. 12 is a diagram illustrating a configuration example of a main table of the device management unit of the higher layer processing device. (1) shows an information table for each hash. The table includes a hash value, a notification target display, a resource consumption index value, and a service priority. As the resource consumption index value, a session processing speed or a CPU usage rate is used. (2) is a figure which shows the structural example of a service information table (conventional apparatus). The table includes a source IP address, a source port, a protocol ID, a destination IP address, a destination port, a service ID, a service type, and a service priority. Explanation of 2)
The invention according to claims 1 and 2 is located at a boundary between different networks or upstream of various servers in an iDC or a corporate data center, and performs processing based on higher layer information of layer 4 or higher of a received packet. In order to ensure system scalability, a high-order packet processing device comprising a plurality of high-level layer processing devices arranged in parallel and distributing packets to these high-level layer processing device groups based on information below L3 of received packets In the layer processing system, a packet distribution device that is calculated from information in the received packet and used to determine the higher layer processing device that is the distribution destination, together with the transfer packet, notifies the higher layer processing device, and the hash value Each time the processing resource is monitored and the result information is stored, the higher layer processing device is congested or When a congestion is predicted, a session group to which processing is handed over to another higher layer processing apparatus is selected in units of hash values, and information necessary for taking over the processing of the selected hash value and the session corresponding to the hash value To the operation management unit of the packet distribution device that monitors the resources of the higher layer processing device unit of the higher layer processing device group, and selects the higher layer processing device that is the takeover destination, and selects the higher layer processing The session takeover information is notified to the device, the congestion state of the higher layer processing device is determined, and at the time of congestion, the packet distribution device determines that the packet corresponding to the hash value is congested from the higher layer processing device. A high-level layer processing system configured to include an operation management unit that changes and transfers a distribution destination to a high-level layer processing device. That.

  The features of the first and second aspects of the present invention are that, as described above, the packet distribution device 3 cooperates with the higher layer processing device 5 to reduce the transfer performance of the packet distribution device 3 from the distribution bias. It is in the point which can correct. This will be described with reference to FIG. Here, in FIG. 2, the packet distribution device 3 performs the packet distribution in the client 1 → server 2 direction, and the packet distribution device 4 performs the distribution in the server 2 → client 1 direction on the return path.

  Since processing related to the session between the client and the server must be performed by a specific higher layer processing apparatus, it is necessary to use the same high layer processing apparatus as the distribution destination in the forward path and the return path. Therefore, in order to distribute the distribution destinations, the client IP address is used as a hash key in both the upstream and downstream stages.

  That is, by selecting the transmission source address as the forward path and the destination address as the hash key for the return path and calculating the hash value, the same hash value is obtained, and the higher layer processing apparatus corresponding thereto is determined and distributed (FIG. 14). reference). The flows (A-1) and (A-1 ′) in FIG. 5 pass through the same higher layer processing apparatus #a because the hash values 101 illustrated in the figure in both flows in the distribution units 32 and 42 are the same. This is the calculated result.

  The black circles in FIGS. 5 and 6 indicate points through which the flow passes. Among the processing contents described at each point, the underlined bold lines indicate the processing of claim 1.

Next, the internal movement of each device corresponding to FIGS. 5 and 6 will be described with reference to FIGS.
1. The packet distribution device 3 receives the packet of the flow (A-1) by the packet reception unit 51 and passes it to the packet identification unit 52. The packet identification unit 52 is a part that identifies a packet that terminates in the present apparatus. In the conventional apparatus, the packet identification unit 52 identifies an ICMP (Inter Control Message Protocol) packet or a keep alive packet for monitoring the state of a higher layer apparatus, To the operation management unit 31.

(A-1) Since the packet is a relay packet to the server 2, the packet is transferred from the packet identification unit 52 to the distribution destination determination unit 53. Here, the distribution destination determination unit 53 performs the hash key extraction and the hash calculation, determines the distribution destination higher layer processing apparatus 5 using the distribution table 54 (see FIG. 14), and determines the packet processing unit 55. hand over. In the conventional apparatus, the packet processing unit 55 creates a frame to which the MAC address of the distribution-destination higher layer processing apparatus 5 is assigned. Here, after performing the process of embedding the calculated hash value in the packet, the packet communication unit 56 From outside the device.
2. The higher layer processing devices # 0 and # 1 receive the packet received from the packet distribution device 3 at the packet receiving unit 71, and after passing through the packet identification unit 72, at the packet information extraction unit 73, are necessary for conventional session processing. In addition to information (information up to L4), the above 1. Extract the hash value given in. As in the session management table of (1) in FIG. 11, the hash value is also stored in the session entry information. In the case of a new session, the session processing speed of the per-hash session processing information table in (2) of FIG. The resource consumption information for each hash (in this case, session processing speed) is collected.

In the case of a new session, the received packet is registered in the session management table and goes to the packet processing unit 76 → the packet transmission unit 77 → the higher layer information processing unit 78, and then the higher layer information, as in the conventional session processing. The route is routed from the processing unit 78 → the packet receiving unit 71 →... → the packet transmitting unit 77 to the outside via the higher layer information processing unit 78. During the user data transfer phase in the existing session, the packet data is transferred from the packet transmission unit 77 to the outside based on the information in the session management table without passing through the higher layer information processing unit 78. Here, when transferring the packet to the subsequent packet sorting apparatus 4, the hash value assigned to the packet by the packet processing unit 76 is erased / removed.
3. The latter-stage packet distribution device 4 (A-1 ′) The same processing is performed.
4). Now, the higher layer processing apparatuses # 0 and # 1 have the above 2. The device management unit 79 periodically picks up the information for each hash collected in (1), and manages the information for each hash value in the form of the information table for each hash in (1) of FIG. Then, the hash value of the delivery candidate is periodically selected, and this is notified to the packet distribution device 3 by the flow (A-2) in FIG.

The internal notification route at this time is from the device management unit 79 to the packet transmission unit 77. Here, in addition to the hash value, session management table information corresponding to the hash is also delivered. Information on the session management table of FIG. 11 (1) having the entry of the selected hash value is collected and transferred to the packet distribution unit 32.
5. The packet distribution device 3 is the same as the above 4. Is identified by the packet identification unit 52 (by IP address and protocol number, etc.) and passed to the message type determination unit 60. The message type determination unit 60 is also a conventional device. The ICMP packet or the keep alive packet described in the above and the transfer of hash information are identified, and the hash information is passed to the higher layer processing apparatus cooperation unit 61.

  Information necessary at the headquarters is extracted and transferred to the distribution management unit 58. In addition to the table of the conventional device of FIG. 15, the distribution management unit 58 holds the received information together with the information from the other higher layer processing device in the session takeover information control table of (3) of FIG. Here, from the resource information for each higher layer processing device in the higher layer processing device status management table of (1) in FIG. 15, the hash destination that is notified from each higher layer processing device is selected, and FIG. (2) holds the information in the higher layer processing apparatus-hash value correspondence table.

Here, the information is stored and the information of the session management table corresponding to the hash value is transferred to the selected higher layer processing apparatus ((A-3) in FIG. 5). The above 4. The session management table information received in (1) is managed, for example, in the form of the session takeover information control table (3) in FIG.
6). The higher layer processing apparatus # 0 also monitors the resource usage status of the entire apparatus as in the conventional apparatus, and is in a congested state or a state where congestion is predicted (indicated by the asterisk in FIG. 5A). 4 above. The notification of the message to the packet distribution device 3 is refrained.
7). The packet distribution device 3 is the same as the above 6. By the monitoring timer of the distribution destination change hash value information table of (1) in FIG. 9, it is detected that the notification message does not come by (* marked part of the packet distribution device 3 in FIG. 5). If detected, the hash value corresponding to the higher layer processing device 5 and the delivery destination are retrieved from the higher layer device-hash value correspondence table in (2) of FIG. 9, and based on the information, the hash value of FIG. Rewrite the distribution management table entry.
8). Further, the packet distribution device 3 has the above-mentioned 7. As described above, the higher layer processing apparatus # 0 is notified that the congestion is determined. This notification is made through the route of the packet transmission unit 56 from the distribution management unit 58 of FIG. 3 through the higher layer processing apparatus cooperation unit 61 ((B-1) of FIG. 6).
9. The higher layer processing apparatus 5-1 has the above 8. Is received via the route of the packet transmission unit 77 → the device management unit 79 (in the case of an implementation in which a communication session with the packet distribution device 3 is statically registered in the session management unit 74 in advance) and notified. The hash value is excluded from the notification target (the notification target validity flag in the per-hash information table of (1) in FIG. 12 is invalidated).
10. The packet distribution device 3 is the same as the above 6. The change in correspondence between the hash value and the distribution destination is notified to the subsequent packet distribution apparatus 4 ((B-2) in FIG. 6: The operations in both apparatuses are the same as in the case of the higher layer processing apparatus 5-1. So explanation is omitted).
11. Above 7. And 10. By the above process 1, the above 1. And 3. The flow assigned to the higher layer processing apparatus 5-1 in step S5 is assigned to the higher layer processing apparatus 5-2 for both the forward path and the return path. Since the service session information is also in the higher layer processing apparatus 5-2, the service can be continued without disconnecting the session ((B-3) and (B-3 ′) in FIG. 6).

  According to the first and second aspects of the present invention, the hash value is originally calculated for selecting a higher layer processing apparatus as a distribution destination, and the packet distribution apparatus transmits new information at the time of packet transfer. Does not require table access to pull out. In the packet transfer, the hash value embedding process is an overhead, but the processing amount is smaller than that of the active session management apparatus, and the processing speed can be increased.

Therefore, in a system composed of a packet distribution device and a higher layer processing device, the number of packet distribution devices that cannot be added as in the higher layer processing device and becomes a speed bottleneck can be increased. In addition, it is possible to avoid the concentration of session load on a specific higher layer processing device due to the distribution without seeing the session by using the dynamic distribution destination change mechanism described above, and to effectively use the entire resources of the higher layer processing device. Can do.
(Explanation of claim 3)
According to the third aspect of the present invention, the information necessary for taking over the processing of the session corresponding to the selected hash value is not transmitted between the packet distribution device and the higher layer processing device group, but via the operation management unit of the packet distribution device. The higher layer processing apparatus acquires information on the takeover destination higher layer processing apparatus from the operation management unit via the switch mechanism arranged in the apparatus, and the apparatus itself transfers the information to the takeover destination apparatus.

  7 and 8 show an operation sequence of claim 3. The underlined portions of the process descriptions S1, S2, and S3 in FIG. 7 are the differences from FIG. The differences from claims 1 and 2 are (A-2) to (A-2 ') in FIG. Only the hash value is notified in (A-2), the delivery destination device (higher layer processing device 5-2) is notified from the packet distribution device in (A-3), and the higher layer processing device 5-1 The session information corresponding to the corresponding hash value is directly transferred to the higher layer processing apparatus 5-2 without going through the packet distribution apparatus 3. For example, although not shown in FIG. 7, it is performed via the L2 switch 6 of FIG.

According to the second aspect of the present invention, the session information transfer processing according to the first and second aspects is not required in the packet processing device, and the function is lightened. Further, in the configuration in which the L2 switch 6 is provided in order to arrange a large number of higher layer processing devices as shown in FIG. 2 in claims 1 and 2, the invention according to this claim is more effective in the resource of the L2 switch by the session information transfer. Consumption can be reduced.
(Explanation of claim 4)
The invention according to claim 4 does not transfer the information necessary for taking over the processing of the session corresponding to the selected hash value to the takeover destination higher layer processing device, and the packet distribution device is simply a congestion state of the higher layer processing device. When congestion occurs, the packet corresponding to the hash value is only distributed from the higher layer process determined to be congested to the takeover destination higher layer processing apparatus.

  According to the fourth aspect of the present invention, session information is not synchronized between the higher layer processing apparatuses of (A-3) of FIG. 5 and (A-2 ′) of FIG. Meaningful in combination with Appendix 21. For example, when services with different service qualities are multiplexed, session disconnection is permitted for low priority services.

According to the fourth aspect of the present invention, it is possible to differentiate services without performing session synchronization.
(Explanation of claim 5)
The invention according to claim 5 does not transfer information necessary for taking over the processing of the session corresponding to the selected hash value to the takeover destination higher layer processing device, and the packet distribution device indicates the congestion state of the higher layer processing device. When congestion occurs, a packet corresponding to the hash value is copied, and the packet is transferred to the takeover destination higher layer processing apparatus together with the higher layer processing apparatus of the current processing system determined to be in the congestion state.

  The invention according to claim 5 is a system in which session information is not synchronized between higher layer processes in advance as in claims 1 to 3 and session disconnection is not caused as in claim 4. If packets are sent to both sides and processing is performed on both sides as described above, the existing session can be continued with the higher layer processing apparatus of the current processing system, and the new session can be processed with the takeover destination higher layer processing apparatus.

  In the current processing system, control is performed so that a new session of the selected hash is not accepted, and only the new session is accepted at the takeover destination, and the rest are discarded. Here, it is identified and controlled by the discrimination information in the packet distribution device shown in appendix 10 to be described later.

According to the fifth aspect of the present invention, the synchronization process of the session information between the higher layer processing devices that can occur periodically in advance can be omitted, and a large number of the higher layer processing devices are arranged as shown in FIG. In the configuration in which the L2 switch 6 is provided, the present claim can reduce consumption of L2 switch resources due to session information transfer.
(Invention 6)
The invention according to appendix 6 does not transfer the information necessary for taking over the processing of the session corresponding to the selected hash value to the takeover destination higher layer processing device, and deploys L2 between the packet distribution device and the higher layer processing device. Using a switch, a VLAN is set in advance between the packet distribution device and a possible pair of two higher layer processing devices, the packet distribution device determines the congestion state of the higher layer processing device, A packet corresponding to the hash value is broadcasted to the VLAN to which the determined higher layer processing apparatus of the current processing system and the takeover destination higher layer processing apparatus belong.

  The invention described in appendix 6 is, for example, in FIG. 2, VLAN 1 is set for the upstream packet sorting device 3 and the higher layer processing devices # 0 and # 1, and VLAN 2 is set for the preceding packet sorting device 3 and the higher layer processing devices # 2 and # 3. If it is set in advance and it is desired to send a packet to both higher layer processing apparatuses # 0 and # 1 for the same purpose as in claim 5, it is broadcast to VLAN1. In claim 5, copy processing is required in the packet distribution device, but in this method, the packet distribution device does not need to copy packets, and VLAN tag assignment and broadcast address assignment are sufficient. In this case, the tag information value of the VLAN to be added is added to the higher layer apparatus-hash value correspondence table in (2) of FIG.

  In addition, the packet transfer device side of the L2 switch 6 in FIG. 2 has a tag VLAN setting, the higher layer processing device side has a port VLAN setting, and the higher layer processing device side needs to remove one stage of the VLAN tag on the transfer side. That is, it is necessary to set a plurality of port VLANs as physical ports. Alternatively, it is necessary to unconditionally remove the VLAN tag by one stage on the higher layer processing apparatus side.

According to the invention described in appendix 6, the copy processing in the packet distribution device as described in claim 5 is not required, and high-speed processing is possible.
(Invention 7)
In the invention described in Appendix 7, the higher layer processing system provides a VLAN configuration service to the outside, and a VLAN corresponding to the VLAN for the outside is also provided between the packet distribution device and the higher layer processing device. As for the VLAN tag value for external use, a value that is not used as the VLAN tag value of the distribution VLAN for activation at the time of congestion is assigned, and the VLAN tag value corresponding to the VLAN service for external use is assigned from the VLAN tag value used at the time of distribution at the time of congestion activation. The value is converted by the L2 switch and rewritten.

  When the higher layer processing system accommodates a plurality of intranet networks, the IP addresses used in each intranet are independent and there is a possibility of collision. Therefore, it is necessary to set a VLAN for each intranet. In such a case, the present invention aims at the same control as that of claim 6 and reserves a VLAN tag space in advance for distribution at the start of congestion.

According to the seventh aspect of the invention, it is possible to accommodate a plurality of intranet networks in which the address collision can occur.
(Explanation of Appendix 8)
The invention according to appendix 8 is characterized in that a higher layer processing apparatus performs conversion / rewriting from a VLAN tag value used at the time of allocation at the start of congestion to a VLAN tag value corresponding to an external VLAN service.

  The present invention is such that the higher layer processing apparatus itself has a VLAN tag value conversion function necessary for processing continuity of the higher layer processing apparatus in Appendix 7.

According to the eighth aspect of the invention, when the general-purpose L2 switch is used, the VLAN tag value conversion function according to claim 7 may avoid the transfer performance, which may deteriorate.
(Invention of Appendix 9)
In the case of the invention described in appendix 9, the high-level layer processing system provides a VLAN configuration service to the outside, and a VLAN corresponding to an external VLAN is also provided between the packet distribution device and the high-level layer processing device. Is characterized in that a VLAN corresponding to an external VLAN and a VLAN to be used at the time of start-up during congestion are provided hierarchically between the packet distribution device and the higher layer processing device.

  The invention described in the supplementary notes is configured to be hierarchized without using the same VLAN tag space for external use and internal distribution as in the supplementary notes 7 and 8.

According to the invention described in Supplementary Note 9, by layering, the VLAN tag value can be freely used, and the usable number can be increased from that of Supplementary Note 7/8, and scalability can be maintained.
(Invention of Appendix 10)
The invention according to appendix 10 is directed to the high-level layer processing apparatus that receives information indicating whether or not the packet distribution device has changed the distribution destination due to congestion of a certain higher-layer processing device, and has received the distribution-destination change packet Is characterized in that differentiation control of processing resource allocation is performed for these distribution destination change sessions, in-service sessions so far, and new service target sessions so far.

  The invention according to the supplementary note is aimed at protecting resources being processed at the delivery destination when session information is not synchronized between higher layer processing apparatuses as in claim 4. In order to assign identification information (flag), it is necessary to add an assignment instruction field to the distribution management table 21 of FIG.

According to the invention of appendix 10, it is possible to protect the processing resources of the processing session at the delivery destination.
(Invention 11)
The invention according to attachment 11 is characterized in that a hash value notification method for notifying each packet from the packet distribution device to a higher layer processing device is embedded in a ToS field of an IP header.

  The invention described in this supplementary note is such that the hash value and the identification information of claim 10 are embedded in the ToS field.

According to the invention described in appendix 11, the transfer address information field can be used in any of the server-to-client and client-to-server directions without damaging the address information field.
(Invention of Appendix 12)
The invention according to appendix 12 is characterized in that, as a method of notifying a hash value to be notified for each packet from the packet distribution device to the higher layer processing device, VLAN is used for packet transfer and is embedded in the Priority field of the VLAN header.

  The invention described in Supplementary Note 12 uses a VLAN and embeds the hash value and the identification information of Supplementary Note 10 in the Priority field of the VLAN header.

  According to the invention described in Supplementary Note 12, a VLAN frame is used and the hash value and the identification information (flag) of Supplementary Note 10 are embedded in the Priority field of the VLAN header.

According to the invention described in Supplementary Note 12, the hash value can be transmitted without damaging, ie, using, the address information field of the forwarding IP layer in any of the server → client and client → server directions.
(Invention 13)
The invention according to attachment 13 is characterized in that, as a method of notifying the hash value to be notified for each packet from the packet distribution device to the higher layer processing device, in the case of an IPv6 packet, it is embedded in the Flow ID field of the IP header.

  In the invention according to the supplementary note, in the case of an IPv6 packet, a hash value is embedded in the Flow ID field of the IP header.

According to the invention described in Supplementary Note 13, the hash value range can be made larger than in the case of Supplementary Note 11/12.
(Invention of Appendix 14)
The invention according to appendix 14 is characterized in that a transfer packet is encapsulated and embedded in an encapsulated header as a method of notifying a hash value for each packet notified from the packet distribution device to a higher layer processing device.

  The invention described in the supplementary note is characterized in that a transfer packet is encapsulated and a hash value is embedded in an encapsulation head. For example, the destination address is a fixed value indicating a capsule, and the source address is used.

According to the invention described in Appendix 14, the hash value range can be made larger than that in the case of Appendix 11/12.
(Invention of Appendix 15)
The invention according to supplementary note 15 is a hash value notification method for notifying each packet when the higher layer processing apparatus provides a server load distribution service and the packet distribution apparatus performs packet distribution from the client to the server. It is embedded in the destination IP address field of the packet.

  In the case of server load balancing, the destination IP address from the client to the server indicates the address of the virtual server. In advance, on the higher layer processing device side, the ID of the virtual server is determined, a table having a relation between the ID and the IP address is provided, the packet distribution device subtracts the virtual server ID from the virtual IP address, and a hash value and If the virtual server ID is embedded, the service can be performed without any trouble by using the server ID-IP address relation table on the higher layer processing apparatus side.

  However, since there is no field that can be used in the same direction from the server to the client, this method does not have a subsequent packet distribution device as shown in FIG. 13, and the higher layer processing device directly accommodates the server (in the case of FIG. 2). Application is limited to a larger storage division loss than the server.

According to the invention described in appendix 15, the hash value range can be made larger than in the case of claim 11/12.
(Invention of Appendix 16)
The invention according to attachment 16 is characterized in that the higher layer processing device uses a session processing speed value as processing resource information to be monitored for each hash value.

  As shown in the premise regarding the higher layer processing apparatus, the software processing of the higher layer processing apparatus mainly occurs at the start of a session, and this processing part tends to be a bottleneck of relay processing. Therefore, a session processing device indicating the number of new sessions within a predetermined time is used as a monitoring parameter for the resource consumption of the higher layer processing device.

According to the invention described in Supplementary Note 16, the session processing speed observation itself can be realized by hardware by using the session management table, and the load of software processing of the higher layer processing apparatus relating to this processing can be reduced.
(Explanation of Appendix 17)
The invention according to appendix 17 is characterized in that weighting corresponding to a service type performed for a session is performed on a session processing speed value and used for selecting a hash value.

  For example, in the case of server load balancing, there is a process of determining a distribution server based on HTTP information, but the processing load differs depending on the range of information referred to by HTTP (for example, only the file extension or the file name itself). In this case, the supplementary note 16 cannot recognize the difference between these loads. Accordingly, a weight is obtained for each service type in advance, and the session processing speed is determined by multiplying the weight (see the service information table in (2) of FIG. 12).

According to the invention described in appendix 17, it is possible to estimate the resource consumption situation more accurately than in the sixteenth aspect.
(Invention of Appendix 18)
The invention according to attachment 18 is characterized in that the higher layer processing device uses a CPU usage rate as processing resource information to be monitored for each hash value.

  The invention described in this supplementary note uses the CPU usage rate as processing resource information.

According to the invention described in Supplementary Note 18, it is possible to estimate the resource consumption situation more accurately than in the case of Supplementary Note 17.
(Invention of Appendix 19)
The invention according to appendix 19 is characterized in that, for the hash value selection method of the higher layer processing apparatus, the smallest non-zero processing resource monitoring result held for each hash value is selected.

According to the invention described in appendix 19, a hash value can be easily selected.
(Invention of Appendix 20)
The invention according to attachment 20 is characterized in that, for the hash value selection method of the higher layer processing apparatus, a processing resource monitoring result held for each hash value is selected that is close to a predetermined value.

  According to the present invention, a hash value can be selected together with a congestion judgment description of a higher layer processing apparatus.

According to the invention described in appendix 20, a reliable control effect (processing amount reduction effect due to delivery) can be obtained.
(Invention of Appendix 21)
The invention according to attachment 21 holds the processing resource monitoring result held for each hash value for each quality of service provided by a predetermined higher layer processing apparatus, and selects a notification hash value based on the service quality It is characterized by that.

  In the invention described in the supplementary note, a hash value is selected in consideration of service quality. Selection is made using the service priorities in the management table for each hash and the service information table in (1) and (2) of FIG. For example, a service with a low service priority is controlled by the combination of claim 4.

According to the invention described in Appendix 21, services can be differentiated.
(Invention of Appendix 22)
In the invention according to attachment 22, the notification hash value is passed as a plurality of lists instead of one at a time, and the packet distribution device distributes the list based on the monitoring resource value of the list and the load status to each higher layer processing device. A higher layer processing apparatus is selected.

  A plurality of candidate hashes are passed to the packet distribution device as a list. After the packet distribution device selects the delivery hash value based on the list, the delivery session information is synchronized.

  According to the invention described in appendix 22, the bucket allocating device can perform a delivery process a plurality of times in accordance with the list, and resource allocation is possible while taking into consideration the effect of delivery and the influence on the delivery destination.

(Supplementary note 1) A high-level layer processing system that is located at a boundary between different networks or upstream of various servers in an iDC or a corporate data center and performs processing based on high-level layer information of layer 4 or higher of a received packet, In order to ensure scalability, a plurality of high-level layer processing devices are arranged in parallel, and in the high-level layer processing system composed of packet distribution devices that distribute packets to these high-level layer processing device groups based on information below L3 of received packets,
The packet distribution device calculates from the information in the received packet and notifies the higher layer processing device together with the transfer packet of the hash value used for the determination of the higher layer processing device that is the distribution destination (step 1).
A high-order layer processing device monitors processing resources for each hash value, retains information on the result, and when the own high-level layer processing device predicts congestion or congestion, a session group hands over the processing to another high-level layer processing device Is selected in units of hash values, and the resources in units of higher layer processing devices of the higher layer processing device group are monitored for the selected hash values and information necessary for taking over the processing of the session corresponding to the hash values. Notify the operation management unit of the packet distribution device (step 2)
The operation management unit selects a takeover destination higher layer processing device, notifies the higher layer processing device of session takeover information, determines the congestion state of the higher layer processing, and at the time of congestion, the packet distribution device The packet corresponding to the hash value is transferred from the higher layer processing device that has been determined to be congested to the higher layer processing device that is the takeover destination (step 3).
A higher layer processing method.

(Supplementary note 2) A high-level layer processing system that is located at a boundary between different networks or upstream of various servers in an iDC or a corporate data center and performs processing based on high-level layer information of layer 4 or higher of a received packet, In order to ensure scalability, a plurality of high-level layer processing devices are arranged in parallel, and in the high-level layer processing system composed of packet distribution devices that distribute packets to these high-level layer processing device groups based on information below L3 of received packets,
A packet distribution device that calculates from the information in the received packet and notifies the higher layer processing device together with the forward packet the hash value used for the determination of the higher layer processing device that is the distribution destination;
A processing resource is monitored for each hash value, information on the result is retained, and when the own higher layer processing device predicts congestion or congestion, a session group that passes processing to another higher layer processing device in units of hash values A packet distribution device that monitors the resources of each higher layer processing device of the higher layer processing device group, and selects the selected hash value and information necessary for taking over the processing of the session corresponding to the hash value. A higher layer processing device to notify the operation management unit;
Selects the takeover destination higher layer processing device, notifies the higher layer processing device of the session takeover information, determines the congestion state of the higher layer processing, and at the time of congestion, the packet distribution device corresponds to the hash value. An operation management unit that changes the distribution destination and transfers the packet from the higher layer processing apparatus that has determined the congestion to the higher layer processing apparatus that is the takeover destination;
A higher layer processing system.

  (Additional remark 3) The information which is required to take over the processing of the session corresponding to the selected hash value is not provided via the operation management unit of the packet distribution device, but is provided between the packet distribution device and the higher layer processing device group The high-level layer processing system according to claim 2, wherein the high-level layer processing apparatus obtains information of the takeover-destination high-level layer processing apparatus from the operation management unit via a mechanism, and the apparatus itself transfers the information to the takeover destination apparatus.

  (Supplementary Note 4) Information necessary for taking over the processing of the session corresponding to the selected hash value is not transferred to the takeover destination higher layer processing device, but the packet distribution device simply determines the congestion state of the higher layer processing device, 4. The high-level layer processing system according to appendix 2 or 3, wherein a packet corresponding to the hash value is only distributed to a takeover destination high-level layer processing apparatus from a high-level layer process determined to be congested at the time of congestion.

  (Supplementary Note 5) The information necessary for taking over the processing of the session corresponding to the selected hash value is not transferred to the takeover destination higher layer processing device, and the packet distribution device determines the congestion state of the higher layer processing device and The packet corresponding to the hash value is copied, and the packet is transferred to the takeover higher layer processing device together with the higher layer processing device of the current processing system determined to be in the congestion state. High layer processing system.

  (Supplementary note 6) The information necessary for taking over the processing of the session corresponding to the selected hash value is not transferred to the takeover destination higher layer processing device, and the L2 switch provided between the packet distribution device and the higher layer processing device is used. The VLAN is set in advance between the packet distribution device and the possible pair of the two higher layer processing devices, and the packet distribution device determines the congestion state of the higher layer processing device and determines that the congestion state is present. 4. The high-level layer processing system according to appendix 2 or 3, wherein a packet corresponding to the hash value is broadcast to a VLAN to which the high-level layer processing device of the processing system and the takeover destination high-level layer processing device belong.

  (Supplementary note 7) When the higher layer processing system provides a VLAN configuration service to the outside, and a VLAN corresponding to an external VLAN is also provided between the packet distribution device and the higher layer processing device The VLAN tag value for the outside is assigned a value that is not used as the VLAN tag value for the distribution VLAN for activation at the time of congestion. From the VLAN tag value used for distribution at the time of activation at the time of congestion, the VLAN tag value corresponding to the VLAN service for the outside is assigned. The high layer processing system according to appendix 6, wherein the L2 switch performs conversion and rewriting.

  (Supplementary note 8) The high-order layer according to supplementary note 7, wherein the high-level layer processing device performs conversion / rewriting from the VLAN tag value used at the time of distribution at the start of congestion to a VLAN tag value corresponding to an external VLAN service. Layer processing system.

  (Supplementary Note 9) When the higher layer processing system provides a VLAN configuration service to the outside, and a VLAN corresponding to an external VLAN is provided between the packet distribution device and the higher layer processing device, the packet The high-layer processing system according to appendix 6, wherein a VLAN corresponding to an external VLAN and a VLAN used when allocating activation during congestion are provided hierarchically between the distribution device and the high-layer processing device.

  (Additional remark 10) The high-order layer processing apparatus which gave the information which shows whether the said packet allocation apparatus changed the allocation destination with the congestion of a certain high-order layer processing apparatus to a packet, and received the allocation destination change packet, The high-order layer according to any one of appendices 2 to 4, characterized in that differentiated control of processing resource allocation is performed for a distribution-destination change session, a session in service so far, and a new session to be serviced so far Processing system.

  (Additional remark 11) As a notification method of the hash value notified for every packet from the said packet distribution apparatus to a high-order layer processing apparatus, it embeds in the ToS field of IP header, The high level in any one of Additional remark 2 thru | or 10 characterized by the above-mentioned Layer processing system.

  (Supplementary note 12) As a hash value notification method for notifying each packet from the packet distribution device to the higher layer processing device, VLAN is used for packet transfer and embedded in the priority field of the VLAN header. A higher layer processing system according to any one of the above.

  (Additional remark 13) As a notification method of the hash value notified for every packet from the said packet distribution apparatus to a high-order layer processing apparatus, in the case of an IPv6 packet, it embeds in the Flow ID field of IP header, Any one of the higher layer processing systems.

  (Supplementary note 14) Any one of Supplementary notes 2 to 10, wherein a transfer value is encapsulated and embedded in an encapsulated header as a method of notifying a hash value to be notified for each packet from the packet distribution device to a higher layer processing device. The higher layer processing system described in 1.

  (Supplementary Note 15) When the higher layer processing apparatus provides a server load distribution service and the packet distribution apparatus performs packet distribution from the client to the server, as a method of notifying the hash value to be notified for each packet, a packet destination 11. The higher layer processing system according to any one of appendices 2 to 10, wherein the higher layer processing system is embedded in an IP address field.

  (Supplementary Note 16) The high layer processing system according to any one of Supplementary Notes 2 to 10, wherein the high layer processing apparatus uses a session processing speed value as processing resource information to be monitored for each hash value.

  (Supplementary note 17) The high-layer processing system according to supplementary note 16, wherein weighting corresponding to a service type to be performed for a session is performed on a session processing speed value and used for selection of a hash value.

  (Supplementary note 18) The high-layer processing system according to any one of supplementary notes 2 to 10, wherein the high-order layer processing apparatus uses a CPU usage rate as processing resource information to be monitored for each hash value.

  (Supplementary note 19) Any one of Supplementary notes 2 to 10, wherein the hash value selection method of the higher layer processing apparatus selects the smallest non-zero processing resource monitoring result held for each hash value. High-layer processing system according to crab.

  (Additional remark 20) About the hash value selection method of the said high-order layer processing apparatus, the thing close | similar to a predetermined value is selected among the processing resource monitoring results currently hold | maintained for every hash value, The additional notes 2 thru | or 10 characterized by the above-mentioned. A higher layer processing system according to any one of the above.

  (Supplementary Note 21) The processing resource monitoring result held for each hash value is held for each quality of service provided by a predetermined higher layer processing apparatus, and a notification hash value is selected based on the service quality The high layer processing system according to any one of appendices 18 to 20.

  (Supplementary note 22) The notification hash value is passed as a plurality of lists instead of one at a time, and the packet distribution apparatus assigns the distribution destination higher layer based on the monitoring resource value of the list and the load status to each higher layer processing apparatus. The high-layer processing system according to appendices 19 to 21, wherein a processing device is selected.

It is a flowchart which shows the principle of this invention method. It is a principle block diagram of the present invention. It is a block diagram which shows the structural example of a packet distribution apparatus. It is a block diagram which shows the structural example of a high level layer processing apparatus. It is a sequence diagram which shows the flow of a process of this invention system. It is a sequence diagram which shows the flow of a process of this invention system. It is a sequence diagram which shows the flow of other processes of this invention system. It is a sequence diagram which shows the flow of other processes of this invention system. It is a figure which shows the structural example of the main table of the distribution process part of a packet distribution apparatus. It is a figure which shows the structural example of the main table of the distribution management part of a packet distribution apparatus. It is a figure which shows the structural example of the session management table of a high level layer processing apparatus. It is a figure which shows the structural example of the main table of the apparatus management part of a high level layer processing apparatus. It is a block diagram which shows one Example of this invention. It is operation | movement explanatory drawing of a conventional system. It is a block diagram which shows the main table of the distribution management part of the conventional packet distribution apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Client 2 Server 3 Previous packet distribution device 4 Rear packet distribution device 5 Higher layer processing device 6 L2 switch 7 L2 switch 8 L2 switch 10 System of the present invention 20 Internet 30 iDC / corporate data center 40 Server form group

Claims (5)

A high-level layer processing system that is located at the boundary between different networks or upstream of various servers in an iDC or corporate data center, and performs processing based on high-level layer information of layer 4 or higher of a received packet, and ensures system scalability Therefore, in a high-level layer processing system comprising a packet distribution device that arranges a plurality of high-level layer processing devices in parallel and distributes packets to these high-level layer processing device groups based on information below L3 of received packets.
The packet distribution device calculates from the information in the received packet and notifies the higher layer processing device together with the transfer packet of the hash value used for the determination of the higher layer processing device that is the distribution destination (step 1).
A high-order layer processing device monitors processing resources for each hash value, retains information on the result, and when the own high-level layer processing device predicts congestion or congestion, a session group hands over the processing to another high-level layer processing device Is selected in units of hash values, and the resources in units of higher layer processing devices of the higher layer processing device group are monitored for the selected hash values and information necessary for taking over the processing of the session corresponding to the hash values. Notify the operation management unit of the packet distribution device (step 2)
The operation management unit selects a takeover destination higher layer processing device, notifies the higher layer processing device of session takeover information, determines a congestion state of the higher layer processing device, and at the time of congestion, the packet distribution device The packet corresponding to the hash value is transferred from the higher layer processing apparatus that has been determined to be congested to the higher layer processing apparatus that is the takeover destination (step 3).
A higher layer processing method. A high-level layer processing system that is located at the boundary between different networks or upstream of various servers in an iDC or a corporate data center, and performs processing based on high-level layer information of layer 4 or higher of a received packet, and ensures system scalability Therefore, in a high-level layer processing system comprising a packet distribution device that arranges a plurality of high-level layer processing devices in parallel and distributes packets to these high-level layer processing device groups based on information below L3 of received packets.
A packet distribution device that calculates from the information in the received packet and notifies the higher layer processing device together with the forward packet the hash value used for the determination of the higher layer processing device that is the distribution destination;
A processing resource is monitored for each hash value, information on the result is retained, and when the own higher layer processing device predicts congestion or congestion, a session group that passes processing to another higher layer processing device in units of hash values A packet distribution device that monitors the resources of each higher layer processing device of the higher layer processing device group, and selects the selected hash value and information necessary to take over the processing of the session corresponding to the hash value. A higher layer processing device to notify the operation management unit;
Selects the takeover destination higher layer processing device, notifies the higher layer processing device of the session takeover information, determines the congestion state of the higher layer processing device, and the packet distribution device supports the hash value at the time of congestion. An operation management unit that changes the distribution destination from the higher layer processing device that has determined the congestion to the higher layer processing device that is the takeover destination and transfers the packet,
A higher layer processing system. Information necessary to take over the processing of the session corresponding to the selected hash value is not via the operation management unit of the packet distribution device, but via the switch mechanism deployed between the packet distribution device and the higher layer processing device group, 3. The high layer processing system according to claim 2, wherein the high layer processing apparatus acquires information on the takeover destination high layer processing apparatus from the operation management unit, and the apparatus itself transfers the information to the takeover destination apparatus. Depending on the attribute of the selected hash value, the packet distribution device does not transfer the information necessary for taking over the processing of the session corresponding to the selected hash value to the takeover destination higher layer processing device, 4. The higher layer according to claim 2, wherein the congestion state is determined, and at the time of congestion, the packet corresponding to the hash value is only distributed to the takeover higher layer processing apparatus from the higher layer processing determined to be congestion. Processing system. The information necessary for taking over the processing of the session corresponding to the selected hash value is not transferred to the takeover destination higher layer processing device, the packet distribution device determines the congestion state of the higher layer processing device, and the 4. The higher layer processing according to claim 2, wherein a packet corresponding to the value is copied, and the packet is transferred to the higher layer processing device of the current processing system determined to be in a congested state to the takeover destination higher layer processing device. system.

Images