CN103532869A - Data flow sharing method and equipment - Google Patents

Data flow sharing method and equipment Download PDF

Info

Publication number
CN103532869A
CN103532869A CN201210226460.7A CN201210226460A CN103532869A CN 103532869 A CN103532869 A CN 103532869A CN 201210226460 A CN201210226460 A CN 201210226460A CN 103532869 A CN103532869 A CN 103532869A
Authority
CN
China
Prior art keywords
port
total flow
network equipment
equipment
self
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201210226460.7A
Other languages
Chinese (zh)
Other versions
CN103532869B (en
Inventor
王�锋
汪洪远
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
New H3C Technologies Co Ltd
Original Assignee
Hangzhou H3C Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou H3C Technologies Co Ltd filed Critical Hangzhou H3C Technologies Co Ltd
Priority to CN201210226460.7A priority Critical patent/CN103532869B/en
Publication of CN103532869A publication Critical patent/CN103532869A/en
Application granted granted Critical
Publication of CN103532869B publication Critical patent/CN103532869B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Data Exchanges In Wide-Area Networks (AREA)
  • Computer And Data Communications (AREA)

Abstract

The present invention discloses a data flow sharing method and equipment. The data flow sharing method is applied for sharing data flow of an equivalent forwarding path of a distributed system. Network equipment is connected with one another through interconnection ports among the equipment. The data flow sharing method comprises the following steps: receiving a counted value of service forwarding port data flow reported by non-control equipment on the equivalent forwarding path by the control equipment in the distributed system, and in combination with the counted value of the service forwarding port on a same equivalent forwarding path, respectively calculating actual total data flow and to-be-allocated total data flow of each network equipment; comparing the actual total data flow and to-be-allocated total data flow of each network equipment by the control equipment, if a fact that the actual total data flow of the network equipment is larger than the to-be-allocated total data flow is found, using the network equipment as shunted equipment, selecting target network equipment for the shunted equipment, and triggering the shunted equipment to forward corresponding data flow to the target network equipment through the interconnection ports between the shunted equipment and the target network equipment according to a difference between the actual total data flow of the shunted equipment and the to-be-allocated data flow.

Description

A kind of traffic sharing method and apparatus
Technical field
The present invention relates to the network equipment Intel Virtualization Technology of communication technical field, relate in particular to a kind of traffic sharing method and apparatus.
Background technology
The technical development of current network device virtualization is extremely rapid, by network equipment Intel Virtualization Technology, many switches can be interconnected at together, forms distributed switching architecture, and as a logic switching entity operation.From the angle of management and configuration, a distributed switching architecture can be regarded a switching equipment as; From the angle of performance, every switch in distributed switching architecture can be formulated local forwarding decision for the second layer on its port or the 3rd laminar flow amount communication service.The typical network virtualization technology of industry mainly comprises IRF (Intelligent Resilient Framework, intelligent elastic framework) and VSS (Virtual Switching System, virtual exchange system) at present.
Important function of network equipment Intel Virtualization Technology has just been to provide across frame polymerizable functional, below by IRF technology, describes the existing local forwarding strategy across frame polymerization in detail.As shown in Figure 1a, distributed system comprises two network equipments, and on two network equipments, configuration service forwards port respectively, and unified condensing together, and flow can forward on port and share in above-mentioned business.
A major issue across frame polymerization technique is exactly how flow forwards, existing IRF and VSS utilization be all local forwarding strategy.As long as there is the port of this equipment in distributed system, the flow flowing into from this equipment can forward port repeat and go out from the business of this equipment.As shown in Figure 1 b, in across frame polymerization situation, because each frame (equipment) inbound traffics are inconsistent, local forwarding strategy causes flow cannot forward in the business of each frame even load sharing between port.
As shown in Fig. 1 c, be another across the traffic sharing situation of frame polymerization, each frame inbound traffics is consistent, but each frame outbound port quantity is different, but also has the problems referred to above.
As can be seen here, need a kind of scheme badly and solve the problem that forwards load balancing between port across frame business.
Summary of the invention
The invention provides a kind of traffic sharing method and apparatus, in order to realize in distributed system each appliance services, forward uniform load between port and share.
For this reason, the embodiment of the present invention adopts following technical scheme:
The embodiment of the present invention provides a kind of traffic sharing method, is applied to the traffic sharing of the forward-path of equal value of distributed system, between the network equipment in described distributed system, by equipment room interconnect port, connects, and the method comprises:
Control appliance in distributed system receives its business on the forward-path of equal value that non-control appliance reports and forwards port flow statistical value, and forward port flow statistical value in conjunction with the business on self same forward-path of equal value, calculate respectively the actual total flow of each network equipment and should distribute total flow;
Described control appliance relatively each network equipment actual total flow and should distribute total flow, if finding that there is the actual total flow of the network equipment is greater than it and should distributes total flow, using this network equipment as by shunting device, for this is by the shunting device select target network equipment, and trigger this by shunting device according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device.
The embodiment of the present invention also provides a kind of traffic sharing method, is applied to the traffic sharing of the forward-path of equal value of distributed system, between the network equipment in described distributed system, by equipment room interconnect port, connects, and the method comprises:
Non-control appliance in distributed system reports the traffic statistics value of its business forwarding port on forward-path of equal value to control appliance;
When described non-control appliance receives after the control command that control appliance sends, according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device; Wherein, described control command is should distribute total flow when actual total flow that control appliance finds that there is the network equipment is greater than it, and this network equipment when by shunting device, is sent by shunting device to this;
When described non-control appliance is from interconnect port receives message, obtain the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, from corresponding port, forward described message.
The embodiment of the present invention also provides a kind of network equipment, be applied to the forward-path traffic sharing of equal value of distributed system, between the network equipment in described distributed system, by equipment room interconnect port, connect, this equipment is the control appliance in distributed system, and described equipment comprises:
Statistical module, for receiving business on the forward-path of equal value that non-control appliance reports, forward port flow statistical value, on the forward-path of equal value reporting according to the non-control appliance receiving, business forwards port flow statistical value, and forward port flow statistical value in conjunction with the business on self same forward-path of equal value, calculate respectively the actual total flow of each network equipment and should distribute total flow;
Load balancing control module, for the actual total flow of each network equipment relatively with should distribute total flow, if finding that there is the actual total flow of the network equipment is greater than it and should distributes total flow, using this network equipment as by shunting device, for this is by the shunting device select target network equipment, and trigger this by shunting device according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device.
The embodiment of the present invention also provides a kind of network equipment, be applied to the traffic sharing of the forward-path of equal value of distributed system, between the network equipment in described distributed system, by equipment room interconnect port, connect, this equipment is the non-control appliance in described distributed system, and this equipment comprises:
Sending module, for reporting the traffic statistics value of its business forwarding port on forward-path of equal value to control appliance;
Load balancing control module, for after receiving the control command of control appliance transmission, according to the actual total flow of self and the difference that should distribute total flow, determines the flow that needs forwarding, and definite traffic sharing model that needs converting flow; Wherein, described control command is should distribute total flow when actual total flow that control appliance finds that there is the network equipment is greater than it, and this network equipment when by shunting device, is sent by shunting device to this;
Packet forwarding module, for the traffic sharing type of determining according to described load balancing control module, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device; And for when from interconnect port receives message, obtain the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, from corresponding port, forward described message.
Compared with prior art, the above embodiment of the present invention has following useful technique effect:
The traffic sharing method that the embodiment of the present invention provides, be applied to the traffic sharing of the forward-path of equal value of distributed system, between each network equipment, by equipment room interconnect port, connect, the outflow of each business forwarding port on forward-path of equal value in control appliance Distribution Statistics formula system, the total flow of calculating the actual total flow of each network equipment and should distributing, if finding that there is the actual total flow of the network equipment is greater than it and should distributes total flow, using this network equipment as by shunting device, for this is by the shunting device select target network equipment, and trigger this by shunting device the mode by load balancing according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device, be greater than under the prerequisite of network equipment port flow mean allocation of the total flow that should distribute meeting actual flow, the business that realized in distributed system forwards uniform load between port and shares.
Accompanying drawing explanation
Fig. 1 a is IRF technology networking schematic diagram of the prior art;
Fig. 1 b is one of traffic sharing schematic diagram across frame polymerization of the prior art;
Fig. 1 c is two of the traffic sharing schematic diagram across frame polymerization of the prior art;
The control appliance that Fig. 2 provides for the embodiment of the present invention triggers the schematic flow sheet of traffic sharing;
The distributed system architecture schematic diagram that Fig. 3 provides for the embodiment of the present invention;
The traffic sharing model schematic diagram that Fig. 4 provides for the embodiment of the present invention;
The structural representation of the control appliance that Fig. 5 provides for the embodiment of the present invention;
The structural representation of the non-control appliance that Fig. 6 provides for the embodiment of the present invention.
Embodiment
The problems referred to above that exist for prior art, the embodiment of the present invention provides the uniform load that business that a kind of traffic sharing scheme that is applied to distributed system (adopting the network equipment of Intel Virtualization Technology) realizes each equipment in distributed system (being the virtual network equipment) forwards between port to share.
A kind of traffic sharing method that the embodiment of the present invention provides, is applied to the traffic sharing of the forward-path of equal value of distributed system, has control appliance in distributed system, and the network equipment except this control appliance is non-control appliance.On control appliance, increased following function: each business in distributed system is forwarded to port and carry out the function of traffic statistics, and trigger actual total flow and be greater than the function that its network equipment that should distribute total flow carries out load balancing.On non-control appliance, increase reporting service and forwarded port flow to the function of control appliance, and by the function of equipment room interconnect port converting flow.
Concrete, non-control appliance can report according to setting cycle the actual flow of its each business forwarding port on forward-path of equal value to control appliance.Control appliance self also can be monitored the actual flow of its business forwarding port on same forward-path of equal value, and the actual flow of the business that can report each non-control appliance forwarding port carries out traffic statistics and calculating with the actual flow that the own service monitoring forwards port, and while found that according to statistical computation the flow of equipment is more, sending controling instruction carries out striding equipment traffic forwarding to trigger this network equipment, to realize the business of each network equipment in distributed system, forwards port flow load balancing.
Referring to Fig. 2, the traffic sharing schematic flow sheet providing for the embodiment of the present invention.In this flow process, each non-control appliance according to the periodic report of setting separately on forward-path of equal value business forward the present flow rate of port, the control appliance also business on the own same forward-path of equal value of monitoring in real time forwards the actual flow of port.On control appliance, also dispose the load balancing cycle, preferred, the time span in load balancing cycle is not less than the report cycle length that business forwards port actual flow.The load balancing cycle arranging on control appliance, while arriving, is carried out following steps:
Step 201, the actual flow that control appliance forwards port according to business on the forward-path of equal value of each equipment (comprising self) carries out traffic statistics operation, comprise: each business on forward-path of equal value of calculating forwards the average discharge of port, and calculates the actual total flow of each network equipment and should distribute total flow according to the average discharge of each business forwarding port on forward-path of equal value.
Wherein, the actual total flow of the network equipment is the cumulative sum that on the forward-path of equal value of this network equipment within the balanced cycle of respective load (an as above equally loaded cycle), each business forwards the actual flow statistical value of port.The computational process that should distribute total flow of the network equipment can be: first control appliance forwards the actual flow of port according to each business on forward-path of equal value, calculate the total flow in the balanced cycle of respective load (an as above equally loaded cycle), then divided by all business on forward-path of equal value, forward the average discharge that port total quantity obtains each business forwarding port on forward-path of equal value by this total flow; Then for each network equipment, by the business of this network equipment, forward the average discharge that port total quantity is multiplied by business forwarding port, what obtain this network equipment should distribute total flow, and concrete calculating can show as following formula:
Flow _ Should be allocated i = ΣFlow _ Actual N × n i . . . . . . . . . . . . . . . . . . ( 1 )
Wherein, Flow_Should be allocated iwhat represent network equipment i should distribute total flow, and ∑ Flow_Actual represents that on the forward-path of equal value of all-network equipment, business forwards port flow statistical value sum, and N represents that all business on forward-path of equal value forward the total quantity of ports, n ithe business that represents network equipment i forwards port number.
Step 202, control appliance is for each network equipment, whether the actual total flow that judges this network equipment is greater than the total flow that this network equipment should distribute, if there is equipment to meet above-mentioned condition, explanation need to be carried out striding equipment traffic forwarding by the flow of this network equipment, proceeds in such cases step 203; If do not have equipment to meet above-mentioned condition, wait for the arrival in next load balancing cycle.Below for convenience of description, the network equipment that meets above-mentioned condition is called by shunting device.
Step 203, control appliance, for determined destination network device (the forwarding operation of the flow that this destination network device is shunted out for bearing) by shunting device, triggers, by shunting device, its flow is transmitted to destination network device by equipment room interconnect port.
Wherein, control appliance is at least to be met the following conditions (being called for convenience of description first condition) by the selected destination network device of shunting device: the actual total flow of destination network device is less than it should distribute total flow.Preferably, control appliance can be from meeting the network equipment of first condition, selects actual total flow and should distribute the network equipment of difference maximum of total flow as destination network device.
After proof load equilibrium treatment, the actual total flow that each business of destination network device forwards port is no more than the total flow that should distribute, and control appliance is for can be met the following conditions (being called for convenience of description second condition) by the selected destination network device of shunting device:
Destination network device should distribute total flow-actual total flow >=by the flow difference Δ of shunting device
Wherein, by the flow difference Δ of shunting device, for this, by the actual total flow of shunting device and its, should be distributed the difference of total flow.
Further, if meet the destination network device of second condition, have a plurality ofly, control appliance can be selected wherein actual total flow and should distribute the network equipment of total flow difference maximum as destination network device.
Further, if larger by the flow difference of shunting device, and any one destination network device all could not independently be born all flows of being shunted, and can use a plurality of destination network devices to share the flow of needs shunting.Such as, the destination network device that meets first condition has a plurality of, but each destination network device does not meet second condition, control appliance can meet a plurality of destination network devices of first condition and select part or armamentarium as destination network device from this, each destination network device is used for sharing partial discharge, thereby answers the part of dispense flow rate to share a plurality of destination network devices to forward by being greater than by shunting device.
If by shunting device, had a plurality of, for each by shunting device, be all that each is by the shunting device select target network equipment in the manner described above.
In this step, if be non-control appliance by shunting device, control appliance, to this by shunting device sending controling instruction, carries out load balancing to trigger by shunting device.In the control command that control appliance sends, carry destination network device sign, and the statistics of control appliance, this statistics can comprise: by the actual total flow of shunting device and its, should be distributed the difference (being designated as Δ) of total flow, the average discharge that business forwards port; Also can only comprise that business forwards the average discharge of port.If control appliance is determined from as by shunting device, omit the step of sending controling instruction, being control appliance is determining from as after by shunting device, according to the relevant statistics of this equipment, adjust the traffic sharing model of this equipment, to the message of respective numbers is transmitted to destination network device by equipment room interconnect port.
Step 204, after being triggered, according to the statistics of wherein carrying and destination network device sign, adjusts the traffic sharing model of this equipment, to corresponding discharge is transmitted to destination network device by equipment room interconnect port by shunting device.
During concrete enforcement, after being triggered by shunting device, by shunting device by and destination network device between interconnect port add in the outbound port list of this equipment, and calculate the actual total flow of this equipment and its difference DELTA that should distribute total flow (for non-control appliance as by the situation of shunting device according to statistics, if comprise this difference DELTA in control command, this step is omitted), the flow that (interconnect port and between destination network device) should forward using this flow difference Δ as this outbound port, calculate the traffic sharing weighted value of each outbound port in the traffic sharing model of this equipment, thereby adjust current traffic sharing model.
Concrete, by shunting device, can be calculated and destination network device between the interconnect port flow (difference DELTA of flow) that should forward, forward the ratio of the average discharge of port with business, and according to this ratio, by setting up the corresponding relation of each outbound port (comprising that business forwards the interconnect port between port and destination network device) and message characteristic value, set up the traffic sharing model of this equipment.Wherein, message characteristic value can obtain in the following manner: by the source IP address of message or object IP address, or out of Memory carries out HASH computing.
After this, the network equipment in distributed system (comprising control appliance or non-control appliance) receives after message, according to its traffic sharing model, carries out message repeating, to realize in its each business, forwards the flow equalization on port.Such as, when above-mentioned, by shunting device, received after message, just can carry out message repeating according to the current traffic sharing model of this equipment.Due in the current traffic sharing model of this shunting device, comprised and destination network device between interconnect port, according to forwarding strategy, have part message and be forwarded to destination network device from the interconnect port of equipment room, destination network device is according to its traffic sharing model, from its business, forward this message of port repeat, thereby realize the traffic forwarding between striding equipment.For example, destination network device, from interconnect port receives the message of being sent by shunting device, is determined the characteristic value of this message, then according to the characteristic value of this message and traffic sharing model, obtain the outbound port information of this message, and forward this message from corresponding outbound port.
In above-mentioned flow process, for non-control appliance as by the situation of shunting device, control appliance is in sending to by the control command of shunting device, entrained statistics can be also the ratio value of this total flow that should be distributed by the flow difference Δ of shunting device (should be distributed the difference of total flow by the actual total flow of shunting device and its) and this equipment (or business forwards port average discharge), like this, by shunting device after receiving this control command, just need not calculate this ratio value again, but according to this ratio value, adjust the traffic sharing model of this equipment.
For by shunting device, due to the outbound port of its traffic sharing model comprise and destination network device between interconnect port, and the traffic sharing weighted value of each outbound port is to need the flow of striding equipment forwarding and the ratio-dependent that should distribute total flow (or business forwards average discharge of port) of this equipment according to this equipment, the corresponding discharge that therefore, this can be received by shunting device according to this traffic sharing model is transmitted to destination network device by equipment room interconnect port.For destination network device, no matter it forwards port or equipment room interconnect port receives message from business, all according to its traffic sharing model, carry out message repeating, thereby forward between port and carry out load balancing in each business of this equipment forward the flow that port and equipment room interconnect port receive from business.
In order to reduce system and the cost of device causing because of frequent starting flow load balance handling process, and the LA Management Room that therefore minimizing brings frequently switches the unsteadiness that shake causes, minimizing striding equipment flow switch takies bandwidth, and the present invention is also optimized on the basis of above-described embodiment.
In a kind of prioritization scheme, control appliance is after carrying out data statistics, when judgement has the actual total flow of the network equipment to be greater than the total flow that this network equipment should distribute, not immediately using this network equipment as by shunting device and trigger load balancing flow process, but further judge whether both flow difference Δs and the ratio of the total flow that should distribute of this network equipment are greater than setting threshold, if, using this network equipment as by shunting device, to trigger by the load balancing flow process of shunting device; Otherwise, do not carry out load balancing.The processing of other step is basic identical with previous embodiment.
Certainly, control appliance also can be when judgement has the actual total flow of the network equipment to be greater than the total flow that this network equipment should distribute, further judge whether both flow difference Δs are greater than setting threshold, if, using this network equipment as by shunting device, to trigger by the load balancing flow process of shunting device; Otherwise, do not carry out load balancing.
In another kind of prioritization scheme, control appliance is preset with threshold value corresponding to each business forwarding port of each network equipment.Control appliance is after carrying out data statistics, when judgement has the actual total flow of the network equipment to be greater than the flow that this network equipment should distribute, not immediately using this network equipment as by shunting device and trigger load balancing flow process, , but further judge in this network equipment that each business forwards the ratio of actual flow of port and the difference of average discharge and this average discharge, whether be greater than default threshold value, if forwarding port, all business of this network equipment all meet above-mentioned condition, using this network equipment as by shunting device, to trigger by the load balancing flow process of shunting device, otherwise, do not carry out load balancing.The processing of other step is basic identical with previous embodiment.
Certainly, control appliance can be also flow value corresponding to the default threshold value of each business forwarding port of each network equipment.Accordingly, control appliance also can be when judgement has the actual total flow of the network equipment to be greater than the flow that this network equipment should distribute, further in this network equipment of judgement, each business forwards the actual flow of port and the difference of average discharge, whether be greater than default threshold value, if, using this network equipment as by shunting device, to trigger by the load balancing flow process of shunting device; Otherwise, do not carry out load balancing.
It should be noted that, in the various embodiments described above, control appliance triggers load balancing judgement and respective handling flow process according to the load balancing cycle of setting, in specific implementation process, control appliance also can be based on user (system manager) instruction, trigger load balancing judgement and respective handling flow process.
For clearer description the above embodiment of the present invention, take the distributed system architecture shown in Fig. 3 below as example, the specific implementation process of the embodiment of the present invention is described.
Referring to Fig. 3, the structural representation of a kind of distributed system providing for the embodiment of the present invention.As shown in Figure 3, distributed system 30 adopts IRF2 (second generation intelligent elastic framework) technology to realize, comprising first network equipment 31 and second network equipment 32.First network equipment 31 comprises that 2 business forward port (the first port and the second port), and second network equipment 32 comprises that 1 business forwards port (the 3rd port).Between second network equipment 32 and first network equipment 31, by equipment room interconnect port, connect, wherein the interconnect port on second network equipment 32 is expressed as the 4th port in the drawings.
First network equipment 31 in this distributed system for carrying out the control appliance of flow load balance, each business that the master control borad of first network equipment 31 receives the business plate report of its business board and second network equipment 32 forwards the actual outflow of port, wherein, the first port of first network equipment 31 and the actual outflow of the second port are respectively 500M, and the actual outflow of the 3rd port of second network equipment 32 is 1G.
When the load balancing cycle arrives, the actual outflow sum that first network equipment 31 calculates all business forwarding ports is 2G, and the average discharge that calculates each business forwarding port is 666M, the total flow that first network equipment 31 should distribute is 1.33G, and the total flow that second network equipment 32 should distribute is 666M.
The actual flow (1G) of the master control borad judgement second network equipment 32 of first network equipment 31 is greater than the total flow 666M that it should distribute, and the actual flow of oneself (1G) is less than the total flow (1.33G) that should distribute, therefore, using second network equipment 32 as by shunting device, to second network equipment 32 sending controling instructions, to trigger it, carry out load balancing.This control command comprises: the actual total flow of second network equipment 32 and the flow difference (333M) that should distribute total flow and the ratio value 1: 2 that should distribute total flow (666M), and the sign of first network equipment 31.
Second network equipment 32 receives after this control command, the equipment room interconnect port (the 4th port) of itself and first network equipment 31 is joined in outbound port list, and according to this ratio value, each outbound port (third and fourth port) of this equipment and the corresponding relation of message characteristic value are set.If message characteristic value comprises 1~6, the corresponding relation of message characteristic value and each outbound port can pass through shown in the traffic sharing model shown in Fig. 4 and table 1:
The corresponding relation of table 1, message characteristic value and outbound port
Outbound port Message characteristic value
The 3rd port 1,3,5,6
The 4th port (with the interconnect port of first network equipment) 2,4
Second network equipment 32 receives after message, determines the characteristic value of this message, according to the corresponding relation of message characteristic value and outbound port, determines the outbound port that message characteristic value is corresponding and E-Packets.Wherein, if the outbound port inquiring is the 4th port, from the 4th port, forward the packet to first network equipment 31.
First network equipment 31 receives after message and (comprises from interconnect port), determines message characteristic value, according to message characteristic value, and by the corresponding relation of query message characteristic value and outbound port, this message is forwarded to port repeat from corresponding service.
Technical conceive based on identical, the embodiment of the present invention also provides a kind of network equipment.
Referring to Fig. 5, one of structural representation of the control appliance providing for the embodiment of the present invention., be applied to the traffic sharing of the forward-path of equal value of distributed system, between the network equipment in this distributed system, by equipment room interconnect port, connect, this equipment is the control appliance in distributed system, this equipment comprises:
Statistical module 51, for receiving business on the forward-path of equal value that non-control appliance reports, forward port flow statistical value, on the forward-path of equal value reporting according to the non-control appliance receiving, business forwards port flow statistical value, and forward port flow statistical value in conjunction with the business on self same forward-path of equal value, calculate respectively the actual total flow of each network equipment and should distribute total flow;
Load balancing control module 52, for the actual total flow of each network equipment relatively with should distribute total flow, if finding that there is the actual total flow of the network equipment is greater than it and should distributes total flow, using this network equipment as by shunting device, for this is by the shunting device select target network equipment, and trigger this network equipment according to the actual total flow of self and should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device.
Load balancing control module 52 specifically for, from actual total flow, be less than its network equipment that should distribute total flow among, for described by the shunting device select target network equipment.
Load balancing control module 52 specifically for, select actual total flow with should distribute the network equipment of the difference maximum of total flow, as destination network device.
Load balancing control module 52 also for, in the actual total flow that finds that there is the network equipment, be greater than after it should distribute total flow, judge the actual total flow of this equipment and should distribute the poor of total flow, with the ratio that should distribute total flow, whether be greater than setting threshold, or judge the poor of actual flow statistical value that each business on the forward-path of equal value of this equipment forwards port and average discharge, whether the ratio with the average discharge statistical value of corresponding service forwarding port, be greater than setting threshold; Load balancing control module 52 is using this network equipment as by the operation of shunting device, and the result that occurs in judgement is in the situation that is.
Load balancing control module 52 specifically for, if this equipment is using self as by shunting device, it is self select target network equipment, and according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device; If this equipment is using non-control appliance as by shunting device, be that this is by the shunting device select target network equipment, and pass through to this by shunting device sending controling instruction, trigger this by shunting device according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device.
When this equipment using self when by shunting device, load balancing control module 52 specifically for, using and destination network device between interconnect port as outbound port, add in the outbound port list of self, and according to the ratio of business forwarding port average discharge on the actual total flow of self and the difference that should distribute total flow and forward-path of equal value, the corresponding relation that each port and message characteristic value in self outbound port list are set, this process is the process that generates traffic sharing model.
Described equipment also comprises packet forwarding module 53, packet forwarding module 53 specifically for, when receiving message from load balancing control module 52, obtain the characteristic value of message, and according to the corresponding relation (traffic sharing model) of each port in self outbound port list and message characteristic value, from corresponding port, E-Packet.
Described equipment also comprises packet forwarding module 53, for when this equipment is using non-control appliance as by shunting device, and when self being chosen as by the destination network device of shunting device, from interconnect port receives message, obtain the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, from corresponding port, forward described message.
Statistical module 51 specifically for, by following formula computing network equipment, should distribute total flow:
Flow _ Should be allocated i = ΣFlow _ Actual N × n i
Wherein, Flow_Should be allocated iwhat represent network equipment i should distribute total flow, and ∑ Flow_Actual represents that on the forward-path of equal value of all-network equipment, business forwards port flow statistical value sum, and N represents that all business on forward-path of equal value forward the total quantity of ports, n ithe business that represents network equipment i forwards port number.
Technical conceive based on identical, the embodiment of the present invention also provides a kind of network equipment.
Referring to Fig. 6, two of the structural representation of the control appliance providing for the embodiment of the present invention., be applied to the traffic sharing of the forward-path of equal value of distributed system, between the network equipment in this distributed system, by equipment room interconnect port, connect, this equipment is the non-control appliance in this distributed system, this equipment comprises:
Sending module 61, for reporting the traffic statistics value of its business forwarding port on forward-path of equal value to control appliance;
Load balancing control module 62, for after receiving the control command of control appliance transmission, according to the actual total flow of self and the difference that should distribute total flow, determines the flow that needs forwarding, and definite traffic sharing model that needs converting flow; Wherein, described control command is should distribute total flow when actual total flow that control appliance finds that there is the network equipment is greater than it, and this network equipment when by shunting device, is sent by shunting device to this.
Packet forwarding module 63, for the traffic sharing model of determining according to load balancing control module 62, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device; And for from interconnect port receives message, obtain the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, from corresponding port, forward described message.
Load balancing control module 62 specifically for, after receiving the control command of control appliance transmission, using and destination network device between interconnect port as outbound port, add in the outbound port list of self, and according to the ratio of business forwarding port average discharge on the actual total flow of self and the difference that should distribute total flow and forward-path of equal value, the corresponding relation of each port and message characteristic value in self outbound port list is set;
Through the above description of the embodiments, those skilled in the art can be well understood to the mode that the present invention can add essential general hardware platform by software and realize, and can certainly pass through hardware, but in a lot of situation, the former is better execution mode.Understanding based on such, the part that technical scheme of the present invention contributes to prior art in essence in other words can embody with the form of software product, this computer software product is stored in a storage medium, comprise that some instructions are with so that a station terminal equipment (can be mobile phone, personal computer, server, or the network equipment etc.) carry out the method described in each embodiment of the present invention.
The above is only the preferred embodiment of the present invention; it should be pointed out that for those skilled in the art, under the premise without departing from the principles of the invention; can also make some improvements and modifications, these improvements and modifications also should be looked protection scope of the present invention.

Claims (20)

1. a traffic sharing method, is applied to the traffic sharing of the forward-path of equal value of distributed system, between the network equipment in described distributed system, by equipment room interconnect port, connects, and it is characterized in that, the method comprises:
Control appliance in distributed system receives its business on the forward-path of equal value that non-control appliance reports and forwards port flow statistical value, and forward port flow statistical value in conjunction with the business on self same forward-path of equal value, calculate respectively the actual total flow of each network equipment and should distribute total flow;
Described control appliance relatively each network equipment actual total flow and should distribute total flow, if finding that there is the actual total flow of the network equipment is greater than it and should distributes total flow, using this network equipment as by shunting device, for this is by the shunting device select target network equipment, and trigger this by shunting device according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device.
2. the method for claim 1, is characterized in that, described control appliance be this by the shunting device select target network equipment, be specially:
Described control appliance is less than from actual total flow among its network equipment that should distribute total flow, for described by the shunting device select target network equipment.
3. method as claimed in claim 2, is characterized in that, described control appliance is selected actual total flow and should be distributed the network equipment of the difference maximum of total flow, as destination network device.
4. the method for claim 1, it is characterized in that, the actual total flow that described control appliance finds that there is the network equipment is greater than after it should distribute total flow, also comprise: judge the actual total flow of this network equipment and should distribute the poor of total flow, with the described ratio that should distribute total flow, whether be greater than the step of setting threshold, or judge that each business on the forward-path of equal value of this network equipment forwards the poor of the actual flow statistical value of port and average discharge, forward the ratio of the average discharge statistical value of port with corresponding service, whether be greater than the step of setting threshold;
Described control appliance is using this network equipment as by the operation of shunting device, and the result that occurs in described determining step is in the situation that is.
5. the method as described in one of claim 1-4, it is characterized in that, if described control appliance is using self as by shunting device, it is self select target network equipment, and according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device;
If described control appliance is using non-control appliance as by shunting device, be that this is by the shunting device select target network equipment, and pass through to this by shunting device sending controling instruction, trigger this by shunting device according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device.
6. method as claimed in claim 5, is characterized in that, described control appliance is according to the actual total flow of self and should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device, be specially:
Described control appliance using and destination network device between interconnect port as outbound port, add in the outbound port list of self, and according to the ratio of business forwarding port average discharge on the actual total flow of self and the difference that should distribute total flow and forward-path of equal value, the corresponding relation of each port and message characteristic value in self outbound port list is set;
Described control appliance receives after message, obtains the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, forwards described message from corresponding port.
7. method as claimed in claim 5, it is characterized in that, if described control appliance is using non-control appliance as by shunting device, and will self be chosen as by the destination network device of shunting device, when described control appliance is from interconnect port receives message, also comprise: obtain the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, from corresponding port, forward described message.
8. the method as described in one of claim 1-4, is characterized in that, described control appliance should distribute total flow by following formula computing network equipment:
Flow _ Should be allocate d i = ΣFlow _ Actual N × n i
Wherein, Flow_Should be allocated iwhat represent network equipment i should distribute total flow, and ∑ Flow_Actual represents that on the forward-path of equal value of all-network equipment, business forwards port flow statistical value sum, and N represents that all business on forward-path of equal value forward the total quantity of ports, n ithe business that represents network equipment i forwards port number.
9. a traffic sharing method, is applied to the traffic sharing of the forward-path of equal value of distributed system, between the network equipment in described distributed system, by equipment room interconnect port, connects, and it is characterized in that, the method comprises:
Non-control appliance in distributed system reports the traffic statistics value of its business forwarding port on forward-path of equal value to control appliance;
When described non-control appliance receives after the control command that control appliance sends, according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device; Wherein, described control command is should distribute total flow when actual total flow that control appliance finds that there is the network equipment is greater than it, and this network equipment when by shunting device, is sent by shunting device to this;
When described non-control appliance is from interconnect port receives message, obtain the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, from corresponding port, forward described message.
10. method as claimed in claim 9, is characterized in that, described according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between be transmitted to destination network device, be specially:
Described by shunting device using and destination network device between interconnect port as outbound port, add in the outbound port list of self, and according to the ratio of business forwarding port average discharge on the actual total flow of self and the difference that should distribute total flow and forward-path of equal value, the corresponding relation of each port and message characteristic value in self outbound port list is set;
When described, by shunting device, received after message, obtain the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, from corresponding port, forward described message.
11. 1 kinds of network equipments, be applied to the forward-path traffic sharing of equal value of distributed system, between the network equipment in described distributed system, by equipment room interconnect port, connect, this equipment is the control appliance in distributed system, it is characterized in that, described equipment comprises:
Statistical module, for receiving business on the forward-path of equal value that non-control appliance reports, forward port flow statistical value, on the forward-path of equal value reporting according to the non-control appliance receiving, business forwards port flow statistical value, and forward port flow statistical value in conjunction with the business on self same forward-path of equal value, calculate respectively the actual total flow of each network equipment and should distribute total flow;
Load balancing control module, for the actual total flow of each network equipment relatively with should distribute total flow, if finding that there is the actual total flow of the network equipment is greater than it and should distributes total flow, using this network equipment as by shunting device, for this is by the shunting device select target network equipment, and trigger this by shunting device according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device.
12. network equipments as claimed in claim 11, is characterized in that, described load balancing control module specifically for, from actual total flow, be less than its network equipment that should distribute total flow among, for described by the shunting device select target network equipment.
13. network equipments as claimed in claim 12, is characterized in that, described load balancing control module specifically for, select actual total flow with should distribute the network equipment of the difference maximum of total flow, as destination network device.
14. network equipments as claimed in claim 11, it is characterized in that, described load balancing control module also for, in the actual total flow that finds that there is the network equipment, be greater than after it should distribute total flow, judge the actual total flow of this equipment and should distribute the poor of total flow, with the described ratio that should distribute total flow, whether be greater than setting threshold, or judge that each business on the forward-path of equal value of this equipment forwards the poor of the actual flow statistical value of port and average discharge, forward the ratio of the average discharge statistical value of port with corresponding service, whether be greater than setting threshold; Described load balancing control module is using this network equipment as by the operation of shunting device, and the result that occurs in described judgement is in the situation that is.
15. network equipments as described in one of claim 11-14, it is characterized in that, described load balancing control module specifically for, if this equipment is using self as by shunting device, it is self select target network equipment, and according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device; If this equipment is using non-control appliance as by shunting device, be that this is by the shunting device select target network equipment, and pass through to this by shunting device sending controling instruction, trigger this by shunting device according to the actual total flow of self with should distribute the difference of total flow, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device.
16. network equipments as claimed in claim 15, it is characterized in that, when this equipment using self when by shunting device, described load balancing control module specifically for, using and destination network device between interconnect port as outbound port, add in the outbound port list of self, and according to the ratio of business forwarding port average discharge on the actual total flow of self and the difference that should distribute total flow and forward-path of equal value, the corresponding relation of each port and message characteristic value in self outbound port list is set;
Described equipment also comprises packet forwarding module, described packet forwarding module specifically for, after receiving message, obtain the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, from corresponding port, forward described message.
17. network equipments as claimed in claim 15, it is characterized in that, described equipment also comprises packet forwarding module, for when this equipment is using non-control appliance as by shunting device, and when self being chosen as by the destination network device of shunting device, from interconnect port receives message, obtain the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, from corresponding port, forward described message.
18. network equipments as described in one of claim 11-14, is characterized in that, described statistical module specifically for, by following formula computing network equipment, should distribute total flow:
Flow _ Should be allocate d i = ΣFlow _ Actual N × n i
Wherein, Flow_Should be allocated iwhat represent network equipment i should distribute total flow, and ∑ Flow_Actual represents that on the forward-path of equal value of all-network equipment, business forwards port flow statistical value sum, and N represents that all business on forward-path of equal value forward the total quantity of ports, n ithe business that represents network equipment i forwards port number.
19. 1 kinds of network equipments, be applied to the traffic sharing of the forward-path of equal value of distributed system, between the network equipment in described distributed system, by equipment room interconnect port, connect, this equipment is the non-control appliance in described distributed system, it is characterized in that, this equipment comprises:
Sending module, for reporting the traffic statistics value of its business forwarding port on forward-path of equal value to control appliance;
Load balancing control module, for after receiving the control command of control appliance transmission, according to the actual total flow of self and the difference that should distribute total flow, determines the flow that needs forwarding, and the traffic sharing model of definite flow that need to forward; Wherein, described control command is should distribute total flow when actual total flow that control appliance finds that there is the network equipment is greater than it, and this network equipment when by shunting device, is sent by shunting device to this;
Packet forwarding module, for the traffic sharing model of determining according to described load balancing control module, by corresponding discharge by and destination network device between interconnect port be transmitted to destination network device; And for when from interconnect port receives message, obtain the characteristic value of described message, and according to the corresponding relation of each port in self outbound port list and message characteristic value, from corresponding port, forward described message.
20. network equipments as claimed in claim 19, is characterized in that,
Described load balancing control module specifically for, after receiving the control command of control appliance transmission, using and destination network device between interconnect port as outbound port, add in the outbound port list of self, and according to the ratio of business forwarding port average discharge on the actual total flow of self and the difference that should distribute total flow and forward-path of equal value, the corresponding relation of each port and message characteristic value in self outbound port list is set.
CN201210226460.7A 2012-07-03 2012-07-03 Data flow sharing method and equipment Active CN103532869B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201210226460.7A CN103532869B (en) 2012-07-03 2012-07-03 Data flow sharing method and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201210226460.7A CN103532869B (en) 2012-07-03 2012-07-03 Data flow sharing method and equipment

Publications (2)

Publication Number Publication Date
CN103532869A true CN103532869A (en) 2014-01-22
CN103532869B CN103532869B (en) 2017-02-15

Family

ID=49934557

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201210226460.7A Active CN103532869B (en) 2012-07-03 2012-07-03 Data flow sharing method and equipment

Country Status (1)

Country Link
CN (1) CN103532869B (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015168856A1 (en) * 2014-05-06 2015-11-12 华为技术有限公司 Multiflow transmission method and device
CN105471761A (en) * 2015-08-07 2016-04-06 北京汉柏科技有限公司 Exchange equipment load adjustment method and device
CN105490825A (en) * 2014-09-15 2016-04-13 阿里巴巴集团控股有限公司 Business processing method and device
CN105703928A (en) * 2014-11-25 2016-06-22 中兴通讯股份有限公司 Traffic adjustment method and traffic adjustment device
CN106888474A (en) * 2017-03-16 2017-06-23 新华三技术有限公司 A kind of load sharing method and device
CN109462639A (en) * 2018-10-26 2019-03-12 新华三技术有限公司合肥分公司 Ports-Extending device management method and device
CN110995607A (en) * 2019-12-19 2020-04-10 锐捷网络股份有限公司 Method, device, network equipment and storage medium for transmitting flow
CN112804145A (en) * 2020-12-31 2021-05-14 锐捷网络股份有限公司 Flow statistical method and device based on segmented identification list
CN113691607A (en) * 2021-08-20 2021-11-23 绿盟科技集团股份有限公司 Flow load balance control method and device and electronic equipment
CN115208904A (en) * 2022-06-29 2022-10-18 深圳星云智联科技有限公司 Flow monitoring method and related equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101035089A (en) * 2007-04-24 2007-09-12 中兴通讯股份有限公司 Method and device for improving the performance of the distributed system
CN101977153A (en) * 2010-11-15 2011-02-16 杭州华三通信技术有限公司 Flow regulation method and equipment
CN102065000A (en) * 2009-11-17 2011-05-18 中国移动通信集团公司 Network congestion control method, device and system
CN102447636A (en) * 2012-01-06 2012-05-09 中兴通讯股份有限公司 Load sharing method and device of data center communication equipment

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101035089A (en) * 2007-04-24 2007-09-12 中兴通讯股份有限公司 Method and device for improving the performance of the distributed system
CN102065000A (en) * 2009-11-17 2011-05-18 中国移动通信集团公司 Network congestion control method, device and system
CN101977153A (en) * 2010-11-15 2011-02-16 杭州华三通信技术有限公司 Flow regulation method and equipment
CN102447636A (en) * 2012-01-06 2012-05-09 中兴通讯股份有限公司 Load sharing method and device of data center communication equipment

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105850085B (en) * 2014-05-06 2019-08-27 华为技术有限公司 The method and apparatus of multiple stream transmission
WO2015168856A1 (en) * 2014-05-06 2015-11-12 华为技术有限公司 Multiflow transmission method and device
CN105850085A (en) * 2014-05-06 2016-08-10 华为技术有限公司 Multiflow transmission method and device
CN105490825A (en) * 2014-09-15 2016-04-13 阿里巴巴集团控股有限公司 Business processing method and device
CN105490825B (en) * 2014-09-15 2019-03-19 阿里巴巴集团控股有限公司 A kind of method and device for business processing
CN105703928A (en) * 2014-11-25 2016-06-22 中兴通讯股份有限公司 Traffic adjustment method and traffic adjustment device
CN105471761A (en) * 2015-08-07 2016-04-06 北京汉柏科技有限公司 Exchange equipment load adjustment method and device
CN106888474A (en) * 2017-03-16 2017-06-23 新华三技术有限公司 A kind of load sharing method and device
CN106888474B (en) * 2017-03-16 2021-08-13 新华三技术有限公司 Load sharing method and device
CN109462639A (en) * 2018-10-26 2019-03-12 新华三技术有限公司合肥分公司 Ports-Extending device management method and device
CN109462639B (en) * 2018-10-26 2021-09-07 新华三技术有限公司合肥分公司 Port expansion equipment management method and device
CN110995607A (en) * 2019-12-19 2020-04-10 锐捷网络股份有限公司 Method, device, network equipment and storage medium for transmitting flow
CN112804145A (en) * 2020-12-31 2021-05-14 锐捷网络股份有限公司 Flow statistical method and device based on segmented identification list
CN112804145B (en) * 2020-12-31 2022-06-21 锐捷网络股份有限公司 Flow statistical method and device based on segmented identification list
CN113691607A (en) * 2021-08-20 2021-11-23 绿盟科技集团股份有限公司 Flow load balance control method and device and electronic equipment
CN113691607B (en) * 2021-08-20 2023-06-02 绿盟科技集团股份有限公司 Flow load balancing control method and device and electronic equipment
CN115208904A (en) * 2022-06-29 2022-10-18 深圳星云智联科技有限公司 Flow monitoring method and related equipment

Also Published As

Publication number Publication date
CN103532869B (en) 2017-02-15

Similar Documents

Publication Publication Date Title
CN103532869A (en) Data flow sharing method and equipment
Oljira et al. A model for QoS-aware VNF placement and provisioning
Yaghmaee et al. Priority-based rate control for service differentiation and congestion control in wireless multimedia sensor networks
Tajiki et al. Joint failure recovery, fault prevention, and energy-efficient resource management for real-time SFC in fog-supported SDN
EP3556066B1 (en) Flow control in a virtualized communication network
Marbach Pricing differentiated services networks: Bursty traffic
CN103294521A (en) Method for reducing communication loads and energy consumption of data center
CN106921584A (en) A kind of distributed network flow control method
CN103428103A (en) Link load control method and stacking device
CN108293200B (en) Device throughput determination
CN110831064B (en) Data transmission method and transmission server in multi-channel concurrent transmission system
US20100180033A1 (en) Hybrid Server Overload Control Scheme for Maximizing Server Throughput
CN103825834A (en) Method and network exchange device for adjusting flow load sharing in EVI
CN103873361B (en) Packet transmitting device and method
CN102209030A (en) Service traffic splitting method, device and system
CN109392013A (en) A kind of method of adjustment and communication device of service quality QoS
US9178826B2 (en) Method and apparatus for scheduling communication traffic in ATCA-based equipment
CN103685057A (en) Traffic statistic method and device
CN107332785A (en) A kind of effective discharge control method based on dynamic duty threshold value
CN109922003A (en) A kind of data transmission method for uplink, system and associated component
CN103475595A (en) System and method for overload control of Parlay X gateway
Yi et al. Energy‐aware disaster backup among cloud datacenters using multiobjective reinforcement learning in software defined network
Guan et al. Virtual network embedding supporting user mobility in 5G metro/access networks
Oljira et al. Mdtcp: Towards a practical multipath transport protocol for telco cloud datacenters
US11012378B2 (en) Methods and apparatus for shared buffer allocation in a transport node

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CP03 Change of name, title or address
CP03 Change of name, title or address

Address after: 310052 Binjiang District Changhe Road, Zhejiang, China, No. 466, No.

Patentee after: Xinhua three Technology Co., Ltd.

Address before: 310053 Hangzhou hi tech Industrial Development Zone, Zhejiang province science and Technology Industrial Park, No. 310 and No. six road, HUAWEI, Hangzhou production base

Patentee before: Huasan Communication Technology Co., Ltd.