CN105847160B - A kind of uncertain incast flow polymerization minimizing grouping based on multidimensional - Google Patents
A kind of uncertain incast flow polymerization minimizing grouping based on multidimensional Download PDFInfo
- Publication number
- CN105847160B CN105847160B CN201610300393.7A CN201610300393A CN105847160B CN 105847160 B CN105847160 B CN 105847160B CN 201610300393 A CN201610300393 A CN 201610300393A CN 105847160 B CN105847160 B CN 105847160B
- Authority
- CN
- China
- Prior art keywords
- server
- transmitting terminal
- grouping
- incast
- transmission
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L45/00—Routing or path finding of packets in data switching networks
- H04L45/48—Routing tree calculation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/41—Flow control; Congestion control by acting on aggregated flows or links
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/70—Admission control; Resource allocation
- H04L47/80—Actions related to the user profile or the type of traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/70—Admission control; Resource allocation
- H04L47/82—Miscellaneous aspects
- H04L47/829—Topology based
Abstract
The invention discloses a kind of uncertain incast flow polymerizations that grouping is minimized based on multidimensional, the transmitting terminal of uncertain incast transmission is pre-processed, it is greater than 1 node elements for frequency of occurrence, all set comprising the node elements only retain the node elements, remaining node elements is deleted；The transmitting terminal of uncertain incast transmission is divided, selected transmitting terminal is grouped as mutual disjoint grouping, the transmitting terminal server in the grouping only has a hop distance between each other；The grouping is cleaned, make same data flow all candidate transmitting terminals only one appear in the final grouping；The incast transmission convergence tree of minimum cost is constructed between the receiving end of transmitting terminal and incast transmission in the grouping, and carries out flow transmission on the incast transmission convergence tree.It is scheduled on optimal incast tree to realize, and polymerize by carrying out caching and flow in transmission process, and then realizes the gain of in-network aggregation.
Description
Technical field
The present invention relates to the field of network transmission of intra-data center data intensive applications, particularly relate to a kind of based on multidimensional
Minimize the uncertain incast flow polymerization of grouping.
Background technique
In the data-intensive applications such as MapReduce, Spark, GraphX, network all plays key effect.Although
These are using different, but they all generally defer to stream calculation mode, i.e., need to transmit in a large amount of in adjacent processing stage
Between calculated result.Many times, these flows are highly relevant.Multi-to-multi shuffle transmission and it is many-to-one
Incast transmits the most of flow for contributing to data center, and the operation of performance and data center to application produces serious shadow
It rings.
In order to solve these flow transmission problems, the network capacity at Future Data center is improved, there has been proposed many nets
Network structure improves this problem, such as Fat-Tree, VL2, BCube and BCN.Other than being studied from network topology structure,
More importantly make full use of the network bandwidth in data center.It is readily conceivable that being made in data by flow scheduling
Intracardiac network is obtained optimizing and be used.However, even if using these required designs optimization method, with being answered in data center
The problem of increase, Internet resources can exhaust at last, and data-intensive application can still encounter network transmission.
Close optimization if be grouped from the angle of application and network, i.e., it, can if considering from flow transmission mode
Can solution to problem it is helpful.The basic starting point of this thinking is to reduce the biography of intermediate result in adjacent processing stage
It is defeated.Therefore this method raising network capacity different from the past and flow scheduling can also combine them further excellent
Change the flow in data center.Since shuffle transmission can be decomposed into grouping incast transmission independent of each other, only
Data aggregate in the net for needing primary study incast to transmit.
For the problem that incast flow carries out in-network aggregation in data center, proposed before can solve.Net cohesion
Conjunction refers to the progress flow aminated polyepichlorohydrin in the equipment of terminal or adjacent phases.In traditional method, the hair of flow transmission
Sending end be it is fixed, our this modes be referred to as determine incast transmission.In determining incast transmission, all flows can be with
It is scheduled on optimal incast tree, and net cohesion may be implemented by carrying out caching and flow polymerization in transmission process
The gain of conjunction.
However, the transmitting terminal of incast transmission can not be known in advance many times.For many incast transmission
Speech, the selection of transmitting terminal does not need setting in special place, and should be the server that those meet certain constraint condition
?.Incast transmission all unascertainable for this transmitting terminal and receiving end, we are defined as not knowing
Incast transmission.
In fact, not knowing incast after the transmitting terminal and receiving end to uncertain incast transmission initialize
Transmission is considered as determining one kind of incast transmission under different situations, and in a series of this possible determining incast transmission
In, the gain of in-network aggregation is different.Thus, it is possible to flow is polymerize with the method for determining incast transmission,
And therefrom select the best one kind of situation.But in determining incast transmission, the building of minimum polymerization tree is np problem, such as
Fruit uses selects optimal polymerization tree from the polymerization tree that many determining incast are transmitted, and it is a series of that complexity is equivalent to solution
Np problem.Therefore solve not know using the method for incast transmission convergence tree is determined incast transmission problem will be can not can
Capable.
Summary of the invention
In view of this, it is an object of the invention to propose that a kind of solution does not know incast transmission problem in the data center
Method.
Based on a kind of above-mentioned purpose uncertain incast flow polymerization for minimizing grouping based on multidimensional provided by the invention
Method, comprising:
The transmitting terminal of uncertain incast transmission is pre-processed, in the transmitting terminal candidate server set of data flow
The number that all node elements occur is found out, 1 node elements is greater than for frequency of occurrence, all includes the node elements
Set only retain the node elements, remaining node elements delete；
It divides, selected transmitting terminal is grouped as non-intersecting between each other to the pretreated transmitting terminal
Grouping, the transmitting terminal server in the grouping only has a hop distance between each other；
The grouping is cleaned, make same data flow all candidate transmitting terminals only one appear in it is final
In the grouping；
The incast transmission of minimum cost is constructed between the receiving end of transmitting terminal and incast transmission in the grouping
Polymerization tree, and flow transmission is carried out on the incast transmission convergence tree.
Further, the pretreated process of transmitting terminal progress of described pair of uncertain incast transmission includes:
The number for finding out all elements appearance is concentrated in the transmitting terminal candidate server of data flow, and frequency of occurrence is greater than
1 node elements node0, all Si comprising node0 only retain node0, remaining element are deleted.
Further, the process that the transmitting terminal of described pair of uncertain incast transmission is divided includes:
Using each of k+1 Route Distinguisher Route Distinguisher to the candidate server collection Si (1 of all data flows
≤ i≤m) and be grouped, and compare that all Route Distinguishers are grouped as a result, obtaining making the least routing of packet count
Identifier.
Further, the process cleaned to the grouping includes:
How much the grouping is subjected to descending arrangement according to contained element, the either element node0 in being grouped to first,
The element node0 is deleted in grouping other than first grouping, and further judges the candidate server collection Sj of data flow
Whether (1≤j≤m) includes node node0, if comprising deleting remaining all node, and deleted in the grouping from Sj
The node of middle deletion；
To subsequent grouping is come, aforesaid operations are also executed；
Wherein, Si is candidate transmitting terminal collection, and m is the item number of data flow, and node0 is the transmitting terminal server node in Si.
Further, minimum cost is constructed between the receiving end of transmitting terminal and the incast transmission in the grouping
The process of incast transmission convergence tree include:
Transmitting terminal and receiving end are extended to one k+2 layers of multistage digraph, wherein the receiving end is located at the 0th rank
Section, the j hop neighbor of the receiving end are the server positioned at stage j, and additional server is selected in per stage.
Further, the process that additional server is selected in per stage includes:
In either phase, for being greater than 1 grouping comprising server number, the server identifier in grouping is only at one
Different in dimension, the data flow of the same grouping from stage k+1, which is forwarded to, jumps common neighbours' server positioned at the 1 of stage k
It is polymerize, if common neighbours' server is the server being grouped in Gi, common neighbours' server mappings is arrived
Otherwise stage k newly adds a server as neighbours' server of stage k+1 in stage k.
Further, the process that additional server is selected in per stage includes:
In either phase, for being equal to 1 grouping, neighbours clothes of the transmitting terminal in selection next stage comprising server number
When business device, those servers as transmitting terminal are paid the utmost attention to.For a transmitting terminal server node0, if in Route Distinguisher
It accords under ej, exist has neighbours' server in some dimension, and neighbours' server appears in other groupings, then
Node0 is sent to stream compression on neighbours' server in same order section.
From the above it can be seen that a kind of uncertain incast stream for minimizing grouping based on multidimensional provided by the invention
Polymerization is measured, the transmitting terminal of uncertain incast transmission is pre-processed, in the transmitting terminal candidate server collection of data flow
The number that all node elements occur is found out in conjunction, 1 node elements is greater than for frequency of occurrence, all includes the node
The set of element only retains the node elements, remaining node elements is deleted；To uncertain incast transmission transmitting terminal into
Row divides, and selected transmitting terminal is grouped as mutual disjoint grouping, the transmitting terminal server phase in the grouping
An only hop distance between mutually；The grouping is cleaned, make same data flow all candidate transmitting terminals only one
It appears in the final grouping；Minimum generation is constructed between the receiving end of transmitting terminal and incast transmission in the grouping
The incast transmission convergence tree of valence, and flow transmission is carried out on the incast transmission convergence tree.To realize optimal
It is scheduled on incast tree, and polymerize by carrying out caching and flow in transmission process, and then realizes in-network aggregation
Gain.
Detailed description of the invention
Fig. 1 is that the present invention is based on multidimensional to minimize showing for the uncertain incast flow polymerization one embodiment being grouped
It is intended to.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, below in conjunction with specific embodiment, and reference
Attached drawing, the present invention is described in more detail.
As shown in Figure 1, for the present invention is based on uncertain one realities of incast flow polymerization that one-dimensional minimizes grouping
The schematic diagram of example is applied, in the present embodiment, the shown uncertain incast flow polymerization packet that grouping is minimized based on one-dimensional
Include following steps:
Step 101: the transmitting terminal of uncertain incast transmission being pre-processed, 1 section is greater than for frequency of occurrence
Point element, all set comprising the node elements only retain the node elements, remaining node elements is deleted.
Step 102: the transmitting terminal of uncertain incast transmission being divided, selected transmitting terminal is grouped as mutually
Between disjoint grouping, the transmitting terminal server in the grouping only has a hop distance between each other.
Step 103: the grouping being cleaned, all only one appearance of candidate transmitting terminal of same data flow are made
In the final grouping.
Step 104: constructing minimum cost between the receiving end of transmitting terminal and incast transmission in the grouping
Incast transmission convergence tree.
Step 105: flow transmission is carried out on the incast transmission convergence tree.
Wherein, step 101, step 102 and step 103 are directly to reduce network flow in uncertain incast transmission
First stage carries out initialization selection to the transmitting terminal of uncertain incast transmission, solution is to minimize transmitting terminal grouping
Problem (MSG, mininal sender group), these selected transmitting terminals are grouped as mutual disjoint grouping,
So that the transmitting terminal server in grouping only has a hop distance between each other.In this way, the flow from same grouping can be jumped 1
It polymerize on neighbours' server.By this method, flow can be reduced in earliest time.
Step 104 and step 105 are that the second stage of network flow is directly reduced in uncertain incast transmission, are being given
After the first stage is selected out transmitting terminal and random receiving end, it will be considered that construct minimum generation between transmitting terminal and receiving end
The incast transmission convergence tree of valence, and propose two kinds of algorithms, it is interstage-bassed and intrastage-based respectively
Algorithm.The transmitting terminal grouping that both algorithms can make full use of the first stage to generate.
The present invention is based on the uncertain incast flow polymerizations that multidimensional minimizes grouping, in the net of relevance flow
The transmission stage rather than only carry out flowing a data aggregate in the receiving end of flow, it is possible to reduce network flow, and capable of more having
Effect ground uses network bandwidth.It first proposed the initialization selection algorithm of transmitting terminal server, which can generate least
Transmitting terminal grouping, polymerize the flow of same grouping.In order to make full use of these grouping institute energy bring gains,
Have also been proposed the developing algorithm of two kinds of incast trees.Experiment shows mentioned algorithm in the research of incast flow in-network aggregation very well
The uncertain incast transmission of performance benefit.From the perspective of reducing network flow and saving Internet resources, do not know
The transmission of incast flow is better than determining incast flow transmission.
Many all follows flowmeter similar to the application such as application, such as Dryad, Pregel and Spark of MapReduce
Calculation mode needs to transmit a large amount of results of intermediate calculations by taking MapReduce as an example between adjacent processing stage to analyze determination
Incast transmission problem.The operation of MapReduce was grouped as by two continuous processing stages.
In the Map stage, mapper task executes map operation to the data of input, generates the sequence of a key-value pair.?
Reducer stage, reducer task execute user-defined reduce to the data of input and operate, usually converging operation.Often
A reducer task be assigned one of codomain section of map input it is unique divide, in the shuffle stage from each
The key-value pair for distributing to it is extracted in the output of mapper.
In general, shuffle transmission includes m transmitting terminal and n receiving end, any pair of transmitting terminal and receiving end it
Between form a data stream.Incast transmission be then to be formed by m transmitting terminal and one of receiving end, each transmitting terminal to
Same receiving end sends data flow.
The network interconnection structure at Future Data center can substantially be divided into following two categories:
The first kind is the network structure using interchanger as core.The structure by interchanger grouping be made into it is tree-like except other
Specific structure, and the network interconnection and routing function are put on interchanger.Typical Representative include Fat-Tree, VL2,
PortLand.Some network equipment producers are dedicated to increasing the function of packet caching and data processing in net to interchanger recently,
Cisco and Juniper has developed programmable ASICs chip to support this function.In addition, common interchanger can also
To carry out the extension of these functions by special equipment.These interchangers based on asic chip, fpga chip and software are all
The in-network aggregation for having programmable data plane incast is supported to transmit.
Second class is the network structure using server as core, wherein main interconnection and routing function are held by server
Load.Bcube, BCN, CamCube belong to this class formation.In practical application, server passes through the programmable exchange board of gigabit
ServerSwitch and OnetSwitch customizes message forwarding capability.As described in previous research work, it is equipped with
The server of ServerSwitch can support new network equipment, such as net interior packet caching.Therefore, using server as the data of core
Central site network structure is that data aggregate brings possibility between realizing stream.If there is multiple data flows are converged in server, arrive first
Data flow can be in server buffer.After all data flows all reach, server can carry out polymerization behaviour to these data flows
Make, forms new data stream, and be transmitted to receiving end.
The topological structure of data center is indicated with a figure, i.e. figure G=(V, E).Wherein V is the set of point, each
Point all represents interchanger or server.E is the set on side, if side (u, v) exists, illustrates the corresponding interchanger kimonos of u, v
There are link connections between business device.
Many data-intensive applications can all be related to multistage processing problem, generate incast transmission problem.These
Incast transmission problem can generally consume a large amount of Internet resources, had certain methods to be suggested solution incast transmission and asked
Topic.But these methods need to know in advance transmitting terminal and the reception of data flow both for incast transmission problem is determined
The position at end.
In many incast transmission, the transmitting terminal of data flow is not fixed to a certain position, as long as it meets about
Beam condition.As soon as this makes for example, map stage mapper can select a host to carry out data mapping from three hosts
The selection for obtaining transmitting terminal in incast transmission becomes flexibly.In order to more preferably indicate this problem, it will be assumed that every number
According to stream all there are three transmitting terminal server it can be selected that the grouping being then likely to occur is combined into 3m if there is m data stream.Separately
Outside, in the reduce stage, reducer can also be scheduled selection out of entire data center, this can make again in incast transmission
Receiving end selection it is more flexible.Based on the above analysis, we it can be concluded that, the transmitting terminal of incast transmission and receiving end are all
Be it is uncertain, below we provide the definition of uncertain incast transmission.
For incast transmission, minimum polymerization tree problem is exactly to find a connected subgraph from figure G=(V, E), this
All incast nodes of a sub- map combining, and spend cost minimum.In the data center of close connection, such as BCube, it is
Incast transmission finds a minimum polymerization tree and has proven to a np hard problem.Similarly, it is transmitted in uncertain incast
The minimum polymerization tree of middle searching is also np hard problem.
Without loss of generality, we illustrate the aggregation problem of uncertain incast transmission with BCube structure.BCube(n,
K) the super cube of broad sense of k+1 dimension n-ary can be conceptualized as.In BCube (n, k), if two are respectively identified as
The server of xkxk-1 ... x1x0 and ykyk-1 ... y1y0 only identify difference in j dimension, then the two servers 1 jump neighbour each other
It occupies.Such two servers are by being identified as yk in BCube (n, k) structure ..., and the interchanger of yj+1yj-1 ... y1y0 is mutual
Connection.As a result it will be appreciated that 1 hop neighbor server of the server in each dimension has n-1, and between them
It is connected by interchanger.If the distance of the two servers is j jump in addition, two servers are different in j dimension.
In order to realize the in-network aggregation of uncertain incast transmission, a set of approximate algorithm is devised to solve the problems, such as this, is calculated
Method takes full advantage of the topological structure of data center.Algorithm is divided into two continuous processing stages, first stage, uncertain
Optimal transmitting terminal is selected in incast transmission, if there is m data stream, when each data flow there can be 3 transmitting terminals, then
It needs to select a kind of optimal situation from 3m kind situation；Second stage needs to construct after selecting transmitting terminal server
Incast polymerization tree is to realize maximization gain.
Incast transmission problem is not known despite studying in the data center of BCube structure, but the present invention is mentioned
Algorithm out be equally applicable to other structures using server as the data center of core.The developing algorithm of incast tree needs benefit
With the topological characteristic of data center's different structure.In the data center using interchanger as core, if its structure be FBFLY and
HyperX, and all employ the novel interchanger of programmable data plane, then it can be equally used for using the algorithm that this chapter is proposed
Incast tree is constructed, because their structure as BCube, belongs to the super cube of broad sense.
In practice, it does not know incast transmission and often selects transmitting terminal from 3m server, and assign one and connect
Receiving end.Select transmitting terminal from many servers as candidate transmitting terminal come when constructing incast tree, cost also in that
The selection of transmitting terminal is different and different.
Transmitting terminal packet count is fewer, and network flow is also fewer.In order to minimize network flow, we will tend to find
A kind of transmitting terminal server subassembly that transmitting terminal packet count can be made as few as possible.And this transmitting terminal the case where
Under, data flow can carry out in-network aggregation in the transmission stage as early as possible, to achieve the purpose that save Internet resources.But
It is that for uncertain incast transmission, finding a grouping can make the maximum transmitting terminal of incast transmission convergence gain be
It is very difficult.This also promotes us to study the minimum transmission grouping problem in uncertain incast transmission problem.
For there is the uncertain incast of m data stream transmission, minimizes transmitting terminal grouping problem (MSG) and just refer to every
Data stream selects a determining transmitting terminal server from candidate server.The transmitting terminal of all data flows can be grouped as
The transmitting terminal of minimum packets number is grouped.The grouping of transmitting terminal is dependent on selected server from candidate server.In order to
The effect for enough realizing in-network aggregation, the data flow issued from same grouping should be able to gather jointly after defeated by 1 jump set
Chalaza is polymerize, to realize the reduction of network flow.
As an embodiment of the present invention, pretreated process packet is carried out to the transmitting terminal of uncertain incast transmission
It includes:
It is greater than 1 node elements for frequency of occurrence, all set comprising the node elements only retain the section
Point element deletes remaining node elements；
As an embodiment of the present invention, include: to the process that is divided of transmitting terminal of uncertain incast transmission
Using each of k+1 Route Distinguisher Route Distinguisher to the candidate server collection Si (1 of all data flows
≤ i≤m) and be grouped, and compare that all Route Distinguishers are grouped as a result, obtaining making the least routing of packet count
Identifier.
As another embodiment of the invention, the process cleaned to the grouping includes:
How much grouping is subjected to descending arrangement according to contained element, its element node0 is investigated in the grouping to coming first,
If in Si (1≤i≤m) including the element, remaining element in Si in addition to node0 is deleted, while in transmitting terminal point
The deleted element is deleted in group；
If not including node0 in Si (1≤i≤m), node0 is deleted from transmitting terminal grouping；
To subsequent grouping is come, aforesaid operations are also executed；
Wherein, Si is candidate transmitting terminal collection, and m is the item number of data flow, and node0 is the transmitting terminal server in Si.
As yet another embodiment of the present invention, between the receiving end of transmitting terminal and incast transmission in the grouping
The process of incast transmission convergence tree for constructing minimum cost includes:
Transmitting terminal and receiving end are extended to one k+2 layers of multistage digraph, wherein the receiving end is located at the 0th rank
Section, the j hop neighbor of the receiving end are the server positioned at stage j, and additional server is selected in per stage.
As another embodiment of the invention, the process that additional server is selected in per stage includes:
In either phase, for being greater than 1 grouping comprising server number, the server identifier in grouping is only at one
Different in dimension, the data flow of the same grouping from stage k+1, which is forwarded to, jumps common neighbours' server positioned at the 1 of stage k
It is polymerize, if common neighbours' server is the server being grouped in Gi, common neighbours' server mappings is arrived
Otherwise stage k newly adds a server as neighbours' server of stage k+1 in stage k.
In some other embodiment of the invention, the process that additional server is selected in per stage includes:
In either phase, for being equal to 1 grouping, neighbours clothes of the transmitting terminal in selection next stage comprising server number
When business device, those servers as transmitting terminal are paid the utmost attention to.For a transmitting terminal server node0, if in Route Distinguisher
It accords under ej, exist has neighbours' server in some dimension, and neighbours' server appears in other groupings, then
Node0 is sent to stream compression on neighbours' server in same order section.
For uncertain incast transmission, most basic idea is exactly using each in k+1 Route Distinguisher
A Route Distinguisher to all data flows candidate server collection Si's (1≤i≤m) and be grouped.By comparing all roads
By identifier be grouped as a result, can obtain making the least Route Distinguisher of packet count in turn.Without loss of generality, we
It is assumed that Optimization route identifier is e0.The grouping as derived from Route Distinguisher e0 can not be directly used in solution and minimize transmitting terminal
Grouping problem.It traces it to its cause, in that these, which are grouped all candidate servers of every data stream, all includes.Therefore, I
After the grouping again carried out cleaning operation, as shown in the function Cleanup () in pseudo-code of the algorithm, as a result, making
All candidate servers of one data stream only have a server and appear in final minimum transmitting terminal grouping.
How much grouping is first subjected to descending arrangement according to contained element, then carries out relevant treatment.For coming first
Grouping, investigates its element node0, if in Si (1≤i≤m) including the element, by remaining member in Si in addition to node0
Element is deleted.This method can make the uncertain transmitting terminal problem in Si become determining problem.In addition, those deleted elements
It should be also deleted in transmitting terminal grouping.If not including node0 in Si (1≤i≤m), by node0 from transmitting terminal yet
It is deleted in grouping.To subsequent grouping is come, such operation is also executed.Final result will generate one and meet constraint condition
Transmitting terminal grouping, it is, every data stream only has 1 server as transmitting terminal.
Simultaneously because the topological structure feature of data center:
First, an element node0 of representative server are likely to appear in the candidate transmitting terminal service of different data streams
In device set Si, i.e., different set Si (1≤i≤m) is it is possible that there is the case where intersection.This is answered in actual data center
It is possible in, because a server can provide data service for multiple tasks.And if a server is more
A task work, the then data flow that these tasks generate can directly be polymerize locally, this will make to flow in data center
The in-network aggregation gain of amount is bigger and can save more Internet resources.Therefore, such server node is being elected to be transmission
There is higher priority when end.
Second, all candidate servers when being divided in the division stage, can at a certain Route Distinguisher e0 quilt
Minimum packets are divided into, but this division result may no longer be that packet count is least after executing cleaning operation, also
A kind of grouping situation for minimum packets that be after saying certain execution division operations be not may make point after executing cleaning operation
Group number is reduced to minimum.Moreover, only one element occurred under some Route Distinguisher is grouped in other Route Distinguisher
It may be 1 hop neighbor under symbol.
In order to make full use of above-mentioned two o'clock property, algorithm for design 1 is as follows:
Above-mentioned algorithm consists of three parts, and is pretreatment respectively, divides, cleaning.
Pretreatment operation is exactly to make full use of first property.The transmitting terminal candidate server of given m data stream
Collect S1 ..., Sm, we are firstly the need of the number for finding out all elements appearance.This can be reached by traversal all elements
Purpose, time complexity are O (m).Notice S1 ..., the sum of element number in Sm is up to 3 × m.For frequency of occurrence
Node elements node0 greater than 1 time, all Si comprising node0 only retain node0, remaining element are deleted.
In order to make full use of second individual character matter, divide the stage, it is right using each Route Distinguisher ei (0≤i≤k)
S1 ..., Sm's and division operation is carried out respectively.Lgroups then has recorded the group result that all Route Distinguishers divide, yuan
Element is up to 3k × m, because at most generating 3 × m grouping when carrying out division operation using single Route Distinguisher.
To sorted grouping, any node node0 in largest packet, due to being in k+1 Route Distinguisher
It is split under symbol, therefore has other k grouping also comprising node node0.Therefore, it is necessary to points other than first grouping
Node0 is deleted in group.In addition, judging whether the candidate server collection Sj (1≤j≤m) of data flow includes node node0, such as
Fruit includes then to delete remaining all node.This method makes uncertain transmitting terminal become determining transmitting terminal.At the same time, from
The node deleted in Sj should also be as deleting from grouping.To remaining grouping, recursive these methods of calling.This recursion method is most
Grouping as few as possible can be generated eventually.
The time complexity of MSG greedy algorithm based on multidimensional is O (k2 × m2).
Prove: in pretreatment stage, most multipair 3 × m element is checked in m-1 set, therefore its time is multiple
Miscellaneous degree is O (m2+m).In the stage of division, need to call k+1 Partition (S, ei) function, and Partition (S, ei) letter
Several time complexities is O (m), so the time complexity in the stage of division is O (k × m).In wash phase, sorting operation
Time complexity is O (k2 × m2).In Cleanup (Lgroups), from most 3k × m represented by the 3rd row of pseudo-code algorithm
The time complexity that at most 3m node is deleted in grouping is O (k × m2).When the 4th row of pseudo-code algorithm updates set Si, and m
3m element in a set is compared, therefore time complexity is O (m2).When pseudo-code the 5th row deletion of node of algorithm,
3m node is compared in most 3k × m groupings, therefore time complexity is O (k × m2).Therefore, third rank
Section, the i.e. time complexity of cleaning operation part are O (k2 × m2).Based on the above analysis, the overall time of three phases is complicated
Degree is O (k2 × m2).
It is selected by the initialization of transmitting terminal server, so that all only 1 server can be made for every data stream
For transmitting terminal.In addition, these servers for being elected as transmitting terminal can be grouped into G1, G2 ... G β.This is allowed for not really
Determine incast transmission problem to degenerate to be that transmitting terminal determines, the uncertain incast transmission problem in receiving end.
In order to sufficiently study the gain of in-network aggregation, further consider to have the incast transmission of any one receiving end to ask
Topic.Such incast transmission has a receiving end R, there is a transmitting terminal set S={ s1, s2 ... s α }.It, will according to defining 3
Transmitting terminal is grouped, group result G1, G2 ... G β.It should be noted that the number α of transmitting terminal server set wants small
In the number m of data flow, this is because some servers can be used as the transmitting terminal of multiple data flows.Therefore, in all data
In transmitting terminal candidate server the set S1 ..., Sm of stream, the server si number occurred is denoted as ci.
Incast transmission problem mentioned immediately above is solved, target is exactly to construct a covering institute in figure G=(V, E)
There is the incast polymerization minimum cost tree of the transmitting terminal and receiving end that participate in incast transmission, so that every transmitting terminal server hair
Data flow out can be transmitted along tree, and receiving end R is received.
In incast transmission, all transmitting terminals and receiving end can be extended the multistage digraph for one k+2 layers.
Wherein, only receiving end was located at for the 0th stage, and positioned at the server of stage j must be the j hop neighbor of receiving end.Consider to send
Grouping G1, G2 ... the G β for holding server is grouped Gi, 1≤i≤β for any one, it has following property at a distance from the r of receiving end
Matter.
If | Gi |=1, and unique server is the j hop neighbor of transmitting terminal in the grouping, then it is located at stage j.
If | Gi | > 1, the server in the grouping is all between each other 1 hop neighbor in given dimension.Therefore this
A little servers are all the j-1 hop neighbor either j hop neighbors of receiving end.They are either positioned at stage j-1 or are to be located at rank
Section j.
Only by these server mappings into multistage digraph, incast polymerization tree cannot be constituted.Problem is
How in per stage to determine minimum number destination server, and how to determine interchanger in successive stages.It is mentioned in front,
The mark of interchanger can be determined by the mark of its associated server.Therefore, the emphasis for constructing incast tree is that
Additional server is selected in per stage.
In each stage, the number of server is fewer, and the data flow number exported to next stage is fewer, because coming from
Data flow on last stage can be polymerize in this stage, and polymerization result is 1 data stream of output.The algorithm that we are mentioned
Original intention is exactly to find out a server in stage j-1, so that stage j server as much as possible is all neighbours with it.In this way, rank
The server number of section j-1 will be reduced.
The server that stage k is recursively found out from stage k+1, until the stage 1.Since stage k+1, the stage, k+1 was only wrapped
Include the server jumped apart from receiving end k+1.These servers come from grouping G1, G2 ... the k+1 in G β jumps server.
These groupings are to be ranked up according to the element number in grouping, and successively handle according to the sequence.
For being greater than 1 grouping comprising server number, i.e., | Gi | > 1, the server identifier in grouping is only at one
It is different in dimension, such as ei.Our idea is that the data flow of the same grouping from stage k+1 is forwarded to positioned at stage k
1 jump common neighbours' server and polymerize, we claim this method to be the polymerization between the stage.Common neighbours' server can lead to
It crosses Route Distinguisher ei to determine, it is only different in ei dimension mark from the server in grouping Gi, the mark and reception that it is tieed up in ei
The ei dimension mark at end is identical.Common neighbours' server can be the server in grouping Gi, can also be not belonging to the grouping.If
It is the server being grouped in Gi, then maps that stage k, otherwise, newly add a server as stage k+1 in stage k
Neighbours' server.
For being equal to 1 grouping comprising server number, i.e., | Gi |=1.A unique server is in stage k in being grouped
On have 1 hop neighbor server in k dimension.If transmitting terminal server randomly chooses a 1 hop neighbor service in stage k
Device, the incast tree generated may not be optimal.A kind of reasonable approach is exactly neighbour of the transmitting terminal in selection next stage
When occupying server, those servers as transmitting terminal are paid the utmost attention to.In this way, the data flow for coming self-grouping Gi and this stage can be with
The polymerization of (inter-stage) between implementation phase.Under special circumstances, such as 1 it is grouped, its 1 hop neighbor server
It is not all transmitting terminal, then the polymerization between this stage just cannot achieve.
Algorithm in a kind of stage is designed in response to this, and flow polymerization is realized within this stage.Consider such one
Transmitting terminal server node0, it is the unique member in some grouping.It, can be in some dimension at Route Distinguisher ej
There is neighbours' server, which appears in other groupings.In this case, node0 is no longer needed number
It is sent to next stage according to circulation, but only that being forwarded in the same stage on neighbours' server of (intra-stage).It is selected
Neighbours' server finally be transmitted to it data flow and own generate data flow polymerize in stage k+1.It will be upper
It states method successively to use in stage k, k-1 ..., 1, in this way, with regard to the polymerization tree of the uncertain incast transmission of Successful construct.
It should be understood by those ordinary skilled in the art that: the discussion of any of the above embodiment is exemplary only, not
It is intended to imply that the scope of the present disclosure (including claim) is limited to these examples；Under thinking of the invention, above embodiments
Or it can also be grouped conjunction between the technical characteristic in different embodiments, step can be realized with random order, and be existed
Many other variations of different aspect present invention as described above, for simplicity, they are not provided in details.
In addition, to simplify explanation and discussing, and in order not to obscure the invention, it can in provided attached drawing
It is connect with showing or can not show with the well known power ground of integrated circuit (IC) chip and other components.Furthermore, it is possible to
Device is shown in block diagram form, to avoid obscuring the invention, and this has also contemplated following facts, i.e., about this
The details of the embodiment of a little block diagram arrangements be height depend on will implementing platform of the invention (that is, these details should
It is completely within the scope of the understanding of those skilled in the art).Elaborating that detail (for example, circuit) is of the invention to describe
In the case where exemplary embodiment, it will be apparent to those skilled in the art that can be in these no details
In the case where or implement the present invention in the case that these details change.Therefore, these descriptions should be considered as explanation
Property rather than it is restrictive.
Although having been incorporated with specific embodiments of the present invention, invention has been described, according to retouching for front
It states, many replacements of these embodiments, modifications and variations will be apparent for those of ordinary skills.Example
Such as, discussed embodiment can be used in other memory architectures (for example, dynamic ram (DRAM)).
The embodiment of the present invention be intended to cover fall into all such replacements within the broad range of appended claims,
Modifications and variations.Therefore, all within the spirits and principles of the present invention, any omission, modification, equivalent replacement, the improvement made
Deng should all be included in the protection scope of the present invention.
Claims (7)
1. a kind of uncertain incast flow polymerization for minimizing grouping based on multidimensional characterized by comprising
The transmitting terminal of uncertain incast transmission is pre-processed, is found out in the transmitting terminal candidate server set of data flow
The number that all node elements occur is greater than frequency of occurrence 1 node elements, all collection comprising the node elements
Conjunction only retains the node elements, remaining node elements is deleted；
The pretreated transmitting terminal is divided, selected transmitting terminal is grouped as disjoint point between each other
Group, the transmitting terminal server in the grouping only have a hop distance between each other；
The grouping is cleaned, make same data flow all candidate transmitting terminals only one appear in it is final described
In grouping；
The incast transmission convergence of minimum cost is constructed between the receiving end of transmitting terminal and incast transmission in the grouping
Tree, and flow transmission is carried out on the incast transmission convergence tree.
2. the uncertain incast flow polymerization according to claim 1 for minimizing grouping based on multidimensional, feature
It is, the transmitting terminal of described pair of uncertain incast transmission carries out pretreated process and includes:
The number that all elements appearance is found out in the transmitting terminal candidate server set of data flow is greater than 1 for frequency of occurrence
Secondary node elements node0, all Si comprising node0 only retain node0, remaining element are deleted；
Wherein, node0 is the transmitting terminal server node in Si；Si is transmitting terminal candidate server set.
3. the uncertain incast flow polymerization according to claim 1 for minimizing grouping based on multidimensional, feature
It is, the process divided to the pretreated transmitting terminal includes:
Using each of k+1 Route Distinguisher Route Distinguisher to the transmitting terminal candidate server set of all data flows
Si is grouped, and compares that all Route Distinguishers are grouped as a result, obtaining making the least Route Distinguisher of packet count；
Wherein, the Si is transmitting terminal candidate server set, and the range of i is 1~m, and m is the item number of data flow.
4. the uncertain incast flow polymerization according to claim 1 for minimizing grouping based on multidimensional, feature
It is, the process cleaned to the grouping includes:
How much grouping is subjected to descending arrangement according to contained element, its element node0 is investigated in the grouping to coming first, if
Include the element in Si, then delete remaining element in Si in addition to node0, while being deleted in transmitting terminal grouping by described
The element removed is deleted；
If not including node0 in Si, node0 is deleted from transmitting terminal grouping；
To subsequent grouping is come, aforesaid operations are also executed；
Wherein, node0 is the transmitting terminal server node in Si；Si is transmitting terminal candidate server set, the range of i is 1~
M, m are the item number of data flow.
5. the uncertain incast flow polymerization according to claim 1 for minimizing grouping based on multidimensional, feature
It is, the incast transmission of minimum cost is constructed between the receiving end of transmitting terminal and the incast transmission in the grouping
Polymerizeing the process set includes:
Transmitting terminal and receiving end are extended to one k+2 layers of multistage digraph, wherein the receiving end is located at the 0th stage, institute
The j hop neighbor of receiving end is stated to select additional server in per stage positioned at the server of stage j.
6. the uncertain incast flow polymerization according to claim 5 for minimizing grouping based on multidimensional, feature
It is, the process that additional server is selected in per stage includes:
In either phase, for being greater than 1 grouping comprising server number, the server identifier in grouping is only in a dimension
Upper difference, the data flow of the same grouping from stage k+1, which is forwarded to, jumps common neighbours' server progress positioned at the 1 of stage k
Polymerization, if common neighbours' server is the server being grouped in Gi, by common neighbours' server mappings to stage
Otherwise k newly adds a server as neighbours' server of stage k+1 in stage k.
7. the uncertain incast flow polymerization according to claim 5 for minimizing grouping based on multidimensional, feature
It is, the process that additional server is selected in per stage includes:
In either phase, for being equal to 1 grouping, neighbours server of the transmitting terminal in selection next stage comprising server number
When, pay the utmost attention to those servers as transmitting terminal；For a transmitting terminal server node0, if in Route Distinguisher ej
Under, exist has neighbours' server in some dimension, and neighbours' server appears in other groupings, then node0 handle
Stream compression is sent on neighbours' server in same order section.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610300393.7A CN105847160B (en) | 2016-05-09 | 2016-05-09 | A kind of uncertain incast flow polymerization minimizing grouping based on multidimensional |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610300393.7A CN105847160B (en) | 2016-05-09 | 2016-05-09 | A kind of uncertain incast flow polymerization minimizing grouping based on multidimensional |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105847160A CN105847160A (en) | 2016-08-10 |
| CN105847160B true CN105847160B (en) | 2019-04-12 |
Family
ID=56591308
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610300393.7A Active CN105847160B (en) | 2016-05-09 | 2016-05-09 | A kind of uncertain incast flow polymerization minimizing grouping based on multidimensional |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105847160B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101067792A (en) * | 2006-05-04 | 2007-11-07 | 国际商业机器公司 | System and method for scalable processing of multi-way data stream correlations |
| CN103139265A (en) * | 2011-12-01 | 2013-06-05 | 国际商业机器公司 | Network transmission self-adaption optimizing method and system in large-scale parallel computing system |
| CN103412635A (en) * | 2013-08-02 | 2013-11-27 | 清华大学 | Energy-saving method and energy-saving device of data center |
-
2016
- 2016-05-09 CN CN201610300393.7A patent/CN105847160B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101067792A (en) * | 2006-05-04 | 2007-11-07 | 国际商业机器公司 | System and method for scalable processing of multi-way data stream correlations |
| CN103139265A (en) * | 2011-12-01 | 2013-06-05 | 国际商业机器公司 | Network transmission self-adaption optimizing method and system in large-scale parallel computing system |
| CN103412635A (en) * | 2013-08-02 | 2013-11-27 | 清华大学 | Energy-saving method and energy-saving device of data center |
Non-Patent Citations (2)
| Title |
|---|
| DCent： 基于双网卡服务器的高可扩展数据中心网络结构;朱桂明等;《计算机研究与发展》;20140531;全文 |
| Research on Hybrid PSO Algorithm with Appended Intensification and Diversification;liu lei 等;《2013 International Conference on Mechatronic Sciences, Electric Engineering and Computer (MEC)》;20131222;全文 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105847160A (en) | 2016-08-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tang et al. | On removing routing protocol from future wireless networks: A real-time deep learning approach for intelligent traffic control | |
| Li et al. | Efficient collective communications in dual-cube | |
| Guo et al. | Exploiting efficient and scalable shuffle transfers in future data center networks | |
| CN104429038A (en) | Route forwarding method, apparatus and system | |
| CN103986661B (en) | A kind of multicast route method for supporting network code | |
| US20060268691A1 (en) | Divide and conquer route generation technique for distributed selection of routes within a multi-path network | |
| CN107786497B (en) | Method and device for generating ACL table | |
| Zahavi et al. | Quasi fat trees for HPC clouds and their fault-resilient closed-form routing | |
| Nosrati et al. | G-CARA: A Global Congestion-Aware Routing Algorithm for traffic management in 3D networks-on-chip | |
| CN108259387A (en) | A kind of exchange system and its routing algorithm built by interchanger | |
| CN103297354B (en) | Server interlinkage system, server and data forwarding method | |
| CN102546380B (en) | Modified tree-based multicast routing scheme | |
| Jouraku et al. | An effective design of deadlock-free routing algorithms based on 2d turn model for irregular networks | |
| Zhu et al. | Efficient hybrid multicast approach in wireless data center network | |
| CN105847160B (en) | A kind of uncertain incast flow polymerization minimizing grouping based on multidimensional | |
| Zheng et al. | Up-and-down routing in mobile opportunistic social networks with bloom-filter-based hints | |
| CN106059914B (en) | A kind of uncertain incast flow polymerization minimizing grouping based on one-dimensional | |
| Mahapatra et al. | Limited multi-path routing on extended generalized fat-trees | |
| CN112995032B (en) | Segment routing traffic engineering method and device based on limited widest path | |
| WO2022269357A1 (en) | Deadlock-free multipath routing for direct interconnect networks | |
| Amin et al. | Agent-based distance vector routing | |
| Liu et al. | Virtual-force-based geometric routing protocol in MANETs | |
| Jingwen et al. | A traffic scheduling scheme for data center networks based on sdn | |
| Qin et al. | Minimum cost multi-path parallel transmission with delay constraint by extending openflow | |
| Nosrati et al. | Regional selection mechanism for traffic-balanced adaptive routing algorithms in mesh-based NoC architectures |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |