CN108833297A - Priority classification method, dispatching method and the device of big data flow - Google Patents
Priority classification method, dispatching method and the device of big data flow Download PDFInfo
- Publication number
- CN108833297A CN108833297A CN201810432953.3A CN201810432953A CN108833297A CN 108833297 A CN108833297 A CN 108833297A CN 201810432953 A CN201810432953 A CN 201810432953A CN 108833297 A CN108833297 A CN 108833297A
- Authority
- CN
- China
- Prior art keywords
- coflow
- port
- serial number
- flow
- data center
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
- H04L47/2425—Traffic characterised by specific attributes, e.g. priority or QoS for supporting services specification, e.g. SLA
- H04L47/2433—Allocation of priorities to traffic types
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/70—Admission control; Resource allocation
- H04L47/80—Actions related to the user profile or the type of traffic
- H04L47/805—QOS or priority aware
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present invention provides priority classification method, dispatching method and the device of a kind of big data flow, is applied to data center, which includes m inbound port and n exit port, and priority classification method includes step 10：Update data center big data flow Coflow ordered set F, F include data center transmitting with Coflow to be transmitted, enable k be F in Coflow serial number, k=1,2 ... K；Step 11：It calculates each Coflow in F and has transmitted flow in m inbound port and n exit port, search each Coflow and transmitted the minimum value of flow in m inbound port and n exit port and be denoted as lk, k=1,2 ... K；Step 12：Update priority indexK=1,2 ... K,With lkDirectly proportional and wkIt is inversely proportional, wkFor the weight of the Coflow of serial number k in F；Step 13：To ψkIt carries out non-decreasing to arrange to obtain ω, the ψ of first item in ωkThe Coflow highest priority of corresponding serial number k, later priority are successively decreased item by item.Coflow priority index proposed by the present invention more can reflect actual demand of the Coflow to bandwidth.
Description
Technical field
The present invention relates to computer field, in particular to a kind of priority classification method of big data flow, dispatching method
And device.
Background technique
In recent years, big data processing framework, such as Map-Reduce, Dryad, Spark, distributed storage are widely used
In data center, a large amount of network flows that these applications generate at runtime bring huge choose to the network of data center
War.Data center network is improved from various aspects such as topology design, routing Design, transmission optimizations, so that data network
Real-time bandwidth and delay are able to satisfy application demand.
Improvement in terms of data center's transmission optimization includes：Classified according to the characteristic of data flow, such as PDQ
(Product Data Quality) or pFabric allow small number then using SJF (Shortest Job First) strategy
More bandwidth are obtained according to stream, thus, to reduction data flow deadline (FCT), and then reduce application delay.
However, in data center, data transmission can not be completed only by a stream, due to there are data dependence,
One group of parallel data flow is generally comprised, when this group data stream is all transmitted, using can just carry out in next step.Therefore,
In such a scenario, only carry out stream rank optimization be it is inadequate, need to carry out the optimization of data flow group rank.
For this phenomenon, California, USA Berkeley University proposes the concept of big data flow (Coflow), with one
Whole removing dispatches one group of context-sensitive data flow.
The current existing method that Coflow is optimized, as Varys uses SEBF (Smallest-Effective-
Bottleneck-First) priority of Coflow is optimized, then uses MADD (Minimum-Allocation-
For-Desired-Duration the bandwidth of Coflow) is determined.But Varys needs to be known in advance the stream size of Coflow, width
The information such as degree, source address, but in practical applications, the stream size of Coflow is often difficult to predict.
Based on this, and there is the method for blind scheduling, such as Aalo, CODA etc., these methods do not need to be known in advance
The stream size of Coflow, is similarly also unable to get optimal result.
Other, are to the method that Coflow is optimized, the Barrat of such as distributed invocation pattern, because this method needs largely
Interchanger is modified, so that distributed scheduling is greatly reduced a possibility that being deployed in actual production environment.
Above it is found that in the method optimized at present to Coflow, lack applied widely, effect of optimization and good side
Method.
Summary of the invention
The present invention provides priority classification method, dispatching method and the device of a kind of big data flow, solves current
The problem that the dispatching method scope of application of Coflow is limited or transmission optimization effect is poor.
The present invention provides a kind of priority classification method of big data flow, is applied to data center, data center's net
Network includes m inbound port and n exit port, and this method includes：
Step 10：Big data flow Coflow the ordered set F, F for updating data center include that data center is transmitting
Coflow and Coflow to be transmitted, enable k be F in Coflow serial number, k=1,2 ... K；
Step 11：It calculates each Coflow in F and has transmitted flow in m inbound port and n exit port, search each
Coflow has transmitted the minimum value of flow in m inbound port and n exit port and has been denoted as lk, k=1,2 ... K；
Step 12：Update priority index ψk, k=1,2 ... K, ψkWith lkIt is directly proportional, ψkWith wkIt is inversely proportional, wkFor serial number in F
For the weight of the Coflow of k；
Step 13：To ψkIt carries out non-decreasing to arrange to obtain ω, the ψ of first item in ωkThe Coflow of corresponding serial number k is excellent
First grade highest, later priority are successively decreased item by item.
The present invention also provides a kind of dispatching methods of big data flow, are applied to data center, the data center network packet
M inbound port and n exit port are included, this method includes：
Step 20：Big data flow Coflow the ordered set F, F for updating data center include that data center is transmitting
Coflow and Coflow to be transmitted, enable k be F in Coflow serial number, k=1,2 ... K；
Step 21：It calculates each Coflow in F and has transmitted flow in m inbound port and n exit port, search each
Coflow has transmitted the minimum value of flow in m inbound port and n exit port and has been denoted as lk, k=1,2 ... K；
Step 22：Update priority index ψk, k=1,2 ... K, ψkWith lkIt is directly proportional, ψkWith wkIt is inversely proportional, wkFor serial number in F
For the weight of the Coflow of k；
Step 23：To ψkIt carries out non-decreasing to arrange to obtain ω, sequentially extracts ψ in ω item by itemkCorresponding Coflow serial number k is obtained
To priority order R；
Step 24：According to priority order R successively checks in every Coflow the flow that whether there is unassigned bandwidth1≤i≤m, 1≤j≤n, 1≤k≤K, whereinOr I indicates serial number
For the inbound port serial number of the Coflow flow of R (k), j indicates the exit port serial number of the Coflow flow of serial number R (k), if
It is to then follow the steps 25；
Step 25：The minimum value in current m inbound port and the remaining bandwidth of n exit port is searched, r is denoted as；
Step 26：By the r bandwidth in the remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j
It distributes toUpdate the remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j, return step
24。
The present invention also provides a kind of priority classification devices of big data flow, are applied to data center, the data center
Network includes m inbound port and n exit port, which includes：
Coflow update module：Big data flow Coflow the ordered set F, F for updating data center include data center
The Coflow and Coflow to be transmitted transmitted, enabling k is the serial number of Coflow in F, k=1,2 ... K；
Port flow computing module：It calculates each Coflow in F and has transmitted flow in m inbound port and n exit port,
Each Coflow is searched to have transmitted the minimum value of flow in m inbound port and n exit port and be denoted as lk, k=1,2 ... K；
Priority index computing module：Update priority index ψk, k=1,2 ... K, ψkWith lkIt is directly proportional, ψkWith wkAt anti-
Than wkFor the weight of the Coflow of serial number k in F；
Coflow categorization module：To ψkIt carries out non-decreasing to arrange to obtain ω, the ψ of first item in ωkCorresponding serial number k's
Coflow highest priority, later priority are successively decreased item by item.
The present invention also provides a kind of dispatching devices of big data flow, are applied to data center, data center network includes m
A inbound port and n exit port, the device include：
Coflow update module：Big data flow Coflow the ordered set F, F for updating data center include data center
The Coflow and Coflow to be transmitted transmitted, enabling k is the serial number of Coflow in F, k=1,2 ... K；
Port flow computing module：It calculates each Coflow in F and has transmitted flow in m inbound port and n exit port,
Each Coflow is searched to have transmitted the minimum value of flow in m inbound port and n exit port and be denoted as lk, k=1,2 ... K；
Priority index computing module：Update priority index ψk, k=1,2 ... K, ψkWith lkIt is directly proportional, ψkWith wkAt anti-
Than wkFor the weight of the Coflow of serial number k in F；
Coflow categorization module：To ψkIt carries out non-decreasing to arrange to obtain ω, sequentially extracts ψ in ω item by itemkCorresponding Coflow
Serial number k obtains priority order R；
Unallocated flow searching module：According to priority order R is successively checked in every Coflow with the presence or absence of unallocated band
Wide flow1≤i≤m, 1≤j≤n, 1≤k≤K, wherein Ori
Indicate the inbound port serial number of the Coflow flow of serial number R (k), j indicates the exit port sequence of the Coflow flow of serial number R (k)
Number, if it is, executing port remaining bandwidth spider module；
Port remaining bandwidth spider module：Search the minimum in current m inbound port and the remaining bandwidth of n exit port
Value, is denoted as r；
Bandwidth allocation module：It will be in the remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j
R bandwidth allocation is givenThe remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j is updated, is returned
Unallocated flow searching module.
Coflow priority index proposed by the present invention has comprehensively considered weight and minimum port flow, so that having sent
Port flow is smaller, and the bigger Coflow of weight possesses higher priority, refers to compared to only consideration weight, the priority of the application
Mark better reflects actual demand of the Coflow to bandwidth.
The Coflow dispatching method of the application allows the higher Coflow of priority preferentially to obtain bandwidth resources, subtracts
The transmission time and application delay of few Coflow, to realize the transmission optimization of Coflow rank.In addition, the scheduling of the application
Method is applicable in based on the Coflow flow and weight calculation priority index transmitted without obtaining the flow of Coflow in advance
Range is wide.By actual test, the application Coflow dispatching method, the deadline ratio of Coflow higher to priority
Varys and blind dispatching method are substantially reduced.
Detailed description of the invention
Fig. 1 is the structure chart of data center network interchanger；
Fig. 2 is the flow chart of the priority classification method of big data flow of the present invention；
Fig. 3 is the flow chart of the dispatching method of big data flow of the present invention；
Fig. 4 is the dispatching method performance test comparison chart of Coflow of the present invention；
Fig. 5 is the structure chart of the priority classification device of big data flow of the present invention；
Fig. 6 is the structure chart of the dispatching device of big data flow of the present invention.
Specific embodiment
To make the objectives, technical solutions, and advantages of the present invention clearer, right in the following with reference to the drawings and specific embodiments
The present invention is described in detail.
The application assumes that data center is the switch architecture of the non-obstruction of a m × n, and Fig. 1 gives one 3 × 4
The switch architecture schematic diagram of non-obstruction has 3 inbound ports (110) and 4 in the interchanger (100) of Tu1Zhong data center
Exit port (120).Based on this it is assumed that on inbound port and exit port that then network congestion only occurs in.
Coflow is defined, is the data flow of one group of related application,OrWherein { } indicates set, and [] indicates list, and wherein k is the sequence of Coflow
Number, for distinguishing different Coflow, k=1,2 ... K.Any Coflow k contains at least one data flowWherein i is to be somebody's turn to do
Data flowInbound port serial number, inbound port serial number corresponds to the source address of the data flow, and j is the data flowExit port sequence
Number, exit port serial number corresponds to the destination address of the data flow, and w is arrangedkFor the weight of Coflow k.
The application defining ideal weight Coflow optimization problem (Idealized Weighted Coflow
Completion Time Minimization, IWCCTM) realization target be：
Wherein, CkIt is that Coflow k is transmitted the moment, the target that formula (1) is realized is to minimize weight Coflow to complete
Moment.
Based on formula (1) it is found that working as the weight w of Coflow kkWeight w less than Coflow ll, C should be madel≤Ck, i.e.,
Coflow k's is transmitted the moment that is transmitted for being later than Coflow l constantly, i.e. the bigger Coflow correspondence of weight transfers
At constantly more early, it is smaller to be transmitted more late respective weights constantly.
The realization target that IWCCTM can be further obtained based on formula (1), as shown in formula (2) and (3)：
Wherein, formula (2) is the restriction to data center's inbound port,The power of expression
The transmission of great all Coflow in Coflow k is time-consuming to add up,Numerical value is uninterrupted, BWi,jFor i inbound port to j
The connotation of the bandwidth of exit port, formula (2) is：Cumulative being less than of transmission time-consuming of the weight greater than all Coflow of Coflow k
Coflow k's is transmitted moment Ck。
Formula (3) is the restriction to data center's exit port, whereinThe power of expression
The transmission of great all Coflow in Coflow k is time-consuming to add up.The connotation of formula (3) is：Weight is greater than Coflow k's
Time-consuming the adding up of the transmission of all Coflow is transmitted moment C less than Coflow kk。
Based on above-mentioned realization target, the present invention proposes the priority classification method of following big data flow, as shown in Fig. 2,
Include the following steps：
Step 10 (S101)：Update data center big data flow Coflow ordered set F, F include data center just
In the Coflow and Coflow to be transmitted of transmission, enabling k is the serial number of Coflow in F, k=1,2 ... K.
Coflow ordered set F can be Coflow set F or Coflow list F, such as F={ FkOr F=[Fk]。
Step 11 (S102)：It calculates each Coflow in F and has transmitted flow in m inbound port and n exit port, search
Each Coflow has transmitted the minimum value of flow in m inbound port and n exit port and has been denoted as lk, k=1,2 ... K.
It enablesThe data volume sent for Coflow k from inbound port i to exit port j.
For any Coflow, it is calculated in the generation flow of all inbound ports and exit port：
Flow of the Coflow k in inbound port i：
Flow of the Coflow k in exit port j：
Minimum value：
Step 12 (S103)：Update priority index ψk, k=1,2 ... K, ψkWith lkIt is directly proportional, ψkWith wkIt is inversely proportional, wkFor F
The weight of the Coflow of middle serial number k；
Step 13 (S104)：To ψkIt carries out non-decreasing to arrange to obtain ω, the ψ of first item in ωkCorresponding serial number k's
Coflow highest priority, later priority are successively decreased item by item.
For example, if in ω first item ψkCorresponding k=5, the then highest priority of Coflow 5.
The method of the application Fig. 2, user can set data center and receive a new Coflow according to the actual situation
Triggering executes once, or is set as being periodically executed.
Further, step 13 further includes：ψ in ω is sequentially extracted item by itemkCorresponding Coflow serial number k obtains preferential level
Sequence R.
For example, the 1st Coflow serial number k is 5 in ω, then (1)=5 R.
Preferentially, ψk=lk/wk, furthermore can also enable ψk=lk 2/wk 2Or other meet " ψkWith lkIt is directly proportional, ψkWith wkAt
The mathematical formulae of inverse ratio ".
Coflow priority index proposed by the present invention has comprehensively considered weight and minimum port flow, so that having sent
Port flow is smaller, and the bigger Coflow of weight possesses higher priority, refers to compared to only consideration weight, the priority of the application
Mark better reflects actual demand of the Coflow to bandwidth.
As shown in figure 3, including the following steps the invention also includes a kind of big data traffic scheduling method：
Step 20 (S201)：Update data center big data flow Coflow ordered set F, F include data center just
In the Coflow and Coflow to be transmitted of transmission, enabling k is the serial number of Coflow in F, k=1,2 ... K；
Step 21 (S202)：It calculates each Coflow in F and has transmitted flow in m inbound port and n exit port, search
Each Coflow has transmitted the minimum value of flow in m inbound port and n exit port and has been denoted as lk, k=1,2 ... K；
Step 22 (S203)：Update priority index ψk, k=1,2 ... K, ψkWith lkIt is directly proportional, ψkWith wkIt is inversely proportional, wkFor F
The weight of the Coflow of middle serial number k；
Step 23 (S204)：To ψkIt carries out non-decreasing to arrange to obtain ω, sequentially extracts ψ in ω item by itemkCorresponding Coflow sequence
Number k obtains priority order R；
Step 24 (S205)：According to priority order R successively checks in every Coflow the stream that whether there is unassigned bandwidth
Amount1≤i≤m, 1≤j≤n, 1≤k≤K, whereinOrI indicates sequence
Number for R (k) Coflow flow inbound port serial number, j indicate serial number R (k) Coflow flow exit port serial number, if
It is to then follow the steps 25；
Step 25 (S206)：The minimum value in current m inbound port and the remaining bandwidth of n exit port is searched, r is denoted as；
Step 26 (S207)：It will be in the remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j
R bandwidth allocation is givenThe remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j is updated, is returned
Step 24.
For example, the remaining bandwidth of the port of inbound port serial number i is I before step 26m-i；The port of exit port serial number j
Remaining bandwidth be In-j；After then executing the step 26, Im-i=Im-i- r, Im-j=Im-j-r。
Preferentially, ψ in step 23 in Fig. 3k=lk/wk。
The Coflow dispatching method of the application allows the higher Coflow of priority preferentially to obtain bandwidth resources, subtracts
The transmission time and application delay of few Coflow, to realize the transmission optimization of Coflow rank.In addition, the scheduling of the application
Method is applicable in based on the Coflow flow and weight calculation priority index transmitted without obtaining the flow of Coflow in advance
Range is wide.
Use facebook flow rate test Varys, Barrat, Aalo and the application dispatching method (Yosemite), test
As a result as shown in figure 4, the abscissa in figure is priority classification, including Significant, Important, Normal,
Unimportant and Lax, priority are sequentially successively decreased, and ordinate is Factor of Improvement, the calculating side of ordinate
Method is：The deadline of each rank Coflow when optimisation strategy is not used and uses each rank Coflow of dispatching method
The ratio between deadline.
It can be found that Coflow for any priority, the performance of Yosemite (the application) algorithm are intended to from Fig. 4
Better than Barrat, Aalo.
Table 1 is the data list of Fig. 4, compared with Varys, Yosemite (the application) for Significant and
The deadline ratio Varys of both priority of Important higher Coflow, Yosemite (the application) at least lack 20%For Normal rank
Coflow, Yosemite (the application) performance are similar with Varys.Unimportant and Lax rank lower for priority
Coflow, Yosemite (the application) are poorer than the performance of Varys.Therefore in terms of comprehensive, the performance of the application dispatching method is wanted
Better than Varys, Barrat, Aalo.
The data list of 1 Fig. 4 of table
As shown in figure 5, it is applied to data center the invention also includes a kind of priority classification device of big data flow,
The data center network includes m inbound port and n exit port, which includes：
Coflow update module：Big data flow Coflow the ordered set F, F for updating data center include data center
The Coflow and Coflow to be transmitted transmitted, enabling k is the serial number of Coflow in F, k=1,2 ... K；
Port flow computing module：It calculates each Coflow in F and has transmitted flow in m inbound port and n exit port,
Each Coflow is searched to have transmitted the minimum value of flow in m inbound port and n exit port and be denoted as lk, k=1,2 ... K；
Priority index computing module：Update priority index ψk, k=1,2 ... K, ψkWith lkIt is directly proportional, ψkWith wkAt anti-
Than wkFor the weight of the Coflow of serial number k in F；
Coflow categorization module：To ψkIt carries out non-decreasing to arrange to obtain ω, the ψ of first item in ωkCorresponding serial number k's
Coflow highest priority, later priority are successively decreased item by item.
Preferably, in the device of Fig. 5, ψk=lk/wk。
Preferentially, in the device of Fig. 5, Coflow categorization module further includes：ψ in ω is sequentially extracted item by itemkIt is corresponding
Coflow serial number k obtains priority order R.
As shown in fig. 6, the invention also includes a kind of dispatching device of big data flow, it is applied to data center, in data
Heart network includes m inbound port and n exit port, which includes：
Coflow update module：Big data flow Coflow the ordered set F, F for updating data center include data center
The Coflow and Coflow to be transmitted transmitted, enabling k is the serial number of Coflow in F, k=1,2 ... K；
Port flow computing module：It calculates each Coflow in F and has transmitted flow in m inbound port and n exit port,
Each Coflow is searched to have transmitted the minimum value of flow in m inbound port and n exit port and be denoted as lk, k=1,2 ... K；
Priority index computing module：Update priority index ψk, k=1,2 ... K, ψkWith lkIt is directly proportional, ψkWith wkAt anti-
Than wkFor the weight of the Coflow of serial number k in F；
Coflow categorization module：To ψkIt carries out non-decreasing to arrange to obtain ω, sequentially extracts ψ in ω item by itemkCorresponding Coflow
Serial number k obtains priority order R；
Unallocated flow searching module：According to priority order R is successively checked in every Coflow with the presence or absence of unallocated band
Wide flow1≤i≤m, 1≤j≤n, 1≤k≤K, wherein Ori
Indicate the inbound port serial number of the Coflow flow of serial number R (k), j indicates the exit port sequence of the Coflow flow of serial number R (k)
Number, if it is, executing port remaining bandwidth spider module；
Port remaining bandwidth spider module：Search the minimum in current m inbound port and the remaining bandwidth of n exit port
Value, is denoted as r；
Bandwidth allocation module：It will be in the remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j
R bandwidth allocation is givenThe remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j is updated, is returned
Unallocated flow searching module.
Preferentially, in the device of Fig. 6, ψk=lk/wk。
The foregoing is merely illustrative of the preferred embodiments of the present invention, not to limit scope of the invention, it is all
Within the spirit and principle of technical solution of the present invention, any modification, equivalent substitution, improvement and etc. done should be included in this hair
Within bright protection scope.
Claims (10)
1. a kind of priority classification method of big data flow, is applied to data center, the data center network includes m and enters
Port and n exit port, which is characterized in that the described method comprises the following steps：
Step 10：Big data flow Coflow the ordered set F, the F for updating the data center include the data center
The Coflow and Coflow to be transmitted transmitted, enabling k is the serial number of Coflow in the F, k=1,2 ... K；
Step 11：It calculates each Coflow in the F and has transmitted flow in the m inbound port and n exit port, search institute
Each Coflow is stated to have transmitted the minimum value of flow in m inbound port and n exit port and be denoted as lk, k=1,2 ... K；
Step 12：Update priority index ψk, k=1,2 ... K, the ψkWith the lkIt is directly proportional, the ψkWith wkIt is inversely proportional, institute
State wkFor the weight of the Coflow of serial number k in the F；
Step 13：To the ψkIt carries out non-decreasing to arrange to obtain ω, the ψ of first item in the ωkCorresponding serial number k's
Coflow highest priority, later priority are successively decreased item by item.
2. the method according to claim 1, wherein the ψk=lk/wk。
3. the method according to claim 1, wherein the step 13 further includes：The ω is sequentially extracted item by item
Middle ψkCorresponding Coflow serial number k obtains priority order R.
4. a kind of dispatching method of big data flow, be applied to data center, the data center network include m inbound port with
N exit port, which is characterized in that the described method comprises the following steps：
Step 20：Big data flow Coflow the ordered set F, the F for updating the data center include the data center
The Coflow and Coflow to be transmitted transmitted, enabling k is the serial number of Coflow in the F, k=1,2 ... K；
Step 21：It calculates each Coflow in the F and has transmitted flow in the m inbound port and n exit port, search institute
Each Coflow is stated to have transmitted the minimum value of flow in m inbound port and n exit port and be denoted as lk, k=1,2 ... K；
Step 22：Update priority index ψk, k=1,2 ... K, the ψkWith the lkIt is directly proportional, the ψkWith wkIt is inversely proportional, institute
State wkFor the weight of the Coflow of serial number k in the F；
Step 23：To the ψkIt carries out non-decreasing to arrange to obtain ω, sequentially extracts ψ in the ω item by itemkCorresponding Coflow serial number
K obtains priority order R；
Step 24：The flow that whether there is unassigned bandwidth is successively checked in every Coflow by the priority order R1≤i≤m, 1≤j≤n, 1≤k≤K, whereinOrI indicates serial number
For the inbound port serial number of the Coflow flow of R (k), j indicates the exit port serial number of the Coflow flow of serial number R (k), if
It is to then follow the steps 25；
Step 25：The minimum value in presently described m inbound port and the remaining bandwidth of n exit port is searched, r is denoted as；
Step 26：By the r bandwidth allocation in the remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j
To describedUpdate the remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j, return step
24。
5. according to the method described in claim 4, it is characterized in that, the ψk=lk/wk。
6. a kind of priority classification device of big data flow, is applied to data center, the data center network includes m and enters
Port and n exit port, which is characterized in that described device includes：
Coflow update module：It includes described for updating big data flow Coflow the ordered set F, the F of the data center
The Coflow and Coflow to be transmitted that data center is transmitting, enabling k is the serial number of Coflow in the F, k=1,2 ... K；
Port flow computing module：Each Coflow in the F is calculated to have transmitted in the m inbound port and n exit port
Flow searches each Coflow and has transmitted the minimum value of flow in m inbound port and n exit port and be denoted as lk, k=
1,2…K；
Priority index computing module：Update priority index ψk, k=1,2 ... K, the ψkWith the lkIt is directly proportional, the ψkWith
wkIt is inversely proportional, the wkFor the weight of the Coflow of serial number k in the F；
Coflow categorization module：To the ψkIt carries out non-decreasing to arrange to obtain ω, the ψ of first item in the ωkCorresponding serial number
The Coflow highest priority of k, later priority are successively decreased item by item.
7. device according to claim 6, which is characterized in that the ψk=lk/wk。
8. device according to claim 6, which is characterized in that the Coflow categorization module further includes：Sequentially mention item by item
Take ψ in the ωkCorresponding Coflow serial number k obtains priority order R.
9. a kind of dispatching device of big data flow, be applied to data center, the data center network include m inbound port with
N exit port, which is characterized in that described device includes：
Coflow update module：Big data quantity flow Coflow the ordered set F, the F for updating the data center include institute
The Coflow and Coflow to be transmitted that data center is transmitting are stated, enables k for the serial number of Coflow in the F, k=1,2 ...
K；
Port flow computing module：Each Coflow in the F is calculated to have transmitted in the m inbound port and n exit port
Flow searches each Coflow and has transmitted the minimum value of flow in m inbound port and n exit port and be denoted as lk, k=
1,2…K；
Priority index computing module：Update priority index ψk, k=1,2 ... K, the ψkWith the lkIt is directly proportional, the ψkWith
wkIt is inversely proportional, the wkFor the weight of the Coflow of serial number k in the F；
Coflow categorization module：To the ψkIt carries out non-decreasing to arrange to obtain ω, sequentially extracts ψ in the ω item by itemkIt is corresponding
Coflow serial number k obtains priority order R；
Unallocated flow searching module：Successively check whether have distribution band in every Coflow or not by the priority order R
Wide flow1≤i≤m, 1≤j≤n, 1≤k≤K, whereinOri
Indicate the inbound port serial number of the Coflow flow of serial number R (k), j indicates the exit port sequence of the Coflow flow of serial number R (k)
Number, if it is, executing port remaining bandwidth spider module；
Port remaining bandwidth spider module：Search the minimum in presently described m inbound port and the remaining bandwidth of n exit port
Value, is denoted as r；
Bandwidth allocation module：By the r band in the remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j
Width is distributed to describedThe remaining bandwidth of the port of inbound port serial number i and the port of exit port serial number j is updated, is returned
Unallocated flow searching module.
10. device according to claim 9, which is characterized in that the ψk=lk/wk。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810432953.3A CN108833297A (en) | 2018-05-08 | 2018-05-08 | Priority classification method, dispatching method and the device of big data flow |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810432953.3A CN108833297A (en) | 2018-05-08 | 2018-05-08 | Priority classification method, dispatching method and the device of big data flow |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108833297A true CN108833297A (en) | 2018-11-16 |
Family
ID=64148431
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810432953.3A Pending CN108833297A (en) | 2018-05-08 | 2018-05-08 | Priority classification method, dispatching method and the device of big data flow |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108833297A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111478808A (en) * | 2020-04-02 | 2020-07-31 | 清华大学 | Method, system, electronic device and storage medium for assisting configuration update verification |
| CN111628940A (en) * | 2020-05-15 | 2020-09-04 | 清华大学深圳国际研究生院 | Flow scheduling method, device, system, switch and computer storage medium |
| CN114567679A (en) * | 2022-03-25 | 2022-05-31 | 阿里巴巴（中国）有限公司 | Data transmission method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105227488A (en) * | 2015-08-25 | 2016-01-06 | 上海交通大学 | A kind of network flow group scheduling method for distributed computer platforms |
| CN105827545A (en) * | 2016-04-21 | 2016-08-03 | 中国科学院信息工程研究所 | Scheduling method and device of TCP co-flows in data center network |
| CN105871738A (en) * | 2016-05-04 | 2016-08-17 | 北京邮电大学 | Flow-scheduling-based link congestion determination method and device in big data service |
| CN106533981A (en) * | 2016-12-19 | 2017-03-22 | 北京邮电大学 | Multi-attribute based big data flow scheduling method and device |
-
2018
- 2018-05-08 CN CN201810432953.3A patent/CN108833297A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105227488A (en) * | 2015-08-25 | 2016-01-06 | 上海交通大学 | A kind of network flow group scheduling method for distributed computer platforms |
| CN105827545A (en) * | 2016-04-21 | 2016-08-03 | 中国科学院信息工程研究所 | Scheduling method and device of TCP co-flows in data center network |
| CN105871738A (en) * | 2016-05-04 | 2016-08-17 | 北京邮电大学 | Flow-scheduling-based link congestion determination method and device in big data service |
| CN106533981A (en) * | 2016-12-19 | 2017-03-22 | 北京邮电大学 | Multi-attribute based big data flow scheduling method and device |
Non-Patent Citations (5)
| Title |
|---|
| HAN ZHANG, XINGANG SHI, XIA YIN, ZHILIANG WANG: "Yosemite: Efficient Scheduling of Weighted Coflows in Data Centers", 《IEEE》 * |
| HENGKY SUSANTO, HAO JIN, AND KAI CHEN: "Stream: Decentralized Opportunistic Inter-Coflow Scheduling for Datacenter Networks", 《IEEE》 * |
| MEHRNOOSH SHAFIEE，JAVAD GHADERI: "Scheduling Coflows in Datacenter Networks: Improved Bound for TotalWeighted Completion Time", 《ACM SIGMETRICS PERFORMANCE EVALUATION REVIEW》 * |
| SHUO WANG, JIAO ZHANG, TAO HUANG, JIANG LIU, TIAN PAN, AND YUNJI: "A Survey of Coflow Scheduling Schemes for Data Center Networks", 《IEEE》 * |
| ZHEN QIU, CLIFF STEIN , YUAN ZHONG: "Experimental Analysis of Algorithms for Coflow scheduling", 《COMPUTER SCIENCE》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111478808A (en) * | 2020-04-02 | 2020-07-31 | 清华大学 | Method, system, electronic device and storage medium for assisting configuration update verification |
| CN111478808B (en) * | 2020-04-02 | 2021-05-25 | 清华大学 | Method, system, electronic device and storage medium for assisting configuration update verification |
| CN111628940A (en) * | 2020-05-15 | 2020-09-04 | 清华大学深圳国际研究生院 | Flow scheduling method, device, system, switch and computer storage medium |
| CN114567679A (en) * | 2022-03-25 | 2022-05-31 | 阿里巴巴（中国）有限公司 | Data transmission method and device |
| CN114567679B (en) * | 2022-03-25 | 2024-04-02 | 阿里巴巴（中国）有限公司 | Data transmission method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108260169B (en) | QoS guarantee-based dynamic service function chain deployment method | |
| CN112738820B (en) | Dynamic deployment method and device of service function chain and computer equipment | |
| CN108566659B (en) | 5G network slice online mapping method based on reliability | |
| CN110297699B (en) | Scheduling method, scheduler, storage medium and system | |
| CN111385202B (en) | Route distribution method based on virtual network function | |
| CN108965014B (en) | QoS-aware service chain backup method and system | |
| CN108684046B (en) | Random learning-based access network service function chain deployment method | |
| CN109614215A (en) | Stream scheduling method, device, equipment and medium based on deeply study | |
| CN103036792B (en) | Transmitting and scheduling method for maximizing minimal equity multiple data streams | |
| CN108833297A (en) | Priority classification method, dispatching method and the device of big data flow | |
| CN108897606B (en) | Self-adaptive scheduling method and system for virtual network resources of multi-tenant container cloud platform | |
| CN103491024B (en) | Job scheduling method and device for streaming data | |
| Wang et al. | Presto: Towards efficient online virtual network embedding in virtualized cloud data centers | |
| CN111245722B (en) | SDN data center network flow forwarding method based on genetic algorithm | |
| CN112003660B (en) | Dimension measurement method of resources in network, calculation force scheduling method and storage medium | |
| CN111130858A (en) | Dynamic multi-target virtual network mapping method under SD-WAN scene | |
| Botero et al. | A novel paths algebra-based strategy to flexibly solve the link mapping stage of VNE problems | |
| CN104954278A (en) | Bee colony optimization based network traffic scheduling method under multiple QoS (quality of service) constraints | |
| CN107483355B (en) | Data center-oriented online scene low-bandwidth overhead traffic scheduling scheme | |
| CN111866623A (en) | High-efficiency virtual optical network survivability mapping method for service reliability | |
| CN114071582A (en) | Service chain deployment method and device for cloud-edge collaborative Internet of things | |
| Hsu et al. | Virtual network mapping algorithm in the cloud infrastructure | |
| CN111611076B (en) | Fair distribution method for mobile edge computing shared resources under task deployment constraint | |
| CN110311863A (en) | A kind of routed path determines method and device | |
| CN110888728B (en) | Task scheduling method of button cluster server |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181116 |
|
| RJ01 | Rejection of invention patent application after publication |