CN108616569B - A kind of Based on Distributed calculates the light Measurement Request dispatching method of application - Google Patents
A kind of Based on Distributed calculates the light Measurement Request dispatching method of application Download PDFInfo
- Publication number
- CN108616569B CN108616569B CN201810253581.8A CN201810253581A CN108616569B CN 108616569 B CN108616569 B CN 108616569B CN 201810253581 A CN201810253581 A CN 201810253581A CN 108616569 B CN108616569 B CN 108616569B
- Authority
- CN
- China
- Prior art keywords
- measurement request
- module
- group
- multicast
- measurement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04B—TRANSMISSION
- H04B10/00—Transmission systems employing electromagnetic waves other than radio-waves, e.g. infrared, visible or ultraviolet light, or employing corpuscular radiation, e.g. quantum communication
- H04B10/40—Transceivers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/16—Arrangements for providing special services to substations
- H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
- H04L12/185—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast with management of multicast group membership
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Electromagnetism (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Small-Scale Networks (AREA)
- Computer And Data Communications (AREA)
- Optical Communication System (AREA)
Abstract
The invention discloses the light Measurement Request dispatching methods that a kind of Based on Distributed calculates application, belong to computer communication field.Building Based on Distributed first calculates the light Measurement Request dispatching platform applied, and collects data information and is simultaneously grouped；If some grouping reaches trigger condition, information group is sent to scheduling trigger module；Then the width for calculating the group submits to light resource allocation and Measurement Request calculates scheduler module, and practical distribution is carried out for each group according to width, allocation result is submitted to Measurement Request distribution module, topology adjustment module and remaining light resource respectively and fills module.Measurement Request distribution module and topology adjustment module establish OCS connection, corresponding Measurement Request is transmitted in starting by dedicated control channel.Last residue light resource filling module preferentially selects available Measurement Request from the lesser Measurement Request group of width.The present invention significantly improves the performance of distributed computing application, and network performance is good, improves the performance of distributed computing application.
Description
Technical field
The invention belongs to computer communication field, specifically a kind of Based on Distributed calculates the light Measurement Request scheduling of application
Method.
Background technique
A large amount of distributed computing applications, these distributed computings application fortune are often run in data center network now
Row is often restricted by network bandwidth, i.e., calculated performance is limited by physical machine in the multiple servers or virtual machine of data center
Between network communication.The network communication in data center how is solved, is existing to promote the performance of distributed computing application
The problem of academia one pays much attention to.
The distributed computing application have the characteristics that one it is common: usually contain a large amount of a pair of of various flow.Such as
When reducer writes storing data to HDFS during MapReduce and in some distributed machines learning algorithm synchronization parameters
When, often data volume is bigger for the network flow used, and it is long there are the time, therefore this kind of network flows are distributed meters
The important component for calculating the network demand of application influences the performance of distributed computing application very big.
Existing distributed computing architecture often regards data center network as a flight data recorder, passes through pair of upper layer application
The encapsulation of network is to transmit similar flow.For example the Hadoop distributed memory system HDFS that is defined using oneself is counted
According to backup, the transmission of this pair of of various flow is carried out using the transmission mode of pipeline flow；Spark uses the P2P of similar BT
Mode handles such a pair of of various flow.But this implementation method be it is inefficient, have much room for improvement.
Existing relatively good method is exactly using handling one-to-many flow based on the optical switching system of fiber coupler
Transmission problem.Because photo-coupler can support the duplication to optical signal, it is very suitable to transmission one-to-many multicast flow.
As shown in Figure 1, the equipment is that some optical ports based on a large-scale optical switching matrix, on optical switching matrix are connected to
On the ToR interchanger (Top of Rack Switch, i.e. machine top cross are changed planes) of data center, other ports are connected to
On photo-coupler.By reconfiguring to optical port, which can transmit out more by any ToR in data center
It broadcasts flow and is transferred on photo-coupler and replicated, is then transferred to again on its corresponding destination ToR, to complete one-to-many
Network flow transmission.
Due to the large capacity of optical transport, the advantages that big bandwidth, low time delay and low energy consumption, make supporting data in this way
One-to-many multicast in center is advantageous when requesting.It is worth noting that, being based on photo-coupler other than this equipment
Optical switching system can all support multicast.As long as corresponding multi-case data to be input to the source of coupler, then will
The output end of coupler is connected to the destination of corresponding multicast.
But in this optical switching system, the mode of network transmission is different from circuit switching, its transmission mode is
There is light resources for the control and scheduling of light Measurement Request based on optical path circuit switching (OCS), therefore in such systems
The problem of distribution.For example, establishing a source ToR if only one light receiver/transmitter on each ToR in data center
After connecting with the OCS of several purpose ToR, this source ToR cannot send light multicast to other ToR for not establishing OCS connection
Data.But in electric network, since all data are swapped by electric interchanger, would not exist in this way
Light resource constraint.Therefore, the reconstruct only connected by OCS appropriate, and light Measurement Request is subjected to rational management,
The advantage of light multicast system can be played to the full extent, to improve the performance of distributed computing application.
The network problem that data center is solved based on this light multicast device has existing research in international conference or phase
Article was delivered in periodical, but existing Measurement Request dispatching method is all only to consider network performance, or only take into account multicast
Request itself.There is no the actual demands for considering Distributed Application, although which results in these technologies can allow in data
The completion speed of Measurement Request in the heart has a certain upgrade, or data center network performance is allowed to have a little promotion, but most
The performance of whole distributed computing application does not reach good effect but.
The essential requirement of distributed computing application should be the deadline of task, i.e., come from user's angle of application
It sees, the Network status being not relevant in this system, as long as the speed that this application is completed is fast, this application effect is exactly
's.In order to probe into distributed computing using real network demand and network characteristic, a kind of document 1: network pumping of cluster application
As；Mosharaf Chowdhury,Ion Stoica；Propose a kind of thought of association stream in October, 2012；This article points out,
The deadline of a task in data center is a stream being most transmitted slowly in all streams as caused by the task
It determines.That is, when considering the network demand of distributed computing application, it should while considering from the same application
All streams.It is good to want the performance for allowing distributed computing to apply, it is necessary to make all streams from the same application all complete as early as possible
At, rather than individually consider the deadline of each stream in network.Since distributed computing application can generate a large amount of network
Traffic requests, and the request of these network flows usually contain it is a large amount of it is one-to-many spread it is defeated；Improve the pass of distributed computing application
Key point be that how cooperative scheduling these all one-to-many streams from the same application, transfer them can as early as possible
At.
But now all of smooth multicast device dispatching method does not all account for this factor.For example, document 2: for number
The light multicast system designed according to central site network；Payman Samadi,Varun Gupta,Junjie Xu,Howard Wang,
Gil Zussman,and Keren Bergman；Propose within 2015 light multicast system optical multicast system
(OMS), wherein the method for using the maximum Measurement Request of prioritised transmission data volume；Document [3]: high-performance is accelerated by light multicast
Data analysis application；YitingXia,T.S.Eugene Ng,Xiaoye Steven Sun；It is adopted in the Blast proposed in 2015
Method is: first acquiring each photoreceiver for each Measurement Request and needs received data volume, then prioritised transmission
The Measurement Request that photoreceiver needs received data volume more；Document 4: to accelerate the mixed of high-performance data center applications design
Light combination electricity multicast；Jinzhen Bao,Baokang Zhao,Dezun Dong,Zhenghu Gong；What in August, 2017 proposed
The HERO Measurement Request that then prioritised transmission by the way of FIFO arrives first.Dispatching algorithm in these systems all considers only
The performance of single multicast or the performance of whole network, and there is no the performances in view of distributed computing application.Although literary
Offer 5: for the multicast scheduling based on adjustability constraint of light data center exchange design；Kamran Keykhosravi,Houman
Rastegarfar,Erik Agrell；Propose the dispatching method specifically for this kind of light multicast systems in March, 2017, but
The document also considers only time delay and throughput performance.After all, these methods are not all effectively in view of using this
The demand of body, and only from the point of view of data center network.
Therefore, in the application of actual distributed computing, although above method can improve the performance of network well,
And the performance of distributed computing application is improved to a certain extent, but there is no in view of the demand of application, also do not have inherently
There is the relevance in view of Measurement Request in data center, is difficult to that distributed computing application is made to reach very excellent in these processes
Performance.
But if directly dispatching the Measurement Request in the system using the thought in document 1, it is also difficult to be applicable in.Cause
Scheduling strategy to propose in the document is to be directed to traditional electric exchange network, and the network model in document assumes that entirely
Network is a large-scale electric interchanger, the scheduling strategy proposed on basis herein.This strategy does not account for light exchange
In network the problem of the limitation of light resource, therefore and it is not suitable for.
Summary of the invention
The present invention is in view of the above-mentioned problems, propose a kind of light Measurement Request dispatching method of Based on Distributed calculating application；
The network demand of distributed computing application and the resource constraint of optical switching network are considered simultaneously.For based on optical coupling
The optical switching system of device defines by being grouped to the Measurement Request of arrival, and to each grouping and calculates new index,
Then it is scheduled.The dispatching method can solve the deficiency in above-mentioned existing method, to largely reduce distributed
The deadline of application is calculated, to improve the performance that distributed computing is applied in data center.
The Based on Distributed calculates the light Measurement Request dispatching method of application, the specific steps are as follows:
Step 1: building Based on Distributed first calculates the light Measurement Request dispatching platform applied；
The light Measurement Request dispatching platform is disposed on the server or has the dedicated network control of computing function single
In member；It collects including Measurement Request information and is provided with grouping module, scheduling trigger module, Measurement Request group width computing module, light
Source distribution calculates scheduler module, Measurement Request distribution module, topology adjustment module and remaining light resource with Measurement Request and fills
Module.
Data channel congestion causes the delay of control information in order to prevent, which is connected by dedicated control channel
Switching equipment and server in data center；Optical switching system is connected by dedicated control channel simultaneously.
It is collected in data center on ToR with grouping module by dedicated control channel Step 2: Measurement Request information is collected
Transceiver port number information, and the Measurement Request information from distributed computing application and is grouped.
According to distributed computing apply in belong to multiple Measurement Requests of the same application and be divided into one group.
Step 3: being directed to each grouping, judge whether the number of Measurement Request in some grouping reaches trigger condition, if
It is that Measurement Request information is collected and the complete Measurement Request information group is sent scheduling trigger module by grouping module, enters
Step 4；Otherwise, newly arrived Measurement Request is continued waiting for；
Trigger condition is defined as: when there is the all-multicast request of a new distributed computing application all here, then reaching
To trigger condition.
Step 4: scheduling trigger module issues instructions to Measurement Request group width computing module, the Measurement Request group is calculated
Width submit to light resource allocation and Measurement Request and calculate scheduler module.
Calculating width, specific step is as follows:
Step 401 is directed to complete Measurement Request group m, and the active ToR node of institute and all purposes ToR for obtaining this group of m save
Point；
Source ToR nodename are as follows: S1, S2 ... Sn；Purpose ToR nodename are as follows: D1, D2 ... Dn.
Step 402, to some source node S/destination node D, find the node need to send/received belong to multicast and ask
The all-multicast of group m is asked to request, and record size respectively；
Measurement Request are as follows: q1, q2 ... qn, corresponding size are s1, s2 ... sn.
Step 403, in the source node S/destination node D, all-multicast request on the node is empty one by one from big to small
It is quasi- to be assigned in the optical sender/receiver that do not request, and record the stream of required transmission in each emittor/receiver
Amount.
Assuming that there is k optical sender/receiver in the source node S/destination node D, it is denoted as t1, t2 ... tk/r1, r1 ...
rk；
Emittor/receiver needs the flow transmitted, is denoted as T1, T2 ... Tk/R1, R2 ... Rk.
When all-multicast number of requests is less than optical sender/receiver quantity on source node S/destination node D, do not divide
The discharge record transmitted required for the optical sender/receiver matched is 0.
Step 404 sorts the flow that emittor/receiver transmits as needed from small to large；And judge whether that there are also surplus
Remaining Measurement Request, if so, by remaining Measurement Request in order one by one virtual assigned to sequence after emittor/receiver
In, and the flow T/R to be transmitted is updated, it is finished until the all-multicast request of the group is all assigned.Otherwise, directly updating will pass
Defeated flow T/R；
Step 405, according to the maximum flow T/R of the source node S/destination node D after update, the minimum that is sent needed for calculating
Time Q；
Step 406, to all ToR source nodes of Measurement Request group m and destination node, find out the minimum of each node respectively
Time Q value；
Step 407 selects the maximum value of all Q values as the width W of Measurement Request group m.
Step 5: repeat the above steps, all Measurement Request group m1 for not completing transmission in triggering moment computing system,
M2 ... .mn corresponding width W1, W2 ... Wn carry out subsequent practical distribution；
Step 6: it is that each Measurement Request group carries out in fact that light resource allocation and Measurement Request, which calculate scheduler module according to width,
Border distribution, and allocation result is submitted to Measurement Request distribution module, topology adjustment module and remaining light resource fill mould respectively
Block.
It is preferential to distribute the lesser Measurement Request group of width under limited smooth resource according to the width that Measurement Request is grouped,
Until light resource uses until exhausted.Limited smooth resource includes on the optical transceiver resource constraint and light multicast device of each ToR
The limitation of coupler quantity.
Specific step is as follows:
Step 601 establishes topological diagram G (v, e), while initializing the quantity k and optical coupling of optical transceiver on each ToR
The quantity z of device；
V is vertex, and each node v represents each Measurement Request being submitted in figure.E is side, i.e., between Measurement Request
Constraint, if two Measurement Requests have identical source node or identical destination node, the point represented by the two Measurement Requests
Between establish connection e, record the original state that established figure is G (v, e).
Step 602, selected from all-multicast request group m1, m2 ... .mn minimum widith W's and include unappropriated multicast
The Measurement Request group m* of request；
It is unappropriated Measurement Request in initial time all-multicast request group m1, m2 ... .mn.
Step 603 successively selects unappropriated Measurement Request q in Measurement Request group m* to be handled, and every processing one
Measurement Request, photo-coupler z subtract 1 certainly；
The specific process is as follows: for the Measurement Request q that currently chooses, judge corresponding point whether in topological diagram G, such as
Fruit is, then Measurement Request q is added in allocation result collection Y and directly carries out practical distribution, and by the point for representing Measurement Request q with
And there is the point on side all to delete from topological diagram G with it；One Measurement Request q of every distribution just needs to consume a photo-coupler.
Step 604 judges whether photo-coupler z is greater than 0, if so, entering step 605；Otherwise, terminate algorithm and return
Result set Y；
Step 605 continues to select next Measurement Request q, and return step 603 does same processing, until may not be used in m*
To continue the Measurement Request of distribution；
Step 606 judges whether there are also remaining points in topological diagram G, and also unappropriated Measurement Request group in system,
If so, entering step 607；Otherwise, 608 are entered step；
Step 607, selection has the Measurement Request group m* of minimum widith, return step 603 from unappropriated group；Otherwise,
Enter step 608；
There is no all-multicast request group in remaining point or system to be all assigned in step 608, topological diagram G, then
It records the remaining optical transceiver quantity k on each TOR and subtracts 1 certainly.
Step 609 judges whether k！=0, if so, topological diagram G (v, e) is reduced to original state, delete in Y
The representative point of the all-multicast request of record, retains the point for having even side with it, return step 602.Otherwise, k==0 terminates
Algorithm returns to allocation result collection Y.
Step 7: Measurement Request distribution module and topology adjustment module control optical switching system by dedicated control channel
OCS connection is established, while corresponding ToR and server being notified to start the Measurement Request in transmission result set Y.
Step 8: the server of data center will inform remaining light resource fill mould after certain Measurement Request is transmitted
Block, the module are attempted to be calculated according to last width as a result, preferentially selecting from the lesser Measurement Request group of width available
Measurement Request.
If just do not operated without any available Measurement Request.
The present invention has the advantages that
1), a kind of Based on Distributed calculates the light Measurement Request dispatching method of application, combines association stream thought, it is contemplated that
The actual demand of distributed computing application, is grouped scheduling to Measurement Request, can significantly promote distributed computing and answer
Performance.
2), a kind of Based on Distributed calculates the light Measurement Request dispatching method of application, it is contemplated that the light of optical transceiver provides
Source constraint carries out the scheduling of light Measurement Request by modeling reasonably distribution light resource, it is good to make network performance, while improving distribution
Formula calculates the performance of application.
Detailed description of the invention
Fig. 1 is the structural schematic diagram of the existing large-scale optical switching matrix of the present invention.
Fig. 2 is the light Measurement Request dispatching method flow chart that a kind of Based on Distributed of the present invention calculates application.
Fig. 3 is the schematic diagram for the light Measurement Request dispatching platform that present invention building Based on Distributed calculates application.
Fig. 4 is the width method flow chart that the present invention was allocated to each Measurement Request and calculated the Measurement Request group.
Fig. 5 be the present invention according to width be the practical method flow diagram distributed of each Measurement Request group.
Fig. 6 is the comparison of the mentioned method of the present invention and the prior art with the growth average completion time of Measurement Request quantity
Figure.
Fig. 7 is the mentioned method of the present invention and the prior art as the growth of optical transceiver quantity on each ToR is averagely completed
The comparison diagram of time.
Specific embodiment
Below in conjunction with drawings and examples, the present invention is described in further detail.
The invention proposes the light Measurement Request dispatching methods that a kind of Based on Distributed calculates application；It is related to distributed meter
Calculation, optical switched technology, data center network, network-control and dispatching device, specifically in the light multicast system of data center
How a kind of method of the Measurement Request that in distributed computing application frequently occurs is dispatched.Distributed computing is considered simultaneously to answer
The resource constraint of network demand and optical switching network.In the method, system is exchanged for the light based on photo-coupler
System defines by being grouped to the Measurement Request of arrival, and to each grouping and calculates new index, then adjusted
Degree；It can solve the deficiency in existing method, so that the deadline of distributed computing application is largely reduced, to mention
The performance that distributed computing is applied in high data center.
As shown in Figure 2, the specific steps are as follows:
Step 1: building Based on Distributed first calculates the light Measurement Request dispatching platform applied；
The light Measurement Request dispatching platform is as shown in figure 3, dispose on the server or have the dedicated of computing function
On network control unit；It collects including Measurement Request information and is calculated with grouping module, scheduling trigger module, Measurement Request group width
Module, light resource allocation and Measurement Request calculate scheduler module, Measurement Request distribution module, topology adjustment module and remaining light
Resource fills module.
Data channel congestion causes the delay of control information in order to prevent, which is connected by dedicated control channel
Switching equipment and server in data center；Optical switching system is connected by dedicated control channel simultaneously.
It is collected in data center on ToR with grouping module by dedicated control channel Step 2: Measurement Request information is collected
Transceiver port number information, and the Measurement Request information from distributed computing application reached and is grouped.
According to distributed computing apply in belong to multiple Measurement Requests of the same application and be divided into one group.
Since the time that is finally completed of distributed computing application is determined by the traffic requests completed the latest,
In scheduling, the relevance between traffic requests is only set up, could more reasonably distribute and dispatches these traffic requests.Cause
This needs these light Measurement Requests being divided into one if containing multiple smooth Measurement Requests in a distributed computing application
Group needs to consider the relevance between them when they are allocated and are dispatched.In this step, there are some key messages
The collection of Measurement Request information and grouping module should be reported to by distributed computing application:
1. each Distributed Application start when should tell this distributed computing of the module apply id and this
Using how many a Measurement Requests included；
2. should include: the source host ip of multicast, multicast using the required information reported for each Measurement Request
All purposes host ip, task id belonging to the data volume size s and this Measurement Request for needing to transmit.
Wherein, source and purpose ip are used to judge the position ToR where source and destination node, and transmitted data amount is to calculate multicast
The number of Measurement Request is for being grouped to Measurement Request in the parameter of request group width, task id and application.
Step 3: being directed to each grouping, Measurement Request information, which is collected, judges Measurement Request in some grouping with grouping module
Number whether reach trigger condition, the complete Measurement Request is believed with grouping module if so, Measurement Request information is collected
Breath group is sent to scheduling trigger module, enters step four；Otherwise, newly arrived Measurement Request is continued waiting for；
Since the calculating of resource allocation and the distribution of light resource need certain time delay, if continually operation comes again
Resource is distributed, very big pressure can be caused to network.Therefore, it is necessary to a suitable trigger conditions, so that scheduling trigger module
Only triggered when suitable.Since all calculate nodes in a distributed computing application can not start or complete simultaneously
Their calculating task, the multiple Measurement Requests for being consequently belonging to the same distributed computing application will not reach simultaneously, and work as this
When the all-multicast request of a distributed computing application does not all arrive at, controller can not just know this Distributed Application institute
The complete attribute of the Measurement Request group of generation, it is exactly inefficient at this moment pleading these multicasts and being allocated scheduling.Therefore it triggers
Condition is defined as: when there is the all-multicast request of a new distributed computing application all here, then reaching device triggering item
Part；Namely reach the Measurement Request number threshold value of each grouping of application layer informing, which is determined by application layer.
Step 4: scheduling trigger module issues instructions to Measurement Request group width computing module, notify the module calculates should
The width of Measurement Request group submits to light resource allocation and Measurement Request calculates scheduler module.
When Measurement Request group width computing module receives the instruction of scheduling trigger module, it is each more that it will collect triggering moment
Broadcast the data volume that request needs to receive and dispatch on related ToR.According to these information, the width of each Measurement Request group is calculated one by one.
The assigned remaining data amount size with the Measurement Request of transmission needs to be counted by distribution before triggering moment
It calculates application and informs the collection of Measurement Request information and grouping module.Real-time Measurement Request information is only obtained, can just make most has
The distribution and scheduling of effect.Since there are relevances between Measurement Request, processes and adjust only for single Measurement Request
Degree be it is nonsensical, need to calculate each Measurement Request group and at least need how long to transmit under limited smooth resource
It completes, remembers that this time is the width W of this Measurement Request group.If this index can be calculated, time-consuming is needed according to
The thought that lesser task is preferentially completed can be obtained close to optimal average task completion time.Due to there is light receipts on ToR
The limitation of hair machine quantity, network system once just can not all send the all-multicast request of a Measurement Request group,
It therefore must be it is to be understood that this Measurement Request group be to encounter bottleneck on which ToR, and find out depositing due to this bottleneck
In the case where not considering to have the intervention of other Measurement Request groups, how long this Measurement Request group needs could complete it
Transmission.Wonder this time, it is necessary first to calculate for some Measurement Request grouping for, it is relative certain
One ToR (if including a ToR in the source destination node of any one Measurement Request in this Measurement Request group, claims this
A ToR be correlation ToR) on all traffic requests of the Measurement Request group are all sent or are received required minimum completely
Time Q.Q value is all calculated to all correlation ToR of this Measurement Request group, the width W of this Measurement Request group is defined as this
The maximum value of a all Q values of Measurement Request group.
As shown in Figure 4, the specific steps are as follows:
Step 401 is directed to complete Measurement Request group m, and the active ToR node of institute and all purposes ToR for obtaining this group of m save
Point；
Source ToR nodename are as follows: S1, S2 ... Sn；Purpose ToR nodename are as follows: D1, D2 ... Dn.
Step 402, to some source node S/destination node D, find the node need to send/received belong to multicast and ask
The all-multicast of group m is asked to request, and record size respectively；
Measurement Request are as follows: q1, q2 ... qn, corresponding size are s1, s2 ... sn.
Step 403, in the source node S/destination node D, all-multicast request on the node is empty one by one from big to small
It is quasi- to be assigned in the optical sender/receiver that do not request, and record the stream of required transmission in each emittor/receiver
Amount.
Assuming that there is k optical sender/receiver in the source node S/destination node D, it is denoted as t1, t2 ... tk/r1, r1 ...
rk；
Emittor/receiver needs the flow transmitted, is denoted as T1, T2 ... Tk/R1, R2 ... Rk.
When all-multicast number of requests is less than optical sender/receiver quantity on source node S/destination node D, do not divide
The discharge record transmitted required for the optical sender/receiver matched is 0.
Step 404 sorts the flow that emittor/receiver transmits as needed from small to large；And judge whether that there are also surplus
Remaining Measurement Request, if so, transmitting of the virtual assigned to after sorting from big to small one by one in order by remaining Measurement Request
In machine/receiver, and the flow T/R to be transmitted is updated, is finished until the all-multicast request of the group is all assigned.Otherwise, directly
It connects and updates the flow T/R to be transmitted；
Step 405, according to the maximum flow T/R of the source node S/destination node D after update, the minimum that is sent needed for calculating
Time Q；
Step 406, to all ToR source nodes of Measurement Request group m and destination node, find out the minimum of each node respectively
Time Q value；
Step 407 selects the maximum value of all Q values as the width W of Measurement Request group m.
This width refers to that this Measurement Request group at least needs how long to pass under existing smooth resource on earth
It has sent.For each Measurement Request group, their corresponding W values can be calculated.
Step 5: repeat the above steps, all Measurement Request group m1 for not completing transmission in triggering moment computing system,
M2 ... .mn corresponding width W1, W2 ... Wn carry out subsequent practical distribution；
Step 6: it is that each Measurement Request group carries out in fact that light resource allocation and Measurement Request, which calculate scheduler module according to width,
Border distribution, and allocation result is submitted to Measurement Request distribution module, topology adjustment module and remaining light resource fill mould respectively
Block.
It is preferential to distribute the lesser Measurement Request group of width under limited smooth resource according to the width that Measurement Request is grouped,
Until light resource uses until exhausted.Limited smooth resource includes on the optical transceiver resource constraint and light multicast device of each ToR
The limitation of coupler quantity.
The preferential the smallest multicast group request of transmission width is proposed the performance applied to distributed computing in data center
It rises.But optical switching system has exchanged some differences with electricity, such as there are optical transceiver port is restricted, optical link resource
Limitation.Although being aware of which Measurement Request group should preferentially distribute, each Measurement Request of distribution, Yi Jiru how are fallen on
What reasonably distributes light resource so that more Measurement Requests can be transmitted or a problem to be solved.In order to
It is calculated by Optimum distribution formula for the purpose of applying, and will have the Measurement Request of transceiver conflict reasonable under limited smooth resource situation
Distribution is gone down, as shown in Figure 5, the specific steps are as follows:
Step 601 establishes topological diagram G (v, e), while initializing the quantity k and optical coupling of optical transceiver on each ToR
The quantity z of device；
V is vertex, and each node v represents each Measurement Request being submitted in figure.E is side, i.e., between Measurement Request
Constraint, if two Measurement Requests have identical source node or identical destination node, the point represented by the two Measurement Requests
Between establish connection e, record the original state that established figure is G (v, e).
Step 602, selected from all-multicast request group m1, m2 ... .mn minimum widith W's and include unappropriated multicast
The Measurement Request group m* of request；
It is unappropriated Measurement Request in initial time all-multicast request group m1, m2 ... .mn.
Step 603 successively selects unappropriated Measurement Request q in Measurement Request group m* to be handled, and every processing one
Measurement Request, photo-coupler z subtract 1 certainly；
The specific process is as follows: for the Measurement Request q that currently chooses, judge corresponding point v whether in topological diagram G,
Practical distribution is directly carried out if it is, Measurement Request q is added in allocation result collection Y, and the point that Measurement Request q will be represented
V and there is the point on side all to delete from topological diagram G with it；One Measurement Request q of every distribution just needs to consume a photo-coupler.
Step 604 judges whether photo-coupler z is greater than 0, if so, entering step 605；Otherwise, do not have in light multicast device
There is more available photo-coupler, terminate algorithm and returns the result collection Y；
Step 605 continues to select next Measurement Request q, and return step 603 does same processing, until may not be used in m*
To continue the Measurement Request of distribution；
Step 606 judges whether there are also remaining points in topological diagram G, and also unappropriated Measurement Request group in system,
If so, entering step 607；Otherwise, 608 are entered step；
Step 607, the Measurement Request group m* that next minimum widith is selected from unappropriated group, return step 603；It is no
Then, 608 are entered step；
There is no all-multicast request group in remaining point or system to be all assigned in step 608, topological diagram G, then
It records the remaining optical transceiver quantity k on each TOR and subtracts 1 certainly.
Step 609 judges whether k！=0, if so, topological diagram G (v, e) is reduced to original state, delete in Y
The representative point of the all-multicast request of record, retains the point for having even side with it, return step 602.Otherwise, k==0 terminates
Algorithm returns to allocation result collection Y.
After obtaining allocation result collection Y, light resource allocation and Measurement Request calculate scheduler module and calculated result are informed multicast
Distribution module, topology adjustment module and remaining light resource is requested to fill module, to ask to multicast in data center network
The control of the control and optical switching system asked.
Step 7: Measurement Request distribution module and topology adjustment module control optical switching system by dedicated control channel
OCS connection is established, while corresponding ToR and server being notified to start the Measurement Request in transmission result set Y.
In this course, Measurement Request distribution module and topology adjustment module carry out network according to allocation result practical
Operation.
Step 8: the server of data center will inform that remaining light resource is filled out after certain any Measurement Request is transmitted
Mold filling block, which attempts to be calculated according to last width as a result, preferentially selection can from width lesser Measurement Request group
Measurement Request.
If just do not operated without any available Measurement Request.
Once there is Measurement Request to complete, the light exchange resource in network system will get off some free time.In order to
Light exchange resource is made full use of, will be checked in existing Measurement Request either with or without the Measurement Request that can directly distribute.According to
The width of the Measurement Request group calculated before searches for all-multicast request from big to small, if there is idle light can be utilized to exchange
The request then directly is distributed by the Measurement Request of resource transmission.Although it is of course also possible to some light exchange resource
Free time gets off, and still can be placed and be dispatched to system under idling-resource limited in this way without any Measurement Request
In, at this moment just do not operate.At the same time, which should continue the Measurement Request that arrival is collected from distributed computing application
Information, if there is new Measurement Request group completely reaches, which will be allocated and adjust to all Measurement Requests again
Degree.
The present invention is emulated, and is compared with existing method, to the average completion time of distributed computing application
It improves significantly.In simulations, entire data center includes 100 ToR, and each ToR has 40 servers to be attached thereto.It is false
If optical link rate is 10Gbps.A certain number of Measurement Requests are generated at random, and the uninterrupted of these Measurement Requests is obeyed
Being uniformly distributed between 500Mbits to 2.5Gbits, their destination number obey being uniformly distributed between 2 to 10.
Assuming that these Measurement Requests reach simultaneously, and do not have new Measurement Request and reach, thus compare the present invention with
The effect of previous Measurement Request dispatching method.Then, these Measurement Requests are grouped into Measurement Request group at random in emulation.One
A Measurement Request group includes 2 to 10 Measurement Requests.Assume that the coupler of light multicast device there are 25 in emulation, each coupler
There are enough ports to support the Measurement Request in emulation.Because the deadline of a Measurement Request group can be directly anti-
The network transmission deadline for answering a distributed computing to apply, in simulations, a Measurement Request group just represents a distribution
Formula calculates application.
As shown in fig. 6, when having 1 optical transceiver on fixed each ToR, the Measurement Request then reached in change system
Quantity, it can be seen from the figure that method advantage of the invention is also relatively previous as the existing Measurement Request amount in system increases
Method promoted it is more obvious.
Shown in Fig. 7 the result is that when having 500 Measurement Requests in fixed system, the light then changed on each ToR is received
Hair machine quantity obtains.As can be seen that in the case where there is the limitation of coupler quantity, with optical transceiver quantity on each ToR
Increase, the coupler on light multicast device can be fully occupied.Under the light exchange resource of same finite, this method can compare
Previous method significantly reduces the time-consuming of distributed computing application.
Claims (3)
1. the light Measurement Request dispatching method that a kind of Based on Distributed calculates application, which is characterized in that specific step is as follows:
Step 1: building Based on Distributed first calculates the light Measurement Request dispatching platform applied；
The light Measurement Request dispatching platform is disposed on the server or is had in the dedicated network control unit of computing function；
It is collected and grouping module, scheduling trigger module, Measurement Request group width computing module, light resource allocation including Measurement Request information
Scheduler module, Measurement Request distribution module, topology adjustment module and remaining light resource, which are calculated, with Measurement Request fills module；
Data channel congestion causes the delay of control information in order to prevent, which connects data by dedicated control channel
Switching equipment and server in center；Optical switching system is connected by dedicated control channel simultaneously；
Step 2: Measurement Request information, which is collected, passes through the receipts on ToR in dedicated control channel collection data center with grouping module
The information of hair machine port number, and the Measurement Request information from distributed computing application, and be grouped；
Step 3: being directed to each grouping, judge whether the number of Measurement Request in some grouping reaches trigger condition, if so,
Measurement Request information is collected and complete Measurement Request information group is sent scheduling trigger module by grouping module, is entered step
Four；Otherwise, newly arrived Measurement Request is continued waiting for；
Step 4: scheduling trigger module issues instructions to Measurement Request group width computing module, the width of the Measurement Request group is calculated
Degree submits to light resource allocation and Measurement Request calculates scheduler module；
Calculating width, specific step is as follows:
Step 401 is directed to complete Measurement Request group m, obtains the active ToR node of institute and all purposes ToR node of this group of m；
Source ToR nodename are as follows: S1, S2 ... Sn；Purpose ToR nodename are as follows: D1, D2 ... Dn；
Step 402, to some source node S/destination node D, find the node need to send/received belong to Measurement Request group
The all-multicast of m is requested, and record size respectively；
Measurement Request are as follows: q1, q2 ... qn, corresponding size are s1, s2 ... sn；
Step 403, in the source node S/destination node D, one by one virtual point from big to small by all-multicast request on the node
It is fitted in the optical sender/receiver that do not request, and records the flow of required transmission in each emittor/receiver；
Assuming that there is k optical sender/receiver in the source node S/destination node D, it is denoted as t1, t2 ... tk/r1, r1 ... rk；
Emittor/receiver needs the flow transmitted, is denoted as T1, T2 ... Tk/R1, R2 ... Rk；
When all-multicast number of requests is less than optical sender/receiver quantity on source node S/destination node D, do not distribute
The discharge record transmitted required for optical sender/receiver is 0；
Step 404 sorts the flow that emittor/receiver transmits as needed from small to large；And judge whether that there are also remaining
Measurement Request, if so, by remaining Measurement Request in order one by one in the emittor/receiver after virtual assigned to sequence,
And the flow T/R to be transmitted is updated, it is finished until the all-multicast request of the group is all assigned；Otherwise, directly updating will transmit
Flow T/R；
Step 405, according to the maximum flow T/R of the source node S/destination node D after update, the minimum time that is sent needed for calculating
Q；
Step 406, to all ToR source nodes of Measurement Request group m and destination node, find out the minimum time of each node respectively
Q value；
Step 407 selects the maximum value of all Q values as the width W of Measurement Request group m；
Step 5: repeat the above steps, all Measurement Request group m1 for not completing transmission in triggering moment computing system,
M2 ... .mn corresponding width W1, W2 ... Wn carry out subsequent practical distribution；
Step 6: it is that each Measurement Request group carries out practical point that light resource allocation and Measurement Request, which calculate scheduler module according to width,
Match, and allocation result is submitted into Measurement Request distribution module, topology adjustment module and remaining light resource respectively and fills module；
It is preferential to distribute the lesser Measurement Request group of width under limited smooth resource according to the width that Measurement Request is grouped, until
Light resource uses until exhausted；Limited smooth resource includes the coupling on the optical transceiver resource constraint and light multicast device of each ToR
The limitation of device quantity；
Specific step is as follows:
Step 601 establishes topological diagram G (v, e), while initializing the quantity k and photo-coupler of optical transceiver on each ToR
Quantity z；
V is vertex, and each node v represents each Measurement Request being submitted in figure；E is side, i.e. constraint between Measurement Request,
If two Measurement Requests have identical source node or identical destination node, built between the point represented by the two Measurement Requests
Vertical connection e, records the original state that established figure is G (v, e)；
Step 602, selected from all-multicast request group m1, m2 ... .mn minimum widith W's and include unappropriated Measurement Request
Measurement Request group m*；
It is unappropriated Measurement Request in initial time all-multicast request group m1, m2 ... .mn；
Step 603 successively selects unappropriated Measurement Request q in Measurement Request group m* to be handled, and one multicast of every processing
Request, photo-coupler z subtract 1 certainly；
The specific process is as follows: for the Measurement Request q that currently chooses, judge corresponding point whether in topological diagram G, if
Be then Measurement Request q is added in allocation result collection Y and directly carries out practical distribution, and by represent Measurement Request q point and
There is the point on side all to delete from topological diagram G with it；One Measurement Request q of every distribution just needs to consume a photo-coupler；
Step 604 judges whether photo-coupler z is greater than 0, if so, entering step 605；Otherwise, terminate algorithm and return the result
Collect Y；
Step 605 continues to select next Measurement Request q, and return step 603 does same processing, until can not be after in m*
The Measurement Request of continuous distribution；
Step 606 judges whether there are also remaining points in topological diagram G, and there are also unappropriated Measurement Request groups in system, if
It is to enter step 607；Otherwise, 608 are entered step；
Step 607, selection has the Measurement Request group m* of minimum widith, return step 603 from unappropriated group；Otherwise, enter
Step 608；
There is no all-multicast request group in remaining point or system to be all assigned in step 608, topological diagram G, then records
Remaining optical transceiver quantity k on each TOR subtracts 1 certainly；
Step 609 judges whether k!=0, if so, topological diagram G (v, e) is reduced to original state, delete recorded in Y
The representative point of all-multicast request, retain the point for having company side with it, return step 602；Otherwise, k==0 terminates algorithm,
Return to allocation result collection Y；
Step 7: Measurement Request distribution module and topology adjustment module are by dedicated control channel, control optical switching system is established
OCS connection, while corresponding ToR and server being notified to start the Measurement Request in transmission result set Y；
Step 8: the server of data center will inform that remaining light resource fills module after certain Measurement Request is transmitted, it should
Module is attempted to be calculated according to last width as a result, preferentially selecting available multicast to ask from the lesser Measurement Request group of width
It asks.
2. the light Measurement Request dispatching method that a kind of Based on Distributed as described in claim 1 calculates application, which is characterized in that
In the step two, according to distributed computing apply in belong to multiple Measurement Requests of the same application and be divided into one group.
3. the light Measurement Request dispatching method that a kind of Based on Distributed as described in claim 1 calculates application, which is characterized in that
The step three, trigger condition is defined as: when there is the all-multicast request of a new distributed computing application all here,
Then reach trigger condition.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810253581.8A CN108616569B (en) | 2018-03-26 | 2018-03-26 | A kind of Based on Distributed calculates the light Measurement Request dispatching method of application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810253581.8A CN108616569B (en) | 2018-03-26 | 2018-03-26 | A kind of Based on Distributed calculates the light Measurement Request dispatching method of application |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108616569A CN108616569A (en) | 2018-10-02 |
| CN108616569B true CN108616569B (en) | 2019-07-12 |
Family
ID=63658891
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810253581.8A Active CN108616569B (en) | 2018-03-26 | 2018-03-26 | A kind of Based on Distributed calculates the light Measurement Request dispatching method of application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108616569B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109348430B (en) * | 2018-10-29 | 2020-05-12 | 电子科技大学 | Multicast scheduling method for multi-channel multi-content base station cell |
| CN110166372B (en) * | 2019-05-27 | 2022-04-19 | 中国科学技术大学 | Method for online scheduling of co-flows in optical circuit switch-based data centers |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8223767B2 (en) * | 2009-12-31 | 2012-07-17 | Telefonaktiebolaget L M Ericsson (Publ) | Driven multicast traffic distribution on link-aggregate-group |
| US8879604B2 (en) * | 2011-07-06 | 2014-11-04 | Cisco Technology, Inc. | Efficient rendezvous for distributed messages in frequency-hopping communication networks |
| US8705557B2 (en) * | 2011-07-07 | 2014-04-22 | Qualcomm Incorporated | Methods and apparatus for supporting multicast communications |
| CN105337832B (en) * | 2014-09-30 | 2018-11-13 | 电子科技大学 | The resource allocation methods of online multicast virtual network |
-
2018
- 2018-03-26 CN CN201810253581.8A patent/CN108616569B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108616569A (en) | 2018-10-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103338163B (en) | Support the software defined network controller of dynamic elasticity scheduling of resource | |
| CN108111931A (en) | The virtual resource section management method and device of a kind of power optical fiber access net | |
| CN104601680B (en) | A kind of method for managing resource and device | |
| CN108566659A (en) | A kind of online mapping method of 5G networks slice based on reliability | |
| CN103152393A (en) | Charging method and charging system for cloud computing | |
| CN106685835A (en) | Method for achieving high-speed distributed routing between computational nodes of data center | |
| CN104468390B (en) | The method of multi-controller load balancing based on distribution centralized architecture model in software defined network | |
| CN104579761A (en) | Automatic nosql cluster configuration system and method based on cloud computing | |
| CN108616569B (en) | A kind of Based on Distributed calculates the light Measurement Request dispatching method of application | |
| CN113011678A (en) | Virtual operation platform operation control method based on edge calculation | |
| CN115314355A (en) | Electric power communication network architecture system and method based on deterministic network | |
| CN110213175A (en) | A kind of intelligent managing and control system and management-control method towards knowledge definition network | |
| CN109286528A (en) | A kind of SDN network multi-controller dispositions method based on time delay | |
| CN115174404B (en) | Multi-device federal learning system based on SDN networking | |
| CN112153153B (en) | Coordinated distributed intra-network resource scheduling method and system and storage medium | |
| CN107147734A (en) | A kind of network traffics thread-level dynamic load balancing method and system based on two level forwarding | |
| CN105391651A (en) | Virtual optical network multilayer resource convergence method and system | |
| CN107018018A (en) | A kind of server delta online upgrading method and system based on SDN | |
| CN103326916B (en) | Intelligent substation automatically divides and optimizes the system and method for VLAN | |
| CN100558071C (en) | A kind of Link Resource Manager method and the transmission network and the network equipment | |
| CN112148381A (en) | Software definition-based edge computing priority unloading decision method and system | |
| CN117221251B (en) | Computing power sensing and routing method and system for software-defined computing power network | |
| CN106549782A (en) | The bandwidth scheduling method and device of association stream in a kind of data center | |
| CN106302249B (en) | A kind of Dynamic Bandwidth Allocation system and distribution method | |
| CN107465628A (en) | The control method and control device of a kind of software defined network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |