CN108241534A

CN108241534A - A kind of task processing, distribution, management, the method calculated and device

Info

Publication number: CN108241534A
Application number: CN201611223651.2A
Authority: CN
Inventors: 李文鹏
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2016-12-27
Filing date: 2016-12-27
Publication date: 2018-07-03

Abstract

This application discloses a kind of processing of task, distribution, management, the method and devices calculated.Wherein, the task processing method, including：Global host node distributes task for each grouping host node, and the mission bit stream of distribution is fed back to each grouping host node；The mission bit stream that each grouping host node is fed back according to global host node, subtask is distributed for the slave node in grouping；After node calculates subtask, the grouping host node feedback into same grouping is based on the result of calculation after the calculating of subtask；The result of calculation received is forwarded by each grouping host node to global host node, and sends the request for obtaining new task to global host node after each forwarding；After global host node receives the request for obtaining new task of grouping host node transmission, new task is distributed, and the new mission bit stream distributed is fed back to grouping host node for grouping host node.The cost to communicate between node is reduced using the above method, accelerates computational efficiency.

Description

A kind of task processing, distribution, management, the method calculated and device

Technical field

This application involves field of computer technology, and in particular to the side that a kind of method, a kind of task of task processing are distributed Method, a kind of method of task management and a kind of method of task computation；The application be related to simultaneously a kind of processing of task device, A kind of task assigned unit, a kind of device of task management and a kind of device of task computation.

Background technology

In the application scenarios of machine learning, especially in the modeling process of credit and financial field, it will usually selection solution The property released preferable linear model is modeled, the reasons such as stability to ensure explanatory and model, it will usually a large amount of Selection contributes highest preceding several features to carry out final modeling on model in feature, on the one hand can be promoted with reduced model On the other hand the feature that some can bring a negative impact from model can also be rejected, carried by the solution to model property released and stability The performance of rising mould type.

A kind of method of feature selecting is preceding to feature selecting, and forward direction feature selecting is only to select a variable every time, is seen It examines and is weighed using variable Marginal Contrinution (MC) index to the influence to model performance, program can traverse All variables select a variable of MC maximums, this is a wheel iteration, can be in the base for the variable chosen in next round iteration On plinth, the variable for traversing remaining variable and selecting MC maximums in the wheel iteration is removed, and so on, it is until selecting N number of variable Only.

At present, feature selection process is that feature and calculate node are grouped, and a group node handles one group of feature.Example Such as：Common A, B, C, D, E in user data, F6 feature, calculate node totally 9, number are respectively 1-9, then feature are divided into three A group, every group of two features, calculate node is also divided into three groups, and every group of 3 nodes e.g., are handled to these three nodal parallels of 1-3 The two features of A, B, each feature are done parallel when processing in tri- intra-nodes of 1-3.Meanwhile other node processings its Its feature.

It can be seen that the calculation amount of feature selection process is very big, often take turns in iteration being required for remaining all variables It is once trained, unit is realized or common Parallel Implementation is difficult the completion selection task under the premise of big data； Parallel computation is not done in feature rank, with the increase of calculate node scale, the cost to communicate between node increases, in addition, When being grouped calculating to feature and calculate node, when the computational efficiency of machine in cluster is inconsistent, slow point of calculating speed Group can form the performance short slab of entire cluster task, even if other packet transaction speed are quickly, still need after processing is completed The handling result of the most slow grouping of latency speed, entire calculating task are decided by the calculated performance of most slow calculating grouping, can be right Computing resource forms great waste.

Invention content

The application provide a kind of method of task processing, a kind of method of task distribution, a kind of method of task management with And a kind of method of task computation, to solve the above problem of the prior art.The application is related to a kind of task processing simultaneously Device, a kind of task assigned unit, a kind of device of task management and a kind of device of task computation.

A kind of task processing method that the application provides, including；

Global host node distributes task, and believe to the task of each grouping host node feedback distribution for each grouping host node Breath；

The mission bit stream that each grouping host node is fed back according to the global host node, son is distributed for the slave node in grouping Task；

Behind the subtask from node calculating, the grouping host node feedback into same grouping is counted based on the subtask Result of calculation after calculation；

The result of calculation received is forwarded, and to the global host node each by each grouping host node The request for obtaining new task is sent after forwarding to the global host node；

It is the grouping after the overall situation host node receives the request for obtaining new task that the grouping host node is sent Host node distributes new task, and the new mission bit stream of distribution is fed back to the grouping host node.

Optionally, the global host node distributes task for each grouping host node, and is grouped host node feedback to each The mission bit stream of distribution, including：

The overall situation host node obtains the mission bit stream of corresponding number according to the number of grouping host node；The task letter Breath, including：The signature identification of the task；

The mission bit stream that the global host node is obtained for each grouping host node distribution, and it is grouped main section to each Point feedback distributes to the mission bit stream of the grouping host node.

Optionally, the mission bit stream that each grouping host node is fed back according to the global host node, in grouping Subtask is distributed from node, including：

Each grouping host node obtains corresponding task according to the mission bit stream that the global host node is fed back；

According to the slave interstitial content in grouping, the task is divided into corresponding from the subtask of interstitial content；

The subtask is evenly distributed described from node.

Optionally, each grouping host node is turned the result of calculation received to the global host node Hair, including:

Each grouping host node, by the grouping received it is all from node based on the subtask calculate after calculating knot Fruit is integrated；

The result of calculation after integration is forwarded to the global host node.

Optionally, the mission bit stream fed back in each grouping host node according to the global host node, in grouping Slave node distribution subtask the step of after, including：

Start timer record temporal information；

The result of calculation received is turned to the global host node performing each grouping host node After the step of hair, stop the timer；

Using the temporal information of the timer record as the calculating time of the mission bit stream.

Optionally, after the step of stopping timer, including：

The calculating time of the mission bit stream that the grouping host node has been recorded to the global host node transmission.

Optionally, the calculating of the mission bit stream recorded in the grouping host node to the global host node transmission After the step of time, including：

The overall situation host node receives the calculating time of the mission bit stream recorded that the grouping host node is sent；

The time that calculates according to having received obtains each total evaluation time for being grouped host node calculating task；

The average calculation times of calculating task are obtained for each grouping host node；

Obtain the shortest grouping host node of average calculation times；

Judge the average calculation times of other grouping host nodes whether more than the grouping host node calculating task obtained Amount to evaluation time；

If so, remove the grouping host node.

Optionally, data are carried out based on message passing interface between the global host node and the grouping host node to lead to Letter.

Optionally, the grouping host node is the slave node of the numerical value minimum of the serial number from node.

Optionally, the task, including：Modeler model feature.

Optionally, the result of calculation received is carried out to the global host node in each grouping host node After the step of forwarding, including：

The overall situation host node receives the result of calculation of each grouping host node forwarding；

The result of calculation more than default value is obtained, using the result of calculation as calculating parameter；

The calculating parameter obtained to each grouping host node feedback.

Optionally, described after the step of calculating parameter of each grouping host node feedback acquisition, including：

Each grouping host node receives the calculating parameter that the global host node is fed back based on the result of calculation；

Slave node into grouping forwards the calculating parameter.

Optionally, it is described to calculate the subtask from node, including：

If described have calculating parameter from node, the subtask is calculated based on the calculating parameter.

Correspondingly, the application also provides a kind of Task Processing Unit, including；

Mission bit stream allocation unit distributes task, and to each grouping for global host node for each grouping host node The mission bit stream of host node feedback distribution；

Subtask allocation unit is grouped mission bit stream of the host node according to the global host node feedback for each, is Slave node distribution subtask in grouping；

Result of calculation feedback unit, for described after node calculates the subtask, the main section of grouping into same grouping Point feedback is based on the result of calculation after the calculating of the subtask；

Task requests transmitting element, it is for each grouping host node that the result of calculation received is main to the overall situation Node is forwarded, and sends the request for obtaining new task to the global host node after each forwarding；

Allocation unit receives the request for obtaining new task of the grouping host node transmission for the global host node Afterwards, it distributes task for the grouping host node and the new mission bit stream of distribution is fed back to the grouping host node.

In addition, the application also provides a kind of method of task distribution, including：

The request of acquisition task is sent to global host node；

Receive the mission bit stream of the global host node feedback；

It is the slave node distribution subtask in grouping according to the mission bit stream.

Optionally, it is described according to the mission bit stream be grouping in slave node distribute subtask, including：

Corresponding task is obtained according to the mission bit stream；

The subtask is evenly distributed described from node.

Optionally, it is described be packet after the step of slave node in grouping distributes subtask according to the mission bit stream It includes：

Receive in grouping it is described from node calculated based on the subtask after the result of calculation fed back；

The result of calculation is integrated；

Optionally, the result of calculation by after integration to the global host node be forwarded the step of after, Including：

Trigger and perform it is described to global host node send acquisition task request the step of.

Start timer record temporal information；

After described the step of being forwarded the result of calculation received to the global host node is performed, stop The only timer；

Using the temporal information of the timer record as the calculating time of the mission bit stream；

The calculating time of the mission bit stream recorded to the global host node transmission.

Optionally, the result of calculation by after integration is forwarded to the global host node the step of it Afterwards, including：

Receive the calculating parameter that the global host node is fed back based on the result of calculation；

Described in into grouping the calculating parameter is forwarded from node.

Optionally, the request that acquisition task is sent to global host node, including：

The request of acquisition task is sent to global host node based on message passing interface.

Optionally, the task, including：Modeler model feature.

Correspondingly, the application also provides a kind of task assigned unit, including：

Task requests transmitting element, for sending the request of acquisition task to global host node；

Mission bit stream receiving unit, for receiving the mission bit stream of the global host node feedback；

Task allocation unit, for being that the slave node in grouping distributes subtask according to the mission bit stream.

In addition, the application also provides a kind of method of task management, including：

Receive the request of acquisition task that grouping host node is sent；

Task and the mission bit stream distributed to the grouping host node feedback are distributed for the grouping host node.

Optionally, in the distributing task for the grouping host node and distribution is fed back to the grouping host node of the task After the step of information, including：

Receive the calculating time of the mission bit stream recorded that the grouping host node is sent；

Obtain the shortest grouping host node of average calculation times；

If so, remove the grouping host node.

Optionally, the request of acquisition task for receiving grouping host node and sending, including：

The request of the acquisition task of grouping host node transmission is received based on message passing interface.

Optionally, the task, including：Modeler model feature.

Receive the result of calculation of each grouping host node forwarding；

The calculating parameter obtained to each grouping host node feedback.

Correspondingly, the application also provides a kind of device of task management, including：

Task requests receiving unit, for receiving the request of acquisition task that grouping host node is sent；

Mission bit stream allocation unit, for distributing task for the grouping host node and dividing to the grouping host node feedback The mission bit stream matched.

In addition, the application also provides a kind of method of task computation, including：

Receive the subtask of grouping host node distribution；

The subtask is calculated based on calculating parameter；

To the grouping host node feedback based on the result of calculation after the calculating of the subtask.

Optionally, in described the step of feeding back the result of calculation after being calculated based on the subtask to the grouping host node Later, including：

Receive the calculating parameter of the grouping host node forwarding.

Correspondingly, the application also provides a kind of device of task computation, including：

Subtask receiving unit, for receiving the subtask of grouping host node distribution；

Subtask computing unit calculates the subtask for being based on calculating parameter；

Result of calculation feedback unit, for feeding back the calculating knot after being calculated based on the subtask to the grouping host node Fruit.

Compared with prior art, the application has the following advantages：

A kind of task processing method, the device of the application offer, the task processing method is every by global host node One grouping host node distribution task, and the mission bit stream distributed is fed back to each grouping host node；It is each grouping host node according to The mission bit stream of the overall situation host node feedback, subtask is distributed for the slave node in grouping；It is described to calculate the son from node After task, the grouping host node feedback into same grouping is based on the result of calculation after the calculating of the subtask；Each main section of grouping The result of calculation received is forwarded, and to the global host node to the main section of the overall situation after each forwarding by point Point sends the request for obtaining new task；The overall situation host node receives the new task of the acquisition for being grouped host node transmission After request, new task is distributed for the grouping host node, and the new mission bit stream of distribution is fed back to the grouping host node. The technical solution is handled by making task in asynchronous packet node, no longer each packet node is distributed unitedly in advance solid Determine the feature of number, but be each packet node assigned characteristics, and setting divides in host node is grouped according to calculating speed Group host node, the grouping host node allocate the resource of feature task, which starts an event loop, use The request of new task is obtained in rear sent to global host node for having handled current task in packet node, accelerates calculating effect Rate.

Description of the drawings

In order to illustrate the technical solutions in the embodiments of the present application or in the prior art more clearly, to embodiment or it will show below There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this Some embodiments described in application, for those of ordinary skill in the art, can also be obtained according to these attached drawings other Attached drawing.

Fig. 1 shows the schematic diagram of task processing frame provided according to an embodiment of the present application；

Fig. 2 shows the flow charts of the method for task processing provided according to an embodiment of the present application；

Fig. 3 shows the schematic diagram of the device of task processing provided according to an embodiment of the present application；

Fig. 4 shows the flow chart of the method for task distribution provided according to an embodiment of the present application；

Fig. 5 shows the schematic diagram of task assigned unit provided according to an embodiment of the present application；

Fig. 6 shows the flow chart of the method for the task management provided according to an embodiment of the present application；

Fig. 7 shows the schematic diagram of the device of the task management provided according to an embodiment of the present application；

Fig. 8 shows the flow chart of the method for the task computation provided according to an embodiment of the present application；

Fig. 9 shows the schematic diagram of the device of the task computation provided according to an embodiment of the present application.

Specific embodiment

It is below in conjunction with the accompanying drawings and specific real in order to be more clearly understood that the above-mentioned purpose of the application, feature and advantage Mode is applied the application is further described in detail.It should be noted that in the absence of conflict, the implementation of the application Feature in example and embodiment can be combined with each other.

Many details are elaborated in the following description in order to fully understand the application.But the application can Much to implement different from other manner described here, those skilled in the art can be in the feelings without prejudice to the application intension Similar popularization is done under condition, therefore, the application is not limited by following public specific implementation.

Embodiments herein provides a kind of method of task processing, a kind of method of task distribution, a kind of task pipe The method of reason and a kind of method of task computation, embodiments herein provide a kind of device of task processing, one simultaneously Kind task assigned unit, a kind of device of task management and a kind of device of task computation.

At present, feature selection process is that feature and calculate node are grouped, and a group node handles one group of feature.Example Such as：Common A, B, C, D, E in user data, F6 feature, calculate node totally 9, number are respectively 1-9, then feature are divided into three A group, every group of two features, calculate node is also divided into three groups, and every group of 3 nodes e.g., are handled to these three nodal parallels of 1-3 The two features of A, B, each feature are done parallel when processing in tri- intra-nodes of 1-3.Meanwhile other node processings its Its feature.It can be seen that the calculation amount of feature selection process is very big, often take turns in iteration be required for remaining all variables into The primary training of row, realizes unit or common Parallel Implementation is difficult the completion selection task under the premise of big data；And Parallel computation is not done in feature rank, with the increase of calculate node scale, the cost to communicate between node increases, in addition, right When feature and calculate node are grouped calculating, when the computational efficiency of machine in cluster is inconsistent, the slow grouping of calculating speed The performance short slab of entire cluster task can be formed, even if other packet transaction speed are quickly, still need after processing is completed etc. Treat the handling result of the most slow grouping of speed, entire calculating task is decided by the calculated performance of most slow calculating grouping, can be to meter It calculates resource and forms great waste.For this problem, the technical solution of the application is by making task in asynchronous packet node Middle processing no longer in advance to the feature of each packet node unified distribution fixed number, but is each point according to calculating speed Group node assigned characteristics, and the setting grouping host node in host node is grouped, the grouping host node is by the resource of feature task It is allocated, which starts an event loop, and the rear to the overall situation of current task is being handled for packet node Host node sends the request for obtaining new task, accelerates computational efficiency..

Although the technical solution of the application is that the modeling process in the application scenarios for machine learning proposes, still, Its application field is not limited to above-mentioned scene, other can be by setting global host node, grouping host node and allocating from section Point carries out the other application scene of parallel computation, the method that the application can also be used to provide, and obtains corresponding advantageous effect.

Before the method handled the task is described in detail, first to times of the method for application task processing Business processing frame illustrates.

It please refers to Fig.1, it illustrates the schematic diagrames of task processing frame provided according to an embodiment of the present application.

It is communicatively coupled between global host node and the packet node of certain amount, wherein, four grouping sections in figure Point is only schematical, and the calculation amount that the number of the packet node in frame can carry out calculating task as needed is increased It adds deduct few.

In this frame, the packet node is by a grouping host node and a certain number of is formed from node.Its In, three in figure are only schematical from node, and the slave interstitial content in each packet node can differ, described from section It is determining according to the level of hardware (computing capability) of the packet node to count out.It is grouped host node and each between node It can communicate, be by packet node when each packet node is communicated between the global host node Grouping host node and the global host node between communicate.

In this frame, also with task pool.Store what task processing frame was calculated in the task pool Task.The task pool can access acquisition task by the grouping host node in the global host node and packet node.

It should be noted that when the task pool is accessed by the global host node, only obtained from the task pool Mission bit stream rather than the specific task of acquisition；It is from the task pool when task pool is accessed by the grouping host node It is middle to obtain specific task.

It should be understood that the overall situation host node is only the grouping host node distribution when accessing the task pool Task, thus only need to get the task mission bit stream (such as：The id of task), it is described grouping host node according to The mission bit stream of the overall situation host node distribution goes to access the task pool, therefrom obtains and is assigned to the specific of the packet node Task.

The various pieces in frame are handled to task below to be described in detail.

The task handles frame, including：Global host node, grouping host node and from node；

The overall situation host node for distributing task for each grouping host node, and is grouped host node feedback point to each The mission bit stream matched；And the request for obtaining new task that respond packet host node is sent, for corresponding grouping host node point With new task, and the new mission bit stream distributed is fed back to the grouping host node；

The grouping host node, for the mission bit stream fed back according to the global host node, for the slave node in grouping Distribute subtask；And after each task computation, result of calculation is forwarded to the global host node, and to The overall situation host node sends the request for obtaining new task；

It is described from node, for calculating the subtask of the grouping host node distribution, and to same grouping after calculating finishes Interior grouping host node feedback is based on the result of calculation after the calculating of the subtask.

Wherein, the grouping host node, further includes：Timer；

The timer, for starting timing when the grouping host node gets task, until performing described each The result of calculation received after the global host node is forwarded, is stopped timing by grouping host node；By described in Calculating time of the temporal information of timer record as the task.

After the timer is stopped, the grouping host node to the global host node transmission recorded described in appoint The calculating time of business.

Correspondingly, the overall situation host node, is additionally operable to the time according to each grouping host node calculating task, removal calculates Performance is less than the packet node of preset threshold value.

The grouping host node in the task handles frame, be serial number in the packet node numerical value it is minimum from Node.

It should be noted that the grouping host node is chosen from calculate node (from node), after which starts Without calculating task, it is responsible for allocating the resource of task.The grouping host node is that rank is 0 in packet node Node, the packet node correspond to a MPI Communicator, i.e.,：Each node is one corresponding in the packet node Rank serial numbers, such as：There are 10 nodes in the packet node, rank is respectively 0-9, then makees the node that rank is 0 To be grouped host node, the node of rank1-9 is used as from node.

Embodiments herein provides a kind of method of task processing.The embodiment of the method for the task processing is as follows：

It please refers to Fig.2, it illustrates the flow charts of the method for task processing provided according to an embodiment of the present application.

The method of the task processing, including：

Step S101, global host node distribute task, and be grouped host node feedback point to each for each grouping host node The mission bit stream matched.

In the present embodiment, this step is to handle the global host node in frame by the task to be implemented, institute It states global host node and distributes task for each grouping host node, and to the mission bit stream of each grouping host node feedback distribution, it can To realize in the following way：The overall situation host node accesses the task pool and obtains mission bit stream, and the task of acquisition is believed Breath evenly distributes to each grouping host node distribution task, and distributes to the grouping host node to each grouping host node feedback Mission bit stream.

It should be noted that in order to evenly distribute the mission bit stream of acquisition to each grouping host node, need in institute State the number that the packet node in distributed type assemblies is determined before global host node accesses the task pool.Section is grouped in cluster The number of point can be determined by log-on message of the packet node in distributed type assemblies.It is determining to divide in distributed type assemblies After the number of group node, perform step S101 overall situations host node and distribute task, and lead to each grouping for each grouping host node During the mission bit stream of node feeding back distribution, including：

It should be understood that after the global host node determines the number of packet node, when accessing the task pool just The mission bit stream of the number of corresponding packet node can be obtained, it is possible to which the task is evenly distributed in distributed type assemblies Packet node on.

It, can be with it should be noted that when obtaining the mission bit stream of the number of corresponding packet node in the global host node The mission bit stream of the number equal to the packet node is obtained, the task letter of the number multiple of the packet node can also be obtained Breath.

Such as：There are 3 packet nodes in cluster, then the global host node obtains 3 mission bit streams from task pool, On 3 packet nodes that 3 mission bit streams of acquisition are evenly distributed in cluster, each packet node is made to get 1 task Information；Or the global host node obtains 3 mission bit streams several again (such as 6) from task pool, and 6 of acquisition are appointed Business information is evenly distributed on 3 packet nodes in cluster, and each packet node is made to get 2 mission bit streams.

It should be understood that the overall situation host node is only the grouping host node distribution when accessing the task pool Task, so only needing to get the mission bit stream of the task.Such as：Each task passes through No. id progress area in task pool Timesharing, it is exactly No. id to get the signature identification in the mission bit stream of the task in this step.

In the specific implementation, the global host node is to be based on to the mission bit stream of each grouping host node feedback distribution The mission bit stream that message passing interface is distributed to each grouping host node feedback.The message passing interface MPI is one across language The communications protocol of speech, for writing parallel computer.Support point-to-point and broadcast.MPI is that an information transmission application program connects Mouthful, including agreement and and semantic description, indicate how it plays its characteristic in various implementations.MPI can ensure the one of message Cause property.

Step S103, the mission bit stream that each grouping host node is fed back according to the global host node, in grouping from Node distributes subtask.

In the present embodiment, this step is the grouping master in each packet node handled by the task in frame What node was implemented, the mission bit stream that each grouping host node is fed back according to the global host node, in grouping Subtask is distributed from node, following manner realization may be used：Each grouping host node is to receive the global host node anti- After the mission bit stream of feedback, the task of the corresponding mission bit stream is obtained from the task pool, and the task cutting is appointed for son Business, the subtask being syncopated as is evenly distributed to the slave node in the packet node.

It should be noted that in order to evenly distribute the subtask being syncopated as to the slave node in the packet node, need The number of the slave node in grouping is determined in cutting.After the number from node is determined, each point of step S103 is performed The mission bit stream that group host node is fed back according to the global host node, when distributing subtask for the slave node in grouping, including：

The subtask is evenly distributed described from node.

It should be understood that after the number for determining the slave node in grouping, the grouping host node in the grouping According to the number from node, by the subtask that the task cutting is the corresponding number from node.

It should be noted that by the task cutting be subtask of the correspondence from the number of node in the grouping host node When, it can be the subtask equal to the number from node by the task cutting, can also be institute by the task cutting It states from the subtask of the number multiple of node.

Such as：There are 2 from node in the grouping, 2 subtasks of the grouping host node individual task cutting (respectively account for The half of predecessor's business), 2 subtasks of cutting are evenly distributed to 2 in being grouped from node, are made each from node Get 1 subtask.

It should be noted that if obtaining 2 tasks from task pool in the grouping host node, and have in the grouping 2 tasks from node, in order to which task is enable uniformly to distribute, are then carried out cutting respectively and (amount to 6 for 3 subtasks by 3 Subtask), 6 subtasks are evenly distributed to 2 in grouping from node, make each to get 2 subtasks from node. When being split to task, if can not uniformly split, the task amount for making the subtask split out is essentially identical.

It should be understood that by task evenly distribute it is described from node after, each distributed from node it is specific son appoint The content of business differs, then described that subtask can be calculated parallel from node.

Corresponding appoint, is obtained according to the mission bit stream that the global host node is fed back performing each grouping host node During business, be each grouping host node fed back according to the global host node mission bit stream (signature identification of task, i.e.,： No. id) it goes to access the task pool, it is obtained and the corresponding specific task of the mission bit stream from the task pool.

Since the level of hardware (computing capability) of packet node is different, in order to determine the calculating of each packet node speed It spends (calculated performance), the technical solution of the present embodiment provides a kind of preferred embodiment, under preferred embodiment, needs to each Each task that packet node calculates carries out timing, is performing step S103 in each grouping host node according to described complete The mission bit stream of office's host node feedback, is distributed for the slave node in grouping after subtask, it is also necessary to perform following steps：

Start timer record temporal information；

Perform each grouping host nodes of step S107 by the result of calculation received to the global host node into After row forwarding, stop the timer；

It it should be noted that can be using the time segment information of the timer record as during the calculating of the mission bit stream Between, in addition to this it is possible to carry out timing using second method, step S103 is being performed in each grouping host node root According to the mission bit stream of the global host node feedback, after distributing subtask for the slave node in grouping, current system is recorded Temporal information, perform each grouping host nodes of step S107 by the result of calculation received to the global host node into Current system time information is recorded again after row forwarding, and the difference of a pair of of temporal information of record is believed as the task The calculating time of breath.

Timing is carried out using the second way, due to only having carried out timing two time points, compared to using timer The recording time information of long period, makes the calculating pressure smaller of packet node, and the resource of occupancy is less.

Since the grouping host node can only record calculating time of being calculated in the grouping for task, so recording times After the calculating time of business, the calculating time by all tasks in cluster is needed to integrate, so described in the stopping After the step of timer, the grouping host node needs the mission bit stream recorded to the global host node transmission The time is calculated, convenient for the global host node to the statistics of calculated performance, to eliminate the influence of long-tail.

Step S105, described after node calculates the subtask, the grouping host node feedback into same grouping is based on institute State the result of calculation after subtask calculates.

In the present embodiment, this step is the slave node in each packet node handled by the task in frame Implemented, described after node calculates the subtask, the grouping host node feedback into same grouping is appointed based on the son Following manner realization may be used in result of calculation after business calculating：It is each to be calculated from node by the grouping host node in same grouping The subtask of distribution, the grouping host node feedback after the subtask has been calculated into same grouping are calculated based on the subtask Result of calculation afterwards.

Step S107, each grouping host node are turned the result of calculation received to the global host node Hair, and the request for obtaining new task is sent to the global host node after each forwarding.

In the present embodiment, this step is the grouping master in each packet node handled by the task in frame What node was implemented, each grouping host node is turned the result of calculation received to the global host node Hair, and the request for obtaining new task is sent to the global host node after each forwarding, following manner realization may be used： Each grouping host node by the same grouping received from node based on the result of calculation fed back behind subtask is calculated, based on message Passing interface is forwarded to the global host node, and is based on message passing interface to the main section of the overall situation after each forwarding Point sends the request for obtaining new task.The main section of grouping for sending the request is carried in the wherein described request for obtaining new task The feature indication of point.

Due to each after node can feed back respective subtask calculating to the grouping host node in packet node Result of calculation, thus to global host node feedback task result of calculation when, need the son that receives of grouping host node The result of calculation of task is integrated, then each grouping host node is main to the overall situation by the result of calculation received Node is forwarded, and is included the following steps：

It should be understood that it is described grouping host node according to the cutting relationship to task, by the corresponding result of calculation in subtask It is integrated, forms the result of calculation of corresponding complete task.

Such as：If obtaining 2 tasks A and B from task pool in the grouping host node, and there are 3 in the grouping From node, in order to which task is enable uniformly to distribute, then 2 tasks A and B are subjected to cutting as 3 subtasks A1, A2 respectively, A3 and B1, B2, B3, then according to cutting relationship by subtask A1, the corresponding result of calculation of A2, A3 is integrated into the calculating task of A, By B1, the corresponding result of calculation of B2, B3 is integrated into the calculating task of B.

It should be understood that the result of calculation received is carried out to the global host node in the grouping host node After forwarding, just illustrate that the packet node completes the calculating process of the task of the global host node distribution, i.e.,：The packet node Interior not have calculating task, then the grouping host node can obtain new task after each forwarding to the global host node transmission Request, obtain new task to the global host node for active and calculated, and without waiting for point whole in cluster Group node is completed to be allocated by the global host node again when secondary calculating task, improves the computational efficiency in cluster.

Step S109, after the overall situation host node receives the request for obtaining new task that the grouping host node is sent, New task is distributed for the grouping host node, and the new mission bit stream of distribution is fed back to the grouping host node.

In the present embodiment, this step is to handle the global host node in frame by the task to be implemented, institute It states global host node and receive the request for obtaining new task that the grouping host node is sent, is received based on message passing interface The request for obtaining new task that the grouping host node is sent.

It should be noted that since the number of packet node having in cluster is more than one, so receiving in this step The request of the new task of acquisition that sends of the grouping host node, may be from several packet nodes, in order to ensure to disappear The consistency of breath, after the request for obtaining new task is received by message passing interface MPI, based on the communications framework, The overall situation host node can be sequentially received the request of the new task of the acquisition, and lead to according to the transmission sequence of each packet node Cross each packet node of the feature indication for being grouped host node differentiation of the transmission of the carrying in the request request.

In the present embodiment, it is described to distribute new task, and feed back to the grouping host node for the grouping host node Following manner realization may be used in the new mission bit stream of distribution：It is single that the overall situation host node accesses the task pool acquisition Mission bit stream, and the mission bit stream of acquisition is distributed into the grouping host node, and distributed to the grouping host node feedback New mission bit stream.

It should be understood that the asking in the new task of the acquisition for receiving each packet node transmission due to the global host node After asking, the corresponding request of meeting successively, so when distributing task for packet node, different from step S101, but it is primary A mission bit stream is obtained from task pool or obtains the mission bit stream of predetermined number.

In the present embodiment, it is based on message during the new mission bit stream to the grouping host node feedback distribution The new mission bit stream that passing interface is distributed to the grouping host node feedback.

Since each grouping host node is performing step S107 by the result of calculation received to the main section of the overall situation After point is forwarded, the timer can be stopped, and the mission bit stream recorded is sent to the global host node Calculate the time, then the layout host node can based on the mission bit stream calculate the time to the calculated performance of each packet node into Row statistics, since the machine software and hardware configuration that task is handled on different grouping node is different, also with external interference factor, Therefore the processing time for leading to each subtask is different, and the processing time for operating in the subtask on the node of poor-performing can be bright The aobvious processing time for being longer than other machines processing subtask becomes the bottleneck of entire task run, caused by eliminating long-tail It influences, specifically comprises the following steps：

Obtain the shortest grouping host node of average calculation times；

If so, remove the grouping host node.

It should be noted that the overall situation host node is in the task recorded the letter for receiving each grouping host node transmission After the calculating time of breath, the calculating time of the mission bit stream can be recorded under the main section of grouping for calculating the mission bit stream, So the total evaluation time that each packet node calculates the calculating task that the packet node has recorded can be directed to.

In the present embodiment, when every wheel iteration starts, packet node can handle each appoint by timer record oneself The business spent calculating time, and handled total times in the iteration is got according to the number of calculating time having recorded Business number, then the average calculation times that calculating task is obtained for each grouping host node, may be used following manner reality It is existing：Each packet node is calculated into the total evaluation time for the calculating task that the packet node has recorded divided by is recorded each The quantity for the mission bit stream that packet node calculates obtains the average calculation times of each grouping host node calculating task.

It should be noted that whether the average calculation times for judging other grouping host nodes are more than the grouping obtained The total evaluation time of host node calculating task refers to：Other grouping host nodes and the shortest grouping of average calculation times are led successively Node is compared, if the average calculation times (calculating the calculating time that a task is spent) of other grouping host nodes are more than The total evaluation time of the grouping host node calculating task of acquisition, then it represents that the main section of fastest grouping is calculated in current iteration After all tasks for distributing to the grouping host node have been calculated in point, average calculation times are more than the grouping host node obtained and calculate The still untreated complete single task of packet node of the total evaluation time of task, it is determined that the grouping host node is long-tail point Group node, and remove the grouping host node.

It should be noted that the long-tail refers to：In systems, task is divided into the subtask of limited quantity, and not With being handled on node, the machine software and hardware configuration for handling task is different, also with external interference factor, therefore causes each The processing time of subtask is different, and the processing time of the subtask operated on the node of poor-performing can be considerably longer than other machines Device handles the processing time of subtask, becomes the bottleneck of entire task run, considerably increases the run time of entire task, this It is exactly long-tail delay phenomenon.

As a preferred embodiment, in task processing method provided by the embodiments of the present application, the task, including： Modeler model feature.Such as：It applies in credit and financial field, task includes：Card number, deposit number, by stages amount, situation Deng.

It should be understood that in the application scenarios of machine learning, modeling is finally modeled by model characteristics, institute Task is distributed by each grouping host node of step S101 overall situations host node, and the task of distribution is fed back to each grouping host node Information is exactly obtained in modeling process, weighs the variable of model performance, the aspect of model can be carried out by feature significance Quantization.

Wherein, to the result of calculation that task computation goes out it is obtained numerical parameter after task feature based conspicuousness calculates. The feature significance includes Marginal Contrinution (MC).

When the task is modeler model feature, it will usually explanatory preferable linear model is selected to be modeled, The reasons such as the stability to ensure explanatory and model can select to contribute model in a large amount of modeler model feature highest Preceding several features carry out final modeling, on the one hand can be with reduced model, the explanatory and stability of lift scheme, the opposing party Face can also reject the feature that some can bring a negative impact from model, the performance of lift scheme, so in the grouping Host node performs step S107 by the result of calculation received after the global host node is forwarded, including：

The calculating parameter obtained to each grouping host node feedback.

It should be noted that task of the result of calculation more than default value is obtained performing, by the calculating knot of the task When fruit is as task parameters, the preset threshold value can be a specific parameter value, which is according to this field The relatively reasonable numerical value or the threshold value that technical staff is rule of thumb configured be in the result of calculation received most Big numerical value, i.e.,：Numerical value max calculation result is obtained from the result of calculation received using the result of calculation of the task as meter Calculate parameter.

After the task in the Modelling feature pond is all had been calculated, the packet node can receive described global main The task parameters that node is fed back based on epicycle feature selecting, the packet node can be by the features based on the calculating parameter It is stored, to carry out next round iteration, in the meter that the global host node is obtained to each grouping host node feedback After the step of calculating parameter, specifically comprise the following steps：

Slave node into grouping forwards the calculating parameter.

Correspondingly, at the execution step S105 calculating subtasks from node, if described have meter from node Parameter is calculated, then the subtask is calculated based on the calculating parameter.

In the above-described embodiment, a kind of task processing method is provided, it is corresponding with above-mentioned task processing method, this Application additionally provides a kind of Task Processing Unit.Since the embodiment of device is substantially similar to the embodiment of method, so description Must be fairly simple, the relevent part can refer to the partial explaination of embodiments of method.Device embodiment described below is only to show Meaning property.The Task Processing Unit embodiment is as follows：

It please refers to Fig.3, it illustrates the schematic diagrames of the device of task processing provided according to an embodiment of the present application.

The device of the task processing, including：Mission bit stream allocation unit 301, calculates knot at subtask allocation unit 303 Fruit feedback unit 305, task requests transmitting element 307 and allocation unit 309；

The mission bit stream allocation unit 301 distributes task, and to every for global host node for each grouping host node The mission bit stream of one grouping host node feedback distribution；

The subtask allocation unit 303, being fed back for each grouping host node according to the global host node for task Information distributes subtask for the slave node in grouping；

The result of calculation feedback unit 305, for described after node calculates the subtask, point into same grouping Group host node feedback is based on the result of calculation after the calculating of the subtask；

The task requests transmitting element 307, for each grouping host node by the result of calculation received to institute It states global host node to be forwarded, and sends the request for obtaining new task to the global host node after each forwarding；

The allocation unit 309 obtains new times for what the global host node reception grouping host node was sent After the request of business, distribute task for the grouping host node and the new mission bit stream of distribution is fed back to the grouping host node.

Optionally, the mission bit stream allocation unit 301, including：Mission bit stream obtains subelement and mission bit stream is anti- Present subelement；

The mission bit stream obtains subelement, is obtained and corresponded to according to the number of grouping host node for the global host node The mission bit stream of number；The mission bit stream, including：The signature identification of the task；

The mission bit stream feeds back subelement, the institute obtained for the global host node for each grouping host node distribution Mission bit stream is stated, and the mission bit stream of the grouping host node is distributed to each grouping host node feedback.

Optionally, the subtask allocation unit 303, including：The task obtains subelement, task divide subelement with And subtask distribution subelement；

The task obtains subelement, the mission bit stream fed back for each grouping host node according to the global host node Obtain corresponding task；

The task divides subelement, for according to the slave interstitial content in grouping, by the task be divided into it is corresponding from The subtask of interstitial content；

Subelement is distributed in the subtask, for evenly distributing the subtask described from node.

Optionally, the task requests transmitting element 307, including：Result of calculation integrates subelement and forwarding subelement；

The result of calculation integrates subelement, will be all from node in the grouping received for each grouping host node Result of calculation after being calculated based on the subtask is integrated；

The forwarding subelement, for the result of calculation after integrating to be forwarded to the global host node.

Optionally, the Task Processing Unit, further includes：Timer initiation unit and timer stop unit；

The timer initiation unit, for being appointed in each grouping host node according to what the global host node was fed back Business information after distributing subtask for the slave node in grouping, starts timer record temporal information；

The timer stop unit, for performing each result of calculation that is grouped host node and will receive After the global host node is forwarded, stop the timer；

Optionally, the Task Processing Unit, further includes：Calculate time transmitting element；

The calculating time transmitting element, for after the stopping timer, the host node that is grouped to be to institute State the calculating time of the mission bit stream that global host node transmission has recorded.

Optionally, the Task Processing Unit, further includes：Calculate time receiving unit, total time computing unit, mean time Between computing unit, shortest time acquiring unit, time judging unit and removal node unit；

The calculating time receiving unit, for what is recorded in the grouping host node to the global host node transmission After the calculating time of the mission bit stream, it is described that the overall situation host node receives having recorded for the grouping host node transmission The calculating time of mission bit stream；

The total time computing unit, for obtaining each grouping host node calculating task according to the calculating time received Total evaluation time；

The average time computing unit, when obtaining the average computation of calculating task for being directed to each grouping host node Between；

The shortest time acquiring unit, for obtaining the shortest grouping host node of average calculation times；

The time judging unit, for judging whether the average calculation times of other grouping host nodes are more than what is obtained It is grouped the total evaluation time of host node calculating task；

The removal node unit, for receiving the judging result of the time judging unit, if so, described point of removal Group host node.

Optionally, the task, including：Modeler model feature.

Optionally, the Task Processing Unit, further includes：Result of calculation receiving unit, calculating parameter acquiring unit and Calculating parameter transmitting element；

The result of calculation receiving unit, for it is described it is each grouping host node by the result of calculation received to After the overall situation host node is forwarded, the overall situation host node receives the calculating knot of each grouping host node forwarding Fruit；

For obtaining the result of calculation more than default value, described calculate is tied for the calculating parameter acquiring unit Fruit is as calculating parameter；

The calculating parameter transmitting element, for the calculating parameter obtained to each grouping host node feedback.

Optionally, the Task Processing Unit, further includes：Calculating parameter receiving unit and calculating parameter retransmission unit；

Calculating parameter receiving unit, for it is described to it is each grouping host node feedback obtain the calculating parameter it Afterwards, each grouping host node receives the calculating parameter that the global host node is fed back based on the result of calculation；

Calculating parameter retransmission unit forwards the calculating parameter for the slave node into grouping.

Optionally, the result of calculation feedback unit 305, if there is calculating parameter from node specifically for described, The subtask is calculated based on the calculating parameter.

In the above-described embodiment, a kind of method of task processing and a kind of device of task processing are provided, in addition, Present invention also provides a kind of methods of task distribution；The embodiment of the method for the task distribution is as follows：

It please refers to Fig.4, it illustrates the flow charts of the method for task distribution provided according to an embodiment of the present application.

The method of the task distribution, including：

Step S401 sends the request of acquisition task to global host node.

In the present embodiment, the method for the task distribution is each the grouping section handled by the task in frame What the grouping host node in point was implemented, the request that acquisition task is sent to global host node may be used such as lower section Formula is realized：The request of acquisition task is sent to the global host node based on message passing interface.It is wherein described to obtain new appoint The feature indication for the grouping host node for sending the request is carried in the request of business.

Step S403 receives the mission bit stream of the global host node feedback.

In the present embodiment, the mission bit stream for receiving the global host node feedback, may be used following manner reality It is existing：The mission bit stream of the global host node feedback is received based on message passing interface.

It should be noted that the mission bit stream refers to：The signature identification of task, i.e.,：No. id.

Step S405 is the slave node distribution subtask in grouping according to the mission bit stream.

In the present embodiment, it is described according to the mission bit stream be grouping in slave node distribute subtask, may be used Following manner is realized：Grouping host node obtains after the mission bit stream for receiving the global host node feedback from the task pool Take the task of the corresponding mission bit stream, and be subtask by the task cutting, by the subtask being syncopated as evenly distribute to Slave node in the packet node.

It should be noted that in order to evenly distribute the subtask being syncopated as to the slave node in the packet node, need The number of the slave node in grouping is determined in cutting.After the number from node is determined, step S405 is performed according to institute State mission bit stream be grouping in slave node distribution subtask when, including：

Corresponding task is obtained according to the mission bit stream；

The subtask is evenly distributed described from node.

In the present embodiment, be the slave node distribution subtask in grouping according to the mission bit stream performing step S405 Later, the calculating after respective subtask calculates can be fed back to the grouping host node from node due to each in packet node As a result, so need to integrate the result of calculation of subtask that receives of grouping host node, following step is specifically included Suddenly：

The result of calculation is integrated；

In the present embodiment, the result of calculation by after integration is forwarded to the global host node, can be with It realizes in the following way：Grouping host node by the same grouping received from node based on calculating the calculating fed back behind subtask As a result, it is forwarded based on message passing interface to the global host node.

It should be understood that the result of calculation received is carried out to the global host node in the grouping host node After forwarding, just illustrate that the packet node completes the calculating process of the task of the global host node distribution, i.e.,：The packet node Interior not have calculating task, then the grouping host node can trigger after each forwarding and perform step S401 and be sent out to global host node The request of acquisition task is sent, obtaining new task to the global host node for active is calculated, and without waiting for cluster Interior whole packet node is completed to be allocated by the global host node again when secondary calculating task, improves the meter in cluster Calculate efficiency.

Since the level of hardware (computing capability) of packet node is different, in order to determine the calculating of each packet node speed It spends (calculated performance), the technical solution of the present embodiment provides a kind of preferred embodiment, under preferred embodiment, needs to grouping Each task that node calculates carries out timing, be that the slave node in grouping divides according to the mission bit stream performing step S405 After subtask, it is also necessary to perform following steps：

Start timer record temporal information；

As a preferred embodiment, in method for allocating tasks provided by the embodiments of the present application, the task, including： Modeler model feature.Such as：It applies in credit and financial field, task includes：Card number, deposit number, by stages amount, situation Deng.

After the task in the Modelling feature pond is all had been calculated, the packet node can receive described global main The task parameters that node is fed back based on epicycle feature selecting, the packet node can be by the features based on the calculating parameter It is stored, to carry out next round iteration, is specifically comprised the following steps：

Grouping host node receives the calculating parameter that the global host node is fed back based on the result of calculation；

Slave node into grouping forwards the calculating parameter.

In the above-described embodiment, a kind of method of task distribution is provided, it is corresponding with the method for above-mentioned task distribution , present invention also provides a kind of task assigned units.Since the embodiment of device is substantially similar to the embodiment of method, institute To describe fairly simple, the relevent part can refer to the partial explaination of embodiments of method.Device embodiment described below is only It is only illustrative.The task assigned unit embodiment is as follows：

Fig. 5 is please referred to, it illustrates the schematic diagrames of task assigned unit provided according to an embodiment of the present application.

The task assigned unit, including：Request transmitting unit 501, mission bit stream receiving unit 503 and subtask Allocation unit 505；

The request transmitting unit 501, for sending the request of acquisition task to global host node；

The mission bit stream receiving unit 503 receives the mission bit stream of the global host node feedback；

The subtask allocation unit 505, for being that the slave node in grouping distributes subtask according to the mission bit stream.

Optionally, the subtask allocation unit 505, including：Task obtain subelement, subtask divide subelement and Subelement is distributed in subtask；

The task obtains subelement, for obtaining corresponding task according to the mission bit stream；

The subtask divides subelement, for according to the slave interstitial content in grouping, the task to be divided into correspondence From the subtask of interstitial content；

Optionally, the task assigned unit, further includes：Result of calculation receiving unit, integral unit and forwarding are single Member；

The result of calculation receiving unit, for it is described be that the slave node in grouping distributes son according to the mission bit stream After task, receive in grouping it is described from node calculated based on the subtask after the result of calculation fed back；

The integral unit, for the result of calculation to be integrated；

The retransmission unit, for the result of calculation after integrating to be forwarded to the global host node.

Optionally, the task assigned unit, further includes：Request sends trigger element；

The request sends trigger element, for the result of calculation by after integration to the global host node into After row forwarding, the request transmitting unit 501 is triggered.

Optionally, the task assigned unit, further includes：Timer initiation unit, timer stop unit are with timely Between transmitting element；

The timer initiation unit, for it is described be that slave node distribution in grouping is appointed according to the mission bit stream After business, start timer record temporal information；

The timer stop unit, for perform it is described by the result of calculation received to the main section of the overall situation After point is forwarded, stop the timer；

The time transmitting element, for recorded to the global host node transmission the calculating of the mission bit stream when Between.

Optionally, the task assigned unit, further includes：Calculating parameter receiving unit and parameter retransmission unit；

The calculating parameter receiving unit, in the result of calculation by after integration to the global host node After being forwarded, the calculating parameter that the global host node is fed back based on the result of calculation is received；

The parameter retransmission unit, for forwarding the calculating parameter from node described in into grouping.

Optionally, the request transmitting unit 501 is obtained specifically for being based on message passing interface to the transmission of global host node Take the request of task.

Optionally, the task, including：Modeler model feature.

In the above-described embodiment, a kind of method of task processing, a kind of device of task processing, a kind of task are provided The method of distribution and a kind of task assigned unit, in addition, present invention also provides a kind of methods of task management；Described The embodiment of the method managed of being engaged in is as follows：

Fig. 6 is please referred to, it illustrates the flow charts of the method for the task management provided according to an embodiment of the present application.

The method of the task management, including：

Step S601 receives the request of acquisition task that grouping host node is sent.

In the present embodiment, the method for the task management be by the task handle frame in global host node into What row was implemented, following manner realization may be used in the request of acquisition task for receiving grouping host node and sending：It is to be based on disappearing It ceases passing interface and receives the request for obtaining new task that the grouping host node is sent.

Step S603, distributing task for the grouping host node and feeding back distribution to the grouping host node for task are believed Breath.

In the present embodiment, it is described to distribute task for the grouping host node and feed back distribution to the grouping host node Following manner realization may be used in mission bit stream：The overall situation host node accesses the task pool and obtains individual task information, and The mission bit stream of acquisition is distributed into the grouping host node, and believe to the new task of the grouping host node feedback distribution Breath.

Due on different grouping node handle task machine software and hardware configuration it is different, also with external interference because Element, therefore cause the processing time of each subtask different, operate in the processing time of the subtask on the node of poor-performing The processing time of other machines processing subtask can be considerably longer than, becomes the bottleneck of entire task run, is made to eliminate long-tail Into influence, step S603 be it is described grouping host node distribute task and to it is described grouping host node feedback distribution task believe After breath, including：

Obtain the shortest grouping host node of average calculation times；

If so, remove the grouping host node.

It should be understood that in the application scenarios of machine learning, modeling is finally modeled by model characteristics, weighing apparatus The variable of model performance is measured, the aspect of model can be quantified by feature significance.

When the task is modeler model feature, it will usually explanatory preferable linear model is selected to be modeled, The reasons such as the stability to ensure explanatory and model can select to contribute model in a large amount of modeler model feature highest Preceding several features carry out final modeling, on the one hand can be with reduced model, the explanatory and stability of lift scheme, the opposing party Face can also reject the feature that some can bring a negative impact from model, the performance of lift scheme, so step S603 is The grouping host node distributes task and after the mission bit stream of the grouping host node feedback distribution, including：

Receive the result of calculation of each grouping host node forwarding；

The calculating parameter obtained to each grouping host node feedback.

In the above-described embodiment, a kind of method of task management is provided, it is corresponding with the method for above-mentioned task management , present invention also provides a kind of devices of task management.Since the embodiment of device is substantially similar to the embodiment of method, institute To describe fairly simple, the relevent part can refer to the partial explaination of embodiments of method.Device embodiment described below is only It is only illustrative.The device embodiment of the task management is as follows：

Fig. 7 is please referred to, it illustrates the schematic diagrames of the device of the task management provided according to an embodiment of the present application.

The device of the task management, including：Task requests receiving unit 701 and mission bit stream allocation unit 703；

The task requests receiving unit 701, for receiving the request of acquisition task that grouping host node is sent；

The mission bit stream allocation unit 703, for distributing task for the grouping host node and being grouped main section to described The mission bit stream of point feedback distribution.

Optionally, the device of the task management, further includes：It calculates time receiving unit, total time computing unit, be averaged Time calculating unit, shortest time acquiring unit, time judging unit and knot-removal unit；

It is described calculating time receiving unit, for it is described for it is described grouping host node distribute task and to it is described grouping lead After the mission bit stream of node feeding back distribution, the calculating of the mission bit stream recorded that the grouping host node is sent is received Time；

The knot-removal unit, for receiving the judging result of the time judging unit, if so, described point of removal Group host node.

Optionally, the task requests receiving unit 701 receives grouping host node specifically for being based on message passing interface The request of the acquisition task of transmission.

Optionally, the task, including：Modeler model feature.

Optionally, the task assigned unit, further includes：Result of calculation receiving unit, calculating parameter acquiring unit with And calculating parameter feedback unit；

The result of calculation receiving unit, for it is described for it is described grouping host node distribute task and to it is described grouping lead After the mission bit stream of node feeding back distribution, the result of calculation of each grouping host node forwarding is received；

The calculating parameter feedback unit, for the calculating parameter obtained to each grouping host node feedback.

In the above-described embodiment, a kind of method of task processing, a kind of device of task processing, a kind of task are provided The method of distribution, a kind of task assigned unit, a kind of method of task distribution and a kind of task assigned unit, in addition, Present invention also provides a kind of methods of task computation；The embodiment of the method for the task computation is as follows：

Fig. 8 is please referred to, it illustrates the flow charts of the method for the task computation provided according to an embodiment of the present application.

The method of the task computation, including：

Step S801 receives the subtask of grouping host node distribution.

In the present embodiment, the method for the task computation is each the grouping section handled by the task in frame What the slave node in point was implemented.

It should be noted that the subtask is according to from the number of node, by the task evenly distribute it is described from Subtask on node.

Such as：There are 4 nodes in the packet node, one of node is grouping host node, and three are from node, then Task is evenly distributed from node, make at three each subtask calculated from node calculation amount be predecessor business be three / mono-.It should be understood that by task evenly distribute it is described from node after, the specific tasks each distributed from node Content differs, then described that the subtask can be calculated parallel from node.

Step S803 calculates the subtask based on calculating parameter.

In the present embodiment, the task based access control parameter calculates the subtask, and following manner realization may be used：It is described From node by based on the calculating parameter being locally stored, the subtask is calculated.

It should be noted that former maximum values taken turns in the task result generated during iteration that the calculating parameter is, i.e.,： Highest feature is contributed with the modeling.If when performing this step, the task of calculating is first round iteration, then described from node sheet There is no store tasks parameter in ground, then directly the subtask can be calculated.

Disk S805, to the grouping host node feedback based on the result of calculation after the calculating of the subtask.

It is in the present embodiment, described to feed back the result of calculation after being calculated based on the subtask to the grouping host node, Following manner realization may be used：It is calculated from node by the subtask of the grouping host node distribution in same grouping, having been calculated After stating subtask the result of calculation after host node feedback is calculated based on the subtask is grouped into same grouping.

It should be noted that the result of calculation can be character conspicuousness information.The feature significance includes Marginal Contrinution(MC)。

In the present embodiment, it is described to be walked to the grouping host node feedback based on the result of calculation after the calculating of the subtask After rapid, including：

Receive the calculating parameter of the grouping host node forwarding.

In the above-described embodiment, a kind of method of task computation is provided, it is corresponding with the method for above-mentioned task computation , present invention also provides a kind of devices of task computation.Since the embodiment of device is substantially similar to the embodiment of method, institute To describe fairly simple, the relevent part can refer to the partial explaination of embodiments of method.Device embodiment described below is only It is only illustrative.The device embodiment of the task computation is as follows：

Fig. 9 is please referred to, it illustrates the schematic diagrames of the device of the task computation provided according to an embodiment of the present application.

The device of the task computation, including：Subtask receiving unit 901, subtask computing unit 903 and calculating knot Fruit feedback unit 905；

The subtask receiving unit 901, for receiving the subtask of grouping host node distribution；

The subtask computing unit 903 calculates the subtask for being based on calculating parameter；

The result of calculation feedback unit 905, after being calculated to the grouping host node feedback based on the subtask Result of calculation.

Optionally, the device of the task computation, further includes：Calculating parameter receiving unit；

The calculating parameter receiving unit, for being calculated described to the grouping host node feedback based on the subtask After result of calculation afterwards, the calculating parameter of the grouping host node forwarding is received.

In a typical configuration, computing device includes one or more processors (CPU), input/output interface, net Network interface and memory.

Memory may include computer-readable medium in volatile memory, random access memory (RAM) and/or The forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable medium Example.

1st, computer-readable medium can be by any side including permanent and non-permanent, removable and non-removable media Method or technology realize that information stores.Information can be computer-readable instruction, data structure, the module of program or other numbers According to.The example of the storage medium of computer includes, but are not limited to phase transition internal memory (PRAM), static RAM (SRAM), dynamic random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), fast flash memory bank or other memory techniques, CD-ROM are read-only Memory (CD-ROM), digital versatile disc (DVD) or other optical storages, magnetic tape cassette, tape magnetic rigid disk storage or Other magnetic storage apparatus or any other non-transmission medium, available for storing the information that can be accessed by a computing device.According to Herein defines, and computer-readable medium does not include non-temporary computer readable media (transitory media), such as modulates Data-signal and carrier wave.

2nd, it will be understood by those skilled in the art that embodiments herein can be provided as method, system or computer program production Product.Therefore, the embodiment in terms of complete hardware embodiment, complete software embodiment or combination software and hardware can be used in the application Form.It can be used moreover, the application can be used in one or more computers for wherein including computer usable program code The computer program product that storage medium is implemented on (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) Form.

Although the application is disclosed as above with preferred embodiment, it is not for limiting the application, any this field skill Art personnel are not being departed from spirit and scope, can make possible variation and modification, therefore the guarantor of the application Shield range should be subject to the range that the application claim is defined.

Claims

1. a kind of task processing method, which is characterized in that including；

Global host node distributes task for each grouping host node, and the mission bit stream of distribution is fed back to each grouping host node；

The mission bit stream that each grouping host node is fed back according to the global host node is appointed for slave node distribution in grouping Business；

Behind the subtask from node calculating, after the grouping host node feedback into same grouping is calculated based on the subtask Result of calculation；

The result of calculation received is forwarded, and forwarding every time by each grouping host node to the global host node The request for obtaining new task is sent to the global host node afterwards；

After the overall situation host node receives the request for obtaining new task that the grouping host node is sent, for the main section of grouping Point distributes new task, and the new mission bit stream of distribution is fed back to the grouping host node.

2. task processing method according to claim 1, which is characterized in that the overall situation host node is each main section of grouping Point distribution task, and the mission bit stream distributed is fed back to each grouping host node, including：

The overall situation host node obtains the mission bit stream of corresponding number according to the number of grouping host node；The mission bit stream, packet It includes：The signature identification of the task；

The mission bit stream that the overall situation host node is obtained for each grouping host node distribution, and it is anti-to each grouping host node The mission bit stream of the grouping host node is distributed in feedback.

3. task processing method according to claim 2, which is characterized in that each grouping host node is according to described complete The mission bit stream of office's host node feedback, subtask is distributed for the slave node in grouping, including：

The subtask is evenly distributed described from node.

4. task processing method according to claim 3, which is characterized in that each grouping host node will receive The result of calculation is forwarded to the global host node, including:

Each grouping host node, by the grouping received it is all from node based on the subtask calculate after result of calculation into Row is integrated；

5. task processing method according to claim 1, which is characterized in that in each grouping host node according to The mission bit stream of global host node feedback, after the step of distributing subtask for the slave node in grouping, including：

Start timer record temporal information；

Performing what the result of calculation received was forwarded by each grouping host node to the global host node After step, stop the timer；

6. task processing method according to claim 5, which is characterized in that it is described stopping the timer the step of it Afterwards, including：

7. task processing method according to claim 6, which is characterized in that main to the overall situation in the grouping host node After the step of calculating time of the mission bit stream that node transmission has recorded, including：

Obtain the shortest grouping host node of average calculation times；

Judge whether the average calculation times of other grouping host nodes are more than the total of grouping host node calculating task obtained Evaluation time；

If so, remove the grouping host node.

8. according to the task processing method described in claim 1-7 any one, which is characterized in that it is described the overall situation host node and Message passing interface is based between the grouping host node into row data communication.

9. task processing method according to claim 8, which is characterized in that the grouping host node is described from node The slave node of the numerical value minimum of serial number.

10. task processing method according to claim 8, which is characterized in that the task, including：Modeler model feature.

11. task processing method according to claim 10, which is characterized in that will be received in each grouping host node To the result of calculation to the global host node be forwarded the step of after, including：

The calculating parameter obtained to each grouping host node feedback.

12. task processing method according to claim 11, which is characterized in that described to each grouping host node feedback After the step of calculating parameter obtained, including：

Slave node into grouping forwards the calculating parameter.

13. task processing method according to claim 12, which is characterized in that it is described to calculate the subtask from node, Including：

14. a kind of Task Processing Unit, which is characterized in that including；

Mission bit stream allocation unit distributes task, and be grouped main section to each for global host node for each grouping host node The mission bit stream of point feedback distribution；

Subtask allocation unit, for the mission bit stream that each grouping host node is fed back according to the global host node, for grouping Interior slave node distribution subtask；

Result of calculation feedback unit, for described after node calculates the subtask, the grouping host node into same grouping is anti- Present the result of calculation after being calculated based on the subtask；

Task requests transmitting element, for each grouping host node by the result of calculation received to the global host node It is forwarded, and sends the request for obtaining new task to the global host node after each forwarding；

Allocation unit, after the request for obtaining new task that the grouping host node is sent is received for the global host node, Task and the new mission bit stream distributed to the grouping host node feedback are distributed for the grouping host node.

A kind of 15. method of task distribution, which is characterized in that including：

The request of acquisition task is sent to global host node；

Receive the mission bit stream of the global host node feedback；

16. the method for task according to claim 15 distribution, which is characterized in that it is described according to the mission bit stream for point Slave node distribution subtask in group, including：

Corresponding task is obtained according to the mission bit stream；

The subtask is evenly distributed described from node.

17. the method for task distribution according to claim 16, which is characterized in that be according to the mission bit stream described After the step of slave node in grouping distributes subtask, including：

The result of calculation is integrated；

18. the method for task distribution according to claim 17, which is characterized in that the calculating knot by after integration Fruit to the global host node be forwarded the step of after, including：

19. the method for task distribution according to claim 17, which is characterized in that be according to the mission bit stream described After the step of slave node in grouping distributes subtask, including：

Start timer record temporal information；

After described the step of being forwarded the result of calculation received to the global host node is performed, stop institute State timer；

20. the method for task distribution according to claim 17, which is characterized in that in the calculating by after integration As a result after the step of being forwarded to the global host node, including：

Described in into grouping the calculating parameter is forwarded from node.

21. the method for the task distribution according to claim 15-20 any one, which is characterized in that described main to the overall situation Node sends the request of acquisition task, including：

22. the method for task distribution according to claim 21, which is characterized in that the task, including：Modeler model is special Sign.

23. a kind of task assigned unit, which is characterized in that including：

A kind of 24. method of task management, which is characterized in that including：

Receive the request of acquisition task that grouping host node is sent；

25. the method for task management according to claim 24, which is characterized in that described for the grouping host node point With task and after the step of mission bit stream of the grouping host node feedback distribution, including：

Obtain the shortest grouping host node of average calculation times；

If so, remove the grouping host node.

26. the method for the task management according to claim 24-25 any one, which is characterized in that described to receive grouping The request for the acquisition task that host node is sent, including：

27. the method for task distribution according to claim 26, which is characterized in that the task, including：Modeler model is special Sign.

28. the method for task distribution according to claim 27, which is characterized in that described for the grouping host node point With task and after the step of mission bit stream of the grouping host node feedback distribution, including：

Receive the result of calculation of each grouping host node forwarding；

The calculating parameter obtained to each grouping host node feedback.

29. a kind of device of task management, which is characterized in that including：

Mission bit stream allocation unit, for distributing task for the grouping host node and feeding back distribution to the grouping host node Mission bit stream.

A kind of 30. method of task computation, which is characterized in that including：

Receive the subtask of grouping host node distribution；

The subtask is calculated based on calculating parameter；

31. the method for task computation according to claim 30, which is characterized in that described anti-to the grouping host node After the step of presenting the result of calculation after being calculated based on the subtask, including：

Receive the calculating parameter of the grouping host node forwarding.

32. a kind of device of task computation, which is characterized in that including：

Result of calculation feedback unit, for feeding back the result of calculation after being calculated based on the subtask to the grouping host node.