CN106407002A - Data processing task execution method and device - Google Patents
Data processing task execution method and device Download PDFInfo
- Publication number
- CN106407002A CN106407002A CN201610702453.8A CN201610702453A CN106407002A CN 106407002 A CN106407002 A CN 106407002A CN 201610702453 A CN201610702453 A CN 201610702453A CN 106407002 A CN106407002 A CN 106407002A
- Authority
- CN
- China
- Prior art keywords
- server
- task
- execute server
- execute
- data processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 239000000284 extract Substances 0.000 claims description 18
- 230000005540 biological transmission Effects 0.000 claims description 14
- 238000012795 verification Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 5
- 239000003550 marker Substances 0.000 claims description 4
- 239000002699 waste material Substances 0.000 description 5
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000000052 comparative effect Effects 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/503—Resource availability
Abstract
The invention relates to a data processing task execution method and device. The method comprises the following steps of: receiving a task distribution notification sent by a distribution server; sending a task executioner request to a sorting server according to the task distribution notification; obtaining an execution server sorting result from the sorting server, wherein the execution server sorting result is generated by the sorting server according to task execution requests sent by the execution servers after receiving the task execution requests; determining a sorting position of a local execution server in the server sorting result, and judging whether the sorting position is in a preset sorting range; and when the sorting position is in the preset sorting range, obtaining a data processing task from the distribution server and executing the data processing task. According to the data processing execution method and device provided by the invention, the condition that the data processing task is repeatedly executed by a plurality of execution servers is avoided, so that the data processing resources consumed during the execution of the data processing task are saved.
Description
Technical field
The present invention relates to technical field of data processing, more particularly to a kind of data processing task execution method and apparatus.
Background technology
In technical field of data processing, when data is processed, generally require to set up data processing task, and will set up
Data processing task send to execute server, execute server gets data processing task and executes.
However, in traditional data processing field, being provided with multiple stage execute server, when multiple stage execution service monitoring arrives
When having data processing task to generate, all can obtain data processing task, and execute the data processing task getting, such multiple stage
Execute server all in execution data processing task, can lead to a data processing task by holding that multiple stage execute server repeats
Row, so that expending excessive resource during a data processing task, causes the waste of data processing resources.
Content of the invention
Based on this it is necessary to be directed to the problem of the wasting of resources in data processing task implementation procedure, provide at a kind of data
Reason task executing method and device.
A kind of data processing task executes method, is applied to execute server, methods described includes：
Receive the task notice distribution that Distributor sends；
Tasks carrying request is sent to sequence server according to described task notice distribution；
Obtain execute server ranking results from described sequence server, described execute server ranking results are by described row
Sequence server, after receiving the tasks carrying request that each execute server sends, asks to generate according to described tasks carrying；
Determine sorting position in described server orders result for the local execute server, judge described sorting position
Whether within the scope of predetermined order；When judging that described sorting position is within the scope of predetermined order, from described distribution service
Device obtains described data processing task and executes.
Wherein in an embodiment, described according to described task notice distribution to sequence server send tasks carrying please
After asking, also include：
Receive the ballot request that described sequence server is initiated, described ballot request is for choosing execution data processing task
Execute server；
Vote information is generated according to described ballot request；
Described vote information is sent to described sequence server, makes described sequence server according to each execution receiving
The vote information that server returns generates execute server ranking results.
Wherein in an embodiment, described according to described task notice distribution to sequence server send tasks carrying please
After asking, also include：
Receive described sequence server and send resource utilization acquisition request；
Request is obtained according to described resource utilization and extracts resource utilization, the resource utilization extracted is back to institute
State sequence server, make described sequence server generate execution according to the resource utilization that each execute server receiving returns
Server orders result.
Wherein in an embodiment, row in described server orders result for the local execute server of described determination
Tagmeme is put, and judges described sorting position whether within the scope of predetermined order, including：
Extract the execute server mark coming first in execute server ranking results；
The execute server extracted mark is compared with self identification；
If the described execute server mark extracted is consistent with described self identification, judge local execute server
Sorting position within the scope of predetermined order.
Wherein in an embodiment, described obtain described data processing task executing, bag from described Distributor
Include：
Send the task acquisition request carrying execute server mark to described Distributor；
Receiving data process task, described data processing task is described Distributor according to execute server sequence knot
Fruit is to described execute server identity verification by rear return.
Above-mentioned data processing task executes method, after the task notice distribution receiving Distributor transmission, according to
Task notice distribution sends tasks carrying request to sequence server, and sequence server is sent out according to each execute server receiving
The tasks carrying request sent is ranked up to execute server generating execute server ranking results.Execute server is from sequence clothes
Business device obtains execute server ranking results, is determining local execute server default according to execute server ranking results
When within sequencing horizon, obtain data processing task from Distributor and execute.So, by execute server ranking results
The quantity of execute server with execution data processing task authority is controlled, it is to avoid multiple execute servers are connecing
When receiving task notice distribution, obtain, from Distributor, the situation that data processing task causes data congestion simultaneously, also avoid
The situation that data processing task is repeated by multiple execute servers, thus disappeared when saving execution data processing task
The data processing resources of consumption.
A kind of data processing task performs device, is applied to execute server, and described device includes：
Notice distribution receiver module, for receiving the task notice distribution of Distributor transmission；
Execution request sending module, please for sending tasks carrying according to described task notice distribution to sequence server
Ask；
Ranking results acquisition module, for obtaining execute server ranking results, described execution from described sequence server
Server orders result by described sequence server after receiving the tasks carrying request that each execute server sends, according to institute
State tasks carrying request to generate；
Sorting position judge module, for determining sequence in described server orders result for the local execute server
Whether position, judge described sorting position within the scope of predetermined order；
Data task acquisition module, for when judging that described sorting position is within the scope of predetermined order, from described point
Send out server to obtain described data processing task and execute.
Wherein in an embodiment, described device also includes：
Ballot request receiving module, for receiving the ballot request that described sequence server is initiated, described ballot request is used
In the execute server choosing execution data processing task；
Vote information generation module, for generating vote information according to described ballot request；
Vote information sending module, for sending described vote information to described sequence server, makes described sequence clothes
Business device generates execute server ranking results according to the vote information that each execute server receiving returns.
Wherein in an embodiment, described device also includes：
Obtain request receiving module, send resource utilization acquisition request for receiving described sequence server；
Resource utilization returns module, extracts resource utilization for obtaining request according to described resource utilization, will carry
The resource utilization got is back to described sequence server, makes described sequence server according to each execute server receiving
The resource utilization returning generates execute server ranking results.
Wherein in an embodiment, described sorting position judge module includes：
Marker extraction module, for extracting the execute server mark coming first in execute server ranking results；
Mark comparing module, for comparing the execute server extracted mark with self identification；
Sequence determining module, if identify consistent with described self identification for the described execute server extracted, sentences
The sorting position of fixed local execute server is within the scope of predetermined order.
Wherein in an embodiment, described data task acquisition module includes：
Task requests sending module, obtains for sending the task of carrying execute server mark to described Distributor
Take request；
Data task receiver module, for receiving data process task, described data processing task is described distribution service
Device according to execute server ranking results to described execute server identity verification by rear return.
Above-mentioned data processing task performs device, after the task notice distribution receiving Distributor transmission, according to
Task notice distribution sends tasks carrying request to sequence server, and sequence server is sent out according to each execute server receiving
The tasks carrying request sent is ranked up to execute server generating execute server ranking results.Execute server is from sequence clothes
Business device obtains execute server ranking results, is determining local execute server default according to execute server ranking results
When within sequencing horizon, obtain data processing task from Distributor and execute.So, by execute server ranking results
The quantity of execute server with execution data processing task authority is controlled, it is to avoid multiple execute servers are connecing
When receiving task notice distribution, obtain, from Distributor, the situation that data processing task causes data congestion simultaneously, also avoid
The situation that data processing task is repeated by multiple execute servers, thus disappeared when saving execution data processing task
The data processing resources of consumption.
Brief description
Fig. 1 is the applied environment figure of data processing task execution system in an embodiment；
Fig. 2 is the schematic flow sheet that in an embodiment, data processing task executes method；
Fig. 3 is to be illustrated according to the flow process that vote information generates the step of execute server ranking results in an embodiment
Figure；
Fig. 4 is to be illustrated according to the flow process that resource utilization generates the step of execute server ranking result in an embodiment
Figure；
Fig. 5 is the step determining execution data processing task authority according to execute server ranking results in an embodiment
Schematic flow sheet；
Fig. 6 is the structured flowchart of data processing task performs device in an embodiment；
Fig. 7 is the structured flowchart of data processing task performs device in another embodiment；
Fig. 8 is the structured flowchart of data processing task performs device in another embodiment；
Fig. 9 is the structured flowchart of sorting position judge module in an embodiment；
Figure 10 is the structured flowchart of data task acquisition module in an embodiment.
Specific embodiment
In order that the objects, technical solutions and advantages of the present invention become more apparent, below in conjunction with drawings and Examples, right
The present invention is further elaborated.It should be appreciated that specific embodiment described herein is only in order to explain the present invention, and
It is not used in the restriction present invention.
Fig. 1 is the applied environment figure of data processing task execution system in an embodiment, and data processing task executes system
System includes execute server 110, Distributor 120 and sequence server 130.Execute server 110 passes through network and distribution
Server 120 connects, and execute server 110 is passed through network and is connected with sequence server 130, wherein execute server 110, distribution
Server 120 and sequence server 130 can be specifically multiple, and Distributor 120 and sequence server 130 can be integrated
In same server.
As shown in Fig. 2 in one embodiment, provide a kind of data processing task execution method, the present embodiment is with the party
Method is particularly applicable in illustrate in the execute server 110 of Fig. 1 data processing task execution system, execute server 110
Upper operation has data processing task configuration processor, and data processing task processing routine is used for implementing data processing task process side
Method, data processing task processing method specifically includes herein below：
S202, receives the task notice distribution that Distributor sends.
Specifically, after Distributor 120 receives the data processing task of terminal transmission, obtain data processing task
Task identification, generates task notice distribution according to task identification, task notice distribution is sent to each execute server, specifically may be used
Task notice distribution to be sent to each execute server 110 in the form of broadcasting.Data processing task can also be in distribution
Server 120 receives the task creation order of terminal transmission, creates data processing task according to task creation order, and for creating
The data processing task built generates task identification.
S204, sends tasks carrying request according to task notice distribution to sequence server.
Specifically, execute server 110, after the task notice distribution receiving Distributor 120 transmission, is extracted and is appointed
Task identification in business notice distribution, generates tasks carrying request according to task identification, and the tasks carrying generating request is sent out
Deliver to sequence server 130.
S206, obtains execute server ranking results from sequence server, execute server ranking results are by the service of sorting
Device, after receiving each execute server transmission tasks carrying request, asks to generate according to tasks carrying.
Specifically, in the tasks carrying request receiving execute server 110 transmission, record is every for sequence server 130
The reception time of individual tasks carrying request, extract the execute server mark in tasks carrying request, each according to record is appointed
The reception time of business execution request is ranked up to the execute server mark from tasks carrying, according to the time of reception to execution
Server identification is ranked up obtaining execute server ranking results, and sequence server 130 extracts appointing in tasks carrying request
Business mark, by corresponding with task identification for execute server ranking results storage, each task identification corresponds to an execute server
Ranking results.Execute server 110 sends the ranking results acquisition request carrying task identification, sequence clothes to sequence server
After business device 130 receives ranking results acquisition request, extract ranking results and obtain task identification in request, lookup task identification pair
The execute server ranking results answered, the execute server finding ranking results are back to and execute clothes by sequence server 130
Business device 110.
S208, determines sorting position in server orders result for the local execute server, judges that sorting position is
No within the scope of predetermined order.
Specifically, execute server 110 receive sequence server 130 return execute server ranking results after,
Sorting position, the sorting position according to self identification and the predetermined order of self identification is searched in execute server ranking results
Position range compares, if the sorting position of self identification within predetermined order position range it is determined that having execution data
The authority of process task；If the sorting position of self identification is not held it is determined that not having within predetermined order position range
The authority of row data processing task.
S210, when judging that sorting position is within the scope of predetermined order, obtains data processing task from Distributor
And execute.
Specifically, after execute server 110 has the authority executing data processing task in determination, to Distributor
120 send task obtains request, and task obtains request and includes task identification.Distributor 120 is receiving the service of execution
After the task that device 110 sends obtains request, task of extracting obtains the task identification in request, at task identification searching data
Reason task, the data processing task finding is sent to execute server 110.
In one embodiment, after execute server 110 gets data processing task from Distributor 120, then
This data processing task is locked by Distributor 120, when Distributor 120 is receiving this data processing task of acquisition
When obtaining request, then refuse this acquisition request；It is specifically as follows the corresponding task identification of this data processing task and add locking mark
Note.
In one embodiment, S210 specifically includes herein below：Send to Distributor and carry execute server
The task of mark obtains request；Receiving data process task, data processing task is that Distributor is arranged according to execute server
Sequence result is to execute server identity verification by rear return.
Specifically, execute server 110 sends the task acquisition carrying execute server mark to Distributor 120
Request.Distributor 120 obtains execute server ranking results from sequence server 130, and Distributor 120 is receiving
Obtain after request to the task that execute server 110 sends, extract the execute server mark in task requests, according to executing clothes
Business device ranking results determine the sorting position of execute server mark, thus verifying that execute server mark is corresponding to execute service
Whether device 110 has the authority of execution data processing task, after being verified, obtains the task identification in request according to task
Search corresponding data processing task, the data processing task finding is back to execute server mark and corresponding executes clothes
Business device 110.
In the present embodiment, receive Distributor transmission task notice distribution after, according to task notice distribution to
Sequence server sends tasks carrying request, and sequence server please according to the tasks carrying that each execute server receiving sends
Ask and execute server is ranked up generate execute server ranking results.Execute server obtains from sequence server and executes clothes
Business device ranking results, when determining the authority with execution data processing task according to execute server ranking results, from distribution
Server obtains data processing task and executes.So, appointed to having execution data processing by execute server ranking results
The quantity of execute server of business authority is controlled, it is to avoid multiple execute servers when receiving task notice distribution,
Obtain, from Distributor, the situation that data processing task causes data congestion simultaneously, it also avoid data processing task multiple
The situation that execute server repeats, thus save the data processing resources being consumed during execution data processing task.
As shown in figure 3, in one embodiment, specifically also include after S204 generating execute server according to vote information
The step of ranking results, this step specifically includes herein below：
S302, receives the ballot request that sequence server is initiated, ballot request is for choosing execution data processing task
Execute server.
Specifically, sequence server 130, after receiving the tasks carrying request that each execute server sends, extracts task
Execute server mark in execution request, generates ballot request, ballot according to mentioning the multiple execute server marks obtaining
Ask the execute server 110 for choosing execution data processing task.
S304, generates vote information according to ballot request.
Specifically, execute server 110, after receiving ballot request, extracts the multiple execute servers in ballot request
Mark, chooses an execute server mark from the multiple execute server marks extracted, according to the execute server chosen
Mark is as vote information.
In one embodiment, execute server 110 receive ballot request after, from storage multiple execute servers
Choose an execute server mark in mark, identified as vote information according to the execute server chosen.
S306, vote information is sent to sequence server, makes sequence server according to each execute server receiving
The vote information returning generates execute server ranking results.
Specifically, execute server 110, after generating vote information, vote information is sent to sequence server 130.Row
Sequence server 130, after receiving the vote information that each execute server sends, extracts the execute server mark in vote information
Know, the poll of statistics each execute server mark, be ranked up obtaining executing clothes according to the poll execute server mark of statistics
Business device ranking results.Specifically can be ranked up from high to low according to poll, can also be ranked up from low to high.
In the present embodiment, after the ballot request receiving server transmission of all sorting, ballot letter is generated according to ballot request
Breath, vote information is sent to sequence server, makes the ballot that sequence server returns according to each execute server receiving
Information is ranked up obtaining execute server ranking results to execute server mark, determines execution data according to execute server
The execute server of process task, it is to avoid data processing task is performed a plurality of times, thus waste data processing resources.
As shown in figure 4, in one embodiment, S204 specifically also includes generating execute server row according to resource utilization
The step of name result, this step specifically includes herein below：
S402, receives sequence server and sends resource utilization acquisition request.
Specifically, sort server 130 after receiving the tasks carrying request that each execute server sends, to each execution
Server sends resource utilization and obtains request, and resource utilization obtains request for obtaining the utilization of resources of each execute server
Rate.
S404, obtains request according to resource utilization and extracts resource utilization, the resource utilization extracted is back to
Sequence server, makes sequence server generate execute server according to the resource utilization that each execute server receiving returns
Ranking results.
Specifically, execute server 110, after receiving resource utilization acquisition request, obtains resource utilization parameters, root
According to resource utilization parameters computing resource utilization rate, resource utilization sends to sequence server 130.Sequence server 130 is connecing
After receiving the resource utilization that each execute server returns, resource utilization is compared, and according to comparative result to resource
Utilization rate corresponding execute server mark is ranked up, can be according to resource utilization from low to high or from high to low suitable
Ordered pair execute server mark is ranked up obtaining execute server ranking results.
In the present embodiment, request is obtained according to the resource utilization that sequence server sends and returns resource utilization, the row of making
Sequence server is ranked up generating execute server ranking results to execute server mark according to resource utilization, according to execution
The resource utilization of server determines the relatively low execute server execution data processing task of resource utilization, improves at data
The efficiency of reason task.
As shown in figure 5, in one embodiment, S208 specifically includes and determines execution number according to execute server ranking results
According to the step of process task authority, this step specifically includes herein below：
S502, extracts the execute server mark coming first in execute server ranking results.
Specifically, execute server 110 is after receiving execute server ranking results, from execute server ranking results
Middle extraction comes first execute server mark.Execute server ranking results can be to identify gained according to execute server
Poll number arranged from high to low, come first for number of votes obtained highest execute server mark；Can also be by
The resource utilization identifying corresponding execute server according to execute server is ranked up to execute server mark from low to high,
Come the execute server mark of first execute server relatively low for resource utilization.
S504, the execute server extracted mark is compared with self identification.
Specifically, execute server 110 is extracting the execute server coming first from execute server ranking results
After mark, the execute server extracted mark is compared with self identification.Self identification is that execute server 110 is corresponding to be held
Row server identification.
S506, if the execute server mark extracted is consistent with self identification, judges local execute server
Sorting position is within the scope of predetermined order.
Specifically, execute server 110 by the execute server extracted mark with self identification compare when, if pass through
Compare, the execute server mark extracted is consistent with self identification, represent that execute server 110 has execution data processing and appoints
The authority of business, then execute server 110 obtain data processing task executing from Distributor.
From execute server ranking results, in this city embodiment, come first execute server mark, the will be come
One execute server mark is compared with self identification, is determined whether with execution data processing task according to comparison result
Authority, this avoid all execute servers all go execute data processing task, cause data processing resources waste feelings
Condition.
As shown in fig. 6, in one embodiment, provide a kind of data processing task performs device 600, this device specifically wraps
Include：Notice distribution receiver module 602, execution request sending module 604, ranking results acquisition module 606, sorting position judge mould
Block 608 data task acquisition module 610.
Notice distribution receiver module 602, for receiving the task notice distribution of Distributor transmission.
Execution request sending module 604, for sending tasks carrying request according to task notice distribution to sequence server.
Ranking results acquisition module 606, for obtaining execute server ranking results, execute server from sequence server
Ranking results, by sorting server after receiving the tasks carrying request that each execute server sends, are asked according to tasks carrying
Generate.
Sorting position judge module 608, for determining sequence in server orders result for the local execute server
Whether position, judge sorting position within the scope of predetermined order.
Data task acquisition module 610, for when judging that described sorting position is within the scope of predetermined order, from distribution
Server obtains data processing task and executes.
In the present embodiment, receive Distributor transmission task notice distribution after, according to task notice distribution to
Sequence server sends tasks carrying request, and sequence server please according to the tasks carrying that each execute server receiving sends
Ask and execute server is ranked up generate execute server ranking results.Execute server obtains from sequence server and executes clothes
Business device ranking results, are judging the sorting position of local execute server in predetermined order according to execute server ranking results
Within the scope of when, obtain data processing task executing from Distributor.So, by execute server ranking results to tool
The quantity of execute server having execution data processing task authority is controlled, it is to avoid multiple execute servers are receiving
During task notice distribution, obtain, from Distributor, the situation that data processing task causes data congestion simultaneously, it also avoid counting
Situation about being repeated by multiple execute servers according to process task, thus consumed when saving execution data processing task
Data processing resources.
As shown in fig. 7, in one embodiment, data processing task performs device 600 specifically also includes：Ballot request connects
Receive module 612, vote information generation module 614 and vote information sending module 616.
Ballot request receiving module 612, for receiving the ballot request that sequence server is initiated, ballot request is used for choosing
The execute server of execution data processing task.
Vote information generation module 614, for generating vote information according to ballot request.
Vote information sending module 616, for sending vote information to sequence server, makes sequence server according to connecing
The vote information that each execute server receiving returns generates execute server ranking results.
In the present embodiment, after the ballot request receiving server transmission of all sorting, ballot letter is generated according to ballot request
Breath, vote information is sent to sequence server, makes the ballot that sequence server returns according to each execute server receiving
Information is ranked up obtaining execute server ranking results to execute server mark, determines execution data according to execute server
The execute server of process task, it is to avoid data processing task is performed a plurality of times, thus waste data processing resources.
As shown in figure 8, in one embodiment, data processing task performs device 600 specifically also includes：Obtain request to connect
Receive module 618 and resource utilization returns module 620.
Obtain request receiving module 618, send resource utilization acquisition request for receiving sequence server.
Resource utilization returns module 620, extracts resource utilization for obtaining request according to resource utilization, will extract
To resource utilization be back to sequence server, make the resource that sequence server returns according to each execute server receiving
Utilization rate generates execute server ranking results.
In the present embodiment, request is obtained according to the resource utilization that sequence server sends and returns resource utilization, the row of making
Sequence server is ranked up generating execute server ranking results to execute server mark according to resource utilization, according to execution
The resource utilization of server determines whether to execute data processing task, improves the efficiency of data processing task.
As shown in figure 9, in one embodiment, sorting position judge module 608 specifically includes：Marker extraction module
608a, mark comparing module 608b and sequence determining module 608c.
Marker extraction module 608a, for extracting the execute server mark coming first in execute server ranking results
Know.
Mark comparing module 608b, for comparing the execute server extracted mark with self identification.
Sequence determining module 608c, if the execute server mark for extracting is consistent with self identification, judges this
The sorting position of the execute server on ground is within the scope of predetermined order.
From execute server ranking results, in the present embodiment, come first execute server mark, first will be come
Execute server mark compare with self identification, according to the sorting position of the comparison result local execute server of judgement
Within the scope of predetermined order, determine the authority with execution data processing task, this avoid all execute servers all
Go to execute data processing task, cause the situation that data processing resources waste.
As shown in Figure 10, in one embodiment, data task acquisition module 610 specifically also includes：Task requests send
Module 610a data task receiver module 610b.
Task requests sending module 610a, obtains for sending the task of carrying execute server mark to Distributor
Take request.
Data task receiver module 610b, for receiving data process task, data processing task is Distributor root
According to execute server ranking results to execute server identity verification by rear return.
In the present embodiment, when Distributor receives task and obtains request, according to execute server ranking results pair
The execute server mark that task obtains in request is verified, to ensure the data safety in Distributor, need not be extra
Setting checking information.
Each technical characteristic of embodiment described above can arbitrarily be combined, for making description succinct, not to above-mentioned reality
The all possible combination of each technical characteristic applied in example is all described, as long as however, the combination of these technical characteristics is not deposited
In contradiction, all it is considered to be the scope of this specification record.
Embodiment described above only have expressed the several embodiments of the present invention, and its description is more concrete and detailed, but simultaneously
Can not therefore be construed as limiting the scope of the patent.It should be pointed out that coming for those of ordinary skill in the art
Say, without departing from the inventive concept of the premise, some deformation can also be made and improve, these broadly fall into the protection of the present invention
Scope.Therefore, the protection domain of patent of the present invention should be defined by claims.
Claims (10)
1. a kind of data processing task executes method, is applied to execute server it is characterised in that methods described includes：
Receive the task notice distribution that Distributor sends；
Tasks carrying request is sent to sequence server according to described task notice distribution；
Obtain execute server ranking results from described sequence server, described execute server ranking results are by described sequence clothes
Business device, after receiving the tasks carrying request that each execute server sends, asks to generate according to described tasks carrying；
Determine sorting position in described server orders result for the local execute server, whether judge described sorting position
Within the scope of predetermined order；
When judging that described sorting position is within the scope of predetermined order, obtain described data processing from described Distributor and appoint
It is engaged in and execute.
2. method according to claim 1 it is characterised in that described according to described task notice distribution to sequence server
After sending tasks carrying request, also include：
Receive the ballot request that described sequence server is initiated, described ballot request is for choosing holding of execution data processing task
Row server；
Vote information is generated according to described ballot request；
Described vote information is sent to described sequence server, makes described sequence server according to each execution service receiving
The vote information that device returns generates execute server ranking results.
3. method according to claim 1 it is characterised in that described according to described task notice distribution to sequence server
After sending tasks carrying request, also include：
Receive described sequence server and send resource utilization acquisition request；
Request is obtained according to described resource utilization and extracts resource utilization, the resource utilization extracted is back to described row
Sequence server, makes described sequence server be generated according to the resource utilization that each execute server receiving returns and executes service
Device ranking results.
4. method according to claim 1 is it is characterised in that the local execute server of described determination is in described server
Sorting position in ranking results, judges described sorting position whether within the scope of predetermined order, including：
Extract the execute server mark coming first in execute server ranking results；
The execute server extracted mark is compared with self identification；
If the described execute server mark extracted is consistent with described self identification, judge the row of local execute server
Tagmeme is put within the scope of predetermined order.
5. method according to claim 1 is it is characterised in that described obtain described data processing from described Distributor
Task simultaneously executes, including：
Send the task acquisition request carrying execute server mark to described Distributor；
Receiving data process task, described data processing task is described Distributor according to execute server ranking results pair
Described execute server identity verification is by rear return.
6. a kind of data processing task performs device, is applied to execute server it is characterised in that described device includes：
Notice distribution receiver module, for receiving the task notice distribution of Distributor transmission；
Execution request sending module, for sending tasks carrying request according to described task notice distribution to sequence server；
Ranking results acquisition module, for obtaining execute server ranking results, described execution service from described sequence server
Device ranking results by described sequence server after receiving the tasks carrying request that each execute server sends, according to described
Business execution request generates；
Sorting position judge module, for determining sequence position in described server orders result for the local execute server
Put, judge described sorting position whether within the scope of predetermined order；
Data task acquisition module, for when judging that described sorting position is within the scope of predetermined order, from described distribution clothes
Business device obtains described data processing task and executes.
7. device according to claim 6 is it is characterised in that described device also includes：
Ballot request receiving module, for receiving the ballot request that described sequence server is initiated, described ballot request is used for selecting
Take the execute server of execution data processing task；
Vote information generation module, for generating vote information according to described ballot request；
Vote information sending module, for sending described vote information to described sequence server, makes described sequence server
The vote information that each execute server according to receiving returns generates execute server ranking results.
8. device according to claim 6 is it is characterised in that described device also includes：
Obtain request receiving module, send resource utilization acquisition request for receiving described sequence server；
Resource utilization returns module, extracts resource utilization for obtaining request according to described resource utilization, will extract
Resource utilization be back to described sequence server, make described sequence server according to each execute server return receiving
Resource utilization generate execute server ranking results.
9. device according to claim 6 is it is characterised in that described sorting position judge module includes：
Marker extraction module, for extracting the execute server mark coming first in execute server ranking results；
Mark comparing module, for comparing the execute server extracted mark with self identification；
Sequence determining module, if identify consistent with described self identification for the described execute server extracted, judges this
The sorting position of the execute server on ground is within the scope of predetermined order.
10. device according to claim 6 is it is characterised in that described data task acquisition module includes：
Task requests sending module, please for sending the task acquisition carrying execute server mark to described Distributor
Ask；
Data task receiver module, for receiving data process task, described data processing task is described Distributor root
According to execute server ranking results to described execute server identity verification by rear return.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610702453.8A CN106407002B (en) | 2016-08-22 | 2016-08-22 | Data processing task executes method and apparatus |
| PCT/CN2017/079649 WO2018036168A1 (en) | 2016-08-22 | 2017-04-07 | Method and device for executing data processing task, execution server, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610702453.8A CN106407002B (en) | 2016-08-22 | 2016-08-22 | Data processing task executes method and apparatus |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106407002A true CN106407002A (en) | 2017-02-15 |
| CN106407002B CN106407002B (en) | 2018-07-13 |
Family
ID=58004231
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610702453.8A Active CN106407002B (en) | 2016-08-22 | 2016-08-22 | Data processing task executes method and apparatus |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106407002B (en) |
| WO (1) | WO2018036168A1 (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018036168A1 (en) * | 2016-08-22 | 2018-03-01 | 平安科技（深圳）有限公司 | Method and device for executing data processing task, execution server, and storage medium |
| CN109582451A (en) * | 2018-11-21 | 2019-04-05 | 金色熊猫有限公司 | Method for scheduling task, system, equipment and readable medium |
| CN109582723A (en) * | 2018-11-30 | 2019-04-05 | 深圳市思迪信息技术股份有限公司 | Distributed ETL collecting method and device |
| CN110009320A (en) * | 2018-01-04 | 2019-07-12 | 财付通支付科技有限公司 | Resource conversion method, device, system, storage medium and computer equipment |
| CN110888925A (en) * | 2019-10-11 | 2020-03-17 | 广州大气候农业科技有限公司 | Data loading and distributing method and device and storage medium |
| CN113452753A (en) * | 2021-05-21 | 2021-09-28 | 北京奇艺世纪科技有限公司 | Data distribution method, device, equipment and storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113901008B (en) * | 2021-11-10 | 2022-09-27 | 上海意略明数字科技股份有限公司 | Data processing method and device, storage medium and computing equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104023042A (en) * | 2013-03-01 | 2014-09-03 | 清华大学 | Cloud platform resource scheduling method |
| CN104618492A (en) * | 2015-02-11 | 2015-05-13 | 北京金和软件股份有限公司 | Data processing system based on multiple servers |
| CN105100280A (en) * | 2015-08-31 | 2015-11-25 | 广州酷狗计算机科技有限公司 | Task allocation method and device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101132393B (en) * | 2006-08-25 | 2010-11-17 | 郑州威科姆科技股份有限公司 | Stream media service dynamic load method |
| US8224825B2 (en) * | 2010-05-31 | 2012-07-17 | Microsoft Corporation | Graph-processing techniques for a MapReduce engine |
| CN103580922B (en) * | 2013-11-08 | 2016-08-17 | 国家电网公司 | A kind of powerline network application server overload controlling method based on hybrid optimization |
| CN104038540B (en) * | 2014-06-03 | 2017-03-22 | 山东乾云启创信息科技股份有限公司 | Method and system for automatically selecting application proxy server |
| CN106407002B (en) * | 2016-08-22 | 2018-07-13 | 平安科技（深圳）有限公司 | Data processing task executes method and apparatus |
-
2016
- 2016-08-22 CN CN201610702453.8A patent/CN106407002B/en active Active
-
2017
- 2017-04-07 WO PCT/CN2017/079649 patent/WO2018036168A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104023042A (en) * | 2013-03-01 | 2014-09-03 | 清华大学 | Cloud platform resource scheduling method |
| CN104618492A (en) * | 2015-02-11 | 2015-05-13 | 北京金和软件股份有限公司 | Data processing system based on multiple servers |
| CN105100280A (en) * | 2015-08-31 | 2015-11-25 | 广州酷狗计算机科技有限公司 | Task allocation method and device |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018036168A1 (en) * | 2016-08-22 | 2018-03-01 | 平安科技（深圳）有限公司 | Method and device for executing data processing task, execution server, and storage medium |
| CN110009320A (en) * | 2018-01-04 | 2019-07-12 | 财付通支付科技有限公司 | Resource conversion method, device, system, storage medium and computer equipment |
| CN110009320B (en) * | 2018-01-04 | 2024-04-16 | 财付通支付科技有限公司 | Resource conversion method, device, system, storage medium and computer equipment |
| CN109582451A (en) * | 2018-11-21 | 2019-04-05 | 金色熊猫有限公司 | Method for scheduling task, system, equipment and readable medium |
| CN109582723A (en) * | 2018-11-30 | 2019-04-05 | 深圳市思迪信息技术股份有限公司 | Distributed ETL collecting method and device |
| CN109582723B (en) * | 2018-11-30 | 2021-08-17 | 深圳市思迪信息技术股份有限公司 | Distributed ETL data acquisition method and device |
| CN110888925A (en) * | 2019-10-11 | 2020-03-17 | 广州大气候农业科技有限公司 | Data loading and distributing method and device and storage medium |
| CN110888925B (en) * | 2019-10-11 | 2022-06-17 | 广州大气候农业科技有限公司 | Data loading and distributing method and device and storage medium |
| CN113452753A (en) * | 2021-05-21 | 2021-09-28 | 北京奇艺世纪科技有限公司 | Data distribution method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106407002B (en) | 2018-07-13 |
| WO2018036168A1 (en) | 2018-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106407002A (en) | Data processing task execution method and device | |
| CN113672937A (en) | Block chain link point | |
| US20170019407A1 (en) | Method and server for providing image captcha | |
| CN105338082A (en) | Load balancing method and load balancing device based on application proxy server | |
| CN108063685B (en) | Log analysis method and device | |
| CN105657677A (en) | Short message sending method, short message gateway and service platform | |
| CN116319777A (en) | Intelligent gateway service processing method based on edge calculation | |
| CN109067645B (en) | Network element equipment connected with NFV virtual security gateway | |
| CN110415044A (en) | Cheat detection method, device, equipment and storage medium | |
| CN111625342B (en) | Data tracing method, device and server | |
| CN109242391A (en) | A kind of cargo recognition methods and device | |
| CN115689656B (en) | Advertisement putting method, device, equipment and storage medium based on Internet of things | |
| CN114201701B (en) | Method and device for identifying operating environment, storage medium, server and client | |
| CN106649678B (en) | Data processing method and system | |
| EP3321879A1 (en) | Article inspection and examination system and method | |
| CN109784052B (en) | Management method for software behavior detection, server, terminal and system | |
| CN109412898A (en) | Characteristic library generating method and device and corresponding flow method for sorting and device | |
| CN111885607A (en) | Intelligent rapid network distribution method and system | |
| CN111030874A (en) | Service arranging method, equipment, storage medium and device | |
| CN114338089B (en) | Anti-attack method, device, equipment and computer readable storage medium | |
| CN117151878A (en) | Transaction processing method and device, electronic equipment and storage medium | |
| CN112905333B (en) | Computing power load scheduling method and device for distributed video intelligent analysis platform | |
| CN110674380B (en) | Data query method, system, readable storage medium and intelligent device | |
| CN116760624B (en) | Network worm detection method, system, storage medium and electronic equipment | |
| CN114071450B (en) | Recognition method and device for card changing behavior of machine changing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |