CN109408220A - A kind of task processing method and device - Google Patents
A kind of task processing method and device Download PDFInfo
- Publication number
- CN109408220A CN109408220A CN201710707452.7A CN201710707452A CN109408220A CN 109408220 A CN109408220 A CN 109408220A CN 201710707452 A CN201710707452 A CN 201710707452A CN 109408220 A CN109408220 A CN 109408220A
- Authority
- CN
- China
- Prior art keywords
- task
- cluster
- message queue
- task processing
- status
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/503—Resource availability
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/508—Monitor
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a kind of task processing method and devices, it is related to field of computer technology, main purpose is to realize the flexible configuration of cluster resource, guarantee the response speed of cluster, main technical schemes of the invention are as follows: the cluster of preset quantity is opened according to task amount to be processed in message queue, task is distributed to the cluster of unlatching according to task processing sequence in the message queue, label has processing status in the message queue；Cluster operating status is updated according to the task processing status in the cluster in task processes, the cluster operating status includes abnormality, normal condition and idle state；Close the cluster of the abnormality and the idle state.Present invention is mainly used for the processing of cluster task.
Description
Technical field
The present invention relates to field of computer technology more particularly to a kind of task processing methods and device.
Background technique
With the development of cloud computing technology, more and more IT service systems are gradually migrated to cloud platform, and building is based on cloud
The service of platform has become main way.So-called cloud platform, the system exactly constructed using cloud computing technology and thought
Platform provides the service platform of data calculating, data storage service and integrated special applications.
Currently, common cloud platform has Ali's cloud, Microsoft Azure, Amazon Web Services etc., each family
Multiple clusters can have been built in cloud platform, are included one group of server independent in each cluster, are passed through cluster operation energy
Enough improve the stability and data-handling capacity of cloud platform.And the Automatic dispatching function that each cloud platform provides can be reasonable
Handle each cluster scaling scene well, distribute reasonable resource for waiting task, by under different scenes to cluster resource
Reasonable disposition can be avoided and the idle situation with task processing capacity deficiency of cluster occurs.
However, Automatic dispatching function provided by existing cloud platform is when coping with different types of task, it can not be to collection
Group's resource carries out flexible configuration, so that cluster response speed in task processes is slow, in addition, when task processing appearance is abnormal
Shi Wufa normal switching-off cluster, leads to the wasting of resources.
Summary of the invention
In view of this, the present invention provides a kind of task processing method and device, main purpose is to realize cluster resource
Flexible configuration guarantees the response speed of cluster.
To solve the above-mentioned problems, present invention generally provides following technical solutions:
On the one hand, the embodiment of the invention provides a kind of task processing methods, comprising:
The cluster that preset quantity is opened according to task amount to be processed in message queue, according to task in the message queue
Processing sequence distributes task to the cluster of unlatching, and label has processing status in the message queue；
Cluster operating status, the cluster are updated according to the task processing status in the cluster in task processes
Operating status includes abnormality, normal condition and idle state；
Close the cluster of the abnormality and the idle state.
Further, the cluster that preset quantity is opened according to task amount to be processed in message queue, according to described
Task processing sequence includes: to the cluster of unlatching distribution task in message queue
The number of clusters for needing to open is determined according to task amount to be processed in message queue；
The cluster that preset quantity is opened according to the number of clusters for needing to open and cluster operating status, according to described
Task processing sequence distributes task to the cluster of unlatching in message queue.
Further, described that cluster operation is updated according to the task processing status in the cluster in task processes
State includes:
Task processing status in the message queue is obtained according to preset time interval, the task processing status includes appointing
It does honest work normal, task abnormity and free time；
In task processes when the task processing status is task abnormity, task is normal or task is completed,
The corresponding cluster operating status for updating task place cluster is abnormality, normal condition or idle state.
Further, after the cluster of the closing abnormality and the idle state, the method is also
Include:
It whether there is the task of untreated completion in cluster after detection closing；
If it exists, the task by the untreated completion is sent in the message queue, and is adjusted described untreated complete
At task processing sequence of the task in the message queue；
Task is distributed to cluster according to the task processing sequence in message queue adjusted.
Further, the task processing sequence according in message queue adjusted includes: to cluster distribution task
The task of the untreated completion is preferentially distributed to cluster according to task processing sequence in message queue adjusted.
Further, cluster fortune is updated according to the task processing status in the cluster in task processes described
After row state, the method also includes:
According to the active degree of the updated cluster operating status setting cluster；
According in the message queue task handle sequencing, according to the sequence of cluster active degree from high to low to
Cluster distributes task.
Further, after the cluster of the closing abnormality and the idle state, the method is also
Include:
After task amount to be processed in the message queue reaches preset threshold, adjusted according to the cluster operating status
Cluster opens quantity.
To achieve the goals above, according to another aspect of the present invention, a kind of storage medium, the storage medium are provided
Program including storage, wherein equipment where controlling the storage medium in described program operation executes described above appoint
Business processing method.
To achieve the goals above, according to another aspect of the present invention, a kind of processor is provided, the processor is used for
Run program, wherein described program executes task processing method described above when running.
On the other hand, the embodiment of the invention also provides a kind of Task Processing Units, comprising:
First allocation unit, for opening the cluster of preset quantity according to task amount to be processed in message queue, according to
Task processing sequence distributes task to the cluster of unlatching in the message queue, and label has processing shape in the message queue
State；
Updating unit, for updating cluster operation according to the task processing status in the cluster in task processes
State, the cluster operating status include abnormality, normal condition and idle state；
Closing unit, for closing the cluster of the abnormality and the idle state.
Further, first allocation unit includes:
Determining module, for determining the number of clusters for needing to open according to task amount to be processed in message queue；
Distribution module, for opening preset quantity according to the number of clusters for needing to open and cluster operating status
Cluster distributes task to the cluster of unlatching according to task processing sequence in the message queue.
Further, the updating unit includes:
Module is obtained, for obtaining task processing status in the message queue, the task according to preset time interval
Processing status includes normal task, task abnormity and free time；
Update module, in task processes when the task processing status be task abnormity, task it is normal or
When person's task is completed, the corresponding cluster operating status for updating task place cluster is abnormality, normal condition or sky
Not busy state.
Further, described device further include:
Detection unit, for detecting whether there is untreated completion in the cluster after closing of the task；
Transmission unit will be described untreated complete if there are the tasks of untreated completion in the cluster after closing for detecting
At task be sent in the message queue, and adjust task of the task of the untreated completion in the message queue
Processing sequence；
Second allocation unit, for distributing task to cluster according to the task processing sequence in message queue adjusted.
Further, second allocation unit is specifically used for according to task processing sequence in message queue adjusted
The task of the untreated completion is preferentially distributed to cluster.
Further, described device further include:
Setting unit, for the active degree according to the updated cluster operating status setting cluster；
Third allocation unit, the sequencing for being handled according to task in the message queue, enlivens journey according to cluster
The sequence of degree from high to low distributes task to cluster.
Further, described device further include:
Adjustment unit, for after task amount to be processed in the message queue reaches preset threshold, according to the collection
Group's operating status adjustment cluster opens quantity.
By above-mentioned technical proposal, technical solution provided in an embodiment of the present invention is at least had the advantage that
A kind of task processing method and device provided in an embodiment of the present invention, by when cluster is operated normally according to message
Task amount to be processed opens the cluster of preset quantity in queue, and the reasonable operation quantity for planning cluster ensure that at task
The wasting of resources is avoided while managing efficiency, while cluster is updated according to task processing status in task processes and runs shape
State, cluster operating status here includes abnormality, normal condition and idle state, and is transported according to updated cluster
Row state closes the cluster of abnormality and idle state, closes the corresponding cluster of idle task in time.With the prior art
Task processing method is compared, and the embodiment of the present invention can close the corresponding cluster of abnormal task when task occurs abnormal in time,
And cluster operating status is updated, each cluster is allowed flexibly to define the operating status of oneself, avoids invalid cluster
Resource occupation.
The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention,
And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can
It is clearer and more comprehensible, the followings are specific embodiments of the present invention.
Detailed description of the invention
By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field
Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention
Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:
Fig. 1 is a kind of task processing method flow chart provided in an embodiment of the present invention；
Fig. 2 is another task processing method flow chart provided in an embodiment of the present invention；
Fig. 3 is that the embodiment of the present invention determines the method flow diagram for opening number of clusters；
Fig. 4 is the method flow diagram that the embodiment of the present invention closes abnormal cluster；
Fig. 5 is the method flow diagram of task of embodiment of the present invention distribution；
Fig. 6 is a kind of composition block diagram of Task Processing Unit provided in an embodiment of the present invention；
Fig. 7 is the composition block diagram of another Task Processing Unit provided in an embodiment of the present invention.
Specific embodiment
The exemplary embodiment that the present invention will be described in more detail below with reference to accompanying drawings.Although showing the present invention in attached drawing
Exemplary embodiment, it being understood, however, that may be realized in various forms the present invention without should be by embodiments set forth here
It is limited.It is to be able to thoroughly understand the present invention on the contrary, providing these embodiments, and can be by the scope of the present invention
It is fully disclosed to those skilled in the art.
The embodiment of the invention provides a kind of task processing methods, as shown in Figure 1, this method goes out in task processes
When existing abnormal task, can timely update cluster operating status, and adjust the task in message queue according to cluster operating status
Processing sequence ensure that task treatment effeciency, provide step in detail below to this embodiment of the present invention:
101, the cluster that preset quantity is opened according to task amount to be processed in message queue, according in the message queue
Task processing sequence distributes task to the cluster of unlatching.
Wherein, message queue be in the transmission process of message save processing task container, and in message queue in
Be marked with task processing status, such as task handle successfully, in task processing or task processing failure information, processing here
Task can calculate task or crawler task dispatching etc. for monitor task, real-time traffic, in addition, collecting as required for different task
Realm type and number of clusters are different, right for needing the higher task of calculating speed that parallel computing trunking can be used
In needing the biggish task of calculation amount that High Performance Computing Cluster can be used, if certainly in message queue waiting task compared with
It is more, then need to open suitable number of cluster, the embodiment of the present invention is to group type and number of clusters without limiting.
It should be noted that may currently have been switched on several clusters before the cluster for opening preset quantity, need
Previously-scanned cluster operating status, if current 10 clusters of scanning are in the open state, if according in message queue wait locate
The number of clusters for the suitable unlatching that the task amount of reason is calculated is 20, then preset quantity is 10, it is only necessary on this basis again
10 clusters are opened, if certainly in the open state without any cluster at this time, preset quantity 20 needs to open
20 clusters.Therefore, the preset quantity of the embodiment of the present invention also needs to be determined according to current cluster operating status.
After determining and needing to open the cluster of preset quantity, according to task processing sequence in message queue to the collection of unlatching
Group's distribution task, each task is corresponding with different clusters here, and the sortord of task processing sequence can be by pre- here
It first can also be by advance will according to sequence serial number to cluster distributed tasks by the task setting sequence serial number in message queue
Task in message queue is according to different task byte orderings, by different task byte orderings result to cluster distributed tasks, originally
Inventive embodiments are to the method for determination of task processing sequence without limiting.
The embodiment of the present invention is opened when cluster is operated normally according to task amount to be processed in message queue
An appropriate number of cluster opens a small amount of cluster if waiting task is less, will not result in waste of resources in this way, if
When task amount to be processed increases, then appropriate number of cluster is opened to handle task according to corresponding proportion, ensure that cluster is appointed
Business treatment effeciency.
102, cluster operating status is updated according to the task processing status in the cluster in task processes.
When cluster to task amount to be processed in the process of processing, can at set time intervals from message queue
Middle acquisition mission bit stream, and the task processing status marked in message queue also can with the processing of task real-time change, such as
Fruit task is being handled, then task processing status is that task is normal, is finished if task is processed, and task processing status is to appoint
Business is completed, if task processing is abnormal, task processing status is task abnormity.
Different task processing statuses can show the different operating status of cluster in cluster, and cluster operating status here can
To include normal condition, abnormality and idle state, if the task processing status in cluster is that task is normal, illustrate to collect
Group is currently running, then updating cluster operating status is normal condition；If the task processing status in cluster is task completion, say
Bright cluster has handled task, then updating cluster operating status is idle state；If the task processing status in cluster is
It is abnormal to illustrate that cluster occurs in task processes for task abnormity, then updating cluster operating status is abnormality.
It should be noted that since cluster is usually that multiple machines are operated together and handle current task, if cluster is transported
Row state is normal condition, then illustrates that the task processing status of all machines in cluster is that task is normal, if cluster runs shape
State is abnormality, then explanation is a certain machine or multiple machine tasks processing failures in cluster, can be waited in cluster at this time
All machines all complete handling at present for task to update cluster operating status again to be abnormality, if cluster operating status is sky
Not busy state then illustrates the task processing status of all machines in cluster for task completion.
103, the cluster of the abnormality and the idle state is closed.
Cluster operating status can be updated to abnormality in task processing failure by cluster, due to the cluster of abnormality
It is unable to operate normally or system setting is unable to operate normally, then the cluster of abnormality is further turned off, to avoid excessive
Cluster resource occupies.Similarly, the cluster of idle state shows that distributing to the cluster for task has been disposed, and opens without keeping
State is opened to occupy excessive cluster resource, is further turned off the cluster of idle state.
For the embodiment of the present invention provide a kind of task processing method, by when cluster is operated normally according to message
Task amount to be processed opens the cluster of preset quantity in queue, and the reasonable operation quantity for planning cluster ensure that at task
The wasting of resources is avoided while managing efficiency, while cluster is updated according to task processing status in task processes and runs shape
State, cluster operating status here includes abnormality, normal condition and idle state, and is transported according to updated cluster
Row state closes the cluster of abnormality and idle state, closes the corresponding cluster of idle task in time.With the prior art
Task processing method is compared, and the embodiment of the present invention can close the corresponding cluster of abnormal task when task occurs abnormal in time,
And cluster operating status is updated, each cluster is allowed flexibly to define the operating status of oneself, avoids invalid cluster
Resource occupation.
It is especially processed in task below in order to which a kind of task processing method proposed by the present invention is explained in more detail
How Cheng Zhong is handled untreated in abnormality cluster and the abnormality cluster when cluster operating status is abnormality
The task of completion, the embodiment of the invention also provides another task processing methods, as shown in Fig. 2, the specific step of this method
Suddenly include:
201, the number of clusters for needing to open is determined according to task amount to be processed in message queue.
For the embodiment of the present invention, after task is sent in message queue, message queue is acted as an intermediary, by task
It is assigned to cluster from message queue, and then guarantees the reliability of task transmitting.Further appointed according to be processed in message queue
Business amount determines the number of clusters for needing to open, and completes for example, the task amount in message queue needs to open 20 clusters 5 hours,
It needs to open 10 clusters 10 hours and complete, it is larger due to opening cluster occupancy resource consumption, not open the cluster more than quantity
Completing speed, The faster the better, and task completion time opens appropriate number of cluster to usual situation according to actual needs.
By determining the number of clusters for needing to open according to task amount to be processed in message queue, can make rational planning for
Task through planning and timely task, it is ensured that cluster completes the waiting task planned in message queue in planned time
Amount, guarantees the response speed of cluster.
202, the cluster that preset quantity is opened according to the number of clusters for needing to open and cluster operating status, according to
Task processing sequence distributes task to the cluster of unlatching in the message queue.
Since the current operating status of different clusters may be different, some clusters are in idle condition, some clusters
Task may be being handled, therefore the current operation of cluster is obtained by scanning cluster operating status before carrying out task distribution
State, the number of clusters combination cluster operating status further opened as needed opens the cluster of preset quantity, specific to flow
Journey schematic diagram can be as shown in figure 3, scanning message queue first judges whether there is waiting task, if it is according to message
Waiting task determines the number of clusters for needing to open in queue, further scans cluster operating status to determine whether existing empty
The cluster of not busy state, the cluster and cluster operating status of opening quantity as needed open the cluster of preset quantity.For example, such as
It is 20 that fruit scanning cluster operating status, which gets the number of clusters for currently having 10 clusters in operating status, and needing to open,
Platform, then preset quantity is 10, so that the number of clusters for being guaranteed at operating status is 20, if scanning cluster operating status
Get in operating status currently without cluster, and needing the number of clusters opened is 20, then preset quantity is 20.
For the embodiment of the present invention, if there is currently the cluster of several operating statuses, in the collection of corresponding operating status
On the basis of group, it is then turned on an appropriate number of cluster, if certainly current all clusters are all in idle state, directly opening is needed
The cluster of quantity is opened, further distributes task to the cluster of unlatching according to task processing sequence in message queue.
203, task processing status in the message queue is obtained according to preset time interval.
In the process of processing, message queue can be distributed periodically to cluster task amount to be processed in message queue
Task, processing task here can calculate task or crawler task dispatching etc. for monitor task, real-time traffic, and in task
Real-time update task processing status in treatment process, such as task handle successfully, task processing in or task processing failure letter
Task processing status can be summarized as normal task, task abnormity and task under normal conditions and completed, if task is just by breath
In processes, then task processing status is that task is normal, is finished if task is processed, and task processing status is that task is complete
At if be abnormal in task processes, task processing status is task abnormity.Further according to preset time interval
Task processing status in message queue is obtained, preset time interval here can be half an hour, 1 hour etc., and the present invention is implemented
Example is without limiting.
It should be noted that task handled by each machine is different due to including several machines in cluster,
The task processing status of machine each in cluster can also be stored in task processing status table by the embodiment of the present invention, thus side
Just the state of each machine current task processing of cluster monitoring.
For the embodiment of the present invention, by obtaining in message queue according to preset time interval, task processing status can be real-time
The processing progress of understanding task can reasonably plan the task processing sequence in message queue when to prevent being abnormal situation,
Guarantee the treatment effeciency of cluster.
204, when the task processing status is task abnormity, task is normal or task is completed in task processes
When, the corresponding cluster operating status for updating task place cluster is abnormality, normal condition or idle state.
It is since cluster is likely to occur abnormal or error, the task processing status got at this time in task processes
Task abnormity further updates cluster operating status according to task processing status in cluster is corresponding, more by the cluster operating status
Newly it is abnormality, shows to encounter problems during cluster processing task, wait and closing, it should be noted that here different
It often may include two kinds of situations, one is can restore abnormal, the problems such as such as network delay, task assignment latency, for that can restore
Cluster can voluntarily restore to operate normally the abnormal cluster usually after waiting some time；Another kind is irrecoverable exception, such as
The unusual condition etc. that network bandwidth exception, cluster hardware problem or system define, is waiting the irrecoverable exception cluster
It being still unable to operate normally after some time, it is generally the case that abnormality here refers to irrecoverable abnormality,
Therefore after irrecoverable abnormal conditions occur, it is different that task processing status, which will be updated the operating status for abnormal cluster and will be updated,
Normal state.
For example, Baidu can to there is " honey jar attack " in the ranking of search result, after repeatedly access Baidu search result, hundred
The degree meeting incoherent data of returning part pretend to be ranking, and in this case, system customized can define Baidu crawler and access
More new engine is abnormality when Baidu's amount reaches certain numerical value, and machine can wait closing after update abnormal state, so both
Resource is saved, the honey jar attack data for also avoiding Baidu lead to the pollution of system data.
Correspondingly, getting the task processing status in cluster is that task is normal, corresponding if task present
Update cluster operating status is normal condition, is finished if task is processed, the task processing status got in cluster is
Task is completed, and the corresponding cluster operating status that updates is idle state.
It should be noted that since cluster is usually that multiple machines are operated together and handle current task, if cluster is transported
Row state is normal condition, then illustrates that the task processing status of all machines in cluster is that task is normal, if cluster runs shape
State is abnormality, then explanation is a certain machine or multiple machine tasks processing failures in cluster, can be waited in cluster at this time
All machines all complete handling at present for task to update cluster operating status again to be abnormality, if cluster operating status is sky
Not busy state then illustrates the task processing status of all machines in cluster for task completion.
The prior art occur abnormal task cluster would generally stop task processing, and it is in the open state wait it is subsequent
Processing, and the cluster operating status can be updated to abnormality there is abnormal task cluster by the embodiment of the present invention, and be waited
Wait close, reduce cluster resource waste.
205, the cluster of the abnormality and the idle state is closed.
Cluster operating status can be updated to abnormality in task processing failure by cluster, due to the cluster of abnormality
It is unable to operate normally or system setting is unable to operate normally, then the cluster of abnormality is further turned off, to avoid excessive
Cluster resource occupies.Similarly, the cluster of idle state shows that distributing to the cluster for task has been disposed, and opens without keeping
State is opened to occupy excessive cluster resource, is further turned off the cluster of idle state, the specific stream for closing abnormality cluster
Journey schematic diagram can be as shown in figure 4, scanning machine Running Status Table first, judges the machine that whether there is abnormality in machine
Device, if it is, closing the machine of abnormality.
206, it whether there is the task of untreated completion in the cluster after detection is closed.
Since the cluster after closing is usually the exception encountered in task processes, while closing is waited, and closed
Cluster be likely present in message queue plan originally and complete and unfinished waiting task, the embodiment of the present invention pass through inspection
Whether there is untreated completion of the task is surveyed in the cluster after closing, and the task of untreated completion is handled, for original
The task that this plan is completed re-starts planning processing, ensure that the timeliness that task is completed.
207, if it exists, the task by the untreated completion is sent in the message queue, and is not located described in adjustment
Manage task processing sequence of completing for the task in the message queue.
There are the tasks of untreated completion in cluster after detection closing will examine to guarantee the timeliness of task processing
The measuring untreated completion of the task is sent in message queue, is left since the task of untreated completion belongs to abnormality cluster
The timeliness of getting off for task, the task is higher compared to the priority of other tasks in message queue, so as to adjust untreated complete
At task processing sequence of the task in message queue.
Usual situation will can not be located when adjusting task processing sequence of the task of untreated completion in message queue
The task that reason is completed comes the Head-of-line of message queue, and then the task of untreated completion is preferentially distributed to the cluster of unlatching,
Certainly sequence of the adjustment task in message queue can also be adapted to according to the task scheduling deadline, to guarantee task energy
Enough it is timely completed.For example, if task is inserted into the position of the head of the queue of message queue compared to having timed, out in the original plan by the task
It sets, if the task compared to having not timed out in the original plan, by task insertion message queue close to the position of head of the queue, the present invention is real
Example is applied without limiting.
208, task is distributed to cluster according to the task processing sequence in message queue adjusted.
Due in message queue adjusted compared to the task processing sequence in not adjusted message queue before, meeting
The corresponding processing sequence of task of untreated completion in the cluster of abnormality closing is shifted to an earlier date, so that priority processing is untreated complete
At legacy tasks.
It should be noted that the side being randomly assigned can be taken for the method for salary distribution of waiting task in message queue
Formula can also take the mode of fixed allocation, and the mode that can also be distributed according to cluster active degree, the embodiment of the present invention is to collection
The method of salary distribution of group is without limiting.
209, according to the active degree of the updated cluster operating status setting cluster.
Cluster can define different active time in normal operating condition under normal conditions, such as handle reptile class and appoint
The active time of cluster can be defined on the period after receiving crawler generic task in message queue by the cluster of business, and processing is searched
The active time of cluster can be defined in message queue by the cluster that index holds up generic task receives search engine times generic task
Period afterwards, the fast speed of the cluster processing task in active time, cluster fast response time, the embodiment of the present invention are logical
The active degree that updated cluster operating status obtains cluster is crossed, the processing capacity of cluster can be understood, in real time so as to basis
The different distribution for adjusting task in message queue according to the cluster of different disposal ability in real time.
210, the sequencing handled according to task in the message queue, according to cluster active degree from high to low suitable
Sequence distributes task to cluster.
Under normal conditions, since the high cluster task processing speed of active degree is fast, message queue can preferentially distribute task
Handled to the high cluster of active degree, and active degree it is low cluster task processing can it is slow, message queue can subtract in right amount
Under absorbed task amount rationally utilizes cluster resource.
In addition, due in message queue there are the task of untreated completion in abnormality cluster, and it is untreated complete
At task usually come the head of the queue of message queue or close to the position of head of the queue, therefore can be preferential by the task of untreated completion
The higher cluster of active degree is distributed to, thus the legacy tasks in priority processing abnormality cluster, to guarantee legacy tasks
There are enough response speeds.Specific task assignment procedure can be with as shown in figure 5, obtain the active degree of cluster first, judgement
Whether cluster is in active time, if it is, preferentially task is obtained from message queue, if it is not, then reducing from Message Queuing
Acquisition task is further handled task after cluster gets task, and the task in update message queue handles shape
State.
211, after task amount to be processed in the message queue reaches preset threshold, according to the cluster operating status
It adjusts cluster and opens quantity.
It should be noted that new task can be also constantly received in message queue during cluster processing task,
And with the continuous accumulation of task amount in message queue, the quantity for having turned on cluster has been unable to satisfy the real-time processing of task, needs
The quantity of cluster unlatching is accordingly adjusted according to cluster operating status.
Processing for a small amount of task, normally-open cluster can maintain the real-time processing of task in message queue, when
After waiting task amount reaches preset threshold in message queue, normally-open number of clusters not can guarantee task completion time,
The cluster of appropriate idle state is further opened according to task amount ratio according to cluster operating status, is appointed with meeting in message queue
The timeliness of business processing.
It can be closed in time for another task processing method provided in an embodiment of the present invention when task occurs abnormal
The corresponding cluster of abnormal task is closed, and updates cluster operating status, each cluster is allowed flexibly to define the operation of oneself
State avoids the resource occupation of invalid cluster, and task untreated in abnormality cluster is further recovered to message team
In column, preferentially unfinished task is distributed to the cluster of normal condition, guarantees task treatment effeciency, is disappeared by reasonably arranging
The ceasing the task in queue and the untreated completion of recycling of the task, guarantees the response speed of cluster, in addition, due in active
The fast speed of the cluster processing task of time, cluster fast response time, by the way that the active degree of cluster is arranged, preferentially by message
Task to be processed distributes to the high cluster of active degree in queue, improves the treatment effeciency of task.
To achieve the goals above, according to another aspect of the present invention, the embodiment of the invention also provides a kind of storage Jie
Matter, the storage medium include the program of storage, wherein equipment where controlling the storage medium in described program operation is held
Row task processing method described above.
To achieve the goals above, according to another aspect of the present invention, the embodiment of the invention also provides a kind of processor,
The processor is for running program, wherein described program executes task processing method described above when running.
Further, as the realization to method shown in above-mentioned Fig. 1 and Fig. 2, another embodiment of the present invention additionally provides one
Kind Task Processing Unit.The Installation practice is corresponding with preceding method embodiment, and to be easy to read, present apparatus embodiment is no longer right
Detail content in preceding method embodiment is repeated one by one, it should be understood that the device in the present embodiment can correspond to reality
Full content in existing preceding method embodiment.The device guarantees the response of cluster for realizing the flexible configuration of cluster resource
Speed, specifically as shown in fig. 6, the device includes:
First allocation unit 31 can be used for opening the collection of preset quantity according to task amount to be processed in message queue
Group distributes task to the cluster of unlatching according to task processing sequence in the message queue, and marking in the message queue has
Business processing status；
Updating unit 32 can be used for updating collection according to the task processing status in the cluster in task processes
Group's operating status, the cluster operating status includes abnormality, normal condition and idle state；
Closing unit 33 can be used for closing the cluster of the abnormality and the idle state.
For the embodiment provides a kind of Task Processing Unit, by when cluster is operated normally according to message
Task amount to be processed opens the cluster of preset quantity in queue, and the reasonable operation quantity for planning cluster ensure that at task
The wasting of resources is avoided while managing efficiency, while cluster is updated according to task processing status in task processes and runs shape
State, cluster operating status here includes abnormality, normal condition and idle state, and is transported according to updated cluster
Row state closes the cluster of abnormality and idle state, closes the corresponding cluster of idle task in time.With the prior art
Task processing method is compared, and the embodiment of the present invention can close the corresponding cluster of abnormal task when task occurs abnormal in time,
And cluster operating status is updated, each cluster is allowed flexibly to define the operating status of oneself, avoids invalid cluster
Resource occupation.
Further, as shown in fig. 7, described device further include:
Detection unit 34 can be used for detecting whether there is untreated completion in the cluster after closing of the task；
Transmission unit 35, if can be used for detecting in the cluster after closing, there are the tasks of untreated completion, by described in not
The task that processing is completed is sent in the message queue, and adjusts the task of the untreated completion in the message queue
Task processing sequence；
Second allocation unit 36 can be used for distributing according to the task processing sequence in message queue adjusted to cluster
Task；
Setting unit 37 can be used for the active degree according to the updated cluster operating status setting cluster；
Third allocation unit 38 can be used for the sequencing handled according to task in the message queue, according to cluster
The sequence of active degree from high to low distributes task to cluster；
Adjustment unit 39 can be used for after task amount to be processed in the message queue reaches preset threshold, according to
The cluster operating status adjustment cluster opens quantity.
Further, second allocation unit 36 specifically can be used for according in message queue adjusted at task
Make the task that sequence preferentially distributes the untreated completion to cluster in order.
Further, first allocation unit 31 includes:
Determining module 311 can be used for determining the cluster number for needing to open according to task amount to be processed in message queue
Amount；
Processing module 312 can be used for being opened according to the number of clusters for needing to open and cluster operating status pre-
If the cluster of quantity, task is distributed to the cluster of unlatching according to task processing sequence in the message queue.
Further, the updating unit 32 includes:
Module 321 is obtained, can be used for obtaining task processing status in the message queue, institute according to preset time interval
The task processing status of stating includes normal task, task abnormity and free time；
Update module 322 can be used in task processes when the task processing status is task abnormity, task
When normal or task is completed, the corresponding cluster operating status for updating task place cluster is abnormality, normal condition
Or idle state.
It can be closed in time for another Task Processing Unit provided in an embodiment of the present invention when task occurs abnormal
The corresponding cluster of abnormal task is closed, and updates cluster operating status, each cluster is allowed flexibly to define the operation of oneself
State avoids the resource occupation of invalid cluster, and task untreated in abnormality cluster is further recovered to message team
In column, preferentially unfinished task is distributed to the cluster of normal condition, guarantees task treatment effeciency, is disappeared by reasonably arranging
The ceasing the task in queue and the untreated completion of recycling of the task, guarantees the response speed of cluster, in addition, due in active
The fast speed of the cluster processing task of time, cluster fast response time, by the way that the active degree of cluster is arranged, preferentially by message
Task to be processed distributes to the high cluster of active degree in queue, improves the treatment effeciency of task.
The Task Processing Unit includes processor and memory, above-mentioned first allocation unit 31, updating unit 32 and pass
It closes unit 33 etc. to store in memory as program unit, above procedure list stored in memory is executed by processor
Member realizes corresponding function.
Include kernel in processor, is gone in memory to transfer corresponding program unit by kernel.Kernel can be set one
Or more, the flexible configuration of cluster resource is realized by adjusting kernel parameter, guarantees the response speed of cluster.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/
Or the forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flashRAM), memory includes at least one storage
Chip.
The embodiment of the invention provides a kind of storage mediums, are stored thereon with program, real when which is executed by processor
The existing task processing method.
The embodiment of the invention provides a kind of processor, the processor is for running program, wherein described program operation
Task processing method described in Shi Zhihang.
The embodiment of the invention provides a kind of equipment, equipment include processor, memory and storage on a memory and can
The program run on a processor, processor perform the steps of when executing program
A kind of task processing method, comprising: the cluster of preset quantity is opened according to task amount to be processed in message queue,
Task is distributed to the cluster of unlatching according to task processing sequence in the message queue, label has place in the message queue
Reason state；Cluster operating status, the cluster are updated according to the task processing status in the cluster in task processes
Operating status includes abnormality, normal condition and idle state；Close the abnormality and the idle state
Cluster.
Further, the cluster that preset quantity is opened according to task amount to be processed in message queue, according to described
It includes: true according to task amount to be processed in message queue that task processing sequence, which distributes task to the cluster of unlatching, in message queue
The number of clusters for needing to open calmly；Preset quantity is opened according to the number of clusters for needing to open and cluster operating status
Cluster distributes task to the cluster of unlatching according to task processing sequence in the message queue.
Further, described that cluster operation is updated according to the task processing status in the cluster in task processes
State includes: to obtain task processing status in the message queue according to preset time interval；Work as institute in task processes
Stating task processing status is abnormality, normal condition or when idle state, the cluster fortune of cluster where updating the task
Row state is abnormality, normal condition or idle state.
Further, after the cluster of the closing abnormality and the idle state, the method is also
It include: whether there is untreated completion in the cluster after detection is closed of the task；If it exists, by the task of the untreated completion
It is sent in the message queue, and it is suitable to adjust task processing of the task of the untreated completion in the message queue
Sequence；Task is distributed to cluster according to the task processing sequence in message queue adjusted.
Further, the task processing sequence according in message queue adjusted includes: to cluster distribution task
The task of the untreated completion is preferentially distributed to cluster according to task processing sequence in message queue adjusted.
Further, cluster fortune is updated according to the task processing status in the cluster in task processes described
After row state, the method also includes: according to the active degree of the updated cluster operating status setting cluster；According to
The sequencing that task is handled in the message queue is appointed according to the sequence of cluster active degree from high to low to cluster distribution
Business.
Further, after the cluster of the closing abnormality and the idle state, the method is also
It include: to be adjusted and collected according to the cluster operating status after task amount to be processed in the message queue reaches preset threshold
Group opens quantity.
Equipment herein can be server, PC, PAD, mobile phone etc..
Present invention also provides a kind of computer program products, when executing on data processing equipment, are adapted for carrying out just
The program code of beginningization there are as below methods step: opening the cluster of preset quantity according to task amount to be processed in message queue,
Task is distributed to the cluster of unlatching according to task processing sequence in the message queue, label has place in the message queue
Reason state；Cluster operating status, the cluster are updated according to the task processing status in the cluster in task processes
Operating status includes abnormality, normal condition and idle state；Close the abnormality and the idle state
Cluster.
It should be understood by those skilled in the art that, embodiments herein can provide as method, system or computer program
Product.Therefore, complete hardware embodiment, complete software embodiment or reality combining software and hardware aspects can be used in the application
Apply the form of example.Moreover, it wherein includes the computer of computer usable program code that the application, which can be used in one or more,
The computer program implemented in usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) produces
The form of product.
The application is referring to method, the process of equipment (system) and computer program product according to the embodiment of the present application
Figure and/or block diagram describe.It should be understood that every one stream in flowchart and/or the block diagram can be realized by computer program instructions
The combination of process and/or box in journey and/or box and flowchart and/or the block diagram.It can provide these computer programs
Instruct the processor of general purpose computer, special purpose computer, Embedded Processor or other programmable data processing devices to produce
A raw machine, so that being generated by the instruction that computer or the processor of other programmable data processing devices execute for real
The device for the function of being specified in present one or more flows of the flowchart and/or one or more blocks of the block diagram.
These computer program instructions, which may also be stored in, is able to guide computer or other programmable data processing devices with spy
Determine in the computer-readable memory that mode works, so that it includes referring to that instruction stored in the computer readable memory, which generates,
Enable the manufacture of device, the command device realize in one box of one or more flows of the flowchart and/or block diagram or
The function of being specified in multiple boxes.
These computer program instructions also can be loaded onto a computer or other programmable data processing device, so that counting
Series of operation steps are executed on calculation machine or other programmable devices to generate computer implemented processing, thus in computer or
The instruction executed on other programmable devices is provided for realizing in one or more flows of the flowchart and/or block diagram one
The step of function of being specified in a box or multiple boxes.
In a typical configuration, calculating equipment includes one or more processors (CPU), input/output interface, net
Network interface and memory.
Memory may include the non-volatile memory in computer-readable medium, random access memory (RAM) and/
Or the forms such as Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is computer-readable Jie
The example of matter.
Computer-readable medium includes permanent and non-permanent, removable and non-removable media can be by any method
Or technology come realize information store.Information can be computer readable instructions, data structure, the module of program or other data.
The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), moves
State random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electric erasable
Programmable read only memory (EEPROM), flash memory or other memory techniques, read-only disc read only memory (CD-ROM) (CD-ROM),
Digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or other magnetic storage devices
Or any other non-transmission medium, can be used for storage can be accessed by a computing device information.As defined in this article, it calculates
Machine readable medium does not include temporary computer readable media (transitory media), such as the data-signal and carrier wave of modulation.
It should also be noted that, the terms "include", "comprise" or its any other variant are intended to nonexcludability
It include so that the process, method, commodity or the equipment that include a series of elements not only include those elements, but also to wrap
Include other elements that are not explicitly listed, or further include for this process, method, commodity or equipment intrinsic want
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including element
There is also other identical elements in process, method, commodity or equipment.
It will be understood by those skilled in the art that embodiments herein can provide as method, system or computer program product.
Therefore, complete hardware embodiment, complete software embodiment or embodiment combining software and hardware aspects can be used in the application
Form.It is deposited moreover, the application can be used to can be used in the computer that one or more wherein includes computer usable program code
The shape for the computer program product implemented on storage media (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.)
Formula.
The above is only embodiments herein, are not intended to limit this application.To those skilled in the art,
Various changes and changes are possible in this application.It is all within the spirit and principles of the present application made by any modification, equivalent replacement,
Improve etc., it should be included within the scope of the claims of this application.
Claims (10)
1. a kind of task processing method characterized by comprising
The cluster that preset quantity is opened according to task amount to be processed in message queue, is handled according to task in the message queue
Sequence distributes task to the cluster of unlatching, and label has processing status in the message queue；
Cluster operating status, the cluster operation are updated according to the task processing status in the cluster in task processes
State includes abnormality, normal condition and idle state；
Close the cluster of the abnormality and the idle state.
2. the method according to claim 1, wherein described open according to task amount to be processed in message queue
The cluster of preset quantity, distributing task to the cluster of unlatching according to task processing sequence in the message queue includes:
The number of clusters for needing to open is determined according to task amount to be processed in message queue；
The cluster that preset quantity is opened according to the number of clusters for needing to open and cluster operating status, according to the message
Task processing sequence distributes task to the cluster of unlatching in queue.
3. the method according to claim 1, wherein it is described in task processes according in the cluster
Task processing status updates cluster operating status
Obtain task processing status in the message queue according to preset time interval, the task processing status include task just
Often, task abnormity and free time；
It is corresponding to update in task processes when the task processing status is task abnormity, task is normal or idle
The cluster operating status of cluster where the task is abnormality, normal condition or idle state.
4. method according to any one of claim 1-3, which is characterized in that it is described close the abnormality and
After the cluster of the idle state, the method also includes:
It whether there is the task of untreated completion in cluster after detection closing；
If it exists, the task by the untreated completion is sent in the message queue, and adjusts the untreated completion
Task processing sequence of the task in the message queue；
Task is distributed to cluster according to the task processing sequence in message queue adjusted.
5. according to the method described in claim 4, it is characterized in that, the task according in message queue adjusted is handled
Sequence distributes task to cluster
The task of the untreated completion is preferentially distributed to cluster according to task processing sequence in message queue adjusted.
6. the method according to claim 1, wherein it is described in task processes according in the cluster
After task processing status updates cluster operating status, the method also includes:
According to the active degree of the updated cluster operating status setting cluster；
According to the sequencing that task in the message queue is handled, according to the sequence of cluster active degree from high to low to cluster
Distribution task.
7. the method according to claim 1, wherein closing the abnormality and the idle shape described
After the cluster of state, the method also includes:
After task amount to be processed in the message queue reaches preset threshold, cluster is adjusted according to the cluster operating status
Open quantity.
8. a kind of Task Processing Unit characterized by comprising
First allocation unit, for opening the cluster of preset quantity according to task amount to be processed in message queue, according to described
Task processing sequence distributes task to the cluster of unlatching in message queue, and label has processing status in the message queue；
Updating unit runs shape for updating cluster according to the task processing status in the cluster in task processes
State, the cluster operating status include abnormality, normal condition and idle state；
Closing unit, for closing the cluster of the abnormality and the idle state.
9. a kind of storage medium, which is characterized in that the storage medium includes the program of storage, wherein run in described program
When control the storage medium where equipment perform claim require 1 to the task processing side described in any one of claim 7
Method.
10. a kind of processor, which is characterized in that the processor is for running program, wherein right of execution when described program is run
Benefit requires 1 to the task processing method described in any one of claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710707452.7A CN109408220A (en) | 2017-08-17 | 2017-08-17 | A kind of task processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710707452.7A CN109408220A (en) | 2017-08-17 | 2017-08-17 | A kind of task processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109408220A true CN109408220A (en) | 2019-03-01 |
Family
ID=65455020
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710707452.7A Pending CN109408220A (en) | 2017-08-17 | 2017-08-17 | A kind of task processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109408220A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110928688A (en) * | 2019-11-29 | 2020-03-27 | 电子科技大学 | Distributed crawler task scheduling system and method |
| CN111813557A (en) * | 2020-07-21 | 2020-10-23 | 北京千丁互联科技有限公司 | Task processing device, method, terminal device and readable storage medium |
| CN111917576A (en) * | 2020-07-28 | 2020-11-10 | 星辰天合（北京）数据科技有限公司 | Control method and device for storage cluster |
| CN112346829A (en) * | 2019-08-07 | 2021-02-09 | 上海云盾信息技术有限公司 | Method and equipment for task scheduling |
| CN112671665A (en) * | 2020-12-16 | 2021-04-16 | 北京百度网讯科技有限公司 | Intelligent traffic scheduling method, device, equipment and storage medium |
| CN113495791A (en) * | 2021-09-07 | 2021-10-12 | 上海燧原科技有限公司 | Task processing system, method and chip |
| CN115118768A (en) * | 2022-06-27 | 2022-09-27 | 平安壹钱包电子商务有限公司 | Task distribution method and device, storage medium and electronic equipment |
| CN115866039A (en) * | 2022-11-29 | 2023-03-28 | 北京达佳互联信息技术有限公司 | Message processing method and device, electronic equipment and storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102073546A (en) * | 2010-12-13 | 2011-05-25 | 北京航空航天大学 | Task-dynamic dispatching method under distributed computation mode in cloud computing environment |
| CN102915254A (en) * | 2011-08-02 | 2013-02-06 | 中兴通讯股份有限公司 | Task management method and device |
| CN103761147A (en) * | 2014-01-15 | 2014-04-30 | 清华大学 | Method and system for managing calculation examples in cloud platforms |
| US20140143787A1 (en) * | 2010-08-30 | 2014-05-22 | Adobe Systems Incorporated | Methods and apparatus for resource management in cluster computing |
| CN104461740A (en) * | 2014-12-12 | 2015-03-25 | 国家电网公司 | Cross-domain colony computing resource gathering and distributing method |
| CN105975334A (en) * | 2016-04-25 | 2016-09-28 | 深圳市永兴元科技有限公司 | Distributed scheduling method and system of task |
| CN106227596A (en) * | 2016-07-13 | 2016-12-14 | 百度在线网络技术（北京）有限公司 | Mission Monitor method and apparatus for task scheduling server |
| CN106339260A (en) * | 2016-08-19 | 2017-01-18 | 北京小米移动软件有限公司 | Task allocation method and device for Jenkins platform |
| CN106375395A (en) * | 2016-08-30 | 2017-02-01 | 厦门中学西渐信息科技有限公司 | Load balancing method and system for node servers |
| CN106371924A (en) * | 2016-08-29 | 2017-02-01 | 东南大学 | Task scheduling method for maximizing MapReduce cluster energy consumption |
| CN107025139A (en) * | 2017-03-21 | 2017-08-08 | 北京天云融创软件技术有限公司 | A kind of high-performance calculation Scheduling Framework based on cloud computing |
-
2017
- 2017-08-17 CN CN201710707452.7A patent/CN109408220A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140143787A1 (en) * | 2010-08-30 | 2014-05-22 | Adobe Systems Incorporated | Methods and apparatus for resource management in cluster computing |
| CN102073546A (en) * | 2010-12-13 | 2011-05-25 | 北京航空航天大学 | Task-dynamic dispatching method under distributed computation mode in cloud computing environment |
| CN102915254A (en) * | 2011-08-02 | 2013-02-06 | 中兴通讯股份有限公司 | Task management method and device |
| CN103761147A (en) * | 2014-01-15 | 2014-04-30 | 清华大学 | Method and system for managing calculation examples in cloud platforms |
| CN104461740A (en) * | 2014-12-12 | 2015-03-25 | 国家电网公司 | Cross-domain colony computing resource gathering and distributing method |
| CN105975334A (en) * | 2016-04-25 | 2016-09-28 | 深圳市永兴元科技有限公司 | Distributed scheduling method and system of task |
| CN106227596A (en) * | 2016-07-13 | 2016-12-14 | 百度在线网络技术（北京）有限公司 | Mission Monitor method and apparatus for task scheduling server |
| CN106339260A (en) * | 2016-08-19 | 2017-01-18 | 北京小米移动软件有限公司 | Task allocation method and device for Jenkins platform |
| CN106371924A (en) * | 2016-08-29 | 2017-02-01 | 东南大学 | Task scheduling method for maximizing MapReduce cluster energy consumption |
| CN106375395A (en) * | 2016-08-30 | 2017-02-01 | 厦门中学西渐信息科技有限公司 | Load balancing method and system for node servers |
| CN107025139A (en) * | 2017-03-21 | 2017-08-08 | 北京天云融创软件技术有限公司 | A kind of high-performance calculation Scheduling Framework based on cloud computing |
Non-Patent Citations (2)
| Title |
|---|
| 姜秀柱主编，高璟，刘鹏副主编: "《计算机网络》", 31 July 2010, 中国矿业大学出版社 * |
| 谢用辉: "《保险IT系统建设》", 30 July 2016, 首都经济贸易大学出版社 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112346829A (en) * | 2019-08-07 | 2021-02-09 | 上海云盾信息技术有限公司 | Method and equipment for task scheduling |
| CN112346829B (en) * | 2019-08-07 | 2023-02-17 | 上海云盾信息技术有限公司 | Method and equipment for task scheduling |
| CN110928688A (en) * | 2019-11-29 | 2020-03-27 | 电子科技大学 | Distributed crawler task scheduling system and method |
| CN111813557A (en) * | 2020-07-21 | 2020-10-23 | 北京千丁互联科技有限公司 | Task processing device, method, terminal device and readable storage medium |
| CN111917576A (en) * | 2020-07-28 | 2020-11-10 | 星辰天合（北京）数据科技有限公司 | Control method and device for storage cluster |
| CN111917576B (en) * | 2020-07-28 | 2023-05-16 | 北京星辰天合科技股份有限公司 | Storage cluster control method and device, computer readable storage medium and processor |
| CN112671665A (en) * | 2020-12-16 | 2021-04-16 | 北京百度网讯科技有限公司 | Intelligent traffic scheduling method, device, equipment and storage medium |
| CN112671665B (en) * | 2020-12-16 | 2023-05-30 | 北京百度网讯科技有限公司 | Intelligent traffic scheduling method, device, equipment and storage medium |
| CN113495791A (en) * | 2021-09-07 | 2021-10-12 | 上海燧原科技有限公司 | Task processing system, method and chip |
| CN115118768A (en) * | 2022-06-27 | 2022-09-27 | 平安壹钱包电子商务有限公司 | Task distribution method and device, storage medium and electronic equipment |
| CN115866039A (en) * | 2022-11-29 | 2023-03-28 | 北京达佳互联信息技术有限公司 | Message processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109408220A (en) | A kind of task processing method and device | |
| US11720403B2 (en) | System for commitment-aware workload scheduling based on anticipated resource consumption levels | |
| CN110727512B (en) | Cluster resource scheduling method, device, equipment and storage medium | |
| EP3629165A1 (en) | Accelerated resource allocation techniques | |
| CN109416647B (en) | System and method for scheduling tasks and managing computing resource allocation for closed loop control systems | |
| US10303128B2 (en) | System and method for control and/or analytics of an industrial process | |
| CN112130963A (en) | Virtual machine task scheduling method and device, computer equipment and storage medium | |
| TW202219761A (en) | Apparatus and method for a closed-loop dynamic resource allocation control framework | |
| WO2020172852A1 (en) | Computing resource scheduling method, scheduler, internet of things system, and computer readable medium | |
| Rejiba et al. | Custom scheduling in kubernetes: A survey on common problems and solution approaches | |
| CN115543577B (en) | Covariate-based Kubernetes resource scheduling optimization method, storage medium and device | |
| CN104156255A (en) | Virtual machine migration method, virtual machine migration device and source physical host | |
| CN109376009A (en) | A kind of method and device of shared resource | |
| CN111026519A (en) | Distributed task priority scheduling method and system and storage medium | |
| CN107479944A (en) | Mix the adaptive thermophoresis dispatching method of virutal machine memory and system under cloud mode | |
| KR20220001418A (en) | Apparatus and method for a resource allocation control framework using performance markers | |
| US10423452B2 (en) | Allocating resources to virtual machines | |
| CN114610475A (en) | Training method of intelligent resource arrangement model | |
| CN104866375B (en) | A kind of method and device for migrating virtual machine | |
| Wang et al. | Real-time middleware for cyber-physical event processing | |
| US7552439B2 (en) | System and method to allow non-deterministic execution in a process control system | |
| US11782765B2 (en) | Method, device, and program product for managing computing system | |
| CN116841720A (en) | Resource allocation method, apparatus, computer device, storage medium and program product | |
| CN117093335A (en) | Task scheduling method and device for distributed storage system | |
| CN106027685A (en) | Peak access method based on cloud computation system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing Applicant after: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. Address before: 100086 Beijing city Haidian District Shuangyushu Area No. 76 Zhichun Road cuigongfandian 8 layer A Applicant before: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190301 |