CN109783228A - Task scheduling and managing method, device, computer system and readable storage system - Google Patents
Task scheduling and managing method, device, computer system and readable storage system Download PDFInfo
- Publication number
- CN109783228A CN109783228A CN201811535241.0A CN201811535241A CN109783228A CN 109783228 A CN109783228 A CN 109783228A CN 201811535241 A CN201811535241 A CN 201811535241A CN 109783228 A CN109783228 A CN 109783228A
- Authority
- CN
- China
- Prior art keywords
- task
- fragment
- gray scale
- production
- task data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003860 storage Methods 0.000 title claims abstract description 29
- 238000000034 method Methods 0.000 title claims abstract description 28
- 239000012634 fragment Substances 0.000 claims abstract description 177
- 238000004519 manufacturing process Methods 0.000 claims abstract description 135
- 238000004590 computer program Methods 0.000 claims description 7
- 230000001960 triggered effect Effects 0.000 claims description 5
- 230000001174 ascending effect Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims 1
- 238000002955 isolation Methods 0.000 abstract description 4
- 239000002956 ash Substances 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 235000002918 Fraxinus excelsior Nutrition 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000013467 fragmentation Methods 0.000 description 2
- 238000006062 fragmentation reaction Methods 0.000 description 2
- 239000010453 quartz Substances 0.000 description 2
- 230000000246 remedial effect Effects 0.000 description 2
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N silicon dioxide Inorganic materials O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of task scheduling and managing method, device, computer system and readable storage system, is based on internet O&M, comprising the following steps: found NameSpace；Mission bit stream is extracted and instantiated, task data is obtained；The task data is associated with the NameSpace；Fragment item is obtained to the task data fragment；Fragment item is exported to the service system；It receives the service system and executes the implementing result exported after fragment item, and according to the implementing result management role data and mission bit stream.The present invention is realized carries out production task scheduling and gray scale task schedule under same domain name, and it ensure that the isolation of production task data and gray scale task data, carry out gray scale publication for operator to provide convenience, expands the scope of application, meet a variety of fragment demands；And it not only ensure that the reliability of task schedule work, it is ensured that the whole efficiency of task schedule work operation.
Description
Technical field
The present invention relates to system O&M technical field more particularly to a kind of task scheduling and managing method, device, departments of computer science
System and readable storage system.
Background technique
Current Portable Batch System device carries out zonal control to server due to cannot achieve, in same domain name
Under cannot achieve gray scale publication, if desired carry out gray scale publication and need to be reset a domain name by operator then to carry out, this side
Formula is that operator causes big inconvenience when carrying out gray scale publication；
Simultaneously as current Portable Batch System device can not way to manage to mission bit stream it is single, therefore, it is impossible to
It monitors the executive condition of important mission bit stream, and then causes the reliability of Portable Batch System device not high.
Summary of the invention
The object of the present invention is to provide a kind of task scheduling and managing method, device, computer system and readable storage system,
It is of the existing technology for solving the problems, such as.
To achieve the above object, the present invention provides a kind of task scheduling and managing method, comprising the following steps:
S1: NameSpace is founded；
S2: extracting and instantiates mission bit stream, obtains task data；
S3: the task data is associated with the NameSpace；
S4: fragment item is obtained to the task data fragment；Fragment item is exported to service system；
S5: it receives the service system and executes the implementing result exported after fragment item, and according to the implementing result management
Task data and mission bit stream.
In above scheme, the NameSpace in the step S1 includes producing NameSpace and gray scale NameSpace, described
Mission bit stream in step S2 can be production task or gray scale task, and task data can be production task data in the step S3
Or gray scale task data；Fragment item in the step S4 includes production fragment item and gray scale fragment item.
In above scheme, the step S2 includes extracting mission bit stream from service system, utilizes client data packet example
Change mission bit stream and obtains task data；The task data includes task configuration information, task address information and task execution letter
Breath；Wherein, production task is extracted from production subsystem, gray scale task is extracted from gray scale subsystem；
The step S3 includes being associated with production task data with production NameSpace, and gray scale task data is dragged out a miserable existence with ash
Name space association, production task data and gray scale task data are isolated；
In above scheme, the step S4 includes: to obtain fragment item to task data fragment, and fragment item is exported to service
The server of system；
It is described to include: to production task data or gray scale task data fragment
According to fragment number to production task data or gray scale task data fragment；By fragment number and production subsystem or gray scale
The sum of the Servers-all of subsystem is divided by；If division result does not have remainder, using the quotient of division result as fragment number；
If division result has remainder, using the quotient of division result as fragment number, using the remainder of division result as redundancy collection number；Or
According to fragment number to production task data or gray scale task data fragment；To production task data or gray scale number of tasks
According to task name carry out Hash operation and obtaining cryptographic Hash；Or
The service code for extracting the TU task unit of production task or gray scale task forms enumerated value to service code assignment,
According to enumerated value to production task or gray scale task fragment.
In above scheme, fragment item or the gray scale fragment item of producing is exported to production subsystem or gray scale subsystem
Server includes:
If the division result does not have remainder, to each server the number of output of production subsystem or gray scale subsystem
For the production fragment item or gray scale fragment item of fragment number；If the division result has remainder, to production subsystem or gray scale
Each server the number of output of system is the production fragment item or gray scale fragment item of fragment number, then the production of redundancy collection number is divided
Piece item or gray scale fragment item are successively appended to corresponding clothes according to the server number order of production subsystem or gray scale subsystem
It is engaged in device；Or
If the cryptographic Hash is odd number, ascending order row is carried out to the number of the server of production subsystem or gray scale subsystem
Column, then the production fragment item or gray scale fragment item are sequentially output to the service for producing subsystem or gray scale subsystem
Device；If the cryptographic Hash is even number, descending arrangement is carried out to the number of the server of production subsystem or gray scale subsystem, then will
The production fragment item or gray scale fragment item are sequentially output the server to the production subsystem or gray scale subsystem.
In above scheme, in the step S5, receives the service system and execute the implementing result exported after each fragment item
It include: that will execute each fragment item after service system receives fragment item；If each fragment item of service system successful execution, will receive by
The content of service system output is the implementing result to run succeeded, if service system executes failure to any fragment item, will be received
It is to execute the implementing result of failure by the content that service system exports.
In above scheme, in the step S5, include: according to the implementing result management role data and mission bit stream
If the content of implementing result is to run succeeded, implementing result described in content is recorded and saved；If implementing result is interior
Hold to execute failure, triggers service system again according to implementing result and execute each fragment item；If the service system had executed
The production task of stipulated number, the content of the implementing result are still when executing failure, and recording and saving content is to execute failure
Implementing result；Or
If the content of implementing result is to run succeeded, implementing result described in content is recorded and saved；If implementing result is interior
Hold to execute failure, triggers service system again according to implementing result and execute each fragment item；If the service system had executed
The production task of stipulated number, the content of the implementing result are still when executing failure, then to delete the task data and task
Information；Or
If the content of implementing result is to run succeeded, implementing result described in content is recorded and saved；If implementing result is interior
Hold to execute failure, the task data and mission bit stream are deleted according to implementing result.
To achieve the above object, the present invention also provides a kind of Portable Batch System devices, comprising:
Module is founded, for founding NameSpace；
Task management module for extracting mission bit stream, and for instantiating mission bit stream, obtains task data；
Relating module, for task data to be associated with NameSpace；
Fragment module for obtaining several fragment items to task data fragment, and several fragment items is exported to institute
State service system；
Task scheduling modules receive the service system and execute the implementing result exported after each fragment item, and according to described
Implementing result management role data and mission bit stream.
To achieve the above object, the present invention also provides a kind of computer systems comprising multiple computer equipments, it is each to calculate
Machine equipment includes memory, processor and stores the computer program that can be run on a memory and on a processor, described
The processor of multiple computer equipments realizes Portable Batch System described in any of the above-described when executing the computer program jointly
The step of method.
To achieve the above object, the present invention also provides a kind of computer-readable storage systems comprising multiple storage mediums,
Computer program is stored on each storage medium, the computer program of the multiple storage medium storage is executed by processor
Shi Gongtong realizes the step of task scheduling and managing method described in any of the above-described.
A kind of task scheduling and managing method, device, computer system and readable storage system provided by the invention, utilize wound
If module founds production NameSpace and gray scale NameSpace under same domain name, task management module is recycled to extract task letter
Cease and to the example after, finally using relating module make production task data with produce NameSpace be associated with, make gray scale task
Data are associated with gray scale NameSpace, are realized and are carried out production task scheduling and gray scale task schedule under same domain name, and
It ensure that the isolation of production task data and gray scale task data, carry out gray scale publication for operator and provide convenience；
Using fragment module and using average stripping strategy, odevity stripping strategy and enumerated value stripping strategy to described
Business data fragmentation obtains fragment item, expands the scope of application, meets a variety of fragment demands；
Differentiated control is carried out to task data using task scheduling modules and using priority policy, wherein Gao Youxian
Grade mode is mainly used for monitoring the executive condition of important mission bit stream, and then guarantees that important mission bit stream can be held
Row, can be by operator's timely learning can not be performed, and takes corresponding remedial measure, ensure that task schedule
The reliability of work；Middle priority mode is mainly used for monitoring the executive condition of more important mission bit stream, protects as far as possible
Having demonstrate,proved more important mission bit stream can be executed；And low priority mode then ensure that processing to task data and hold
Line efficiency, and then ensure that the whole efficiency of task schedule work operation.
Detailed description of the invention
Fig. 1 is the flow chart of task scheduling and managing method embodiment one of the present invention；
Fig. 2 be task scheduling and managing method embodiment one of the present invention between Portable Batch System device and service system
Work flow diagram；
Fig. 3 is the program module schematic diagram of Portable Batch System Installation practice two of the present invention；
Fig. 4 is the hardware structural diagram of computer equipment in computer system embodiment three of the present invention.
Appended drawing reference:
1, Portable Batch System device 2, service system 3, computer equipment
11, module 12, task management module 13, relating module are founded
14, fragment module 15, task scheduling modules 31, memory 32, processor
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that described herein, specific examples are only used to explain the present invention, not
For limiting the present invention.Based on the embodiments of the present invention, those of ordinary skill in the art are not before making creative work
Every other embodiment obtained is put, shall fall within the protection scope of the present invention.
A kind of task scheduling and managing method, device, computer system and readable storage system provided by the invention, are suitable for
System O&M technical field provides one kind for Portable Batch System device and is based on founding module, task management module, association mould
The task scheduling and managing method of block, fragment module and task scheduling modules founds production using module is founded under same domain name
NameSpace and gray scale NameSpace, recycle task management module extract mission bit stream and to the example after, finally utilize
Relating module is associated with production task data with production NameSpace, is associated with gray scale task data with gray scale NameSpace, real
Show and carried out production task scheduling and gray scale task schedule under same domain name, and ensure that production task data and gray scale are appointed
The isolation for data of being engaged in carries out gray scale publication for operator and provides convenience；
Using fragment module and using average stripping strategy, odevity stripping strategy and enumerated value stripping strategy to described
Business data fragmentation obtains fragment item, expands the scope of application, meets a variety of fragment demands；
Differentiated control is carried out to task data using task scheduling modules and using priority policy, wherein Gao Youxian
Grade mode is mainly used for monitoring the executive condition of important mission bit stream, and then guarantees that important mission bit stream can be held
Row, can be by operator's timely learning can not be performed, and takes corresponding remedial measure, ensure that task schedule
The reliability of work；Middle priority mode is mainly used for monitoring the executive condition of more important mission bit stream, protects as far as possible
Having demonstrate,proved more important mission bit stream can be executed；And low priority mode then ensure that processing to task data and hold
Line efficiency, and then ensure that the whole efficiency of task schedule work operation.
Embodiment one:
Please refer to Fig. 1 and Fig. 2, a kind of task scheduling and managing method of the present embodiment, using Portable Batch System device 1,
The following steps are included:
S1: NameSpace (NameSpace i.e. namespace) is founded；
Specifically, the NameSpace includes production NameSpace and gray scale NameSpace；Wherein, production NameSpace and
Gray scale NameSpace is founded under same domain name.
Further, this step includes founding production NameSpace and gray scale NameSpace.
Further, service system 2 includes production subsystem and gray scale subsystem, produces the title and life of NameSpace
The title of produce surviving of son system is consistent, and the title of the gray scale NameSpace is consistent with the title of gray scale subsystem.
Further, the gray scale NameSpace it is entitled it is described production NameSpace title ending increase "-
abt”。
S2: extracting and instantiates mission bit stream, obtains task data；
Specifically, the mission bit stream can be production task or gray scale task, the mission bit stream is by least one task
Unit is constituted.
Specifically, access service system 2, extracts mission bit stream from service system 2, instantiated using client data packet
Mission bit stream obtains task data；The client data packet includes task configuration class, task address class and task execution class；Institute
Stating task data includes task configuration information, task address information and task execution information；Wherein, task execution information includes not
Execute, run succeeded and execute failure.
Further, the task data can be production task data or gray scale task data, mention from production subsystem
Production task is taken, gray scale task is extracted from gray scale subsystem.
Further, this step includes load client data packet, instantiates mission bit stream using client data packet, obtains
Obtain task data；
Production task data then include the task configuration information of the production task, task address information and task execution letter
Breath；
Gray scale task data then includes the task configuration information of the gray scale task, task address information and task execution letter
Breath.
Optionally, the client data Bao Kewei product cash flow client data packet can also monitor for monthly index
Client data packet, the client data packet can also be used for the data to production task and gray scale task instances for other
Packet.
S3: task data is associated with NameSpace；
Specifically, this step includes being associated with production task data with production NameSpace, by gray scale task data and ash
The name space of dragging out a miserable existence association production task data and gray scale task data are isolated, and then realizes that isolation production task and gray scale are appointed
The effect of business；
Further, production task data and gray scale task data respectively further comprise leader node information；
The leader node information of the production task determines in the following manner:
To the production task data creation byte number under production NameSpace, according to the byte number to several lifes
It produces task data to be ranked up, selects in the sequence the smallest production task data of serial number as producing all lifes under NameSpace
The leader node for producing task data will produce the leader node information in production task data all under NameSpace, write
For the task configuration information of the leader node；
Byte number is created to the gray scale task data under gray scale NameSpace, according to the byte number to several ashes
Degree task data is ranked up, and selects in the sequence the smallest gray scale task data of serial number as all ashes under gray scale NameSpace
The leader node of degree task data writes the leader node information in gray scale task data all under gray scale NameSpace
For the task configuration information of the leader node.
S4: several fragment items are obtained to task data fragment；Several fragment items are exported to service system 2；
Wherein, the fragment item includes production fragment item and gray scale fragment item.
Specifically, the step S4 the following steps are included:
S41: the setting triggered time makes to trigger scheduler program according to triggered time triggering task schedule frame；
S42: using task schedule frame to task data fragment, and several fragment items are obtained；
S43: several fragment items are exported to the server of service system 2 using task schedule frame；
Wherein, the fragment Xiang Kewei production fragment item or gray scale fragment item.Production fragment item is exported to production subsystem
The server of system, by the server of gray scale fragment item output gray level subsystem.
Specifically, the step S42 includes: based on average stripping strategy
S42-01: using task schedule frame according to fragment number to task data fragment, make to produce fragment item or gray scale point
The quantity of piece item is consistent with fragment number；
S42-02: fragment number and the sum of production subsystem or the Servers-all of gray scale subsystem are divided by；
S42-03: if fragment number can not be had by the division result of all production subsystems or the server of gray scale subsystem
There is remainder, then using the quotient of division result as fragment number；If fragment number can not be by all production subsystems or gray scale subsystem
The division result of server there is remainder, then using the quotient in division result as fragment number, and by the remainder in division result
As redundancy collection number；
Further, the step S43 includes: based on average stripping strategy
If the division result does not have remainder, to each server the number of output of production subsystem or gray scale subsystem
For the production fragment item or gray scale fragment item of fragment number；
If the division result has remainder, each server the number of output to production subsystem or gray scale subsystem is
The production fragment item or gray scale fragment item of fragment number, then by the production fragment item of redundancy collection number or gray scale fragment item, according to production
The server number order of subsystem or gray scale subsystem is successively appended in corresponding server；
Such as:
If the sum of the server of all production subsystems is 3, fragment number is 8, then every server is assigned to
Production fragment item is: 1=[0,1,6], 2=[2,3,7], 3=[4,5]；
If the sum of the server of all production subsystems is 3, fragment number is 10, then every server is assigned to
Production fragment item is: 1=[0,1,2,9], 2=[3,4,5], 3=[6,7,8].
Specifically, the step S42 includes: based on odevity stripping strategy
S42-11: production task data or gray scale task data fragment are made according to fragment number using task schedule frame
The quantity for producing fragment item or gray scale fragment item is consistent with fragment number；
S42-12: Hash fortune is carried out using task name of the task schedule frame to production task data or gray scale task data
It calculates and obtains cryptographic Hash；
Further, the step S43 includes: based on odevity stripping strategy
If the cryptographic Hash is odd number, ascending order row is carried out to the number of the server of production subsystem or gray scale subsystem
Column, then the production fragment item or gray scale fragment item are sequentially output to the service for producing subsystem or gray scale subsystem
Device；
If the cryptographic Hash is even number, descending row is carried out to the number of the server of production subsystem or gray scale subsystem
Column, then the production fragment item or gray scale fragment item are sequentially output to the service for producing subsystem or gray scale subsystem
Device；
Such as: production subsystem has 3 servers, and fragment number is 2, carries out Hash to the task name of production task data
Operation simultaneously obtains cryptographic Hash as odd number, then the production fragment item that every server is assigned to is: 1=[0], 2=[1], 3=[]；It is raw
Produce surviving of son system has 3 servers, and fragment number is 2, carries out Hash operation to the task name of production task data and obtains Hash
Value is even number, then the production fragment item that every server is assigned to is: 3=[0], 2=[1], 1=[].
Specifically, the step S42 includes: based on enumerated value stripping strategy
S42-21: the service code of the TU task unit of production task or gray scale task is extracted；
S42-22: enumerated value is formed to service code assignment；
S42-23: will be had same according to enumerated value to production task or gray scale task fragment using task schedule frame
The TU task unit of enumerated value is attributed to same production fragment item or gray scale fragment item.
Further, the production fragment item or gray scale fragment item are sequentially output to the production subsystem or gray scale
The server of system
Optionally, task schedule frame is that (Quartz is that OpenSymphony open source group is woven in Job to Quartz
Another open source projects of the field scheduling) application.
Optionally, enumerated value can be time, place, personage or other name entities.
Such as: the service code of the TU task unit of production task is 0,1,2, wherein 0 indicates Beijing, and 1 indicates Shanghai, 2 tables
Show Guangzhou；Enumerated value, the i.e. Beijing 0=, the Shanghai 1=, the Guangzhou 2=are formed to service code assignment；
The server of production subsystem is respectively A, B, C, then the production fragment item of each server will in production subsystem
By A=[0], B=[1], C=[2] switch in A=[Beijing], and B=[Shanghai], C=[Guangzhou] are conducive to operator to fragment
The management of item.
S5: it receives the service system 2 and executes the implementing result exported after each fragment item, and according to the implementing result pipe
Task data and mission bit stream are managed, to realize the task schedule to production task and the task schedule to gray scale task.
Wherein, before obtaining implementing result, the task execution information of production task data or gray scale task data is
It is not carried out；
Specifically, the step S5 includes: based on high priority mode
S5-01 service system 2 will execute each fragment item after receiving several described fragment items；
If each fragment item of 2 successful execution of S5-02 service system, the content that is exported by service system 2 will be received for execute at
The implementing result of function, if service system 2 executes failure to any fragment item, will receive the content exported by service system 2 is to hold
The implementing result of row failure；
If the content of S5-03 implementing result is to run succeeded, implementing result described in content is recorded and saved, and by number of tasks
According to task execution information be modified to carry out success；If the content of implementing result is to execute failure, touched again according to implementing result
Hair service system 2 executes each fragment item；If the service system 2 had executed the production task of stipulated number, described to execute knot
The content of fruit is still that record and save content be to execute the implementing result to fail when executing failure, and by the task of task data
Execution information is modified to carry out failure.
Optionally, the stipulated number can be for three times.
Further, the step S5 includes: based on middle priority mode
S5-11: service system 2 will execute each fragment item after receiving several described fragment items；
S5-12: if each fragment item of 2 successful execution of service system, the content that is exported by service system 2 will be received for execute at
The implementing result of function, if service system 2 executes failure to any fragment item, will receive the content exported by service system 2 is to hold
The implementing result of row failure；
S5-13: if the content of implementing result is to run succeeded, recording and saving implementing result described in content, and by task
The task execution information of data is modified to carry out success；If the content of implementing result is to execute failure, again according to implementing result
Triggering service system 2 executes each fragment item；If the service system 2 had executed the production task of stipulated number, the execution
As a result content is still when executing failure, then to delete the task data and mission bit stream.
Optionally, the stipulated number can be for three times.
Specifically, the step S5 includes: based on low priority mode
S5-21: service system 2 will execute each fragment item after receiving several described fragment items；
S5-22: if each fragment item of 2 successful execution of service system, the content that is exported by service system 2 will be received for execute at
The implementing result of function, if service system 2 executes failure to any fragment item, will receive the content exported by service system 2 is to hold
The implementing result of row failure；
S5-23: if the content of implementing result is to run succeeded, recording and saving implementing result described in content, and by task
The task execution information of data is modified to carry out success；If the content of implementing result is to execute failure, deleted according to implementing result
The task data and mission bit stream.
Embodiment two:
Please continue to refer to Fig. 3, a kind of Portable Batch System device 1 is shown, comprising:
Module 11 is founded, for founding NameSpace；
Task management module 12 for extracting mission bit stream, and for instantiating mission bit stream, obtains task data；
The instantiation module loading has client data packet；
Relating module 13, for task data to be associated with NameSpace；
Fragment module 14, for obtaining several fragment items to task data fragment, and by several fragment items export to
The service system 2；The fragment module 14 has triggering scheduler program and task schedule frame, and can set the triggered time；
Task scheduling modules 15 execute each fragment item for triggering service system 2, and obtain implementing result.
Optionally, the module 11 of founding is that (ZooKeeper is one distributed to ZooKeeper, point of open source code
Cloth application program coordination service).
The technical program is based on internet O&M, using module 11, task management module 12 and relating module 13 is founded, in fact
The effect that production environment publication and the publication of gray scale environment are completed under same domain name is showed；Utilize fragment module 14 and task schedule
Module 15 plays under production environment and gray scale environment, to the excellent of production task data and gray scale task data issuing process
Change effect.
Embodiment three:
To achieve the above object, the present invention also provides a kind of computer system, which includes multiple computers
The component part of equipment 3, the Portable Batch System device 1 of embodiment two is dispersed in different computer equipments, computer
Equipment can be smart phone, tablet computer, laptop, desktop computer, rack-mount server, the blade for executing program
Formula server, (including the service composed by independent server or multiple servers of tower server or Cabinet-type server
Device cluster) etc..The computer equipment of the present embodiment includes, but is not limited to: that depositing for connection can be in communication with each other by system bus
Reservoir 31, processor 32, as shown in Figure 4.It should be pointed out that Fig. 4 illustrates only the computer equipment with component-, still
It should be understood that, it is not required that implement all components shown, the implementation that can be substituted is more or less component.
In the present embodiment, memory 31 (i.e. readable storage medium storing program for executing) includes flash memory, hard disk, multimedia card, card-type memory
(for example, SD or DX memory etc.), random access storage device (RAM), static random-access memory (SRAM), read-only memory
(ROM), electrically erasable programmable read-only memory (EEPROM), programmable read only memory (PROM), magnetic storage, magnetic
Disk, CD etc..In some embodiments, memory 31 can be the internal storage unit of computer equipment, such as the computer
The hard disk or memory of equipment.In further embodiments, memory 31 is also possible to the External memory equipment of computer equipment, example
The plug-in type hard disk being equipped in such as computer equipment, intelligent memory card (Smart Media Card, SMC), secure digital
(Secure Digital, SD) card, flash card (Flash Card) etc..Certainly, memory 31 can also both be set including computer
Standby internal storage unit also includes its External memory equipment.In the present embodiment, memory 31 is installed on meter commonly used in storage
Calculate machine equipment operating system and types of applications software, such as embodiment one Portable Batch System device program code etc..
In addition, memory 31 can be also used for temporarily storing the Various types of data that has exported or will export.
Processor 32 can be in some embodiments central processing unit (Central Processing Unit, CPU),
Controller, microcontroller, microprocessor or other data processing chips.The processor 32 is commonly used in control computer equipment
Overall operation.In the present embodiment, program code or processing data of the processor 32 for being stored in run memory 31, example
Portable Batch System device is run, such as to realize the task scheduling and managing method of embodiment one.
Example IV:
To achieve the above object, the present invention also provides a kind of computer-readable storage systems comprising multiple storage mediums,
Such as flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory), random access storage device (RAM), static state
Random access storage device (SRAM), electrically erasable programmable read-only memory (EEPROM), may be programmed read-only memory (ROM)
Read-only memory (PROM), magnetic storage, disk, CD, server, App are stored thereon with computer using store etc.
Program, program realize corresponding function when being executed by processor 32.The computer readable storage medium of the present embodiment is appointed for storing
Business dispatching managing device, realizes the task scheduling and managing method of embodiment one when being executed by processor 32.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can be realized by means of software and necessary general hardware platform, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.
The above is only a preferred embodiment of the present invention, is not intended to limit the scope of the invention, all to utilize this hair
Equivalent structure or equivalent flow shift made by bright specification and accompanying drawing content is applied directly or indirectly in other relevant skills
Art field, is included within the scope of the present invention.
Claims (10)
1. a kind of task scheduling and managing method, which comprises the following steps:
S1: NameSpace is founded；
S2: extracting and instantiates mission bit stream, obtains task data；
S3: the task data is associated with the NameSpace；
S4: fragment item is obtained to the task data fragment；Fragment item is exported to service system；
S5: it receives the service system and executes the implementing result exported after fragment item, and according to the implementing result management role
Data and mission bit stream.
2. task scheduling and managing method according to claim 1, which is characterized in that the NameSpace packet in the step S1
Production NameSpace and gray scale NameSpace are included, the mission bit stream in the step S2 can be production task or gray scale task, institute
Stating task data in step S3 can be production task data or gray scale task data；Fragment item in the step S4 includes production
Fragment item and gray scale fragment item.
3. task scheduling and managing method according to claim 2, which is characterized in that the step S2 includes from service system
Middle extraction mission bit stream obtains task data using client data packet instantiation mission bit stream；The task data includes appointing
Business configuration information, task address information and task execution information；Wherein, production task is extracted from production subsystem, from gray scale
Gray scale task is extracted in subsystem；
The step S3 includes being associated with production task data with production NameSpace, gray scale task data and gray scale is named empty
Between be associated with, production task data and gray scale task data are isolated.
4. task scheduling and managing method according to claim 2, which is characterized in that the step S4 includes: to number of tasks
Fragment item is obtained according to fragment, fragment item is exported to the server of service system；
It is described to include: to production task data or gray scale task data fragment
According to fragment number to production task data or gray scale task data fragment；By fragment number and production subsystem or gray scale subsystem
The sum of the Servers-all of system is divided by；If division result does not have remainder, using the quotient of division result as fragment number；If phase
Division result has remainder, then using the quotient of division result as fragment number, using the remainder of division result as redundancy collection number；Or
According to fragment number to production task data or gray scale task data fragment；To production task data or gray scale task data
Task name carries out Hash operation and obtains cryptographic Hash；Or
The service code for extracting the TU task unit of production task or gray scale task forms enumerated value to service code assignment, according to
Enumerated value is to production task or gray scale task fragment.
5. task scheduling and managing method according to claim 4, which is characterized in that described to produce fragment item or gray scale point
Piece item is exported to the server of production subsystem or gray scale subsystem
It is point to each server the number of output of production subsystem or gray scale subsystem if the division result does not have remainder
The production fragment item or gray scale fragment item of the piece number；If the division result has remainder, to production subsystem or gray scale subsystem
Each server the number of output be fragment number production fragment item or gray scale fragment item, then by the production fragment item of redundancy collection number
Or gray scale fragment item, corresponding server is successively appended to according to the server number order of production subsystem or gray scale subsystem
In；Or
If the cryptographic Hash is odd number, ascending order arrangement is carried out to the number of the server of production subsystem or gray scale subsystem, then
The production fragment item or gray scale fragment item are sequentially output to the server to the production subsystem or gray scale subsystem；If institute
Stating cryptographic Hash is even number, carries out descending arrangement to the number of the server of production subsystem or gray scale subsystem, then by the life
It produces fragment item or gray scale fragment item is sequentially output to the server for producing subsystem or gray scale subsystem.
6. task scheduling and managing method according to claim 1, which is characterized in that in the step S5, receive the clothes
It includes: that will execute each fragment item after service system receives fragment item that business system, which executes the implementing result exported after each fragment item,；
It is the implementing result to run succeeded by receiving by the content that service system exports if each fragment item of service system successful execution, if
Service system executes failure to any fragment item, and will receive by the content that service system exports is to execute the implementing result of failure.
7. task scheduling and managing method according to claim 1, which is characterized in that in the step S5, held according to described
Row results management task data and mission bit stream include:
If the content of implementing result is to run succeeded, implementing result described in content is recorded and saved；If the content of implementing result is
Failure is executed, service system is triggered again according to implementing result and executes each fragment item；If the service system had executed regulation
The production task of number, the content of the implementing result are still when executing failure, and recording and saving content is to execute holding for failure
Row result；Or
If the content of implementing result is to run succeeded, implementing result described in content is recorded and saved；If the content of implementing result is
Failure is executed, service system is triggered again according to implementing result and executes each fragment item；If the service system had executed regulation
The production task of number, the content of the implementing result are still when executing failure, then to delete the task data and mission bit stream；
Or
If the content of implementing result is to run succeeded, implementing result described in content is recorded and saved；If the content of implementing result is
Failure is executed, the task data and mission bit stream are deleted according to implementing result.
8. a kind of Portable Batch System device characterized by comprising
Module is founded, for founding NameSpace；
Task management module for extracting mission bit stream, and for instantiating mission bit stream, obtains task data；
Relating module, for task data to be associated with NameSpace；
Fragment module for obtaining several fragment items to task data fragment, and several fragment items is exported to the clothes
Business system；
Task scheduling modules receive the service system and execute the implementing result exported after each fragment item, and according to the execution
Results management task data and mission bit stream.
9. a kind of computer system comprising multiple computer equipments, each computer equipment include memory processor and deposit
Store up the computer program that can be run on a memory and on a processor, which is characterized in that the place of the multiple computer equipment
Reason device realizes the step of any one of claim 1 to 7 task scheduling and managing method jointly when executing the computer program.
10. a kind of computer-readable storage system comprising multiple storage mediums are stored with computer journey on each storage medium
Sequence, which is characterized in that the computer program of the multiple storage medium storage realizes right when being executed by processor jointly
It is required that the step of any one of 1 to 7 task scheduling and managing method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811535241.0A CN109783228A (en) | 2018-12-14 | 2018-12-14 | Task scheduling and managing method, device, computer system and readable storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811535241.0A CN109783228A (en) | 2018-12-14 | 2018-12-14 | Task scheduling and managing method, device, computer system and readable storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109783228A true CN109783228A (en) | 2019-05-21 |
Family
ID=66496990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811535241.0A Pending CN109783228A (en) | 2018-12-14 | 2018-12-14 | Task scheduling and managing method, device, computer system and readable storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109783228A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110131285A1 (en) * | 2009-11-30 | 2011-06-02 | International Business Machines Corporation | Task management system associating with contact information and method thereof |
| CN103473272A (en) * | 2013-08-20 | 2013-12-25 | 小米科技有限责任公司 | Data processing method, device and system |
| US20150172140A1 (en) * | 2013-12-18 | 2015-06-18 | Thomson Reuters Global Resources (Trgr) | System and method for dynamically scheduling network scanning tasks |
| US9118697B1 (en) * | 2006-03-20 | 2015-08-25 | Netapp, Inc. | System and method for integrating namespace management and storage management in a storage system environment |
| CN106936899A (en) * | 2017-02-25 | 2017-07-07 | 九次方大数据信息集团有限公司 | The collocation method of distributed statistical analysis system and distributed statistical analysis system |
| CN107807845A (en) * | 2017-10-16 | 2018-03-16 | 昆仑智汇数据科技(北京)有限公司 | A kind of incremented data parallel processing apparatus and method |
| CN108958920A (en) * | 2018-07-13 | 2018-12-07 | 众安在线财产保险股份有限公司 | A kind of distributed task dispatching method and system |
-
2018
- 2018-12-14 CN CN201811535241.0A patent/CN109783228A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9118697B1 (en) * | 2006-03-20 | 2015-08-25 | Netapp, Inc. | System and method for integrating namespace management and storage management in a storage system environment |
| US20110131285A1 (en) * | 2009-11-30 | 2011-06-02 | International Business Machines Corporation | Task management system associating with contact information and method thereof |
| CN103473272A (en) * | 2013-08-20 | 2013-12-25 | 小米科技有限责任公司 | Data processing method, device and system |
| US20150172140A1 (en) * | 2013-12-18 | 2015-06-18 | Thomson Reuters Global Resources (Trgr) | System and method for dynamically scheduling network scanning tasks |
| CN106936899A (en) * | 2017-02-25 | 2017-07-07 | 九次方大数据信息集团有限公司 | The collocation method of distributed statistical analysis system and distributed statistical analysis system |
| CN107807845A (en) * | 2017-10-16 | 2018-03-16 | 昆仑智汇数据科技(北京)有限公司 | A kind of incremented data parallel processing apparatus and method |
| CN108958920A (en) * | 2018-07-13 | 2018-12-07 | 众安在线财产保险股份有限公司 | A kind of distributed task dispatching method and system |
Non-Patent Citations (1)
| Title |
|---|
| 刘一田;刘士进;: "多租户高可用并行任务调度框架", 计算机系统应用, no. 12, 15 December 2016 (2016-12-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109582433B (en) | Resource scheduling method and device, cloud computing system and storage medium | |
| CN111124277B (en) | Deep learning data set caching method, system, terminal and storage medium | |
| CN109683806A (en) | Storage device and its operating method and computing system including storage device | |
| CN103049317B (en) | Based on the high concurrent data no write de-lay system and method for queue under cloud environment | |
| CN109558260B (en) | Kubernetes fault elimination system, method, equipment and medium | |
| CN113626286B (en) | Multi-cluster instance processing method and device, electronic equipment and storage medium | |
| CN110427258B (en) | Resource scheduling control method and device based on cloud platform | |
| CN110737504B (en) | Fault-tolerant method, system, terminal and storage medium for deep learning model training | |
| CN107645410A (en) | A kind of virtual machine management system and method based on OpenStack cloud platforms | |
| CN103414712A (en) | Management system and method of distributed virtual desktop | |
| CN105843937A (en) | Data processing method and device | |
| CN108694076A (en) | Multinode storage operation | |
| CN107844711B (en) | Data manipulation permission partition method, application server and computer readable storage medium | |
| US10305817B1 (en) | Provisioning system and method for a distributed computing environment using a map reduce process | |
| CN111045802B (en) | Redis cluster component scheduling system and method and platform equipment | |
| CN108255994A (en) | A kind of database version management method based on database snapshot | |
| CN110162344A (en) | A kind of method, apparatus, computer equipment and readable storage medium storing program for executing that current limliting is isolated | |
| EP4206915A1 (en) | Container creation method and apparatus, electronic device, and storage medium | |
| US8539048B2 (en) | Electronic device and method for loading configuration files using the same | |
| CN111401752A (en) | Flywheel energy storage frequency modulation sharing processing method, device and system and electronic equipment | |
| CN107528871A (en) | Data analysis in storage system | |
| CN113424152A (en) | Workflow-based scheduling and batching in a multi-tenant distributed system | |
| CN112612605B (en) | Thread allocation method, thread allocation device, computer equipment and readable storage medium | |
| CN113792038A (en) | Method and apparatus for storing data | |
| CN109783228A (en) | Task scheduling and managing method, device, computer system and readable storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |