CN108874528A - Distributed task scheduling storage system and distributed task scheduling storage/read method - Google Patents
Distributed task scheduling storage system and distributed task scheduling storage/read method Download PDFInfo
- Publication number
- CN108874528A CN108874528A CN201710319768.9A CN201710319768A CN108874528A CN 108874528 A CN108874528 A CN 108874528A CN 201710319768 A CN201710319768 A CN 201710319768A CN 108874528 A CN108874528 A CN 108874528A
- Authority
- CN
- China
- Prior art keywords
- task
- service
- service node
- storage
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5022—Workload threshold
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/508—Monitor
Abstract
This application discloses a kind of distributed task scheduling storage systems and distributed task scheduling storage/read method.One specific embodiment of the system includes：Gateway server, the task requests sent for receiving each operation system, and each task requests are assigned to the service node in task storage cluster, wherein, task requests are task storage request or task read requests, and each task requests include the task type for the needing to store or read of the task；Data persistence divides library cluster, including multiple databases, and each database is used to store the task of corresponding task type；The task type of task storage cluster, including multiple service nodes, the task that each service node is used to store as needed stores each task into corresponding database, and the task type of the task for reading as needed reads each task.The distributed task scheduling storage system that the embodiment provides can be used for storing the task of different business systems.
Description
Technical field
This application involves field of computer technology, and in particular to technical field of memory more particularly to distributed task scheduling storage
System and distributed task scheduling storage/read method.
Background technique
Either in large-scale Internet application, or in enterprise's level framework, different operation systems can exist
The totally different task of a large amount of types needs to store and read.The mode that the prior art generallys use Distributed Message Queue carries out task
Storage and reading, and different operation systems needs individual storage system to store and read task data, and this makes not
Task data with operation system stores dispersion, is unfavorable for statisticalling analyze.
Summary of the invention
The purpose of the application is to propose a kind of improved distributed task scheduling storage system and distributed task scheduling storage/reading
Method, to solve the technical issues of background section above is mentioned.
In a first aspect, the embodiment of the present application provides a kind of distributed task scheduling storage system, which includes：Gateway service
Device, the task requests sent for receiving each operation system, and each task requests are assigned to the service in task storage cluster
Node, wherein task requests are task storage request or task read requests, and each task requests include needing to store or read
The task type of task；Data persistence divides library cluster, including multiple databases, and each database is for storing corresponding task class
The task of type；Task storage cluster, including multiple service nodes, the task for the task that each service node is used to store as needed
Type stores each task into corresponding database, and the task type of the task for reading as needed reads each
Business.
In some embodiments, above system further includes unified configuration center, for constructing distributed task scheduling storage system
In a variety of services, and setting task storage cluster needs the quantity respectively serviced disposed；Task storage cluster further includes control
Center processed, for being serviced for the deployment of each service node is at least one.
In some embodiments, unified configuration center is also used to monitor the non-load balanced case of each service node, and is supervising
The transfer request for shifting the service is sent when measuring the load imbalance of service node to control centre, so that control centre is by portion
At least one service affixed one's name in first service node is transferred to second service node, wherein the load capacity of first service node is big
In the load capacity of second service node.
In some embodiments, unified configuration center is also used to monitor the load capacity of each service, and is monitoring there are clothes
When the load capacity of business is greater than corresponding first preset threshold, the number of the service in increase task storage cluster is sent to control centre
The increase service request of amount；Unified configuration center is also used to be less than corresponding second in the load capacity for monitoring presence service default
When threshold value, the reduction service request of the quantity of the service in reduction task storage cluster is sent to control centre.
In some embodiments, unified configuration center is also used to monitor the loading condition of each service node, and is monitoring
There are when the service node that load capacity is greater than third predetermined threshold value, the increase for increasing the quantity of service node is sent to control centre
Service node request；Unified configuration center is also used in the load capacity for monitoring presence service node less than the 4th preset threshold
When, the reduction service node request for the quantity for reducing service node is sent to control centre.
In some embodiments, control centre is also used to increase service request or reduction service request in response to receiving,
Increase or decrease the service in task storage cluster；Control centre is also used in response to receiving the request of increase service node or subtracting
Few service node request, increases or decreases the service node in task storage cluster.
In some embodiments, a variety of services include the service of responsibility chain, Discrete service, accelerate in service and Warning Service
It is at least one；Each service in task chain of the responsibility chain service for determining storage or reading task, and by each service according to pre-
If sequence be added in task chain；Gateway server is also used to that task requests are assigned to responsibility chain based on load-balancing algorithm
Service.
In some embodiments, above system further includes data buffer storage cluster, for storing each task in service node
Each task is cached when into database corresponding with the task type of the task, and is formed not according to the task type of each task
Same distributed queue.
In some embodiments, accelerate the task type of task of the service for reading as needed from data buffer storage cluster
In corresponding distributed queue read the task of caching.
In some embodiments, Warning Service is used to be greater than the 5th in response to the quantity for counting on not being read for task pre-
If threshold value, warning message is sent to user by unified configuration center；Warning Service is also used to there is abnormal shape in response to determining
The task of state sends warning message to user by unified configuration center.
In some embodiments, each task includes unique task identification；Discrete service is used in response to detecting each
Business has sole task mark, by the discrete storage of each task for needing to store to data corresponding with the task type of the task
Library.
Second aspect, this application provides a kind of distributed task scheduling storage/read method, this method includes：Gateway service
Device receives the task requests that each operation system is sent, and each task requests are assigned to the service node in task storage cluster,
Wherein, task requests are task storage request or task read requests, and each task requests include the needing to store or read of the task
Task type；Each task is stored to data persistence in response to receiving task storage request and divides library collection by each service node
In group in database corresponding with the task type of the task, alternatively, each service node is in response to receiving task read requests,
The task type of reading as needed for task reads each task；Wherein, data persistence point library cluster includes multiple databases,
And each database is used to store the task of corresponding task type.
In some embodiments, the above method further includes：It is used in unified configuration center building distributed task scheduling storage system
In a variety of services of storage/reading task, and set the quantity of each service；The control centre of task storage cluster configures from unified
Center obtains each service, and is at least one service of each service node deployment.
In some embodiments, the above method further includes：Unified configuration center monitors the load balancing feelings of each service node
Condition, and the transfer request for shifting the service is sent to control centre in the load imbalance for monitoring service node；In control
At least one service for being deployed in first service node is transferred to second service in response to receiving the transfer request of service by the heart
Node, wherein the load capacity of first service node is greater than the load capacity of second service node.
In some embodiments, the above method further includes：Unified configuration center monitors the load capacity of each service, and is monitoring
To presence service load capacity be greater than corresponding first preset threshold when, to control centre send increase task storage cluster in should
The increase service request of the quantity of service；Unified configuration center is less than corresponding second in advance in the load capacity for monitoring presence service
If when threshold value, the reduction service request of the quantity of the service in reduction task storage cluster is sent to control centre；Control centre
Increase service request or reduction service request in response to receiving, increases or decreases the service disposed in task storage cluster.
In some embodiments, the above method further includes：Unified configuration center monitors the loading condition of each service node, and
When the load capacity for monitoring presence service node is greater than third predetermined threshold value, the number for increasing service node is sent to control centre
The increase service node of amount is requested；Unified configuration center is in the load capacity for monitoring presence service node less than the 4th preset threshold
When, the reduction service node request for the quantity for reducing service node is sent to control centre；Control centre increases in response to receiving
Add service node to request or reduce service node request, increases or decreases the service node in task storage cluster.
In some embodiments, a variety of services include the service of responsibility chain, Discrete service, accelerate in service and Warning Service
It is at least one；Each task is stored to data persistence in response to receiving task storage request and divides library cluster by each service node
In in database corresponding with the task type of the task, including：Gateway server is based on load-balancing algorithm and stores task
Request is assigned to responsibility chain service；Task storage cluster start the service of responsibility chain, according to chain-of-responsibilpattern pattern from each service node according to
It is secondary to call each service for being used for store tasks, by task storage to corresponding database.
In some embodiments, each service node is in response to receiving task read requests, reading as needed for task
Task type read each task, including：Gateway server is based on load-balancing algorithm and task read requests is assigned to responsibility
Chain service；Each service node is in response to receiving task read requests, corresponding with the task type of task for needing to read
Task is read in database.
In some embodiments, each task includes unique task identification；The above method further includes：Task storage cluster tune
With Discrete service, in response to detecting that each task is identified with sole task, Discrete service arrives the discrete storage of each task and this
The corresponding database of the task type of task.
In some embodiments, the above method further includes：It, will in each service node in response to receiving task storage request
While each task storage divides database corresponding with the task type of the task in the cluster of library to data persistence, by each task
It is cached to data buffer storage cluster, and forms distributed queue corresponding with the task type of each task in data buffer storage cluster；
In response to receiving task read requests, task storage cluster, which calls, accelerates service to read task from distributed queue.
In some embodiments, the above method further includes：Service node calls Warning Service to count stored each
The quantity of not being read in business for task, and when the quantity for counting on not being read for task is greater than five preset thresholds, lead to
It crosses unified configuration center and sends warning message to user；And service node calls the task of Warning Service monitoring abnormality,
And determining there are when the task of abnormality, warning message is sent to user by unified configuration center.
The third aspect, this application provides a kind of servers, including：One or more processors；Storage device, for depositing
Store up one or more programs；When one or more programs are executed by one or more processors, so that one or more processors
Realize any of the above-described method.
Fourth aspect, this application provides a kind of computer readable storage mediums, are stored thereon with computer program, special
Sign is that the program realizes any of the above-described method when being executed by processor.
Distributed task scheduling storage system provided by the embodiments of the present application, gateway server therein are receiving each business system
Each request, can be assigned to the service node of task storage cluster by the task storage request or task read requests that system is sent,
Each service node can be by the task storage for needing to store to the task type with the task in the cluster of data persistence point library
In corresponding database and the task type of task that each service node can also be read as needed reads each task, from
And the distributed task scheduling storage system is made to can store the task of different business systems publication, and make the distributed task scheduling
Storage system can correspondingly read the task of different business systems storage.
Detailed description of the invention
By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the application's is other
Feature, objects and advantages will become more apparent upon：
Fig. 1 shows the structural schematic diagram of one embodiment of the distributed task scheduling storage system according to the application；
Fig. 2 shows the flow charts according to distributed task scheduling storage/read method one embodiment of the application；
Fig. 3 shows the flow chart of another embodiment of the distributed task scheduling storage method according to the application；
Fig. 4 shows the flow chart of another embodiment of the distributed task scheduling read method according to the application；
Fig. 5 shows the structure of the computer system of the terminal device or server that are suitable for being used to realize the embodiment of the present application
Schematic diagram.
Specific embodiment
The application is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining related invention, rather than the restriction to the invention.It also should be noted that in order to
Convenient for description, part relevant to related invention is illustrated only in attached drawing.
It should be noted that in the absence of conflict, the features in the embodiments and the embodiments of the present application can phase
Mutually combination.The application is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.
Fig. 1 shows the structural schematic diagram of one embodiment of the distributed task scheduling storage system according to the application.Such as figure
Shown, the distributed task scheduling storage system in the present embodiment may include：Gateway server 101,102 sum number of task storage cluster
Divide library cluster 103 according to persistence.
In the present embodiment, above-mentioned gateway server 101 can receive the task requests that different operation systems is sent.This
In, task requests can be to store to the task of the distributed task scheduling storage system store tasks and request, alternatively, task requests are also
It can be the task read requests that task is read from the distributed task scheduling storage system；Above-mentioned task requests may include needing to deposit
The task type of storage or reading for task, i.e. task storage request include the task type for needing store tasks, and task reading is asked
Seek the task type including needing to read task.Above-mentioned each task requests can be assigned to task storage by gateway server 101
Service node in cluster 102.
In the present embodiment, above-mentioned task storage cluster 102 may include multiple service nodes, as shown in Figure 1, task is deposited
Accumulation 102 may include service node 1, service node 2 and service node 3 etc..Each service node can according to need storage
Task type the storing of the task will be needed to store into the corresponding database in data persistence point library cluster 103.Here,
Data persistence point library cluster 103 may include multiple databases, as shown in Figure 1, data persistence point library cluster 103 can wrap
Include database 1, database 2, database 3 etc..Each database can store the task of corresponding task type.Above-mentioned gateway server
101 when receiving task read requests, and the task type that can according to need the task of reading reads each times for needing to read
Business.As it can be seen that the distributed task scheduling storage system may be implemented to carry out the task that different business systems are issued unified storage and reading
It takes.
In some optional implementations of the present embodiment, above-mentioned distributed task scheduling storage system can also include system
One configuration center 104, as shown in Figure 1.This, which unifies configuration center 104, can be used for constructing in the distributed task scheduling storage system
A variety of services, and the quantity respectively serviced for needing to dispose for the setting of above-mentioned task storage cluster 102.Above-mentioned task storage cluster
102 can also include control centre, as shown in Figure 1, the control centre can be each service node in task storage cluster 102
Dispose above-mentioned at least one service.
In some optional implementations of the present embodiment, above-mentioned unified configuration center 104 can also monitor task and deposit
The non-load balanced case of each service node in accumulation 102, and when the case where load imbalance occurs in service node to control
Center sends the transfer request of transfer service, in order to which control centre can shift the service being deployed on first service node
Guarantee each service node load balancing to second service node.For example, unified configuration center 104 is monitoring first service section
The load capacity of point is greater than the load capacity of second service node, and because the load capacity of first service node is greater than second service node
Load capacity the phenomenon that causing load imbalance, at this time this unify that configuration center 104 can send to control centre will deployment
Service on first service node is transferred to the request of the transfer service of second service node.Control centre is receiving this turn
The partial service being deployed on first service node can be transferred on second service node when moving request, so that first service
Node and second service node can be with load balancing.It is saved as it can be seen that above-mentioned control centre can be deployed in different services by transfer
Service on point, so that each service node load balancing in task storage cluster 102.
In some optional implementations of the present embodiment, above-mentioned unified configuration center 104 can also monitor each service
Load capacity, and the load capacity for monitoring a certain service be greater than its corresponding to the first preset threshold when, to control centre send out
Send the increase service request of the quantity of the service in increase task storage cluster 102.Control centre receive increase service ask
When asking, it can correspondingly increase the service in task storage cluster.
Further, if above-mentioned unified configuration center 104 monitors that the load capacity of a certain service is corresponding less than the service
Second preset threshold can be asked to the reduction service that control centre sends the quantity of the service in reduction task storage cluster 102
It asks.Control centre can correspondingly reduce the service in task storage cluster 102 when receiving reduction service request.As it can be seen that
Above-mentioned distributed task scheduling storage system can increase on demand or delete service therein.
In some optional implementations of the present embodiment, above-mentioned unified configuration center 104 can also monitor each service
The loading condition of node, and there are load capacity to be greater than the preset service node of third in monitoring task storage cluster 102
When, the increase service node request of the quantity of service node in increase task storage cluster 102 is sent to control centre.In control
The heart can increase the service node in task storage cluster 102 when receiving increase service node request.
Further, if above-mentioned unified configuration center 104 monitors that there are load capacity less than the in task storage cluster 102
When four preset service nodes, it can subtract to the quantity that control centre sends service node in reduction task storage cluster 102
Few service node request.Control centre is when receiving reduction service node request, it is possible to reduce in task storage cluster 102
Service node.As it can be seen that above-mentioned distributed task scheduling storage system may be implemented to increase on demand or delete service node, so that task is deposited
Accumulation 102 can be with dynamic retractility, and scalability is good.It is appreciated that control centre can increase or decrease service according to demand
With the quantity of service node, the distributed task scheduling storage system is allowed to support the storage or reading of high concurrent task.
In some optional implementations of the present embodiment, above-mentioned unified configuration center 104 can also construct responsibility chain
At least one of service, Discrete service, acceleration service and Warning Service, as shown in Figure 1.Above-mentioned control centre can will be above-mentioned
Each service node of each service arrangement in task storage cluster 102.It is understood that above-mentioned unified configuration center 104 is also
The type of building service can be increased according to the demand of store tasks or reading task, for example, if the distributed task scheduling stores
System needs the quantity of statistics task, then this, which unifies configuration center 104, can also construct for counting storage or reading task
Statistical fractals.
Above-mentioned responsibility chain service is determined for the service needed in the task chain for storing or reading task, and will need
Each service be added in the task chain according to preset sequence so that each task can be with during task chain executes
State rollback or label etc. are carried out in any step.Above-mentioned gateway server 101 is also based on load-balancing algorithm for task
Storage request or task read requests are assigned to each responsibility chain service.
In some optional implementations of the present embodiment, above-mentioned distributed task scheduling storage system can also include data
Cache cluster 105.The data buffer storage cluster 105 stores the needing to store of the task to correspondence in above-mentioned task storage cluster 102
Database while, can be by each task buffer into the data buffer storage cluster 105.Also, it is cached to data buffer storage cluster
Each task in 105 can also be differently formed different distributed queues according to task type, it can by same task class
The task of type forms a distributed queue.Above-mentioned each service node is deployed in the service when receiving task read requests
Acceleration service in node can be from corresponding with the task type for needing to read task in above-mentioned data buffer storage cluster 105
The task of its caching is read in distributed queue, so as to accelerate the speed of task reading, and avoids the occurrence of same task quilt
Repeat the phenomenon that reading.Optionally, in order to keep above-mentioned data buffer storage cluster 105 and above-mentioned data persistence point library cluster 103
The synchronization of middle data, above-mentioned unified configuration center 104 can also construct relay services etc..
In some optional implementations of the present embodiment, the Warning Service for being deployed in service node can be also used for uniting
The quantity of not being read in meter distributed task scheduling storage system for task, and work as in the distributed task scheduling storage system and exist not
When the quantity of being read for task is greater than five preset thresholds, which can be by unifying configuration center 104 to user
Warning message is sent, allows user to understand system in time and overstocks and can take appropriate measures there are task.It is above-mentioned pre-
Police uniform business can be also used for the task for the abnormality for determining that distributed task scheduling storage system whether there is, and determine that there are different
When the task of normal state, which can send warning message to user by unified configuration center 104, so that user can
With the task of abnormality in understanding system in time and can take appropriate measures.For example, Warning Service can divide determining
Outdated task is deposited in cloth task storage system, and (storage is into distributed task scheduling storage system and in such as 1 day preset time
Not being read inside for task can be outdated task) when, warning message is sent to user.Here warning message can pass through postal
The forms such as part or short message show user.
In some optional implementations of the present embodiment, the needing to store of the task may include unique task mark
Know, the Discrete service being deployed on service node, can be with when detecting that task that needs store has unique task identification
The task that above-mentioned needs store discretely is stored in database corresponding with the task type of the needs store tasks, each
Unique task identification of business can guarantee that it may only once be stored, and ensure that the idempotence and nonrepeatability of task.
When above-mentioned gateway server 101 receives task read requests, the service node in task storage cluster 102 can also be direct
Task is read from corresponding database.
The distributed task scheduling storage system provided by the above embodiment of the application, gateway server 101 are receiving each industry
When the task that business system is sent stores request or task read requests, each request can be assigned to task storage cluster 102
Service node, each service node can by need the storing of the task store into data persistence point library cluster 103 with this
In the corresponding database of the task type of business and the task type of task that each service node can also be read as needed
Each task is read, so that the distributed task scheduling storage system can store the task of different business systems publication, and can be with
Correspondingly read the stored task of different business systems.
It should be noted that in gateway server 101, task storage cluster 102 in distributed task scheduling storage system
Each service node etc. can be server, and correspondingly, distributed task scheduling storage/read method is generally stored by distributed task scheduling
Each server in system executes.
Next, referring to FIG. 2, it illustrates one realities of distributed task scheduling storage/read method according to the application
Apply the flow chart of example.As shown, the distributed task scheduling storage/read method of the present embodiment may include steps of：
Step 201, gateway server receives the task requests that each operation system is sent, and the distribution of each task requests is taken office
The service node being engaged in storage cluster.
In the present embodiment, the electronic equipment of distributed task scheduling storage/read method operation thereon is (such as shown in Fig. 1
Distributed task scheduling storage system in each server), gateway server therein can be by wired connection mode or wireless
Connection type receives the task requests that each operation system is sent, and each task requests are further assigned to task storage cluster
Service node in.Here, task requests can be task storage request or task read requests, and task storage request can
To include the task type for needing store tasks, task read requests may include the task type for needing to read task.It is above-mentioned
Task storage cluster may include multiple service nodes.It should be pointed out that above-mentioned radio connection may include but unlimited
In 3G/4G connection, WiFi connection, bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, with
And other currently known or exploitation in the future radio connections.
Step 202, each service node is in response to receiving task storage request, by each task storage to data persistence point
In the cluster of library in database corresponding with the task type of the task.
In the present embodiment, when the service node in above-mentioned task storage cluster receives what above-mentioned gateway server was sent
When task storage request, the task type that service node can according to need the task of storage is stored to data persistence point
In the cluster of library in database corresponding with the task type of the task.Wherein, data persistence point library cluster may include multiple
Database, and each database can be used for storing the task of corresponding at least one task type.
Step 203, service node is in response to receiving task read requests, the task type of reading as needed for task
Read each task.
In the present embodiment, when the service node in above-mentioned task storage cluster receives what above-mentioned gateway server was sent
When task read requests, service node can determine the task type for needing to read task, then reading as needed for task
Task type read task.
Distributed task scheduling storage/read method provided by the above embodiment of the application, gateway server can will be different
The service node in task storage cluster that the task requests of operation system are sent, in order to which each service node can according to need
Storage or the task type for reading task store each task into corresponding database, so as to realize to not of the same trade or business
The task of business system publication carries out unified storage or reading.
With continued reference to FIG. 3, it illustrates according to another embodiment of the distributed task scheduling storage method of the application
Flow chart.As shown, distributed task scheduling storage method provided in this embodiment may include steps of：
Step 301, a variety of services of store tasks are used in unified configuration center building distributed task scheduling storage system, and
Set the quantity of each service.
In the present embodiment, for storing distributed task scheduling storage system (such as Fig. 1 of the task of each operation system publication
It is shown) it may include unified configuration center, this, which unifies configuration center, can construct in distributed task scheduling storage system for storing
A variety of services of task, and set the quantity of each service.
In some optional implementations of the present embodiment, above-mentioned unified configuration center can construct the service of responsibility chain,
Discrete service, Warning Service etc., and unified configuration center according to the other services of increase in demand of store tasks or can also be deleted
Except existing service.
Step 302, the control centre of task storage cluster obtains each service from unified configuration center, and is each service node
At least one service of deployment.
It in the present embodiment, may include control centre in the task storage cluster of distributed task scheduling storage system, the control
Center processed can obtain each service from above-mentioned unified configuration center, and each service is deployed on demand each in task storage cluster
Service node.After each service node disposes at least one service, above-mentioned unified configuration center can also be obtained respectively for control centre
Service the distribution situation in the task storage cluster.
Step 303, gateway server receives the task storage request that each operation system is sent, and each task is stored and is requested
The service node being assigned in task storage cluster.
In this embodiment, the gateway server in distributed task scheduling storage system can receive task from each operation system
Storage request.Then, each task received storage request can be assigned to each in task storage cluster by gateway server
Service node, in order to which each service node can store the task that above-mentioned task storage request needs to store.It is appreciated that above-mentioned
Gateway server can receive the task storage request that multiple operation systems are sent, and each task storage request can request storage more
A task, here without limitation.
In some optional implementations of the present embodiment, above-mentioned gateway server is receiving task storage request
When, security protection and authentication can also be carried out.When gateway server determines received task storage request, there is no peaces
Full hidden danger and when passing through authentication, received task can be stored each service node that request is sent by gateway server.
Step 304, gateway server is based on load-balancing algorithm and task storage request is assigned to responsibility chain service.
In the present embodiment, the unified configuration center in distributed task scheduling storage system can be constructed in advance and be used in the system
In multiple services of store tasks, such as responsibility chain service, and the quantity respectively serviced needed in the system is set.Then, task
Control centre in storage cluster can obtain each service from above-mentioned unified configuration center, and each service is deployed in task on demand
Storage cluster each service node.
In the present embodiment, the storage request of each task can be assigned to by above-mentioned gateway server based on load-balancing algorithm
The responsibility chain service being deployed on each service node stores each in order to which each service node can use responsibility chain service acquisition
Service required for business, and store each task for needing to store.Each task that above-mentioned load-balancing algorithm can store needs
It is evenly distributed to different responsibility chain services, accelerates the speed of task storage.
Step 305, task storage cluster starts the service of responsibility chain, successively calls according to chain-of-responsibilpattern pattern from each service node
Each service for store tasks.
In the present embodiment, the service node in above-mentioned task storage cluster can call responsibility chain service, according to responsibility
Chain pattern successively calls the task chain for storing the service composition store tasks that each required by task is wanted from each service node, passes through
The processing respectively serviced in task chain, each service node can store the task of each task storage request request storage.
In some optional implementations of this implementation, above-mentioned unified configuration center can construct in advance appoints for storing
Discrete service, Warning Service of business etc. are appointed so that service node can use the service of responsibility chain and call one by one on demand for storing
Each service of business.
In some optional implementations of the present embodiment, unified configuration center can be also used for monitoring and be deployed in each clothes
The load capacity for the service being engaged on node, and monitoring that be greater than service setting there are the load capacity of a certain service first is default
When threshold value, the increase service request of the quantity for the service disposed in increase task storage cluster is sent to above-mentioned control centre.
Control centre, can be on the basis for the load balancing for guaranteeing each service node in the increase service request for receiving the service
On, correspondingly increase the service disposed in task storage cluster.
Further, unified configuration center is monitoring to be less than the second of service setting there are the load capacity of a certain service
When preset threshold, unified configuration center can send the service disposed in reduction task storage cluster to above-mentioned control centre
The reduction service request of quantity.Control centre can guarantee each service section in the reduction service request for receiving the service
On the basis of the load balancing of point, the service disposed in task storage cluster is correspondingly reduced.As it can be seen that being deployed in task storage
Each service in cluster can be according to actual increase in demand or deletion, and scalability is good.
In some optional implementations of the present embodiment, unified configuration center can be also used for monitoring each service node
Non-load balanced case, and in the load imbalance for monitoring service node to control centre send transfer be deployed in service section
The transfer request of service in point, so that each service node in task storage cluster can be with load balancing.Specifically, when unified
Configuration center monitors that the load capacity of first service node is greater than the load capacity of second service node and therefore causes load uneven
When weighing apparatus, the service for being deployed in first service node can be transferred to second service node by above-mentioned control centre, so that appointing
Each service node load balancing being engaged in storage cluster.
In some optional implementations of the present embodiment, above-mentioned unified configuration center is monitoring the negative of each service node
When carrying situation, it can also be sent when the load capacity for monitoring presence service node is greater than third predetermined threshold value to control centre
Increase the increase service node request of the quantity of service node.Control centre receive increase service node request when, can be with
Service node in increase task storage cluster.It is appreciated that task storage cluster, after increasing service node, control centre can
The service disposed on other service nodes is transferred to newly-increased service node, so that each service node can be with load balancing.
Further, above-mentioned unified configuration center is in the load capacity for monitoring presence service node less than the 4th preset threshold
When, the reduction service node request for the quantity for reducing service node can be sent to control centre.Control centre subtracts receiving
When few service node request, the service node that needs are deleted can be determined, and the service disposed on the service node is transferred to
Other service nodes then delete the service node, to reduce the quantity of the service node in the task storage cluster.It can
See, the service node in task storage cluster dynamically can increase or delete according to the actual needs, and scalability is good.
Step 306, task storage cluster calls Discrete service, in response to detecting that each task is identified with sole task,
The discrete storage of each task is arrived database corresponding with the task type of the task by Discrete service.
In the present embodiment, it may include unique task identification that task storage request, which needs each task stored,.Work as clothes
When being engaged in node starting responsibility chain service according to chain-of-responsibilpattern pattern store tasks, task storage cluster can call Discrete service, root
According to the sole task mark for needing storing for task to have, data corresponding with the task type of the task are arrived into the storage of each task
In library, so as to avoid repetition store tasks.
The task of each operation system publication can divide library collection to same data persistence according to the different storages of task type
In different databases in group, and data persistence divides the quantity of the database in the cluster of library can be according to the actual needs
It increases or decreases, without multiple corresponding task memory spaces are arranged for different operation systems, improves storage money
The utilization rate in source.
The distributed task scheduling storage method provided by the above embodiment of the application, can use and dispose in task storage cluster
The service of responsibility chain and Discrete service on service node etc. arrive the task in different business systems according to task identification storage
In database corresponding with its task type, so that the task of different business systems can be stored uniformly, responsibility is called
Chain service can guarantee that task can be with rollback or label during storage according to the task identification store tasks of task, and keep away
Exempt from same task to be redundantly stored.
With continued reference to FIG. 4, it illustrates according to another embodiment of the distributed task scheduling read method of the application
Flow chart.As shown, the distributed task scheduling read method of the present embodiment may include steps of：
Step 401, for reading a variety of services of task in unified configuration center building distributed task scheduling storage system, and
Set the quantity of each service.
In the present embodiment, the unified configuration center in distributed task scheduling storage system (such as shown in Fig. 1) can be preparatory
A variety of services are constructed, and set the quantity respectively serviced for needing to dispose in task storage cluster.Above-mentioned each service can be used for from
Distributed task scheduling storage system reads stored each task, for example, unified configuration center can construct the service of responsibility chain,
Accelerate service and Warning Service etc..
Step 402, the control centre of task storage cluster obtains each service from unified configuration center, and is each service node
At least one service of deployment.
In the present embodiment, task storage cluster may include control centre, which can configure according to unified
The quantity respectively serviced that the task storage cluster of center setting needs to dispose obtains each service from unified configuration center, and to appoint
The above-mentioned at least one service of each service node deployment being engaged in storage cluster.For example, each service node can dispose responsibility chain clothes
At least one of business, acceleration service and Warning Service etc..
Step 403, gateway server receives the task read requests that each operation system is sent, and by each task read requests
The service node being assigned in task storage cluster.
In the present embodiment, the gateway server in distributed task scheduling storage system can receive what each operation system was sent
Task read requests, the task read requests can be used for reading task stored in distributed task scheduling storage system, and
And task read requests can also include the task type for needing to read task.Then, gateway server can read each task
Request is taken to be assigned to each service node in task storage cluster, in order to which each service node can according to need reading task
Task type reads task.
Step 404, gateway server is based on load-balancing algorithm and task storage request is assigned to responsibility chain service.
In the present embodiment, control centre saves the responsibility chain service arrangement that unified configuration center constructs in advance in each service
Point, above-mentioned gateway server can the algorithm based on load balancing the storage request of each task is assigned to and is deployed on service node
Responsibility chain service.
Step 405, task storage cluster starts the service of responsibility chain, successively calls according to chain-of-responsibilpattern pattern from each service node
For reading each service of task.
In the present embodiment, the service node in above-mentioned task storage cluster can call responsibility chain service, according to responsibility
Chain pattern successively calls the task chain for reading the service composition reading task that each required by task is wanted from each service node, so that
Each service node can read the task that each task read requests request is read.
It is understood that, distribution in the present embodiment identical as distributed task scheduling storage method in above-described embodiment
During formula task is read, unified configuration center can also monitor the loading condition of each service and each service node, and control
Center processed can increase or decrease the quantity of service and/or service node according to the monitoring result of unified configuration center, here not
It repeats again.
Step 406, in response to receiving task read requests, task storage cluster, which calls, accelerates service from distributed queue
Middle reading task.
It in the present embodiment, can also include data buffer storage cluster in distributed task scheduling storage system.It stores and collects in task
Group in service node receive task storage request, by each task storage to data persistence divide in the cluster of library with the task
While task type corresponding database, task can also be stored into above-mentioned data buffer storage cluster, and slow in the data
It deposits and forms distributed queue corresponding with the task type of each task in cluster.
Service node in task storage cluster can call after receiving task read requests and be deployed in a certain service
Acceleration service on node reads the task type of task from the corresponding distributed queue in data buffer storage cluster as needed
Reading task.Optionally, above-mentioned service node can also divide the database in the cluster of library to read task from data persistence, still
Compared with the database divided in the cluster of library from data persistence reads task, service node reads task from data buffer storage cluster
It can accelerate the speed of task reading.And reading task from above-mentioned distributed queue can prevent same task to be repeated reading
It takes.
In some optional implementations of the present embodiment, the service node in task storage cluster receives task
Storage request divides the storage of each task to the same of database corresponding with the task type of the task in the cluster of library to data persistence
When, each task can also be carried out local cache by above-mentioned service node, and form distribution corresponding with the task type of each task
Formula queue.Service node in task storage cluster directly can read caching from local after receiving task read requests
The purpose for accelerating reading task equally also may be implemented in task.
In some optional implementations of this example, the service node in task storage cluster, which can call, to be deployed in
Warning Service therein counts the quantity of not being read in each task being stored in distributed task scheduling storage system for task,
And when the quantity for counting on not being read for task is greater than five preset thresholds, is sent and reported to user by unified configuration center
Alert information.Further, above-mentioned service node may call upon abnormal shape in Warning Service monitoring distributed task scheduling storage system
The task of state, and determining there are when the task of abnormality, warning message is sent to user by unified configuration center.
The distributed task scheduling read method that the above embodiments of the present application provide, can use and dispose in task storage cluster
The service of responsibility chain and acceleration service on service node etc., read the task type of task, rapidly from data as needed
Corresponding task is read in cache cluster, and calls the service of responsibility chain that can guarantee that task can carry out during reading
Rollback or label.
Below with reference to Fig. 5, it illustrates the terminal device/server computers for being suitable for being used to realize the embodiment of the present application
The structural schematic diagram of system 500.Terminal device/server shown in Fig. 5 is only an example, should not be to the embodiment of the present application
Function and use scope bring any restrictions.
As shown in figure 5, computer system 500 includes central processing unit (CPU) 501, it can be read-only according to being stored in
Program in memory (ROM) 502 or be loaded into the program in random access storage device (RAM) 503 from storage section 508 and
Execute various movements appropriate and processing.In RAM 503, also it is stored with system 500 and operates required various programs and data.
CPU 501, ROM 502 and RAM 503 are connected with each other by bus 504.Input/output (I/O) interface 505 is also connected to always
Line 504.
I/O interface 505 is connected to lower component：Importation 506 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 507 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 508 including hard disk etc.；
And the communications portion 509 of the network interface card including LAN card, modem etc..Communications portion 509 via such as because
The network of spy's net executes communication process.Driver 510 is also connected to I/O interface 505 as needed.Detachable media 511, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 510, in order to read from thereon
Computer program be mounted into storage section 508 as needed.
Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description
Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium
On computer program, which includes the program code for method shown in execution flow chart.In such reality
It applies in example, which can be downloaded and installed from network by communications portion 509, and/or from detachable media
511 are mounted.When the computer program is executed by central processing unit (CPU) 501, limited in execution the present processes
Above-mentioned function.It should be noted that computer-readable medium described herein can be computer-readable signal media or
Computer readable storage medium either the two any combination.Computer readable storage medium for example can be ---
But be not limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above group
It closes.The more specific example of computer readable storage medium can include but is not limited to：With being electrically connected for one or more conducting wires
Connect, portable computer diskette, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed it is read-only
Memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory
Part or above-mentioned any appropriate combination.In this application, computer readable storage medium, which can be, any include or stores
The tangible medium of program, the program can be commanded execution system, device or device use or in connection.And
In the application, computer-readable signal media may include in a base band or the data as the propagation of carrier wave a part are believed
Number, wherein carrying computer-readable program code.The data-signal of this propagation can take various forms, including but not
It is limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be computer
Any computer-readable medium other than readable storage medium storing program for executing, the computer-readable medium can send, propagate or transmit use
In by the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to：Wirelessly, electric wire, optical cable, RF etc., Huo Zheshang
Any appropriate combination stated.
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the application, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of the module, program segment or code include one or more use
The executable instruction of the logic function as defined in realizing.It should also be noted that in some implementations as replacements, being marked in box
The function of note can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are actually
It can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it to infuse
Meaning, the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart can be with holding
The dedicated hardware based system of functions or operations as defined in row is realized, or can use specialized hardware and computer instruction
Combination realize.
As on the other hand, present invention also provides a kind of computer-readable medium, which be can be
System described in above-described embodiment included；It is also possible to individualism, and without in the supplying system.Above-mentioned calculating
Machine readable medium carries one or more program, when said one or multiple programs are executed by the system, so that should
System：Gateway server receives the task requests that each operation system is sent, and each task requests are assigned to task storage cluster
In service node, wherein task requests be task storage request or task read requests, each task requests include need to store
Or the task type of reading for task；Each service node is in response to receiving task storage request, by each task storage to data
Persistence is divided in the cluster of library in database corresponding with the task type of the task, alternatively, each service node is in response to receiving
The task type of task read requests, reading as needed for task reads each task；Wherein, data persistence divides library cluster packet
Multiple databases are included, and each database is used to store the task of corresponding task type.
Above description is only the preferred embodiment of the application and the explanation to institute's application technology principle.Those skilled in the art
Member is it should be appreciated that invention scope involved in the application, however it is not limited to technology made of the specific combination of above-mentioned technical characteristic
Scheme, while should also cover in the case where not departing from foregoing invention design, it is carried out by above-mentioned technical characteristic or its equivalent feature
Any combination and the other technical solutions formed.Such as features described above has similar function with (but being not limited to) disclosed herein
Can technical characteristic replaced mutually and the technical solution that is formed.
Claims (23)
1. a kind of distributed task scheduling storage system, which is characterized in that including：
Gateway server, the task requests sent for receiving each operation system, and each task requests are assigned to task
Service node in storage cluster, wherein the task requests are task storage request or task read requests, each task
Request includes the task type for the needing to store or read of the task；
Data persistence divides library cluster, including multiple databases, and each database is used to store appointing for corresponding task type
Business；
The task storage cluster, including multiple service nodes, the task that each service node is used to store as needed
Task type stores each task into corresponding database, and the task of the task for reading as needed
Type reads each task.
2. system according to claim 1, which is characterized in that further include unified configuration center, for constructing the distribution
A variety of services in formula task storage system, and set the number for each service that the task storage cluster needs to dispose
Amount；
The task storage cluster further includes control centre, for disposing at least one service for each service node.
3. system according to claim 2, which is characterized in that the unified configuration center is also used to monitor each service
The non-load balanced case of node, and send and shift to the control centre when monitoring the load imbalance of the service node
The transfer request of the service, so that the control centre shifts at least one the described service for being deployed in first service node
To second service node, wherein the load capacity of the first service node is greater than the load capacity of the second service node.
4. system according to claim 2, which is characterized in that the unified configuration center is also used to monitor each service
Load capacity, and monitor there are the load capacity of the service be greater than corresponding first preset threshold when, into the control
The heart sends the increase service request for increasing the quantity of the service in the task storage cluster；
The unified configuration center is also used to monitoring that there are the load capacity of the service to be less than corresponding second preset threshold
When, the reduction service request for reducing the quantity of the service in the task storage cluster is sent to the control centre.
5. system according to claim 4, which is characterized in that the unified configuration center is also used to monitor each service
The loading condition of node, and monitor there are load capacity be greater than third predetermined threshold value the service node when, to the control
Center processed sends the increase service node request for the quantity for increasing the service node；
The unified configuration center is also used to when monitoring that there are the load capacity of the service node less than four preset thresholds,
The reduction service node request for the quantity for reducing the service node is sent to the control centre.
6. system according to claim 5, which is characterized in that the control centre is also used in response to receiving the increasing
Add service request or the reduction service request, increases or decreases the service in the task storage cluster；
The control centre is also used in response to receiving the increase service node request or reduction service node request,
Increase or decrease the service node in the task storage cluster.
7. system according to claim 2, which is characterized in that it is described it is a variety of service include the service of responsibility chain, Discrete service,
Accelerate at least one of service and Warning Service；
The responsibility chain service is used to determine each service for storing or reading in the task chain of the task, and will be each described
Service is added in the task chain according to preset sequence；
The gateway server is also used to that the task requests are assigned to the responsibility chain service based on load-balancing algorithm.
8. system according to claim 7, which is characterized in that further include data buffer storage cluster, for being saved in the service
Point caches each task when storing each task into the database corresponding with the task type of the task, and
Different distributed queues is formed according to the task type of each task.
9. system according to claim 8, which is characterized in that described for accelerating service for reading as needed
The task type of business corresponding distributed queue from the data buffer storage cluster reads the task of caching.
10. system according to claim 7, which is characterized in that the Warning Service in response to counting on for not read
The quantity of the task taken is greater than the 5th preset threshold, sends warning message to user by the unified configuration center；
Warning Service is also used in response to determining there are the task of abnormality, by the configuration center of unifying to the user
Send the warning message.
11. system according to claim 7, which is characterized in that each task includes unique task identification；
The Discrete service is used in response to detecting that each task is identified with sole task, each described by need to store
Database corresponding with the task type of task described in this is arrived in the discrete storage of task.
12. a kind of distributed task scheduling storage/read method, which is characterized in that including：
Gateway server receives the task requests that each operation system is sent, and each task requests are assigned to task storage collection
Service node in group, wherein the task requests are task storage request or task read requests, each task requests packet
Include the task type for the needing to store or read of the task；
Each service node is in response to receiving the task storage request, by each task storage to data persistence point
In the cluster of library in database corresponding with the task type of the task, alternatively, each service node is described in response to receiving
The task type of task read requests, reading as needed for task reads each task；
Wherein, data persistence point library cluster includes multiple databases, and each database is for storing correspondence
The task of task type.
13. according to the method for claim 12, which is characterized in that further include：
For storing/reading a variety of services of the task in unified configuration center building distributed task scheduling storage system, and set
The quantity of fixed each service；
The control centre of the task storage cluster obtains each service from the unified configuration center, and is each service
At least one service of node deployment.
14. according to the method for claim 13, which is characterized in that the method also includes：
The unified configuration center monitors the non-load balanced case of each service node, and is monitoring the service node
The transfer request for shifting the service is sent when load imbalance to the control centre；
The control centre in response to receiving the transfer request of the service, by be deployed in first service node at least one
The service is transferred to second service node, wherein the load capacity of the first service node is greater than the second service node
Load capacity.
15. according to the method for claim 13, which is characterized in that the method also includes：
The unified configuration center monitors the load capacity of each service, and is monitoring that the load capacity there are the service is greater than
When corresponding first preset threshold, the quantity for increasing the service in the task storage cluster is sent to the control centre
Increase service request；
The unified configuration center is when monitoring to be less than corresponding second preset threshold there are the load capacity of the service, to institute
It states control centre and sends the reduction service request for reducing the quantity of the service in the task storage cluster；
The control centre increases or decreases described in response to receiving the increase service request or the reduction service request
The service disposed in task storage cluster.
16. according to the method for claim 15, which is characterized in that the method also includes：
The unified configuration center monitors the loading condition of each service node, and is monitoring that there are the service nodes
When load capacity is greater than third predetermined threshold value, the increase for sending the quantity for increasing the service node to the control centre services section
Point request；
The unified configuration center is when monitoring that there are the load capacity of the service node less than four preset thresholds, to described
Control centre sends the reduction service node request for the quantity for reducing the service node；
The control centre requests in response to receiving increase service node request or the reductions service node, increase or
Reduce the service node in the task storage cluster.
17. according to the method for claim 13, which is characterized in that a variety of services include the service of responsibility chain, discrete clothes
At least one of business, acceleration service and Warning Service；
Each service node is in response to receiving the task storage request, by each task storage to lasting data
Change divides in the cluster of library in database corresponding with the task type of the task, including：
The gateway server is based on load-balancing algorithm and task storage request is assigned to the responsibility chain service；
The task storage cluster starts the responsibility chain service, successively calls according to chain-of-responsibilpattern pattern from each service node
For storing each service of the task, by task storage to the corresponding database.
18. according to the method for claim 17, which is characterized in that each service node is described in response to receiving
Task read requests, the task type of reading as needed for task read each task, including：
The gateway server is based on load-balancing algorithm and the task read requests is assigned to the responsibility chain service；
Each service node is in response to receiving the task read requests, in the task class with the task for needing to read
The task is read in the corresponding database of type.
19. according to the method for claim 17, which is characterized in that each task includes unique task identification；
The method also includes：
The task storage cluster calls the Discrete service, in response to detecting that each task is identified with sole task,
Each discrete storage of task is arrived database corresponding with the task type of task described in this by the Discrete service.
20. according to the method for claim 17, which is characterized in that the method also includes：
In each service node in response to receiving the task storage request, by each task storage to data persistence
While dividing database corresponding with the task type of the task in the cluster of library, by each task buffer to data buffer storage collection
Group, and distributed queue corresponding with the task type of each task is formed in the data buffer storage cluster；
In response to receiving the task read requests, the task storage cluster calls the acceleration service from the distribution
The task is read in queue.
21. according to the method for claim 17, which is characterized in that further include：
The number of not being read in each task that the service node calls the Warning Service statistics stored for task
Amount, and when the quantity for counting on the task not being read is greater than five preset thresholds, pass through the unified configuration center
Warning message is sent to user；And
The service node calls the task of the Warning Service monitoring abnormality, and is determining that there are the tasks of abnormality
When, the warning message is sent to the user by the unified configuration center.
22. a kind of server, including：
One or more processors；
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
The now method as described in any in claim 12-21.
23. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
The method as described in any in claim 12-21 is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710319768.9A CN108874528A (en) | 2017-05-09 | 2017-05-09 | Distributed task scheduling storage system and distributed task scheduling storage/read method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710319768.9A CN108874528A (en) | 2017-05-09 | 2017-05-09 | Distributed task scheduling storage system and distributed task scheduling storage/read method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108874528A true CN108874528A (en) | 2018-11-23 |
Family
ID=64287796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710319768.9A Pending CN108874528A (en) | 2017-05-09 | 2017-05-09 | Distributed task scheduling storage system and distributed task scheduling storage/read method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108874528A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109561153A (en) * | 2018-12-17 | 2019-04-02 | 郑州云海信息技术有限公司 | Distributed memory system and business switch method, device, equipment, storage medium |
| CN110008750A (en) * | 2019-04-09 | 2019-07-12 | 广东绍林科技开发有限公司 | A kind of data-storage system and method with block chain distributed management characteristic |
| CN110046040A (en) * | 2019-04-09 | 2019-07-23 | 厦门网宿有限公司 | Distributed task scheduling processing method and system and storage medium |
| CN110351119A (en) * | 2019-06-06 | 2019-10-18 | 苏州元核云技术有限公司 | A kind of distributed file storage system and file memory method |
| CN110764912A (en) * | 2019-10-25 | 2020-02-07 | 东北大学 | Self-adaptive task scheduler and method |
| CN110825512A (en) * | 2019-11-08 | 2020-02-21 | 东南数字经济发展研究院 | Universal distributed system design and C + + implementation thereof |
| CN111190901A (en) * | 2019-12-12 | 2020-05-22 | 平安医疗健康管理股份有限公司 | Business data storage method and device, computer equipment and storage medium |
| CN111669327A (en) * | 2020-06-16 | 2020-09-15 | 珠海麦宁科技有限公司 | Service providing system, method, storage medium and service node |
| CN112035579A (en) * | 2019-06-03 | 2020-12-04 | 杭州海康威视数字技术股份有限公司 | Graph management method, data storage method, data query method, device and storage medium |
| CN112068956A (en) * | 2020-08-24 | 2020-12-11 | 北京首汽智行科技有限公司 | Load balancing method based on redis cache and server |
| CN112333249A (en) * | 2020-10-26 | 2021-02-05 | 新华三大数据技术有限公司 | Business service system and method |
| CN112561326A (en) * | 2020-12-15 | 2021-03-26 | 青岛海尔科技有限公司 | Task execution method and device, storage medium and electronic device |
| CN112948104A (en) * | 2019-12-11 | 2021-06-11 | 中盈优创资讯科技有限公司 | Load balancing data acquisition method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102970379A (en) * | 2012-12-19 | 2013-03-13 | 中国电子科技集团公司第十五研究所 | Method for realizing load balance among multiple servers |
| CN105069170A (en) * | 2015-08-31 | 2015-11-18 | 中国科学院遥感与数字地球研究所 | Mass spacial information storage and service integrated machine system |
| CN106302640A (en) * | 2016-07-27 | 2017-01-04 | 乐视控股（北京）有限公司 | Data request processing method and device |
| WO2017050141A1 (en) * | 2015-09-24 | 2017-03-30 | 网宿科技股份有限公司 | Distributed storage-based file delivery system and method |
-
2017
- 2017-05-09 CN CN201710319768.9A patent/CN108874528A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102970379A (en) * | 2012-12-19 | 2013-03-13 | 中国电子科技集团公司第十五研究所 | Method for realizing load balance among multiple servers |
| CN105069170A (en) * | 2015-08-31 | 2015-11-18 | 中国科学院遥感与数字地球研究所 | Mass spacial information storage and service integrated machine system |
| WO2017050141A1 (en) * | 2015-09-24 | 2017-03-30 | 网宿科技股份有限公司 | Distributed storage-based file delivery system and method |
| CN106302640A (en) * | 2016-07-27 | 2017-01-04 | 乐视控股（北京）有限公司 | Data request processing method and device |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109561153A (en) * | 2018-12-17 | 2019-04-02 | 郑州云海信息技术有限公司 | Distributed memory system and business switch method, device, equipment, storage medium |
| CN110008750B (en) * | 2019-04-09 | 2021-01-15 | 广东绍林科技开发有限公司 | Data storage system and method with block chain distributed management characteristic |
| CN110008750A (en) * | 2019-04-09 | 2019-07-12 | 广东绍林科技开发有限公司 | A kind of data-storage system and method with block chain distributed management characteristic |
| CN110046040A (en) * | 2019-04-09 | 2019-07-23 | 厦门网宿有限公司 | Distributed task scheduling processing method and system and storage medium |
| CN110046040B (en) * | 2019-04-09 | 2021-11-16 | 厦门网宿有限公司 | Distributed task processing method and system and storage medium |
| CN112035579B (en) * | 2019-06-03 | 2024-02-20 | 杭州海康威视数字技术股份有限公司 | Graph management, data storage and data query methods, devices and storage medium |
| CN112035579A (en) * | 2019-06-03 | 2020-12-04 | 杭州海康威视数字技术股份有限公司 | Graph management method, data storage method, data query method, device and storage medium |
| CN110351119A (en) * | 2019-06-06 | 2019-10-18 | 苏州元核云技术有限公司 | A kind of distributed file storage system and file memory method |
| CN110764912A (en) * | 2019-10-25 | 2020-02-07 | 东北大学 | Self-adaptive task scheduler and method |
| CN110764912B (en) * | 2019-10-25 | 2022-09-09 | 东北大学 | Self-adaptive task scheduler and method |
| CN110825512A (en) * | 2019-11-08 | 2020-02-21 | 东南数字经济发展研究院 | Universal distributed system design and C + + implementation thereof |
| CN112948104A (en) * | 2019-12-11 | 2021-06-11 | 中盈优创资讯科技有限公司 | Load balancing data acquisition method and device |
| CN112948104B (en) * | 2019-12-11 | 2024-01-05 | 中盈优创资讯科技有限公司 | Load balancing data acquisition method and device |
| CN111190901B (en) * | 2019-12-12 | 2023-02-07 | 深圳平安医疗健康科技服务有限公司 | Business data storage method and device, computer equipment and storage medium |
| CN111190901A (en) * | 2019-12-12 | 2020-05-22 | 平安医疗健康管理股份有限公司 | Business data storage method and device, computer equipment and storage medium |
| CN111669327A (en) * | 2020-06-16 | 2020-09-15 | 珠海麦宁科技有限公司 | Service providing system, method, storage medium and service node |
| CN112068956A (en) * | 2020-08-24 | 2020-12-11 | 北京首汽智行科技有限公司 | Load balancing method based on redis cache and server |
| CN112333249A (en) * | 2020-10-26 | 2021-02-05 | 新华三大数据技术有限公司 | Business service system and method |
| CN112333249B (en) * | 2020-10-26 | 2022-07-22 | 新华三大数据技术有限公司 | Business service system and method |
| CN112561326A (en) * | 2020-12-15 | 2021-03-26 | 青岛海尔科技有限公司 | Task execution method and device, storage medium and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108874528A (en) | Distributed task scheduling storage system and distributed task scheduling storage/read method | |
| CN109257320A (en) | Message storage method and device | |
| CN110365752A (en) | Processing method, device, electronic equipment and the storage medium of business datum | |
| CN109697075A (en) | File updating method, system and device | |
| CN108733476A (en) | A kind of method and apparatus executing multitask | |
| CN106227596A (en) | Mission Monitor method and apparatus for task scheduling server | |
| CN107666525A (en) | The method and apparatus of cluster container IP distribution | |
| CN109947565A (en) | Method and apparatus for distributing calculating task | |
| CN109408205A (en) | Method for scheduling task and device based on hadoop cluster | |
| CN103078759B (en) | The management method and device of calculate node, system | |
| CN110377416A (en) | Distributed subregion method for scheduling task and device | |
| CN110427304A (en) | O&M method, apparatus, electronic equipment and medium for banking system | |
| CN110046178A (en) | The method and apparatus of distributed data inquiry | |
| CN108234319A (en) | The transmission method and device of a kind of data | |
| CN108366110A (en) | A kind of website data interactive system and method | |
| CN108234638A (en) | A kind of data processing method and device based on content distributing network CDN | |
| CN108399046A (en) | File operation requests treating method and apparatus | |
| CN106686112A (en) | Cloud file transmission system and method | |
| CN110300170A (en) | Block chain distributed storage method for down loading, equipment and storage medium | |
| CN114169801A (en) | Workflow scheduling method and device | |
| CN110321242A (en) | Data processing method and device | |
| CN110399394A (en) | The data traversal method, apparatus and calculate node of calculate node | |
| CN110113176A (en) | Information synchronization method and device for configuration server | |
| CN109684059A (en) | Method and device for monitoring data | |
| CN110349076A (en) | The processing method and processing device of data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181123 |