CN111309469A - Method for unified management and scheduling of data requests - Google Patents
Method for unified management and scheduling of data requests Download PDFInfo
- Publication number
- CN111309469A CN111309469A CN202010123691.XA CN202010123691A CN111309469A CN 111309469 A CN111309469 A CN 111309469A CN 202010123691 A CN202010123691 A CN 202010123691A CN 111309469 A CN111309469 A CN 111309469A
- Authority
- CN
- China
- Prior art keywords
- data
- data request
- dispatching platform
- identification number
- scheduling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
- G06F9/524—Deadlock detection or avoidance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Abstract
The invention provides a method for uniformly managing and scheduling data requests, which comprises the following steps: s1, the data request dispatching platform provides a uniform data request inlet; s2, the data request dispatching platform extracts the identification number of the processing data and stores the identification number into the intermediate table; s3, the data request dispatching platform extracts data from the intermediate table; and S4, the data request dispatching platform puts the request package into a message queue according to the data identification number and waits for processing. The interface protection system based on the Redis distributed lock, which is used for uniformly managing and scheduling the data request, can effectively reduce the burden of a user on accessing the interface in a large batch on the service, and logically avoids the possibility that the user intentionally calls the interface to swipe tickets, crawl data information and participate in activities. The problem that the service function cannot be performed due to excessive resource occupation caused by deadlock of a common lock is solved.
Description
Technical Field
The invention belongs to the field of data scheduling, and particularly relates to a method for uniformly managing and scheduling data requests.
Background
At present, many companies adopt a unified scheduling scheme to solve the problem of data scheduling under a complex system, a platform monitors requests and the like of a service system, a url carries a parameter message to call an interface or execute tasks in the form of mails, and each time, the tasks are processed to a thread pool of a single machine through load balancing to execute subsequent operations.
Summary of the invention
In view of the above, the present invention provides a method for performing unified management and scheduling on data requests, so as to solve the problems in the prior art that a bottleneck is met depending on a single-machine resource processing capability and data operations are easy to dirty read and write when a data amount is too large and requests are too frequent. .
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
a method for carrying out unified management scheduling on data requests comprises the following steps:
s1, the data request dispatching platform provides a uniform data request inlet;
s2, the data request dispatching platform extracts the identification number of the processing data and stores the identification number into the intermediate table;
s3, the data request dispatching platform extracts data from the intermediate table;
and S4, the data request dispatching platform puts the request package into a message queue according to the data identification number and waits for processing.
Further, in S1, the server provides a unified data request entry for limiting the backend service to process only requests from the unified entry, and ensuring that all requests can be managed.
Further, S2 includes the following steps: and the data request dispatching platform compares the extracted identification number with the record table, and stores the type and time information of the request into an intermediate table after the verification is passed.
Further, when the data request scheduling platform processes the message in S4, the service sharing lock is acquired according to the message identification number.
Further, after the data request scheduling platform message processing is completed in S4, a record is recorded in the processing history table, and the shared lock is released.
Compared with the prior art, the method for uniformly managing and scheduling the data requests has the following advantages that:
the interface protection system based on the Redis distributed lock, which is used for uniformly managing and scheduling the data request, can effectively reduce the burden of a user on accessing the interface in a large batch on the service, and logically avoids the possibility that the user intentionally calls the interface to swipe tickets, crawl data information and participate in activities. The problem that the service function cannot be performed due to excessive resource occupation caused by deadlock of a common lock is solved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the invention without limitation. In the drawings:
fig. 1 is a flowchart illustrating a method for performing unified management and scheduling on data requests according to an embodiment of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments of the present invention may be combined with each other without conflict.
In the description of the invention, it is to be noted that, unless otherwise explicitly specified or limited, the terms "mounted", "connected" and "connected" are to be construed broadly, e.g. as being fixed or detachable or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the creation of the present invention can be understood by those of ordinary skill in the art through specific situations.
The invention will be described in detail with reference to the following embodiments with reference to the attached drawings.
As shown in fig. 1, a method for performing unified management scheduling on data requests includes the following steps:
s1, the data request dispatching platform provides a uniform data request inlet;
s2, the data request dispatching platform extracts the identification number of the processing data and stores the identification number into the intermediate table;
s3, the data request dispatching platform extracts data from the intermediate table;
and S4, the data request dispatching platform puts the request package into a message queue according to the data identification number and waits for processing.
In S1, the server provides a unified data request entry for restricting the back-end service to only handle requests from the unified entry, ensuring that all requests can be managed.
The step of S2 further includes: and the data request dispatching platform compares the extracted identification number with the record table, and stores the type and time information of the request into an intermediate table after the verification is passed.
And when the data request dispatching platform processes the message in the S4, acquiring the service sharing lock according to the message identification number.
And when the data request scheduling platform message processing is completed in the S4, recording and recording in the processing history table, and releasing the sharing lock.
The interface protection system based on the Redis distributed lock can effectively reduce the burden of a user on service caused by accessing the interface in a large batch, and can logically prevent the possibility that the user intentionally calls the interface to swipe tickets, crawl data information and participate in activities. The problem that the service function cannot be performed due to excessive resource occupation caused by deadlock of a common lock is solved.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the invention, so that any modifications, equivalents, improvements and the like, which are within the spirit and principle of the present invention, should be included in the scope of the present invention.
Claims (5)
1. A method for unified management scheduling of data requests is characterized by comprising the following steps:
s1, the data request dispatching platform provides a uniform data request inlet;
s2, the data request dispatching platform extracts the identification number of the processing data and stores the identification number into the intermediate table;
s3, the data request dispatching platform extracts data from the intermediate table;
and S4, the data request dispatching platform puts the request package into a message queue according to the data identification number and waits for processing.
2. The method of claim 1, wherein the method comprises: in S1, the server provides a unified data request entry for restricting the back-end service to only handle requests from the unified entry, ensuring that all requests can be managed.
3. The method of claim 1, wherein the method comprises: the step of S2 further includes: and the data request dispatching platform compares the extracted identification number with the record table, and stores the type and time information of the request into an intermediate table after the verification is passed.
4. The method of claim 1, wherein the method comprises: and when the data request dispatching platform processes the message in the S4, acquiring the service sharing lock according to the message identification number.
5. The method of claim 1, wherein the method comprises: and when the data request scheduling platform message processing is completed in the S4, recording and recording in the processing history table, and releasing the sharing lock.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010123691.XA CN111309469A (en) | 2020-02-27 | 2020-02-27 | Method for unified management and scheduling of data requests |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010123691.XA CN111309469A (en) | 2020-02-27 | 2020-02-27 | Method for unified management and scheduling of data requests |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111309469A true CN111309469A (en) | 2020-06-19 |
Family
ID=71145213
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010123691.XA Pending CN111309469A (en) | 2020-02-27 | 2020-02-27 | Method for unified management and scheduling of data requests |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111309469A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112035525A (en) * | 2020-08-25 | 2020-12-04 | 上海中通吉网络技术有限公司 | Bill verification and cancellation method, device, equipment and storage medium for express international business |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101478439A (en) * | 2009-01-20 | 2009-07-08 | 中兴通讯股份有限公司 | Uniform service management platform, interface machine, service processing system and process method |
| CN107633025A (en) * | 2017-08-30 | 2018-01-26 | 苏州朗动网络科技有限公司 | Big data business processing system and method |
| US10327098B1 (en) * | 2018-04-30 | 2019-06-18 | Paypal, Inc. | Locations platform for managing and providing of user experiences |
| CN110471774A (en) * | 2019-06-28 | 2019-11-19 | 苏宁云计算有限公司 | A kind of data processing method and device based on unified task schedule |
| CN110489222A (en) * | 2019-08-22 | 2019-11-22 | 深圳前海微众银行股份有限公司 | Method for scheduling task, system, cluster server and readable storage medium storing program for executing |
-
2020
- 2020-02-27 CN CN202010123691.XA patent/CN111309469A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101478439A (en) * | 2009-01-20 | 2009-07-08 | 中兴通讯股份有限公司 | Uniform service management platform, interface machine, service processing system and process method |
| CN107633025A (en) * | 2017-08-30 | 2018-01-26 | 苏州朗动网络科技有限公司 | Big data business processing system and method |
| US10327098B1 (en) * | 2018-04-30 | 2019-06-18 | Paypal, Inc. | Locations platform for managing and providing of user experiences |
| CN110471774A (en) * | 2019-06-28 | 2019-11-19 | 苏宁云计算有限公司 | A kind of data processing method and device based on unified task schedule |
| CN110489222A (en) * | 2019-08-22 | 2019-11-22 | 深圳前海微众银行股份有限公司 | Method for scheduling task, system, cluster server and readable storage medium storing program for executing |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112035525A (en) * | 2020-08-25 | 2020-12-04 | 上海中通吉网络技术有限公司 | Bill verification and cancellation method, device, equipment and storage medium for express international business |
| CN112035525B (en) * | 2020-08-25 | 2024-04-09 | 上海中通吉网络技术有限公司 | Bill verification method, device, equipment and storage medium for express international business |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107729139B (en) | Method and device for concurrently acquiring resources | |
| CN101501645B (en) | Computer micro-jobs | |
| US8156226B2 (en) | Ordering provisioning request execution based on service level agreement and customer entitlement | |
| CN107665234A (en) | Method for processing business, device, server and storage medium | |
| CN105700937A (en) | Multi-thread task processing method and device | |
| US8874754B2 (en) | Load balancing in handwritten signature authentication systems | |
| US9535749B2 (en) | Methods for managing work load bursts and devices thereof | |
| CN111290917A (en) | YARN-based resource monitoring method and device and terminal equipment | |
| CN108984125B (en) | Resource allocation method, device and storage medium | |
| CN103036946A (en) | Method and system for processing file backup on cloud platform | |
| CN1975655B (en) | Method and apparatus for managing access to storage | |
| CN101719852B (en) | Method and device for monitoring performance of middleware | |
| CN109901918B (en) | Method and device for processing overtime task | |
| CN107766160A (en) | Queue message processing method and terminal device | |
| CN107515781B (en) | Deterministic task scheduling and load balancing system based on multiple processors | |
| CN108280150A (en) | A kind of distribution asynchronous service distribution method and system | |
| CN108255607A (en) | Task processing method, device, electric terminal and readable storage medium storing program for executing | |
| CN107943584A (en) | The processing method and processing device of batch transaction request | |
| CN111078436A (en) | Data processing method, device, equipment and storage medium | |
| CN108616668B (en) | Method for monitoring dialing task, electronic device and computer readable storage medium | |
| CN111309469A (en) | Method for unified management and scheduling of data requests | |
| CN112132462A (en) | Customer service dispatching method, device, equipment and storage medium | |
| CN103677975A (en) | System and method for having access to internal objects of application systems | |
| CN108021486B (en) | The method and storage medium that electronic device, collage-credit data are handled | |
| CN107015855A (en) | A kind of asynchronous service centralized dispatching method and device for supporting time parameter method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200619 |