CN113657863A - Centralized operation processing system and method thereof - Google Patents
Centralized operation processing system and method thereof Download PDFInfo
- Publication number
- CN113657863A CN113657863A CN202110969373.XA CN202110969373A CN113657863A CN 113657863 A CN113657863 A CN 113657863A CN 202110969373 A CN202110969373 A CN 202110969373A CN 113657863 A CN113657863 A CN 113657863A
- Authority
- CN
- China
- Prior art keywords
- task
- queue
- module
- service
- tasks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 80
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000003860 storage Methods 0.000 claims abstract description 16
- 238000012550 audit Methods 0.000 claims abstract description 12
- 238000012544 monitoring process Methods 0.000 claims abstract description 10
- 238000002955 isolation Methods 0.000 claims description 52
- 238000007726 management method Methods 0.000 claims description 34
- 230000008569 process Effects 0.000 claims description 22
- 238000009826 distribution Methods 0.000 claims description 9
- 238000012216 screening Methods 0.000 claims description 5
- 238000003672 processing method Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 11
- 230000007246 mechanism Effects 0.000 description 5
- 238000011084 recovery Methods 0.000 description 4
- 238000012163 sequencing technique Methods 0.000 description 4
- 238000012423 maintenance Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000007261 regionalization Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000012946 outsourcing Methods 0.000 description 1
- 238000012954 risk control Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/71—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/75—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06316—Sequencing of tasks or work
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0635—Risk analysis of enterprise or organisation activities
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Theoretical Computer Science (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Economics (AREA)
- Operations Research (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Marketing (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Game Theory and Decision Science (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a centralized operation processing system and a method thereof, the system comprises a foreground and a background which are connected with each other, the foreground is provided with a scanning module, the background is provided with a centralized sorting module, a slicing module, a centralized entry module, a rule checking module, a quit checking module, a customized service module, a hierarchical data processing module and a hierarchical storage module, the method comprises the following steps: the foreground receives the client request, scans the received data and transmits the scanned data to the background; and the background performs service input, service audit, service processing and service monitoring on the scanned data according to the service scene requirements, and returns the service processing result data to the foreground. Compared with the prior art, the invention can effectively improve the service processing efficiency and reduce the cost and risk by the intensive processing of the background service.
Description
Technical Field
The invention relates to the technical field of business data processing, in particular to a centralized operation processing system and a method thereof.
Background
The traditional operation mode mostly adopts a mode of 'network full function and teller full flow', namely a decentralized operation mode: the network node is used as the initiation and landing point of the service, and all the processes (service processing, file management, risk monitoring and operation management) of one service are completed in one network node.
However, with the continuous richness of related products and the continuous increase of transaction amount and transaction types, the types and processing flows of corresponding services are more and more complex, and the transaction complexity is also increased, so that the limitation of the traditional decentralized operation mode is more and more obvious: firstly, the business process is unreasonable, each branch organization has background operation, resources and professional skills are difficult to share, the data processing cost is high, and the business process and the operation standard are different among the branch organizations, so that the service quality and the customer experience are inconsistent; secondly, the service requirement on the foreground teller is high, and each teller must be skilled in mastering various transactions, so that the teller has high labor intensity, low working efficiency, low service processing efficiency and relatively long waiting time of clients; thirdly, risk control links are multiple, business risk points are distributed in each business outlet and each business link, risks are difficult to control, and traditional after-the-fact supervision is often poor in effect.
Disclosure of Invention
The present invention aims to overcome the above-mentioned drawbacks of the prior art and provide a centralized operation processing system and method thereof, which effectively improve the efficiency of service processing and reduce the cost and risk by performing intensive processing on background services.
The purpose of the invention can be realized by the following technical scheme: a centralized operation processing system comprises a foreground and a background which are connected with each other, wherein the foreground is provided with a scanning module, the scanning module is used for scanning data and then transmitting the scanned data to the background, the background is provided with a centralized sorting module, a slicing module, a centralized entry module, a rule checking module, a piece returning checking module and a customized service module, and the centralized sorting module is used for classifying the scanned data so as to obtain image data through screening;
the slicing module is used for identifying effective areas of the image data;
the centralized entry module is used for entering, checking and rechecking the image data;
the rule auditing module is used for comparing and confirming data consistency;
the quit auditing module is used for performing quit auditing on the service data needing to be quit;
the customized service module is used for customizing a corresponding data processing flow according to the user requirement.
Furthermore, the background is provided with a layered data processing module, the layered data processing module is respectively connected with the centralized clearing module, the slicing module, the centralized entry module, the rule auditing module, the quitting auditing module and the customizing service module, the layered data processing module comprises a service access unit, a transaction distribution unit and a task management unit, and the service access unit is used for acquiring service data to be executed; the transaction distribution unit is used for distributing specific tasks to the task management unit; the task management unit is used for managing and scheduling the public task queue and the customized task queue in a centralized mode.
Furthermore, the hierarchical data processing module is connected with a hierarchical storage module, the hierarchical storage module comprises a Redis database, a JVM memory unit and a DB database, and the Redis database is used for storing detailed information of services and tasks after service access;
the JVM memory unit is used for storing task information of a job task object in a task management layer;
and the DB database is used for storing the clearing, inputting and auditing parameterized configuration data of each service variety and the flow data after the service is finished.
Further, the task management unit comprises a queue manager, a task manager and a data accessor which are sequentially connected, wherein the task manager is connected with the JVM memory unit, the data accessor is connected with the Redis database, and the queue manager is used for dividing the queue into different areas to store tasks in different states;
the task manager is used for acquiring a task ID;
the data accessor is used for acquiring task information.
Further, the queue area comprises a hosting queue, a buffer queue, a backup queue, a hosting queue isolation area, an isolation area buffer queue and an isolation area backup queue, and the hosting queue is used for storing initialized tasks; the cache queue is used for storing cache tasks of a hosted area; the backup queue is used for storing tasks which are taken by a teller and temporarily stored; the managed queue isolation area is used for storing initialization tasks isolated by overtime; the isolation region cache queue is used for storing cache tasks of an isolation region; the isolation area backup queue is used for storing the tasks which are taken by the isolation area role teller and are temporarily stored.
A centralized operation processing method comprises the following steps:
s1, the foreground receives the client request, scans the received data and transmits the scanned data to the background;
and S2, the background performs service entry, service audit, service processing and service monitoring on the scanned data according to the service scene requirements, and returns the service processing result data to the foreground.
Further, the service entry in step S2 includes image data classification screening, slicing processing, and centralized entry, and the service audit includes rule audit and return audit.
Further, the service processing procedure in step S2 includes the following steps:
s21, dividing the queue into different areas to correspondingly store tasks in different states, wherein the tasks comprise a hosting queue, a buffer queue, a backup queue, a hosting queue isolation area, an isolation area buffer queue and an isolation area backup queue;
and S22, distributing the tasks in the queue pool, picking up the tasks from the task pool, and scheduling the tasks in the task pool.
Further, the specific process of allocating the task pool task in step S22 is as follows: initializing the task, and storing the initialized task in a hosting queue;
monitoring the task amount of a cache pool, and calling a snapshot generating service when the task amount of the cache pool is lower than a preset threshold value;
and the snapshot generating service acquires tasks in batch from the hosting queue according to the priority order and presses the tasks into the cache queue.
Further, the specific process of retrieving the task from the task pool in step S22 is as follows: according to the priority sequence, picking up the task with the highest priority from the task pool with the highest priority;
acquiring a task ID from a JVM memory;
and acquiring corresponding task information from the Redis database according to the task ID.
Further, the specific process of scheduling the task in the task pool in step S22 is as follows: moving overtime tasks in the managed queue to an isolation area;
updating the priority sequence, and reordering the tasks in the queue;
migrating all tasks corresponding to a certain service in the queue to another queue in batch;
and when the task amount of the cache queue is lower than a preset threshold value, triggering asynchronous snapshot generating service.
Compared with the prior art, the invention has the following advantages:
according to the invention, the centralized sorting module, the slicing module, the centralized entry module, the rule audit module, the return audit module and the customized service module are arranged at the background, so that data scanned by the foreground can be transmitted to the background to carry out service intensive processing, and the background is used for completing service entry, service audit, service processing and service monitoring, thereby greatly improving the automatic processing degree of the system, avoiding repeated development and scattered investment, effectively saving cost, being capable of controlling risk points from scattered to centralized, realizing unified control, reducing manual links in each service flow and greatly reducing operation risks generated by base-level and counter workers; in addition, the concentration of background operation can effectively reduce the operation cost, the service with repeated flow is extracted and transferred to the shared operation center, so that the operation processing is more standardized, the operation efficiency is improved, the error probability of the service is reduced, and the supervision and control capability is enhanced.
The invention adopts a layered data processing module and a layered storage module, the layered data processing module is respectively responsible for service access, transaction distribution and centralized management and scheduling of tasks, and each layer adopts a high-cohesion low-coupling implementation mode, thereby improving the reusability, easy maintenance and expansibility of the system; the task scheduling efficiency, the system concurrency and the throughput are improved by utilizing the hierarchical storage module; thereby further improving the efficiency of service processing.
The invention divides the queue into different areas to correspondingly store tasks in different states, thereby facilitating centralized task management and scheduling and simultaneously improving the task management efficiency; by setting the task pool isolation region, various service scenes can be met; when the invention distributes the tasks of the queue pool, the tasks are initialized in the hosted area and the tasks are received in the cache area, and the intermediate tasks are scheduled and linked through the snapshot production service, thereby reducing the system performance consumption brought by real-time sequencing and ensuring the stability of the received tasks.
Drawings
FIG. 1 is a schematic diagram of the system of the present invention;
FIG. 2 is a schematic flow diagram of the process of the present invention;
FIG. 3 is a schematic diagram of an embodiment of an application process;
FIG. 4 is a diagram illustrating a functional architecture of a background in an embodiment;
FIG. 5 is a schematic diagram of a hierarchical data processing module in an embodiment;
FIG. 6 is a diagram illustrating a hierarchical storage process according to an embodiment;
FIG. 7 is a diagram illustrating an exemplary queue structure;
FIG. 8 is a diagram illustrating task allocation of a queue pool in an embodiment;
FIG. 9 is a task pool task getting diagram in an embodiment;
FIG. 10 is a diagram illustrating task scheduling of task pools according to an embodiment;
FIG. 11 is a diagram illustrating task state changes in an embodiment;
the notation in the figure is: 1. foreground, 2, background, 101, scanning module, 201, centralized clearing module, 202, slicing module, 203, centralized entry module, 204, rule auditing module, 205, quit auditing module, 206, customized service module, 207, hierarchical data processing module, 208 and hierarchical storage module.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments.
Examples
To facilitate a clearer description of the embodiments, the following terms are first paraphrased:
task pool: the standard operation time of task nodes (namely form slices) in various business varieties is combed in advance by a certain type of queues with common or similar tasks. The standard job time length refers to the time length from the start of task generation to the submission of the entered task. And integrating a plurality of task pools according to the trend of standard operation time length, and independently setting the task pools for special service types. All task pools have the same menu entry.
Caching the pool: and a cache pool is respectively arranged below each task pool to grab the tasks in batches at variable time so as to reduce the loss of the task sequencing on the system. And the head line parametrizes and sets the upper limit and the lower limit of the tasks of the cache pool, the upper limit is the maximum value of the tasks accessed by the cache pool, the lower limit is the minimum value of the tasks accessed by the cache pool, and once the minimum value is reached, the system automatically captures the tasks in batches from the task pool. And the queuing rule of the tasks in the cache pool is consistent with that of the task pool.
The task dispatching mechanism comprises the following steps: for the difference of the same service in the aspects of processing speciality, regionality and the like, the system needs to realize the assignment of central tasks according to different service rules so as to realize independent task management, monitoring and professional cultivation of manpower.
Task priority queuing rules:
a. default rules are: and queuing according to the residual processing time (slice standard processing time-processed time) of the nodes, wherein the shorter the residual processing time is, the higher the priority is.
b. An additional rule: the sequence of each business variety can be parameterized in real time in the task pool.
Overtime task queuing rules: the overtime task refers to a task whose processed time length is longer than the standard processing time length. The timed-out task defaults to a general task, the more timed-outs, the higher the priority.
Isolation region: the method can be used for isolating overtime tasks of partial service types in real time through parameterization setting, temporarily not processing the overtime tasks, and also can be used for arranging a teller in the authority of the isolation area to process the tasks of the isolation area, and meanwhile, the center can open a valve of the isolation area at any time to recycle and process the tasks.
A task recovery mechanism: each task has the property of the overtime recovery time length in the task pool, and can be parameterized in advance, and once the processed time length of the task exceeds the preset overtime recovery time length, the system automatically recovers the task. And the task is automatically released to the task pool and then has the highest priority.
A snapshot management module: the method is mainly used for acquiring the current most urgent TOPN tasks from the task pool and storing the tasks into the snapshot pool, thereby reducing the system performance consumption brought by real-time sequencing.
Standard working hours: refers to the standard length of time from task pickup to task submission, which is generally related to the workload and complexity of the task.
The wait-able duration: the time length from task generation to task extraction is generally related to the timeliness of the service, and the waiting time length of each task is unified for one service type.
Standard time length: refers to the standard processing time from task generation to task completion submission, which is equal to standard man-hour + waitable time.
The overtime recovery time is as follows: the system automatically recycles the task once the time length is exceeded, and the time length is longer than the standard time length.
As shown in fig. 1, a centralized operation processing system includes a foreground 1 and a background 2 that are connected to each other, the foreground 1 is provided with a scanning module 101, the scanning module 101 is used for scanning and transmitting data to the background, the background 2 is provided with a centralized sorting module 201, a slicing module 202, a centralized entry module 203, a rule auditing module 204, a quit auditing module 205, a customized service module 206, a hierarchical data processing module 207 and a hierarchical storage module 208, wherein the hierarchical data processing module 207 is respectively connected with the centralized sorting module 201, the slicing module 202, the centralized entry module 203, the rule auditing module 204, the quit auditing module 205, the customized service module 206 and the hierarchical storage module 208;
the centralized sorting module 201 is used for sorting the scanned data to obtain image data by screening;
the slicing module 202 is used for identifying an effective area of the image data;
the centralized entry module 203 is used for entering, checking and rechecking the image data;
the rule auditing module 204 is used for comparing and confirming data consistency;
the quit auditing module 205 is used for performing quit auditing on the service data needing to be quit;
the customization service module 206 is used for customizing a corresponding data processing flow according to the user requirement;
the hierarchical data processing module 207 comprises a service access unit, a transaction distribution unit and a task management unit, wherein the service access unit is used for acquiring service data to be executed; the transaction distribution unit is used for distributing specific tasks to the task management unit; the task management unit is used for managing and scheduling a public task queue and customizing a task queue in a centralized mode, and comprises a queue manager, a task manager and a data accessor which are sequentially connected, wherein the task manager is connected with a JVM memory unit, the data accessor is connected with a Redis database, and the queue manager is used for dividing the queue into different areas to store tasks in different states: the queue area comprises a hosting queue, a buffer queue, a backup queue, a hosting queue isolation area, an isolation area buffer queue and an isolation area backup queue, wherein the hosting queue is used for storing initialized tasks; the buffer queue is used for storing buffer tasks of the hosted area; the backup queue is used for storing tasks which are taken by a teller and temporarily stored; the managed queue isolation area is used for storing the initialization tasks isolated by overtime; the isolation region cache queue is used for storing cache tasks of the isolation region; the isolation area backup queue is used for storing tasks which are taken by the isolation area role teller and are temporarily stored;
the task manager is used for acquiring a task ID;
the data accessor is used for acquiring task information. (ii) a
The hierarchical storage module 208 comprises a Redis database, a JVM memory unit and a DB database, wherein the Redis database is used for storing detailed information of services and tasks after the services are accessed;
the JVM memory unit is used for storing task information of a job task object in the task management layer;
and the DB database is used for storing the clearing, inputting and auditing parameterized configuration data of each service variety and the flow data after the service is finished.
The above system is applied to practice to realize a centralized operation job processing method, as shown in fig. 2, including the following steps:
s1, the foreground receives the client request, scans the received data and transmits the scanned data to the background;
and S2, the background performs service input (including image data classification screening, slicing processing and centralized input processes), service audit (including rule audit and quit audit processes), service processing and service monitoring on the scanned data according to the service scene requirements, and returns the service processing result data to the foreground.
The service processing procedure in step S2 includes the following steps:
s21, dividing the queue into different areas to correspondingly store tasks in different states, wherein the tasks comprise a hosting queue, a buffer queue, a backup queue, a hosting queue isolation area, an isolation area buffer queue and an isolation area backup queue;
s22, allocating the tasks in the queue pool, getting the tasks from the task pool, and scheduling the tasks in the task pool, wherein the specific process of allocating the tasks in the task pool is as follows: initializing the task, and storing the initialized task in a hosting queue;
monitoring the task amount of a cache pool, and calling a snapshot generating service when the task amount of the cache pool is lower than a preset threshold value;
the snapshot generating service acquires tasks in batch from the hosting queue according to the priority order and presses the tasks into a cache queue;
the specific process of picking up the tasks from the task pool comprises the following steps: according to the priority sequence, picking up the task with the highest priority from the task pool with the highest priority;
acquiring a task ID from a JVM memory;
acquiring corresponding task information from a Redis database according to the task ID;
the specific process of scheduling the tasks in the task pool comprises the following steps: moving overtime tasks in the managed queue to an isolation area;
updating the priority sequence, and reordering the tasks in the queue;
migrating all tasks corresponding to a certain service in the queue to another queue in batch;
and when the task amount of the cache queue is lower than a preset threshold value, triggering asynchronous snapshot generating service.
By applying the technical scheme, the existing business processing flow is decomposed into a mode that a website is used as a foreground to initiate, a head office is used as a background to perform centralized processing, namely, the business is cut between the foreground and the background, each link only processes one part of the business, and meanwhile, the processing of each link is simplified and standardized as much as possible so as to reduce the labor capacity of the website and use low-cost personnel or perform outsourcing on part of processing links. As shown in fig. 3, the centralized common business process is unified and integrated:
1, the centralized service uniformly scans data through a network scanning post and then enters central centralized processing;
2, entering a central clearing post to classify the data types of the image data;
3, according to the type of the data sorted by the sorting post, whether the data enter the slicing post to identify the effective area of the image data can be selected;
4, entering a centralized recording service of the center to realize centralized recording of two-recording, one-correcting and one-rechecking of the image data;
5, for different column entry values needing to be compared, entering a central rule checking post for data consistency comparison and confirmation;
and 6, if the service finally has a refund, entering a central refund auditing post to carry out uniform refund auditing treatment.
The embodiment constructs the integral framework of the centralized job processing platform at the background:
1. service publicization: as shown in fig. 4, the common operation part of the centralized service is abstracted into a plurality of individual public services, a uniform service is provided for each service system in the form of a service component, the service can be freely combined according to the actual scene, a real service-oriented architecture is realized, the service expansion and the integration of the central post are facilitated, and the centralized auditing module and the central uniform task engine module can customize corresponding services according to the user requirements.
2. Parameter configuration: the centralized operation platform is currently accessed to thirty types of services, such as clearing, entering and quitting service parameter pre-configuration, the front-end page performs unified information display and logic control, the parameters are only required to be configured for the access of new services, and no change is required for background flow control and front-end information display.
3. Layering a system: as shown in fig. 5, the centralized operation platform architecture system adopts a hierarchical structure, and is divided into a service access layer, a transaction processing layer, and a task management layer. And the system is respectively responsible for service access, transaction distribution and centralized management and scheduling of tasks. Each layer adopts a high-cohesion low-coupling implementation mode, and the reusability, easy maintenance and expansibility of the system are improved.
4. Storage regionalization: as shown in fig. 6, the centralized job platform adopts a hierarchical storage manner, and stores the hierarchical storage manner in the Redis memory database, the JVM memory area, and the DB2 database, respectively.
Redis in-memory database: the method is used for storing detailed information of the service and the task after the service is accessed, and the interaction speed with the program is improved.
JVM memory area: the task management method is used for storing the task information of the job task object in the task management layer and improving the management and scheduling efficiency of the task.
DB2 database: the method is used for storing parameterized configuration data of various service varieties such as score clearing and entry and the like and flow data after the service is finished.
The queue structure is designed as follows:
a) the queue structure adopts a regionalization design. As shown in fig. 7, a physical queue is actually divided into six areas in the memory, which are a managed queue, a cache queue, a backup queue, a managed queue isolation area, an isolation area cache queue, and an isolation area backup queue. The system is used for storing initialized tasks, cache tasks of a hosting area, tasks which are taken by teller and are temporarily stored, initialized tasks which are isolated overtime, cache tasks of an isolation area and tasks which are taken by teller and are temporarily stored by the role of the isolation area. The queues are divided into different areas to store tasks in different states, responsibility of all the departments is not affected mutually, and meanwhile, the departments are closely connected, centralized task management and scheduling are facilitated, and task management efficiency is improved.
b) And allocating the tasks of the queue pool. As shown in fig. 8, all tasks that enter the centralized job processing platform management are initially placed in the managed queue. And when the task amount of the cache pool is lower than the lower limit, calling a snapshot generating service, wherein the snapshot generating service acquires tasks in batches from the escrow queue according to the priority and presses the tasks into the cache queue for the teller to get. The tasks are initialized in the hosted area and the tasks are received in the cache area, and intermediate tasks are scheduled and linked through the snapshot production service, so that the system performance consumption caused by real-time sequencing is reduced, and meanwhile, the stability of the tasks received by the teller is ensured.
c) And (4) task acquisition of the task pool. As shown in fig. 9, a task with the highest priority in the task pool with the highest priority is retrieved according to teller permission, and in order to improve efficiency and convenience of task pool management, task objects in the task pool are stored in an application JVM memory and only contain pure task information of the task (that is, only contain relevant information for task identification and task ordering, such as task ID, generation time, and waiting time). After the task management layer receives the task, the service access layer obtains corresponding detailed task information from a Redis memory database according to the task ID, and then the detailed task information is returned to the teller operation page to display the task.
d) And scheduling tasks in the task pool. As shown in fig. 10, after the service accesses the centralized operation platform, the task is initialized and stored in the managed queue through operation allocation, and then the task may enter the isolation area or the cache pool in different manners. The task acquisition is completely transparent to the teller, the teller does not need to pay attention to the authority of the teller, the task can be acquired by the next time, and the task can be processed after the task pushed by the system is received. The teller authority is adjusted by the EUIF system, so that the task processing efficiency is improved, and the manpower resource scheduling of the center is facilitated.
And (4) isolating overtime (manual triggering), moving the overtime task in the managed queue to an isolation area, and storing the overtime task in the managed queue if the overtime task is not manually triggered.
Priority adjustment (manual triggering), resetting of business variety priority, re-ordering of tasks in queue according to priority
And (4) batch migration (manual triggering), wherein all tasks of a certain business variety in the queue are migrated to another queue.
And (4) snapshot generation (system triggering), and when the task quantity of the cache queue is lower than the lower limit, triggering asynchronous snapshot generation service.
e) A task state mechanism. As shown in fig. 11, the task has the following states:
ready: the state of the initialized task, at this time, the task is carried out to a queue hosted area or a buffer area or is not pressed into the queue;
operation: when the task is picked up, the task enters a queue backup area;
isolation: after the task is put into the overtime isolation, the task is in the state of the isolation area;
and (3) completing: status when task is completed; completion tasks no longer exist in the queue;
exception: a state when the task continuously times out more than a certain number of times; the exception task no longer exists in the queue.
In conclusion, the technical scheme adopts a unified common node menu inlet, and can automatically push processing tasks; setting a plurality of priority queuing mechanisms to support flexible management of a center; a task pool isolation area is arranged, so that various service scenes can be met; the data regional storage is adopted, so that the task scheduling efficiency, the system concurrency and the throughput are improved; through configuration of service parameters, dynamic display of tasks can be achieved, development cost is reduced, and service access time is shortened. According to the technical scheme, the situation that different current business processing rules are not uniform and are difficult to be matched with a background standard processing mode direction is considered, so that management pressure is caused; the background task queuing mechanism is not perfect enough, and currently, the auditing rules for each service entry are not uniform; in order to standardize the processing rules of the centralized operation business and unify the processing modes of various business varieties, the technical scheme can strengthen the unified management of the centralized business by uniformly processing the background centralized operation, thereby assisting the center to better finish the distribution and the dispatching of manpower, improving the production efficiency and reducing the operation and maintenance cost of the center.
Claims (10)
1. The centralized operation processing system is characterized by comprising a foreground (1) and a background (2) which are connected with each other, wherein the foreground (1) is provided with a scanning module (101), the scanning module (101) is used for scanning and transmitting data to the background (2), the background (2) is provided with a centralized sorting module (201), a slicing module (202), a centralized entry module (203), a rule checking module (204), a piece returning checking module (205) and a customized service module (206), and the centralized sorting module (201) is used for classifying the scanned data to obtain image data through screening;
the slicing module (202) is used for identifying effective areas of the image data;
the centralized entry module (203) is used for entering, checking and rechecking the image data;
the rule auditing module (204) is used for comparing and confirming data consistency;
the quit auditing module (205) is used for performing quit auditing on the service data needing to be quit;
the customization service module (206) is used for customizing the corresponding data processing flow according to the user requirement.
2. The centralized operation processing system according to claim 1, wherein the background (2) is provided with a hierarchical data processing module (207), the hierarchical data processing module (207) is respectively connected with a centralized sorting module (201), a slicing module (202), a centralized entry module (203), a rule auditing module (204), a quitting auditing module (205), and a customization service module (206), the hierarchical data processing module (207) comprises a service access unit, a transaction distribution unit, and a task management unit, and the service access unit is used for acquiring service data to be executed; the transaction distribution unit is used for distributing specific tasks to the task management unit; the task management unit is used for managing and scheduling the public task queue and the customized task queue in a centralized mode.
3. The centralized operation processing system according to claim 2, wherein the hierarchical data processing module (207) is connected with a hierarchical storage module (208), the hierarchical storage module (208) comprises a Redis database, a JVM memory unit and a DB database, the Redis database is used for storing detailed information of services and tasks after service access;
the JVM memory unit is used for storing task information of a job task object in a task management layer;
and the DB database is used for storing the clearing, inputting and auditing parameterized configuration data of each service variety and the flow data after the service is finished.
4. The centralized operational job processing system according to claim 2, wherein the task management unit comprises a queue manager, a task manager and a data accessor, which are connected in sequence, the task manager is connected with the JVM memory unit, the data accessor is connected with a Redis database, and the queue manager is configured to divide a queue into different regions to store tasks in different states;
the task manager is used for acquiring a task ID;
the data accessor is used for acquiring task information.
5. The centralized operational job processing system of claim 4, wherein the queue areas comprise managed queues, buffer queues, backup queues, managed queue isolation zones, isolation zone buffer queues, isolation zone backup queues, and the managed queues are used for storing initialized tasks; the cache queue is used for storing cache tasks of a hosted area; the backup queue is used for storing tasks which are taken by a teller and temporarily stored; the managed queue isolation area is used for storing initialization tasks isolated by overtime; the isolation region cache queue is used for storing cache tasks of an isolation region; the isolation area backup queue is used for storing the tasks which are taken by the isolation area role teller and are temporarily stored.
6. A centralized operation job processing method using the centralized operation job processing system according to claim 1, comprising the steps of:
s1, the foreground receives the client request, scans the received data and transmits the scanned data to the background;
and S2, the background performs service entry, service audit, service processing and service monitoring on the scanned data according to the service scene requirements, and returns the service processing result data to the foreground.
7. The method according to claim 6, wherein the business process procedure in step S2 includes the following steps:
s21, dividing the queue into different areas to correspondingly store tasks in different states, wherein the tasks comprise a hosting queue, a buffer queue, a backup queue, a hosting queue isolation area, an isolation area buffer queue and an isolation area backup queue;
and S22, distributing the tasks in the queue pool, picking up the tasks from the task pool, and scheduling the tasks in the task pool.
8. The method according to claim 7, wherein the specific process of allocating the task pool task in step S22 is as follows: initializing the task, and storing the initialized task in a hosting queue;
monitoring the task amount of a cache pool, and calling a snapshot generating service when the task amount of the cache pool is lower than a preset threshold value;
and the snapshot generating service acquires tasks in batch from the hosting queue according to the priority order and presses the tasks into the cache queue.
9. The method according to claim 8, wherein the specific process of retrieving tasks from the task pool in step S22 is as follows: according to the priority sequence, picking up the task with the highest priority from the task pool with the highest priority;
acquiring a task ID from a JVM memory;
and acquiring corresponding task information from the Redis database according to the task ID.
10. The method according to claim 9, wherein the specific process of scheduling the tasks in the task pool in step S22 is as follows: moving overtime tasks in the managed queue to an isolation area;
updating the priority sequence, and reordering the tasks in the queue;
migrating all tasks corresponding to a certain service in the queue to another queue in batch;
and when the task amount of the cache queue is lower than a preset threshold value, triggering asynchronous snapshot generating service.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110969373.XA CN113657863A (en) | 2021-08-23 | 2021-08-23 | Centralized operation processing system and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110969373.XA CN113657863A (en) | 2021-08-23 | 2021-08-23 | Centralized operation processing system and method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113657863A true CN113657863A (en) | 2021-11-16 |
Family
ID=78492033
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110969373.XA Pending CN113657863A (en) | 2021-08-23 | 2021-08-23 | Centralized operation processing system and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113657863A (en) |
Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103365711A (en) * | 2013-07-03 | 2013-10-23 | 南京邮电大学 | Task scheduling mechanism and method applied to business platform of internet of Things |
| CN103473696A (en) * | 2013-09-03 | 2013-12-25 | 周吉 | Method and system for collecting, analyzing and distributing internet business information |
| CN104243581A (en) * | 2014-09-15 | 2014-12-24 | 成都北纬航信网络科技有限责任公司 | Multi-level cache method and system achieving high cohesion and low coupling |
| CN105553699A (en) * | 2015-12-09 | 2016-05-04 | 东北大学 | Wind turbine generator set state monitoring private cloud system and method built from point of data processing |
| CN106127571A (en) * | 2016-06-17 | 2016-11-16 | 深圳市雁联计算系统有限公司 | A kind of lightness operation processing method and processing device |
| CN109492040A (en) * | 2018-11-06 | 2019-03-19 | 深圳航天智慧城市系统技术研究院有限公司 | A kind of system suitable for data center's magnanimity short message data processing |
| CN111190798A (en) * | 2020-01-03 | 2020-05-22 | 苏宁云计算有限公司 | Service data monitoring and warning device and method |
| CN111190963A (en) * | 2019-12-27 | 2020-05-22 | 中思博安科技（北京）有限公司 | Block chain application back-end system |
| CN111292179A (en) * | 2020-02-13 | 2020-06-16 | 中国建设银行股份有限公司 | Counter service processing system, method, equipment and storage medium |
| CN111382177A (en) * | 2020-03-09 | 2020-07-07 | 中国邮政储蓄银行股份有限公司 | Service data task processing method, device and system |
| CN111639911A (en) * | 2020-05-27 | 2020-09-08 | 中信银行股份有限公司 | Asset hosting instruction online processing method and device, storage medium and electronic equipment |
| CN111967822A (en) * | 2020-08-13 | 2020-11-20 | 中铁工程装备集团技术服务有限公司 | WeChat public number-based shield machine service platform, service method and framework |
| CN111970529A (en) * | 2020-08-21 | 2020-11-20 | 网易（杭州）网络有限公司 | Information processing method and device for live broadcast room, storage medium and electronic device |
| CN112214338A (en) * | 2020-11-05 | 2021-01-12 | 北京科东电力控制系统有限责任公司 | Internet of things cloud platform based on flexible deployment of micro-services |
| CN112598038A (en) * | 2020-12-15 | 2021-04-02 | 中国建设银行股份有限公司 | Method and system for centralized drawing and processing of bills in same city |
| CN112734529A (en) * | 2021-01-13 | 2021-04-30 | 上海群之脉信息科技有限公司 | Retail intelligent operation system |
-
2021
- 2021-08-23 CN CN202110969373.XA patent/CN113657863A/en active Pending
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103365711A (en) * | 2013-07-03 | 2013-10-23 | 南京邮电大学 | Task scheduling mechanism and method applied to business platform of internet of Things |
| CN103473696A (en) * | 2013-09-03 | 2013-12-25 | 周吉 | Method and system for collecting, analyzing and distributing internet business information |
| CN104243581A (en) * | 2014-09-15 | 2014-12-24 | 成都北纬航信网络科技有限责任公司 | Multi-level cache method and system achieving high cohesion and low coupling |
| CN105553699A (en) * | 2015-12-09 | 2016-05-04 | 东北大学 | Wind turbine generator set state monitoring private cloud system and method built from point of data processing |
| CN106127571A (en) * | 2016-06-17 | 2016-11-16 | 深圳市雁联计算系统有限公司 | A kind of lightness operation processing method and processing device |
| CN109492040A (en) * | 2018-11-06 | 2019-03-19 | 深圳航天智慧城市系统技术研究院有限公司 | A kind of system suitable for data center's magnanimity short message data processing |
| CN111190963A (en) * | 2019-12-27 | 2020-05-22 | 中思博安科技（北京）有限公司 | Block chain application back-end system |
| CN111190798A (en) * | 2020-01-03 | 2020-05-22 | 苏宁云计算有限公司 | Service data monitoring and warning device and method |
| CN111292179A (en) * | 2020-02-13 | 2020-06-16 | 中国建设银行股份有限公司 | Counter service processing system, method, equipment and storage medium |
| CN111382177A (en) * | 2020-03-09 | 2020-07-07 | 中国邮政储蓄银行股份有限公司 | Service data task processing method, device and system |
| CN111639911A (en) * | 2020-05-27 | 2020-09-08 | 中信银行股份有限公司 | Asset hosting instruction online processing method and device, storage medium and electronic equipment |
| CN111967822A (en) * | 2020-08-13 | 2020-11-20 | 中铁工程装备集团技术服务有限公司 | WeChat public number-based shield machine service platform, service method and framework |
| CN111970529A (en) * | 2020-08-21 | 2020-11-20 | 网易（杭州）网络有限公司 | Information processing method and device for live broadcast room, storage medium and electronic device |
| CN112214338A (en) * | 2020-11-05 | 2021-01-12 | 北京科东电力控制系统有限责任公司 | Internet of things cloud platform based on flexible deployment of micro-services |
| CN112598038A (en) * | 2020-12-15 | 2021-04-02 | 中国建设银行股份有限公司 | Method and system for centralized drawing and processing of bills in same city |
| CN112734529A (en) * | 2021-01-13 | 2021-04-30 | 上海群之脉信息科技有限公司 | Retail intelligent operation system |
Non-Patent Citations (9)
| Title |
|---|
| 俞俊;院仁杰;袁晓东;: "高性能数据拯救系统DataSafe的设计", 通信电源技术, no. 09 * |
| 张明龙, 冯博琴, 刘芳: "区分服务的动态队列管理策略", 计算机工程与应用, no. 30 * |
| 张晓丹;高姗姗;: "重组柜面业务流程 促进网点营销服务转型", 金融电子化, no. 02 * |
| 张波;: "重庆图书馆新馆网络信息系统的解决方案", 科技情报开发与经济, no. 15 * |
| 林跃东;: "组件配置技术在中间业务处理平台的应用", 华南金融电脑, no. 03 * |
| 胡皓鹏;孙勇;阙华坤;: "超大规模计量自动化系统高效集中采集技术研究", 工业控制计算机, no. 12 * |
| 赵艳;: "商业银行反洗钱业务的信息化系统设计――以H行为例", 航空财会, no. 02 * |
| 郝正腾: "银行业后台业务集中运营的问题与对策研究", 中国乡镇企业会计, no. 05, pages 229 - 230 * |
| 陈飞;吴旭春;纪建鑫;: "运营集中平台构建运营体系新格局", 金融电子化, no. 09 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100444121C (en) | Batch task scheduling engine and dispatching method | |
| CN105354762B (en) | Power customer service work order identification and distribution system and method | |
| CN111061788B (en) | Multi-source heterogeneous data conversion integration system based on cloud architecture and implementation method thereof | |
| US8219431B2 (en) | Workflow management system, method and device for managing a workflow including plural hierarchically-classified tasks | |
| US5295065A (en) | Resource-lot association coordinator | |
| US20020062239A1 (en) | Program planning management system | |
| AU2003204420B2 (en) | Systems and methods for work list prediction | |
| JPH04232566A (en) | Method and system for controlling allotment of resource to lot | |
| CN111125444A (en) | Big data task scheduling management method, device, equipment and storage medium | |
| CN107968802A (en) | The method, apparatus and filtering type scheduler of a kind of scheduling of resource | |
| CN101082970A (en) | Method for realizing government affairs examination and approval workflow engines system | |
| CN110351384A (en) | Big data platform method for managing resource, device, equipment and readable storage medium storing program for executing | |
| CN103701934A (en) | Resource optimal scheduling method and virtual machine host machine optimal selection method | |
| CN113741883B (en) | RPA lightweight data middling station system | |
| CN112801607A (en) | Management service platform and construction method | |
| CN108268546B (en) | Method and device for optimizing database | |
| CN113298503A (en) | Government affair-oriented workflow management system and database and table dividing method thereof | |
| CN109902919A (en) | Server assets management method, device, equipment and readable storage medium storing program for executing | |
| CN106910004A (en) | A kind of overall process blower fan workmanship monitoring system based on workflow | |
| CN112200484A (en) | Intelligent case division system and method based on strategy definition | |
| CN107544844A (en) | A kind of method and device of lifting Spark Operating ettectiveness | |
| CN113657863A (en) | Centralized operation processing system and method thereof | |
| CN101212342A (en) | Multi-version network element network management method | |
| CN116051062A (en) | Online IT resource allocation and optimization system and method | |
| CN115756308A (en) | Optimization method and system for cloud data migration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |