CN116308850A - Account checking method, account checking system, account checking server and storage medium - Google Patents
Account checking method, account checking system, account checking server and storage medium Download PDFInfo
- Publication number
- CN116308850A CN116308850A CN202310565184.5A CN202310565184A CN116308850A CN 116308850 A CN116308850 A CN 116308850A CN 202310565184 A CN202310565184 A CN 202310565184A CN 116308850 A CN116308850 A CN 116308850A
- Authority
- CN
- China
- Prior art keywords
- reconciliation
- file
- task
- thread
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000007781 pre-processing Methods 0.000 claims abstract description 47
- 230000011218 segmentation Effects 0.000 claims abstract description 47
- 230000000977 initiatory effect Effects 0.000 claims abstract description 33
- 230000003203 everyday effect Effects 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 2
- 238000000638 solvent extraction Methods 0.000 claims 3
- 238000004364 calculation method Methods 0.000 description 7
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000018109 developmental process Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 1
- FFBHFFJDDLITSX-UHFFFAOYSA-N benzyl N-[2-hydroxy-4-(3-oxomorpholin-4-yl)phenyl]carbamate Chemical compound OC1=C(NC(=O)OCC2=CC=CC=C2)C=CC(=C1)N1CCOCC1=O FFBHFFJDDLITSX-UHFFFAOYSA-N 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/12—Accounting
- G06Q40/125—Finance or payroll
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention provides a checking method, a checking system, a checking server and a storage medium, wherein the checking method of the invention can generate a processor thread allocation strategy and a checking file segmentation strategy in advance based on resource configuration information of the server; after receiving the reconciliation task request, initiating a file preprocessing thread according to a processor thread allocation policy; paging and dividing a source accounting file based on an accounting file dividing strategy through a file preprocessing thread; when checking accounts, initiating a plurality of checking account task threads according to a processor thread allocation strategy and generating a summary task thread; processing the reconciliation tasks of the paging reconciliation sub-files through a plurality of reconciliation task threads; and finally, summarizing the account checking results of a plurality of paging account checking sub-files through a summarizing task thread. According to the invention, the source account checking file is divided into a plurality of independent account checking files based on the hardware capability of the server, and the account checking tasks are processed simultaneously through concurrent tasks, so that the account checking efficiency is improved.
Description
Technical Field
The invention relates to the technical field of payment reconciliation, in particular to a reconciliation method, a reconciliation system, a reconciliation server and a storage medium.
Background
In large enterprises, as the enterprises develop, different bank payments are accessed, and for each payment, account checking is required on the next day so as to cut funds with the banks. With the development of enterprises, business expansion and more bank payment access are performed, the transaction scale is greatly increased, and the existing single file sequential reconciliation method is not suitable for the reconciliation processing of large-batch reconciliation files, and the reconciliation efficiency is too low.
Accordingly, the prior art is still in need of improvement and development.
Disclosure of Invention
The main object of the present invention is to solve the technical problems mentioned in the background art in the prior art.
The first aspect of the present invention provides a reconciliation method, which includes:
acquiring resource configuration information of a server in advance, and generating a processor thread allocation strategy and a reconciliation file segmentation strategy based on the resource configuration information;
receiving a reconciliation task request initiated by a client, and acquiring a source reconciliation file corresponding to the reconciliation task request;
initiating a file preprocessing thread according to the processor thread allocation strategy;
paging and dividing the source reconciliation file based on the reconciliation file division policy through the file preprocessing thread to obtain a plurality of paging reconciliation sub-files;
initiating a plurality of reconciliation task threads according to the processor thread allocation policy and generating a summary task thread;
processing the reconciliation tasks of the paging reconciliation sub-files through a plurality of reconciliation task threads;
and summarizing the reconciliation results of the paging reconciliation sub-files through the summarization task thread to generate a reconciliation report of the source reconciliation file.
In an optional implementation manner of the first aspect of the present invention, the pre-obtaining resource configuration information of the server, and generating a processor thread allocation policy and a reconciliation file segmentation policy based on the resource configuration information includes:
acquiring CPU core quantity information in the resource configuration information of the server;
calculating the number of tasks which can be processed simultaneously by the server based on the CPU core number information;
establishing a simulation program, and obtaining average time consumption of each step of the checking complete flow by starting a thread to simulate the complete flow of checking;
thread allocation proportions of the steps of the reconciliation whole process are determined based on the average time consumption of the steps of the reconciliation whole process.
In an optional implementation manner of the first aspect of the present invention, the pre-obtaining resource configuration information of the server, and generating the processor thread allocation policy and the reconciliation file segmentation policy based on the resource configuration information further includes:
acquiring the average character length of each account checking record of a source account checking file counted by historical data;
calculating the average character length of each account checking record of a source account checking file based on historical data statistics to obtain the average occupied memory of each account checking record;
acquiring available memory information in the resource configuration information of the server;
calculating the account checking record number which can be processed by the server memory at the same time based on the average occupied memory and the available memory information of each account checking record;
and taking the account checking record quantity which can be processed simultaneously in the server memory as the dividing basis of the account checking file.
In an optional implementation manner of the first aspect of the present invention, the pre-obtaining resource configuration information of the server, and generating the processor thread allocation policy and the reconciliation file segmentation policy based on the resource configuration information further includes:
acquiring the average file size of a single source account checking file counted by historical data;
acquiring available storage space information in the resource configuration information of the server;
the number of source ledgers that a server can process simultaneously is determined based on the average file size of the single source ledgers of the available storage space information and historical data statistics.
In an optional implementation manner of the first aspect of the present invention, the performing, by the file preprocessing thread, paging segmentation on the source reconciliation file based on the reconciliation file segmentation policy, to obtain a plurality of paging reconciliation subfiles includes:
acquiring the account checking record quantity which can be processed simultaneously in a server memory;
acquiring a preset memory occupation redundancy condition;
calculating the reasonable account checking record number of the paging account checking sub-file based on the memory occupation redundancy condition and the account checking record number which can be processed by the memory of the server at the same time;
and performing paging segmentation on the source account checking file based on the reasonable account checking record quantity to obtain a plurality of paging account checking sub-files.
In an optional implementation manner of the first aspect of the present invention, the initiating a plurality of reconciliation task threads and generating one summary task thread according to the processor thread allocation policy includes:
acquiring the thread allocation proportion of each step of the reconciliation whole process;
acquiring the total number of threads which can be started by a CPU of the server;
calculating the total number of threads of the reconciliation task based on the total number of threads which can be started by the CPU of the server and the thread allocation proportion of each step of the reconciliation full flow;
obtaining the configuration number of the reconciliation task threads by subtracting 1 from the total number of threads;
initiating the configured number of reconciliation task threads and generating one summary task thread.
In an optional implementation manner of the first aspect of the present invention, after the initiating a file preprocessing thread according to the processor thread allocation policy, before the performing, by the file preprocessing thread, paging segmentation on the source reconciliation file based on the reconciliation file segmentation policy, obtaining a plurality of paging reconciliation subfiles includes:
generating a source reconciliation file task according to the configuration of the file downloading query every day;
checking whether the source reconciliation file task can be executed or not at fixed time intervals, and if so, modifying the source reconciliation file task to be processed;
and checking whether the source reconciliation file task to be processed exists or not, and sequentially distributing the source reconciliation file task to be processed to the file preprocessing thread for processing.
A second aspect of the present invention provides a reconciliation system comprising:
the policy generation module is used for acquiring resource configuration information of the server in advance and generating a processor thread allocation policy and a reconciliation file segmentation policy based on the resource configuration information;
the request receiving module is used for receiving a reconciliation task request initiated by a client and obtaining a source reconciliation file corresponding to the reconciliation task request;
the preprocessing thread initiating module is used for initiating a file preprocessing thread according to the processor thread allocation strategy;
the file segmentation module is used for carrying out paging segmentation on the source reconciliation file based on the reconciliation file segmentation strategy through the file preprocessing thread to obtain a plurality of paging reconciliation sub-files;
the accounting thread initiating module is used for initiating a plurality of accounting task threads according to the processor thread allocation strategy and generating a summary task thread;
the reconciliation task execution module is used for simultaneously processing the reconciliation tasks of the paging reconciliation sub-files through a plurality of reconciliation task threads;
and the reconciliation task summarizing module is used for summarizing the reconciliation results of the paging reconciliation sub-files through the summarizing task thread to generate a reconciliation report of the source reconciliation files.
A third aspect of the present invention provides a reconciliation server comprising: a memory and at least one processor, the memory having instructions stored therein, the memory and the at least one processor being interconnected by a line;
the at least one processor invokes the instructions in the memory to cause the reconciliation server to perform the reconciliation method as defined in any one of the preceding claims.
A fourth aspect of the invention provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements a reconciliation method as defined in any of the preceding claims.
The beneficial effects are that: the invention provides a checking method, a checking system, a checking server and a storage medium, wherein the checking method of the invention can generate a processor thread allocation strategy and a checking file segmentation strategy in advance based on resource configuration information of the server; after receiving the reconciliation task request, initiating a file preprocessing thread according to a processor thread allocation policy; paging and dividing a source accounting file based on an accounting file dividing strategy through a file preprocessing thread; when checking accounts, initiating a plurality of checking account task threads according to a processor thread allocation strategy and generating a summary task thread; processing the reconciliation tasks of the paging reconciliation sub-files through a plurality of reconciliation task threads; and finally, summarizing the account checking results of a plurality of paging account checking sub-files through a summarizing task thread. According to the invention, the source account checking file is divided into a plurality of independent account checking files based on the hardware capability of the server, and the account checking tasks are processed simultaneously through concurrent tasks, so that the account checking efficiency is improved.
Drawings
FIG. 1 is a schematic diagram of one embodiment of a reconciliation method of the present invention;
FIG. 2 is a schematic diagram of one embodiment of a reconciliation system of the present invention;
FIG. 3 is a diagram of one embodiment of a reconciliation server of the present invention.
Detailed Description
It should be noted in advance that the terms "first," "second," "third," "fourth," and the like in the description and in the claims of the invention and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.
For easy understanding, the following describes a specific flow of an embodiment of the present invention, referring to fig. 1, and the first aspect of the present invention provides a reconciliation method, where the reconciliation method includes:
s100, acquiring resource configuration information of a server in advance, and generating a processor thread allocation strategy and a reconciliation file segmentation strategy based on the resource configuration information; the invention analyzes and calculates the number of tasks which can be processed by the server at the same time based on the resource configuration information, the number of accounting data which can be processed by the server at the same time, the accounting file segmentation size and the number of accounting files which can be processed by the server at the same time, and then obtains a processor thread allocation strategy and an accounting file segmentation strategy based on the number of tasks which can be processed by the server at the same time, the number of accounting data which can be processed by the server at the same time, the accounting file segmentation size and the number of accounting files which can be processed by the server at the same time;
s200, receiving a reconciliation task request initiated by a client, and acquiring a source reconciliation file corresponding to the reconciliation task request; in the invention, the account checking task request initiated by the client can be one or a plurality of account checking task requests initiated simultaneously, and the source account checking file corresponding to the request can be a plurality of account checking files;
s300, initiating a file preprocessing thread according to the processor thread allocation strategy; in the invention, the file preprocessing thread is mainly used for preprocessing the source account checking file, when the source account checking file has a plurality of copies, the preprocessed content comprises the steps of dividing the source account checking file, carrying out similar merging processing on a plurality of source account checking files in advance, and initiating the file preprocessing thread to mainly determine the number of the file preprocessing threads according to the thread allocation strategy of the processor;
s400, performing paging segmentation on the source reconciliation file based on the reconciliation file segmentation strategy through the file preprocessing thread to obtain a plurality of paging reconciliation sub-files; in the invention, the number of the file preprocessing threads is multiple, when the number of the source reconciliation files is only 1, the plurality of the file preprocessing threads process 1 source reconciliation file together, when the number of the source reconciliation files is also multiple, the plurality of the source reconciliation files are sequentially distributed to each file preprocessing thread, the rest of the source reconciliation files wait for the paging processing after the file preprocessing threads process the distributed source reconciliation files, and the reconciliation file segmentation strategy is mainly to determine the number of data strips on each page reconciliation file in the paging process of the source reconciliation files;
s500, initiating a plurality of reconciliation task threads according to the processor thread allocation policy and generating a summary task thread; in the invention, when data checking is carried out, the paging checking sub-files are respectively processed simultaneously through a plurality of checking task threads, and the checking result of each checking task thread is collected through a summarizing task thread; in terms of task configuration level of miscellaneous thread allocation, the step of establishing the reconciliation task in step S500 is to firstly establish a source reconciliation file processing task for task status summarization, generate a sub-file for each page according to the divided page data, record the sub-file task, and establish a relationship between the sub-file task and the source reconciliation file task after the whole content of the source reconciliation file is divided;
s600, processing the reconciliation tasks of the paging reconciliation sub-files through a plurality of reconciliation task threads; in the invention, a plurality of paging accounting sub-files are also sequentially distributed to the accounting task threads of the trunk, and each accounting record in the paging accounting sub-files is read by the accounting task threads in an HTTP mode;
and S700, summarizing the reconciliation results of the paging reconciliation sub-files through the summarization task thread, and generating a reconciliation report of the source reconciliation file. In the invention, when a plurality of source reconciliation files exist, a plurality of reconciliation task threads and 1 summary task thread which are private for each source reconciliation file exist.
In terms of task level, the step of step S700 is to read the sub-file task, and update the source reconciliation file task to be executing and update the execution status of the sub-file task to be executing; and paging and reading the contents of the subfiles, checking, wherein each checking record is in a normal service state for checking success and failure, and represents checking completion, and if checking operation is abnormal due to database or code problems, the checking operation represents checking failure. After each account checking record is checked, updating the execution state of the account checking record to be completed, checking that all the execution states of the account checking records in the sub-files are completed, and updating the execution state of the sub-tasks to be completed after the execution of the sub-task account checking file is completed. Checking that the reconciliation states of all the reconciliation records in the sub-files are complete, updating the sub-file task state is successful, if one of the reconciliation records fails, updating the sub-file task state is failed, deleting the sub-files in the sub-tasks according to the abnormal information of the reconciliation records, and reducing the occupation of the hard disk space.
And the summarizing task thread is used for scanning the sub-file task states, starting summarizing all sub-file task reconciliation states when the sub-file task execution states of the same source reconciliation file task relationships are completed, updating the source reconciliation file task reconciliation state to be successful if all the sub-file task reconciliation states are successful, updating the source reconciliation file task reconciliation state to be failed if one sub-file task reconciliation state is failed, and recording all abnormal information of the failed sub-file tasks to the source reconciliation file task. For redundancy processing, a task processing timeout may be set, after the source accounting file task starts processing, after the execution time exceeds the timeout, and the task state is not updated to be clear success or failure, the task may be reset, and the waiting step may be re-executed.
According to the method, the system and the device, the problems that the accounting files are bigger and bigger along with the business scale development and the accounting operation is slower and bigger are solved by processing the file bill segmentation and accounting tasks in a mode, automatic retry and triggering of the tasks are supported, and the execution state, the accounting state and abnormal information of the accounting tasks of the source accounting files can be checked in real time.
In an optional implementation manner of the first aspect of the present invention, the pre-obtaining resource configuration information of the server, and generating a processor thread allocation policy and a reconciliation file segmentation policy based on the resource configuration information includes:
acquiring CPU core quantity information in the resource configuration information of the server; calculating the number of tasks which can be processed simultaneously by the server based on the CPU core number information; establishing a simulation program, and obtaining average time consumption of each step of the checking complete flow by starting a thread to simulate the complete flow of checking; thread allocation proportions of the steps of the reconciliation whole process are determined based on the average time consumption of the steps of the reconciliation whole process.
In the step of the invention, for example, assuming that the number of CPU cores of the server is 4, a single CPU core simultaneously processes only one thread, considering the time-consuming blocking of I/O, setting the over-frequency of the number of service threads as CPU core number which is 8 threads, establishing a simulation program, requiring the fixed thread to be 1, randomly selecting a plurality of source account checking files, using a synchronous mode and embedding points to measure and calculate the time consumption, wherein the time consumption of each source account checking file is 1:3 after the file preprocessing (including file merging and file dividing) and the file account checking steps; so 2 threads are allocated to perform file preprocessing and file splitting operations and 6 threads are allocated to perform reconciliation tasks.
In an optional implementation manner of the first aspect of the present invention, the pre-obtaining resource configuration information of the server, and generating the processor thread allocation policy and the reconciliation file segmentation policy based on the resource configuration information further includes:
acquiring the average character length of each account checking record of a source account checking file counted by historical data; calculating the average character length of each account checking record of a source account checking file based on historical data statistics to obtain the average occupied memory of each account checking record; acquiring available memory information in the resource configuration information of the server; calculating the account checking record number which can be processed by the server memory at the same time based on the average occupied memory and the available memory information of each account checking record; and taking the account checking record quantity which can be processed simultaneously in the server memory as the dividing basis of the account checking file.
In the present invention, the server available memory information is mainly used for calculating the number of the accounting records that the server can process at the same time, for example, in this step of the present invention, it is assumed that the server available memory information M is 16G, according to the average character length L of one accounting record of the source accounting file counted by the history, according to the calculation formula of the memory occupied by the character string
Calculating to obtain an average occupied memory K of 500B for the statement, reserving partial resources for the server to treat other conditions, and only using the available memory with the partial proportion J of 80% after the server is started, wherein the occupied memory with the occupied coefficient P of 3 times is required for the task of processing the file, so that the total simultaneous processing of the memory can be calculated>
Stripe data, i.e.)>
Stripe data.
In an optional implementation manner of the first aspect of the present invention, the pre-obtaining resource configuration information of the server, and generating the processor thread allocation policy and the reconciliation file segmentation policy based on the resource configuration information further includes:
acquiring the average file size of a single source account checking file counted by historical data; acquiring available storage space information in the resource configuration information of the server; the number of source ledgers that a server can process simultaneously is determined based on the average file size of the single source ledgers of the available storage space information and historical data statistics.
In the invention, the available storage space information of the server is mainly used for calculating the quantity of the source accounting files which can be processed by the server at the same time (namely, calculating the maximum quantity of the segmented accounting files which can be processed by the server at the same time), in the step, the condition of determining the quantity X of the source accounting files is that the condition of assuming that the available storage space information of the server is N and the size of the source accounting files to be processed is M
。
In an optional implementation manner of the first aspect of the present invention, the performing, by the file preprocessing thread, paging segmentation on the source reconciliation file based on the reconciliation file segmentation policy, to obtain a plurality of paging reconciliation subfiles includes:
acquiring the account checking record quantity which can be processed simultaneously in a server memory; acquiring a preset memory occupation redundancy condition; calculating the reasonable account checking record number of the paging account checking sub-file based on the memory occupation redundancy condition and the account checking record number which can be processed by the memory of the server at the same time; and performing paging segmentation on the source account checking file based on the reasonable account checking record quantity to obtain a plurality of paging account checking sub-files.
In the invention, the document accounting thread N1 (comprising a plurality of accounting task threads and a summary task thread) distributed according to the processor thread distribution strategy is utilized to execute the accounting tasks, the total memory of the server obtained by the previous calculation can be processed and recorded in common and divided by the accounting document thread N1, and the memory occupation is halved for redundancy, namely
Is->
Stripe data, soThe number of accounting records processed simultaneously by each thread is +.>
The number of accounting records which can be processed simultaneously for each thread is +.>
And the strip data distributes the number of the reconciliation records recorded in each page of the source reconciliation file.
In an optional implementation manner of the first aspect of the present invention, the initiating a plurality of reconciliation task threads and generating one summary task thread according to the processor thread allocation policy includes:
acquiring the thread allocation proportion of each step of the reconciliation whole process; acquiring the total number of threads which can be started by a CPU of the server; calculating the total number of threads of the reconciliation task based on the total number of threads which can be started by the CPU of the server and the thread allocation proportion of each step of the reconciliation full flow; obtaining the configuration number of the reconciliation task threads by subtracting 1 from the total number of threads; initiating the configured number of reconciliation task threads and generating one summary task thread.
In the invention, each step of the full reconciliation process mainly comprises document preprocessing (comprising the merging of similar reconciliation documents and the segmentation of the reconciliation documents after merging) and document reconciliation; each source account checking file exemplified before is subjected to file downloading file merging, file segmentation and file account checking steps, average time consumption is taken, the time consumption is approximately 1:3, and the number of service threads is set as
Namely 8 threads, the total number of threads of the file reconciliation steps in the whole reconciliation process can be calculated to be 6, and 5 threads in the 6 threads are used for the reconciliation processing of the page reconciliation subfiles of each page, and the other 1 threads are used for collecting the reconciliation results of the 5 reconciliation threads.
In an optional implementation manner of the first aspect of the present invention, after the initiating a file preprocessing thread according to the processor thread allocation policy, before the performing, by the file preprocessing thread, paging segmentation on the source reconciliation file based on the reconciliation file segmentation policy, obtaining a plurality of paging reconciliation subfiles includes:
generating a source reconciliation file task according to the configuration of the file downloading query every day; checking whether the source reconciliation file task can be executed or not at fixed time intervals, and if so, modifying the source reconciliation file task to be processed; and checking whether the source reconciliation file task to be processed exists or not, and sequentially distributing the source reconciliation file task to be processed to the file preprocessing thread for processing. In the invention, the source account checking file task to be processed is distributed to the file preprocessing thread in sequence for processing, namely the source account checking file corresponding to the source account checking file task to be processed is sequentially transmitted to the file preprocessing thread for processing.
Briefly, the process of the reconciliation method of the present invention can be understood as: s1: collecting server resource information; s2: calculating the size of the paging file and the concurrency of the reconciliation task according to the server resource; s3: file preprocessing and file segmentation; s4: generating an account checking task; s5: executing a reconciliation task; s6: and (5) summarizing task results.
Step S2, calculating the number of tasks which can be processed simultaneously according to the size of the paging file and the concurrency of the reconciliation tasks by the acquired number of CPU cores of the server and a calculation formula in the server resource calculation; calculating the file segmentation size and the upper limit of the account checking record which is processed at most simultaneously when the task is processed through the acquired available memory capacity of the server; through the available hard disk space of the collected server, the number of segmented reconciliation documents can be calculated at most, that is to say, the number of source reconciliation documents can be processed simultaneously.
The file preprocessing in step S3 includes: task configuration of different business reconciliation requirements, task execution. Downloading and decrypting source account checking files of different formats and sources; data processing of the source account checking files needing to be combined; the step S3 of dividing the text comprises dividing the file according to the calculated file division size; step S4, generating a reconciliation task, namely generating the reconciliation task from the divided files, generating a summary task of the source reconciliation file, and carrying out task association; step S5, executing the reconciliation task, and executing the paging reconciliation task and the summarizing task through threads; and summarizing the task results, namely summarizing the tasks of the source reconciliation document after the task processing of the segmented document is completed.
Referring to fig. 2, a second aspect of the present invention provides a reconciliation system comprising:
the policy generation module 10 is configured to obtain resource configuration information of a server in advance, and generate a processor thread allocation policy and a reconciliation file segmentation policy based on the resource configuration information;
the request receiving module 20 is configured to receive a reconciliation task request initiated by a client, and obtain a source reconciliation file corresponding to the reconciliation task request;
a preprocessing thread initiating module 30, configured to initiate a file preprocessing thread according to the processor thread allocation policy;
the file segmentation module 40 is configured to perform, by using the file preprocessing thread, page segmentation on the source reconciliation file based on the reconciliation file segmentation policy, to obtain a plurality of page reconciliation sub-files;
the reconciliation thread initiating module 50 is configured to initiate a plurality of reconciliation task threads according to the processor thread allocation policy and generate a summary task thread;
the reconciliation task execution module 60 is configured to process the reconciliation tasks of the paged reconciliation sub-files simultaneously through a plurality of reconciliation task threads;
and the reconciliation task summary module 70 is configured to summary the reconciliation results of the paged reconciliation sub-files through the summary task thread, and generate a reconciliation report of the source reconciliation file.
In an alternative embodiment of the second aspect of the present invention, the policy generation module 10 includes:
a CPU information obtaining unit configured to obtain CPU core number information in the resource configuration information of the server;
the processing task number acquisition unit is used for calculating the number of the simultaneous processing tasks of the server based on the CPU core number information;
the checking simulation unit is used for establishing a simulation program, and simulating the whole checking process by starting a thread to obtain the average time consumption of each step of the whole checking process;
the thread proportion obtaining unit is used for determining the thread allocation proportion of each step of the reconciliation whole process based on the average time consumption of each step of the reconciliation whole process.
In an alternative embodiment of the second aspect of the present invention, the policy generation module 10 further includes:
the average length acquisition unit of the account checking records is used for acquiring the average character length of each account checking record of the source account checking file counted by the historical data;
the account checking record memory occupation obtaining unit is used for calculating the average character length of each account checking record of the source account checking file based on historical data statistics to obtain the average occupied memory of each account checking record;
a server available memory information obtaining unit, configured to obtain available memory information in the resource configuration information of the server;
the account checking record processing quantity obtaining unit is used for calculating the account checking record quantity which can be processed by the server memory at the same time based on the average occupied memory and the available memory information of each account checking record; and taking the account checking record quantity which can be processed simultaneously in the server memory as the dividing basis of the account checking file.
In an alternative embodiment of the second aspect of the present invention, the policy generation module 10 further includes:
the source account checking file average file size acquisition unit is used for acquiring the average file size of a single source account checking file counted by historical data;
a server available storage space acquiring unit, configured to acquire available storage space information in the resource configuration information of the server;
and the file processing quantity acquisition unit is used for determining the quantity of the source reconciliation files which can be processed by the server at the same time based on the average file size of the single source reconciliation files counted by the available storage space information and the historical data.
In an alternative embodiment of the second aspect of the present invention, the file splitting module 40 includes:
the account checking record quantity loading unit is used for acquiring the account checking record quantity which can be processed simultaneously in the memory of the server;
the redundancy condition acquisition unit is used for acquiring preset memory occupation redundancy conditions;
the reasonable account checking record number calculation unit is used for calculating the reasonable account checking record number of the paging account checking sub-file based on the memory occupation redundancy condition and the account checking record number which can be processed by the memory of the server at the same time;
and the file paging unit is used for paging and dividing the source account checking files based on the reasonable account checking record quantity to obtain a plurality of paging account checking sub-files.
In an alternative embodiment of the second aspect of the present invention, the reconciliation thread initiation module 50 comprises:
the allocation proportion loading unit is used for acquiring the thread allocation proportion of each step of the reconciliation whole flow;
the CPU thread total number acquisition unit is used for acquiring the total number of threads which can be started by the CPU of the server;
the accounting thread total number calculation unit is used for calculating the total number of threads of the accounting task based on the total number of threads which can be started by the CPU of the server and the thread allocation proportion of each step of the accounting full flow;
the accounting task thread number calculation unit is used for obtaining the configuration number of the accounting task threads through subtracting 1 from the total number of threads;
and the thread initiating unit is used for initiating the reconciliation task threads with the configuration quantity and generating one summarization task thread.
In an alternative embodiment of the second aspect of the present invention, the reconciliation system further comprises:
the source account checking file task generating module is used for generating a source account checking file task according to the configuration of file downloading inquiry every day;
the source account checking file task state changing module is used for checking whether the source account checking file task can be executed or not at fixed time intervals, and if so, modifying the source account checking file task to be processed;
and the source account checking file task allocation module is used for checking whether the source account checking file task to be processed exists or not, and sequentially allocating the source account checking file task to be processed to the file preprocessing thread for processing.
Fig. 3 is a schematic diagram of a reconciliation server according to an embodiment of the invention, which may vary considerably in configuration or performance, and may include one or more processors 80 (central processing units, CPU) (e.g., one or more processors) and memory 90, one or more storage media 100 (e.g., one or more mass storage devices) storing applications or data. The memory and storage medium may be transitory or persistent. The program stored on the storage medium may include one or more modules (not shown), each of which may include a series of instruction operations in the reconciliation server. Still further, the processor may be configured to communicate with a storage medium and execute a series of instruction operations in the storage medium on the flower manuscript generation.
The accounting server of the present invention may also include one or more power supplies 110, one or more wired or wireless network interfaces 120, one or more input/output interfaces 130, and/or one or more operating systems, such as Windows Serve, mac OS X, unix, linux, freeBSD, etc. It will be appreciated by those skilled in the art that the reconciliation server structure illustrated in fig. 3 is not limiting in detail and may include more or fewer components than illustrated, or certain components in combination, or a different arrangement of components.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, or may be a volatile computer readable storage medium, having stored therein instructions that, when executed on a computer, cause the computer to perform the steps of the reconciliation method.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the system or the unit described above may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a read-only memory (ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method of reconciliation, the method comprising:
acquiring resource configuration information of a server in advance, and generating a processor thread allocation strategy and a reconciliation file segmentation strategy based on the resource configuration information;
receiving a reconciliation task request initiated by a client, and acquiring a source reconciliation file corresponding to the reconciliation task request;
initiating a file preprocessing thread according to the processor thread allocation strategy;
paging and dividing the source reconciliation file based on the reconciliation file division policy through the file preprocessing thread to obtain a plurality of paging reconciliation sub-files;
initiating a plurality of reconciliation task threads according to the processor thread allocation policy and generating a summary task thread;
processing the reconciliation tasks of the paging reconciliation sub-files through a plurality of reconciliation task threads;
and summarizing the reconciliation results of the paging reconciliation sub-files through the summarization task thread to generate a reconciliation report of the source reconciliation file.
2. The reconciliation method of claim 1, wherein the pre-obtaining the resource configuration information of the server and generating the processor thread allocation policy and the reconciliation file partitioning policy based on the resource configuration information comprises:
acquiring CPU core quantity information in the resource configuration information of the server;
calculating the number of tasks which can be processed simultaneously by the server based on the CPU core number information;
establishing a simulation program, and obtaining average time consumption of each step of the checking complete flow by starting a thread to simulate the complete flow of checking;
thread allocation proportions of the steps of the reconciliation whole process are determined based on the average time consumption of the steps of the reconciliation whole process.
3. The reconciliation method of claim 2, wherein pre-obtaining resource configuration information for a server and generating a processor thread allocation policy and a reconciliation file partitioning policy based on the resource configuration information further comprises:
acquiring the average character length of each account checking record of a source account checking file counted by historical data;
calculating the average character length of each account checking record of a source account checking file based on historical data statistics to obtain the average occupied memory of each account checking record;
acquiring available memory information in the resource configuration information of the server;
calculating the account checking record number which can be processed by the server memory at the same time based on the average occupied memory and the available memory information of each account checking record;
and taking the account checking record quantity which can be processed simultaneously in the server memory as the dividing basis of the account checking file.
4. The reconciliation method of claim 3, wherein pre-obtaining the resource configuration information of the server and generating the processor thread allocation policy and the reconciliation file partitioning policy based on the resource configuration information further comprises:
acquiring the average file size of a single source account checking file counted by historical data;
acquiring available storage space information in the resource configuration information of the server;
the number of source ledgers that a server can process simultaneously is determined based on the average file size of the single source ledgers of the available storage space information and historical data statistics.
5. The reconciliation method of claim 3, wherein the obtaining, by the file preprocessing thread, the paged-out source reconciliation file based on the reconciliation file splitting policy, the paged-out reconciliation subfiles comprising:
acquiring the account checking record quantity which can be processed simultaneously in a server memory;
acquiring a preset memory occupation redundancy condition;
calculating the reasonable account checking record number of the paging account checking sub-file based on the memory occupation redundancy condition and the account checking record number which can be processed by the memory of the server at the same time;
and performing paging segmentation on the source account checking file based on the reasonable account checking record quantity to obtain a plurality of paging account checking sub-files.
6. The method of checking out according to claim 2, wherein initiating a plurality of checking out task threads and generating one summary task thread according to the processor thread allocation policy comprises:
acquiring the thread allocation proportion of each step of the reconciliation whole process;
acquiring the total number of threads which can be started by a CPU of the server;
calculating the total number of threads of the reconciliation task based on the total number of threads which can be started by the CPU of the server and the thread allocation proportion of each step of the reconciliation full flow;
obtaining the configuration number of the reconciliation task threads by subtracting 1 from the total number of threads;
initiating the configured number of reconciliation task threads and generating one summary task thread.
7. The method according to claim 1, wherein after the initiating a file preprocessing thread according to the processor thread allocation policy, the performing, by the file preprocessing thread, paging segmentation on the source reconciliation file based on the reconciliation file segmentation policy, to obtain a plurality of paging reconciliation subfiles includes:
generating a source reconciliation file task according to the configuration of the file downloading query every day;
checking whether the source reconciliation file task can be executed or not at fixed time intervals, and if so, modifying the source reconciliation file task to be processed;
and checking whether the source reconciliation file task to be processed exists or not, and sequentially distributing the source reconciliation file task to be processed to the file preprocessing thread for processing.
8. A reconciliation system, the reconciliation system comprising:
the policy generation module is used for acquiring resource configuration information of the server in advance and generating a processor thread allocation policy and a reconciliation file segmentation policy based on the resource configuration information;
the request receiving module is used for receiving a reconciliation task request initiated by a client and obtaining a source reconciliation file corresponding to the reconciliation task request;
the preprocessing thread initiating module is used for initiating a file preprocessing thread according to the processor thread allocation strategy;
the file segmentation module is used for carrying out paging segmentation on the source reconciliation file based on the reconciliation file segmentation strategy through the file preprocessing thread to obtain a plurality of paging reconciliation sub-files;
the accounting thread initiating module is used for initiating a plurality of accounting task threads according to the processor thread allocation strategy and generating a summary task thread;
the reconciliation task execution module is used for simultaneously processing the reconciliation tasks of the paging reconciliation sub-files through a plurality of reconciliation task threads;
and the reconciliation task summarizing module is used for summarizing the reconciliation results of the paging reconciliation sub-files through the summarizing task thread to generate a reconciliation report of the source reconciliation files.
9. A reconciliation server, the reconciliation server comprising: a memory and at least one processor, the memory having instructions stored therein, the memory and the at least one processor being interconnected by a line;
the at least one processor invoking the instructions in the memory to cause the reconciliation server to perform the reconciliation method of any of claims 1-7.
10. A computer readable storage medium having a computer program stored thereon, which when executed by a processor implements the reconciliation method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310565184.5A CN116308850B (en) | 2023-05-19 | 2023-05-19 | Account checking method, account checking system, account checking server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310565184.5A CN116308850B (en) | 2023-05-19 | 2023-05-19 | Account checking method, account checking system, account checking server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116308850A true CN116308850A (en) | 2023-06-23 |
| CN116308850B CN116308850B (en) | 2023-09-05 |
Family
ID=86801703
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310565184.5A Active CN116308850B (en) | 2023-05-19 | 2023-05-19 | Account checking method, account checking system, account checking server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116308850B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765179A (en) * | 2019-10-18 | 2020-02-07 | 京东数字科技控股有限公司 | Distributed account checking processing method, device, equipment and storage medium |
| CN113486059A (en) * | 2021-07-23 | 2021-10-08 | 重庆富民银行股份有限公司 | Multi-thread-based account checking method and system |
| CN113656194A (en) * | 2021-08-12 | 2021-11-16 | 京东科技控股股份有限公司 | Account checking result data notification method and device, electronic device and storage medium |
| CN113743078A (en) * | 2021-03-29 | 2021-12-03 | 北京京东拓先科技有限公司 | Data checking method, device and storage medium |
| CN114461385A (en) * | 2021-12-28 | 2022-05-10 | 深圳力维智联技术有限公司 | Thread pool scheduling method, device and equipment and readable storage medium |
| CN114997990A (en) * | 2022-05-31 | 2022-09-02 | 中国银行股份有限公司 | Distributed account checking method, device and system |
| WO2023273235A1 (en) * | 2021-06-29 | 2023-01-05 | 深圳前海微众银行股份有限公司 | Data comparison method, apparatus and device for file, and storage medium |
-
2023
- 2023-05-19 CN CN202310565184.5A patent/CN116308850B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765179A (en) * | 2019-10-18 | 2020-02-07 | 京东数字科技控股有限公司 | Distributed account checking processing method, device, equipment and storage medium |
| CN113743078A (en) * | 2021-03-29 | 2021-12-03 | 北京京东拓先科技有限公司 | Data checking method, device and storage medium |
| WO2023273235A1 (en) * | 2021-06-29 | 2023-01-05 | 深圳前海微众银行股份有限公司 | Data comparison method, apparatus and device for file, and storage medium |
| CN113486059A (en) * | 2021-07-23 | 2021-10-08 | 重庆富民银行股份有限公司 | Multi-thread-based account checking method and system |
| CN113656194A (en) * | 2021-08-12 | 2021-11-16 | 京东科技控股股份有限公司 | Account checking result data notification method and device, electronic device and storage medium |
| CN114461385A (en) * | 2021-12-28 | 2022-05-10 | 深圳力维智联技术有限公司 | Thread pool scheduling method, device and equipment and readable storage medium |
| CN114997990A (en) * | 2022-05-31 | 2022-09-02 | 中国银行股份有限公司 | Distributed account checking method, device and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116308850B (en) | 2023-09-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11422853B2 (en) | Dynamic tree determination for data processing | |
| US20180165348A1 (en) | Distributed storage of aggregated data | |
| US8732118B1 (en) | Distributed performance of data aggregation operations | |
| US5881283A (en) | Job scheduling analysis method and system using historical job execution data | |
| US6889243B1 (en) | Job scheduling analysis method and system using historical job execution data | |
| US20130332446A1 (en) | Efficient partitioning techniques for massively distributed computation | |
| JP5730386B2 (en) | Computer system and parallel distributed processing method | |
| CN103077197A (en) | Data storing method and device | |
| US20070226342A1 (en) | Transaction request processing system and method | |
| US20060230085A1 (en) | Key sequenced clustered I/O in a database management system | |
| US20150180960A1 (en) | Using a same program on a local system and a remote system | |
| JP2022524177A (en) | Transaction parallel execution method, device and storage medium | |
| CN104182295A (en) | Data backup method and data backup device | |
| CN116308850B (en) | Account checking method, account checking system, account checking server and storage medium | |
| CN115578180A (en) | Bank outlet cash resource management method and device | |
| US8290935B1 (en) | Method and system for optimizing database system queries | |
| US8296336B2 (en) | Techniques for efficient dataloads into partitioned tables using swap tables | |
| WO2014049738A1 (en) | Large-scale specifications transmission system, large-scale specifications transmission method, and server | |
| CN106611034A (en) | Method and apparatus for big size file blocking for distributed processing | |
| CN113590566B (en) | Method, device, equipment and storage medium for optimizing sequence file storage based on heap structure | |
| CN113127548B (en) | File merging method, device, equipment and storage medium | |
| JP2010061604A (en) | Consistency verification system, verification method, and program | |
| CN113495896A (en) | Management method, device, equipment and medium of transaction processing system | |
| CN117689243A (en) | Data processing method and device | |
| JP2720775B2 (en) | File parallel processing unit |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |