WO2007071984A2

WO2007071984A2 - Method and system for running a batch process

Info

Publication number: WO2007071984A2
Application number: PCT/GB2006/004789
Authority: WO
Inventors: Caroline Peeke; Martin Philips; Andrew Pocock; Phillip Connor
Original assignee: Misys Plc
Priority date: 2005-12-19
Filing date: 2006-12-19
Publication date: 2007-06-28
Also published as: GB0525741D0; WO2007071984A3

Abstract

A method is disclosed that allows batch processes such as the daily batch process to be run within a batch processing environment. An account batch process is initiated after a cut-off time between a first business day and a second business day. The accounts are selected one by one and processed by applying a plurality of batch functions to them without interruption within a short period of time. Whenever a transaction or enquiry is requesting access to an account during the batch process, the transaction or enquiry is delayed for a short period of time while the requested account is being processed. Thus, the method allows transactions and enquiries to operate on the main database even when the overnight batch process is running and ensures that transactions and enquiries initiated after the cut-off time are processed with the updated account data.

Description

METHOD AND SYSTEM FOR RUNNING A BATCH PROCESS

This invention relates to a method and system for running a batch process. The invention is particularly concerned with updating a large amount of data organised by accounts, to update account information. In preferred embodiments the invention relates to a method and framework that will allow batch processes such as the so- called daily batch process to be defined, scheduled, run and monitored within a batch processing environment.

Batch processes typically perform operations on batches of data, without user intervention, for example running as background tasks. They are often performance intensive.

Traditional banking or accounting systems have an overnight or daily batch process that runs for a period of several hours during each night to update the customer accounts and during this period the system is not available for processing new transactions. With the advent of round-the-clock Internet and telephone banking and operations across multiple time zones, there has been increasing pressure to provide 24-7 transaction processing and enquiry capabilities.

A form of 24-7 operation is often achieved by taking a copy of the customer and account data before starting the overnight batch process, using this copy to process transactions and enquiries during the batch period and then re-applying the transactions to the main database alter the batch processes have completed. This solution is not entirely satisfactory for several reasons: first, transactions are processed against out-of-date account data and this can lead to inconsistencies when the transactions are re-applied to the main database after the overnight batch process has completed. For example, a withdrawal transaction may be authorised that would not have been authorised if the results of the batch process had been known at the time. Hence, the solution may allow only a restricted set of transactions to occur during the overnight batch period because of the risk of inconsistencies and the difficulty of re-applying the transactions to the main database. In addition, there may be interruptions of service lasting several minutes while the database is being copied at the start of the overnight batch process and while the transactions are being re- applied after the end of the process. Finally, the solution is relatively expensive to develop because additional program logic is needed to re-apply the transactions to the main database.

It has been suggested in US 2001/0056433 to emulate online-like transaction processing of account information contained in batch process files. To this end, a batch file containing a plurality of records is first read and the records relating to the same one out of a plurality of accounts are identified, after which the batch processes are run on the records on an account-by-account basis. This, however, means that different accounts perform their batch processing and change their business date at different times during the daily batch process. Thus, there is no guarantee that a customer accessing his account after the batch process has started would have access to the updated data. Moreover, the need to create a work queue organised by account before processing the records adds complexity to the process.

Viewed from one aspect, the present invention provides a method of running a batch process to update a plurality of accounts on a data processing system, comprising applying to each account at least one batch function, the accounts being updated sequentially in accordance with a default sequence, wherein, during running of the batch process, if there is detected a request to access an account that has not yet been processed, execution of the request is delayed, application of the batch function to that account is carried out in advance of the position of that account in the default sequence, and the request is executed.

It will be appreciated that the default sequence could be a predetermined order or could be random. In either case, if a request is detected to access an account that has not been updated, the account will be updated on what may be, in the context of a particular embodiment of the invention, a prioritised basis, or as soon as possible, or substantially immediately.

The invention also extends to a data-processing system configured to carry out the method, and to a computer program carrying instructions which, when executed on a data-processing system, will configure the data-processing system to carry out the method. The program may be provided on a physical carrier such as a CD-ROM or a DVD, or as signals from a remote location, such as by means of a software download over the Internet.

In a preferred embodiment, the present invention provides a method that allows transactions and enquiries to operate on a main database even when an overnight batch process is running and ensures that transactions and enquiries initiated after a predetermined cut-off time are processed with the updated account data.

In one embodiment, a file of inter-account relationships is maintained that identifies groups of related accounts that have to be processed in a specific order or in a synchronized mode during the batch process, wherein the relationships are respected notwithstanding any deviations from the default sequence in the order in which accounts are updated.

Preferably, the maintaining of the file of inter-account relationships comprises modifying the file whenever a function is called that controls or affects an inter- account transaction.

Preferably, at least one batch function may include a close-of-business function that uses a first business day as a processing date and a start-of-business function that uses a second business day as processing date. The first business day and the second business day maybe successive business days. - A -

Preferably, at least one batch function includes changing a processing business date of a selected account or group of related accounts from the first business day to the second business day.

Preferably, at least one batch function includes at least one of: an interest accrual function; a payment processing function; an interest posting function; and a balance rollup function.

Preferably, the step of processing a selected account or group of related accounts is run in parallel on parallel systems.

Preferably, calculation-intensive processes are run before commencing applying the at least one batch function to each account.

The invention may be viewed from a number of aspects. Thus, viewed from another aspect, there is provided a method of applying a plurality of batch functions to a plurality of accounts, the method comprising maintaining a file of inter-account relationships that identifies groups of related accounts that have to be processed in a specific order or in a synchronized mode during the batch process, and initiating an account batch process after a cut-off time between a first business day and a second business day. The account batch process includes: selecting one by one each of the accounts or groups of related accounts that have not yet been processed; processing the selected account or group of related accounts by applying the plurality of batch functions to the selected account or group of related accounts without interruption within a short period of time; detecting that a transaction or enquiry is requesting access to an account out of the plurality of accounts; and delaying the transaction or enquiry until the requested account has been processed. The step of selecting one by one each of the accounts or groups of related accounts that have not yet been processed comprises identifying any requested account or group of related accounts that has not yet been processed and selecting the requested account as soon as it has been identified. In a preferred embodiment, the overnight batch process is organised on an account- by-account basis, so that all batch functions, such as interest accrual and payment processing, are applied consecutively within a short period of time to one account before progressing to apply them all to the next account. If a transaction or enquiry is initiated after the end-of-day cut-off time, it cannot proceed until the batch processing has been completed for the account or accounts involved in the transaction. The transaction or enquiry is delayed momentarily while this batch processing is completed and is then allowed to proceed as normal. Thus, the process ensures that inquiries and transactions initiated after the cut-off time are processed against the account data as at the next business day.

Advantageously, a batch scheduling process is used to control access to the account data. When it detects that a transaction or enquiry is requesting access to an account or group of related accounts that have not yet had the batch process applied, it schedules these accounts for immediate processing outside the normal account sequence and momentarily delays the transaction or enquiry until this batch processing is completed.

Preferably, the batch functions include "close of business" functions that use the first business day as a processing date and "start of business" functions that use the second business day as a processing date. By running "close of business" and "start of business" functions for each account in a single transaction, it is possible to reduce considerably the length of time for which the account is unavailable.

According to one embodiment, the method is applied to a bank batch process, and the batch functions include at least one of: an interest accrual function; a payment processing function; an interest posting function; and a balance rollup function.

Advantageously, the batch functions include a function to change the processing business date of the selected account or group of related accounts from the first business day to the second business day. It is possible to check the processing business date of each account to determine whether it has already been processed. Advantageously, the close of business batch process is performed on a daily, weekly, monthly, quarterly or yearly basis, or based on any other time period. Throughout this specification, the expression "daily batch process" will be used as a generic term to describe the periodic batch processes that run after "close of business".

In one embodiment, the first business day and the second business day are successive business days. Throughout this specification, the term "business day" is used as a generic term to designate a period of time between two successive, predetermined cut-off times.

Advantageously, the processing of selected accounts or groups of related accounts is run in parallel on parallel systems.

Advantageously, the maintaining of a file of inter-account relationships comprises modifying the file whenever an account function allows an inter-account transaction.

Advantageously, calculation-intensive processes are run before the cut-off time. Hence the time required to run the account batch process on each account is reduced.

According to a second aspect of the invention, there is provided a method of running a bank batch process comprising the following sequence of processes: applying the method described hereinabove to a plurality of customer accounts; applying the method described hereinabove to a plurality of nostro accounts; and applying the method described hereinabove to a plurality of internal accounts.

According to a third aspect of the invention, there is provided a method of processing bank accounts comprising: applying the method described hereinabove to a plurality of customer accounts; applying to the first business day any transaction requesting access to any account out of the plurality of accounts before the cut-off time; and applying to the second business day any transaction requesting access to any account out of the plurality of accounts after the cut-off time.

According to a fourth aspect of the invention, there is provided a bank account management system including: means for maintaining a file of inter-account relationships that identifies groups of related accounts that have to be processed in a specific order or in a synchronized mode during the batch process; means for detecting that, after a cut-off time between a first business day and a second business day, a transaction or enquiry is requesting access to an account or a group of related accounts out of the plurality of accounts that has not yet been processed, and for delaying the transaction or enquiry until the identified account has been processed; and means for initiating and running an account batch process after the cut-off time, the account batch process, including means for selecting one by one each of the accounts or groups of related accounts that have not yet been processed, said means being responsive to the detecting means so as to select the requested account as soon as it has been detected, and means for processing the selected account or group of related accounts by applying a plurality of batch functions to the selected account or group of related accounts, without interruption, within a short period of time.

Advantages and features of the invention will become more apparent from the following description of a specific embodiment of the invention, given as a non- restrictive example only, and represented in the accompanying drawings.

Figure 1 is a diagrammatic illustration of a time line showing different batch processes run during a business day.

Figure 2 illustrates an account daily batch process within a daily batch process.

Figure 3 illustrates a framework for handling the daily batch process.

Figures 4A and 4B illustrate a run-time environment for running the daily batch process. The figures together illustrate a method and framework for performing daily batch processes in a bank alongside the business-hours input providing a 24-7 environment. It is assumed that the bank has a plurality of branches or sets of branches. The daily batch process is run branch-by-branch within a bank-wide window of time, during which each branch's daily batch process is run. This means in particular that one branch's end-of-day process can run whilst other branches remain open. Each branch has a cut-off point after which the branch's daily batch process is run and after which newly-occurring business-hours transactions in a 24-7 environment are applied to the following business day.

Referring to figure 1, the daily batch process 10 is run within a window of time during each business day cycle using the following synchronisation points: a Start of Bank daily batch process synchronisation point 12, which is the point at which the "close of business" occurs for a business date; an end of all branches' daily batch processes synchronisation point 14, which is he point after which all branches have completed their daily batch processing allowing bank-wide processes to be completed; and an End of Bank daily batch process synchronisation point 16, which is the point at which the end of "close of business" for a business date is complete for all branches and for the bank.

These bank-wide synchronisation points ensure that all transactions always occur over two adjacent business days. Within these synchronisation points, branches are able to run their own daily batch processes at different times.

The daily batch process performs branch batch processes including all "close of business" and "start of business" processes. The end of all branches' daily batch processes synchronisation point 14 is provided to await the completion of all branch daily processes, after which any bank-wide processes are completed.

At weekends the daily batch process can be run on Friday evening and can perform all "close of business" and "start of business" processes. Once the daily batch process has completed, the bank business day will be Monday and all "start of business" processes will have been performed, e.g. cheques due for clearing on Monday will have cleared in time for customer ATM and Internet usage over the weekend. The bank may choose to schedule standalone, bank-wide processes such as standing orders and pay/receives on the Monday morning. The batch process framework provides a scheduling mechanism for "actual" date and time as well as "business" date and time.

Figure 1 illustrates the different batch processes performed within a business day. Background processes 20, standalone batch processes 22 and the daily batch process 10 each run concurrently with business-hours transactions 23. The daily batch process 10 is performed account-by- account within a branch (or a set of regional branches) of the bank.

To minimise inter-account transactions within a branch or a set of regional branches, the following process sequence can be adopted: perform branch start processes 29; perform a customer accounts daily batch process 24 (illustrated here as comprising Deal account and Movement account batch processes) for all customer accounts; perform a nostro accounts daily batch process 26 for all nostro accounts; perform an internal accounts daily batch process 28 for all internal accounts; and perform branch end processes 30.

This sequence performs all customer account processes before nostro and internal account processes to ensure that all customer postings updating nostro and internal accounts are complete before the nostro and internal accounts run their daily batch processes.

Daily batch processes are run account-by-account so that all of an account's daily processes can be performed within seconds as one logical transaction. Account updates are preferably multi-streamed so as all to run concurrently whilst still allowing business-hours input. The daily batch process within each branch is run by performing "close of business" transactions, i.e. transactions that use the current day as processing date, and "start of business" transactions, i.e. transactions that use the next day as processing date, for each account as a single transaction; hence, an account is updated in as short a time slot as possible, reducing the account lockout time to business-hours transaction levels. This account-based processing enables individual accounts to be accessed in a true 24-7 mode whereby any business-hours transaction after the branch cut-off time will trigger the daily batch process for the affected accounts) without affecting performance of the business-hours transaction.

The account-based batch processing also allows a seamless switchover between business dates for each account since, once an account has been processed, subsequent business-hours transactions will be for the next business date.

To ensure that the number of processes during an account's daily batch process remains manageable, it may be appropriate to run calculation-intensive processes (such as interest accrual calculation) before the daily batch process. These calculation-intensive processes update work files, which are later used during the daily batch process to update the database, thus removing the calculations from the daily batch process. Moreover, standalone processes such as standing orders may be run as a separate batch process that the bank can schedule depending on its own requirements. Whenever possible, the generation of customer document data for use by a customer correspondence product should preferably be performed as part of the associated business-hours transaction. File reorganisation can also be performed as a standalone batch process that the bank can schedule depending on its own requirements. Deferred tasks, e.g. base rate changes and credit risk management customer limits creation, can also run as background batch processes triggered by the corresponding business-hours transaction, if necessary.

To ensure that end-of-day reporting facilities are available, all concurrent batch processes populate work files that are available for subsequent reporting. Further, a data consistency point is made available after an account's balance rollup process, which extracts data from the database for statutory- and general-ledger reporting.

The definition of inter-account transactions within a branch may require one account to be processed before another during the daily batch process, e.g. an account used to fund a balance order must be processed before the account that it funds. Therefore, all related accounts have to be processed together during the daily batch process. To this end, a static file of inter-account relationships is maintained during business hours by any function that allows inter-account transactions during the daily batch process, e.g. balance orders, deal settlements, cheque deposit processing, nominated interest/charges on accounts and standing orders.

In certain circumstances, it may be advisable to limit the business-hours transactions available during the daily batch process. For example, the bank might choose to restrict the use of static data maintenance transactions such as account type changes and rate changes during the daily batch process.

One consequence of account-based processing is that different accounts perform their daily batch processing and change their business date at different times during the daily batch process. To ensure that the accounts all have a logical point at which they enter the next business day, each branch (or set of branches or region) has a specific cut-off time at which the business date is assumed to have changed, after which the daily batch processes will run and any subsequent transactions to the branch will be applicable to the next business day.

Figure 2 indicates how the daily batch process handles the cut-off time. More specifically, figure 2 illustrates an account daily batch process within a branch daily batch process within the bank daily batch process. The branch daily batch process has a specific cut-off time 40 after which accounts are processed. Once all branches have been processed, the bank- wide processes are performed. The diagram also shows a user attempting to access an account before (42) and after the branch cut-off time, in particular during the account daily batch process (44) and after the daily batch process has been completed (46).

Business-hours transactions before the branch cut-off (42) are applied to the current business date whereas transactions after the cut-off (44, 46) are applied to the next business date; hence, when the transaction is initialised after the cut-off time, the input and enquiry framework first checks to see if the account's daily batch process has run and, if not, this process is triggered without delay, while the transaction is momentarily put on hold. This, of course, may trigger multiple requests for daily batch processes to be run against related accounts before the account can be processed.

Figure 3 illustrates the components of a batch process framework that allows batch processes such as the daily batch process to be defined, scheduled, run and monitored within a batch-processing environment The data required include base processes 60, 62, 64, 66, each of which is a set of code that provides some form of functionality. Base processes may or may not require an input selection, may or may not issue an error message and may or may not update or output tables. In this framework, base processes cannot be run independently. They must first be associated to a data set and scheduled to run. To this end, process requests 70, 72, 74 are provided, which link a data selection and base process together. Process requests can then be scheduled for running as required, via a run time environment. Most process requests require selection data 80 to be entered that will be used for processing, e.g. account, customer and branch base processes require a corresponding selection identifying the range of items, e.g. accounts, that need to be processed. All or most base processes are at the account level where the data required for the base process are read using a separate process driver.

More complex process requests such as the daily batch process can be entered as separate process requests and combined to be run as one linked process request for ease of use. Preferably, all process requests are capable of being multi-streamed. The type of multi-streaming available may depend on the type of process. Branch multi- streaming allows the bank to define ranges of branches as well as a number of processors for each branch range.

Figures 4A and 4B illustrate the components of the runtime environment that enable scheduled batch process requests to be run. The daily batch process is initiated by a scheduler 100. The scheduler is a software component run continuously, which monitors branch cut-off times and submits scheduled requests at the cut-off points. More specifically, the daily batch process scheduler 100 selects the linked daily batch process request and sends a linked process request message to a dispatcher 102 at the branch cut-off time.

The dispatcher 102 receives and validates process request messages, which can originate from the scheduler 100 or from the input and enquiry system 104, and dispatches the message to a suitable control mechanism, a controller 106, based on whether the process request is linked or not.

The daily batch process request message is sent to the linked process request controller 106, which is used to control and call one or more individual process request controllers 108, 110, 112. Each process request is preferably submitted sequentially rather than concurrently.

On receipt of a process request message, each process request controller 108, 110, 112 determines whether multi-streaming is required. If so, the request is forwarded to a primary request multi-stream controller 114; otherwise the request is forwarded to a secondary process request multi-stream controller 116. The process request controllers 108, 110, 112 ensure that completion of each forwarded request is within the timeout period and, once complete or after the timeout period, update an audit database with the relevant status. The primary process request multi-stream controller 114 splits the process request using primary multi-stream characteristics and forwards appropriate messages to a secondary process request multi-stream controller 116. The secondary process request multi-stream controller reads the records required to drive the base processes and passes the number of records defined on a specific field of the process request to a process request driver 120. The process request driver 120 receives one or more records from a secondary process request multi-stream controller where a selection is specified. It may also receive no records where no selection data have been specified. The process request driver then calls up base requests as required. If no records are passed, the base processes are called up directly and committed. If records are passed then each record is processed in turn and, for each record, the driver will call each relevant base process that matches the base process's pre-conditions.

When a user attempts to access an account via an input and enquiry framework 104 during the daily batch process, the input and enquiry framework 104 first checks the date of the account data to determine if the daily batch process has already been applied to the account. If not, the input or enquiry is momentarily delayed, while the enquiry framework sends a message to the dispatcher 102 so as to initiate batch processing for the specific account or group of related accounts that is the subject of the transaction or enquiry. The date of the account is changed at the end of processing and the input and enquiry framework allows the input or enquiry to proceed.

While preferred embodiments of the invention have been described, it is to be understood by those skilled in the art that the invention is naturally not limited to these embodiments. Many variations are possible.

The framework is preferably a Java-based framework. However, alternative network-programming languages could also be used.

The daily batch process described so far is a synchronous process; however, the system can be easily extended to use an asynchronous process that would employ the current batch framework, using a targeted message and a timeout value. While the method has been described in relation to a bank with several branches, it is to be understood that it can also be applied to a bank with only one branch, or to any batch process within a 24-7 operation environment.

Claims

1. A method of running a batch process to update a plurality of accounts on a data processing system, comprising applying to each account at least one batch function, the accounts being updated sequentially in accordance with a default sequence, wherein, during running of the batch process, if there is detected a request to access an account that has not yet been processed, execution of the request is delayed, application of the batch function to that account is carried out in advance of the position of that account in the default sequence, and the request is executed.

2. A method as claimed in claim 1 wherein a file of inter- account relationships is maintained that identifies groups of related accounts that have to be processed in a specific order or in a synchronized mode during the batch process, and wherein said relationships are respected notwithstanding any deviations from the default sequence in the order in which accounts are updated.

3. A method as claimed in claim 2, wherein the maintaining of the file of inter- account relationships comprises modifying the file whenever a function is called that controls or affects an inter-account transaction.

4. A method as claimed in any preceding claim, wherein the at least one batch function may include a close-of-business function that uses a first business day as a processing date and a start-of-business function that uses a second business day as processing date.

5. A method as claimed in claim 4, wherein the first business day and the second business day are successive business days.

6. A method as claimed in claim 4 or 5, wherein the at least one batch function includes changing a processing business date of a selected account or group of related accounts from the first business day to the second business day.

7. A method as claimed in any preceding claim, wherein the at least one batch function includes at least one of: an interest accrual function; a payment processing function; an interest posting function; and a balance rollup function.

8. A method as claimed in any preceding claim, wherein the step of processing a selected account or group of related accounts is run in parallel on parallel systems.

9. A method as claimed in any preceding claim, further comprising running calculation-intensive processes before commencing applying the at least one batch function to each account.

10. A data processing system configured to carry out the method as claimed in any preceding claim.

11. A computer program carrying instructions which, when executed on a data- processing system, will configure the data-processing system to carry out a method as claimed in any preceding claim.

12. A method of applying a plurality of batch functions to a plurality of accounts, the method comprising: maintaining a file of inter-account relationships that identifies groups of related accounts that have to be processed in a specific order or in a synchronized mode during the batch process; and- initiating an account batch process after a cut-off time between a first business day and a second business day, the account batch process including: selecting one by one each of the accounts or groups of related accounts that have not yet been processed; processing the selected account or group of related accounts by applying the plurality of batch functions to the selected account or group of related accounts without interruption within a short period of time; detecting that a transaction or enquiry is requesting access to an account out of the plurality of accounts; and delaying the transaction or enquiry until the requested account has been processed, wherein the step of selecting one by one each of the accounts or groups of related accounts that have not yet been processed comprises identifying any requested account or group of related accounts that has not yet been processed and selecting the requested account as soon as it has been identified.

13. A method as claimed in claim 12, wherein the batch functions include close- of-business functions that use the first business day as a processing date and start-of- business functions that use the second business day as processing date.

14. A method as claimed in claim 12 or 13, wherein the batch functions include at least one of: an interest accrual function; a payment processing function; an interest posting function; and a balance rollup function.

15. A method as claimed in claims 12 to 14, wherein the batch functions include changing a processing business date of the selected account or group of related accounts from the first business day to the second business day.

16. A method as claimed in claim 12, wherein a close-of-business batch process is performed on a daily, weekly, monthly, quarterly or yearly basis, or based on any other time period.

17. A method as claimed in claims 12 to 16, wherein the first business day and the second business day are successive business days.

18. A method as claimed in claims 12 to 17, wherein the step of processing the selected account or group of related accounts is run in parallel on parallel systems.

19. A method as claimed in claims 12 to 18, wherein the step of maintaining a file of inter-account relationships comprises modifying the file whenever an account function allows an inter-account transaction.

20. A method as claimed in claims 12 to 19, further comprising a step of running calculation-intensive processes before the cut-off time.

21. A method of running a bank batch process comprising the following sequence of processes: applying the method of any one of claims 12 to 20 to a plurality of customer accounts; then applying the method of any one of claims 12 to 20 to a plurality of nostro accounts; and then applying the method of any one of claims 12 to 20 to a plurality of internal accounts.

22. A method of processing bank accounts comprising: applying the method of any one of claims 12 to 20 to a plurality of customer accounts; applying to the first business day any transaction requesting access to any account out of the plurality of accounts before the cut-off time; applying to the second business day any transaction requesting access to any account out of the plurality of accounts after the cut-off time.

23. A bank account management system including: means for maintaining a file of inter-account relationships that identifies groups of related accounts that have to be processed in a specific order or in a synchronized mode during the batch process; means for detecting that, after a cut-off time between a first business day and a second business day, a transaction or enquiry is requesting access to an account or a group of related accounts out of the plurality of accounts that has not yet been processed and for delaying the transaction or enquiry until the identified account has been processed; and means for initiating and running an account batch process after the cut-off time, the account batch process including: means for selecting one by one each of the accounts or groups of related accounts that have not yet been processed, said means being responsive to the detecting means so as to select the requested account as soon as it has been detected; and means for processing the selected account or group of related accounts by applying the plurality of batch functions to the selected account or group of related accounts without interruption within a short period of time.