AU2002100233A4

AU2002100233A4 - Automatic data aggregation system and method

Info

Publication number: AU2002100233A4
Application number: AU2002100233A
Authority: AU
Inventors: Philip Tole
Original assignee: HP JDV Ltd
Current assignee: HP JDV Ltd
Priority date: 2001-04-06
Filing date: 2002-03-27
Publication date: 2002-06-06
Anticipated expiration: 2010-03-27

Description

P/00/011 Regulation 3.2

AUSTRALIA

Patents Act 1990

ORIGINAL

SPECIFICATION

INNOVATION PATENT Invention Title: Automatic data aggregation system and method The following statement is a full description of this invention, including the best method of performing it known to us: FruIhills It\U'+ I r'r-nte 1k March 2002 (1:2 i v~lrea e e uournvIC:IYIuUUUII +u i o ozoL Prnted 27 March 2002 (15:02) page 2 AUTOMATIC DATA AGGREGATION SYSTEM AND METHOD The present invention relates generally to automated methods and systems for data aggregation, and in particular to computerised methods and systems for processing and aggregating data from a variety of different sources. The invention is suitable for use in facilitating the management of a portfolio of assets maintained for individuals or organisations by multiple asset managers, and it will be convenient to describe the invention in relation to that exemplary, non-limiting application.

Various companies provide users with computerised or Internet-based asset management systems to enable users to view details of a number of assets held by them at any time. Some systems of this type enable a user to enter details of assets held within various asset classes, and details of the value of those assets and transactions performed in relation to those assets. A user is frequently provided with a global view of the net worth of their asset portfolio, and is able to compare the performance of the various assets maintained within the portfolio.

Rather than requiring the manual entry of this information, some portfolio management systems are adapted to automatically extract asset-related information, such as the share or unit price of an asset or the value of funds held within an account, from one or more financial sources. The portfolio is then updated and selected details displayed to the user by the portfolio management system.

However, the completeness and accuracy of known portfolio management systems is not optimal. Due to the wide variety of financial institutions from which a user may have acquired or transacted in one or more assets, and the variety of data formats and data delivery systems used by various financial service providers, existing portfolio management systems frequently enable a user to only monitor a portion of their asset portfolio. Moreover, existing portfolio management systems are frequently complex, and difficult and expensive to develop and maintain.

There currently exists a need to provide a data aggregation system and Melbourne\004017801 Printed 21 March 2002 (10:48) method that ameliorates or overcomes one or more of the disadvantages of known prior art.

Moreover, there exists a need to develop a data aggregation method and system that facilitates the uniform collection, processing and/or presentation of data from diverse sources to a user.

There also exists a need to develop a data aggregation method and system that is convenient and simple to develop and maintain.

With this in mind, one aspect of the present invention provides a method of aggregating data, including: obtaining source data in one or more different data formats from one or more data sources; consolidating the source data into two or more output data files each having a common data format; and deriving consolidated information from the output data files.

Preferably, the source data is consolidated by: creating a data map for each data source, the data defining selected data source fields from each data source and corresponding output data fields in one or more output data files into which data from each selected source data field is to be mapped; for each data source, mapping data from the selected source data fields into the corresponding output data fields in accordance with the data map; and merging data in the output data fields into a consolidated output file.

The data map may also define one or more business rules governing the mapping of the data.

The sources from which source data is obtained may be financial data sources characterising different types of financial information. Different financial sources may be used to obtain the different types of source data. Source data Melbourne\004017801 Printed 21 March 2002 (10:48) obtained may include data characterising transactions involving assets and/or liabilities, ownership of assets, and/or reconciliation of the transactions.

The consolidated output files may include: a transaction file consolidating transaction information characterising transactions involving the assets and/or liabilities; a security file consolidating securities information characterising listed securities, such as equities, bonds and managed funds; a client file, consolidating owner information characterising owners of the assets, such as information identifying the client, account and profile data; and/or a reconciliation file consolidating reconciliation information enabling the reconciliation of the transaction information.

The method may further include: validating the currency and format of the source data.

The method may further include: where the source data characterises one or more transactions, verifying the presence of data fields required to record a complete business transaction.

Another aspect of the invention provides a data aggregation system for performing the steps of the method described above.

A further aspect of the present invention provides a data mapper for consolidating the source data described above into two or more output data files each having a common data format.

Yet another aspect of the present invention provides computer software including code for executing the steps described above.

A still further aspect of the present invention provides computer software including code for consolidating the source data described above into two or more output data files each having a common data format.

Melbourne\004017801 Printed 21 March 2002 (10:48) The following description refers in more detail to the various features of the present invention. To facilitate an understanding of the invention, reference is made in the description to the accompanying drawings where the data aggregation system and method are illustrated in a preferred embodiment. It is to be understood that the invention is not limited, however, to the preferred embodiment as illustrated in the drawings.

In the drawings: Figure 1 is a schematic diagram showing one embodiment of a data aggregation system according to the present invention; Figure 2 is a schematic diagram illustrating the conceptual operation of the data aggregation system of Figure 1; and Figure 3 is a data flow diagram illustrating the performance of a data mapping function by the data aggregation system of Figure 1.

Referring now to Figure 1, there is shown generally a data aggregation system 1 for use, in this example, in the management of a portfolio of assets. The data aggregation system 1 includes a portfolio management tool 2 that presents a consolidated view to a user of their asset holdings and liabilities, transactions involving those assets or liabilities, as well as details of security-based assets, such as the prices and dividends paid. The data displayed in the portfolio management tool 2 is obtained from a number of different financial data sources. A data mapping tool 3 maps input source data obtained from those financial data sources into consolidated output data files required by the portfolio management tool 2. A consolidated view of the assets and asset-related information is obtained by a user from remote user terminals 4 and 5, interconnected to the portfolio management tool 2 by a communications network, such as the Internet 6.

The data mapping tool 3 includes a data base engine 7, UNIX script engine 8 and email gateway 9. The data mapping tool 3 extracts and maps various sources of data into multiple consolidated output files in a format required by the portfolio Melbourne\004017801 Printed 21 March 2002 (10:48) management tool 2. The various financial data sources from which source data is obtained, in general, have differing data formats and data delivery mechanisms.

In this example, source data is required to be obtained from one or more data sources providing source data in File Tansfer Protocol (FTP) format 10, via a computer network such as the Internet 6. One or more data sources are obtained by the data mapper 3 by way of email delivery 11 over the Internet 6, whilst other financial data sources provide source data which may be downloaded by the data mapper 3 in Hypertext Transfer Protocol (HTTP) format. Other exemplary data sources 13 and 14 may provide source data in the form of one or more data tables directly useable by the database engine 7, and deliverable either via an export batch engine 15 or in response to a database query of a database server 16. In addition to existing financial datasources, the database engine 7 may be provided with manually entered financial data, for example, via a manual data entry terminal 17 and associated server 18. The email, or messaging, gateway 9 converts source data delivered to the data mapper 3 from a financial data source using Simple Mail Transfer Protocol (SMTP) or other like protocol into a UNIX convertible format.

The UNIX script engine 8 acts to extract source data from the data table delivered from the email gateway 9 or obtained via FTP or HTTP download. The UNIX script engine 8 may include a Java engine to facilitate the HTTP file copy process.

The UNIX script engine 8 includes shell script directories that are executed on the source data received in order to convert source data received into one or more data tables recognisable by the database engine 7. For example, the database engine 7 may use a database management system supporting Structured Query Language (SQL), and the UNIX script engine may convert the source data received into data table recognisable by this language. Similarly, data may be extracted from the database server 17 associated with the financial data source 16 by means of a database query using SQL or like language. The export batch engine 15 may also provide source data from the financial data source 13 in an SQL recognisable format.

Melbourne\004017801 Printed 21 March 2002 (10:48) Figure 2 illustrates the conceptual operation of the data mapper 3. Various financial data sources 20 to 22 provide source data to the data mapper 3. After conversion into a data format useable by the database engine 7, the data mapper 3 performs file validation 23 on the source data to check that the data file is current and has a valid data format. The data mapper 3 then performs a business logic function 24 to ensure that all data files relating to a transaction associated with a user asset include all required information to record a complete business transaction. The data mapper 3 also performs exception handling, providing exception reporting and information to accompanying erroneous or missing data or transactions in the data tables provided from the financial data sources 20 to 22.

The data mapper 3 also checks that the data for appropriate transactions is present by determining whether the data for the previous day's transactions has been provided or generating an error message when the data is for any day other than the previous day.

Importantly, the data mapper 3 performs a data consolidation function 26, namely consolidating source data into multiple output data files of common data format for use by the portfolio management tool 2 in deriving consolidated asset information for use by clients. In particular, the data mapper 3 acts to generate a client file 27, a transaction file 28, a security file 29 and a reconciliation file The security file consolidates general securities information characterising listed securities, such as equities, bonds and managed funds. The data mapper 3 sources securities data from selected financial data sources 20 to 22. The securities data generally includes the recognised or other identifier of the security, the security name, sector category and price. In addition to automatic extraction of source data, securities information can also be added to the database engine 7 via the manual entry terminal 18.

The client file consolidates owner information characterising owners of assets. The client, account and profile details are generally sourced from proprietary systems of financial service provider in a variety of different data Melbourne\004017801 Printed 21 March 2002 (10:48) formats. The client file consolidates this information into a standardised format for use by the portfolio management tool 2. Importantly, the portfolio management tool can produce data at an individual account level, collect data from all accounts linked to a single individual, and group data relating to a series of individuals into a household position.

The transaction file consolidates transaction information characterising transactions involving assets and/or liabilities. From this data, the portfolio management tool performs performance calculations and asset management for each client. The reconciliation file consolidates reconciliation information enabling the reconciliation of the transaction information in the transaction file. This process ensures that the portfolio management tool 2 has accurately represented the entire business transaction.

Selected financial data sources 20 to 22 may be used to obtain different types of source data required to create the consolidated output data files 27 to 30. In one practical embodiment developed by the Applicant, the data mapper 3 has interfaces to selected financial data sources in order to separately obtain each of the following types of information, required for generation of the consolidated security file 29: details of securities listed on one or more stock exchanges and derivatives detailed data for retail managed funds close-of-day pricing for securities listed on stock exchanges detailed data and pricing for wholesale funds data for term deposits pricing for exchange traded options detailed data and pricing for international shares and data for stock exchange traded securities security master and pricing data for semi government and corporate bonds The data mapper 3 also has interfaces to financial data sources to obtain the Melbourne\004017801 Printed 21 March 2002 (10:48) 9 following transaction information for use in generating the transaction file 28: transactions for client equity trading (including margin lending) and other financial transactions, such as banking 0 transaction details from a centralised electronic clearing settlement and transfer system recording stock exchange transactions transaction details from financial service providers, such as banks 0 transactions for retail managed funds 0 details of corporate actions and other transactions on securities held by individual financial service providers 0 cash and securities transactions for wholesale managed funds for individual financial service providers 0 details of transactions for exchange traded options S details of transactions for term deposits management by individual financial service providers.

The data mapper 3 additionally has interfaces to various financial data sources in order to generate the reconciliation file cash balance details for client trust accounts provided by individual financial service providers bonus sponsor holdings recorded by the electronic clearing settlement and transfer system of stock exchanges cash balance for a variety of accounts maintained for clients by individual financial service providers balances of retail managed funds 0 balances of all securities traded by clients of individual financial service providers 0 balances of wholesale management funds Melboume\004017801 Printed 21 March 2002 (10:48) client holdings of exchange traded options term deposit balances of accounts maintained by individual financial service providers.

The data mapper 3 similarly generates the client file 27 by accessing client, account and a profile detail of selected organisations. The client file 27 is a client based, rather than account based, data relationship model able to reflect data at an individual account level, client level or at a household level. The data mapper 3 extends the richness of information to allow for a representation of investments at a client level, which may include two or more accounts, and, at household level, may include two or more direct accounts and clients containing multiple accounts. The establishment of such relationships can be generated based on input source data, which may be electronically captured or configured within the data mapper 3 for specific application.

This client based data relationship model can be used to populate functionality rich Customer Relationship Management solutions concurrently with Portfolio Management solutions. One of the strengths of this model is the ability to have the same data to co-exist in several levels of the client model, without the traditional contention and inability to reconcile.

Accordingly, a user of the data (for example, investment advisors, direct clients and management) is able to view the client in a highly flexible manner enhancing their ability to extract greater marginal value out of the underlying data than would otherwise be the case in an account based relationship model.

The consolidation of the source data derived from the financial data sources to 22 into the multiple output data files 27 to 30, each having a common data format suitable for use by the portfolio management tool 2, is performed in a manner illustrated in Figure 3. As seen in this Figure, the source data is provided in a series of source data tables or like data structures, here illustrated by exemplary data tables 40 and 41. For each data source, a data map 42, 43 is created. In this Melbourne\004017801 Printed 21 March 2002 (10:48) example, data maps 42, 43 represent transaction data maps for mapping transaction source data into the consolidated transaction file 28, however the process illustrated in Figure 3 is representative of that used to generate each of the consolidated output data files 27 to Each of the data maps 42, 43 defines selected source data fields from each financial data source and corresponding output data fields into which data from each selected source data field is to be mapped. In this way, a variety of source data fields can be mapped into a limited number of output data fields, here referenced Field 1 to Field n, having a common format. The number of fields required to be processed by the portfolio management tool 2 is therefore minimised, and the data can be easily manipulated and compared. The data mapper 3 not only maps the data but also interprets it.

After the creation of the data maps, the data from each financial data source accessed by the data mapper 3 is mapped from selected source data fields into the corresponding output data fields in accordance with the data maps 42, 43. The data from the output data fields are then translated into one of the consolidated output files 27 to In addition, each data map may define one or more business rules governing the mapping of the data from the selected source data field to a corresponding output data field. The business rules may define, for example, conditions that must be met in order for the data mapping to occur. These conditions may relate to the value of data in other input source data fields from one or more selected data sources.

Finally, it is to be understood that various modifications and/or additions to the data aggregation method and system may be made without departing from the ambit of the present invention as described above. It will be appreciated that the invention is not limited to applications relating to the management of assets and liabilities, but may be used in a variety of other applications, such as customer relationship management, risk management, tax calculations and the like where data Melbourne\004017801 Printed 21 March 2002 (10:48) from multiple sources is required to be aggregated. The invention thus enables a client based, rather than account based, data relationship model to be developed for many differing applications.

Melbourne\004017801 Printed 21 March 2002 (10:48)

Claims

1. A method of aggregating data, including: obtaining source data in one or more different data formats from one or more data sources; consolidating the source data into two or more output data files each having a common data format; and deriving consolidated information from the output data files.

2. A method according to claim 1, wherein the source data is consolidated by: creating a data map for each data source, the data defining selected data source fields from each data source and corresponding output data fields in one or more output data files into which data from each selected source data field is to be mapped; for each data source, mapping data from the selected source data fields into the corresponding output data fields in accordance with the data map; and merging data in the output data fields into a consolidated output file.

3. A method according to claim 2, wherein the data map also defines one or more business rules governing the mapping of the data.

4. A method according to any one of the preceding claims, wherein the sources from which source data is obtained are financial data sources characterising different types of financial information. A method according to any one of the preceding claims, the method further including: where the source data characterises one or more transactions, verifying the presence of data fields required to record a complete business transaction. Dated: 21 March, 2002 HP JDV LIMITED FREEHILLS CARTER SMITH BEADLE Patent Attorneys for the Applicant Melbourne\004017801 Printed 21 March 2002 (10:48)