EP4659135A1

EP4659135A1 - A system and method for preserving data obfuscation during collaboration using prior obfuscated data

Info

Publication number: EP4659135A1
Application number: EP24786727.8A
Authority: EP
Inventors: Paul DE BEER
Original assignee: Omnisient International Ltd
Current assignee: Omnisient International Ltd
Priority date: 2023-10-06
Filing date: 2024-10-04
Publication date: 2025-12-10
Also published as: ZA202405525B; GB2634311A; WO2025073875A1

Abstract

A computer implemented system and method for securely processing data sets and identifying patterns and data correlations between data sets to provide insights from unconventional combined datasets.

Description

A System and Method for Preserving Data Obfuscation During Collaboration Using Prior Obfuscated Data

Technical Field

[001] This patent application relates to a system and technical method for securely processing, matching and analysing data sets to enable advanced collaboration on data between different parties while preserving obfuscated data.

Background

[002] This patent application relates to a system and method for securely processing data sets and identifying patterns and data correlations between data sets to provide insights from unconventional combined datasets.

Summary

[003] According to one aspect, a computer implemented method for preserving data obfuscation collaboration is provided. The method may comprise: storing a first party obfuscated data set and a second party obfuscated data set in a data store, wherein each obfuscated data set contains a plurality of line items, and each line item contains obfuscated data fields relating to sensitive information of data subjects that has been obfuscated and non-obfuscated data fields relating to the data subjects; establishing a connection between a first party and a second party whereby the first party requests access to the second party’s obfuscated data set by: transmitting a connection requestion to the first party or the second party; and receiving from the first party or the second party an authorisation of acceptance of the connection request; providing access to the at least one proprietary data set of the second party by: receiving from the second party data access criteria, the data access criteria including one or more fields and one or more line items to which access is allowed; transmitting the data access criteria to the first party; and receiving from the first party an acceptance of the data access criteria from the first party and creating a secure datastore for uploading data from the first party; executing an additional obfuscation method on the first party obfuscated data set to generate an additional obfuscated first party data set in which one or more of the non-obfuscated data fields of the first party obfuscated data set have been obfuscated and storing the additional obfuscated first data set in a secure data store; loading the second party obfuscated data set to the secure data store; executing a matching algorithm over the additional obfuscated first party data set and the second party obfuscated data set to match line items from the additional obfuscated first party data set with corresponding line items from the second party obfuscated data set, wherein the matching algorithm matches line items in the obfuscated data sets by matching one or more of the data fields from each set; creating, based on the data access criteria, a third data set including at least some matched line items from the additional obfuscated first party data set and the second party obfuscated dataset, wherein each matched line item contains obfuscated data fields and non-obfuscated data from the additional obfuscated first party data set and the second party obfuscated data set; allowing the first party to only view the non-obfuscated data of the third data set and wherein none of the obfuscated data or the non-obfuscated data can be exported out of the secure data store; providing a web access tool to the first party to enable execution of data analysis algorithms on the third data set and display the results of any data analysis algorithms executed on the third data set to the first party.

[004] The data access criteria may include an indication of the data within the second party obfuscated data set that are allowed to be accessed, one or more defined fields and line items that are allowed to be accessed, and whether the line items from all data subjects or only those relating to matching data subjects can be accessed.

[005] Executing an additional obfuscation method on the first party obfuscated data set to generate an additional obfuscated first party data set may comprise analysing the first party obfuscated data set to identify data fields from the non-obfuscated data which may be uniquely linked to a data subject, and obfuscating or excluding the identified data field from the additional obfuscated first party data set.

[006] The matching algorithm may be executed over the data sets to match a line item from the first data set with a corresponding line item from the second data using a third party data set to assist in the matching process, whereby if one or more obfuscated data fields and nonobfuscated data fields from the third party data set match one or more corresponding fields in the additional obfuscated first party data set and second party obfuscated data set, the line items are matched through association with a corresponding line item in the third party data set.

[007] The method may further comprise training a machine learning model using the third data set, and the method may further comprise generating at least one prediction using the machine learning model based on new data received from the second party. The at least one prediction may relate to obfuscated-preserving credit scores or risk profiles.

[008] The matching algorithm may comprise matching data fields from each line item from a first data set with one or more data fields from other line items from a second data set to identify one or more line items from the second data set that match the line item from the first data set, the matching algorithm then applying a weight to each matching line item and then selecting the line item with the highest matching weight. The weight may be the estimated probability that the two matched line items originated from the same data subject.

[009] According to another aspect of the invention, a system for securely processing data sets is provided. The system includes a communications module for receiving and transmitting data to and from the system, a memory for storing data to be used by the system, and a processor executing software thereon, the processor adapted to control the system to perform the method described above.

[010] According to another aspect of the invention, a non-transitory, computer-readable medium is provided. The computer-readable medium comprises program instructions which, when executed, cause a process to perform the method described above.

[011] According to some aspects, a computer implemented method for preserving data obfuscation collaboration is provided. The method may comprise: storing a plurality of obfuscated data sets in a data store, wherein each obfuscated data set contains a plurality of line items, and each line item contains obfuscated data fields relating to sensitive information of data subjects that has been obfuscated, and non-obfuscated data fields, relating to the data subjects; connecting data owner parties whereby a first party requests access to at least one other party’s proprietary data set; receiving and transmitting the authorisation of acceptance or rejection of a connection request; receiving and transmitting data access authorisation with relevant data field access criteria; receiving and transmitting the acceptance of data access criteria from the first party and creating a secure datastore for uploading data from the first party; executing an obfuscation method on the non-obfuscated data fields relating to non-sensitive information of the data subjects of the first party data set; loading the data from the second party to the secure data store; executing a matching algorithm over the data sets to match line items from the first obfuscated data set with corresponding line items from the second obfuscated data set, the matching algorithm matches line items in the obfuscated data sets by matching one or more of the data fields from each set; creating and storing a third data set including at least some matched line items from the first and second data sets, wherein each matched line item contains obfuscated data fields and non-obfuscated data from the first and second data sets; allowing the first party to only view the non-obfuscated data of the third data set and wherein none of the obfuscated data or the non- obfuscated data can be exported out of the data store; providing a web access tool to the first party to enable them to execute data analyses algorithms on the third data set; and displaying, via the web access tool, the results of any data analysis algorithms executed on the third data set to the first party. [012] In a further embodiment the access permission messages received may include detailed criteria on what portion of data the first party can access, the criteria includes one or more defined fields and line items that are allowed to be accessed and whether the line items from all data subjects can be accessed or only those relating to mutual data subjects.

[013] In a further embodiment, prior to executing the matching algorithm, the first data set is analysed to identify data fields from the non-obfuscated data which may be uniquely linked to a data subject, and the identified data field is either obfuscated or excluded from the third data set.

[014] In the computer implemented method the matching algorithm is executed over the data sets to match a line item from the first data set with a corresponding line item from the second data uses an additional data set of a third party to assist in the matching process, whereby if one or more obfuscated data fields and non-obfuscated data fields from the third party match one or more corresponding fields in the first and second data sets, the line items are matched through association with a corresponding line item in the third party data set.

[015] In a further embodiment of the computer implemented method wherein the plurality of obfuscated data sets are received from the first and second parties, the obfuscated data sets having been created by the first and second parties executing an obfuscating data algorithm on the their own raw data, wherein each received obfuscated data set contains a plurality of line items and each line item contains obfuscated data fields relating to sensitive information of data subjects that has been obfuscated and non-obfuscated data fields relating to nonsensitive information of the data subjects.

[016] In the data analyses algorithms applied on the third data set create a machine learning model that can be used to generate predictions using updated data from the second party.

[017] In one embodiment the data analyses algorithm can result in a machine learning model wherein the predictions relate to obfuscated- preserving credit scores or risk profiles.

[018] In one embodiment the matching algorithm matching data fields from each line item from a first data set with one or more data fields from other line items from the second data set to identify one or more line items from the second data set that match the line item from the first data set, the matching algorithm then applying a weight to each matching line item and then selecting the line item with the highest matching weight thereby estimating the probability that the two matched line items originated from the same data subject. [019] According to some aspects, the present technology also relates to a system for securely processing data sets. The system may comprise: a communications module for receiving and transmitting data to and from the system; a memory for storing data to be used by the system; a processor executing software thereon, the processor adapted to control the system to: store a plurality of obfuscated data sets in a data store in the memory, wherein each obfuscated data set contains a plurality of line items, and each line item contains obfuscated data fields relating to sensitive information of data subjects that has been obfuscated and non- obfuscated data fields relating to non-sensitive information of the data subjects; connecting data owner parties whereby a first party request access to at least one other party proprietary data set; receiving and transmitting the authorisation of acceptance or rejection of a connection request; receiving and transmitting data access authorisation with relevant data field access criteria; receiving and transmitting the acceptance of data access criteria from the first party and creating a secure datastore for uploading data from the first party executing an obfuscation method on the non-obfuscated data fields relating to nonsensitive information of the data subjects of the first party data set; loading the data from the second party to the secure data store; executing a matching algorithm over the data sets to match line items from the first obfuscated data set with corresponding line items from the second obfuscated data set, the matching algorithm matches line items in the obfuscated data sets by matching one or more of the data fields from each set; creating and storing a third data set including at least some matched line items from the first and second data sets, wherein each matched line item contains obfuscated data fields and non-obfuscated data from the first and second data sets; allowing the first party to only view the non-obfuscated data of the third data set and wherein none of the obfuscated data or the non- obfuscated data can be exported out of the data store; providing a web access tool to the first party to enable them to execute data analyses algorithms on the third data set; and displaying, via the web access tool, the results of any data analysis algorithms executed on the third data set to the first party. [020] According to some aspects of the present technology, there is provided a non- transitory, computer-readable medium comprising program instructions for obfuscated preserving data collaboration or securely processing data sets, the program instructions, when executed by one or more processors of a computer system, cause the one or more processors to perform any one more of the methods of the embodiments disclosed herein.

[021] According to some aspects of the present technology, there is provided a system comprising: at least one computing device configured to provide one or more applications for obfuscated preserving data collaboration or securely processing data sets the one or more applications comprising program instructions that, when executed by one or more processors, cause the one or more processors to perform any one more of the methods of the embodiments disclosed herein.

Brief Description of the Drawings

[022] Figure 1 Diagram providing an overview of the preferred pre- processing and components;

[023] Figure 2 Diagram illustrating the workflow to create a connection between data sharing parties Party A and B;

[024] Figure 3 Exemplary screens provided by a user interface which display aspects of an embodiment of the present disclosure;

[025] Figure 4 Illustration of the process whereby Party B provides data access to Party A;

[026] Figure 5 Exemplary screens provided by a user interface which display aspects of an embodiment of the present disclosure;

[027] Figure 6A Diagram defining the project components;

[028] Figure 6B Illustration of the process whereby Party A creates project to use Party B data;

[029] Figure 7 Exemplary screens provided by a user interface which display aspects of an embodiment of the present disclosure; [030] Figure 8 Illustration of the process whereby additional obfuscation on Party A data;

[031] Figure 9 Illustration the execution of the platform ‘Best Match’ algorithm;

[032] Figure 10 Exemplary screens provided by a user interface which display aspects of an embodiment of the present disclosure;

[033] Figure 11 Party A access project for analysis and ML development;

[034] Figure 12 Schematic illustration of Example 1 : Credit Risk assessments using

Grocery Retail data;

[035] Figure 13 Schematic illustration of Example 2: Optimise lead generation with data from a 3rd party;

[036] Figure 14 Schematic illustration of example components to implement the system;

[037] Figure 15 Schematic illustration of example components of the central server;

[038] Figure 16 Schematic illustration of Example 3: Evaluate long-term impact of government initiative;

[039] Figure 17 Schematic illustration of Example 4: Correlation between healthy food spend and medical claims/expenses;

Detailed Description with Reference to the Drawings

[040] The present technology relates generally to the field of data collaboration, more specifically, to a system and method that enables the unconventional combination of data sets from different parties (e.g. in different industries), and the use of advanced analytical tools in a unique data-obfuscated conserving way.

[041] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of an embodiment of the present technology. It will be evident, however, to one skilled in the art that the present technology may be practiced without these specific details. [042] The present technology relates to a system and method for preserving data obfuscation during collaboration between at least two collaborating parties.

[043] Data referred to herein may be any form of data. For example, data may refer to scientific or engineering data, measured data, simulated data, calculated data, information etc. The present technology is applicable to all types of data provided that certain data is to be obfuscated (e.g. sensitive data generally not to be disclosed outside a particular party).

[044] A data subject referred to herein pertains to a specific entity such as, but not limited to, a vehicle, robot, book, building, company or even an individual.

[045] For the purpose of illustration, in the detailed description and drawings, the collaborating parties will be referred to as Party A and Party B, although more than two parties can be collaborating at the same time.

[046] The preferred implementation of the system is after data from various parties (100A, 100B) have been obfuscated using a consistent technique (102) such as replacing the data with tokens generated by a hashing algorithm like SHA256 (or encryption of the data), and uploaded onto the data collaboration platform (106) via a network (104).

[047] For the data collaboration between parties to work, it is necessary that the requirements for the obfuscation technique (102) includes that the same criteria for all the parties are used and that separate obfuscation tokens are created for each field item of sensitive data, wherein each obfuscation token is the same for the same input data, and that non- sensitive data fields are not obfuscated (114A, 114B).

[048] The data collaboration platform may include multiple data stores (108), a server computer (110), and a user interface (112). The data collaboration platform (106) is preferably in a computing domain separate and independent from the potential data owner collaboration parties, and in one embodiment such computing domain is in a cloud computing domain.

[049] The data store (108) serves as the custodian of obfuscated data while any data collaboration project is in progress. The data store can be made up of different data storage locations and models. For example, the data store may include a data warehouse and a data lake or a cluster of servers. [050] Each potential collaborating party can view, grant access, delete, modify and analyse proprietary uploaded data via the user interface (112) using a web access tool. A potential collaborating party has no visibility or access to any other party proprietary data until agreement to collaborate has been reached and access to data has been granted.

[051] From the user interface (112) Party A can request a connection with Party B (202), which can be accepted or declined (204). If accepted a connection is created between the parties (206) which enables the Party A and Party B, now connected, to consider and grant data set access on the secure platform to each other without actual data transmission.

[052] From the user interface (112) Party B selects a data set (402) before which access will be granted, selecting a particular connection and party to which access is granted (404). In this step Party B may set the detailed criteria on which specific data elements Party A may access (406).

[053] The data access criteria for the specific data elements may include one or more fields and one or more line items to which access is allowed and may include specific criteria such as whether the line items from all data subjects can be accessed or only those relating to mutual (the same) data subjects.

[054] Party B may specify a time period of how long the data set will be available to Party A whereafter access to the data set is revoked and all projects where the data was used is no longer accessible and locked.

[055] The data access criteria can at any time during the project be changed, inclusive of allowing a time extension.

[056] The data access criteria is automatically communicated via the system to Party A.

[057] Party A can either accept or decline (410) the data access criteria from Party B. If the data access criteria is accepted the data access is set up by the system (412), and the newly shared data set is made available for selection within projects for Party A.

[058] Through the user interface (112) Party A can view the shared data sets, and request a project to be created on the platform (Figure 6B). A project consists of a separate dedicated secure data store (602), having a unique set of access credentials (604) managed by the central server (110) enabling the Party A to work on the data within the defined criteria set by the data collaborating party, Party B. In the project within the platform, Party A will have a set of analytical tools (606) to perform data preparation, analysis and machine learning.

[059] A new project is created (608) by Party A via the user interface (112) by submitting a request. Party A must select at least one of their own data sets (610) to be included in the new project as well as any data set already shared with Party A (612) still within the active data access period. (614). Party A can add multiple shared data sets form multiple different Companies into a single project.

[060] The new project request (616) is received on the platform and allocates a dedicated secure data store (618).

[061] All fields with sensitive information were obfuscated before being uploaded to the platform. Party A can however re-identify an original data subject if there exists a field in their data set with a value unique to that particular data subject - as well as any data shared by Party B data matched to that data subject. For this reason, an additional obfuscation (620) algorithm is executed on Party As data set to ensure that re-identification of data is not possible. This is only necessary for Party As data as they are the only party who will have access to the matched project data.

[062] The additional obfuscation is illustrated by Figure 8, and starts with firstly replacing all obfuscated tokens and unique customer identifiers with surrogate keys (802). The system then identifies other fields in the data set that could potentially function as a unique identifier and removes these fields (804). This is followed by a final step that identifies any field that contains a unique value for any particular data subject (806), and if these fields are numeric, it determines the best compromise between rounding off a number field and removing any unique records. Any non-numeric unique records are removed (810).

[063] Once the obfuscation step is completed Party As data set is matched to the Party B’s data set (622). In this context, matching refers to the process whereby line items from Party As data set in the project is matched with line items from Party B’s data set through a Best Match algorithm applying the data access criteria provided by Party B (624).

[064] The Best Match algorithm is illustrated in Figures 9A, 9B and 9C. [065] The matching process step begins by identifying the obfuscated sensitive data fields common between Party A’s and Party B’s data sets (902). Using only the common fields a matching table is created for each data set that contains the various pre-defined combinations with an acceptable level of accuracy (904), these matching tables are used to find all possible matches between the line items (906). The matches for each line item from Party A are ranked based on an accuracy score (908) and the best match with the highest accuracy score is selected (910). The selected matches are then used to create a linking table in the project data store (912).

[066] The accuracy score represents the estimated probability that the two matched line items originated from the same data subject. The higher the accuracy score the higher the likelihood that the data subject from Party A is the same data subject as the data subject from Party B.

[067] With the additional obfuscated Party A data set (620), and the link table created within the project data store (622), the Party B data set access criteria can be applied (624) and a corresponding data set created within the project data store (626). The Party B data set can only be created inside the project after the matching algorithm is completed as some of the access criteria may depend on the matching.

[068] When the final data sets have been created in the secure project data store (the claimed third data set), Party A as project owner on the platform (604) is allowed to access and work on the specific project’s data via the user interface (112). It provides the user with the permission to delete the project and assign project permissions to other users from Party A. No user is able to export any of the project data from the platform or move it across to other projects.

[069] The project users are provided with multiple analytics tools (606) to prepare and analyse the matched data sets from Party A and Party B inside the project data store (602). Examples of the tools include Power Bl dashboards, and SQL and Python coding environments.

[070] Some of the most valuable use cases for the technology is where Party A would use the project data to train a machine learning (ML) model to improve a project i.e. Party A would train an ML model to use Party B data as an input to predict or model a future situation for Party A. [071] A very simplified process flow, illustrating a use case, is shown in Figure 11. Party A would access a project (1102) via the user interface (112) and start with an exploratory data analysis of the matched data sets (1104). Creating a target variable on Party A data (1106) establishing an event outcome the ML model will be trained to predict. Features can be created on Party B data (1108), which form the inputs the model needs to use to calculate a prediction for the target variable. Party A would then train an ML model on the target from Party A and the features from Party B (1110).

[072] Depending on the use case, the present technology provides the tools for the ML model to be used to calculate ongoing predictions for future events based on updated features data.

[073] The data analyses algorithms applied on the matched data sets results in a machine learning model that can be used to generate predictions using updated data from Party B.

[074] The above description focusses on the simplest case of two collaborating parties.

[075] The illustration can for example apply on predicting whether a new credit applicant will default on loan repayments predicting a credit score of a risk profile.

[076] In the following non limiting examples the present technology is illustrated to display the wide range of data collaboration use cases that are enabled by the method and system claimed in this present technology. These examples illustrate how the present technology lends itself to more complex and unconventional scenarios for data collaboration.

[077] In one embodiment, the present technology is applied to enable multiple credit providers to assess the payment default risk of new credit applicants using data from a grocery retailer.

[078] In this embodiment data collaboration platform is deployed in a cloud environment and receives obfuscated data sets from one or more credit providers (each can be considered a separate Party A1 , A2, A3....) and a grocery retailer (Party B).

[079] Figure 12 provides a high-level illustration of the data flow between the parties.

[080] The credit providers could include banks or micro lenders, and each can use the platform to build a credit risk scorecard (i.e. machine learning model) using obfuscated data from the grocery retailer - and use that scorecard to assess the credit risk of future credit applications using the retailer’s data.

[081] In this embodiment the grocery retailer will obfuscate sales data linked to a data subject and upload it to the platform.

[082] The sales data will contain a customer’s obfuscated sensitive information as well as transaction details such as date, value and the products purchased. The sales data will also cover a long period of time to enable it to be used to build a machine learning model. The retailer’s sales data will be used to generate features as input (independent variables) into the machine learning model.

[083] Each credit provider will obfuscate a data set of historic credit applications and upload it to the platform. The data will at minimum contain certain obfuscated sensitive information, as well as the application date and whether a particular account subsequently went into arrears or not.

[084] This account performance will be used to create a target (dependent variable that the user wants to predict) to build the machine learning model.

[085] The data set can also include other auxiliary fields such features captured from the credit application, features received from a credit bureau and prediction outcomes of existing credit risk scorecards.

[086] Using the user interface (112) of the system each credit provider will send a connection request (202) to the grocery retailer, who can either accept or decline the request (204) using the user interface (112).

[087] The grocery retailer then grants access to their data (402) to each credit provider separately (404) and specifies the access criteria (406) to be applied (i.e. , time period, fields to include, line items to include, only access data from mutual customers or all customers) on user interface (112).

[088] Each credit provider will create a new project (608) using the user interface (112) and select their application data (610), as well as the data the retailer has granted access to (612), to be included in the project. [089] The system will create the project with its dedicated secure data store (618), run the additional obfuscation step on the credit provider’s data (620), run the Best Match Algorithm (622) and apply the data access criteria on the matched data set from the retailer (624).

[090] The credit provider is assigned the project owner role (604) and can now use the analytics tools provided by the platform (606) to analyse the data (1104), prepare the target variable (1106), build features (1108) and train a machine learning model (1110).

[091] In another embodiment, the method of the present technology is applied to enable an insurance Party and lead providers to develop a machine learning model to optimise lead generation using a third party data set.

[092] The data collaboration platform is deployed in a cloud environment and receives obfuscated data sets from an insurance Party (Party A) interested in buying ‘hot leads’ to sell their product to a party who are not already a customer, a third-party data owner (Party B1) with access to descriptive information on individuals for a broad segment of the market, and one or more lead providers (Party B2 and B3) with opted-in marketing lists.

[093] Figure 13 provides a high-level illustration of the data flow between the parties.

[094] A ‘hot lead’ can be defined as a party who has recently indicated interest in the product offered by a Party and provided their details and consent to be contacted by the Party to conclude the sale.

[095] A lead provider means a Party with an opted in (consent to market) list to whom messages would be sent to test interest in a specific product offered by another Party. If the response is positive, a ‘hot lead’ is identified that is sent to the other Party to conclude the product sale. This activity is also known as lead generation.

[096] The third-party data owner would generally be a Party with access to descriptive information on data subjects for a broad segment of the market (for example a credit bureau or data aggregator) to ensure sufficient overlap of data subjects from both the insurance Party and the lead providers.

[097] This example illustrates the system’s ability to enable the insurance Party to optimise the leads received from lead providers by utilising data owned by a third party. This assists the lead providers to obtain more efficiency from the messages they send as well as the insurance Party in receiving better quality leads from the lead providers. The third party would generally receive monetary compensation for the use of their data in the machine learning model.

[098] To start, the insurance Party will obfuscate historic lead and sales data and upload the obfuscated data to the platform. The lead and sales data will contain a data subject’s obfuscated sensitive information as well as further information on when the lead was received, whether the lead converted into a sale, and the value of the sale. The insurance Party’s data will be used as targets in the machine learning model to predict an individual's likelihood of becoming a lead (showing interest in the product) and converting into a valuable sale.

[099] Next the third party will obfuscate its data and upload the obfuscated data to the platform. The third party’s data will be used as features into the machine learning model.

[100] Using the user interface (112) the insurance Party will send a connection request to the third party (202), who can either accept or decline the request (204) using the user interface (112).

[101] The third-party grants access to their data (402) to the insurance Party (404) and specifies the access criteria (406) to be applied (i.e. , time period, fields to include, line items to include, in this case they need to provide access to data from all data subjects and not just the same data subjects) on the user interface (122).

[102] The insurance Party creates a new project (608) on the user interface (112) and selects their lead and sales data (610), as well as the data the third party has granted access to (612), to be included in the project.

[103] The system will create the project with its dedicated secure data store (618), run the additional obfuscation on the insurance Party’s data (620), run the best match algorithm (622) and apply the data access criteria on the matched data set from the third party (624).

[104] The insurance Party is assigned the project owner role (604) and can now use the analytics tools provided by the platform (606) to analyse the data (1104), prepare the target variable (1106), build features (1108) and train a machine learning model (1110). [105] The insurance Party’s data will be used as targets in the machine learning model and the third party’s data as features to predict an individual's likelihood of becoming a lead (showing interest in the product) and converting into a valuable sale.

[106] The machine learning model is used to generate predictions for the third party’s entire data set, not just the records matched to the Party data.

[107] Next the lead providers obfuscate their opted-in lists and uploads the obfuscated data to the platform. The opted-in lists will contain a data subject’s obfuscated sensitive information as well as a unique reference number that only exists in the lead provider system and is therefore not sensitive information, and which will enable the lead provider to reidentify the data subjects selected.

[108] Using the user interface (122) the insurance Party will send a connection request (202) to each lead provider, who can either accept or decline the request (204) using the user interface (112).

[109] The lead provider grants access to their data (402) to the insurance Party (404) and specifies the access criteria (406) to be applied (i.e. , time period, fields to include, line items to include, in this case they need to provide access to data from all data subjects and not just identical data subjects) on the user interface (112).

[110] The insurance Party selects the lead provider data to be included in the existing project. The system will move the data into the project’s secure data store.

[111] The insurance Party then joins the predictions from the third-party data onto the lead providers data and can use the analytics tools (606) provided by the platform to analyse how many of the lead provider’s list is predicted to have a high likelihood of being interested in their product and converting to a valuable sale.

[112] In another embodiment, the present technology is applied to enable a government department to use data from a private party such as a grocery retailer, as well as resources from a research institute, to assess the longer-term impact of a government initiative like a grant on recipients. [113] In this embodiment data collaboration platform is deployed in a cloud environment and receives obfuscated data sets from a government department (Party A) and a grocery retailer (Party B).

[114] Figure 16 provides a high-level illustration of the data flow between the parties.

[115] In this embodiment the grocery retailer will obfuscate sales data linked to a specific data subject and upload it to the platform.

[116] The sales data will contain a data subject’s obfuscated sensitive information as well as transaction details such as store location, date, value and the products purchased. The sales data will also cover a long period of time to enable it to be used to measure increase in spend by the grant recipients.

[117] The government department will obfuscate a data set of historic grant recipients and upload it to the platform. The data will at minimum contain the data subject’s obfuscated sensitive information as well as the start and end date of receiving the grant.

[118] Using the user interface (112) of the government department will send a connection request (202) to the grocery retailer, who can either accept or decline the request (204) using the user interface (112).

[119] The grocery retailer then grants access to their data (402) to the government department (404) and specifies the access criteria (406) to be applied (i.e. , time period, fields to include, line items to include, in this embodiment the grocery retailer will enable access for all their data subjects to enable the selection of control groups similar to the grant recipients) on user interface (112).

[120] The government department will create a new project (608) using the user interface (112) and select their grant recipient data (610), as well as the data the retailer has granted access to (612), to be included in the project.

[121] The system will create the project with its dedicated secure data store (618), run the additional obfuscation on the credit provider’s data (620), run the best match algorithm (622) and apply the data access criteria on the matched data set from the retailer (624). [122] The government department is assigned the project owner role (604) and can provide access to just the single project to external research consultants loaded as users on the government department’s profile (628). The research consultants can now use the analytics tools provided by the platform (606) to analyse the data and create control groups comparable to the grant recipients (1104), create the target variable needed to measure spend (1106), define input features as the different groups (grant vs control) and any control features (1108) and train a machine learning model to infer the long-term impact of the grant on spend after the recipient stopped receiving it (1110).

[123] In another embodiment, the present technology is applied to enable a medical aid to use data from a grocery retailer, to assess the impact of healthy food purchases on claims.

[124] In this embodiment the data collaboration platform is deployed in a cloud environment and receives obfuscated data sets from a medical aid (Party A) and a grocery retailer (Party B).

[125] Figure 17 provides a high-level illustration of the data flow between the parties.

[126] In this embodiment the grocery retailer will obfuscate sales data linked to a data subject and upload it to the platform.

[127] The sales data will contain a customer’s obfuscated sensitive information as well as transaction details such as store location, date, value and the products purchased. The sales data will also cover a long period of time to enable it to be used to evaluate a data subject’s spend on healthy products over time.

[128] The medical aid will obfuscate a data set of their member base, including historic claims and upload it to the platform. The data will at minimum contain certain obfuscated sensitive information, as well as details on any claims made by the member (claim type, value, date etc.).

[129] Using the user interface (112) of the medical aid will send a connection request (202) to the grocery retailer, who can either accept or decline the request (204) using the user interface (112).

[130] The grocery retailer then grants access to their data (402) to the medical aid (404) and specifies the access criteria (406) to be applied (i.e. , time period, fields to include, line items to include, in this embodiment the grocery retailer will only need to provide access to mutual customers) on user interface (112).

[131] The medical aid will create a new project (608) using the user interface (112) and select a data subject’s and its claim data (610), as well as the data the retailer has granted access to (612), to be included in the project.

[132] The system will create the project with its dedicated secure data store (618), run the additional obfuscation step on the credit provider’s data (620), run the Best Match Algorithm (622) and apply the data access criteria on the matched data set from the retailer (624).

[133] The medical aid is assigned the project owner role (604) and can now use the analytics tools provided by the platform (606) to analyse the data and determine the correlation between healthy food spend and claims.

[134] Various components may be provided for implementing the method described above with reference to Figures 1 to 11.

[135] Referring to the accompanying figures, a system for implementing the methodologies described herein typically includes a central server (1410) that is connected to a plurality of remote computers (1412).

[136] It will be appreciated that in the illustrated embodiment only four remote computers (1412) are illustrated but in a real life application there will likely be a large number of remote computes (1412) connected to the central server (1410).

[137] Referring to Figures 14 and 15, the central server (1410) typically includes one or more processors (1514) and memory (1516).

[138] It will be appreciated that the one or more processors (1514) may be any of the following: Central Processing Units (CPU), Graphics Processing Unit (GPU), microprocessors, field programmable gate arrays (FPGA), application specific integrated circuits (ASICs) or any other suitable computing processing device.

[139] Sometimes, the one or more processors (1514) may be located at the same location or different places (for example, in a distributed, cluster or cloud-based computing system) and some software units may be designed to manage and/or process data for remote devices. [140] The memory (1516) is used to create a secure data store in which sets of data from different third parties is stored.

[141] The central server (1410) includes a communications module (1520) for communicating with the remote computers (1412). This allows the receiving and transmitting of data to and from the central server (1410).

[142] The central server (1410) typically has a user interface (112) by means of which a user can input data and instructions to the central server (1410).

[143] The processor (1514) is operably coupled to the memory (1516), communications module (1520) and user interface (112).

[144] The processor (1514) typically has software executing thereon and is programmed to control the system to perform the methodology described above.

[145] The remote computers (1412) include one or more processors and one or more memory components (not shown in Figure 14). The remote computers represent the owners of the data.

[146] The remote computers (1412) are connected to the central server (1410) via a communications network (1418).

[147] The communications network (1418) could be a Mobile Communications Network, a Local Area Network (LAN), a Wireless Local Area Network (WLAN), a Wide Area Network (WAN) such as the Internet or a combination of these.

ALTERNATIVES:

[148] The above advantages are exemplary, and these or other advantages may be achieved by the invention. Further, the skilled person will appreciate that not all advantages stated above are necessarily achieved by embodiments described herein.

[149] In the foregoing specification, the invention has been described with reference to specific examples of embodiments of the invention. It will be evident, however, that various modifications and changes may be made therein without departing from the broader spirit and scope of the invention as set forth in the appended claims.

[150] Accordingly, unless implied or stated otherwise, any connections may for example be direct connections or indirect connections. The connections may be illustrated or described in reference to being a single connection, a plurality of connections, unidirectional connections, or bidirectional connections. However, different embodiments may vary the implementation of the connections. For example, separate unidirectional connections may be used rather than bidirectional connections and vice versa. In addition, plurality of connections may be replaced with a single connection that transfers multiple signals serially or in a time- multiplexed manner. Likewise, single connections carrying multiple signals may be separated out into various different connections carrying subsets of these signals. Therefore, many options exist for transferring signals.

[151] Those skilled in the art will recognize that the boundaries between logic blocks or software components are merely illustrative and that alternative embodiments may merge logic blocks or software components or circuit elements or impose an alternate decomposition of functionality upon various logic blocks or circuit elements. Thus, it is to be understood that the architectures depicted herein are merely exemplary, and that in fact many other architectures can be implemented which achieve the same functionality.

[152] Any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality is achieved. Hence, any two components herein combined to achieve a particular functionality can be seen as "associated with" each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated can also be viewed as being "operably connected," or "operably coupled," to each other to achieve the desired functionality.

[153] Furthermore, those skilled in the art will recognize that boundaries between the above described operations are merely illustrative. The multiple operations may be combined into a single operation, a single operation may be distributed in additional operations and operations may be executed at least partially overlapping in time. Moreover, alternative embodiments may include multiple instances of a particular operation, and the order of operations may be altered in various other embodiments. [154] Also for example, in one embodiment, the illustrated examples may be implemented as circuitry located on a single integrated circuit or within a same device. Alternatively, the examples may be implemented as any number of separate integrated circuits or separate devices interconnected with each other in a suitable manner.

[155] Also for example, the examples, or portions thereof, may implemented as soft or code representations of physical circuitry or of logical representations convertible into physical circuitry, such as in a hardware description language of any appropriate type.

[156] Also, the present technology is not limited to physical devices or units implemented in nonprogrammable hardware but can also be applied in programmable devices or units able to perform the desired device functions by operating in accordance with suitable program code, for example a Field Programmable Gate Array (FPGA). The terms “processing device”, “computing device”, “ processor” includes such programmable devices.

[157] However, other modifications, variations and alternatives are also possible. The specifications and drawings are, accordingly, to be regarded in an illustrative rather than in a restrictive sense.

[158] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of other elements or steps then those listed in a claim. Furthermore, the terms "a" or "an," as used herein, are defined as "one, or more than one." Also, the use of introductory phrases such as "at least one" and "one or more" in the claims should not be construed to imply that the introduction of another claim element by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim element to inventions containing only one such element, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an." The same holds true for the use of definite articles. Unless stated otherwise, terms such as "first" and "second" are arbitrarily used to distinguish between the elements such terms describe. Thus, these terms are not necessarily intended to indicate temporal or other prioritization of such elements. The mere fact that certain measures are recited in mutually different claims does not indicate that a combination of these measures cannot be used to advantage.

[159] As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words “include,” “including,” and “includes” mean “including, but not limited to’ the listed item(s).

[160] Various units, circuits, or other components may be described as “configured to” perform a task or tasks. In such contexts, “configured to” is a broad recitation of structure generally meaning “having circuitry that” performs the task or tasks during operation. As such, the unit/circuit/component can be configured to perform the task even when the unit/circuit/component is not currently on. In general, the circuitry that forms the structure corresponding to “configured to” may include hardware circuits. Similarly, various units/circuits/components may be described as performing a task or tasks, for convenience in the description. Such descriptions should be interpreted as including the phrase “configured to.”

[161] Unless specifically stated otherwise as apparent from the foregoing discussions, it is appreciated that throughout the present description of embodiments, discussions utilizing terms such as “collecting,” “selecting”, “creating,” “running,” “replaying,”, “collecting,” “fixing,” “communicating,” “deploying,” “running,” “comparing,” “predicting,” “mapping,” or the like, refer to the actions and processes of an integrated circuit, an ASIC, a memory device, a computer system, or similar electronic computing device. The memory device or similar electronic computing device manipulates and transforms data represented as physical (electronic) quantities within the devices’ registers and memories into other data similarly represented as physical quantities within the devices’ memories or registers or other such information storage, transmission, or display devices.

[162] Aspects of the present technology are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present technology. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus or processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. [163] These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

[164] The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

[165] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present technology. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that in some embodiments, one or more blocks may be omitted, where appropriate. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[166] It is to be understood that the described embodiments of the invention are illustrative only and that modifications thereof may occur to those skilled in the art. Accordingly, this invention is not to be regarded as limited to the embodiments disclosed, but is to be limited only as defined by the appended claims herein. It will further be understood that any features described in relation to any particular embodiment may be featured in combinations with other embodiments, for avoidance of doubt. [167] The following is a list of embodiments of the invention that may or may not be claimed at a later time:

1. A computer implemented method for preserving data obfuscation during collaboration between parties, the method including: storing a plurality of obfuscated data sets in a data store, wherein each obfuscated data set contains a plurality of line items, and each line item contains obfuscated data fields relating to sensitive information of data subjects that has been obfuscated and non-obfuscated data fields relating to the data subjects; connecting data owner parties whereby a first party requests access to at least one other party’s proprietary data set; receiving and transmitting the authorisation of acceptance or rejection of a connection request; receiving and transmitting data access authorisation with relevant data field access criteria; receiving and transmitting the acceptance of data access criteria from the first party and creating a secure datastore for uploading data from the first party; executing an obfuscation method on the non-obfuscated data fields relating to non-sensitive information of the data subjects of the first party data set; loading the data from the second party to the secure data store; executing a matching algorithm over the data sets to match line items from the first obfuscated data set with corresponding line items from the second obfuscated data set, the matching algorithm matches line items in the obfuscated data sets by matching one or more of the data fields from each set; creating and storing a third data set including at least some matched line items from the first and second data sets, wherein each matched line item contains obfuscated data fields and non-obfuscated data from the first and second data sets; allowing the first party to only view the non-obfuscated data of the third data set and wherein none of the obfuscated data or the non-obfuscated data can be exported out of the data store; providing a web access tool to the first party to enable them to execute data analyses algorithms on the third data set; and displaying, via the web access tool, the results of any data analysis algorithms executed on the third data set to the first party.

2. A computer implemented method according to embodiment 1 wherein the received access permission message includes detailed criteria on what portion of data the first party can access, the criteria includes one or more defined fields and line items that are allowed to be accessed and whether the line items from all data subjects can be accessed or only those relating to similar data subjects.

3. A computer implemented method according to embodiment 1 or embodiment 2 wherein, prior to executing the matching algorithm, the first data set is analysed to identify data fields from the non-obfuscated data which may be uniquely linked to a data subject, and the identified data field is either obfuscated or excluded from the third data set.

4. A computer implemented method according to any preceding embodiment wherein the matching algorithm executed over the data sets to match a line item from the first data set with a corresponding line item from the second data uses an additional data set of a third party to assist in the matching process, whereby if one or more obfuscated data fields and non-obfuscated data fields from the third party match one or more corresponding fields in the first and second data sets, the line items are matched through association with a corresponding line item in the third party data set.

5. A computer implemented method according to any preceding embodiment wherein the plurality of obfuscated data sets are received from the first and second parties, the obfuscated data sets having been created by the first and second parties executing an obfuscating data algorithm on the their own raw data, wherein each received obfuscated data set contains a plurality of line items and each line item contains obfuscated data fields relating to sensitive information of data subjects that has been obfuscated and non- obfuscated data fields relating to non-sensitive information of the data subjects.

6. A computer implemented method according to any preceding embodiment wherein the data analyses algorithms applied on the third data set results in a machine learning model that can be used to generate predictions using updated data from the second party.

7. A computer implemented method according to embodiment 6 wherein the predictions relate to obfuscated-preserving credit scores or risk profiles.

8. A computer implemented method according to any preceding embodiment wherein the matching algorithm matching data fields from each line item from a first data set with one or more data fields from other line items from the second data set to identify one or more line items from the second data set that match the line item from the first data set, the matching algorithm then applying a weight to each matching line item and then selecting the line item with the highest matching weight. A computer implemented method according to embodiment 8 wherein the weight is the estimated probability that the two matched line items originated from the same data subject. A system for securely processing data sets, the system including: a communications module for receiving and transmitting data to and from the system; a memory for storing data to be used by the system; a processor executing software thereon, the processor adapted to control the system to: store a plurality of obfuscated data sets in a data store in the memory, wherein each obfuscated data set contains a plurality of line items, and each line item contains obfuscated data fields relating to sensitive information of data subjects that has been obfuscated and non- obfuscated data fields relating to non-sensitive information of the data subjects; receive a request from a first party to link their obfuscated data set with the obfuscated data set of a second party; transmit a permission request to the second party; receive a permission authorisation message from the second party; access the obfuscated data sets from the first and second party; execute a matching algorithm over the data sets to match line items from the first obfuscated data set with corresponding line items from the second obfuscated data set, the matching algorithm matches line items in the obfuscated data sets by matching one or more of the data fields from each set; create a third data set including at least some matched line items from the first and second data sets, wherein each matched line item contains obfuscated data fields and non-obfuscated data from the first and second data sets; store the third data set securely in a data store in the memory, wherein the first party is able to view only the non-obfuscated data of the third data set and wherein none of the obfuscated data or the non-obfuscated data can be exported out of the data store; provide a web access tool to the first party to enable them to execute data analyses algorithms on the third data set; and display, via the web access tool, the results of any data analysis algorithms executed on the third data set to the first party.

Claims

1. A computer implemented method for preserving data obfuscation during collaboration between parties, the method including: storing a first party obfuscated data set and a second party obfuscated data set in a data store, wherein each obfuscated data set contains a plurality of line items, and each line item contains obfuscated data fields relating to sensitive information of data subjects that has been obfuscated and non-obfuscated data fields relating to the data subjects; establishing a connection between a first party and a second party whereby the first party requests access to the second party’s obfuscated data set by: transmitting a connection requestion to the first party or the second party; and receiving from the first party or the second party an authorisation of acceptance of the connection request; providing access to the at least one proprietary data set of the second party by: receiving from the second party data access criteria, the data access criteria including one or more fields and one or more line items to which access is allowed; transmitting the data access criteria to the first party; and receiving from the first party an acceptance of the data access criteria from the first party and creating a secure datastore for uploading data from the first party; executing an additional obfuscation method on the first party obfuscated data set to generate an additional obfuscated first party data set in which one or more of the nonobfuscated data fields of the first party obfuscated data set have been obfuscated and storing the additional obfuscated first data set in a secure data store; loading the second party obfuscated data set to the secure data store; executing a matching algorithm over the additional obfuscated first party data set and the second party obfuscated data set to match line items from the additional obfuscated first party data set with corresponding line items from the second party obfuscated data set, wherein the matching algorithm matches line items in the obfuscated data sets by matching one or more of the data fields from each set; creating, based on the data access criteria, a third data set including at least some matched line items from the additional obfuscated first party data set and the second party obfuscated dataset, wherein each matched line item contains obfuscated data fields and non-obfuscated data from the additional obfuscated first party data set and the second party obfuscated data set; allowing the first party to only view the non-obfuscated data of the third data set and wherein none of the obfuscated data or the non-obfuscated data can be exported out of the secure data store; providing a web access tool to the first party to enable execution of data analysis algorithms on the third data set and display the results of any data analysis algorithms executed on the third data set to the first party.

2. A computer implemented method according to claim 1 wherein the data access criteria include an indication of the data within the second party obfuscated data set that are allowed to be accessed, one or more defined fields and line items that are allowed to be accessed, and whether the line items from all data subjects or only those relating to matching data subjects can be accessed.

3. A computer implemented method according to claim 1 or claim 2 wherein executing an additional obfuscation method on the first party obfuscated data set to generate an additional obfuscated first party data set comprises analysing the first party obfuscated data set to identify data fields from the non-obfuscated data which may be uniquely linked to a data subject, and obfuscating or excluding the identified data field from the additional obfuscated first party data set.

4. A computer implemented method according to any preceding claim wherein the matching algorithm is executed over the data sets to match a line item from the first data set with a corresponding line item from the second data using a third party data set to assist in the matching process, whereby if one or more obfuscated data fields and non-obfuscated data fields from the third party data set match one or more corresponding fields in the additional obfuscated first party data set and second party obfuscated data set, the line items are matched through association with a corresponding line item in the third party data set.

5. A computer implemented method according to any preceding claim wherein the method further comprises training a machine learning model using the third data set.

6. A computer implemented method according to claim 5, wherein the method further comprises generating at least one prediction using the machine learning model based on new data received from the second party.

7. A computer implemented method according to claim 6 wherein the at least one prediction relates to obfuscated-preserving credit scores or risk profiles.

8. A computer implemented method according to any preceding claim wherein the matching algorithm comprises matching data fields from each line item from a first data set with one or more data fields from other line items from a second data set to identify one or more line items from the second data set that match the line item from the first data set, the matching algorithm then applying a weight to each matching line item and then selecting the line item with the highest matching weight.

9. A computer implemented method according to claim 8 wherein the weight is the estimated probability that the two matched line items originated from the same data subject.

10. A system for securely processing data sets, the system including: a communications module for receiving and transmitting data to and from the system; a memory for storing data to be used by the system; a processor executing software thereon, the processor adapted to control the system to perform the method of any preceding claim.

11. A non-transitory, computer-readable medium comprising program instructions which, when executed, cause a process to perform the method of any of claims 1 to 9.