AU2017201362A1 - Group selection method and apparatus - Google Patents

Abstract

H:\stp\Interwoven\NRPortbl\DCC\STP\13232366_ 1.docx-28/02/2017 - 32 A method for selecting a group of entities, the method including in one or more electronic processing devices, obtaining behaviour data indicative of a historical behaviour of a plurality of entities, obtaining attribute data indicative of attributes of the plurality of entities, analysing the behaviour data to determine at least one metric indicative of a projected future behaviour for each of the plurality of entities, determining selection constraints and selecting a group of entities using attributes of the plurality of entities, the selection constraints and at least one metric, wherein the selection is performed so as to ensure the selection constraints are met and to attempt to obtain a desired projected future behaviour for the group. Fig. 1 Obtain behaviour 100 data Obtain attribute data 110 Calculate plurality of 120 metrics Determine selection 130 constraints Select combination 140 Fig. 1

Description

GROUP SELECTION METHOD AND APPARATUS

Background of the Invention [0001] The present invention relates to a method and system for selecting a group of entities, and in particular to a method and system for selecting a group of entities based on a predicted behaviour of the entities.

Description of the Prior Art [0002] The reference in this specification to any prior publication (or information derived from it), or to any matter which is known, is not, and should not be taken as an acknowledgment or admission or any form of suggestion that the prior publication (or information derived from it) or known matter forms part of the common general knowledge in the field of endeavour to which this specification relates.

[0003] There are a range of different circumstances in which it is desirable to select a group from a number of different available entities. For example, this can be performed in order to select a team of sports players for inclusion in a fantasy sports competition, to select a number of shares for inclusion in an investment portfolio, to select a number of social media posts relating to a particular topic, or the like. All such processes typically involve having a user review information regarding historical behaviours, and in particular performance of the entities, together with other related information, using this in order to make a selection. However, the amount of data that needs to be reviewed, particularly when dealing with large numbers of entities, makes this process very difficult, meaning users are required to make selections largely based on instinct rather than analytical analysis of the available data, particularly for individuals with little or no data analytics experience. This places many individuals or organisations at a severe disadvantage when making such selections, and makes it likely that any resulting selection will be of only limited value.

Summary of the Present Invention [0004] In one broad form an aspect of the present invention seeks to provide a method for selecting a group of entities, the method including [0005] In one or more electronic processing devices: obtaining behaviour data indicative of a historical behaviour of a plurality of entities; obtaining attribute data indicative of attributes of the plurality of entities; analysing the behaviour data to determine at least one metric indicative of a projected future behaviour for each of the plurality of entities; determining selection constraints; and, selecting a group of entities using: attributes of the plurality of entities; the selection constraints; and, at least one metric, wherein the selection is performed so as to: ensure the selection constraints are met; and, attempt to obtain a desired projected future behaviour for the group.

[0006] In one embodiment the selection constraints define at least one of: restrictions on at least one attribute for each entity; restrictions on at least one attribute for the group; and, restrictions on the at least one metric.

[0007] In one embodiment the selection constraints are determined at least one of: by retrieving predefined selection constraints; and, in accordance with user input commands.

[0008] In one embodiment the method includes selecting the group of entities by: displaying, for at least some of the plurality of entities, an indication of: the at least one metric; and, the attributes; and, determining the selection in accordance with user input commands.

[0009] In one embodiment the method includes, determining selection of at least one entity by: generating an entity indication indicative of, for at least some of the plurality of entities: the at least one metric; and, the attributes; and, providing the entity indication to a client device via a communications network, the client device being responsive to the entity indication to: display a representation indicative of, for at least some of the plurality of entities: the at least one metric; and, the attributes; and, determine selection of at least one entity in accordance with user input commands; generating an entity selection indication indicative of the selected entity; and, providing the entity selection indication to the one or more processing devices.

[0010] In one embodiment the method includes selecting the group of entities using an optimisation algorithm.

[0011] In one embodiment the method includes using the optimisation algorithm to at least one of: optimise a cumulative value for at least one selected metric for the group; optimise individual values for at least one selected metric for each entity in the group; and, ensure the selection constraints are met.

[0012] In one embodiment the method includes selecting a plurality of groups of entities, each group including a different combination of entities.

[0013] In one embodiment the method includes selecting a plurality of groups of entities by at least one of: using multiple different optimisation algorithms; optimising multiple different metrics; and, using different selection criteria.

[0014] In one embodiment the method includes selecting a plurality of groups of entities by: selecting multiple entities, the multiple entities including more entities than in the group; and, selecting multiple groups from the multiple entities so that each group is a subset of the multiple entities.

[0015] In one embodiment the method includes: determining at least one entity selection in accordance with user input commands, the at least one entity selection including at least one of: at least one entity to be included in the group; and, at least one entity to be excluded from the group; and, selecting the group in accordance with the at least one entity selection.

[0016] In one embodiment the method includes: modifying at least one metric in accordance with user input commands; and, selecting the group using the at least one modified metric.

[0017] In one embodiment the at least one metric includes: a historical performance variance; a projected performance; a projected performance to entity value ratio; a comparison of a projected performance and a historical performance; an expected performance derived based on an entity value; and, a comparison of a historical performance and an expected performance.

[0018] In one embodiment the method includes: calculating an average entity value for a number of entities; calculating an average performance for the number of entities; and, calculating an expected performance per unit entity value using the average entity value and the average performance.

[0019] In one embodiment the method includes: selecting a set including a number of entities; and calculating the metrics for the set.

[0020] In one embodiment the method includes selecting the set at least partially in accordance with the attributes of the entities.

[0021] In one embodiment: the attributes include at least one of: an entity type; and, an entity value; the selection criteria define: a combination of entity types; and, a total entity value for the plurality of entities.

[0022] In one embodiment the method includes retrieving at least one of the behaviour data and attribute data from a remote data source.

[0023] In one embodiment the method includes at least one of: filtering the behaviour data; and, cleaning the behaviour data.

[0024] In one embodiment the method includes analysing the behaviour data using at least one predictive model.

[0025] In one embodiment the method includes: analysing the behaviour data using a plurality of predictive models to determine a plurality of metric scores for a given metric; and, combining the plurality of metric scores to determine the given metric.

[0026] In one embodiment the method includes: obtaining historical behaviour data; segmenting the historical behaviour data into a training set and a testing set; using the training set to train at least one predictive model; and testing the at least one predictive model using the testing set.

[0027] In one embodiment the method includes, iteratively improving the model using the training and testing sets.

[0028] In one embodiment the method includes, providing an indication of a group to a user client device via a communications network, the user client device being responsive to the indication to display a representation of the group to a user.

[0029] In one broad form an aspect of the present invention seeks to provide a system for selecting a group of entities, the system including [0030] In one or more electronic processing devices that: obtain behaviour data indicative of a historical behaviour of a plurality of entities; obtain attribute data indicative of attributes of the plurality of entities; analyse the behaviour data to determine at least one metric indicative of a projected future behaviour for each of the plurality of entities; determine selection constraints defining restrictions on combinations of attributes for the group; and, select a group of entities using: the attributes and the selection constraints to thereby ensure the selection constraints are met; and, the metrics to attempt to obtain a desired projected future behaviour for the group.

[0031] It will be appreciated that the broad forms of the invention and their respective features can be used in conjunction, interchangeably and/or independently, and reference to separate broad forms is not intended to be limiting.

Brief Description of the Drawings [0032] An example of the present invention will now be described with reference to the accompanying drawings, in which: - [0033] Figure lisa flowchart of an example of a method for selecting a group of entities; [0034] Figure 2 is a schematic diagram of an example of a distributed computer architecture; [0035] Figure 3 is a schematic diagram of an example of a processing system of Figure 2; [0036] Figure 4 is a schematic diagram of an example of a client device of Figure 2; [0037] Figure 5 is a flowchart of an example of a model generation process; [0038] Figures 6A to 6C are a flowchart of an example of a method for selecting a group of entities; [0039] Figure 7 is a flowchart of an example of a method of calculating metrics; [0040] Figure 8A is a schematic diagram of an example of a representation of entities and respective metrics; and, [0041] Figure 8B is a schematic diagram of an example of a selected group.

Detailed Description of the Preferred Embodiments [0042] An example of a process for selecting a group of entities will now be described with reference to Figure 1.

[0043] For the purpose of illustration, it is assumed that the process is performed at least in part using one or more electronic processing devices forming part of one or more processing systems, such as servers, personal computers or the like, and which may optionally be connected to one or more client devices, such as mobile communications devices, personal computers, or the like, via a network architecture, as will be described in more detail below. Whilst reference will be made generally to a single processing device, this is for the purpose of ease of explanation only and it will be appreciated that in practice functionality could be distributed across multiple processing devices, and the term should not be considered as limiting.

[0044] For the purpose of the following explanation, the term "entity" will be understood to include anything for which historical behaviour can be recorded and used to project future expected behaviour. For example, entities could include people, legal entities, such as corporations, partnerships, trusts, or the like, financial entities, including financial securities, such as stocks or shares, digital entities, such as social media posts, news articles, or the like.

[0045] It will also be appreciated that the nature of the behaviour will similarly vary depending on the type of entity, and the particular circumstances. For example, in the event that the entity is a sportsperson, the behaviour data could be indicative of previous performance ratings of the sportsperson, including but not limited to statistics, such as goals scored, passes completed or the like. Performance could be based solely on the basis of performance of the entity, but could also take into account the performance of a group with which the entity is associated, for example including aspects of team performance, such as team wins in the case of team sports. In the event that the entities are financial securities, the performance data could relate to financial performance, including a price, value and/or dividend associated with the security, whilst in the case of the entity being a social media post, this could correspond to a number of followers, how well the post is trending, a number of views, or the like. Thus, the behaviour is typically indicative of some form of performance of the entity, although this is not necessarily essential and any form of behaviour could be used.

[0046] In this example, at step 100, the processing device obtains behaviour data indicative of a historical behaviour of a plurality of entities. The behaviour data can be obtained in any appropriate manner and this could include receiving behaviour data from a data source, retrieving the behaviour data from a repository, such as a database, determining the behaviour data in accordance with user inputs, or any other appropriate mechanism. The form of the behaviour data will vary depending on the behaviour and the data source, but typically the behaviours data is times series data including a number of values representing changes in a respective variable over time, although this is not essential and any form of behaviour data could be used.

[0047] At step 110, the processing device obtains attribute data indicative of attributes of the plurality of entities. Again, the attribute data could be determined in any appropriate manner and could be received from a remote data source, retrieved from a store, entered manually, or otherwise determined. For example the attribute data could form part of metadata associated with the entity, particularly if the entity is a digital entity, such as a social media post or the like. The attribute data could be obtained from the same data source as the behaviour data, although this is not essential, and alternatively could be obtained from a different source.

[0048] In contrast to the behaviour data, which is indicative of a behaviour or performance of the entities, the attribute data relates to one or more specific properties of the entities and is typically substantially invariant. For example, in the case of a sportsperson, the attribute data could be indicative of a name, position played, a salary or value, or the like. In the case of financial securities, the attribute could be indicative of a name, a type of the security, a financial institute associated with the security, such as an exchange via which the security can be traded, or the like. Similarly, in the case of social media posts, the attribute could include an indication of the platform used for the social media post to be disseminated, could be indicative of the subject matter of the social media post, a hashtag or other suitable identifier.

[0049] At step 120, the processing device analyses the behaviour data to determine at least one metric indicative of a projected future behaviour for each of the plurality of entities. This can be achieved in any appropriate manner, but typically involves creating one or more mathematical models, which can be used to project future behaviour based on historical behaviour. In one example, this is achieved utilising machine learning predictive models, as will be described in more detail below, but it will be appreciated that this is not essential and any suitable form of mathematical model, such as multi variable analysis, monte carlo simulation, or the like, could be employed. In one particular example, the mathematical model can be predefined and retrieved as required, although again this is not essential.

[0050] At step 130, the processing device determines selection constraints. The selection constraints are used to place constraints over the entities that can be selected to form part of the group. The selection constraints can relate to the group as a whole, such as to define a number of entities to be included in the group, a total entity value across the group, types of entities to be included within the group, or the like and/or could include constraints relating to individual entities that are to be included, such as to specify maximum individual value, or the like. The selection constraints generally include constraints specific to attributes of the entities, but may additionally and/or alternatively include constraints on behaviour data, such as particular performance requirements, optimisation techniques, or the like, as will become apparent from the following description.

[0051] At step 140, at least one combination of entities is selected using the attributes, the selection constraints and the at least one metric. The selection can be performed in any appropriate manner and could be performed manual and/or automated procedures, such as an optimisation process, depending on the preferred implementation. In general, the selection is performed to ensure the selection constraints are met, as well as to attempt to obtain a desired projected future behaviour for the group. For example, this can be performed in order to attempt to maximise an accumulative metric score for the group.

[0052] By way of an example, in the case of sportspersons forming part of a sports team, this could be used in order to attempt to maximise a projected score for the sports team, whilst ensuring cost constraints are met and that players provided in each of required playing roles. This could be used for the purpose of fantasy sports style games, but could also be used in a professional context, for example to assist a team manager with organising player transfers, picking a player squad or the like. In the case of financial securities, this could be performed in order to maximise a total value and/or total dividend for the securities, for a given acquisition value and different spread of markets sectors, whereas in the case of social media posts, this could be performed in order to maximise social media post views, followers, trending or the like.

[0053] In any event, it will be appreciated that the above described process utilises a combination of historical behaviour data indicative of historical behaviour of the entities, together with attribute data and selection constraints, in order to allow a group of entities to be selected. The process can be performed in an automated, semi-automated or manual fashion, to provide a desired balance between a degree of control for a user versus simplicity in terms of the ability to automatically generate a group with substantially no user input. Despite this, allowing user intervention allows a wide range of different groups to be generated from the same underlying data, maximising the flexibility of the group selection process.

[0054] It will be appreciated that the above describe that the above described techniques can be used in a wide range of circumstances. For example, the approach could be used to select a team for participation in a fantasy sports league, to select securities for investment, social media posts for publication or the like. Irrespective of the intended use, this allows a large amount of data, in the form of historic behaviour data and additional attribute data, to be analysed and interpreted, which would not otherwise be achievable.

[0055] A number of further features will now be described.

[0056] In one example, the selection constraints are used to control the selection of entities within the group. The selection constraints can define restrictions on one or more attributes for each entity, or for the entire group. For example, in the case of a sports team, the selection constraints could place constraints on individuals, requiring that a combination of player roles be utilised in order to ensure the group corresponds to an entire team that could be fielded. Alternatively, selection constraints could be used to set a minimum or maximum value for each entity, or specifying a maximum combined salary for group members. In addition to this, the selection constraints could provide restrictions on the at least one metric, which is used to perform the selection, for example setting a minimum value for a projected score associated with the entity, limiting a performance variance for group members, or the like. In another example this can be performed in order to encourage or avoid stacking of players. For example, maximising the covariance of players who share a particular attribute can be used to pick complementary players from a given team, so as to benefit from points for both scoring and assisting.

[0057] The selection constraints can be predefined and retrieved from a database, and/or could be determined in accordance with user input commands. For example, if the user is utilising the group selection process in order to choose a team for inclusion in a competition, such as a fantasy sports league, a number of the constraints will be defined by the rules governing participation in the competition. This will include selection constraints, such as a total value of players, a total salary of players, combinations of required player positions or the like. Additionally and/or alternatively, at least some of the constraints could be manually defined, for example allowing the user to require or prohibit the inclusion of a particular entity, placing constraints over the attributes, such as a value of any one entity, or the like.

[0058] As previously mentioned, the group of entities can be determined manually. In this example, the method typically includes displaying, for at least some of the entities, an indication of at least one metric and/or attribute. Following this, the processing device determines selection of the entities in accordance with user input commands. This process could be performed entirely locally, or may be performed in a distributed fashion, for example by having the processing device generate an entity indication indicative of the at least one metric and/or attribute before providing the entity indication to a client device via a communications network, thereby allowing the client device to display a representation indicative of the metric and/or attributes for the entities. This allows a user to make a selection, with the client device determining selection of at least one entity in accordance with user input commands, generating an entity selection indicative of the selected entity and provides the entity selection to the one or more processing devices. It will be appreciated however that any suitable selection mechanism may be used.

[0059] As an alternative to performing manual selection, selection can be performed utilising automated processes. In one example, this includes selecting a group of entities using an optimisation algorithm. The optimisation algorithm can be adapted to optimise a cumulative value for at least one metric for the group and/or to optimise individual values for at least one selected metric for each entity in the group, for example to maximise a score associated with a specific entity type. The optimisation is also typically performed in order to ensure selection constraints are met. It will be appreciated that any appropriate optimisation algorithm can be used and that as such optimisation algorithms are known, these will not described in further detail.

[0060] It will also be appreciated that a combination of manual and automated processes could be used, for example allowing an optimisation algorithm to be used to select a draft group, with this being manually modified to generate a final group. Alternatively, a number of entities of interest could be identified manually by the user, with the group being selected from the entities of interest using the optimisation algorithm.

[0061] Whilst the method can include selecting a single group, in another example, a plurality of different groups of entities can be selected, with each group including a different combination of entities. This can be performed in order to for example allow users to submit multiple teams for participation in a fantasy sports league. Similarly, this could be utilised in order to allow users to establish multiple different securities portfolios or the like.

[0062] The plurality of groups of entities can be selected in a variety of ways. In one example, this is achieved by using multiple different optimisation algorithms, with each optimisation algorithm being used in order to select one or more groups of entities. Alternatively, this can be performed by optimising multiple different metrics. For example, this could be performed in order to optimise a score metric, to reduce a performance variability metric, or the like. Additionally and/or alternatively, the plurality of groups can be determined by using a different selection criteria for each group. It will also be appreciated that combinations of these approaches can be performed, for example using different selection criteria with multiple different optimisation algorithms, which are used to determine multiple different metrics.

[0063] In one particular example, the method of selecting a plurality of groups includes selecting multiple entities, the multiple entities including more entities than in the group, and the selecting multiple groups from the multiple entities so that each group is a subset of the multiple entities. Thus, this approach allows an initial step of selection to be performed to narrow down to a pool of potential entities, with different groups being selected from the pool of entities.

[0064] In a further example, the method can include determining at least one entity selection in accordance with user input commands, with the at least one entity selection including an indication of at least one entity to be included in the group or at least one entity to be excluded from the group. The group selection is then performed in accordance with the at least one entity selection. Thus, it will be appreciated that this can be used in order to allow users to influence the selection process, for example allowing the user to specify entities that should or should not be included in the group.

[0065] In a further example, the method can include modifying at least one metric in accordance with user input commands and selecting the group utilising the at least one modified metric. In this regard, it will be appreciated that the metrics are indicative of projected behaviour for the entities. A user may have particular experience or a particular opinion regarding the likely behaviour for an entity which falls outside the scope of projected behaviour that can be predicted utilising mathematical models. Accordingly, allowing a user to manually alter the metrics enables the user to take this into account. For example, this allows users to take into account information which may not be available as a part of the historical behaviour data, such as information regarding a recent player injury, recent change of management of a company issue shares, or the like. Thus, enabling a user to alter the metric allows this to be automatically taken into account in the group selection process. Additionally, this can be used to allow users to control creation of different groups, for example allowing the user to create multiple groups of unique entities, where each group has at least a minimum number of different entities in each group.

[0066] The above described system can utilise a range of different metrics. Example metrics include, but are not limited to, a historical performance variance, a projected performance, a projected performance to entity value ratio, a comparison of a projected performance and a historical performance, an expected performance derived based on an entity value and a comparison of a historical performance and an expected performance. It will be appreciated however that these are simply example metrics and a range of different metrics could be calculated depending for example on the nature of the behaviour data, the prediction models used, or the like, and the list is therefore not intended to be limiting.

[0067] The metrics can be calculated in any one of a number of ways, depending on the nature of the metric, the form of the available data, and the preferred implementation. In one example, this involves calculating an average entity value for a number of entities, calculating an average performance for the number of entities and then calculating an expected performance per unit entity value using the entity value and the average performance. The entities historical performance can then be compared to the expected performance to determine if the entity historically outperforms or underperforms the expected performance. This can then be used by the user to ascertain a likelihood of the entity exceeding projected performance. It will also be appreciated that projected performance can then be determined or interpreted taking this information into account.

[0068] It will be appreciated from this that in one example the method includes selecting a set including a number of entities, and calculating the metrics for the set. The set could be determined for example in accordance with attributes of the entities, allowing similar entities to be compared so that determination of relative metrics is more meaningful. For example, this could be used to ensure that metrics for players within a team are determined by comparing players to other players in a similar position. For example, it is pointless comparing the number of goals scored by a goalkeeper versus a striker. Similarly, it may be desirable to compare the performance of securities in one market sector to those in the same market sector, rather than in different market sectors, which may have significantly different performance characteristics.

[0069] The nature of the attributes will vary depending upon the nature of the entity. In one example, the attributes include at least one of an entity type and an entity value. In this case, selection criteria typically define a combination of entity types and a total entity value for the plurality of entities. Thus, for example, this could correspond to defining entity types in the form of player positions and entity values in the form of a player value or salary. This then allows a team to be created having a combination of particular player positions and a total value or salary cap. However, it will be appreciated that other attributes and respective selection criteria can be used.

[0070] In one example, the method includes retrieving the behaviour data and attribute data from a remote data source. The remote data source can be the same data source for both the attribute and behaviour data, but more typically is a different remote data source. For example, the attribute data could be retrieved from a data source associated with a company operating a fantasy league competition, with the behaviour data being retrieved from a data source having information regarding player performance in a real-life league in which players participate. It will be appreciated however that this will depend upon the particular implementation and in particular behaviour and attribute data used, so for example attribute and behaviour data for shares or other financial securities could be obtained from a share market on which the shares are traded.

[0071] When behaviour data is received, particularly from a remote data source, it is typically necessary to ensure the data is in a form suitable for analysis. This generally involves tasks such as filtering and/or cleaning the data. For example, the behaviour data may include information regarding every player in a league, whereas the fantasy league may only operate in respect of certain players, in which case the behaviour data can be filtered to remove non-participating players. The data is also typically cleaned, in order to remove extraneous or unneeded data, such as behavioural information not relevant to determining metrics, making the subsequent analysis more straightforward.

[0072] Typically, the behaviour data is analysed using at least one predictive model. More typically, the behaviour data is analysed using a plurality of different predictive models, with each model being used to determine a respective metric score for a given metric. The plurality of metric scores are then combined in order to determine the given metric. Such an ensemble approach allows fluctuations that may arise for one or more given models to be accommodated and in particular balanced by other models, leading to more accurate projections for individual entities.

[0073] The predictive models can be obtained in any appropriate manner. In one example, these can be obtained by performing a mathematical analysis of prior behaviour using this to generate a mathematical model. Alternatively however, this can be performed utilising machine learning or other similar techniques. When machine learning is used, historical behaviour data can be obtained for the plurality of entities, with the historical behaviour data being segmented into a training and testing set. The training and testing sets can be established based on temporal separation, for example, taking behaviour data for a previous twelve months and then segmenting this into six months of training data and six months of testing data. Alternatively, this could be performed by segmenting based on the entities, so that data for half of the players may be used as training data, with data for the other half being used as testing data. The training set is then used to train at least one prediction model, with the testing set being used to test the resulting predictive model. This process can be performed repeatedly, thereby iteratively improving the model until the model is able to make sufficiently accurate projections. Such prediction modelling techniques are known in the art and one example can be found in the Microsoft Azure "Predictive Experiment" modelling systems, and these will not therefore be described in further detail.

[0074] Whilst the above described method can be performed on a standalone computer system, in one example, the method is performed using a client server arrangement, with analysis of behaviour data being performed by the server, and with results being displayed on a client device, such as a tablet, mobile phone or the like. An example of such an arrangement will now be described in more detail with reference to Figure 2.

[0075] In this example, a number of processing systems 210 are provided coupled to one or more client devices 230, via one or more communications networks 240, such as the Internet, and/or a number of local area networks (LANs).

[0076] Any number of processing systems 210 and client devices 230 could be provided, and the current representation is for the purpose of illustration only. The configuration of the networks 240 is also for the purpose of example only, and in practice the processing systems 210 and client devices 230 can communicate via any appropriate mechanism, such as via wired or wireless connections, including, but not limited to mobile networks, private networks, such as an 802.11 networks, the Internet, LANs, WANs, or the like, as well as via direct or point-to-point connections, such as Bluetooth, or the like.

[0077] In this example, the processing systems 210 are adapted to provide to perform analysis of and provide access to data allowing this to be displayed via the client devices 230. Whilst the processing systems 210 are shown as single entities, it will be appreciated they could include a number of processing systems distributed over a number of geographically separate locations, for example as part of a cloud based environment. Thus, the above described arrangements are not essential and other suitable configurations could be used.

[0078] An example of a suitable processing system 210 is shown in Figure 3. In this example, the processing system 210 includes at least one microprocessor 300, a memory 301, an optional input/output device 302, such as a keyboard and/or display, and an external interface 303, interconnected via a bus 304 as shown. In this example the external interface 303 can be utilised for connecting the processing system 210 to peripheral devices, such as the communications networks 230, databases 211, other storage devices, or the like. Although a single external interface 303 is shown, this is for the purpose of example only, and in practice multiple interfaces using various methods (eg. Ethernet, serial, USB, wireless or the like) may be provided.

[0079] In use, the microprocessor 300 executes instructions in the form of applications software stored in the memory 301 to allow the required processes to be performed. The applications software may include one or more software modules, and may be executed in a suitable execution environment, such as an operating system environment, or the like.

[0080] Accordingly, it will be appreciated that the processing system 210 may be formed from any suitable processing system, such as a suitably programmed PC, web server, network server, or the like. In one particular example, the processing system 210 is a standard processing system such as an Intel Architecture based processing system, which executes software applications stored on non-volatile (e.g., hard disk) storage, although this is not essential. However, it will also be understood that the processing system could be any electronic processing device such as a microprocessor, microchip processor, logic gate configuration, firmware optionally associated with implementing logic such as an FPGA (Field Programmable Gate Array), or any other electronic device, system or arrangement.

[0081] As shown in Figure 4, in one example, the client device 230 includes at least one microprocessor 400, a memory 401, an input/output device 402, such as a keyboard and/or display, an external interface 403, and typically a card reader 404, interconnected via a bus 405 as shown. In this example the external interface 403 can be utilised for connecting the transaction terminal 220 to peripheral devices, such as the communications networks 230 databases, other storage devices, or the like. Although a single external interface 403 is shown, this is for the purpose of example only, and in practice multiple interfaces using various methods (eg. Ethernet, serial, USB, wireless or the like) may be provided. The card reader 404 can be of any suitable form and could include a magnetic card reader, or contactless reader for reading smartcards, or the like.

[0082] In use, the microprocessor 400 executes instructions in the form of applications software stored in the memory 401, and to allow communication with one of the processing systems 210.

[0083] Accordingly, it will be appreciated that the client device 230 be formed from any suitably programmed processing system and could include suitably programmed PCs, an Internet terminal, lap-top, or hand-held PC, a tablet, a smart phone, or the like. However, it will also be understood that the client device 230 can be any electronic processing device such as a microprocessor, microchip processor, logic gate configuration, firmware optionally associated with implementing logic such as an FPGA (Field Programmable Gate Array), or any other electronic device, system or arrangement.

[0084] Examples of the processes for selecting groups will now be described in further detail. For the purpose of these examples it is assumed that one or more respective processing systems 210 are servers. In one example, the servers 210 could host analysis services that are accessed by the client devices 230 allowing group selection to be performed, although alternatively this could be performed by the client device 230. Different servers 210 also typically act as data sources allowing behaviour and/or attribute data to be accessed as required. The servers 210 typically execute processing device software, allowing relevant actions to be performed, with actions performed by the server 210 being performed by the processor 300 in accordance with instructions stored as applications software in the memory 301 and/or input commands received from a user via the FO device 302. It will also be assumed that actions performed by the client devices 230, are performed by the processor 400 in accordance with instructions stored as applications software in the memory 401 and/or input commands received from a user via the EO device 402.

[0085] However, it will be appreciated that the above described configuration assumed for the purpose of the following examples is not essential, and numerous other configurations may be used. It will also be appreciated that the partitioning of functionality between the different processing systems may vary, depending on the particular implementation.

[0086] An example of the process for generating a predictive model will now be described with reference to Figure 5. For the purpose of these examples, reference will be made to the selection of a sports team for use in a fantasy sports league. The sports team is assumed to be a selection of different players that are members of actual sports teams, which have participated in sports competitions over one or more seasons. It will be appreciated that this is for illustrative purposes only and is not intended to be limiting, so the technique could also be applied to other entities, including but not limited to financial securities, social media posts, or the like.

[0087] In this example, at step 500, historical behaviour data is obtained. The historical data is typically behaviour data that has been collected for a plurality of entities over a previous time period. The historical data is typically distinct from the behaviour data utilised in order to generate the metrics. For example, when analysing player performance for use in a fantasy league, it is typical to utilise behaviour data from a relatively recent time period, such as the last few weeks or months, or an immediately previous season. In contrast, the historical data can be obtained from a prior time period, such as one or more prior seasons.

[0088] The historical data typically contains redundant records and other unnecessary information and accordingly is typically filtered and cleaned at step 505. Such filtering can include removing certain prior games, for example such as null results, abandoned games or the like, whilst cleaning the data typically includes applying a filter to remove unwanted columns of data, delete missing data rows or the like.

[0089] At step 510, the historical data is segmented into training and testing data. This can be performed by segmenting the data temporally, or by segmenting the data by splitting the data into different entity groups. A basic model is then created at step 515, with this being used to generate metrics from the training data at step 520. Typically, a number of default models are available and accordingly these are typically used to create the initial models at step 515.

[0090] The generated metrics are then compared to the testing data at step 525 to determine if the model is acceptable. If not, at step 535, the model is refined based on a disparity between predictions made and actual results recorded, using the testing data. In this regard, differences between the calculated metrics and the testing data are utilised in an iterative feedback loop in order to refine model parameters, thereby ensuring that the resulting model more accurately reflects real system behaviour. This process is therefore repeated a number of times as required until such time as the model is deemed acceptable at step 530, at which time the model can then be stored for subsequent usage at step 540.

[0091] The above described process is typically performed using a number of different base models, allowing a number of scenario specific models to be created, with these being used in parallel in order to increase the overall effectiveness of the modelling process, relative to the testing data.

[0092] An example of a process for selecting a group will now be described in more detail with reference to Figures 6A to 6C.

[0093] In this example, at step 600, a user selects a select group option. In this regard, the user typically access the system using a client device 230, by having the client device display a user interface presented for example via a webpage hosted by the server 210, or an application hosted by the client device 230. The option may specify whether the user wishes to select an automated, semi-automated or substantially manual process, and may also allow other information to be specified, such as the nature of the group to be created, or the like. Whilst shown as a single step, it will be appreciated that in practice multiple different step could be performed, such as selecting a type of group to be created and then specifying one or more options relating to the group to be created.

[0094] At step 605, the server 210 determines if the metrics required for the current group selection are current. In this regard, metrics will need to be generated on a periodic basis, such as each time additional games are played, but once created, these can be stored and reused as required. If the metrics are current, the process simply proceeds to step 635, otherwise metrics are created. It will also be appreciated that once metrics are stored, these can be reused as required, so that for example previous historic metrics could be used if desired. This might be useful for example to examine historical metrics from a previous equivalent time period in a previous season to examine how players perform at a given time point within the season.

[0095] In order to create the metrics, at step 610 the server 210 retrieves behaviour data from a suitable data store, such as a store of statistical data relating to games played within the sporting league, operating to filter and clean the behaviour data as needed at step 615. Such processes are known in the art, and can be automated with suitable knowledge of the format of the retrieved data, and these steps will not therefore be described in further detail.

[0096] The behaviour data is then applied to the predictive data models at step 620, allowing a projected behaviour for the entities to be determined at step 625. The projected behaviour typically involves determining a predicted overall score for the respective player, but may also include determining more specific information, such as a likely number of goals, tackles, or the like. It will be appreciated that depending on the complexity of the models, this could be a general projection, such as an average number of goals per game, or may be a more complex projection taking into account the opponents, and their own recent performance, thereby predicting different scores for different upcoming games. This process is typically performed for multiple different models, thereby obtaining multiple different projections, which are then combined in order to determine resulting metrics.

[0097] Accordingly, it will be appreciated that the above described process typically involves collecting data from a recent number of games N, forwarding this to the Microsoft Azure API, and receiving prediction results typically from an ensemble of models, including a Monte Carlo simulation, random forest, decision tree, or the like.

[0098] At step 630, the metrics are then calculated from the different projections, and stored for subsequent use. The nature of the metrics and the manner in which these are calculated will vary depending upon the preferred implementation and nature of the entities whose behaviour is being assessed.

[0099] An example metric calculation process will now be described with reference to Figure 7.

[0100] For the purpose of this example, reference is made to a generic score as an indicator of performance. Whilst the score could refer to a number of goals scored or the like, more typically the score is an overall rating for the player obtained during a game, and is a composite rating calculated based on a number of different aspects of the player’s and/or teams performance. This could include a number of goals scored, a number of passes completed, a number of successful tackles, whether the team won or lost, or the like. It will be appreciated that the composite scores could be calculated in a different manner depending on the respective player’s role within the team, so that a defender’s score could be calculated in a different manner to an attacker’s score. The scores are typically determined by the operator of the league, or a related party, but this will depend on the particular circumstances.

[0101] In this example, at step 700, a historical score variance is calculated based on the behaviour data. The historical score variance is utilised in order to understand a degree of variation in each players performance, and it is typically based on a standard deviation or the like. This can be used in order to allow a player volatility to be assessed, for example to determine whether a player is playing consistently, and hence likely to obtain a similar score each time they play, or is prone to good or bad games.

[0102] At step 705, a historical cost to score ratio is calculated for multiple entities, based on the average score and current salary or transfer value for multiple players, with this being used to calculate an expected score at step 710, in the form of an expected score for the player given their current value. This can be performed for all players in the league, or more typically is performed for certain types of player, so that for example a different expected score would be determined for a defender, as opposed to a midfielder or attacker.

[0103] The historical scores for an entity are then compared to the expected score, with this being utilised in order to determine metrics indicative of how often the expected scores are exceeded, exceeded by a determined threshold amount, such as 133%, or fallen below by a threshold amount, such as by 66%. This in turn can be used as an indicator of the player’s volatility.

[0104] At step 720, a projected score for the player is calculated based on the projected score obtained from multiple models, with this being used to calculate a projected entity cost per score ratio at step 725. The projected entity cost per score ratio is calculated to that of other players to determine predictive metrics at step 730, specifically whether the respective player cost per score ratio is above the average cost per score ratio for the multiple entities.

[0105] It will be appreciated that the above described process can be repeated a number of times for different sets of multiple entities, such as entities in a common player role or the like. It will be appreciated from this that a range of different statistical techniques can be used in order to derive the metrics and the examples provided above are for the purpose of illustration only and are not intended to be limiting.

[0106] Having calculated or retrieved pre-calculated metrics, at step 635 the server 210 retrieves entity attributes for the entities from a respective data source. In the current example, the attributes such as the player role, player name, player value or the like, could be obtained from the operator of the fantasy sports league, or the real league depending on the rules of the fantasy sports league competition.

[0107] Selection constraints are then determined at step 640. The selection constraints may also be obtained from the operator of the fantasy league, and could include restrictions on the total player value within a team, the number of different player roles within the team, or the like. The selection constraints can also include manually defined constraints, which can be initially set to a default value and then modified during the following process, as will be described with more detail below.

[0108] At step 645, the server 210 determines if an automatic group selection is to be performed, typically based on one or more of the options selected at step 600 above.

[0109] If automatic selection is to be performed, the server 210 uses an optimisation algorithm at step 650 in order to select a group of players from all of the available players at step 655. The selection is performed in accordance with the selection criteria, and is typically performed in order to optimise one or more metrics for the team, for example to maximise the overall score, minimise score variance, or the like. Any form of optimisation algorithm can be used and in one example, a linear optimiser, such as a Google OR-TOOLS linear optimiser, such as an Apps Script's Linear Optimization Service, Glop (Google Linear OPtimiser) SCIP, from Zuse-Institut Berlin, can be used.

[0110] Alternatively however, at step 660, an entity representation indicative of entities, the associated entity attributes and derived metrics can be displayed. This is typically displayed to the user via the client device 230, for example by displaying the representation as part of a webpage or having appropriate data transferred to the client device 230 allowing this to be displayed by an application hosted by the client device.

[0111] An example of an entity representation is showed in Figure 8A.

[0112] In this example, the entity representation corresponds to players forming part of a fantasy league. In this indication, the representation includes a number of columns 801, ...814. This includes a player column 801, a team name column 802, an opponent column 803, a position column 804 and a price column 805. Metrics are then displayed, including a predicted score column 806, a score for the last five games column 807, a standard deviation 808 and a number of games on which the prediction is based 809. Column 810 shows a cost per score, column 811 shows a cost variation based on a comparison of cost per score versus an expected cost per score, columns 812 to 814 describe a historical performance and in particular a comparison of the historical score to an expected score with column 812 showing a percentage of games in which a player has scored at least their expected score (‘consistency’), column 813 a percentage of games over 133% of their expected score (‘upside’) and column 814 a percentage of games a player has scored under 66% of an expected score.

[0113] At this stage, additional advanced options can be used to filter the players displayed. The nature of the options can vary depending on the preferred implementation, but could include defining numbers of players to be displayed, a salary cap per player, an exposure, a number of players from each of a number of different teams, or the like.

[0114] At step 665, a user selected combination is determined in accordance with user input commands. Thus, the user manually selects a number of players based on the information presented, with an indication of the selection being provided to the server 210, each from the application executed by the client device 230, or through interaction with the website.

[0115] Irrespective of whether automatic or manual selection is performed, the server 210 determines whether further input is required and if so the process returns to step 645. Thus, it will be appreciated that this allows a partial selection to be made manually and/or automatically, with additional parts of the selection process being performed automatically or manually, depending on the preferred implementation.

[0116] Once a group has been selected, results can be displayed at step 675. An example of a representation of the results are shown in Figure 8B.

[0117] In this example, the results representation includes a position column 821, a player column 822, a team column 823, a salary column 824, a projection column 825 and a change option column 826. A remaining salary 827 and total score 828 are shown. Thus, this allows a user to view a created team, view information regarding the salary of each team member and the projected score for each team member, together with a cumulative score and a remaining salary compared to a salary cap defined in the selection constraints.

[0118] At step 680, the user determines if the outcome is acceptable, and if not, optionally performs modification of any one or more of selection criteria, optimisation techniques, required or precluded entities or the like at step 685. These modifications are typically performed in order to allow the user to influence the selection process in some manner and in particular, to allow alternative sets of results to be achieved.

[0119] For example, the user could select to exclude one or more members of the group, and then operate to repeat the process, so that different group members are determined. Similarly, the user could select one or more group members to be retained, with the process being repeated in order to select different other team members.

[0120] The user could select different parameters for the optimisation algorithm. This could include for example, selecting to have the algorithm maximise an overall score whilst minimising a score variance and then subsequently maximise the score with no limit on the variance. Not limiting the variance, will allow players that can have outlier performances to be included, which can lead to the potential for a higher overall score, but with the risk that the higher score might be less likely to be achieved.

[0121] Alternatively, the user can manually alter one or more of the projected metrics, to see how this influences the selection process. This can be used for example if the user believes the respective player is likely to perform better or worse than projected.

[0122] It will therefore be appreciated that this process can be performed multiple times, with a new group being performed on each iteration allowing a user to explore a range of different optimisation options, with little or no understanding of how the metrics are calculated and the groups of players chosen. This can be repeated until an acceptable outcome is achieved at step 680, at which point the results can be saved at step 690.

[0123] Whilst the above described example has focused on the determination of a single group with each iteration, it will be appreciated that this is not essential and instead multiple different groups can be created with each interaction, allowing users to more rapidly explore differently optimised groups, thereby helping users more rapidly arrive at a preferred group.

[0124] Accordingly, the above described process allows a user to gain access to results of mathematical modelling in order to understand the likely behaviour, and in particular the performance of an entity in future, based on information derived from historical behaviour. This process can be broadly considered to include a projection step, used to predict future performance and an analysis step used to determine a group, with this latter step optionally being performed multiple times in parallel, or allow multiple different groups to be determined.

[0125] Throughout this specification and claims which follow, unless the context requires otherwise, the word “comprise”, and variations such as “comprises” or “comprising”, will be understood to imply the inclusion of a stated integer or group of integers or steps but not the exclusion of any other integer or group of integers.

[0126] Persons skilled in the art will appreciate that numerous variations and modifications will become apparent. All such variations and modifications which become apparent to persons skilled in the art, should be considered to fall within the spirit and scope that the invention broadly appearing before described.

Claims (25)

1. THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS: 1) A method for selecting a group of entities, the method including in one or more electronic processing devices: a) obtaining behaviour data indicative of a historical behaviour of a plurality of entities; b) obtaining attribute data indicative of attributes of the plurality of entities; c) analysing the behaviour data to determine at least one metric indicative of a projected future behaviour for each of the plurality of entities; d) determining selection constraints; and, e) selecting a group of entities using: i) attributes of the plurality of entities; ii) the selection constraints; and, iii) at least one metric, wherein the selection is performed so as to: (1) ensure the selection constraints are met; and, (2) attempt to obtain a desired projected future behaviour for the group.
2. 2) A method according to claim 1, wherein the selection constraints define at least one of: a) restrictions on at least one attribute for each entity; b) restrictions on at least one attribute for the group; and, c) restrictions on the at least one metric.
3. 3) A method according to claim 2, wherein the selection constraints are determined at least one of: a) by retrieving predefined selection constraints; and, b) in accordance with user input commands.
4. 4) A method according to any one of the claims 1 to 3, wherein the method includes selecting the group of entities by: a) displaying, for at least some of the plurality of entities, an indication of: i) the at least one metric; and, ii) the attributes; and, b) determining the selection in accordance with user input commands.
5. 5) A method according to claim 4, wherein the method includes, determining selection of at least one entity by: a) generating an entity indication indicative of, for at least some of the plurality of entities: i) the at least one metric; and, ii) the attributes; and, b) providing the entity indication to a client device via a communications network, the client device being responsive to the entity indication to: i) display a representation indicative of, for at least some of the plurality of entities: (1) the at least one metric; and, (2) the attributes; and, ii) determine selection of at least one entity in accordance with user input commands; iii) generate an entity selection indication indicative of the selected entity; and, iv) provide the entity selection indication to the one or more processing devices.
6. 6) A method according to any one of the claims 1 to 5, wherein the method includes selecting the group of entities using an optimisation algorithm.
7. 7) A method according to claim 6, wherein the method includes using the optimisation algorithm to at least one of: a) optimise a cumulative value for at least one selected metric for the group; b) optimise individual values for at least one selected metric for each entity in the group; and, c) ensure the selection constraints are met.
8. 8) A method according to any one of the claims 1 to 7, wherein the method includes selecting a plurality of groups of entities, each group including a different combination of entities.
9. 9) A method according to claim 8, wherein the method includes selecting a plurality of groups of entities by at least one of: a) using multiple different optimisation algorithms; b) optimising multiple different metrics; and, c) using different selection criteria.
10. 10) A method according to claim 8 or claim 9, wherein the method includes selecting a plurality of groups of entities by: a) selecting multiple entities, the multiple entities including more entities than in the group; and, b) selecting multiple groups from the multiple entities so that each group is a subset of the multiple entities.
11. 11) A method according to any one of the claims 1 to 10, wherein the method includes: a) determining at least one entity selection in accordance with user input commands, the at least one entity selection including at least one of: i) at least one entity to be included in the group; and, ii) at least one entity to be excluded from the group; and, b) selecting the group in accordance with the at least one entity selection.
12. 12) A method according to any one of the claims 1 to 11, wherein the method includes: a) modifying at least one metric in accordance with user input commands; and, b) selecting the group using the at least one modified metric.
13. 13) A method according to any one of the claims 1 to 12, wherein the at least one metric includes: a) a historical performance variance; b) a projected performance; c) a projected performance to entity value ratio; d) a comparison of a projected performance and a historical performance; e) an expected performance derived based on an entity value; and, f) a comparison of a historical performance and an expected performance.
14. 14) A method according to any one of the claims 1 to 13, wherein the method includes: a) calculating an average entity value for a number of entities; b) calculating an average performance for the number of entities; and, c) calculating an expected performance per unit entity value using the average entity value and the average performance.
15. 15) A method according to any one of the claims 1 to 14, wherein the method includes: a) selecting a set including a number of entities; and b) calculating the metrics for the set.
16. 16) A method according to claim 15, wherein the method includes selecting the set at least partially in accordance with the attributes of the entities.
17. 17) A method according to any one of the claims 1 to 16, wherein: a) the attributes include at least one of: i) an entity type; and, ii) an entity value; and, b) the selection criteria define: i) a combination of entity types; and, ii) a total entity value for the plurality of entities.
18. 18) A method according to any one of the claims 1 to 17, wherein the method includes retrieving at least one of the behaviour data and attribute data from a remote data source.
19. 19) A method according to any one of the claims 1 to 18, wherein the method includes at least one of: a) filtering the behaviour data; and, b) cleaning the behaviour.
20. 20) A method according to any one of the claims 1 to 19, wherein the method includes analysing the behaviour data using at least one predictive model.
21. 21) A method according to claim 20, wherein the method includes: a) analysing the behaviour data using a plurality of predictive models to determine a plurality of metric scores for a given metric; and, b) combining the plurality of metric scores to determine the given metric.
22. 22) A method according to claim 20 or claim 21, wherein the method includes: a) obtaining historical behaviour data; b) segmenting the historical behaviour data into a training set and a testing set; c) using the training set to train at least one predictive model; and d) testing the at least one predictive model using the testing set.
23. 23) A method according to claim 22, wherein the method includes, iteratively improving the model using the training and testing sets.
24. 24) A method according to claim 1, wherein the method includes, providing an indication of a group to a user client device via a communications network, the user client device being responsive to the indication to display a representation of the group to a user.
25. 25) A system for selecting a group of entities, the system including in one or more electronic processing devices that: a) obtain behaviour data indicative of a historical behaviour of a plurality of entities; b) obtain attribute data indicative of attributes of the plurality of entities; c) analyse the behaviour data to determine at least one metric indicative of a projected future behaviour for each of the plurality of entities; d) determine selection constraints defining restrictions on combinations of attributes for the group; and, e) select a group of entities using: i) the attributes and the selection constraints to thereby ensure the selection constraints are met; and, ii) the metrics to attempt to obtain a desired projected future behaviour for the group.