GB2474240A - A method for matching users probability estimates - Google Patents

A method for matching users probability estimates Download PDF

Info

Publication number
GB2474240A
GB2474240A GB0917471A GB0917471A GB2474240A GB 2474240 A GB2474240 A GB 2474240A GB 0917471 A GB0917471 A GB 0917471A GB 0917471 A GB0917471 A GB 0917471A GB 2474240 A GB2474240 A GB 2474240A
Authority
GB
United Kingdom
Prior art keywords
user
probability
event
users
outcome
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
GB0917471A
Other versions
GB0917471D0 (en
Inventor
Dylan Evans
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University College Cork
Original Assignee
University College Cork
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University College Cork filed Critical University College Cork
Priority to GB0917471A priority Critical patent/GB2474240A/en
Publication of GB0917471D0 publication Critical patent/GB0917471D0/en
Publication of GB2474240A publication Critical patent/GB2474240A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06QDATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/04Forecasting or optimisation, e.g. linear programming, "travelling salesman problem" or "cutting stock problem"
    • G06F17/608

Abstract

Users are required to provide probability estimates of what the outcome of a particular event will be or the truth of a particular proposition. Users are paired together and a wager formed between the pair of users with one user taking the position that there will be one particular outcome/truth of proposition and the other user taking the position that the opposite outcome/truth of proposition will occur. The user who takes the position of the eventual outcome/truth of proposition is rewarded and the other party is penalised. Preferably this method is used to train the accuracy of probability estimates provided by users.

Description

"A method for improving the accuracy of probability estimates"

Introduction

This invention relates to a method for improving the accuracy of probability estimates.

There are many professions that require individuals to make probability estimates relating to the outcome of an event. For example, weather forecasters must make probability estimates as to the likelihood of precipitation occurring in a given region at a given time, medical practitioners must make a probability estimate as to the cause of certain symptoms being displayed by a patient before initiating a course of treatment, and market traders must make probability estimates as to the likelihood of a particular stock rising or falling in value. The accuracy with which these individuals can make the probability estimates has a direct effect on their success or failure in their chosen field.

However, the majority of the population are relatively poor at making probability estimates and their estimates are very often wildly inaccurate. Even professionals with many years of experience in a field where they are required to make probability estimates on a frequent basis can display poor calibration. This is due primarily to universal biases and the fact that a person's natural disposition and their personality can have a significant bearing on their probability estimation performance. A person having an optimistic or carefree disposition often has a tendency to exhibit overconfidence in their probability estimation and a person having a pessimistic or cautious disposition often has a tendency to exhibit under-confidence in their probability estimation.

Perhaps of more significance than individual preferences are universal biases. For example, a person may exhibit a cognitive bias such as being overconfident in general.

If a person is overconfident, they will estimate that there is more of a probability of an outcome occurring than there is. These characteristics cause inaccuracies in their probability estimates.

It is an object of the present invention to provide a method for improving the accuracy of probability estimates that overcomes at least some of the problems with the known methods that is simple and effective to perform.

Statements of Invention

According to the invention there is provided a method for improving the accuracy of user probability estimates comprising the steps of: requesting a user to provide a probability estimate of an outcome of an event; receiving the probability estimate from the user; receiving a probability estimate from at least one other user; pairing each of the users with one other user; creating a wager between each pair of users; obtaining the outcome of the event; comparing the probability estimate received from the user with the outcome of the event; notifying the user of the outcome of the event; and rewarding one of the pair of users and penalizing the other of the pair of users based on the accuracy of their probability estimates when compared with the outcome of the event.

By providing such a method, it will be possible to improve the accuracy of user probability estimates over time. The user will be able to gauge their performance against their prior performance and their peers. By providing such a method, it is not "gameable", or in other words, it is not possible for a person to manipulate the result of the method by cheating. The method is therefore useful as a training tool.

In one embodiment of the invention there is provided a method in which the step of pairing each of the two or more users with one other user comprises pairing users with identical probability estimates.

In another embodiment of the invention there is provided a method in which when users with identical probability estimates are paired together, one of the users is selected at random to wager that a particular outcome of the event will occur and the other user is selected to wager that the opposite outcome of the event will occur.

In a further embodiment of the invention there is provided a method in which the step of pairing each of the two or more users with one other user comprises pairing the probability estimates in order of decreasing proximity.

In one embodiment of the invention there is provided a method in which when two users with different probability estimates are paired together, one of the users is selected to wager that a particular outcome of the event will occur and the other user is selected to wager that the opposite outcome of the event will occur.

In another embodiment of the invention there is provided a method in which the user that provided a probability estimate that a particular outcome is more likely to occur than the other user is selected to wager that that particular outcome of the event will occur and the other user is selected to wager that the opposite outcome of the event will occur.

In a further embodiment of the invention there is provided a method in which the mean value of the two estimates is calculated and an amount proportional to the mean value is taken as a wager value.

In one embodiment of the invention there is provided a method in which an amount proportional to the wager value is frozen in an account of the user selected to wager that that particular outcome of the event will occur.

In another embodiment of the invention there is provided a method in which a second amount proportional to the wager value is frozen in an account of the user selected to wager that that opposite outcome of the event will occur.

In a further embodiment of the invention there is provided a method in which the step of rewarding one of the pair of users and penalizing the other of the pair of users comprises unfreezing the account of the user whose probability estimate relating to the outcome of the event was correct.

In one embodiment of the invention there is provided a method in which the step of rewarding one of the pair of users and penalizing the other of the pair of users comprises debiting the account of the user whose probability estimate relating to the outcome of the event was incorrect by the amount frozen in their account.

In another embodiment of the invention there is provided a method in which the step of rewarding one of the pair of users and penalizing the other of the pair of users comprises crediting the account of the user whose probability estimate relating to the outcome of the event was correct by the amount frozen in the other users account.

In a further embodiment of the invention there is provided a method comprising the step of building a trained user calibration curve based on the probability estimates received from the user.

In one embodiment of the invention there is provided a method comprising the step of measuring the progress of an individual by calculating a score for the user based on the area between the calibration curve and an identity line.

In another embodiment of the invention there is provided a method in which the score is inversely proportional to the size of the area between the calibration curve and the identity line.

In a further embodiment of the invention there is provided a method comprising the initial step of building an untrained user calibration curve and the subsequent step of comparing the trained user calibration curve and the untrained user calibration curve.

In one embodiment of the invention there is provided a method in which the step of rewarding or penalizing the user based on the accuracy of their probability estimate comprises the step of crediting or debiting a user account.

In another embodiment of the invention there is provided a method in which one of the events is a future event whose outcome is not yet known to the user.

In a further embodiment of the invention there is provided a method in which the proposition describes an historical event.

In one embodiment of the invention there is provided a method in which the event is a fact.

In another embodiment of the invention there is provided a computer program for implementing the method.

In a further embodiment of the invention there is provided a computer readable medium carrying a computer program for implementing the method.

In one embodiment of the invention there is provided a training product for training the probability estimation of a user comprising a computer program having program instructions for carrying out the method steps of: requesting a plurality of users to each provide a probability estimate of an outcome of an event; receiving the probability estimates from the users; pairing each of the users with one other user; creating a wager between each pair of users; obtaining the outcome of the event; comparing the probability estimate received from the user with the outcome of the event; notifying the users of the outcome of the event; and rewarding one of the each pair of users and penalizing the other of the each pair of users based on the accuracy of their probability estimates when compared with the outcome of the event.

Detailed Description of the Invention

The invention will now be more clearly understood from the following description of some embodiments thereof given by way of example only with reference to the accompanying drawings, in which:-Figure 1 is a schematic representation of a probability estimation graph; and Figure 2 is a diagrammatic representation of a computer network in which the method according to the invention can be performed.

Referring to the drawings and initially to Figure 1 thereof, there is shown a schematic representation of a probability estimation graph, indicated by the reference numeral 1.

The probability estimation graph comprises a Y axis 3 which represents the real probability of an event occurring, an X axis 5 which represents the estimated probability of an event occurring and a straight line 7 which is the ideal state where the estimated probability equals the real probability. The zone 9 represents an under-confidence in estimated probability and the zone 11 represents an over-confidence in estimated probability.

If an individual is asked over a period of time what the likelihood of there being rain the following day is, and they respond that there is a 50% chance of there being rainfall the following day and it is found that the frequency of there being rain was 50% on each of the days, they would be precisely on the straight line 7 as the estimated probability, or prediction, was equal to the real probability, or frequency. This point is indicated on the probability estimation graph with the numeral 13. If however they predicted that there was a 75% chance of rainfall the following day and it is found that the frequency of there being rain was only 50%, they would be at point 15 in the zone 11 indicative that they were overconfident of there being rain, or in other words, they were overconfident of there being a particular outcome. If on the other hand they predicted that there was a 25% chance of rainfall the following day and it is found that the frequency of there being rain was 50%, they would be at point 17 in the zone 9 indicative that they were under-confident of there being rain, or in other words, they were under-confident of there being a particular outcome.

Due to a person's cognitive biases, they can have a tendency to be either overconfident in their approach in which case they are most frequently in the zone 9, or under-confident in their approach, in which case they are most frequently in the zone 11. Even more likely is the scenario where they will be overconfident in some regions and underconfident in other regions. Over time, an individual's performance can be plotted as their calibration curve. The calibration curve is indicated by the reference numeral 19.

The calibration curve 19 shows an instance where the individual is overconfident in some areas and underconfident in other areas. In the area marked "A", the individual displays underconfidence and in the area marked "B" the individual displays overconfidence. For example, if asked what the probability of there being rain the following day was and they estimate that there is a 10% chance of rain the following day, indicated by point 21, when it is found that there is a 30% frequency of there actually being rain the following, they were underconfident that there would be rain in those situations where there was a 30% chance of rainfall. Similarly, if asked what the probability of there being rain the following day was and they estimate that there is a 90% chance of rain the following day, indicated by point 23, when it is found that there is only a 50% frequency of there being rain the following day, they were overconfident that there would be rain. The result is the calibration curve 19 which is unique for each individual. It is desirable to train the individual so that the calibration curve 19 indicative of their performance gets closer and closer over time to the line 7, the ideal where the estimated probability or prediction is the same as the real probability or frequency. In other words, as the individual improves, the area between the calibration curve 19 and the identity line 7 will decrease in size.

Referring to Figure 2, there is shown a diagrammatic representation of a computer network, indicated generally by the reference numeral 30, in which the invention according to the invention can be performed. The computer network 30 comprises a plurality of computers 31, represented by PC's, each having a user interface 33 and a keyboard 35 for receiving user inputted data. Alternatively, the apparatus could be any electronic device with a user interface 33 and means for receiving user inputted data capable of running a program implementing the method according to the invention including and not limited to a laptop, a palmtop, a personal digital assistant, a handheld device, a mobile telephone, an electronic games console and the like. The PCs further comprise means to communicate with each other and other devices over a communications network, represented in this case by the internet 37. The computer network 31 further comprises a server 39, preferably a web server hosting a web site operating part of the method according to the present invention. The server 39 has memory (not shown).

In use, the server 39 is accessed by each of the computers 31 and the server polls the users of the computers to provide probability estimates of an outcome of an event. This could be in the form of an electronic mail or a web page with a web form for completion by the user. In one example, the users are asked to estimate the probability that a certain event will occur or that a particular proposition is true. Each user privately states their estimate by entering the estimate in a web form, preferably as a percentage or as a number between zero and one. In a particularly preferred embodiment, there are provided a plurality of percentage values, evenly spaced in increments of 10% that the user may select from. In such an instance the user is selecting the probability that they believe a certain proposition is true. The event may be a historical event or alternatively could be a future event, the outcome of which is not yet known. For example, the event could be the likelihood of there being rainfall the following day.

Each of the users transmits their probability estimate to the server 39 which stores the estimates in memory. The server then pairs each of the user's probability estimates with one other user's probability estimate. The server pairs any identical probability estimates first, followed by the next closest probability estimates and so on until all of the probability estimates are paired off in order of decreasing proximity. Once paired, if there is an uneven number of probability estimates leaving an unpaired probability estimate, this unpaired probability estimate is discarded or paired with a dummy. If there is an uneven number of probability estimates, one of the probability estimates may be selected at random to be excluded from the pairing exercise. If a dummy is used, there are a number of ways in which the wager value for the dummy could be set, for example by selecting the wager value of the dummy at random, by duplicating the next most extreme bet value of the remaining bets or by using a median value of the other bets.

The server then creates a wager between each pair of probability estimates and once the outcome of the event is known, the server rewards one of the pair of users and penalizes the other of the pair of users based on the accuracy of their probability estimates when compared with the outcome of the event. The wager could comprise a points wager or a monetary value. In order to set the wager, for the pairs of non-identical estimates, the person who made the higher estimate (or in other words, that the event is more likely to occur) is taken to be betting that the event will occur or that the proposition is true, while the person who made the lower estimate is taken to be betting that the event will not occur or that the proposition is not true. For the pairs of identical estimates, the person betting that the event will occur or that the proposition is true is determined randomly.

The mean of the two estimates is calculated, and a number of credits proportional to the mean are frozen in the account of the person betting that the event will occur or that the proposition is true. Likewise, a number of credits proportional to one hundred minus the mean (if a percentage value is used for the probability estimate, or one minus the mean if the estimates were between zero and one) is frozen in the account of the person betting that the event will not occur or that the proposition is not true. For each wager, therefore, the total amount of frozen credits is proportional to one hundred (or to one).

Once the outcome of the event is known, or in other words when it is determined whether or not the proposition is true, all the wagers are reconciled. The wagers are reconciled by un-freezing the relevant amount of frozen credits in the account of the person in each pair who was taken to have placed the winning bet. The relevant amount of frozen credits in the account of the person in each pair who was taken to have placed the losing bet is transferred to the account of the winning bettor with whom they were -10-paired in this round. Preferably, users are provided with information and feedback about the outcome and their performance on a regular basis so that they may learn from their mistakes and build on their strengths.

For example, in one embodiment of the invention, the method requires users to estimate the probability of there being rainfall in the region the following day. User A may estimate that there is a 30% probability of there being rainfall, User B may estimate that there is a 40% probability of there being rainfall, User C may estimate that there is a 50% probability of there being rainfall and User D may estimate that there is also a 40% probability of there being rainfall. The server 35 pairs User B with User D as they both estimated that there is a 40% probability of there being rainfall the following day. The server then pairs User A with User C. One of User B and User D are selected at random to be the user that is estimating that the event will occur whereas the other of User B and User D will be selected as estimating that the event will not occur. For the purpose of this example, User B will be assumed to have been selected as estimating that the event will occur and User D will be assumed to have been selected that the event will not occur. In that case, 40 credits will be frozen in User B's account and 60 credits (100 credits minus the wager of User B, 40 credits) will be frozen in User D's account. If there is rainfall the following day, the 40 credits in User B's account are unfrozen and 60 credits are deducted from User D's account and credited to User B's account. If on the other hand there is no rain the following day, the 60 credits in User D's account will be unfrozen and 40 credits will be debited from User B's account and credited to User D's account.

In the case of User A and User C, User C estimated that there was a 50% probability that there would be rain the following day whereas user A estimated that there was only a 30% probability of there being rain the following day. The mean of the two wagers is taken as the wager, in this case 40% and as user C predicted that it was more likely to rain, they are allocated the side of the wager that it will rain the following day. 40 credits is frozen in User C's account and 60 credits (100 minus the wager of User C, 40 credits) is frozen in User A's account. If there is rainfall the following day, the 40 credits in User C's account are unfrozen and 60 credits are deducted from User A's account and credited to User C's account. If on the other hand there is no rain the following day, the -11 -credits in User A's account will be unfrozen and 40 credits will be debited from User C's account and credited to User A's account.

The questions presented to the user will largely depend on who the users are. For example, if one were to provide the method to a group of medical students, one would probably use true/false statements relating to their medical curriculum, such as: * Cannabinoid receptors are expressed on T cells.

* Demyelination reduces the speed of nerve conduction.

* Inflammatory changes constitute a significant aspect of multiple sclerosis.

* There are no physiological effects of thyroid hormone on the liver.

* The tumbler test is a way of confirming the non-blanching nature of a rash.

If the software were used with the general public, questions relating to current affairs could be used, such as: * A federal government run health insurance plan will be approved by the US Congress before midnight Eastern Time on 31 December 2009.

* A majority will vote YES in the 2009 Irish Referendum on the Lisbon Treaty.

* The new Harry Potter film will gross over $15M in its opening weekend.

* A venue in North America will host the 2016 Summer Olympics.

* A cap and trade system for emissions trading will be established before midnight Eastern Time on 31 Dec 2010.

* The US Unemployment Rate in December 2009 will be greater than 10%.

Again, the above questions are only indicative of the type of question that the student/member of the public could be asked. They would be able to provide a probability that the proposition were true based on their knowledge and their certainty that the answer is correct. For example, if the medical student is absolutely certain that the proposition "Cannabinoid receptors are expressed on T cells" was true, they may wager 100% (which may be equivalent to 100 points or other point value). Similarly, if they were almost certain but had a small doubt about their answer, they may wager 80% or 90% depending on the level of doubt and if they were uncertain they may wager between 40% and 60% on whether the proposition were true.

-12 -Preferably, each user will undertake an initial calibration test to generate an untrained user calibration curve before they start engaging with the method properly. This will provide a base level of competence that can be compared to future calibration curves.

As the user carries out the method, a trained user calibration curve can be generated and compared if desired with the untrained user calibration curve. A score for each calibration curve can be calculated by dividing the area between the calibration curve 19 into half the total area of the graph. The initial calibration test would typically comprise a number of true false statements or statements that the user had to provide their probability estimate that the proposition is either true or false. The questions could include, but are not limited to, true/false statements such as the following: * The melting point of tin is higher than the melting point of aluminum.

* In English, the word "quality" is more frequently used that the word "speed".

* Any male pig is referred to as a hog.

* The Model T was the first car produced by Henry Ford.

* When rolling 2 dice, a roll of 7 is more likely than a 3.

* No one has ever been reported to have been hit by any object that fell from space.

* Sir Christopher Wren was a British anthropologist.

* Pakistan does not border Russia.

* Italian has more words than any other language.

* The month of August is named after a Greek god.

* The deepest ocean trench is deeper than the Grand Canyon.

* Abraham Lincoln was the first president born in a log cabin.

* The population of Spain is higher than the population of Libya.

Once the user has answered these and/or other questions, it will be possible to generate an untrained calibration curve for the individual. It may be preferable to ensure that the user makes a wide range of selections prior to generating the initial calibration curve. For example, in a system that allows a user to estimate in increments of 10% probabilities from 0% probability to 100% probability, it may be desirable to require the user to have completed a minimum number of answers in each section rather than simply allow the -13-user to have estimated in a very small range of the possible selections. For example, it may be necessary for the user to have estimated each possible probability value three times (e.g. 0% probability three times, 10% probability three times, 20% probability three times and so on) until all options have been selected a minimum of three times.

In order to plot the calibration curve, reference is made to the publication by Sarah Liechtenstein et al [Lichtenstein, S., B. Fischhoff, et al. (1982). "Calibration of probabilities: The state of the art to 1980", Judgement Under Uncertainty: Heuristics and Biases. D. Kahneman, P. Slovic and A. Tversky. Cambridge, Cambridge University Press: 306-334.] the entire contents of which are incorporated herein by way of reference. In order to plot a calibration curve one first collects many probability estimates for items for which the correct answer is already known or will shortly be known (this is similar to the initial calibration test described above). Similar probability estimates are then grouped together (e.g. all estimates between 0.6 and 0.69 are placed in the same bin. In the example described above, all estimates in which 30% (0.30) was estimated are grouped together, all estimates in which 40% (0.40) was estimated are grouped together and so on). Within each category, the proportion of items for which the proposition is true (i.e. the event actually occurred) is calculated and plotted (on the ordinate) against the mean probability estimate (on the abscissa).

In a calibration curve for a perfectly calibrated person, all points would fall on the identity line. The extent to which any real person approximates this ideal can be judged visually by looking at how closely the plotted data points cluster around the identity line, but it would be preferable to replace this informal judgment with a single numerical measure.

The publication by Servan-Schreiber et al [Servan-Schreiber, E., J. Wolfers, et al. (2004). Prediction Markets: Does Money matter?" Electronic Markets 14(3): 243-251.] incorporated herein by way of reference in its entirety, suggests several such measures, including the square root of the mean squared error. In this way, a numerical value can be given to indicate how accurate a person is at estimating the probability of an event.

The server may store user accounts with a points tally and will be able to update the profiles as the method is implemented. Furthermore, league tables and the like could be published showing the individuals with the most points. It can be seen from the above therefore that the method could be used in an evaluation process for certain -14 -professionals where probability estimation is important to their line of work, for example doctors, traders or weather forecasters. Furthermore, the tool could be used to train or calibrate an individual from time to time to compensate for their natural disposition. The method can be used as a game and bets could be placed on the outcomes and the ability of individuals to more accurately predict the outcome of certain types of events better than their peers. Ideally, the method could be used in the assessment of candidates and their suitability to carry out certain functions.

It is envisaged that the questions can either be directed to certain geographic areas or the answers could be paired with individuals who are in the same specific geographic area. The geographic areas could be determined by the default location of the user or by the location of the computer. This can be important for certain questions where the location may be relevant, for example, will it rain or who will win the local elections? In such instances, it may be preferable to either state the geographic area, for example, will it rain in Cork City Centre tomorrow or simply ask the question will it rain tomorrow and ascertain the geographic area of the user that responded to the query and pair that user with a user in the same geographic area.

Furthermore, it is envisaged that in certain embodiments, there will be provided sub-groups whereby a number of users can develop a community and have their own separate league table for the community. In these instances, pairing may be done within the community. If for example the invention was used in a gaming or betting capacity, in this case there may be groups within the wider community who may wish to play against each other as well as across the wider community. In such an instance, players score against the wider community but can also score separately within their own group. If desired, these groups could be set up and managed by the individual rather than by an administrator.

In the embodiment described, the method is implemented in software. The Language(s) used to develop the training tool are PHP and JavaScript. The database is a MySQL database but equally well either XHTML, SQL could have been used. In order to operate the method, any PHP and MySQL compliant web-server would suffice, for example a server running Apache with PHP and MySQL modules. The database comprises a plurality of tables including: account, Admin, admin_setting, answer, assignment, -15-authorized, bad_attempt, country, creates, ip_blacklist, message, online, online_admin, quiz, quiz_answers, quiz_history, quiz_results, student, student_pairs and transfer In the embodiments shown in Figure 2, the user devices are PCs but again it will be understood that any computing device capable of communication with other computing devices over a communication network, with a user interface, means for receiving user inputted data and capable of running a program implementing at least part of the method according to the invention could be used instead. The means for receiving user inputted data 25 could comprise a keyboard, touchpad, stylus and touchpad, a touch screen, voice recognition software and microphone and or a pointing device such as a mouse or a tracker ball. The user interface 23 could comprise a VDU and/or an audio unit.

Another possible application of the present invention would be to provide useful survey data to public bodies and private enterprise. The estimates generated by the participants in the method might contain valuable information about public opinion that third parties would be prepared to purchase. For example, the method could have targeted questions regarding the confidence of the user in the government being able to bring the economy out of recession and based on the person's probability estimate profile, it may be possible to extract useful and meaningful information from the responses. The data may be used for marketing purposes, on an individual level whereby a users personal preferences and profile may be determined or on a community level over the entire group of users or within a smaller group of users within the larger community.

In the embodiment described, the users place wagers that are paired against other players. Instead of a points value, the reward could be a score, a qualification or a permission to proceed to a more complex level whereas the penalty could be a point deduction, a disqualification or a demotion/maintenance at a particular level. In some instances, reference is made to the outcome of an event and in other instances reference is made to the truth of a proposition. It will be understood that these have been used interchangeably throughout the specification and the outcome of an event will be understood to mean the truth of a proposition and vice versa.

It will be further understood that the method according to the present invention will be performed largely in software and therefore the present invention extends also to computer programs, on or In a carrier, comprising program Instructions for causing a computer to carry out the method. The computer program may be In source code format, object code format or a format intermediate source code and object code. The computer program may be stored on or In a carrier Including any computer readable medium, including but not limited to a floppy disc, a CD, a DVD, a memory slick, a tape, a RAM, a ROM, a PROM, an EPROM, a hardware circuft or a transmissible carrier such as a carrier signal when transmitted either wirelessly and/or through wire and/or cable.

In this specification the terms comprlse comprises, comprised and comprising" and the terms "include, includes, included and kicluding" are all deemed totally interchangeable and should be afforded the widest possible interpretation.

The kiventlon is in no way limited to the embodiment hereinbefore described but may be varied in both construction and detail within the scope of the specification. -17-

GB0917471A 2009-10-06 2009-10-06 A method for matching users probability estimates Withdrawn GB2474240A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
GB0917471A GB2474240A (en) 2009-10-06 2009-10-06 A method for matching users probability estimates

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB0917471A GB2474240A (en) 2009-10-06 2009-10-06 A method for matching users probability estimates

Publications (2)

Publication Number Publication Date
GB0917471D0 GB0917471D0 (en) 2009-11-18
GB2474240A true GB2474240A (en) 2011-04-13

Family

ID=41393907

Family Applications (1)

Application Number Title Priority Date Filing Date
GB0917471A Withdrawn GB2474240A (en) 2009-10-06 2009-10-06 A method for matching users probability estimates

Country Status (1)

Country Link
GB (1) GB2474240A (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040015429A1 (en) * 2000-09-18 2004-01-22 Tighe Joseph Michael Bet matching system
EP1777666A1 (en) * 2005-10-13 2007-04-25 Cinnober Financial Technology AB Method, system and business model for electronic betting

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040015429A1 (en) * 2000-09-18 2004-01-22 Tighe Joseph Michael Bet matching system
EP1777666A1 (en) * 2005-10-13 2007-04-25 Cinnober Financial Technology AB Method, system and business model for electronic betting

Also Published As

Publication number Publication date
GB0917471D0 (en) 2009-11-18

Similar Documents

Publication Publication Date Title
Benjamin et al. Religious identity and economic behavior
Macey et al. eSports, skins and loot boxes: Participants, practices and problematic behaviour associated with emergent forms of gambling
Primi et al. The development and testing of a new version of the cognitive reflection test applying item response theory (IRT)
Buser et al. Gender, competitiveness, and career choices
Cueva et al. Cognitive (ir) reflection: New experimental evidence
Rao Familiarity does not breed contempt: Generosity, discrimination and diversity in Delhi schools
Lefebvre et al. Tax evasion and social information: an experiment in Belgium, France, and the Netherlands
Yukselturk et al. Exploring the link among entry characteristics, participation behaviors and course outcomes of online learners: An examination of learner profile using cluster analysis
Lavy Gender differences in market competitiveness in a real workplace: Evidence from performance‐based pay tournaments among teachers
Datta Gupta et al. Gender matching and competitiveness: Experimental evidence
Williams et al. Quinte longitudinal study of gambling and problem gambling
Wohl et al. Facilitating responsible gambling: The relative effectiveness of education-based animation and monetary limit setting pop-up messages among electronic gaming machine players
Eckel et al. Forecasting risk attitudes: An experimental study using actual and forecast gamble choices
Zittel Lost in technology? Political parties and the online campaigns of constituency candidates in Germany's mixed member electoral system
Simmons et al. Intuitive biases in choice versus estimation: Implications for the wisdom of crowds
Williams et al. Best practices in the population assessment of problem gambling
Pleskac et al. Development of an automatic response mode to improve the clinical utility of sequential risk-taking tasks.
Andersen et al. Preference heterogeneity in experiments: Comparing the field and laboratory
Johnes Determinants of student wastage in higher education
Benjamin et al. Thin-slice forecasts of gubernatorial elections
Booth et al. Gender differences in risk behaviour: does nurture matter?
Faff et al. On the linkage between financial risk tolerance and risk aversion
Klein et al. Perceived control and the optimistic bias: A meta-analytic review
Francis et al. Gambling motives: Application of the reasons for gambling questionnaire in an Australian population survey
Gainsbury et al. A digital revolution: Comparison of demographic profiles, attitudes and gambling behavior of Internet and non-Internet gamblers

Legal Events

Date Code Title Description
WAP Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1)