GB2546991A - Digitally mediated user classification - Google Patents

Abstract

A method of classifying a user of a vending apparatus comprising; obtaining information about the user via at least one sensor 310, transmitting the information to a remote device for analysis 320, receiving a signal indicating whether to permit or prevent dispensing of a product 350, and dispensing if permitted 360. The method may include the user requesting a product 300. The remote device may determine the users eligibility based on the users identity or apparent age. The user may have a profile that the device can access, which may include users age, pictures, voice samples or other biometric information. The signal may indicate that a local operator must assess the user in person. A separate method at the remote device may present the information to a remote operator for analysis 330. The remote device may calculate an age verification confidence rating

Description

DIGITALLY MEDIATED USER CLASSIFICATION

This invention relates to the field of digitally mediated user classification.

Age verification is often required as part of a transaction for goods or services; in the UK for example it is required by law that a customer must be at least 18 years old to be allowed to purchase alcohol. Many stores may enact a policy such as ‘Challenge 25’, which requires staff to ask any customers appearing under a certain age (in this case, 25) to provide photographic ID as proof of age. This may result in a large number of prospective customers that are required to present ID, which may present a burden to staff at a store.

Such a burden may be particularly inconvenient in the context of the use of a self-serve checkout. Self-serve checkouts have become increasingly popular in recent years as they enable a single member of staff to supervise a plurality of checkouts simultaneously, thus allowing a greater number of checkouts to be provided (which generally results in a shorter queuing time for customers) without increasing the staffing needs for a store. A drawback that may be associated with such an arrangement is encountered when a plurality of prospective customers are attempting to purchase age-restricted products at the same time. This may result in the single member of staff being required to check the photographic ID of a plurality of prospective customers simultaneously, or at the very least making a determination of their age by sight, and then approving or denying the sale of the product as a result. As a consequence of this requirement a large queue may be formed, resulting in customer dissatisfaction. A further scenario in which current age verification methods may be considered inadequate is in the field of vending machines. Vending machines selling age-restricted products are generally rare due to the lack of discrimination of customers; if a customer is able to use the machine, they are able to buy the product. One method that has been used to address this problem is the adult-only vending machine produced by Intel and Kraft Foods for the dispensing of sample products to adults only. This device relies on image analysis to group consumers by gender and as a member of one of four different age brackets and provides a product in response to this. The software provided analyses images in order to determine measurements such as the distance between a consumer’s eyes as a basis for the grouping.

However there are a number of drawbacks associated with such an arrangement, such as those related to the computer-implemented method for determining the age of the prospective customer. The ability of computers to correctly and reliably estimate age is known to be rather poor relative to the ability of a person to perform the same task, and therefore computer-implemented age verification methods are known to generally be poor. In the arrangement described above for example, a simple measurement of facial features may not be sufficiently accurate to comply with legal requirements regarding the sale of age-restricted products if it allows underage customers to purchase restricted goods. Additionally access to age-restricted products may be denied to those who are old enough to be allowed to purchase them in the opposite scenario.

These problems are evidenced by the fact that such systems are not used to provide age-restricted products in the current market, with devices such as vending machines that supply age-restricted products being located exclusively in age-restricted environments such as bars.

Similar problems may be associated with other restrictions on providing goods, for example that of prescription drugs. Currently prescription drugs are distributed by a pharmacy which is staffed by a number of human operators, but it may be advantageous to automate at least a part of the distribution process such that a number of drugs could be obtained without having to communicate with a human. Such an arrangement may provide benefits to service time and dispensary operating efficiency, but it may also be seen as impractical as the dispensing of drugs should only be performed for a specific drug and recipient (such as a person who has been prescribed a drug, or a doctor who is wishing to distribute the drug to patients themselves); therefore the identification of the recipient is required before dispensing drugs. As a result of such problems, automated dispensing arrangements are not implemented for drugs which should only be dispensed to authorised recipients.

It is an object of the present invention to mitigate or alleviate the above-described problems, and to provide a reliable, digitally mediated user classification arrangement.

The present disclosure is defined by claim 1.

Further respective aspects and features of the disclosure are defined in the appended claims.

Embodiments of the disclosure will now be described with reference to the accompanying drawings, in which:

Figure 1 schematically illustrates a vending arrangement;

Figure 2 schematically illustrates a digitally mediated age verification method; and

Figure 3 schematically illustrates a digitally mediated age verification method.

Referring now to Figure 1, this schematically illustrates a vending arrangement. This comprises a vending machine 100 as an example of a vending apparatus, although other examples such as self-serve checkouts and devices (such as the Sony® PlayStation 4®) for providing access to online stores may be suitable vending apparatuses. Such apparatus may allow a consumer to buy or otherwise obtain a physical product, either in person or to be delivered at a later time, or a digital product such as a movie or game download. It is envisaged in the present disclosure that such products may have an age restriction, preventing those below a certain age from legally purchasing a product, or some other restriction on the dispensing of a product to a user of the vending apparatus.

The vending machine 100 comprises a selection panel 110 to allow a prospective customer to indicate which product it is that they would like to purchase, for example using a number pad to input a number corresponding to a product. The vending machine 100 is also provided with one or more sensors 120 such as a camera, microphone or fingerprint scanner; it should be noted that any other sensor known in the art may be provided instead of or in addition to any of these examples. A speaker 130 may also be provided, for example so that announcements may be made by the vending machine 100. A transmitter/receiver component 140 is provided to transmit at least information obtained from one or more sensors and receive at least a signal indicating whether to permit or prevent a product being dispensed. The vending machine 100 comprises a processor 150 that is operable to implement any of the methods described below. The vending machine 100 also comprises a dispenser 160 operable to dispense a product to the prospective user if the received signal indicates that the dispensing is permitted.

The vending machine 100 is provided with power from a power supply 170, and is connected to remote device 190 via a network 180. The network 180 may be a local network, or it may be an internet connection. Connections are shown as being wired connections, but wireless connections may instead be used if it is considered appropriate. The remote device 190 may be on-site (such as in a different room) or in a completely different location to the vending apparatus, and provides at least communication and processing capability.

In an embodiment in which a device such as the Sony® PlayStation 4® is the vending apparatus, the selection panel 110 is replaced with an input device such as a controller which allows a user to communicate their selection.

Figure 2 schematically illustrates a digitally mediated age verification method. At an initial step 200 there is a detection of a request to obtain a product by a prospective user of a vending apparatus such as the vending machine 100 of Figure 1. This detection also comprises an identification of the product, or at least an identification that the dispensing of the product is intended to be restricted in some way; for example, an age restriction or a restriction such that only authorised people may be allowed to obtain the item. This could be implemented by recognition of an item, such as at a self-serve checkout when a user scans a barcode that is associated with a specific product, or by any other means, for example an attempt to use a vending machine that only supplies age-restricted goods.

At a step 210 one or more sensors are used to obtain information about a prospective user of the vending apparatus. For example, in the context of the apparatus schematically illustrated in Figure 1, the camera 120 could be used to capture an image of the prospective customer, the microphone 130 used to capture a voice input of the prospective customer and/or the fingerprint scanner 140 used to obtain an image of a prospective customer’s fingerprints. As described above, any other sensors known in the art may be provided, and a plurality of such sensors may be provided rather than just a single one. This information may be analysed locally by the vending apparatus or transmitted to be analysed remotely, or a combination of the two could be implemented.

In one example, a camera is used to capture images that may be used to estimate the age of a person in the image. Such an image may simply capture the face of a person, or it may show a user’s full body. ID may also be provided to the vending apparatus as a proof of age as part of the obtaining information from sensors step. This could for example be in the form of showing a driving licence to a camera and transmitting an image of the licence to the remote device for subsequent comparison with an image of the customer captured previously. Similarly, a fingerprint scan may be compared with scans held on record at the remote device or a trusted third party. Alternatively, or in addition, a form of identification may be able to be identified using other means; an example of this is the electronic chip that is currently used in many passports. Such a form of identification may comprise the provision of a prospective customer’s image and date of birth by a third party rather than having to determine this information from an image of the ID document. This may aid comparison of a captured image of the user to an ID document, especially in the case that a low-quality image is obtained by the vending apparatus.

At a step 220 transmission of the obtained information to a remote device for analysing information is performed. Such an analysis of the transmitted information comprises performing at least one of image analysis, voice analysis, data comparison or biometric analysis, for example.

Data that is transmitted may be examined by a computer, such as performing a digital analysis on an image or voice sample. In the context of the metrics described above, a processor could perform an image analysis to determine a person’s height or to detect wrinkled skin. An audio analysis of a voice sample could comprise a detection of the pitch or any other property of the audio.

The most reliable indicators of age are height, hair colour and skin texture. Height can generally be used to distinguish children from adults, although of course adults of short stature may be miss-classified if this is the only metric used.

Meanwhile grey or partially grey hair can be reasonably assumed to be indicative of an older person.

Alternatively or in addition, skin texture can be used to estimate the user’s age. Skin texture can be evaluated by selecting regions of skin on the face (for example adjacent to and parallel with the eyes and lips) and normalising the image in that region before measuring the variance of pixel values within the region. Smooth skin will have low variance, whilst older skin and wrinkled skin will have a higher variance. There is a rough correlation between this variance and the age of the user.

Alternatively or in addition a frequency analysis of the regions can be performed to detect wrinkle features, which cause a characteristic peak in image frequency components distinct from either skin (higher frequency) or shading due to facial curvature (lower frequency).

In any event, optionally the device may classify the user’s age to the degree of granularity possible based upon the indicators used. Hence for example it may classify a user as a child (e.g. as a function of short height, non-grey hair, and/or smooth skin tone), an adult (e.g. as a function of normal height, mostly non-grey hair and less smooth skin tone), or a retired person (e.g. as a function of grey or partially grey hair and wrinkled skin tone).

In general, a better estimation of a person’s age may be generated using a more advanced analysis method, but this requires either more processing power to be supplied to perform the analysis or more time for the analysis to be performed in. It is usually desirable to reduce the amount of time a user spends waiting for an analysis to be performed, and as a result an increase in processing power may be considered the more appropriate option.

However because of the associated costs of providing a large amount of computing power such a processing system would be impractical to supply locally to each vending apparatus. Providing access to such a system on a remote basis is a much more efficient arrangement, as the vending apparatuses could each request an age classification via a network connection. Providing such an estimate would require a trivial amount of time for a suitable processing system, and so a single system could be used to provide estimations for a whole network of vending apparatuses.

Alternatively or in addition to step 220, at a step 230 information obtained by the one or more sensors is presented to a remote human operator for additional analysis. The remote operator is able to assess the information to assist in making a determination of whether the prospective customer is old enough to purchase an age-restricted product. In one example, this step comprises the transmission of an image of the prospective customer to a remotely located person who is able to determine whether the person appears to be old enough to purchase the product or not

In some embodiments the step 230 may be performed before the analysis by the remote device. In such an arrangement, the analysis by the remote device may not be performed at all if the remote operator is sufficiently convinced that the prospective user of the vending apparatus should be permitted to obtain a product. For example, if the remote operator believes the prospective customer is old enough upon seeing an image of the prospective customer; in line with the ‘Challenge 25’ scheme described above, for example, if an operator believes that the prospective customer looks over 25 they may approve the sale without requesting to view any ID or using image analysis or the like.

Alternatively, information may not be presented to a remote operator if the computer-based analysis of the information obtained by the sensors is considered to be conclusive; for example, if the image analysis indicates that a prospective user is approximately 80 years old there is no need for this to be independently verified by a remote operator if the analysis is considered to be sufficiently reliable. Conversely, if the prospective user is estimated to be 23 years old then a remote operator may be required to approve the supply of a product manually in keeping with the Challenge 25 scheme.

At a step 240 the vending apparatus receives a signal indicating whether to permit or prevent a product being dispensed to the prospective user. A determination of whether a customer is permitted to make a purchase may be stored locally by the vending apparatus so as to allow a customer to make further purchases without being referred to an operator again. Further purchases may refer to either a user purchasing further items in the same transaction, or an embodiment in which the vending apparatus creates a profile for the user and simply relies on identification of the user for future transactions rather than having to undertake the entire verification process again.

At a step 250 a product is dispensed to the prospective user if the received signal indicates that the dispensing is permitted. In the context of the vending apparatus being a device that enables access to an online store, this step may comprise the dispatch of an item for delivery to the customer or the beginning of a download of a file.

It should be understood that the method described above with reference to Figure 2, as well as that described below with reference to Figure 3, may be applied to the provision of products other than those which are age restricted. For example, a corresponding method could be applied to determine whether a person attempting to purchase (or otherwise acquire) a product is authorised to do so; an example of this is a doctor obtaining pharmaceutical products from a vending machine, as in such an example members of the public should not be allowed to access these products freely without a prescription.

The corresponding method to that described with reference to Figure 2 for the remote device comprises a first step of receiving information about a prospective user of a vending apparatus from one or more sensors associated with the vending apparatus, a second step of analysing the received information, and a third step of transmitting a signal to the vending apparatus indicating whether to permit or prevent a product being dispensed to the prospective user. Further steps may be performed and additional features provided in conjunction with this process in line with those described with reference to the above method; an example is the analysis of the received information by a remote operator in addition to or instead of processing by the remote device.

Figure 3 schematically illustrates a digitally mediated age verification method that comprises a step of generating an age verification confidence rating (AVCR). This is an example of a method in which a remote device identifies whether the prospective user appears to meet age requirements for the dispensation of a product.

At a step 300 there is an identification of a request to obtain a product by a prospective user.

At a step 310 one or more sensors are used to obtain information about the prospective customer, as in the corresponding step 210 described above with reference to Figure 2.

At a step 320 the vending apparatus transmits the obtained information to a remote device for analysing information obtained by the one or more sensors. An analysis is performed by the remote device so as to generate an indication as to whether or not the prospective user is old enough to legally obtain the requested product.

In addition to, or instead of, the step 320 at a step 330 information obtained by the one or more sensors is presented to a remote operator in order to obtain a further estimation of the prospective user’s age. Such an estimate may be used when generating the AVCR. This step may instead be performed after the step 340 if the AVCR based upon analysis by the remote device is considered to be inconclusive, or not performed at all if the AVCR is considered to be sufficiently conclusive. If this step is performed in response to an inconclusive AVCR, the estimate of the remote operator may be used as a final decision as to whether the dispensing of a product proceeds or it may be used in the generation of a new AVCR rating.

At a step 340 an AVCR is generated by the remote device from the results of the analysing of the transmitted information in order to estimate the prospective user’s age, the age verification confidence rating being indicative of the certainty of a prospective user being old enough to be permitted to obtain an age-restricted product. In an exemplary embodiment, a prospective customer enters their date of birth as an input to the vending apparatus, provides some form of identification document and has their image captured by a camera associated with the vending apparatus. A signal to be transmitted to the vending apparatus may be generated to indicate that a product should be dispensed if the age verification confidence rating exceeds a threshold value.

The AVCR is generated using the provided information; as a non-limiting example, an AVCR could be comprised of: 1. A system-level age estimation from camera sensors; 2. A system-level age estimation from microphone sensors; 3. Additional age estimates from any additional sensors; 4. Examined identification documents; and 5. A date of birth entered by the prospective customer.

It will appreciated that not all of the above-listed factors may be required. For example, the date of birth may be laborious to enter and so may not be requested or may take the form of the user being requested to indicate an age range that they fall within instead (such as 18-25, 25-40, 40-65, or 65+) as this simplifies the input whilst still providing age-related information. A combination of a number of these factors (or any other appropriate estimates or indications of age) may provide a determination of whether the prospective customer appears to be old enough to purchase an age-restricted product. For example, a value between 0 and 1 may be produced where a value of 0 represents a certainty that the prospective customer is not old enough to purchase an age-restricted item and a value of 1 represents a certainty that the prospective customer is old enough to do so. A value of over 0.9 could be sufficient to allow for a sale to proceed, as this represents an almost-certainty, or any other threshold value could be used. A different value of certainty may be required for different age-restricted products; for example, the threshold could be lower for a 15-rated game than for a box of matches as it may be considered that a box of matches would be more dangerous if supplied irresponsibly.

Each of the factors listed above may also have an associated weighting that corresponds to how reliable the provided information is. For example, the date of birth entered by a user could be given a low weighting due to the fact that an underage person would be likely to simply provide a false date of birth in order to increase their chances of being allowed to purchase an age-restricted product. The weighting given to a computer-implemented estimation of a prospective customer’s age could vary depending on how reliable the estimation method is perceived to be. An exemplary AVCR calculation is provided below.

Consider an arrangement in which age estimation is performed using only the input date of birth and computer-implemented estimates of a prospective customer’s age based upon a captured image and voice sample. The results of the analysis the transmitted information may be associated with a corresponding weighting to be used when calculating the AVCR. Each of these pieces of information also has an associated accuracy, wherein the accuracy is a measure of how likely it is that a prospective customer is old enough to purchase an age-restricted product and the weighting is a indicative of how reliable the measure is.

The date of birth of a user may be taken to have 100% accuracy, as given a specific date of birth it is possible to ascertain with 100% accuracy whether a person with that date of birth is old enough to purchase an age-restricted product. However in view of the prospective customer’s ability (and motivation, as the prospective customer would like to obtain the product) to be dishonest this measure may be assigned a low weighting value, such as 0.1. Alternatively, if the provided date of birth does not indicate that the prospective customer is old enough to purchase the age-restricted product then there is an accuracy of 0% assigned. However, for the calculation of this example it is assumed that the prospective customer has entered a date of birth that would allow them to purchase the age-restricted product.

Assuming that estimations based on voice and appearance are equally reliable, they may receive an equal weighting of 0.45. If the estimate of the prospective customer based upon an image results in a 70% certainty of the prospective customer being old enough and a voice-based estimate being 50% certain, a calculation of the AVCR would be as appears below. An appropriate formula is: AVCR = W(1)A(1) + W(2)A(2) + W(3)A(3) in which W(n) is the weight assigned to the ntfl estimate method and A(n) is the accuracy of the determination, by the nth estimate method, that the prospective customer is old enough. In the above-described example, the following sum is obtained: AVCR = 0.1 * 1 + 0.45 * 0.7 + 0.45 * 0.5 wherein the percentages have been converted to decimals for ease of calculation. The AVCR in this example is: AVCR = 0.1 + 0.315 + 0.225 = 0.64

An AVCR of 0.64 indicates that based upon the estimates used there is a 64% likelihood that the prospective customer is old enough to buy the age-restricted product. This may be sufficient for some products to allow a sale to proceed without further intervention by a remote operator, or it may be low enough to require the remote operator to intervene and make a decision.

In the latter case, a camera associated with the vending apparatus may be used to provide an image to the remote operator in order for the operator to make a judgment as to the age of the prospective customer.

The remote operator or remote device is then operable to transmit a signal indicating that the outcome of the age verification method is any one of at least the following 4 decisions: 1. Allow the transaction to proceed; 2. Request further evidence of the age of the prospective customer; 3. Cancel the transaction; or 4. Manually inspect the prospective customer in person.

If the fourth option is selected, then a local operator is informed that an inspection of the prospective customer is required in person. The local operator will then make an independent assessment of the age of the prospective customer, and decide whether or not the provision of the product should be allowed. This may be based on either a by-eye assessment of age, or by further examination of identification documents. The local operator may then either approve the provision of the product locally, for example by entering a security code to the vending apparatus, or may send a signal to the remote operator to inform them whether to transmit a signal to allow or cancel the transaction.

At a step 350 a signal indicating whether to permit or prevent a product being dispensed to the prospective user is received by the vending apparatus. As described above, this signal may indicate other options (such as indicating that the product should not be dispensed until a local operator is able to assess the eligibility of the prospective user in person) rather than being a mere binary decision.

At a step 360 a product is dispensed to the prospective user if the received signal indicates that the dispensing is permitted. This may comprise the provision of a physical good to a user of the vending apparatus, or the vending apparatus may provide a copy of a digital product such as a movie or game to a user. In embodiments of the present disclosure, a process may be initiated by which a 3D printer associated with the vending device is controlled to print an object in response to product dispensing being permitted. So, the product to be dispensed may be one or more of a digital product, a physical product, or a physical product created (for example, printed) in response to digital data.

As described above, this method could be applied to the provision of products that are restricted subject to criteria other than age. In an embodiment in which confirmation of a user’s identity is required, the remote device identifies whether the prospective user is permitted to obtain the product to be dispensed based upon the prospective user’s identity. Image processing may be performed to compare a captured image to a previously stored image, and a corresponding confidence rating (‘Identity Verification Confidence Rating’, IVCR) may be generated to indicate how likely it is that the two images are of the same person. A prospective user of a vending apparatus may pre-register their details in order to create a profile that may be accessed by an appropriate vendor.

The prospective user has an associated profile that the remote device may access, the profile comprising one or more of age information, images of the prospective user, voice samples of the prospective user or any other biometric identifiers. Any other appropriate information may also be included in the prospective user’s associated profile. By having such a profile it is not essential to perform processing to determine the user’s age when they are attempting to obtain a product from a vending apparatus, it is only required that identification of the user is performed as this allows the correct profile to be accessed and thus the verified age of the user may be obtained from the profile associated with the user.

Consequently it should be understood that the methods of Figure 2 or Figure 3 may be carried out without the first step of identifying the attempted purchase of an age-restricted product; for example, in the context of an online account it may be preferable for a user to perform age verification upon creating the account rather than waiting for an attempted purchase to perform age verification.

The techniques described above may be implemented in hardware, software or combinations of the two. In the case that a software-controlled data processing apparatus is employed to implement one or more features of the embodiments, it will be appreciated that such software, and a storage or transmission medium such as a non-transitory machine-readable storage medium by which such software is provided, are also considered as embodiments of the disclosure.

Claims (15)

1. A digitally mediated user classification method for use with a vending apparatus, the method comprising: using one or more sensors to obtain information about a prospective user of the vending apparatus; transmitting the obtained information to a remote device for analysing information obtained by the one or more sensors; receiving a signal indicating whether to permit or prevent a product being dispensed to the prospective user; and dispensing a product to the prospective user if the received signal indicates that the dispensing is permitted.

2. The method of claim 1, wherein the remote device identifies whether the prospective user is permitted to obtain the product to be dispensed based upon the prospective user’s identity.

3. The method of claim 1, wherein the remote device identifies whether the prospective user appears to meet age requirements for the dispensation of the product.

4. The method of claim 1, the method comprising detecting a request to obtain a product by the prospective user.

5. The method of claim 1, wherein the prospective user has an associated profile that the remote device may access, the profile comprising one or more of age information, images of the prospective user, voice samples of the prospective user or any other biometric identifiers.

6. The method of claim 1, wherein the received signal may indicate that the product should not be dispensed until a local operator is able to assess the eligibility of the prospective user in person.

7. The method of claim 1, wherein the vending apparatus is a device providing access to an online store.

8. A digitally mediated user classification method for use at a remote device, the method comprising: receiving information about a prospective user of a vending apparatus from one or more sensors associated with the vending apparatus; analysing the received information; and transmitting a signal to the vending apparatus indicating whether to permit or prevent a product being dispensed to the prospective user.

9. The method of claim 8, wherein the information analysis comprises presenting information to a remote operator for additional analysis.

10. The method of claim 8, wherein the analysing of the received information comprises performing at least one of image analysis, voice analysis, data comparison or biometric analysis.

11. The method of claim 10, wherein: an age verification confidence rating is generated from the results of the analysing of the received information in order to estimate the prospective user’s age, the age verification confidence rating being indicative of the likelihood of a prospective user being old enough to be permitted to obtain an age-restricted product; and the transmitted signal indicates that a product should be dispensed if the age verification confidence rating exceeds a threshold value.

12. The method of claim 11, wherein the results of analysing the received information are associated with a corresponding weighting to be used when calculating the age verification confidence rating.

13. Computer software which, when executed by a computer, causes the computer to carry out the method of any one of the preceding claims.

14. A machine-readable non-transitory storage medium which stores computer software according to claim 13.

15. A vending apparatus for use with a digitally mediated user classification method, the apparatus comprising: one or more sensors operable to obtain information about a prospective user of the vending apparatus; a transmitter operable to transmit the obtained information to a remote device for analysing the information obtained by the one or more sensors; a receiver operable to receive a signal indicating whether to permit or prevent a product being dispensed to the prospective user; and a dispenser operable to dispense a product to the prospective user if the received signal indicates that the dispensing is permitted.