WO2018191784A1

WO2018191784A1 - Eyeglasses ordering system and digital interface therefor

Info

Publication number: WO2018191784A1
Application number: PCT/AU2018/050355
Authority: WO
Inventors: Bjorn Peter Alan Russell
Original assignee: SPEQS Limited
Priority date: 2017-04-19
Filing date: 2018-04-19
Publication date: 2018-10-25

Abstract

In accordance with various embodiments of the present invention, a system and methods are provided that may be used for the selection of glasses from a library of frames; then apply a selected frame to the user's facial image, simulating the realistic glasses appearance on each frame of a video sequence in real-time, in an unconstrained wearer simulation environment and on a commercially available consumer device. (VDS). It id provided that the system will, with little effort from the user, obtain critical optical dispensing measurements necessary for the ordering and manufacturing of the completed spectacle unit, in an unconstrained wearer simulation environment and on a commercially available consumer device. (DMM). It will also be provided an online order for placing the spectacle manufacture order on behalf of the user, and take payment for such, in an unconstrained wearer simulation environment and on a commercially available consumer device. (ECI)

Description

Eyeglasses Ordering System and Digital Interface therefor CROSS REFERENCE TO RELATED APPLICATIONS

The entire disclosure of Australian Provisional patent application 2017901427 is incorporated herein by reference.

The entire disclosure of United States Provisional patent application 61 /874,506 is incorporated herein by reference.

TECHNICAL FIELD

[0001 ] The present invention relates to online system for ordering goods and services.

[0002] The invention has been devised particularly, although not necessarily solely, in relation to online system for ordering prescription eyeglasses.

BACKGROUND ART

[0003] The following discussion of the background art is intended to facilitate an understanding of the present invention only. The discussion is not an acknowledgement or admission that any of the material referred to is or was part of the common general knowledge as at the priority date of the application.

[0004] Spectacles, glasses, sunglasses (herein referred to as spectacles or glasses) are primarily used to improve a wearer's vision, however, they are often also worn enhance or alter the appearance of a person's face for the purposes of aesthetics and image. Determining the alteration of the appearance of a face by wearing spectacles is a time-consuming task requiring the user to visit an optical store, often many optical stores, for the purpose of trying on glasses and assessing the aesthetic appearance. Consumers may wish to try on many different spectacles and view them in different conditions, in combination with different Glasses, hair, or clothing styles before selecting a desired appearance. They may also wish to view the appearance using a variety of facial expressions and poses to further help in their evaluation, for example by changing their facial expression, hair style, and pose whilst viewing their face in a mirror. This immediate visual feedback is important in enabling the consumer to form opinions regarding their appearance with different glasses styles. The physical wearing of many different spectacle looks is a time consuming and potentially costly process and, in addition, each spectacle frame may need to be adjusted to assess not only aesthetics, but also fit, which further increases time and effort. Despite these negative factors, the value of physically trying on glasses adds a great deal to a consumers purchasing decision, and means it is still the most popular method of purchasing spectacles.

[0005] Methods of synthesising, or simulating, the appearance of glasses on a subject's face have the potential to greatly reduce the time and cost required to visualise different spectacle appearances. This potential can only truly be realised if the synthesised appearance gives a true and realistic representation of the physical product when applied to the face of the subject and is achievable in a natural, accessible and immediate way. The potential of any spectacle simulation system is reduced if the consumer faces technical or usability barriers, such as a lengthy and difficult process, the need to visit a fixed location, or a complex or inadequate interface.

[0006] An ideal system of glasses simulation must have no difficult or time consuming setup, must simulate accurate and realistic glasses appearance, and should be deployed on widely available consumer hardware, such as smartphones. Additionally, it should also allow the subject to view the synthesised appearance in a variety of poses and expressions naturally, in real time, and with little effort. To maximise the accessibility of such a system to consumers, the system should also be capable of performing in a typical consumer environment, such as in the home or outdoors, and must therefore be able to operate with very few constraints on lighting conditions, image resolution or the subject's position in the image. We will refer to this scenario hereafter as an unconstrained wearer simulation environment.

[0007] Previous methods that have been developed for simulating glasses looks, often called Virtual Try On systems, or VTO, have often been limited to single image systems. The dependence on a single static image severely limits the subject's ability to visualise the appearance of the physical glasses appearance in a natural way by removing their ability to experiment with expression and pose. Furthermore, the systems do not alter, in real time, dynamic differences occurring in shadow detail, lighting induced highlights, and reflections, further removing the ability to achieve a true 'in mirror' view. For example, US20150235428A1 Systems and methods for generating a 3d model of a user for a virtual try-on product

[001 22] Further,_there are several prior art systems provided on internet web sites that allow the modification of single images of a subject for the purpose of overlying a spectacle frame image to produce a virtual try on image^

[0008] These prior art systems have been fairly widely deployed but have had limited impact on consumer behaviour due to their simplistic nature and unrealistic resultant image. As previously stated, these systems are fundamentally limited to a single image of the subject, and so fail to provide a natural equivalent to the physical process of trying on glasses such as wearing glasses and viewing in a mirror. The systems are also somewhat limited in the images that they are able to process, requiring good lighting and high-resolution images as input which is a further barrier to wide scale adoption in the spectacle wearing consumer space.

[0009] Additionally, the overall system must be able to seamlessly obtain various measurements which are required for the spectacle ordering and manufacturing process.

[001 0] Many systems have been described, for example, US6508553B2 describes, as typical for such prior art methods, a method for obtaining the values required. The system requires a specific routine to be performed by the wearer of the glasses, requires an operator for the system, does not offer a real time live viewing experience, and measures crudely the dimensions from determining the pupil centre as the measurement reference point. Gross limitations exist with such methods, particularly where the pupil centre may not be coincident with the eye's visual axis due to various reasons, such as anatomical variation or previous iris injury causing an irregular shape and or located pupil.

[001 1 ] Additionally, the overall system must be able to seamlessly place the spectacle lens and spectacle frame manufacturing order with the chosen or appointed spectacle lens producing laboratory for manufacture of the chosen spectacles, and take payment for such, with little or no effort from the user apart from actually making or authorising the payment. To date, no prior art systems have shown this capability.

[001 2] Ordering of the eyeglasses from optometrist services, in particular prescription eyeglasses, typically requires the physical presence of the wearer of the eyeglasses (also referred as the glasses user or simply the user) at the premises of the optometrist services to conduct initially an eye test to be undertaken by a qualified optometrist for issuance of the prescription for manufacturing of the lenses to be mounted onto a frame.

[001 3] Further, after the eye test has been conducted, the user needs to select - unless the user already has done so while waiting for the eye test to be conducted - one or more frames onto which the lenses based on the prescription resulting from the eye test will be mounted. Typically, selection of the frame(s) of the eyeglasses may take a relative long time due to delays in the user making up her/his mind and/or on occasions a relative large quantity of costumers may be present at the optometrist service premises which cannot be served all at once by the salespersons of the optometrist service. However, solutions have been developed to be able to select the frames to which the lenses (manufactured based on the prescriptions have issued by an optometrist) will be fitted. Some of these solutions are online systems (also referred to as Virtual Try On system - VTO) that permit viewing a multitude of frames and selecting one or more for purchase thereof. Some VTO systems are adapted to permit superposing onto a static image of the potential purchase of the frame(s).

[0014] Furthermore, once the user has selected the frame(s), a further test needs to be conducted for fitting the eyeglasses to the particular face of the user. Fitting of the eyeglasses to the face of the user comprises measuring specific parameters for adjusting the eyeglasses to ensure that the eyeglasses properly fit onto the user's face so that the user's eyesight is accurately corrected when using the eyeglasses. Measuring these specific parameters typically needs to be done by qualified personal located at the premises of the optometrist. Thus, the user will have to further wait for the measurement to be undertaken by the qualified personnel. [001 5] The specific parameters are Monocular Pupillary Distance, Segments Heights for Bifocals, Multifocals and Single Vision Lenses, Back Vertex Distance and Pantascopic Tilt.

[001 6] Currently, measurement of these specific parameters for adjustment of the frame(s) requires the user to physically have the selected frame on the her/his face. Further, in the vast majority of current systems for measuring the specific parameters , the user - while carrying the frame - need to apply a scaling piece of equipment over the top of their glasses to allow a photo of the user to be taken for measuring of the specific parameters. The measurements of the specific parameters may be conducted using the photograph of the user taken while the user is physically wearing the frame to be adjusted and using a computer for perusing the photograph of the user's face to determine the particular values of the specific parameters used for adjusting the eyeglasses.

[001 7] With the advent of the internet, online systems for offering for sale and ordering frames of eyeglasses have been made available to the public; some of these online systems even include the option of providing systems for permitting the users of virtually putting these glasses onto their faces to confirm whether a particular frame suits the user's face.

[001 8] However, these systems for ordering online frames of eyeglasses do not necessarily include the option of measuring the specific parameters required for adjusting eyeglasses to the user's face. Thus, after the user has selected the frame; to ensure that the lenses will be properly issued, the user still needs to travel to the premises of an optometrist service for measuring of the specific parameters.

[001 9] Below are exposed several of the reasons that currently, in order to ensure that the frames properly fitted, it is best to conduct measurement of the specific parameters premises of an optometrist service.

[0020] Whilst there are many Virtual Try On (VTO) systems described in prior art, they are all static image based, i.e. they rely on an image that is captured, the spectacle frame is overlaid on the face image, and that constructed image of face with frame is viewed by the user, often with the image able to be rotated and so forth in order to view the image at various angles. Whilst these VTO systems are acceptable for viewing the frame, they have the limitation of being unable to present the user with an as-worn view in real time, i.e. as if the user were trying on the glasses and viewing themselves in a mirror. Hence, they are largely unrealistic in view, and do not take into account shadowing, appearance in different lighting, and so forth.

[0021 ] Currently, whilst the spectacle frame selection may be possible with the prior art VTO systems, the user must ultimately either 1 ) visit an optical store for measurements to be taken, where the optical store staff usually complete the required steps for glasses manufacture, 2) use some other separate measurement system to obtain the parameters required and then enter such detail into yet another system so as the glasses may be manufactured, 3) guess, or estimate the measurements, 4) rely on the spectacle seller of manufacturer to guess or estimate the measurements.

[0022] Typically, a person trained in the art of optical dispensing is required to assist in the manufacture of spectacles. This person, the Optical Dispenser, or Dispenser, as they are commonly known, is required to properly adjust the spectacle frame to suit the wearer's head, usually by adjusting the nose pad positions and angles (with a metal frame or rimless frame), by adjusting the temple angle(s) to achieve a straight or horizontal fitting with appropriate Pantascopic tilt of the frame, and to achieve a generally comfortable fitting for the wearer.

[0023] After having satisfactory achieved the basic frame adjustments, the dispenser is required to obtain critical measurements whilst the frame is in place on the wearer's head. These measurements allow for alignment of the visual axis of the eye with the optical axis of the spectacle lens, alignment of such axis allowing for the eye's rotational axis in such lenses as aspheric lenses, the fitting cross location for occupational and progressive addition lenses, and segment location with bifocal or trifocal lenses.

[0024] These measurements are often done via various methods. One common method is to physically place a 'dot' with a marking pen or similar on the demonstration lens of the spectacle frame directly on front of and in line with the center of the eye's pupil, and to then measure using a ruler or such the dimensions required from that dot. Errors common with this method arise from poor wearer's head posture creating incorrect alignment with the dot x and or y axis, incorrect dot location due to error from the dispenser arising from parallax error, the dot being placed in an estimated central pupil position being not the actual central pupil position, and the dot being arbitrarily placed in the pupil center location as opposed to the first Purkinje image (which gives a true location of the visual axis of the eye) off the cornea. Further errors arise from the limited ability of the dispenser to accurately measure the parameters from the dot, in part due to estimating the center of the dot itself as the measurement starting point, estimating the deepest part of the inner reverse bevel of the spectacle frame, movement of the frame angles due to flexure in flexible type spectacle frames, parallax error, and often, a general 'fudge' of the numbers if the dispenser feels they do not quite look right.

[0025] This method requires a highly experienced and skilled optical dispenser, and trainees often make errors, resulting in a less than ideal outcome being suffered.

[001 23] Another common method of obtaining the dispensing measurements is to use some photographic based dispensing system, whereby a reference apparatus is clipped or attached by some method to the front of the spectacle frame to be measured, the wearer shall then put in place on their head the spectacle frame and reference frame combination, be instructed to look ahead at some target whilst remaining very still, and a photographic image of the person's head is captured, most often in two planes, one from directly in front, and one from the side (90 degrees to the first image). The system then displays the image with some lines overlaid which can be moved by using a computer mouse. The dispenser then moves the lines to align with various desired locations about the spectacle frame, and also places a cross or such in front of the wearer's eye, usually at the first Purkinje image. The computerised system then calculates the desired parameters from the location of the lines and crosses. Errors arise commonly from this method via way of poor wearer's head posture creating incorrect alignment, hence all measurements that follow will be incorrect. Additionally, the dispenser must often guess as to where the inner reverse bevel of the frame is actually located, and sometimes the Purkinje image appears to be in an odd location and the dispenser may simply move the pupil cross to a more standard appearing location, or the Purkinje image may not be visible. Reflections off the demonstration lens may also cause unwanted photographic artifacts that hinder the estimating of measurement locations.

[0026] Additionally, since the wearer views some target at a near distance, and that distance is unknown, the system itself must make some estimations as to what the distance PDs would be, given that the system is in fact capturing images of the near PDs. This is often based upon some arbitrary formulae.

[0027] Additionally, if the spectacle frame has not been acceptably adjusted for the user, or the addition of the reference apparatus creates an unnatural frame seating position, then all following measurements will be incorrect.

[0028] Additionally, errors from the optical dispenser estimating some parameters, which must still be done manually by the optical dispenser, lead to errors, i.e. Estimating the back vertex distance and Pantascopic tilt as worn.

[0029] This method requires specialised equipment and a highly experienced and skilled optical dispenser, and trainees often make errors, resulting in a less than ideal outcome being suffered.

[0030] Yet another common method used when the user visits an optical store for measurements to be obtained, relies on measurements based off obtaining Purkinje images to be used as the visual axis reference point, with the optical dispenser having to align in some way their measurement device with the Purkinje image produced. This is usually done with a pupilometer when simple monocular or binocular pupillary distance (PD) is required. The wearer shall look into the pupilometer whilst the optical dispenser views into the pupilometer from the opposite end and aligns a movable line with the Purkinje image produced by a light inside the pupilometer.

[0031 ] This method, whilst relatively simple for even a trainee to employ, does not give any measurements other than PD, hence is not useful for the vast array of spectacle lenses dispensed today, for example, multifocal or aspheric lenses. [0032] Yet another method is for the optical dispenser to simply estimate all of the parameters from aligning a PD rule, which is in essence a ruler, by aligning the zero mark with the estimated pupil centre and reading off the scale at various locations about the frame. On occasion, the optical dispenser may obtain the PD using the PD rule by measuring the distance from the temporal iris boundary (limbus) on one eye, the nasal limbus of the other eye, obtaining a distance PD, and sometimes the dispenser will merely divide this distance by two to obtain monocular distance PDs. This is the most basic and most crude method, almost always leading to errors and less than ideal outcomes. It is, however, still widely used, particularly where the expense of computerised equipment is prohibitive to the business.

[0033] Further, optical dispensers often estimate this distance (the Back Vertex distance) which is usually grossly inaccurate. There are measurement instruments available in the form of small inside calipers, which physically measure from the spectacle lens to the closed eyelid. Errors arise with these devices due to eyelid thickness estimations and frame movement. Generally, these instruments are not widely used.

[0034] Photographic based systems can measure the distance by having the operator move a line or some such marker to the corneal surface, along with another line or marker to the lens surface, then calculating the distance.

[0035] In theory, these methods may, with considerable care, produce reliable results.

[0036] However, they all measure to the posterior surface of a demonstration lens that is fitted to the spectacle frame for display purposes. When an actual prescription lens is fitted, the BVD may be altered, as the back surface curvature of the lens is a major component in the lens design, and changes with almost every prescription. Some lenses, for instance, may have very steep, or highly concave rear surface topology, whereas other lenses may be almost completely flat. Differences in back surface radii may therefore alter the fitted BVD by several millimeters.

[0037] Furthermore, with traditional optical dispensing, or optical dispensing using prior art VTO systems, once the dispensing measurements are obtained, they must be entered or transmitted in some way to an optical laboratory for lens manufacture. The optical dispenser must ensure that the send or transmitted information is accurate, populated in the correct fields or electronic systems or written in the correct area for writing based systems, correct parameters for the lens type chosen, etc. Hence, a human checking system is paramount, as errors will result in an unacceptable outcome for the wearer. Errors during this process are commonplace, particularly with less experienced or trainee optical dispensers, and with more sophisticated lens designs that may be poorly understood by the optical dispenser.

[0038] It is against this background that the present invention has been developed.

SUMMARY OF INVENTION

[0039] In accordance with various embodiments of the present invention, a system and methods are provided that may be used for the selection of glasses from a library of frames; then apply a selected frame to the user's facial image, simulating the realistic glasses appearance on each frame of a video sequence in real-time, in an unconstrained wearer simulation environment and on a commercially available consumer device. (VDS)

[0040] It will be provided that the system will, with little effort from the user, obtain critical optical dispensing measurements necessary for the ordering and manufacturing of the completed spectacle unit, in an unconstrained wearer simulation environment and on a commercially available consumer device. (DMM)

[0041 ] It will be provided that the system will, with little effort from the user, be capable of placing the spectacle manufacture order on behalf of the user, and take payment for such, in an unconstrained wearer simulation environment and on a commercially available consumer device. (ECI)

[0042] According to a first aspect of the invention there is provided a virtual-try-on method for a user to virtually try on a frame of spectacles, the method comprising the steps of:

[0043] displaying in a graphical interface a real-time video of the user; [0044] displaying in the graphical interface at least one spectacle frame for selection by the user.

[0045] providing a real-time video of a 3D representation of the selected spectacle frame;

[0046] combining the real-time video of the user with the real-time video of a 3D representation of the selected spectacle frame to create a combined real-time video, the combined real-time video depicting the user wearing the 3D representation of the selected spectacle frame; and

[0047] displaying in the graphical interface the combined real-time video depicting the user wearing the selected frame.

[0048] Preferably, the method further comprises the step of deriving image capture environment parameters and user visual characteristics from the real time image of the face of the user.

[0049] Preferably, the method further comprises the step extracting a correspondence between the chosen spectacle frame and the face of the user in the real-time image;

[0050] Preferably, the method further comprises processing the combined real-time video for adapting the selected spectacle frame to the image capture environment parameters and the visual and anatomical characteristics of the user.

[0051 ] Preferably, the method further comprises applying the adapted spectacle frame to an image of the face of the user based on the correspondence, thereby generating a simulated look real-time video image of at least the user's head wearing the selected spectacle frame.

[0052] Preferably, wherein generating the simulated look real-time video image comprises a simulated look image on each frame of a video sequence in real time to generate the simulated look real-time video image. [0053] Preferably, wherein the correspondence between the selected spectacle frame and the face of the user may be defined on a basis of extracting a plurality of locations corresponding to an adaptive set of feature locations.

[0054] Preferably, wherein the correspondence is defined on the basis of extracting a plurality of facial data of the user.

[0055] Preferably, wherein the simulated look real-time video image is calibrated in the context of the image capture environment parameters so as to generate an overall appearance matching the image capture environment parameters.

[0056] Preferably, the method further comprises a step of compensating colors of the glasses and facial appearance, based on facial features of the subject, and/or based on lighting in the image capture environment.

[0057] Preferably, wherein the simulated image simulated look real-time video image comprises a plurality of independent layers corresponding to the physical parameters of the environment and surrounding the user.

[0058] Preferably, wherein the least one of the plurality of independent layers may be color shifted to simulate different colors of some reflections.

[0059] Preferably, the method further comprises the step of receiving a glasses-wearing template created by the user.

[0060] Preferably, the method further comprises the step of displaying in a graphical interface an eyeglass ordering interface to permit the user to select frames from a multitude of frames.

[0061 ] Preferably, the method further comprises the step of displaying a first screen area for displaying an image of the user's face onto which the digital representation of the frame will be superimposed.

[0062] Preferably, the method further comprises the step of generating a second screen area for displaying a multitude of images of frames to be selected by the user. [0063] Preferably, wherein the virtually trying-on by the user is conducted by the user on a mobile phone.

[0064] According to a second aspect of the invention there is provided a method for measuring at least one parameter used for adjusting eyeglasses to a user of the eyeglasses, the method comprising the steps of:

[0065] generating a simulated look real-time video image of at least the user's head wearing the selected spectacle frame as defined in the first aspect of the invention; and

[0066] processing the digital information for measuring at least one parameter used for adjusting the selected spectacle frame to the user.

[0067] Preferably, the method further comprises the step of obtaining the reference scale using a reference object of standard size for location in a capture plane for recordal of the image and processing.

[0068] Preferably, the method further comprises guiding the user for location of the reference object to the capture plane.

[0069] Preferably, the method further comprises the step of virtually placing the reference object in a desired measurement plane.

[0070] Preferably, the method further comprises displaying a brighter box within a darker background for guidance of the user.

[0071 ] Preferably, wherein a facial recognition process is conducted for guidance of the user.

[0072] Preferably, the method further comprises the step of accounting for variations in the distance between the user and means for generating the real-time video of the user. [0073] Preferably, the method further comprises the step of adjusting the size of the 3D representation of the selected spectacle frame if variations in the distance between the user and means for generating the real-time video of the user occur.

[0074] Preferably, wherein the parameter comprises near pupillary distance or distant pupillary distance.

[0075] Preferably, wherein the near pupillary distance comprises near monocular distance o near binocular pupillary distance.

[0076] Preferably, wherein the distant pupillary comprises distant monocular distance o distant binocular pupillary distance.

[0077] Preferably, wherein the parameter comprises segment heights.

[0078] Preferably, wherein the parameter comprises the Back Vertex Distance.

[0079] Preferably, wherein the Back Vertex Distance is calculated taking into account the lens form to be fitted to the selected spectacle frames.

[0080] Preferably, wherein the Back Vertex Distance is calculated using the facial data generated via VDS, along with known, scaled, correctly fitted Glasses data from VDS, and DMM predictions of lens back surface values.

[0081 ] Preferably, wherein the parameter comprises the Pantascopic tilt.

[0082] Preferably, the method further comprises the step of determine whether the value of the angle of the Pantascopic tilt is suitable for the particular frame selected by the user.

[0083] Preferably, the method further comprises the step of suggesting to the user another frame for selection thereof.

[0084] Preferably, wherein the measurement of parameters is conducted for single vision lenses, bifocal lenses or multifocal lenses. [0085] Preferably, the method further comprises the step measuring the location of the 1 st Purkinje images.

[0086] Preferably, the method further comprises the emission of light for eliciting a light reflection off the anterior surface of the cornea of both eyes.

[0087] Preferably, the method further comprises storing the location of the location of the 1 st Purkinje images for future reference.

[0088] Preferably, wherein the pupillary distance and the segment heights are measured either from 1 st Purkinje images or from an area substantially coinciding with the center of the pupils of the user's eyes.

[0089] Preferably, wherein the virtually trying-on by the user is conducted by the user on a mobile phone.

[0090] According to a third aspect of the invention there is provided a virtual-try-on system for users to virtually try-on spectacles on one first remote device of the user, the method comprising

[0091 ] a first data storage for storing data associated with users and suppliers of spectacles frames;

[0092] a second data storage for storing images of spectacles frames to be selected by the users;

[0093] a third data storage for storing images of the users;

[0094] at least one graphical display for displaying to at least one user a plurality of screen areas for displaying at least real-time videos of a user and of the spectacle frames;

[0095] a computer processor for executing programs instructions and for retrieving data stored in any of the first to third data storage; [0096] input means for permitting communication between the user and the computer processor to permit the user to provide instructions to the computer processor; and

[0097] computing means in communication with the computer processor, for storing program instructions for execution by the computer processor to conduct the method as defined in the first aspect of the invention.

[0098] Preferably, the one first remote device of the user comprises a mobile phone.

[0099] According to a fourth aspect of the invention there is provided a system for measuring at least one parameter used for adjusting eyeglasses to users of the eyeglasses by virtually trying on spectacles on one first remote device of the user, the system comprising the steps of:

[001 00] a first data storage for storing data associated with users and suppliers of spectacles frames;

[001 01 ] a second data storage for storing images of spectacles frames to be selected by the users;

[001 02] a third data storage for storing images of the users;

[001 03] at least one graphical display for displaying to at least one user a plurality of screen areas for displaying at least real-time videos of a user and of the spectacle frames;

[001 04] a computer processor for executing programs instructions and for retrieving data stored in any of the first to third data storage;

[001 05] input means for permitting communication between the user and the computer processor to permit the user to provide instructions to the computer processor ; and

[001 06] computing means in communication with the computer processor, for storing program instructions for execution by the computer processor to conduct the method as defined in the second aspect of the invention. [001 07] Preferably, the system further comprises fourth data storage for storing data representative of the location of the 1 st Purkinje images

[001 08] Preferably, the one first remote device of the user comprises a mobile phone.

[001 09] According to a fifth aspect of the invention there is provided an online system for a user to order eyeglasses from a remote location, the system comprising:

[001 10] a measuring system for measuring at least one parameter for adjusting eyeglasses to the user of the eyeglasses, the measuring system being the system as defined in any one of the fourth aspect of the invention;

[001 1 1 ] at least one first remote device of the user located at the remote location and adapted to be connected to the measuring system; the first remote device comprising the at least one graphical display of the measuring system and being adapted for the user to interact with the online system through an eyeglass ordering interface for providing one or more user's images, selecting one or more frames and filing out an order form for ordering the eyeglasses to be adjusted based on the one or more parameters measured by the measuring system; and

[001 12] at least one second remote device of an optometrist service adapted to be connected to at least to the first remote device; the second remote device being adapted to process the order form for delivery of the eyeglasses ordered by the user; wherein

[001 13] the online system further comprises computing means to display in the at least one graphical display of the user's remote an eyeglass ordering interface to permit the user to select frames from a multitude of frames and order the eyeglasses manufactured based on the at least one measured parameter and comprising the selected frame. [001 14] According to a sixth aspect of the invention there is provided an online system for a user to order eyeglasses from a remote location, the system comprising:

[001 15] a virtual-try-on system for a user to virtually try on a spectacles frame as defined in the third aspect of the invention;

[001 16] at least one first remote device of the user located at the remote location and adapted to be connected to the measuring system; the first remote device comprising the at least one graphical display of the measuring system and being adapted for the user to interact with the online system through an eyeglass ordering interface for providing one or more user's images, selecting one or more frames and filing out an order form for ordering the eyeglasses to be adjusted based on the one or more parameters measured by the measuring system; and

[001 17] at least one second remote device of an optical dispensing service adapted to be connected to at least to the first remote device; the second remote device being adapted to process the order form for delivery of the eyeglasses ordered by the user; wherein

[001 18] the online system further comprises computing means to display in the at least one graphical display of the first remote device of the user an eyeglass ordering interface to permit the user to select frames from a multitude of frames and order the eyeglasses manufactured based on the at least one measured parameter and comprising the selected frame.

[001 19] Preferably, the computing means are adapted to permit the user to order the eyeglasses directly to the optical dispensing services.

[001 20] Preferably, the ordering the eyeglasses is done via a single click for forwarding the order to the optical dispensing services.

[001 21 ] Preferably, the one first remote device of the user comprises a mobile phone. [001 22] Preferably, the system further comprises data storage means for storing the data of the user for future reference. Preferably, the online system is adapted to refer the user to an optical provider for undergoing a traditional in-office eye exam.

[001 23] Preferably, the online system is adapted to access the booking system of the optical provider for booking purposes.

BRIEF DESCRIPTION OF THE DRAWINGS

[001 24] Further features of the present invention are more fully described in the following description of several non-limiting embodiments thereof. This description is included solely for the purposes of exemplifying the present invention. It should not be understood as a restriction on the broad summary, disclosure or description of the invention as set out above. The description will be made with reference to the accompanying drawings in which:

Figure 1 illustrates a particular arrangement of a system in accordance with an embodiment of the present invention for ordering eyeglasses online by a user of eyeglasses;

Figure 2 illustrates a block diagram of the method in accordance with an embodiment of the present invention for ordering eyeglasses using the system shown in figure 1 ;

Figure 3 shows a graphical user interface of a computing device of the user of eyeglasses for ordering eyeglasses online using the system shown in figure 1 ;

Figure 4 shows the graphical user interface shown in figure 3 during measurement of a first group of specific parameters used for fitting the frames ordered using the system shown in figure 1 ;

Figure 5 shows the graphical user interface shown in figure 3 during measurement of a second group of the specific parameters used for fitting the frames ordered using the system shown in figure 1 ; Figure 6 schematically depicts the basic components of the adaptive, calibrated glasses simulation device in accordance with an embodiment of the invention;

Figure 7 shows a flowchart showing the calibration phase and simulation phase of glasses simulation in accordance with embodiments of the present invention;

Figure 8 shows a flowchart of the Create Look 206 process shown in figure 7;

Figure 9 shows a flowchart of the Simulate Look 21 1 process shown in figure 7;

Figure 10 is a detail of a front view of the user showing the 1 st Purkinge Image in the users eyes;

Figure 1 1 shows a representation of the ocular globe showing the visual axis and the optical axis;

Figure 12 shows a shows a representation of both eyes of a user showing the Right Monocular Pupillary Distance;

Figure 13 shows a representation of both eyes of a user showing the Fitting Height;

Figure 14 shows is the corneas the user showing the Back Vertex Distance (BVD);

Figure 15 is a schematic view of a frame and an ocular globe for showing the Pantascopic Tilt;

Figure 1 6 is an enlarged view of the area A shown in figure 1 5;

Figure 1 7 shows a representation of both eyes of a user using a frame showing illustrating the bifocal segment height; and Figure 1 8 shows a representation of both eyes of a user using a frame showing illustrating the monocular pupillary distance.

DESCRIPTION OF EMBODIMENT(S)

[001 24] The present invention comprises a system 1 0 for enabling individuals (users) to order eyeglasses online. In a particular arrangement, the system 10 displaces a multitude of images of different frames 14 (see figure 3) so that users may view the images of frames 14 and select one or more frames 14 from the multitude of frames 14 (see figure 3). The images may be photographs or pictorial representations of the frames 14 that may be uploaded from databases 25 of frame suppliers operatively connected to the system 10 via, for example, a remote device 22.

[001 25] Further, the system 10 permits remotely measuring the specific parameters required for adjusting the eyeglasses to be prepared, for example, based on the eyeglasses prescription issued to the particular user after having had an eye test. Thus, measurement of the specific parameters do not need to be conducted while the user is physically wearing the frames 12 selected by her/him.

[001 26] The fact that the system 10 allows remotely measuring these specific parameters results in that the user does not need to travel to the premises of an optometrist service for measuring these specific parameters. Thus, the user may select particular frames 14 prior to attending an appointment at a particular optometrist service for conducting an eye test for issuance of the prescription. Alternatively, the user may, after having undergone the eye test, return to her/his home and remotely finalise the process for ordering the prescription eyeglasses by selecting the frames 12 at her/his home; thus, the user neither needs to spent time selecting the frames at the premises of the optometrist services premises nor wait to be served by the staff of the optometrist service for conducting measurement of the specific parameters needed to adjust the eyeglasses to the user's face.

[001 27] Furthermore, particular embodiments of the invention relate to a system and methods for automated synthesis of the appearance of spectacles and sunglasses as worn by users in a video sequence in real time, and more particularly, to synthesising the appearance of spectacles in an unconstrained consumer environment where this synthesis occurs entirely on widely available consumer hardware.

[001 28] Also particular embodiments of the invention relate to a system and methods for automated optical dispensing parameter measurement, more particularly, off 1 st Purkinje image location. The user may remain unaware during the process. And particular embodiments of the invention relates to a system of automated optical appliance ordering and sales. Furthermore, additional modules may be added to enable Rx retrieval from cloud based medical records storage, or via traditional request methods using email.

Moreover, aspects of various embodiments of the invention include, without limitation:

[001 29] An automatic calibration process by which the simulated glasses are matched to the user's attributes (i.e. facial features and anatomical facial features) and camera parameters of the subject and capture environment to provide a more realistic experience even in unconstrained environments.

[001 30] Spectacle simulation based on robust facial data and/or adaptive facial features rather than pre-defined facial features, which achieves robustness to unconstrained wearer simulation environments.

[001 31 ] A realistic and efficient method of matching the simulated glasses to the lighting conditions in the captured images.

[001 32] An efficient method of simulation of multiple pairs of glasses using a library database of rendered frames, enabling the invention to be customisable and compactly deployed.

[001 33] An efficient method for obtaining highly accurate optical dispensing measurements using 1 st Purkinje image based methodology.

[001 34] An efficient method for optical appliance order placement direct with optical manufacturing laboratories. [001 35] An efficient method for collecting payment for ordered spectacles via various methods.

[001 36] Referring now to figure 1 .

[001 37] Figure 1 shows a particular arrangement of a system 10 in accordance with an embodiment of the invention. The system 10 comprises an Eyeglass Ordering Platform 1 8 that is adapted to interact with the users, the optometric services as well as, for example, with third parties such as the suppliers of the frames 14 to be ordered by the user using the system 10.

[001 38] As is shown in figure 1 , interaction between the user, the optometrist service and the frame suppliers and the Eyeglass Ordering Platform 18 occurs via remote devices 1 6, 20 and 22.

[001 39] In a particular arrangement, the Eyeglass Ordering Platform 1 8 comprises serves 24 for including software for running the Eyeglass Ordering Platform 18 and one or more computer processors, such as one or more commercially available Central Processing Units (CPUs) in the form of one-chip microprocessors, coupled to a communication device configured to communicate via a communication network. The communication device may be used to communicate, for example, with one or more of the remote devices 1 6, 20 and 22.

[00140] The Eyeglass Ordering Platform 18 further comprises an input devices (for example, a mouse and/or keyboard to enter persona details among other) and an output device (for example, a computer monitor to display reports and/or aggregated results to an administrator and permit the administrator to manage and control the Eyeglass Ordering Platform 18).

[00141 ] Further, the system 10 includes computing means having data sharing software program installed and/or directly integrated within the system 1 0. The computing means comprise electronic devices that include the laptops 16, 20 and 22 26 (or PC and similar computing and communication devices such as mobile phones) and servers 24 and 25. The computing means each include at least one CPU and operating system, RAM, ROM storage and TCP/IP adapter card. Computing means are each coupled to input and output (I/O) devices through an I/O interface. The I/O interface includes any system for exchanging information to or from an external device. I/O devices include any known type of external device such as a display device (e.g., monitor), keyboard, mouse devices, printer, speakers, handheld device, facsimile, microphone, or webcam.

[00142] The Eyeglass Ordering Platform 18 further includes database farms incorporated in the serves 24. The database farms store information required for the Eyeglass Ordering Platform 1 8 to generate the application for ordering of the eyeglasses and for submission of the application to, for example, the optometrist service.

[00143] The information stored in the database farms may include, for example, account information associated with the users of the system 1 0 and account information associated with suppliers and the optometric services. According to some embodiments, the database farms may include a report database including summary reports that have been generated for the users of the system, the administrator of the system, and/or the suppliers or optometrist services. Note that the databases described herein are examples, and additional and/or different information may be stored therein. Moreover, various databases might be split or combined in accordance with any of the embodiments in accordance with the present invention.

[00144] The computer means of the Eyeglass Ordering Platform 1 8 performs instructions of the programs, and thereby operate in accordance with any of the embodiments of the present invention to be described below. For example, the computer means may generate a face image 50 the user as well as the digital representations 46 of the frames 12 for superimposition on the face image 50 of the user. Superposition of the digital representations 46 of the frames 1 2 on the face image permits (1 ) the user to view on a display how the selected frames 12 appear on their face to decide whether a particular frame 1 2 would be appropriate for selection by the user and (2) measurement the specific parameters (also referred to as dispensing parameters) for adjusting eyeglasses to be ordered by the user. [00145] The system permitting a user to view on a display how the selected frames 1 2 appear on their face to decide whether a particular frame 12 would be appropriate for selection by the user is referred to as the Virtual Dispense System (VDS).

[00146] The system measurement the specific parameters (also referred to as dispensing parameters) for adjusting eyeglasses to be ordered by the user is referred to as the Dispense Measurement Module (DMM).

[00147] The programs of the Eyeglass Ordering Platform 18 may be stored in a compressed, uncompiled and/or encrypted format. The programs may furthermore include other program elements, such as an operating system, a database management system, and/or device drivers used by the computer processor to interface with any of the peripheral devices such printers 28, and the remote devices 16, 20 and 22, among others.

[00148] As used herein, information may be "received" by or "transmitted" to, for example: (i) Eyeglass Ordering Platform 1 8 from another device; or (ii) a software application or module within the Eyeglass Ordering Platform 18 from another software application, module, or any other source.

[00149] As mentioned earlier, the system 1 0 may interact with users, optometrist services and suppliers; interaction occurs through the computing means such as remote user device 1 6, 20, 22 (such as PC, laptops or mobile phones) associated and/or communicated with the computing means of Eyeglass Ordering Platform 18 included in the serves 24. Interaction includes transmissions of signals, representative of particular information, between the computing means for delivering data (such as images of the user's face and representations of the frames 14 to be supplied by the supplier as well as personal details of the users, the optometrist services and the suppliers), requesting that particular actions and decisions should be taken by the users, optometrist and suppliers and signals representative of the actions and decisions taken, among others.

[001 50] As used herein, the remote devices 1 6, 20, 22, 26, and computing means included in servers 24 described herein may exchange information between each other via any communication network which may be one or more of a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a proprietary network, a Public Switched Telephone Network (PSTN), a Wireless Application Protocol (WAP) network, a Bluetooth network, a wireless LAN network, and/or an Internet Protocol (IP) network such as the Internet, an intranet, or an extranet. Note that any device described herein may communicate via one or more such communication networks.

[001 51 ] The remote user devices 16, 20, 22 and 26 might comprise, for example, personal computers (PCs) as shown in figure 1 , wireless phones, tablets and/or laptop computers, among others. These devices 16, 20, 22 and 26 comprise digital graphical interfaces 30 as well as input devices such as touch screens, mouse devices or keyboards, and cameras and audio and video systems to interact with the Eyeglass Ordering Platform 18.

[001 52] For example, the remote user device 16 is adapted to transmit data representative of the information pertinent to the personal information of the user, and images of the face of the user such as front view and side view of the user to the Eyeglass Ordering Platform 18 permitting the interaction between the Eyeglass Ordering Platform 18 and the user for ordering the eyeglass. Communication between the Eyeglass Ordering Platform 18 and the remote user device 16 may via a web browser 44 and/or a plug-in for a web browser to be described below with reference to figure 3.

[001 53] Moreover, the system 1 0 comprises printer means 28 for printing any relevant information, images or reports.

[001 54] As mentioned before, the Eyeglass Ordering Platform 18 may interact with the users for ordering the eyeglasses, selecting the frame(s) 14, and measuring the specific parameters for adjusting the eyeglasses to the face of the particular user; this interaction between the Eyeglass Ordering Platform 18 and the user may be done through the remote user device 1 6; this may be done through a web browser 44 in the particular case where the remote user device 16 comprises a PC, laptop or tablet. In alternative arrangements interaction between the Eyeglass Ordering Platform 18 and the user may occur via a portable communication device such as a mobile phones through a software application (i.e. an app).

[001 55] Figure 3 illustrates a particular arrangement of the web browser 44 displayed in a graphical interface of the remote user device 26 when associated and/or connected to the Eyeglass Ordering Platform 1 8. The web browser 44 allows the Eyeglass Ordering Platform 18 to interact with the user through an eyeglass ordering interface 32 displayed on the graphical interface 30 of the remote user device 16. The eyeglass ordering interface 32 comprises a screen area 48 for displaying an image of a user's face (the face image 50 of the user shown in figure 4) onto which the digital representation 46 of the frame 12 will be superimposed. The face image 50 of the user is obtained by a user's image to be processed by the processing means of the Eyeglass Ordering Platform 18 so as to generate the face image 50 of the user and display the face image 50 in the screen area 48 as is shown in figure 4 and 5.

[001 56] In the arrangement shown in the figures, the image of the user may be captured by a camera 52 for processing by the processing means of the Eyeglass Ordering Platform 18 to generate the face image 50 - the camera may operatively connected to the processing means of the user's remote device 16 and located, for example, at a centre upper location of the frame surrounding the graphical display 30 of the remote user device 1 6. In alternative arrangements, the user may upload a photo of her/him for processing by the processing means of the Eyeglass Ordering Platform 18 to generate the face image 50. At this stage, the processing means of the Eyeglass Ordering Platform 18 may measure the specific measurements and issue the corresponding application form for ordering the eyeglasses for submission to the optometrist service which will process the order and forward the physical frames 12 to the user of the eyeglasses

[001 57] Further, as shown in figure 3, the eyeglass ordering interface 32 comprises a plurality of windows that permit the Eyeglass Ordering Platform 1 8 to interact with the user. Each window permits the Eyeglass Ordering Platform 18 (while interacting with the user) performing the steps of the method for ordering the prescription eyeglasses as outlined in figure 2. To generate the eyeglass application for ordering of the eyeglasses, the Eyeglass Ordering Platform 18 provides to the user each of the windows via the eyeglass ordering interface 32.

[001 58] In a particular arrangement, access to a particular window is obtained when the user clicks on the tab 34 of the particular window. Alternatively, the Eyeglass Ordering Platform 1 8 may automatically activate a particular window after the Eyeglass Ordering Platform 18 and user have completed their interaction in a previous window.

[001 59] Referring now to figure 2.

[001 60] Figure 2 shows a group of steps that are included in the method for ordering the eyeglasses using the system 10.

[001 61 ] The first step is used for accessing the system 10 for ordering of the prescription glasses. In accordance with the arrangement shown in the figures, access to the system 10 occurs accessing the particular website via the web browser 44. Once access has been gained to the website of the system 10, the user may go to the particular web page used for ordering the eye glasses and that is shown in figures 3 to figure 5.

[001 62] The first tab of this webpage is the tab for the user to input her/his personal details as well as any other information (such as for example payment details) required for finalising and submitting the application for ordering the eyeglasses.

[00163] The second tab allows the user to provide at least images of her or him; in particular, the user needs to provide a front image of her/his face and a side image of her/his face. The front image will be used for measuring a first group of specific measurements such as Monocular and binocular Pupillary Distance, Segments Heights for Bifocals, Multifocals and Single Vision Lenses; the side image will used for measuring the Pantascopic Tilt and Back Vertex Distance. [001 64] The front and side images may be captured via the camera 52 incorporated in user's remote device 16; alternatively, the user may upload one of her/his images for use by the system 10.

[001 65] After the user's images have been delivered to the Eyeglass Ordering Platform 18 as described before, the computing means of the Eyeglass Ordering Platform 18 proceeds to process the images by detecting the user's face images 50 from the images supplied by the user and post the face images 50 onto the screen area 48 as is shown in figure 4 and figure 5.

[001 66] The third tab relates to confirming whether the face images (front and side) have been properly posted on the screen area 46 and that they are proper representations of the user's face. The user may return to the previous image for capturing or uploading other images if generation of the previous face images 50 did not result in proper representations of the user's face.

[001 67] The fourth tab permits selection by the user of the particular frame as will described below. In particular arrangements of the present embodiment of the invention, this particular step includes a virtual try-on eyeglasses interface permitting users to mount the images of particular frames 14 on the users' face image 50 for the users to confirm whether the particular frame 14 actually suits her/his particular face.

[001 68] The images of the frames 14 to offered to the user may be graphically displayed at a menu 38 located at a particular location (such as the left hand side of the graphical interface 30) of the eyeglass ordering interface 32 by the processing means of the Eyeglass Ordering Platform 1 8. The menu 38 may be a rolling menu that permits visualising a multitude of images of frames 14 by pressing the up and down arrows 40 and 42 located at the upper and lower ends of the menu 38 - alternatively, drop down menus may be used.

[001 69] Selection of a particular frame 14 may occur by, for example, moving a cursor (using a mouse device) displayed on the graphical display over the particular row of the rolling menu 38 where the particular frame 14 is shown and pressing the left hand control of the mouse device to select the particular row. Selecting the particular row indicates to the processing means of the Eyeglass Ordering Platform 18 that the particular frame 14 (shown in that row) has been selected by the user and that a digital representation 46 of that particular frame 14 should be superimposed onto the image of the user's face for measuring the specific parameters and for, for example, conducting a virtual try-on for confirming whether the selected frame 12 suits the face of the user.

[001 70] In accordance with the present embodiment of the invention, the superimposition onto the user's face 50 of the digital representations 36 of the selected frame 12, such as a virtual image of the selected frame 12, defines an augmented reality as is shown in figure 4.

[001 71 ] As will be described with reference to the method of operation of the system 1 0, the processing means of the Eyeglass Ordering Platform 18 comprises means for generating the virtual image (the digital representations 36 of the selected frame 12) .

[001 72] The fifth tab relates to measuring the specific parameters used for ordering the prescription glasses. As mentioned before, a screen area 48 is provided for displaying face images 50 of the user and for superimposing representations 36 of the selected frames 1 2 for permitting the processing means peruse the augmented reality for measuring of the specific parameters used for adjusting the eyeglasses to the user's face.

[001 73] In particular, in a first instance (see figure 4) a front face image 50 is displayed in the screen area 48 to permit the processing means of the Eyeglass Ordering Platform 18 to process the front face image 50 by perusing the image 50 in order to measure the Monocular Pupillary Distance, Segments Heights for Bifocals, Multifocals and Single Vision Lenses parameters; in a second instance (see figure 5) a side face image 50 is displayed in the screen area 48 to permit the processing means of the Eyeglass Ordering Platform 18 to process the front face image 50 by perusing the image 50 in order to measure the Pantascopic Tilt and the Back Vertex Distance. [001 74] Once the processing means of the Eyeglass Ordering Platform 18 has finalised measurement of the specific parameters, the measurement are stored for use in preparing the application form.

[001 75] The sixth tab permits issuance of the corresponding application form for ordering the eyeglasses for submission to the optometrist service which will process the order and forward the physical frames 12 to the user of the eyeglasses. In this particular step, the user may have the option of adding additional requests to the application form such as colour of the frames among others. Also, if the user already has a valid prescription, the user may include the details of the valid prescription for ordering prescription glasses; this particular option (permitting submitting a valid prescription) is particularly useful because it permits ordering prescription glasses without having to travel to the premises of the optometrist service - the advantage of this option is that it allows replacing any lost or damaged prescription glasses without the user having to go to the optometrist service, provided the user has a valid eyeglasses prescription.

[001 76] The eight tab relates to submitting the filled out application form. In this step, the processing means of the Eyeglass Ordering Platform 18 fills out automatically the application form for submission with the optometrist services. Prior submission of the filled out application form the Eyeglass Ordering Platform 18 may interact with the user requesting any additional information such as acceptance of the terms and conditions of the eyeglass ordering service and confirmation of payment.

[001 77] After submission of the application with the optometrist services, the application is processed and the physical frame(s) 1 2 is/are delivered to the user as is shown in figure 1 .

[001 78] Referring now to figures 1 to 15; figures 1 to 1 5 relate to the Virtual Dispensing system (VDS) and the Dispense Measurement Module (DMM) in accordance with an embodiment of the invention.

[001 79] In accordance with an embodiment of the present invention, a method is provided for real time simulation of an application of at least one spectacle frame to the face of a user, where the user is disposed in an image capture environment subject to uncertain lighting conditions. The method has steps of:

[001 80] capturing a real time image of the face of the user within the image capture environment;

[00181 ] deriving image capture environment parameters and user visual characteristics from the real time image of the face of the user;

[00182] receiving a desired spectacle frame to try on;

[00183] extracting a correspondence between the chosen spectacle frame and the face of the user in the real-time image;

[00184] adapting the chosen spectacle frame to the image capture environment parameters and the user's visual and anatomical characteristics; and

[00185] applying the adapted spectacle frame to an image of the face of the user based on the correspondence, thereby generating a composite image of the face with the chosen spectacle frame in place.

[00186] In accordance with other embodiments of the invention, the correspondence between the chosen spectacle frame and the face of the user may be defined on a basis of extracting a plurality of locations corresponding to the adaptive set of feature locations. The correspondence may also be defined on the basis of extracting a plurality of facial data. The generated spectacle simulation appearance may be calibrated in a context of image capture environment parameters so as to generate an overall appearance matching the image capture environment parameters.

[00187] In accordance with further embodiments of the present invention, there may be a further step of compensating colors of the glasses and facial appearance, based on facial features of the subject, and/or based on lighting in the image capture environment. The adapted glasses appearance look may include a plurality of independent layers corresponding to physical parameters surrounding the subject and the environment. At least one of the plurality of independent layers may be color shifted to simulate different colors of some reflections.

[001 88] In accordance with yet further embodiments of the invention, the step of receiving the chosen glasses appearance may include receiving a glasses-wearing template created by the user. In any of the foregoing embodiments, there may, additionally, be a step of displaying the composite image.

[001 89] With reference now to the Virtual Dispensing system (VDS), the VDS comprises a virtual-try-on method for a user to virtually try on a frame of spectacles, the method comprising the steps of:

[001 90] displaying in a graphical interface a real-time video of the user;

[001 91 ] displaying in the graphical interface at least one spectacle frame for selection by the user.

[001 92] providing a real-time video of a 3D representation of the selected spectacle frame;

[001 93] combining the real-time video of the user with the real-time video of a 3D representation of the selected spectacle frame to create a combined real-time video, the combined real-time video depicting the user wearing the 3D representation of the selected spectacle frame; and

[001 94] displaying in the graphical interface the combined real-time video depicting the user wearing the selected frame.

[001 95] In the VDS one first step is the generation of a non-scaled augmented reality of user's head (referred to also as: "the combined real-time video depicting the user wearing the 3D representation of the selected spectacle frame"); activation of the VDS is done by activating the tab VDS that is displayed on the device screen as is shown in figure 4. The generation of the non-scaled augmented reality of user's head is first described with reference to figure 6. As shown in figure 6, a monocular video camera 100 is positioned to capture the face of a subject 1 04 during a facial performance, and a computational device 101 (contained in servers 24) contains a central processing unit and memory (data storage). The data storage is capable of storing all of the entities (such as the glasses templates 202 and Adaptive Expression Extractor 207) and data of the system (such as the facial data 300).

[001 96] In particular, image-capture environment parameters and user visual characteristics may be derived from the real-time image of the face of the user. Examples of image capture environment parameters include lighting spectrum and lighting direction and intensity. Examples of user visual characteristics include facial features and anatomical facial features.

[001 97] The memory also contains a program in the form of a series of instructions which, when executed by the central processing unit, and applied to the data and entities, simulates glasses on the images captured by the camera. The system 99 is configured so that the video of the subject is captured by camera 100 and streamed to computational device 101 for processing and generation of the simulated look image 21 2 in real-time.

[001 98] In one embodiment of the invention, the computational device displays the simulated look image stream on a physical display 1 02 for the Subject 104 to view. In another embodiment of the invention, the simulated look images are stored within the computational unit's memory for off-line analysis, by computational device or by another system.

[001 99] In another embodiment of the invention, the simulated look images are streamed to another device over a network for viewing by a third-party subject.

[00200] In another embodiment of the invention, the Input Device 1 03 is used by the subject to set parameters and select glasses, which modify the output of the glasses simulation, for example color and style parameters as described below. Input device maybe a touch screen, keyboard, pointing device. Any other input device capable of capturing this information falls within the scope of the present invention. [00201 ] One embodiment of the invention provides for the production of a continuous stream of images which simulate a specific glasses appearance giving the impression of a realistic physical application of spectacles onto the subject's face. The production of this stream is separated into two phases; a Calibration Phase 200 and a Simulation Phase 201 .

[00202] The Calibration Phase receives an input from the subject, who may also be referred to herein as a "user". The glasses appearance that is indicated by the user may be referred to herein as the "glasses look." The Calibration Phase creates the glasses look in such a way that it is tailored to the subject's anatomical facial features and the image capture environment, in which case the look may now be referred to as the "calibrated glasses look".

[00203] The Simulation Phase uses the calibrated glasses look to produce Simulated Look Images where the glasses look is mapped accurately onto the subjects face and lighting is convincingly matched to create the desired realistic output image stream. The Simulation Phase is able to produce Simulated Look Images at typical video frame rates directly on consumer mobile hardware.

[00204] Methods for generating Simulated Look Images from images of a subject captured by camera are now described with reference to the flowchart depicted in figure 7. The following is an overview of the system and each component will be described in detail below.

[00205] The Calibration Phase uses the Create Look module 206 to create a Look Profile 208 (referred to also as "non-scaled augmented reality of user's head" or "the combined real-time video depicting the user wearing the 3D representation of the selected spectacle frame"); non-scaled augmented reality of user's head (referred to also as: "the combined real-time video depicting the user wearing the 3D representation of the selected spectacle frame"); from a set of Glasses Templates 202, User Input 203 and derived information about the subject's facial anatomy and various camera characteristics. Facial features and camera parameters are derived with reference to facial data produced by an Adaptive Expression Extractor 207 and an Image Frame 400 obtained from the camera. The facial data 301 contain information on the subject's 3D pose and expression in the image which can be used to accurately identify regions of the image over which to calculate the facial features and camera parameters. The resulting Look Profile combines one or more generic Glasses Templates and modifies the result to match the subject's facial features and image capture environment.

[00206] The Simulation Phase simulates a look on each image captured from the camera at the desired frame-rate, typically between 30-60fps. The simulated look is based on the Look Profile and is constructed with reference to facial data. The facial data and the captured image are passed to the Simulate Look 21 1 module which maps the glasses look accurately and precisely onto the expression of the subject in the image and matches lighting conditions convincingly to give the impression of a realistic physical application of Glasses on to the subject's face.

[00207] Facial Data

[00208] The invention is designed to operate in an unconstrained wearer simulation environment. Achieving reliable facial data in such a scenario is a challenging task. Facial tracking and expression measurement approaches are available in the prior-art, for example, Active Appearance Models (AAMs) as taught in Cootes, et al., "Active appearance models" , Proc. European Conf. on Computer Vision, vol. 2, pp. 484-98, (Springer, 1998) , however the authors are not aware of a system capable of performing with the accuracy and precision required without some form of constraint on image capture such as controlled lighting, constrained head pose or constrained subject expression.

[00209] The Compute facial data 205 module (shown in figure 7) implements a system using the facial data produced containing information about the neutral 3D shape of a subject, their expression encoded as a set of activation values for each of a set of basic facial features and expressions which, when combined together, accurately and precisely reconstruct the subject's appearance on each frame. The facial data also contain information on the position and orientation of the user in 3D space and a camera with parameters through which to perform a projective mapping between 3D space and image coordinates. [00210] The facial data also enables the retrieval of a set of feature locations in the image frame corresponding to physical locations on the subject's face. These locations are adaptive and may differ from frame-to-frame.

[0021 1 ] In particular, the feature locations in successive frames or images may adapt to the specific appearance and image capture conditions of each frame of a video sequence. The feature adaptation system may be referred to herein as an adaptive tracker. Adaptive tracking may be achieved by selecting an adaptive set of feature locations during calibration, and, during video tracking potentially refining that set further based on the characteristics of each frame.

[00212] As used herein, the term "adaptive set" shall mean a set of elements (inclusive of values or parameters), the elements of which may be varied in time without the set losing its characterisation as the originally identified set. When elements of the adaptive set of features have been modified, the adaptive set of features may be referred to herein as an "adapted set of features," or, equivalents, as an "adapted feature set."

[00213] Create Look

[00214] Methods for creating a Look Profile by combining multiple Glasses Templates are now described with reference to the flowchart depicted in figure 8.

[00215] The invention contains a number of Glasses Templates each of which represent a different frame model that has been scanned and rendered accurately, and hold information about the frame, such as but not limited to wrap angle, bevel position and depth, nose pad area and angles of fit, eye size and bridge size.

[00216] As the glasses look is produced, the glasses cover a specific area of the face and the glasses template data is used combined with facial data to produce accurate location points for the simulated look image.

[00217] In one embodiment of the invention, examples of Glasses Templates include, but are not limited to, plastic spectacle frames, metallic spectacle frames, rimless spectacle frames, and sunglasses. [00218] Each Glasses Template specifies the coverage of the Glasses component with reference to a set of facial feature points. In one embodiment of the invention the coverage is specified as a four-channel image; 3 color intensity (Red, Green and Blue) and one transparency (Alpha) value for each pixel.

[00219] In another embodiment of the invention, the Glasses Template is specified as a single base color and a two channel image; intensity and transparency for each pixel. Any other suitable representation of such a template is included within the scope of the invention. Using the facial feature points, Glasses Templates can be mapped into a consistent image space, and from that space, onto the image of a subject as described below. This consistent image space may be referred to herein as a "facial reference space."

[00220] Glasses Templates are defined with respect to a specific spectacle frame features, but need to look realistic when applied to subjects of arbitrary facial features and varying facial features. To enable this, the reference facial features parameter is also stored as part of each Glasses Template.

[00221 ] Compute Look Geometry

[00222] A critical component of the invention is the modification of the Glasses Templates to match the facial features and image capture environment of the subject in the Input Image. This requires measurements to be taken from the subject's face in the image, and the simulated Glasses to be rendered precisely into the image. The Compute Look Geometry 301 module is used to generate a mapping between Glasses Templates, Look Profiles and the subject's face in the Input Image. This mapping is such that the Glasses overlays the target facial features in the image frame precisely and accurately. This mapping is implemented using the facial reference space as a common space that both Glasses Templates and the Input Image can be transformed into.

[00223] In one embodiment of the invention, mappings are generated from the Input Image to the facial reference space with respect to 2D image coordinates. The facial data contain the 2D image coordinates of a set of adaptive facial features each of which is associated with a specific facial location. Although the set of features measured will typically vary from frame to frame, their facial locations are defined in the facial reference space. Using standard image warping techniques, such as piece-wise bilinear or thin-plate spline transforms, the Input Image can be mapped to the facial reference space.

[00224] Precision and realism are critical components of the invention and, in general, the adaptive features are too sparse to define this mapping to the required level of detail.

[00225] For this reason, the set of adaptive features is enhanced using knowledge of anatomical facial structure data before defining the mapping. In one example, extra features which refine the mapping of smooth facial boundaries, such as lip and jaw edges, can be added to the adaptive feature set by fitting spline curves to the members of the original sparse set belonging to these features. Another example is to use knowledge of the circularity of the iris and its occlusion by the eyelids to refine the mapping in the eye region by imposing these constraints. Other prior knowledge about the geometry of the face can be represented in this way and is included within the scope of the invention.

[00226] In another embodiment of the invention, the mappings are defined with reference to the facial data using a 3D geometric representation of the subject. The facial data contain information about the shape of the subject's head, its position and orientation in 3D space and measurements of the subject's expression in the Input Image. It is widely known in the prior art how to produce a 3D geometry with shape variation parameters compatible with the facial data. When applying the facial data to the 3D geometry in the standard way, the vertices of the geometry project precisely onto the subject in the image providing a mapping between the Input Image and the facial reference space. In the prior art terminology for 3D methods, the equivalent to the facial reference space is referred to as a UV space, for example as described in "UV mapping", http://en.wikipedia.org/wiki/UV_mapping, which associates a 2D image with 3D geometry. The Glasses Templates are mapped to the facial reference space in the same way as above.

[00227] Measure Calibration Parameters [00228] The Measure Calibration Parameters module 302 measures various properties from the image such as lighting, camera parameters and the facial features of the subject. These parameters are used to match the Look Profile to the subject's appearance in the image to ensure a realistic final Simulated Look Image. Measuring image parameters during calibration and using them to alter the Simulated Look greatly enhances the realism of the simulated Glasses in the Simulated

[00229] Look Images.

[00230] Since the capture conditions are expected to remain relatively constant throughout a session, calibration is only required on certain frames, for example; the first frame; when a different subject appears before the camera; or when lighting conditions change significantly.

[00231 ] In an unconstrained wearer simulation environment very little is known in advance about the camera parameters or lighting conditions.

[00232] The facial data contain estimates of camera parameters such as focal length, position and orientation, from which projective transforms can be calculated, but image formation parameters such as white-balance are not available.

[00233] White balance can exhibit large variation across consumer cameras and correctly accounting for white balance changes is crucial to obtaining a realistic simulation of a specific Glasses look. The facial features of the subject also has a significant effect on the appearance of a Glasses product when physically applied and so must also be taken into account. These and other parameters cannot be measured directly in a consumer capture environment and so must be estimated using the Input Image and prior knowledge.

[00234] In one embodiment of this invention camera white balance is estimated using prior knowledge of facial features groups. The observed skin-tone, RGB, contaminated by the camera's white balance, is measured in the Input Image as the average value for each color channel in the subject's facial region. The closest uncontaminated facial features distribution prior is selected as the target facial features. The camera white balance RGBW^b is calculated as the offset between RGB and the selected distribution's mean RGB value. This value is applied to correct the Look Profile to match the subject's camera as described below.

[00235] The effect of applying the Look Profile to the subject's facial features is achieved in HSV (Hue, Saturation, Value) color space as a scaling of the saturation and value channels. The observed and target RGB values are converted into a HSV representation and saturation and value scaling parameters are calculated as the ratio between their per-channel mean values as follows:

[00236] s^s = s7 s^l

[00237] s^v = ν^'Ί

[00238] where S', S¹ are the saturation channels from the image and the reference respectively, V', V are the value channels from the image and the reference respectively. Both the saturation scale, s^s, and value scale, v^s, parameters are applied to match the Look Profile to the subject's facial features as described below.

[00239] Select Glasses Templates.

[00240] The user is given control over the Glasses simulation system by being allowed to set various parameters through Input Device. These parameters control the set of glasses chosen, i.e. the Glasses templates that will be applied to the simulated look, allowing the subject to construct different glasses styles to view and compare. In the Select Glasses Templates module 303, the User Input is used to select a subset of Glasses Templates combined to create the Look Profile.

[00241 ] Apply Calibration Parameters

[00242] The parameters from the Apply Calibration Parameters 306 module are applied to the Look Profile to match the Glasses to the subject's facial features and capture conditions. To take account of the subject's facial features, each pixel in the Look Profile is modified in the HSV color space using the following equations:

[00243] S' = s^sS [00244] V = s^vV

[00245] where s^s and s^v are defined above and S and V are the saturation and value channels of the HSV representation respectively.

[00246] White balance adjustment of the Look Profile to match that of the subject's camera, is achieved by adding RGBW^b, computed during the calibration phase, to the Look Profile pixels.

[00247] Simulate Look

[00248] The Simulate Look method for generating a Simulated Look Image from an Image Frame, facial data and a Look Profile is now described with reference to the flowchart depicted in figure 9. The Simulate Look module uses the Compute Look Geometry module, described above, to calculate a mapping between the image and the Look Profile using either 2D or 3D techniques. This mapping is used by the Simulate Lighting module 402 to calculate the lighting component and apply them to the Look Profile to produce a realistic Glasses simulation that matches the physical image capture conditions. Finally, the Simulated Look Image is produced by the Render Look module which combines the Look Profile with the Input Image using the Image to Glasses mapping. Each of these sub-methods are described in detail below.

[00249] Simulate Lighting

[00250] The Simulate Lighting 402 sub-method of Simulate Look 21 1 applies pixel- wise intensity adjustment to the Look Profile such that the lighting levels in the resulting Look Profile match the corresponding lighting levels in the Image Frame.

[00251 ] In one embodiment of the invention, the mapping between the Look Profile and the image is used to sample the Red, Green and Blue image intensities in the Image Frame corresponding to each pixel in the Look Profile. The sampled RGB values are converted to a single intensity value for each pixel. Using these measured image intensity values, each sample of the Look Profile is altered as follows: [00252] If |ⁱ < l

[00253] di = (Ι'— i)max(RGB^d)/l [00254] Else

[00255] di = (Γ— I) (max(1— RGB^d))/(1— I) [00256] RGB¹ = RGB^d + wdl

[00257] where RGB' is the lighting modified Look Profile pixel consisting of a red green and blue value.

[00258] RGB^d is the input Look Profile pixel consisting of a red green and blue value.

[00259] I' is the measured intensity from the Image Frame at the location corresponding to each Look Profile pixel

[00260] I is the intensity of the RGB mean facial features value computed in the Measure Calibration Parameters module

[00261 ] w is a parameter which controls the overall strength of the lighting effect.

Finally, RGB¹ is truncated to lie in the range [0,1 ].

[00262] The effect of these lighting equations is to shift the RGB values of the Look Profile towards white or black as the lighting in the image varies from that measured in the calibration frame without changing the perceived color of each pixel of the Look Profile. Variations in lighting across the subject's face, such as illumination gradients and cast shadows, are accurately captured and a realistic final image is obtained with lighting that closely matches the image capture environment.

[00263] The Simulate Lighting method is designed to produce the desired effect whilst remaining amenable for calculation in real-time at high-resolution. The calculation can also be efficiently implemented on Graphical Processing Unit (GPU) devices as each pixel is independent from its neighbours. [00264] Render Look

[00265] The final step is undertaken by the Render Look module 403, it creates a Simulated Look Image by rendering the Look Profile on the Image Frame. The rendering process can be achieved using either 2D or 3D rendering utilising the Look Geometry mapping to warp the Look Profile precisely and accurately onto the subject's face in the Image Frame.

[00266] One embodiment of the invention uses Look Geometry computed in 2D, and the rendering process involves overlaying the Look Profile on the face using the Look Profile's alpha channel and Alpha Combination, described above.

[00267] Another embodiment of the invention is where the Look Geometry is computed in 3D, such that the rendering process involves the projection of the 3D geometry onto the image plane and rendering of the Look Profile respecting the alpha channel and other aspects of the 3D geometry including occlusion. Both methods are standard rendering techniques well-known in prior art and, as any other rendering method, is encompassed within the scope of this invention.

[00268] Another embodiment of the invention is the ability for the user to view in real time the rendered images produced, whilst wearing their existing corrective or non-corrective eyewear. For example, the user wears their glasses to view the 99 Device, however, the VDS algorithms detect and remove their glasses, reconstruct the virtual space where the glasses were located. It is now as if the user is not wearing glasses and they are free to enjoy the real time viewing of Simulated Look Images with virtual glasses overalayed as described above. The VDS system tracks in real time and continues to remove the user's own eyewear from the constructed real time video feed images.

[00269] Referring now to the Dispense Measurement Module (DMM), this module relates to the process for measuring measurement the specific parameters (also referred to as dispensing parameters) for adjusting eyeglasses to be ordered by the user. In particular, the module executes method for measuring at least one parameter used for adjusting eyeglasses to a user of the eyeglasses, the method comprising the steps of: [00270] generating a simulated look real-time video image of at least the user's head wearing the selected spectacle frame as described earlier in relation to the VDS; and

[00271 ] processing the digital information for measuring at least one parameter used for adjusting the selected spectacle frame to the user.

[00272] The DMM is activated when, from the user interface of the VDS, the user presses, or selects, the DMM tab that is displayed on the device screen as shown in figure 5.

[00273] As described earlier, the attainment of dispensing measurements has means that the user must endure some process or procedure, usually in an in-office type location, for example a physical optical store location, and have someone experienced in the art complete the measurement process. This is most often inconvenient to the consumer who wishes to purchase online. Often is the case, this inconvenient procedure also excludes the consumers choice of online purchase of spectacles.

[00274] The consumer may use a VDS or VTO system to 'try on' glasses virtually, and online, for the purpose of intended purchase online. However, since the required dispensing measurements are not available, the transaction for completed prescription eyewear cannot proceed beyond that initial try on phase.

[00275] If the consumer is to obtain the required measurements from a physical store location, then that store location must have in stock, the chosen glasses frame from which to obtain the measurements when worn on the face of the consumer. Since there are many thousands of frames available, with most stores stocking varying and different frame ranges, and sizes, it may be very difficult to find the desired frame in a convenient store location.

[00276] Furthermore, even if the frame is found in store, most optical stores and their staff may be quite reluctant to spend the required time and effort in fitting such frame and obtaining all the required measurements, then providing that data to the consumer, enabling the consumer to purchase online with one of their perceived online competitors. In other words, in this scenario, the physical store does all the work for no possible reward, and so may refuse the consumer request.

[00277] In another scenario, the store may charge a fee to obtain and supply the required measurement data, hence adding to the cost for the consumer. Additionally, the consumer must then, without being trained in the art, enter or correctly supply the obtained data into an optical provider's dispensing system, where a slight error, or a measurement not converted to the correct format for that particular system, will result in an undesirable finished product outcome.

[00278] In another scenario, if the store were to have the frame available, and charge a fee to supply the dispensing data, the store may offer to price match the online offer price in order to obtain the sale from that consumer, hence the online supplier loses the customer, even though that customer had chosen their glasses with their VTO system, which initiated the sale.

[00279] Furthermore, as previously noted, the precise measurements required for some modern advanced spectacle lens designs prefer a level of accuracy that, until now, could only be obtained with large, fixed, specialised equipment operated by experienced users.

[00280] The current situation, as we see, presents problems for a) the consumer, b) the online seller, and c) the physical stores within the optical industry.

[00281 ] The purpose of the DMM is to allow for automatic capture of optical dispensing parameters and their accurate measurement during the VDS phase of the customer interaction. It eliminates the need for physical measurements by someone trained in the art, and eliminates the consumer inconvenience of obtaining that data. It further eliminates the need for physical stores to deal with the problem when such consumers present requesting only such data be provided. It further eliminates the online optical supplier from 'losing' that consumer transaction. Hence, there are many positives to implementation of a true, highly capable DMM within the VDS system. [00282] In accordance with an embodiment of the present invention, a system and method is provided to obtain all optical dispensing measurements, regardless of spectacle lens type, and regardless of the spectacle frame chosen. The method has steps of:

[00283] Obtain reference scale to be applied during VDS

[00284] Produce 1 st Purkinje images off anterior corneal surface

[00285] Primary Measurements: Find 1 st Purkinje images and complete measurement of Near Monocular Pupillary Distance, Distance Monocular Pupillary Distance, Pupil Fitting Height, Bifocal Fitting Height, Trifocal Fitting Height, Near Pupil fitting height for Single Vision Near glasses to be used for reading where the optical axis will benefit from inferior relocation

[00286] Measure, and or calculate Back Vertex Distance (BVD)

[00287] Measure, and or calculate Pantascopic Tilt

[00288] Correctly supply and or populate the obtained measurements into an optical provider's dispensing system, with correctly converted values if required, so as to ensure the outcome of finest quality of visual performance

[00289] The steps above shall be described in detail below.

[00290] It should be understood that whilst all measurements involve the counting of pixels, these pixel counts are converted to millimeters for the final output measurement; with the exception of Pantascopic tilt, where conversion to millimeters is not required as this in an angular value.

[00291 ] Obtain reference scale to be applied during VDS

[00292] To accurately measure a parameter in the virtual, or augmented reality space, we must first have at least one reference of known size in order to scale against that known reference size. [00293] The scaling is performed in a fashion by using a ratio against the known size, and has been described in prior art, for example US6535223B1 P5 (15-67), P6 (1 -44). For our invention, we advise use of a credit card, or store loyalty type plastic card, as they may be turned to show the rear side (so as not to enable a potential security breach by viewing the front side), as these cards are universal in size and readily available.

[00294] The user of our VDS system may at anytime convenient to them, record the reference. The process to do so is to simply engage the reference capture module by selecting the 'true fit' icon on the display screen of the device used, as this icon is present continuously at a small corner location so as not to intrude on the viewing experience. Once selected, the user is instructed to hold the reference object, preferably a card as described (coins, rulers, etc may also be used), in the capture plane (i.e.. in the facial plane), and within a frame or two of the live video feed, the system records the image and processes against it's know size.

[00295] To place the reference object in a suitable location, the user is guided by a guidance system consisting of a brighter box within a darker background, at about the user's forehead region but not covering the user's eyes. This altered contrast display makes it easy for the user to understand where to place the reference object.

[00296] This 'primary guidance system' makes use of advanced facial recognition technology, to locate the correct region to be highlighted to the user.

[00297] Now the VDS system can scale the Glasses template, i.e. the glasses frame selected, to exact sizing in real time so as to give the most realistic look possible, in conjunction with the previously described VDS system. The scale ratio is applied during the final stage of 301 Compute Look Geometry in the previously described VDS module.

[00298] It should be appreciated that from this point, the system knows the dimensions exactly of the facial features and will automatically scale any subsequently selected Glasses templates that the user may choose. [00299] It should additionally be appreciated that the VDS system accounts for variance in subject distance from the camera, for example if the user brings the camera closer in order to enlarge their image on the display screen, or moves further away in order to obtain a more complete overall image appreciation, the VDS system will alter the sizing of the rendered glasses (also referred as to 3D representation of the selected spectacle frame)in real time so as to give a truly 'real' mirror image, and is termed herein 'gross planar compensation', and accounts for altered distance between the subject and the 99 System.

[00300] Hence, the reference scale may be obtained very early in the VDS experience, or obtained at any point thereafter.

[00301 ] Furthermore, once the reference object has been captured, the user's anatomical facial features (obtained from module 205 Compute Facial Data) are analyzed by the DMM to determine if, and if so by how much, compensation should be applied in order to virtually place the reference object from the plane of the user's forehead into the plane of the user's anterior eye corneal surface, and is known herein as 'fine planar compensation'. As described above, 'gross planar compensation' is available for the VDS module in sizing the rendered Simulated Look Image.

[00302] All of this takes place seamlessly and without the user being aware.

[00303] It should be appreciated that whilst the capture of the reference object, and the scaling ratios are similar to that described in prior art , the capability of planar compensation, i.e., to compensate the scaled sizing as required for true accuracy of representation at grossly different image planes, "gross planar compensation" (altered distance between the subject and the 99 System), or at slightly different image planes, "fine planar compensation" (slightly further or closer subject distance from where the reference object was captured), has not been described previously.

[00304] The capability of planar compensation is particularly useful because being able to adjust the scaling ratio to be accurate at the plane of the anterior eye is not known. No system uses "fine planar compensation" by where the reference object is effectively virtually placed in the desired measurement plane. (For example, someone may have a very prominent forehead against which they place the reference object, however, they also have very deep set eyes, and the front of their eyes may be, lets say, 20mm behind the forehead plane. Since the persons head has been reconstructed virtually, we can know this distance and use it to ensure measurements taken at the eyes' plane is accurate, even though the reference for this calculation was in a plane 20mm from the eyes' plane.)

[00305] Produce Purkinje images

[00306] Once the reference is captured, and the DMM recognises that a Simulated Look Image is being displayed to the user, a 'secondary guidance system' is automatically employed consisting of a red irregular quadrilateral image, constructed with lines and located around the edge region of the display, that becomes a green regular rectangle when the device is held in a suitable position for measurement capture. The quadrilateral-to-rectangle method, controlled by a system internal to the DMM, utilises the internal functions of the device gyroscope and accelerometer, with the intended goal of directing the user to hold the 99 System in front of the face, vertical to the ground and perpendicular to the face, with eyes viewing in a straight ahead position (primary gaze position).

[00307] In another embodiment, the user may selectively engage or disengage the secondary guidance system, effectively engaging or disengaging the DMM measurement sector.

[00308] Once the system recognises that the device is in the desired position, the entire device display screen momentarily emits bright, white light. The timing is in the region of 40ms, enough time for 2 complete frames to be imaged at 60fps. (Longer timings may be utilised in devices that are running at lower than 60fps with live video feed.) Given the short time frame of around 40ms, and the relatively low light intensity (compared to flash photography for example),the user is not adversely affected in terms of light adaptation or pupil constriction. The bright light burst is sufficient to elicit a reflection of light off the anterior surface of the cornea of both eyes. It is this reflection that is termed the 1 st Purkinje image, and is known in optical physics to represent most accurately the visual axis of the eye - see figure 1 1 for a description of the visual axis and the optical axis of an ocular globe of a user.

[00309] In another embodiment, if the device is fitted with a front camera flashlight system, then that flashlight system may be utilised in order to produce the 1 st Purkinje images.

[00310] It should be appreciated that since the eye is not a centred optical system, it does not possess a true optical axis. Nevertheless, it is possible to establish an axis of observation, relative to the eye's visual axis, by using Purkinje images, and this shall be herein referred to as the Visual Axis. From figure 1 1 , it can be seen that if measurements are taken from the pupil centre, or the observed geometrical or Optical axis, we have the resultant angle Alpha in relation to the Visual Axis, and distance x representing the error in estimating the location of the Visual Axis. It is for this reason, that measurements should be take from the Visual Axis of the eye (as by utilising the 1 st Purkinje image), and preferably not from the pupil centre or other such arbitrary location as commonly described in prior art, for example US6535223B1 C3 (34-49).

[0031 1 ] It should be appreciated that the provision of Purkinje image formation and or capture for use in relation to an automated optical dispensing systems is new and has not been described previously.

[00312] It should be appreciated that once a Purkinje image capture event has been completed, the location of the Purkinje images for this user is known, and this known location may be used in any subsequent 'try on' session using the VTO, hence eliminating the need for Purkinje image capture to ever be repeated, i.e.. The known Purkinje image locations are used as the measurement starting point for any particular frame subsequent.

[00313] Primary Measurements

[00314] After having completed the sequence of scaling, and produced the Purkinje images, and since the live video feed was capturing continuously, the DMM module now, through iris and or pupil recognition, finds the bright spot within the dark pupil zone, i.e. the system locates and maps to the 1 st Purkinje image. Furthermore, since the wearer had imposed at that capture moment a correctly fitted and scaled Simulated Look Image containing the chosen Glasses, the DMM system now automatically counts pixels in various locations:

[00315] From the Right eye Purkinje image, horizontally, {or horizontal equivalent (if the user had their head tilted, for example)} to the centre of the frame's nose bridge.

[00316] This shall be known as the Right Monocular Pupillary Distance.

[00317] From the Left eye Purkinje image, horizontally, {or horizontal equivalent (if the user had their head tilted, for example)} to the centre of the frame's nose bridge.

[00318] This shall be known as the Left Monocular Pupillary Distance.

[00319] In the case where horizontal equivalent is deemed necessary by the DMM system, an imaginary line shall be constructed by the system intersecting the right and left Purkinje images, and a right-angled triangle constructed down from one Purkinje image to another line running horizontally across the display device from the other Purkinje image - see figure 1 2.

[00320] Now horizontal pixels and vertical pixels may be counted (the sides of the triangle), and using Pythagoras' theorem, the triangle's hypotenuse is calculated. Given that the scale is known, the DMM uses the overall distance between the 2 Purkinje images (also known as the overall Pupillary Distance), to ratio off and determine the distance in the oblique plane (the hypotenuse) to give the desired pupillary distance to the frames centerline, i.e. to the centre of the frame's nose bridge.

[00321 ] It should be noted that this calculation method obsoletes the requirement of fixed head position during measurement capture, and adds convenience to the user as the process of capture is not required to be repeated. [00322] Although the calculation is well known, it is provided here for reference. With points in a pixelated image, each point has a x and y coordinate, hence the number of pixels representing the hypotenuse (h) will be V((xi - Χ2)² + (y1 -y2)² ) wherein Χ1 -Χ2 is the length of the adjacent side of the triangle and yi-y2 is the length of the opposite side of the triangle.

[00323] Notes on Pupillary Distances

[00324] As may be appreciated from the above descriptions, when the Purkinje images are captured, the user will be looking at the display device at some close distance, typically around 30 or 40cm. Hence the Right and Left Pupillary distances described so far, are in fact, Near Pupillary Distances. Furthermore, for the sake of high accuracy, these Near Pupillary Distances (NPDs) may need to be adjusted for spectacle lens manufacture if NPDs are being used, as most manufacturing laboratories work off standard inset values pertaining to a working distance of 40cm. Hence, if the screen were being viewed at 50cm during image capture, the NPD value measured shall ideally be adjusted accordingly if the discrepancy is large enough.

[00325] The value adjustment may be calculated by the DMM if the screen to face distance is known, as may be the case with devices that employ technologies including but not limited to depth sensing cameras, proximity sensors, or triangulation using multiple video capture images in real time, or the like which can supply such distance information. If this distance is known, and the overall pupillary distance is known, then through mathematics of triangles and eyes centre of rotation information, the true NPD at 40 cm (or any other distance) may be calculated with accuracy.

[00326] Whilst such small differences are largely ignorable in most cases, although consumers with very high prescriptions, or very close viewing distance preference, for example, may benefit from such adjusted PD values, and can avoid unwanted and fatiguing induced base out prismatic effect.

[00327] For true Distance Pupillary Distances (DPDs), we need to account for the convergence of the eyes when viewing at near distance, and calculate back to what the DPDs would truly be if the user were attending to a distant target. As above, if the DMM can extract distance data representing the display to face measurement, then an easy calculation from known human ocular rotation parameters can be employed.

[00328] In another embodiment of the DMM capture sequence, the user shall be instructed via voice command, and or via text on the screen, to briefly look up just over the top of the device at something far away. Eye tracking technology in-built, and widely described in prior art, shall recognise when the user has shifted gaze upwards and over the screen, and the Purkinje images can now be captured whilst the eyes are truly in a DPD posture. Since the capture camera of smartphones are universally high up in the screen area, viewing over the top represents a small vertical misalignment which may be desirable to avoid for Fitting Heights measurement, and this small misalignment shall be compensated for by the DMM algorithms.

[00329] In another embodiment of the DMM capture sequence, which is capable of very fast capture via the live video feed, the DMM may make two, successive Purkinje sequences, and in doing so may take one measurement image for heights, then another measurement image with over the top viewing for DPDs.

[00330] In another embodiment of the DMM capture sequence, the DMM may monitor the live video feed and track the eye pupils in real time, and if the 99 device is deemed to be in a suitable position whilst simultaneously the eyes fixation point is also deemed suitable, then the DMM may automatically instigate a Purkinje image production and capture event.

[00331 ] In another embodiment of the DMM capture sequence, the DMM may monitor the live video feed and track the eye pupils in real time, and if the 99 device is deemed to be in a suitable position whilst simultaneously the eyes fixation point is also deemed suitable, and there is an ascertainable Purkinje image in each pupil by way of lighting from the unconstrained wearer simulation environment, then the DMM may automatically instigate a Purkinje image capture event. [00332] It should be appreciated that an automated method for obtaining near and or distance pupillary distance measurements using Purkinje images and a readily available consumer hardware device has not been described previously.

[00333] Fitting Heights

[00334] From the Left eye Purkinje image vertically down to the inner lower edge of the glasses. This is the Left fitting Height, as used in most multifocal lens designs.

[00335] The fitting heights are parameters measured and necessary for the manufacture of glasses containing but not limited to multifocal lenses, aspheric single vision lenses and occupational lenses. The parameter shall be measured:

[00336] From the Right eye Purkinje image vertically down to the inner lower edge of the glasses. This is the Right fitting Height, as used in most multifocal lens designs.

[00337] From the Left eye Purkinje image vertically down to the inner lower edge of the glasses. This is the Left fitting Height, as used in most multifocal lens designs.

[00338] In rimless and semi-rimless (nylon) construction, these Right and Left fitting Height measurements are sufficient. However, in plastic and metallic frames with full rims that completely cover the lens outer edge, there is a groove, or reverse bevel as it is sometimes known inside the innermost portion of the frame's rim, and the measurement must now extend down into that groove, as that is the true distance to which the lens will be manufactured. The groove depth is typically around 0.5mm. As the Glasses templates are produced for use in the VDS system, these groove depths too are measured, and so at this point the DMM system can simply adjust the final measurement to allow for the extra groove depth.

[00339] Where the users head was tilted during image capture, the method described previously using a constructed right angle triangle and Pythagoras' theorem shall be applied here also, with the hypotenuse representing the Fitting Height (H) - see figure 13 [00340] It should be appreciated that an automated method for obtaining lens fitting height measurements using Purkinje images and a readily available consumer hardware device has not been described previously.

[00341 ] Near Pupil fitting height for Single Vision Near glasses to be used for reading where the optical axis will benefit from inferior relocation

[00342] On occasion, it may be desirable to deliberately locate the optical axis of spectacle lenses much lower in the frame that usual. I.e. the fitting heights will be deliberately set low. Such a decision is normally only considered by someone of considerable optical dispensing experience and knowledge of physical optics.

[00343] The most usual case, is where the wearer has a significant difference in refractive error between the two eyes, and the optical power required for reading or close work is moderate to high. Often, the refractive difference between the eyes will be greater than 2 Dioptres, and the near prescription will be over 4 Dioptres in one eye. Viewing through traditionally set near vision lenses in this scenario may often lead to vertical diplopia, most particularly if the wearer has only recently fallen into such a refractive group, for example following cataract surgery to one eye. In such a case, when parameter values are breached from what is considered 'normal', the DMM may choose to set the fitting height low by up to 15mm, hence avoiding an unwanted visual outcome. The DMM module will first verify with the user, via a question, as to whether or not they have had recent eye surgery, and after calculating the likely induced prismatic imbalance, decide where to place the optical axis for the user.

[00344] It should be appreciated that this level of scrutiny for lens optical axis placement is unusual in every optical practice setting, as the level of detail and calculation can be exhausting. The DMM however, is capable of almost seamless customisation of the optical glasses product specific for the user's visual needs.

[00345] It should be appreciated that an automated method for determining sophisticated alteration of lens fitting height measurements using a readily available consumer hardware device has not been described previously. [00346] Back Vertex Distance (BVD)

[00347] An accurate Back Vertex Distance (BVD), as depicted in figure 14, is important whenever the optical prescription is of magnitude 4 Dioptres or above. In such cases, either a) the back vertex distance must be matched from prescription supplied BVD, or b) the prescription must be converted so as to be equivalent at the new as-worn BVD.

[00348] Often, in practice, this is difficult, as the finished product as-worn BVD is not known until the glasses are manufactured and the wearer places them on their face. Alternatively, it may be difficult to judge whether the frame itself has enough adjustment potential to be situated acceptably so as to achieve the desired BVD one completed.

[00349] The combination of VDS, which can know accurately the fitted position of the frame, and the DMM which can, once lens selection has occurred, compute the frame complete with as-finished lenses fitted.

[00350] If during the VDS the wearer turns their head beyond about 80 degrees to the side, and DMM is engaged (i.e. running in the background), then DMM will capture images for later use. One such use is to calculate the BVD.

[00351 ] The finished rendered frame and lens combination can be re-rendered onto the users face, either in real time if desired by the user (to assess aesthetics of lens options), or at a later time in the background processing of the DMM, where pixels are counted along the horizontal meridian from the apex of the eye cornea to the back surface of the virtualized lens. Once this measurement is obtained (BVD), the DMM can automatically alter the refractive prescription as necessary.

[00352] The particular method for measuring the BVD in accordance with the present embodiment is calculated taking into account the lens form (the shape of the lens. In this particular embodiment of the invention to obtain BVD, the facial data generated via VDS, along with known, scaled, correctly fitted Glasses data from VDS, and DMM predictions of lens back surface values can be combined and used to predict what the BVD will be once the finished product is manufactured. Once this predicted measurement is obtained (BVD), the DMM can automatically alter the refractive prescription as necessary.

[00353] It should be appreciated that an automated method for obtaining Back Vertex Distance measurements using a readily available consumer hardware device has not been described previously.

[00354] Pantascopic Tilt

[00355] The Pantascopic angle, or as more commonly known, the Pantascopic Tilt, (see figure 15) is a critical parameter for the ordering and production of the newest and most advanced spectacle lens designs available. Such advanced spectacle lens designs rely on many variables, one such variable involves the rotation of the eye about it's rotational axis, and the resulting incident angle of the eyes visual axis with respect to the lens surface that results, and varies, as the eye looks down, for example. The lens design computers need to know at what angle the lens shall be held in front of the eye, i.e., Pantascopic tilt angle, (and at what distance, i.e.. BVD), so the design may account for the lens-eye interaction.

[00356] Without doubt, these spectacle designs will become commonplace over time, hence any dispensing system that does not account for the acquirement of Pantascopic tilt will be destined for obsolescence.

[00357] Furthermore, systems not allowing for the capture of Pantascopic tilt, by default, deny their users the opportunity to enjoy the best enhanced vision available via the use of, for example, double sided freeform single vision and multifocal lenses.

[00358] There is no prior art that addresses Pantascopic Tilt.

[00359] If during the VDS the wearer turns their head beyond about 45 degrees to the side, and DMM is engaged (i.e. running in the background), then DMM will capture images for later use. One such use is to calculate the Pantascopic tilt. The calculation relies on counting pixels, and a complete image of the frame is not required. The constructed diagram against which the calculation is performed is shown in figures 15 and 16. Pixels are counted for the base (b) and height (h) of the constructed triangle, and the Pantascopic angle (PA) is given by the trigonometric formula

[00360] Tan(PA) = b / h

[00361 ] In another embodiment, because scaling is unimportant for this angular value, the image capture may be obtained at any stage in the VDS process, hence DMM engagement is not required for the capture, although DMM involvement is required to perform the calculation.

[00362] Once the Pantascopic Tilt has been calculated, if the tilt angle is deemed unsuitable as it falls outside a defined parameter range, the user shall be prompted to try a different frame, and or from the Glasses Templates, a similar design with greater angular tilt may be suggested to the user by the VDS. A lens selection, or change of lens selection will also trigger DMM re-evaluation of the Pantascopic tilt to ensure suitability.

[00363] It should be appreciated that an automated method for obtaining Pantascopic Tilt values using a readily available consumer hardware device has not been described previously.

[00364] Electronic Commerce Interface (ECI) Module

[00365] Traditionally, as mentioned above, an optical dispenser trained in the art is required to verify measurements and the optical prescription and manually enter, or manually check that they have been entered with correct protocol values, into an optical ordering system, so as that data may be sent to an optical laboratory where the spectacle lenses will be manufactured and prepared (cut to size and shape) for fitting into the chosen spectacle frame. Whilst this may sound straightforward, it should be noted that it is in fact a detailed process, and a fully automated system has not been previously described. [00366] The DMM supplies the ECI with all dispensing measurements required to place an order for completed optical products with optical manufacturing laboratories.

[00367] The ECI module interfaces with the DMM in a both-way communications protocol, whereby one module can extract or be supplied information from the other. For instance, The ECI may acquire the user's optical prescription and immediately make it available to the DMM for calculation of spectacle lens parameters in order to calculate BVD. Or in another scenario, the DMM may extract from the ECI the chosen spectacle lens type, including refractive index information, in order to supply the VDS lens information so the most realist view possible in the Simulated Look Image is possible.

[00368] In another example, the user may have chosen a preferred Glasses (spectacle frame) via VDS, and the system has acquired dispensing data through DMM, and that user has now just supplied optical prescription data into the system and chosen a specific type of lens.

[00369] The system now invites the user to 'try on' the completed pair of spectacles, with as-fitted spectacle lenses in place via VDS i.e.. The VDS has lens data, shapes etc supplied to it for rendering the image from DMM, and DMM has supplied to it the optical prescription data and lens type from ECI (and also frame data from VDS).

[00370] In the above example, the user could, if desired, decide to 'try on' another glasses style, and the VDS and DMM would work together to immediately render a predicted as-finished appearance of this new style in the Simulated Look Image, and in the background have DMM running and preparing to capture new Purkinje images for this new style, and hence be very quickly able to render a fully accurate representation with the Simulated Look Image, and be able to feed back to the ECI the calculated optical dispensing measurements for this new style.

[00371 ] Whenever such a scenario may be encountered, each glasses style and associated data is stored against the user's profile, and such profile may be temporary in the case of an unregistered user, or permanent and future retrievable in the case of a registered user.

[00372] Whilst the ECI has a secondary role of interfacing with DMM, it's primary role is interfacing with:

[00373] the user

[00374] optical providers

[00375] optical laboratories

[00376] payment systems

[00377] The role this interface operates with respect to the above shall be described below:

[00378] ECI Interfacing with the User

[00379] A vital component in any optical correction eyewear solution is the Optical Prescription, herein referred to as the Ήχ'. The Rx specifies what strength, or optical power, in Dioptres, the spectacle lens should provide in order to correct the vision of the intended wearer, or user, of the finished appliance.

[00380] The user typically obtains the Rx from an optical provider, for example, an Optometrist.

[00381 ] The Rx may be provided to the user in written form, electronically via email, electronically via email attachment, or made available via download from some cloud or other storage based system.

[00382] Also common is for the optical provider, for example, an optometrist, to provide the Rx to an optical dispenser, so that optical dispenser may order the spectacles, and the Rx might be provided by way of any of the aforementioned methods. [00383] It should be appreciated that once a Purkinje image capture event has been completed, the location of the Purkinje images for this user is known, and this known location may be used in any subsequent 'try on' session using the VTO, hence eliminating the need for Purkinje image capture to ever be repeated, i.e.. The known Purkinje image locations are used as the measurement starting point for any particular frame subsequent."

[00384] The ECI has the capability of requesting, and retrieving automatically the Rx from optical providers. A 'enter or get Rx' icon is displayed to the user on 99 Device during the ECI phase of gathering required user information. (For glasses ordering, or for completed glasses Simulated Look Image rendering, etc)

[00385] The 'enter or ger Rx' icon prompts the user to either:

[00386] Manually enter the Rx details if they have the Rx details and wish to do so

[00387] Authorise the ECI to retrieve the Rx from an optical provider

[00388] Be referred to an optical provider to undergo a traditional eye exam

[00389] Begin the Automated Eye Exam module (AEE)

[00390] Manually enter the Rx

[00391 ] If the user chooses to manually enter the Rx, then an Rx entry screen is provided to the user for this purpose and is relatively straightforward.

[00392] Alternatively, if the user has the Rx stored electronically on their smartphone, the ECI can search the phone, including photographs, in order to find and retrieve the Rx and automatically populate the Rx fields in the ECI customer record.

[00393] Alternatively, if the user has a written prescription, they may photograph that Rx and ECi will employ Optical Character Recognition to the photograph and retrieve the Rx and automatically populate the Rx fields in the ECI customer record. [00394] While the DMM accurately supplies the necessary optical dispensing parameters, it is possible for the customer to override them. For example, If a customer's pupillary distance (PD) is included in their optometrist provided Rx, the customer might choose to manually enter their PD into the Rx, and therefore overriding the value generated by the DMM. Where manually entered values are input, the DMM runs analysis and ensures the entered values 'make sense'. Where a predefined comparison range is breached, the value entered will not be accepted and the user alerted to either a) re-enter the value, or b) accept the DMM value.

[00395] Similarly, when a manually entered Rx is submitted, it shall be analysed by ECI to ensure it complies with a known set of mostly 'normal' expected values, and if the comparison parameters are breached, the user shall be alerted to very carefully check the inputted Rx values before proceeding.

[00396] If the manually entered Rx contains nonsensical entry(ies), the ECI shall refuse the entry and alert the user to either a) re-enter the value(s), b) authorise ECI to retrieve the Rx, c) book appointment for an eye exam, or d) begin AEE.

[00397] Lens Choice

[00398] Once the system is aware of the glasses style chosen, and the Rx, the system now must be aware of what type of lens the user desires (or requires), and any additional lens options that may be available.

[00399] Often, the Rx shall contain much of this information and be automatically populated by ECI during the 'get Rx' process.

[00400] However, often such information is not provided, and it is left to the user to decide which, if any, options are to be included. Such options include, but are not limited to, higher refractive index lens materials, specialised lens treatments and coatings, photochromic lenses, polarised sunglass lenses, etc.

[00401 ] The user shall be presented with a options 'menu' to choose from that lists available options and prices. [00402] Only available options for the lens type selected shall be shown. For example, if Bifocal Lens is selected, 1 .74 index lens option will not appear, as Bifocal lenses are not available in 1 .74 index materials.

[00403] Where the user is unaware of what an option is, or does, they may select an information icon using 99 device to be shown a short educational video explaining the features and benefits of that option, before being returned to the 'menu' screen.

[00404] Whenever a lens type, or refractive index type is altered in selection, the ECl feeds this new selection information to DMM which recalculates and in turn returns relevant data to ECl. Additionally, DMM feeds the new updated data to VDS in case the user requires a further 'try on', which shall be offered to the user, and so VDS shall be ready to create the updated Simulated Look Image. If a new 'try on' is partaken, the user shall be returned to the previous screen afterwards, in this case, the 'menu' screen.

[00405] ECl interfacing with optical provider

[00406] If the user chooses to authorise ECl to retrieve the Rx from a provider, a release of information request statement is authorised by the user, and the user enters via way of input device or voice, the know provider's Name and location and or any other unique identifiers that shall allow the ECl to find the provider.

[00407] The ECl connects to the Internet and searches for the provider. Having found the provider's contact information, ECl either completes that provider's request for Rx online form, completes a request for Rx on behalf of the user in a 'contact us' box, or more usually, will email that provider a completed request for Rx authorised by the user on behalf of the user. In all cases, the provider is requested to return the Rx via email to the ECl module, which will then automatically populate the Rx fields in the ECl customer record. Where the Rx is returned in written, or scanned format, the ECl shall use Optical Character Recognition to convert the written Rx into machine readable format and automatically populate the Rx fields in the ECl customer record. [00408] Where the Rx is made available via download from a storage device or platform, or electronic patient record database, the ECl shall automatically complete this process and proceed to automatically populate the Rx fields in the ECl customer record. Where such retrieval systems have security systems in place that require entry of email address for Rx to be sent to, the ECl shall input it's own email address in order to receive the Rx directly. Similarly, if a verification code is first sent by the retrieval system, the ECl shall supply it's own contact details in order to retrieve the Rx directly.

[00409] Be referred to an optical provider

[00410] Should the user prefer to be referred to an optical provider to undergo a traditional in-office eye exam, ECl will gather information from the user as to where and when they would like to attend for such eye exam. The ECl will the search the area(s) indicated by the user and give alternatives available. After selection of provider, ECl will use the provider's online booking system, and or offer the system to the user, to make the appointment.

[0041 1 ] Where made available by the provider's practice management system or the like, ECl will request that Rx data be made available directly to ECl so that the user's Rx shall be populated in ECl. Alternatively, ECl shall have the user's authorisation to retrieve Rx on the user's behalf, and will proceed to do so as described above.

[00412] ECl Interfacing with Optical Laboratories

[00413] The Rx spectacle lens typically consist of, and or is sometimes defined in the prescription by, some or all of:

[00414] Front surface curvature (sometimes referred to as Front Base Curve, or simply Base Curve)

[00415] Back surface curvature [00416] Thickness of lens between Front and Back curvatures, measured along the optical axis

[00417] Refractive Index

[00418] Lens type

[00419] With the exception of Lens Type, the components above combine to determine the optical power of the lens, and the shape of the lens.

[00420] It should be appreciated that the DMM module, after having been supplied by the ECl module the Rx, can calculate using known lens manufacturer information such as base curve, and assess suitability for the intended frame and Rx combination, then make changes to the lens parameters if desired to optimise the finished outcome. For example, if a glasses frame with a curved front (wrap around style) is chosen, and the typical Rx lens for the given Rx will have a relatively flat base curve (flat front), then DMM can determine which steeper base curve will be more suitable, calculate the required back curve and thickness to suit the new steeper front base curve, taking into account refractive index, and supply that information back to ECl as special manufacturing instructions to be sent to the lens laboratory for manufacture. Hence the lens will be custom ordered and manufactured to DMM specifications to best suit the chosen frame.

[00421 ] In its simplest function, ECl is able to send Rx and dispensing data directly to optical laboratories for the manufacture of the required spectacle lenses.

[00422] ECl interfacing with Payment systems

[00423] The ECl is fully integrated with an e-commerce and payment system.

When the user is satisfied with a specific pair of glasses, the product can be added to a virtual shopping bag. Every item in the customer's virtual shopping bag can be linked to a set of parameters generated by VDS/DMM/ECI based on a person and or product combination. [00424] Therefore, a single virtual shopping bag can include eyewear for different people, each with vastly different and unique parameters necessary for optimal optical dispensing.

[00425] This allows a family to place a single order for multiple people.

[00426] When the customer is satisfied with their virtual shopping bag, containing spectacle frame and or eyewear products, their respective DMM parameters (although not directly visible to the customer), Rx (or Rx input method selection) and lens customisation options, they are presented with the price and can decide to pay and place an order using a variety of payment methods, including but not limited to credit card, EFT, health insurance rebate schemes, paypal™ and other automated or electronic payment systems.

[00427] Sometimes, at this payment stage, users may ultimately decide that the final pricing is not suitable, and would like to quickly explore other alternatives.

[00428] For this reason, there are two icons present at this time on 99 Device, "alter frame", and "alter lens".

[00429] Selecting "alter lens" takes the user to the lens 'menu' screen, which operates as previously detailed under Lens Choice.

[00430] Selecting "alter frame" takes the user back to VDS Simulated Look Image and is ready for new or different frame selections, and has DMM engaged as described earlier. If the user selects a new frame DMM can recapture Purkinje Images then return the user to the ECI payment screen.

[00431 ] In another embodiment, selecting "alter frame" takes the user back to VDS Simulated Look Image and is ready for new or different frame selections, and has DMM engaged as described earlier. If the user selects a new frame DMM can calculate by comparing previous spectacle frame parameters against the newly selected spectacle frame parameters and accurately extrapolate the updated dimensional measurements. The user is returned to the ECI payment screen. [00432] After payment is executed, the order is recorded in ECI, and ECI automatically sends the optical dispensing details to an optical manufacturing laboratory as a production order. ECI then continues to interface with that laboratories electronic systems for the purposes of tracking and ETA and the like.

[00433] Customer Data Unit (CDU)

[00434] The ECI feeds certain data directly into the Customer Data Unit, herein referred to as CDI. CDI stores securely, specific customer data and information, including but not limited to:

[00435] Rx

[00436] ASP

[00437] SKU items purchased

[00438] Payment types used

[00439] Purchase dates

[00440] Purchase frequency

[00441 ] Differences in SKU categories purchased (i.e. change in fashion styles)

[00442] The CDU can be used to detect, identify and predict, changing and or emerging trends in fashion, age, purchase behaviour, etc; and is capable of generating real and predicted data sets that are useful for retailing optical products, specifically eyewear, and specifically in the online marketplace. It can also trigger and send reminder letters, special offers, discount codes and the like to targeted individuals or groups.

[00443] The present embodiments of the invention are particularly advantageous.

In fact, The Virtual Dispense System has several advantages over the more traditional methods and prior art VTO systems. [00444] Firstly, as the frame is fitted virtually and with great accuracy, it can instantly be adjusted to fit correctly the wearer's head, and all such required measurements around temple angles and Pantascopic tilt may be calculated from the data obtained during the rendering process. For example, if the wearer's ears are of unequal height, the DMM system will know by how much the variation is and can calculated the different side temple angles when the frame is level horizontal to the wearer's face.

[00445] In addition, since the VDS images are captured and displayed in real time, a sequence of images is available for the system to use, and the most correct alignment image may be selected to be used for the measurements by the DMM.

[00446] Furthermore, the absence of any external reference device eliminates errors prone to such systems.

[00447] Because various spectacle frame parameters can be know in advance, captured at the time of entering the frame details into the VDS, these parameters can be accurately measured and later made available during the DMM dispensing process, hence eliminating estimation errors arising from such in more traditional methods. Such parameters may include Frame Wrap Angle, Frame base curve, depth of inner reverse bevel, actual bridge width, actual eye size, and standard side temple angles from which Pantascopic tilt may be calculated.

[00448] Additionally, Pantascopic tilt may be measured and or calculated directly from the images captured during the frame simulation (VTO) process.

[00449] The pupil center or first Purkinje image reference points can be automatically and accurately found with the DMM dispensing system, as the DMM technology can locate to the accuracy of a pixel dimension the central location of these landmarks.

[00450] Of great importance and further point of difference with the VDS system, is its ability to capture the critical measurement of Back Vertex Distance (BVD), which is defined as the distance from the anterior corneal surface to the posterior surface of the spectacle lens. [00451 ] Furthermore, with traditional optical dispensing, or optical dispensing using prior art VTO systems, once the dispensing measurements are obtained, they must be entered or transmitted in some way to an optical laboratory for lens manufacture. The optical dispenser must ensure that the send or transmitted information is accurate, populated in the correct fields or electronic systems or written in the correct area for writing based systems, correct parameters for the lens type chosen, etc. Hence, a human checking system is paramount, as errors will result in an unacceptable outcome for the wearer. Errors during this process are commonplace, particularly with less experienced or trainee optical dispensers, and with more sophisticated lens designs that may be poorly understood by the optical dispenser.

[00452] The VDS system in accordance with the present embodiment of the invention is sufficiently intelligent to be able to automatically select the correct data sets for specific lens designs, never incorrectly populates fields, and can communicate directly with lens manufacturer's systems, eliminating understanding or communication errors. For instance, some manufactures use a Boxed measurement system, whilst others use a Datum based measurement system. The VTO technology can automatically convert measurements to either protocol.

[00453] Once the lens data has been sent to the lens laboratory for manufacture, the dispenser's job is complete.

[00454] The optical dispensing process typically therefore consists of three major components, 1 ) Frame selection and fitting, 2) Measurements, and 3) Order filling for manufacture and payment.

[00455] Prior art VTO systems, prior art Measurement systems and prior art ordering systems cannot a) complete all three steps, b) cannot duplicate the level of accuracy of the , and c) cannot eliminate the optical dispenser from the procedure. Additionally, prior art systems cannot be completed by the user, cannot be done at the user's convenience anywhere at anytime, and are furthermore cannot complete the process on a smartphone or portable device. [00456] The VDS system offers true live, scaled, real time video simulation of frame try on; does not need an optical dispenser at any point, and as such, the optical dispenser is obsoleted; the user may complete the process by using a smartphone or such portable device and can do so at their convenience at any time; offers accurate measurement capability, including BVD and Pantascopic tilt; and has fully integrated final dispense and e-commerce solutions.

[00457] Modifications and variations as would be apparent to a skilled addressee are deemed to be within the scope of the present invention.

[00458] Throughout this specification, unless the context requires otherwise, the word "comprise" or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.

[00459] Furthermore, various different terminologies are used in this specification which relate to graphical user interface elements, such as buttons, elements, regions, areas, media, and windows. It should be understood that these and other terminology refer to graphics that are displayable on a display screen and that can be interacted with (e.g., via touch screen, mouse devices and/or keyboard) or be associated with other graphical elements that can be interacted with by a user.

[00460] Further, the terms servers, computers, PCs, tablets, or similar computing devices as well as communication devices such as cellphones are all considered to at least have processors for executing software stored in memory, communications components (e.g., Ethernet or wifi) for communicating with other devices across a data network, as well as input devices, such as touch screens, mouse devices, or keyboards, to interact with the software. The software may at least include an operating system, a web browser and sales mechanism software.

[00461 ] Furthermore, As used in this description and in the appended claims, the term "image" refers to any multidimensional representation, whether in tangible or otherwise perceptible form or otherwise, whereby a value of some characteristic is associated with each of a plurality of locations corresponding to dimensional coordinates of an object in physical space, though not necessarily mapped one-to- one thereonto. Thus, for example, the graphic display of the spatial distribution of some feature, such as temperature, in one or more colors constitutes an image. So, also, does an array of numbers in a computer memory or holographic medium. Similarly, "imaging" refers to the rendering of a stated physical characteristic in terms of one or more images.

Claims

1 . A virtual-try-on method for a user to virtually try on a frame of spectacles, the method comprising the steps of: displaying in a graphical interface a real-time video of the user; displaying in the graphical interface at least one spectacle frame for selection by the user. providing a real-time video of a 3D representation of the selected spectacle frame; combining the real-time video of the user with the real-time video of a 3D representation of the selected spectacle frame to create a combined real-time video, the combined real-time video depicting the user wearing the 3D representation of the selected spectacle frame; and displaying in the graphical interface the combined real-time video depicting the user wearing the selected frame.

2. A method according to claim 1 further comprising the step of deriving image capture environment parameters and user visual characteristics from the real time image of the face of the user.

3. A method according to claims 1 or 2 further comprises the step extracting a correspondence between the chosen spectacle frame and the face of the user in the real-time image;

4. A method according to any one of claims 1 to 3 further comprising processing the combined real-time video for adapting the selected spectacle frame to the image capture environment parameters and the visual and anatomical characteristics of the user.

5. A method according to claim 4 further comprising applying the adapted spectacle frame to an image of the face of the user based on the correspondence, thereby generating a simulated look real-time video image of at least the user's head wearing the selected spectacle frame.

6. A method according to claim 5 wherein generating the simulated look real-time video image comprises a simulated look image on each frame of a video sequence in real time to generate the simulated look real-time video image.

7. A method according to claims 5 or 6 wherein the correspondence between the selected spectacle frame and the face of the user may be defined on a basis of extracting a plurality of locations corresponding to an adaptive set of feature locations.

8. A method according to any one of claims 5 to 7 wherein the correspondence is defined on the basis of extracting a plurality of facial data of the user.

9. A method according to any one claims 5 to 8 wherein the simulated look realtime video image is calibrated in the context of the image capture environment parameters so as to generate an overall appearance matching the image capture environment parameters.

10. A method according to any one claims 5 to 9 further comprising step of compensating colors of the glasses and facial appearance, based on facial features of the subject, and/or based on lighting in the image capture environment.

1 1 . A method according to any one claims 5 to 1 0 the simulated image simulated look real-time video image comprises a plurality of independent layers corresponding to the physical parameters of the environment and surrounding the user.

12. A method according to any one claims 2 to 12 wherein the least one of the plurality of independent layers may be color shifted to simulate different colors of some reflections.

13. A method according to any one claims 1 to 12 further comprising the step of receiving a glasses-wearing template created by the user.

14. A method according to any one of claims 1 to 13 further comprising the step of displaying in a graphical interface an eyeglass ordering interface to permit the user to select frames from a multitude of frames.

15. A method according to any one of claims 1 to 14 further comprising the step of displaying a first screen area for displaying an image of the user's face onto which the digital representation of the frame will be superimposed.

16. A method according to any one of claims 1 to 15 further comprising the step of generating a second screen area for displaying a multitude of images of frames to be selected by the user.

17. A method according to any one of claims 1 to 1 6 wherein the virtually trying by the user is conducted by the user on a mobile phone.

18. A method for measuring at least one parameter used for adjusting eyeglasses to a user of the eyeglasses, the method comprising the steps of: generating a simulated look real-time video image of at least the user's head wearing the selected spectacle frame as defined in claims 5 to 17; and processing the digital information for measuring at least one parameter used for adjusting the selected spectacle frame to the user.

19. A method according to claim 18 further comprising the step of obtaining the reference scale using a reference object of standard size for location in a capture plane for recordal of the image and processing.

20. A method according to claim 19 further comprising guiding the user for location of the reference object to the capture plane.

21 . A method according to claim 20 further comprising the step of virtually placing the reference object in a desired measurement plane.

22. A method according to claims 20 or 21 further comprising displaying a brighter box within a darker background for guidance of the user.

23. A method according to any one of claims 20 to 23 wherein a facial recognition process is conducted for guidance of the user.

24. A method according to any one of claims 19 to 23 further comprising the step of accounting for variations in the distance between the user and means for generating the real-time video of the user.

25. A method according to claim 24 further comprising the step of adjusting the size of the 3D representation of the selected spectacle frame if variations in the distance between the user and means for generating the real-time video of the user occur.

26. A method according to any one of claims 18 to 25 wherein the parameter comprises near pupillary distance or distant pupillary distance.

27. A method according to claim 26 wherein the near pupillary distance comprises near monocular distance o near binocular pupillary distance.

28. A method according to claim 26 wherein the distant pupillary comprises distant monocular distance o distant binocular pupillary distance.

29. A method according to any one of claims 18 to 25 wherein the parameter comprises segment heights.

30. A method according to any one of claims 18 to 25 wherein the parameter comprises the Back Vertex Distance.

31 . A method according to claim 30 wherein the Back Vertex Distance is calculated taking into account the lens form to be fitted to the selected spectacle frames.

32. A method according to claim 31 wherein the Back Vertex Distance is calculated using the facial data generated via VDS, along with known, scaled, correctly fitted Glasses data from VDS, and DMM predictions of lens back surface values.

33. A method according to any one of claims 18 to 25 wherein the parameter comprises the Pantascopic tilt.

34. A method according to claim 33 further comprising the step of determine whether the value of the angle of the Pantascopic tilt is suitable for the particular frame selected by the user.

35. A method according to claim 34 further comprising the step of suggesting to the user another frame for selection thereof.

36. A method according to any one of claims 18 to 35 wherein the measurement of parameters is conducted for single vision lenses, bifocal lenses or multifocal lenses.

37. A method according to any one of claims 19 to 36 further comprising the step measuring the location of the 1 st Purkinje images.

38. A method according to claim 37 further comprising the emission of light for eliciting a light reflection off the anterior surface of the cornea of both eyes.

39. A method according to claims 37 or 38 further comprising storing the location of the location of the 1 st Purkinje images for future reference.

40. A method according to any one of claims 19 to 39 wherein the pupillary distance and the segment heights are measured either from 1 st Purkinje images or from an area substantially coinciding with the center of the pupils of the user's eyes.

41 . A virtual-try-on system for users to virtually try-on a spectacles frame, the method comprising a first data storage for storing data associated with users and suppliers of spectacles frames; a second data storage for storing images of spectacles frames to be selected by the users; a third data storage for storing images of the users; at least one graphical display for displaying to at least one user a plurality of screen areas for displaying at least real-time videos of a user and of the spectacle frames; a computer processor for executing programs instructions and for retrieving data stored in any of the first to third data storage; input means for permitting communication between the user and the computer processor to permit the user to provide instructions to the computer processor; and computing means in communication with the computer processor, for storing program instructions for execution by the computer processor to conduct the method as defined in any one of claims 1 to 1 7.

42. A system for measuring at least one parameter used for adjusting eyeglasses to users of the eyeglasses, the system comprising the steps of: a first data storage for storing data associated with users and suppliers of spectacles frames; a second data storage for storing images of spectacles frames to be selected by the users; a third data storage for storing images of the users; at least one graphical display for displaying to at least one user a plurality of screen areas for displaying at least real-time videos of a user and of the spectacle frames; a computer processor for executing programs instructions and for retrieving data stored in any of the first to third data storage; input means for permitting communication between the user and the computer processor to permit the user to provide instructions to the computer processor ; and computing means in communication with the computer processor, for storing program instructions for execution by the computer processor to conduct the method as defined in any one of claims 18 to 40. A system according to claim 41 further comprising fourth data storage for storing data representative of the location of the 1 st Purkinje images

43. An online system for a user to order eyeglasses from a remote location, the system comprising: a measuring system for measuring at least one parameter for adjusting eyeglasses to the user of the eyeglasses, the measuring system being the system as defined in any one of claims 42; at least one first remote device of the user located at the remote location and adapted to be connected to the measuring system; the first remote device comprising the at least one graphical display of the measuring system and being adapted for the user to interact with the online system through an eyeglass ordering interface for providing one or more user's images, selecting one or more frames and filing out an order form for ordering the eyeglasses to be adjusted based on the one or more parameters measured by the measuring system; and at least one second remote device of an optometrist service adapted to be connected to at least to the first remote device; the second remote device being adapted to process the order form for delivery of the eyeglasses ordered by the user; wherein the online system further comprises computing means to display in the at least one graphical display of the user's remote an eyeglass ordering interface to permit the user to select frames from a multitude of frames and order the eyeglasses manufactured based on the at least one measured parameter and comprising the selected frame.

44. An online system for a user to order eyeglasses from a remote location, the system comprising: a virtual-try-on system for a user to virtually try on a spectacles frame as defined in claim 41 ; at least one first remote device of the user located at the remote location and adapted to be connected to the measuring system; the first remote device comprising the at least one graphical display of the measuring system and being adapted for the user to interact with the online system through an eyeglass ordering interface for providing one or more user's images, selecting one or more frames and filing out an order form for ordering the eyeglasses to be adjusted based on the one or more parameters measured by the measuring system; and at least one second remote device of an optical dispensing service adapted to be connected to at least to the first remote device; the second remote device being adapted to process the order form for delivery of the eyeglasses ordered by the user; wherein the online system further comprises computing means to display in the at least one graphical display of the first remote device of the user an eyeglass ordering interface to permit the user to select frames from a multitude of frames and order the eyeglasses manufactured based on the at least one measured parameter and comprising the selected frame.

45. An online system according to claims 44 or 45 wherein the computing means are adapted to permit the user to order the eyeglasses directly to the optical dispensing services.

46. An online system according to claims 44 or 45 wherein ordering the eyeglasses is done via a single click for forwarding the order to the optical dispensing services.

47. An online system according to any one of claims 44 to 47 wherein the one first remote device of the user comprises a mobile phone.

48. An online system according to any one of claims 44 to 48 further comprising data storage means for storing the data of the user for future reference.

49. An online system according to any one of claims 44 to 49 wherein the online system is adapted to refer the user to an optical provider for undergoing a traditional in-office eye exam.

50. An online system according to claim 50 wherein the online system is adapted to access the booking system of the optical provider for booking purposes.

An online system according to any of claims 44 to 43 wherein the one first remote device of the user comprises a mobile phone.