GB2593702A

GB2593702A - Method and system for eyewear fitting

Info

Publication number: GB2593702A
Application number: GB2004622.3A
Authority: GB
Inventors: John Myerscough-Jackopson Peter
Original assignee: FUEL 3D TECHNOLOGIES Ltd; Fuel 3d Tech Ltd
Current assignee: FUEL 3D TECHNOLOGIES Ltd; Fuel 3d Tech Ltd
Priority date: 2020-03-30
Filing date: 2020-03-30
Publication date: 2021-10-06
Also published as: GB202004622D0

Abstract

A computer-implemented method for fitting a model of eyewear to a face of a subject. The method comprises, with an imaging apparatus such as a camera, capturing a sequence of images from a plurality of viewpoints of the face of the subject, each image capturing a respective portion of the face; and displaying the images sequentially on a display. For each image, computing a three-dimensional face model of the portion of the face captured in said image; for each displayed image, using the respective three-dimensional face model to compute a fitted eyewear position and orientation for the respective face portion displayed in said image; and superimposing a model of eyewear onto each displayed image at the respective fitted eyewear position and orientation. Additional modelling data such as depth data and modelling using point cloud. The display of images sequentially may be in response to user input controlling which images are displayed and from which viewpoint. The user may initiate a process of fabricating eyewear according to any of the eyewear models displayed.

Description

METHOD AND SYSTEM FOR EYEWEAR FITTING

FIELD OF INVENTION

The present disclosure relates to a method and system for forming a model of at least one item of eyewear located on a face of a subject.

BACKGROUND

Traditionally, opticians fitting new eyewear to a customer's face manually measure facial dimensions to determine which eyewear frame sizes fit best. The optician asks the customer to remove their old eyewear and uses a ruler or other manual measuring device to determine distances between and the depth of facial features such as the eyes, nose and ears.

Once a suitable frame size has been determined, the optician provides the customer with samples of different frame styles of that size for the customer to try on in front of a mirror to help them choose what style of frame they like best. However, customers who wear and need eyewear and who are asked to remove their old eyewear are typically not able to see well at all distances so are unable to see what they look like in the mirror when trying on the sample frames. Workarounds such as moving the mirror closer to the customer are impractical and do not allow the customer to see themselves wearing the sample frames at all angles.

More recently, imaging techniques have been developed to automate the measurement of the dimensions of the customer's face. Intemafional patent applications WO/2017/077279 and WO/2018/154271 describe methods for computing a 3D face model of a face of a subject and for obtaining the position of eyewear on the subject's face in a high enough level of precision (typically of the order of a few microns) to enable eyewear to be fitted accurately. Such methods allow a static 3D model of a subject's face to be generated and rendered on a display screen in a virtual environment after a snapshot of the depth information has been captured. 3D models of eyewear having the correct size can be superimposed onto the 3D model of the face in the virtual environment at a position on the face corresponding to that in which the eyewear would be worn, allowing an optician to cycle through many different frame styles to assist the subject in determining what style they like best. Unlike when a mirror is used by a subject to view themselves wearing different frame styles, a subject being shown his/her face in a virtual environment on a display screen can put his/her old eyewear back on so that they can see again. This ensures they do not need to sit very close to the display screen and thus ensures that such systems are more practical and less of an inconvenience for the subject. The 3D model can also be rotated and viewed at different zoom settings to give the subject a good overall view of themselves in different frame styles.

However, despite significant advances in computer graphics that allow near photorealistic texture and lighting to be rendered, they do not yet achieve the imaging accuracy of a photograph. Indeed, humans typically find near-life-like 3D models to be unsettling. The more the model resembles a life-like human, the more likely the subject's response to the model will be negative. This effect is known in the art as the "uncanny valley" effect. The "uncanny valley" effect can significantly influence how the subject responds to seeing different frame styles on their own near-life-like 3D face. If they are unsettled, they are less likely to view frame styles that suit them well positively and are less likely to want to use such eyewear fitting systems in future.

An improved apparatus and method is required, preferably one which enables subjects to obtain a more accurate impression of how they would appear wearing specific eyewear.

STATEMENT OF INVENTION

In general terms, the present disclosure proposes capturing images of a subject's face using an imaging device (such as, for example, a camera) and, for each image, determining corresponding modelling data of a visible portion of the subject's face relative to the imaging device sufficient (e.g. together with the image itself) to form a three-dimensional model of at least part of the face, for example using the techniques described in WO/2017/077279 and WO/2018/154271, indicating the positions in three-dimensional space of an array of points on the face. Instead of displaying a rendered 3D model of the subject's face, the captured images themselves are shown on a display, thus avoiding the "uncanny valley" effect. Additionally, a fitted eyewear position and orientation for each image is computed based on the three-dimensional model of the face. The computed fitted eyewear positions and orientations correspond to where the subject would normally wear well-fitting eyewear. A rendered 3D model of eyewear is superimposed on each displayed image at the corresponding computed fitted eyewear position and orientation, that is as a composite image of the face and the eyewear. This allows the subject to virtually try on the eyewear in an accurate, well-fitted position.

By displaying the captured images (for example a video stream) of the subject's own face, the disclosure allows the subject to see what the eyewear would like on an actual image of themselves, rather than on a rendered 3D model of their face. The "uncanny valley" effect described above thus does not occur.

Particularly for a subject who requires vision-correction eyewear, the images are captured in a first phase in which the subject is not wearing the vision-correcting eyewear, and then displayed to the subject, including the rendered 3D model of eyewear (typically different eyewear from the vision-correcting eyewear), at a time at which the subject is wearing the vision-correcting eyewear.

Note that the position and orientation of the superimposed eyewear based on determined depth data is preferably significantly more accurate than the accuracy it is possible to achieve using traditional augmented reality (AR) applications such as known social media filter/overlay techniques. Such techniques typically use feature detection (for example edge detection) on a live video stream to detect and track the position of the face and its features to superimpose props for entertainment purposes.

In such cases, no three-dimensional model is formed and the resulting superimposed prop is not able to be accurately positioned or oriented. For example, as soon as a subject rotates their face too far in one direction on a live video stream, such techniques are unable to track the facial features that might be used to compute a fitted eyewear position and orientation. For example, such techniques would not be able to accurately position a 3D model of eyewear on a subject who has turned the side of their face to the camera because the facial features that are needed to do this are not in view. In contrast and in accordance with the present disclosure, by capturing images and determining the associated modelling data first a full 3D profile of the subject's face is obtained. By computing the corresponding fitted eyewear position and orientation for those viewpoints, the 3D eyewear model can be then superimposed at the correct position and orientation when those images are played back to a subject on a display (e.g. when the subject is wearing vision-correcting eyewear).

A first expression of the invention is a computer-implemented method for displaying a sequence of images showing eyewear positioned on a face of a subject, the method comprising: with an imaging device, capturing a sequence of images from a plurality of respective viewpoints of the face of the subject, each image capturing a respective portion of the face; for each image, computing a corresponding three-dimensional face model of the portion of the face captured in said image; displaying the images sequentially on a display for each displayed image, using the respective three-dimensional face model to compute a fitted eyewear position and orientation for the respective face portion displayed in said image; and superimposing a first model of eyewear onto each displayed image at the respective fitted eyewear position and orientation.

A second expression of the invention is a system for displaying a sequence of images showing eyewear positioned on a face of a subject, comprising: an imaging device; a display; and a processor, wherein the imaging device is configured to capture a sequence of images from a plurality of viewpoints of the face of the subject, each image representing a respective portion of the face; wherein the display is configured to display the images sequentially; wherein the processor is configured to: for each image, compute a three-dimensional face model of the portion of the face captured in said image; for each displayed image, use the respective three-dimensional face model to compute a fitted eyewear position and orientation for the respective face portion displayed in said image and superimpose a model of eyewear onto each displayed image at the respective fitted eyewear position and orientation.

BRIEF DESCRIPTION OF THE DRAWINGS

An embodiment of the invention will now be described for the sake of example only with reference to the following figures in which: Figure 1 is a flow diagram of steps according a method according to the present disclosure.

Figures 2a schematically shows a system according to the present disclosure.

Figure 2b schematically shows an imaging apparatus according to the present disclosure Figure 2c schematically shows an imaging apparatus according to the present

disclosure.

Figures 3a-3d schematically show a method according to the present disclosure.

Figures 4a-4c schematically show images displayed sequentially on a display

according to the present disclosure.

Figure 5 schematically shows an exemplary data-processing apparatus according to the present disclosure.

DETAILED DESCRIPTION

Figure 1 is a flow diagram of steps of a computer-implemented method according to the present disclosure for fitting a model of eyewear to a face of a subject. The method 100 comprises capturing 101 by an imaging device a sequence of images from a plurality of viewpoints of the face of the subject. Each image may capture a respective portion of the face. The sequence of images may be, for example, a sequence of frames of a video stream of the subject's face or a plurality of still images of the subject's face taken separately.

As described below, each image may be captured together with (i.e. at substantially the same time as) additional modelling data which allows a three-dimensional model of the face to be calculated. In one example, the additional modelling data may comprise depth data (that is, the respective distances from the imaging apparatus to a plurality of positions on the face), for example in the form of a point cloud. The modelling data may be stored in a memory in association with that image.

For each captured image, a respective three-dimensional face model of the portion of the face captured in that image is computed 102. In the case that additional modelling data was captured and stored for each image, the computation of the three-dimensional face model may use the stored additional modelling data. For example, in the case that the modelling data is a point-cloud of distances, the distance data may be retrieved from the memory and used to generate the three-dimensional face model.

In some forms of the method, modelling data associated with multiple ones of the images is used in combination to form the three-dimensional model. For example, in some cases it may be assumed that the 3D shape of some or all of the subject's face is the same for different ones of the images. In this case, the modelling data corresponding to multiple ones of the images may be used to form a three-dimensional model of that that part of the subject's face. The three-dimensional model can then be rotated and/or translated (e.g. using the respective modelling data for each of the images), to form some or all of the respective three-dimensional model for each image of the image sequence For each displayed image, the respective three-dimensional face model is used to compute 103 a fitted eyewear position and orientation for the respective face portion displayed in that particular image. The fitted eyewear position corresponds to the position an optician would fit an item of eyewear at or on the subject's face (e.g. to correct their vision). The fitted eyewear orientation corresponds to the orientation the eyewear would be at, relative to the imaging apparatus, had there been eyewear present in the image.

The captured images are then displayed 104 sequentially on a display. This may comprise, for example, playing back the video stream on the display or displaying captured still images one after another on the display. Note that the sequence in which the images are displayed may not be the same sequence in which they were captured. For example, the system may include a process (e.g. performed based on input ("display control input") received from a user (who may be the subject), e.g. concurrently with the sequential display of the images) of defining the order in which the images are displayed.

For example, the display of images sequentially may be in response to user input (e.g. controlling a slider on the display) that controls which of the images is displayed at a given time thereby providing the user with the ability to "rotate" their face on the display and view it from any of the viewpoints from which the images were captured. This could include "rotation" of views from above, below, front, side and/or back of the face, depending on what views were captured. For this purpose, for each of the images, the system may generate corresponding direction data indicating a direction in which the subject's face is facing in the image, and then the image displayed at any time may be selected as the image for which the corresponding direction data indicates a direction which is closest to a direction specified by the display control input.

A model of eyewear is superimposed 105 onto each displayed image at the respective fitted eyewear position and orientation. The model of eyewear may be based on pre-existing data (a "template") describing the eyewear. It may comprise, for example, of information indicative of the dimensions of the eyewear, for example, in the form of a point cloud, polygon or triangle mesh, and/or other 3D surface model.

In the case that the relative position of the face and the imaging device is such that, if the subject had been wearing the eyewear shown by the 3D model in reality, part of it would have been occluded, then it is preferably occluded in the same way in the display also. For example, if the captured image shows one side of the subject's face, such that only that side of the eyewear would have been visible if the subject had been wearing the eyewear, the display shows only that side of the eyewear.

Note that steps 104 and 105 may be performed concurrently. Each image is used to generate a respective composite image of the subject's face and the superimposed model of the eyewear, and those composite images are successively displayed.

As described above, by displaying the captured images on the display instead of a rendered version of the three-dimensional face model, the "uncanny valley" effect is avoided without affecting the accuracy of the fitted eyewear position and orientation when the eyewear is superimposed on the displayed images.

As described above, the modelling data Of any) may be depth data.

Alternatively, it may be a second image of the face captured at substantially the same time that the (first) image is captured, by a second imaging device at a position offset from the position of the (first) imaging device. In this case, the three-dimensional model may be captured by stereoscopy, combining the image captured by the imaging device with the second image. Note however that it is possible for the three-dimensional model for each image to be computed by stereoscopy without using second images captured by a second imaging device (e.g. in the absence of any additional modelling data being captured or stored). This can be done using a structure-from-motion analysis, exploiting the motion of the (first) imaging device relative to the face. The technique may use a stereoscopic technique based on at least two images captured by the first imaging device at successive times at which the imaging device is in different respective positional relationships to the face. One of these images may be the image with which the computed three-dimensional model is associated.

In another method according to the present disclosure, the face of the subject may be illuminated successively, at respective times separated by very short intervals, from at least three directions, with one or more energy sources. Each image captured by the imaging device may correspond to an image captured when the face is illuminated from one of these directions, and the corresponding additional model data may be images captured when the face is illuminated from the other directions a very short time later or earlier.

By illuminating the face in at least three directions and observing the diffuse reflected energy under each illumination direction, photometric data, for example photometric stereo data, may be generated for a plurality of respective positions on the face. The photometric data may in turn be used to generate the above described three-dimensional face model. Unlike 3D data collected using time-of-flight measurement techniques used by some 3D scanners, for example those found in smartphones, photometric stereo data can be generated without structured light, lasers, or other expensive components and may also be used to safely and accurately image eyes, something that is necessary for accurately fitting eyewear to a subject's face.

The method according to the present disclosure may further comprise replacing the superimposed first model of eyewear with a second model of eyewear on each displayed image. Whilst the second eyewear model may have a different style, texture, shape, surface features and/or decorations to the first eyewear model, its fitted position and orientation may be unchanged (for example because it sits at the same position on the subject's face as the first eyewear model). The same fitted eyewear position and orientation is thus used for both eyewear models and it is thus not necessary to recompute the fitted position and orientation each time the eyewear model is replaced. The first and second eyewear models may be based on respective templates retrieved from a library of eyewear models provided by, for example, a manufacturer of eyewear.

The replacement of the first model of eyewear with the second model of eyewear may be performed responsive to user input (e.g. from the subject). For example, a slider on a user interface may be provided allowing a user (e.g. the subject) to swipe through a plurality of models of eyewear. Alternatively, the replacement of one eyewear model with another may occur automatically after a predetermined time delay. The second model of eyewear can then be replaced by a third eyewear model in the same way, and so on until the subject finds an eyewear model he/she likes best. In this way, the subject is able to view themselves virtually wearing different models of eyewear in a perfectly fitted position without physically needing to try each eyewear on physically.

Typically, the display of the images in steps 104 and 105 is not performed in real time (i.e. while step 101 is still being carried out). The subject may thus continue to wear his/her own, old eyewear while viewing the display so will not need to sit very close to or squint at the display thus improving the experience the subject has while fitting eyewear.

The user (the subject or another person) may initiate a process of fabricating eyewear according to any of the eyewear models. For example, the subject or the other person may supply input to cause the eyewear to be fabricated, e.g. by transmitting a command signal to an eyewear fabrication unit to cause the eyewear fabrication unit to fabricate the eyewear. At least one parameter of the fabricated eyewear may be set based on one or more of the three-dimensional face models.

Figures 2a schematically shows a system 200 for fitting a model of eyewear to a face 201 of a subject according to the present disclosure. The system 200 comprises an imaging apparatus 202, a display 203 and a processor 204. The imaging apparatus 202, display 203, and processor 204 may all be part of the same data-processing apparatus such as, for example, a computer, smartphone, laptop and/or other mobile device. Alternatively, they may be separate components connected to each other by one or more physical and/or network connections. The imaging apparatus 202 includes a camera 205 (a video camera or a still camera) configured to capture a sequence of images of the face 201 of the subject, each image capturing a respective portion of the face.

The relative positions of the imaging apparatus 202 and the face 201 are preferably changed while the sequence of images of the face 201 is captured, so that each image represents a different "view" of the face from a respective viewpoint. Figures 2b and 2c schematically show the imaging apparatus 202 capturing images from different viewpoints of the face 201 of the subject relative to Figure 2a. Different portions of the face 201 are captured in each image. For example, in Figure 2a, a side portion of the face 201 is captured, in Figure 2b a front portion of the face 201 is captured, and in Figure 2c a rear portion of the face 201 is captured. The display 203 is configured to display the captured images sequentially, as is described above in connection with the method of the present disclosure.

The system may further comprise one or more sensors 206 for capturing modelling data for each of the images captured by the imaging apparatus 202. The sensor(s) 206 may be depth sensor(s) for measuring the distance from the imaging apparatus 202 to the closest object in each of a plurality of directions. The face 201 may be located to intercept with a plurality of these directions, so that the sensor obtains the distances from the imaging apparatus 202 of multiple points on the face 201.

Alternatively, the sensor 206 may be a second camera arranged to take a second image of the face whenever the (first) imaging apparatus 205 does so.

Alternatively, the sensor 206 may comprise one or more energy source configured to illuminate the face 201 of the subject successively in at least three directions. The one or more energy sources may comprise one or more light emitting diodes, for example, of a mobile device such as a smartphone. The camera 205 may be used to capture the images described above as images when the face is illuminated by ambient light or when the face is illuminated in one of the three directions. The corresponding modelling data may comprise images captured by the camera 205 at very slightly later or earlier times when the face is illuminated in other directions.

The sensor 206 described herein may further be configured to capture one or more views of the environment around a subject, for example the environment in front of the subject's face. In this way, the background lighting conditions under which the subject's face is imaged may be determined and used to reduce noise. For example, light may reflect from the environment onto the subject's face and inadvertently generate noise in the modelling data. The noise may be caused by the background lighting conditions having an effect on the brightness, colour, contrast, and/or shadows present on the subject's face as it is imaged. By capturing one or more views of the environment around the subject, the contribution the background lighting conditions have to the modelling data may be determined and compensated for in the modelling data. For example, the views of the environment around the subject may be used to model reflectance of the environment around the subject, and thus estimate, for example, the colour, brightness and direction of background lighting conditions.

As described above in connection the method 100 of the present disclosure, the processor 204 is configured to, for each captured image, compute a three-dimensional face model of the portion of face captured in said image. It does this using the additional data associated with each of the images, and, in some implementations using the image itself.

For example, in the case that the additional data is depth data, the processor 204 may form the three dimensional model using known algorithms.

In the case that the additional data (and optionally the images) comprise photographs captured when the face is illuminated in respective directions, the computing of the three-dimensional face model by the processor may comprise using photometric techniques to generate the three-dimensional face model, as is described above.

The processor 204 is further configured to, for each displayed image, use the respective three-dimensional face model to compute a fitted eyewear position and orientation for the respective face portion displayed in said image and superimpose a model of eyewear onto each displayed image at the respective fitted eyewear position and orientation.

Optionally, the system 200 may comprise an eyewear fabrication unit (not shown) for fabricating eyewear according to the model of eyewear. Alternatively, the system 200 may comprise a communication interface (not shown) for transmitting a command to fabricate the eyewear to an external eyewear fabrication unit, to cause the eyewear fabrication unit to fabricate the eyewear. In either case, the system 200 may control the eyewear fabrication unit to fabricate the eyewear based on at least one parameter which is determined based on one or more of the three-dimensional face models.

Figures 3a-3d illustrate the method described above from the capture of an image to the computing of a fitted eyewear position and orientation for the respective face portion displayed in that image. In particular, Figure 3a schematically shows a captured image 301 of a face 201, for example from the viewpoint of the subject of Figure 2a, that may be displayed on the display 203. As is illustrated in Figure 3b, the computed three-dimensional face model corresponds to the portion of the face 201 visible in the captured image 301 and may be a point cloud or other representation of information indicative of the distances from the imaging apparatus to different positions on the face.

Once the three-dimensional face model is computed, the fitted eyewear position and orientation may be determined, as illustrated in Figure 3c. An eyewear model template 303 is positioned as an eyewear model 305 on the captured images of the subject's face, as shown in Figure 3d.

Figures 4a-4c schematically show three images 401, 402, 403 displayed sequentially on the display 203 of the system of the present disclosure. The three displayed images 401, 402, 403, are captured from the viewpoints illustrated respectively in Figures 2a-2c. Namely, the image 401 of Figure 4a corresponds to a view from the side of the subject's face 201 shown in Figure 2a, the image 402 of Figure 4b corresponds to a view from the front of the subject's face 201, and the image 403 of Figure 4c corresponds to a view from the rear of the subject's face. In each displayed image 401, 402 403, a model of eyewear 305 derived from the template 303 is superimposed at the respective fitted eyewear position and orientation. In each displayed image, a different corresponding portion of the model of eyewear 305 is visible depending on which face portion is present in the image. Known occlusion culling algorithms may be used to determine which portions of the model of eyewear 305 would be visible in the image and accordingly which portions to render when the model of eyewear 305 is superimposed on the displayed image.

Although the invention has been described in terms of preferred embodiments as set forth above, it should be understood that these embodiments are illustrative only and that the claims are not limited to those embodiments. Those skilled in the art will be able to make modifications and alternatives in view of the disclosure which are contemplated as falling within the scope of the appended claims. Each feature disclosed or illustrated in the present specification may be incorporated in the invention, whether alone or in any appropriate combination with any other feature disclosed or illustrated herein.

For example, the imaging apparatus may comprise a camera of a mobile device configured to capture a red, green, and blue pixel (RGB) video stream, wherein the sequence of images described herein comprises one or more images from the RGB video stream.

Also provided herein, as shown by the technical architecture in Figure 5, is an exemplary data-processing apparatus comprising means for carrying out the steps of the methods of any of the above embodiments.

The data-processing apparatus may comprise a processor 500, for example the processor of the system described herein, that is in communication with memory devices including secondary storage 501 (such as disk drives), read only memory (ROM) 502, random access memory (RAM) 503. The processor 500 may be implemented as one or more CPU chips, which are cheaper than GPUs. The data-processing apparatus may further comprise input/output (I/O) devices 504, and network connectivity devices 505.

The secondary storage 501 is typically comprised of one or more disk drives or tape drives and is used for non-volatile storage of data and as an over-flow data storage device if RAM 503 is not large enough to hold all working data. Secondary storage 501 may be used to store programs which are loaded into RAM 503 when such programs are selected for execution.

In this embodiment, the secondary storage 501 has an order processing component 501a comprising non-transitory instructions operative by the processor 500 to perform various operations of the method of the present disclosure. The ROM 502 is used to store instructions and perhaps data which are read during program execution. The secondary storage 501, the RAM 503, and/or the ROM 502 may be referred to in some contexts as computer readable storage media and/or non-transitory computer readable media.

I/O devices 504 may include printers, video monitors, liquid crystal displays (LCDs), plasma displays, touch screen displays, keyboards, keypads, switches, dials, mice, track balls, voice recognizers, card readers, paper tape readers, or other well-known input devices.

The network connectivity devices 505 may take the form of modems, modem banks, Ethernet cards, universal serial bus (USB) interface cards, serial interfaces, token ring cards, fiber distributed data interface (FDDI) cards, wireless local area network (WLAN) cards, radio transceiver cards that promote radio communications using protocols such as code division multiple access (CDMA), global system for mobile communications (GSM), long-term evolution (LIE), worldwide interoperability for microwave access (WiMAX), near field communications (NFC), radio frequency identity (RFID), and/or other air interface protocol radio transceiver cards, and other well-known network devices. These network connectivity devices 505 may enable the processor 500 to communicate with the Internet or one or more intranets. With such a network connection, it is contemplated that the processor 500 might receive information from the network, or might output information to the network in the course of performing the above-described method operations. Such information, which is often represented as a sequence of instructions to be executed using processor 500, may be received from and outputted to the network, for example, in the form of a computer data signal embodied in a carrier wave.

The processor 500 executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk based systems may all be considered secondary storage 501), flash drive, ROM 502, RAM 503, or the network connectivity devices 505. While only one processor 500 is shown, multiple processors may be present. Thus, while instructions may be discussed as executed by a processor, the instructions may be executed simultaneously, serially, or otherwise executed by one or multiple processors.

Although the technical architecture is described with reference to a computer, it should be appreciated that the technical architecture may be formed by two or more computers in communication with each other that collaborate to perform a task. For example, but not by way of limitation, an application may be partitioned in such a way as to permit concurrent and/or parallel processing of the instructions of the application. Alternatively, the data processed by the application may be partitioned in such a way as to permit concurrent and/or parallel processing of different portions of a data set by the two or more computers. In an embodiment, virtualizabon software may be employed by the technical architecture to provide the functionality of a number of servers that is not directly bound to the number of computers in the technical architecture. In an embodiment, the functionality disclosed above may be provided by executing the application and/or applications in a cloud computing environment. Cloud computing may comprise providing computing services via a network connection using dynamically scalable computing resources. A cloud computing environment may be established by an enterprise and/or may be hired on an as-needed basis from a third party provider.

It is understood that by programming and/or loading executable instructions onto the technical architecture, at least one of the CPU 500, the RAM 503, and the ROM 502 are changed, transforming the technical architecture in part into a specific purpose machine or apparatus having the novel functionality taught by the present disclosure. It is fundamental to the electrical engineering and software engineering arts that functionality that can be implemented by loading executable software into a computer can be converted to a hardware implementation by well-known design rules.

Also provided herein is a computer program and a computer-readable storage medium for storing the computer program, comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the methods of any of the above embodiments.

Claims

CLAIMS: 1. A computer-implemented method for displaying a sequence of images showing eyewear positioned on a face of a subject, the method comprising: with an imaging device, capturing a sequence of images from a plurality of respective viewpoints of the face of the subject, each image capturing a respective portion of the face; for each image computing a three-dimensional face model of the portion of the face captured in said image; displaying the images sequentially on a display for each displayed image, using the respective three-dimensional face model to compute a fitted eyewear position and orientation for the respective face portion displayed in said image; and superimposing a first model of eyewear onto each displayed image at the respective fitted eyewear position and orientation.
2. A computer-implemented method according to claim 1 in which the three-dimensional model is computed by stereoscopy.
3. A computer-implemented method according to claim 1 or claim 2 further including capturing modelling data associated with each image, said computing of the three-dimensional face model for each image being performed using the modelling data associated with the corresponding image.
4. A computer-implemented method according to claim 3 in which the modelling data comprises depth data indicative of the distances from the imaging device to multiple respective positions on the face.
5. A computer-implemented method according to claim 3 in which the modelling data comprises at least one photograph captured by a camera at a position offset from the imaging device, the three-dimensional model being formed by stereoscopy.
The method of claim 3, comprising: with one or more energy sources, successively illuminating the face of the subject in at least three directions, the modelling data comprising photographs captured when the face is illuminated in respective multiple ones of the directions; and wherein computing the three-dimensional face model of the respective face portion captured in each image comprises: using the modelling data to generate said three-dimensional face model by photometry.
7. The method of any preceding claim, comprising: replacing the superimposed first model of eyewear with a second model of eyewear on each displayed image at the fitted eyewear position and orientation.
8. The method of claim 5, wherein replacing the model of eyewear with the second model of eyewear is performed responsive to an input of a user.
9. The method of any preceding claim, wherein the fitted eyewear position corresponds to a position on the face of the subject at which fitted eyewear corrects vision of the subject.
10. The method of any preceding claim comprising receiving display control input from a user, and defining the order in which the images are displayed during said sequential display of the images based on the display control input.
11. A method of fabricating eyewear comprising: displaying a sequence of images showing eyewear positioned on a face of a subject by a method according to any preceding claim, and based on received input, initiating fabrication of an item of eyewear according to the first model.
12. A system for displaying a sequence of images showing eyewear positioned on a face of a subject, comprising: an imaging device; a display; and a processor, wherein the imaging device is configured to capture a sequence of images from a plurality of viewpoints of the face of the subject, each image representing a respective portion of the face; wherein the display is configured to display the images sequentially; wherein the processor is configured to: for each image, compute a three-dimensional face model of the portion of the face captured in said image; for each displayed image, use the respective three-dimensional face model to compute a fitted eyewear position and orientation for the respective face portion displayed in said image and superimpose a model of eyewear onto each displayed image at the respective fitted eyewear position and orientation.
13. The system of claim 12 in which the processor is configured to compute the three-dimensional model for each image by stereoscopy.
14. The system of claim 12 or claim 13 in which each of the images is associated with corresponding captured modelling data, and in which the processor is configured to compute the three-dimensional model for each image using the modelling data associated with the image.
15. The system of claim 14, comprising a depth sensor arranged to capture the modelling data as depth data indicative of the distances from the imaging device to multiple respective positions on the face.
16. The system of claim 14 further comprising one or more energy sources configured to illuminate the face of the subject successively in at least three directions.
17. The system of claim 16, wherein the one or more energy sources comprise one or more light emitting diodes (LEDs) of a mobile device.
18. A computer program comprising instructions which, when the program is executed by a computer having an imaging device and a display, cause the computer to carry out the steps of any of the methods of claims 1-10.
19. A computer-readable storage medium having stored thereon a computer program according to claim 18.