EP4652582A1

EP4652582A1 - Fit prediction based on feature detection in image data

Info

Publication number: EP4652582A1
Application number: EP24705273.1A
Authority: EP
Inventors: Idris Syed Aleem; Mayank BHARGAVA; Sushant Umesh Kulkarni
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2023-01-18
Filing date: 2024-01-04
Publication date: 2025-11-26
Also published as: WO2024155447A1

Abstract

A system and method of predicting fit of a wearable device from image data obtained by a computing device together with position and orientation of the computing device is provided. The system and method may include capturing a series of frames of image data, and detecting one or more fixed features in the series of frames of image data. Position and orientation data associated with the capture of the image data is combined with the position data related to the one or more fixed features, to extract depth data from the series of frames of image data. A three-dimensional model is generated based on the extracted depth data. The three-dimensional model and/or key points extracted therefrom, can be processed by a simulator and/or a machine learning model to predict fit of the wearable device for the user.

Description

FIT PREDICTION BASED ON FEATURE

DETECTION IN IMAGE DATA

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation of, and claims priority to, U.S. Nonprovisional Patent Application No. 18/156,156, filed on January 18, 2023, entitled “FIT PREDICTION BASED ON DETECTION OF METRIC FEATURES IN IMAGE DATA,” and to U.S. Non-provisional Patent Application No. 18/157,498, filed on January 20, 2023, entitled “FIT PREDICTION BASED ON FEATURE DETECTION IN IMAGE DATA.” the disclosures of which are incorporated by reference herein in their entireties.

TECHNICAL FIELD

[0002] This relates in general to the detection of scale from image data, and in particular to the detection of scale of facial features from image data together with position and/or orientation data, to predict fit of a wearable device.

BACKGROUND

[0003] A manner in which a wearable device fits a particular wearer may be dependent on features specific to the wearer, how the wearable device interacts with features associated with the specific body part at which the wearable device is worn by the wearer, and the like. In some situations, a wearer may want to customize a wearable device for fit and/or function. For example, when fitting a pair of glasses, the wearer may want to customize the glasses to incorporate selected frame(s), prescription/ corrective lenses, a display device, computing capabilities, and other such features. Many existing systems for procurement of these types of wearable devices do not provide for accurate fitting and customization without access to a retail establishment and/or without the assistance of a technician and/or without access to specialized equipment. Existing virtual systems may provide a virtual try-on capability, but may lack the ability to accurately size the wearable device from images of the wearer without specialized equipment. This may result in improper fit of the delivered product. In the case of a head mounted wearable device, such as smart glasses that include display capability and computing capability, improper fit may compromise the functionality.

SUMMARY

[0004] S\ stems and methods are described herein that provide for the selection, sizing and/or fitting of a head mounted wearable device based on a series of frames of two-dimensional image data of a user. The series of frames of two- dimensional image data may be captured via an application executing on a computing device operated by the user. In some examples, the sizing and/or fitting of the head mounted wearable device may be accomplished based on the series of image data together with motion or movement related data associated with the computing device. The series of frames of image data may be captured via an application executing on a computing device operated by the user. A user mesh is generated, representative of the head, for example a portion of the head, such as the face of the user, based on one or more facial landmarks detected within the series of frames of two-dimensional image data. Changes in position of the one or more facial landmarks in the sequential image frames are correlated with changes in position and/or orientation of the computing device provided by position/orientations sensors of the computing device to determine depth data. The depth data may include depth values associated with a plurality of different points detected within the image data. The depth data is used to develop one or more depth maps which are fused to in turn generate a three- dimensional mesh, or a three-dimensional model, that is representative of the face and/or head of the user. The three-dimensional mesh, or model, and/or facial and/or cranial and/or ophthalmic measurements extracted therefrom, are provided to a simulator, to predict fit of a head mounted wearable device for the user.

[0005] In one general aspect, the proposed solution in particular relates to a (computer-implemented) method, in particular a method for partially or fully automated selection, sizing and/or fitting of a head mounted wearable device to userspecific requirements, the method including capturing current image data, via an application executing on a computing device operated by a user, the current image data including (a representation of) a head of the user; detecting at least one fixed feature in the current image data; detecting a change in a position and an orientation of the computing device, from a previous position and a previous orientation corresponding to the capturing of previous image data, to a cunent position and a current orientation corresponding to the capturing of the current image data; detecting a change in a position of the at least one fixed feature between the current image data and the previous image data; correlating the change in the position and the orientation of the computing device with the change in the position of the at least one fixed feature; generating a three-dimensional model of the head of the user based on depth data extracted from the correlating of the change in position and orientation of the computing device with the change in position and orientation of the at least one fixed feature; and predicting, by a machine learning model accessible to the computing device, a fit of a head mounted wearable device on the head of the user based on the three-dimensional model of the head of the user.

[0006] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium storing executable instructions that when executed by at least one processor of a computing device are configured to cause the at least one processor to: capture, by an image sensor of the computing device, current image data, the current image data including a head of a user; detect at least one fixed feature in the current image data; detect a change in a position and an orientation of the computing device, from a current position and a current orientation corresponding to the capture of the current image data, to a previous position and a previous orientation corresponding to the capture of previous image data including the head of the user; detect a change in a position of the at least one fixed feature between the current image data and the previous image data; correlate the change in the position and the orientation of the computing device with the change in the position of the at least one fixed feature; generate a three-dimensional model of the head of the user based on depth data extracted from the correlation of the change in position and orientation of the computing device with the change in position and orientation of the at least one fixed feature; and predict, by a machine learning model accessible to the computing device, a fit of a head mounted w earable device on the head of the user based on the three-dimensional model of the head of the user.

[0007] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the at least one fixed feature includes at least two facial landmarks that are representative of a facial measurement, including at least one of: a distance betw een a first ear saddle point and a second ear saddle point representative of a head width of a user; a distance between an outer comer portion of a right eye and an outer comer portion of a left eye of the user; a distance between an inner comer portion of the right eye and an inner comer portion of the left eye of the user; or a distance between a pupil of the right eye and a pupil of the left eye of the user.

[0008] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the at least one fixed feature includes a distance between at least two fixed elements detected in a background area surrounding the head of the user.

[0009] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the executable instructions cause the at least one processor to detect the change in the position and the orientation of the computing device, including: detect the previous position and the previous orientation of the computing device in response to receiving previous data provided by an inertial measurement unit of the computing device at the capture of the previous image data; detect the current position and the current orientation of the computing device in response to receiving current data provided by the inertial measurement unit of the computing device at the capture of the current image data; and determine a magnitude of movement of the computing device corresponding to the change in the position and the orientation of the computing device based on a comparison of the current data and the previous data.

[0010] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the executable instructions cause the at least one processor to: associate the magnitude of the movement of the computing device to a change in a measurement associated with the at least one fixed feature; and determine depth data based on the associating.

[0011] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the executable instructions cause the at least one processor to: repeatedly capture image data as the computing device is moved relative to the user to capture image data from a plurality’ of different positions and orientations of the computing device relative to the head of the user; correlate a plurality of changes in position and orientation of the computing device with a corresponding plurality' of changes in position of the at least one fixed feature detected the image data; determine depth data as the image data is repeatedly captured from the plurality’ of different positions and orientations based on the correlating; and develop the three-dimensional model of the head of the user for predicting the fit of the head mounted wearable device based on the repeatedly capturing of the image data by the computing device from the plurality of different positions and orientations and the depth data determined from the repeatedly capturing of the image data. [0012] In some aspects, the techniques described herein relate to a non- transitory computer-readable medium, wherein the executable instructions cause the at least one processor to: generate the three-dimensional model of the head of the user; extract at least one measurement from the three-dimensional model of the head of the user; and select a head mounted wearable device, from a plurality of available head mounted wearable devices, based on the at least one measurement, the at least one measurement including at least one of: a cranial measurement determined based on distance between two fixed facial features detected in the current image data and the previous image data; or an ophthalmic measurement determined based on a distance between two optical features detected in the current image data and the previous image data.

[0013] In some aspects, the techniques described herein relate to a computer- implemented method, including: capturing current image data, via an application executing on a computing device (operated or operable by a user), the current image data including (a representation of) a nose of the user captured at a current position and a current orientation of the computing device; detecting at least one fixed feature in the current image data; detecting a change in a position and an orientation of the computing device; detecting a change in a position of the at least one fixed feature between the current image data and previous image data captured at a previous position and a previous orientation of the computing device; correlating the change in the position and the orientation of the computing device with the change in the position of the at least one fixed feature; generating a three-dimensional model of the nose of the user based on depth data extracted from the correlating of the change in position and orientation of the computing device with the change in position and orientation of the at least one fixed feature; and simulating, by a simulation engine accessible to the computing device, a fit of a head mounted wearable device on a head of the user based on the three-dimensional model of the nose of the user.

[0014] In some aspects, the techniques described herein relate to a computer- implemented method, wherein the at least one fixed feature includes at least two facial features that are representative of a fixed measurement associated with the nose of the user. [0015] In some aspects, the techniques described herein relate to a computer- implemented method, wherein the fixed measurement includes at least one of: a width of the nose at a root end portion of the nose; or a slope of the nose along a nasal ridge of the nose.

[0016] In some aspects, the techniques described herein relate to a computer- implemented method, wherein the width of the nose is representative of a distance between a right end portion of the nose at the root end portion of the nose, and a left end portion of the nose at the root end portion of the nose.

[0017] In some aspects, the techniques described herein relate to a computer- implemented method, wherein the at least two facial features includes three facial features, including: a sellion at the root end portion of the nasal ridge of the nose; a tip of the nose at a distal end portion of the nasal ridge of the nose; and an ala at a lower end portion of the nose, corresponding to a first lower end portion and a second lower end portion of the nose.

[0018] In some aspects, the techniques described herein relate to a computer- implemented method, wherein the fixed measurement includes: a nose height, representative of a distance between the root end portion of the nose and at least one of the first lower end portion or the second lower end portion of the nose; and a nose depth, representative of a distance between the tip of the nose and at least one of the first lower end portion or the second lower end portion of the nose, wherein the slope is the quotient of the nose height divided by the nose depth.

[0019] In some aspects, the techniques described herein relate to a computer- implemented method, further including: generating, by the simulation engine, a simulated fit of the head mounted wearable device based on the simulating; selecting a pair of adjustment pads, from a plurality of adjustment pads, that are selectively couplable to the head mounted wearable device; and generating a simulated adjusted fit of the head mounted wearable device including the pair of adjustment pads.

[0020] In some aspects, the techniques described herein relate to a computer- implemented method, wherein the pair of adjustment pads includes a first adjustment pad that is selectively couplable to a first rim portion of the head mounted wearable device and a second adjustment pad that is selectively couplable to a second rim portion of the head mounted wearable device.

[0021] In some aspects, the techniques described herein relate to a computer- implemented method, wherein generating the simulated adjusted fit includes adjusting at least one of: a position of a bridge portion of the head mounted wearable device along a nasal ridge of the nose on the three-dimensional model of the nose; an angular position of a front frame portion of the head mounted wearable device relative to the nasal ridge of the nose on the three-dimensional model of the nose.

[0022] In some aspects, the techniques described herein relate to a computer- implemented method, wherein detecting the change in the position and the orientation of the computing device includes: detecting the previous position and the previous orientation of the computing device in response to receiving previous data provided by an inertial measurement unit of the computing device at the capturing of the previous image data; detecting the current position and the current orientation of the computing device in response to receiving current data provided by the inertial measurement unit of the computing device at the capturing of the current image data; and determining a magnitude of movement of the computing device corresponding to the change in the position and the orientation of the computing device based on a comparison of the current data and the previous data.

[0023] In some aspects, the techniques described herein relate to a computer- implemented method, wherein correlating the change in the position and the orientation of the computing device with the change in the position of the at least one fixed feature includes: associating the magnitude of the movement of the computing device to a change in a measurement associated with the at least one fixed feature; and determining depth data based on the associating.

[0024] In some aspects, the techniques described herein relate to a computer- implemented method, further including: repeatedly capturing image data as the computing device is moved relative to the user to capture image data from a plurality of different positions and orientations of the computing device relative to the head of the user; correlating a plurality of changes in position and orientation of the computing device with a corresponding plurality of changes in position of the at least one fixed feature detected the image data; determining depth data as the image data is repeatedly captured from the plurality of different positions and orientations based on the correlating; and developing the three-dimensional model of the nose of the user for predicting the fit of the head mounted wearable device based on the repeatedly capturing of the image data by the computing device from the plurality of different positions and orientations and the depth data determined from the repeatedly capturing of the image data. [0025] In some aspects, the techniques described herein relate to a computer- implemented method, wherein predicting, by the simulation engine accessible to the computing device, the fit of the head mounted wearable device includes: generating the three-dimensional model of the nose of the user; extracting at least one measurement from the three-dimensional model of the head of the user; and selecting a head mounted wearable device, from a plurality of available head mounted wearable devices, based on the at least one measurement.

[0026] In some aspects, the techniques described herein relate to a computer- implemented method, wherein the at least one measurement includes at least one of: a nose width based on distance between two fixed facial features detected in the current image data and the previous image data; or a nose slope determined based on nose height and a nose depth, the nose height being based on a distance between two fixed facial features detected in the current image data and the previous image data, and the nose depth being based on a distance between two fixed facial features detected in the current image data and the previous image data.

[0027] In some aspects, the techniques described herein relate to a system, including: a computing device, including: an image sensor; at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to: capture current image data, the current image data including ahead of a user; detect at least one fixed feature in the current image data; capture previous image data, the previous image data including the head of the user; detect the at least one fixed feature in the previous image data; detect a change in a position and an orientation of the computing device, from a previous position and a previous orientation corresponding to the capture of the previous image data, to a current position and a current orientation corresponding to the capture of the current image data; detect a change in a position of the at least one fixed feature between the current image data and the previous image data; correlate the change in the position and the orientation of the computing device with the change in the position of the at least one fixed feature; generate a three-dimensional model of the head of the user based on depth data extracted from the change in position and orientation of the computing device correlated with the change in position and orientation of the at least one fixed feature; and predict a fit of a head mounted wearable device on the head of the user based on the three-dimensional model of the head of the user.

[0028] In some aspects, the techniques described herein relate to a system, wherein the instructions cause the at least one processor to: generate the three- dimensional model of the head of the user; extract at least one measurement from the three-dimensional model of the head of the user; and select a head mounted wearable device, from a plurality of available head mounted wearable devices, based on the at least one measurement, the at least one measurement including at least one of: a cranial measurement determined based on distance between two fixed facial features detected in the current image data and the previous image data; or an ophthalmic measurement determined based on a distance between two optical features detected in the current image data and the previous image data.

[0029] In some aspects, the techniques described herein relate to a system, wherein the at least one fixed feature includes a plurality of fixed features, including: at least one facial landmark defined by at least two fixed facial features; and at least one fixed element defined by at least two fixed key points detected in a background area surrounding the head of the user.

[0030] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG. 1A illustrates an example system.

[0032] FIG. IB illustrates an example wearable device worn by a user and an example computing device held by a user.

[0033] FIG. 2A is a front view of one of the example wearable devices shown in FIG. 1A.

[0034] FIG. 2B is a rear view of the example wearable device shown in FIG. 2A.

[0035] FIG. 2C is a front view of an example handheld computing device shown in FIGs. 1A and IB.

[0036] FIGs. 2D-2F illustrate example ophthalmic fit measurements associated with the example w earable device shown in FIGs. 2A and 2B.

[0037] FIG. 3 is a block diagram of a system, in accordance w ith implementations described herein.

[0038] FIG. 4A illustrates an example computing device in an image capture mode. [0039] FIG. 4B illustrates an example display portion of the example computing device shown in FIG. 4A.

[0040] FIGs. 5A-5F illustrate example image data capture using an example computing device.

[0041] FIGs. 6A-6D illustrate example image data capture using an example computing device.

[0042] FIGs. 7A-7D illustrate example three-dimensional mesh models of a face and/or head of a user from two-dimensional image data.

[0043] FIG. 8 illustrates an example fitting image.

[0044] FIG. 9 is a flowchart of an example method.

[0045] FIG. 10A illustrates use of an example computing device in an image capture mode.

[0046] FIG. 10B illustrates an example display portion of the example computing device shown in FIG. 10A.

[0047] FIGs. 10C and 10D illustrate example landmarks detectable in image data captured by the example computing device shown in FIG. 10 A.

[0048] FIGs. 11 A-l 1 J illustrate example image data capture using an example computing device.

[0049] FIGs. 12A-12D illustrate example three-dimensional mesh models of a nose of a user generated from two-dimensional image data.

[0050] FIG. 13 illustrates an example fitting image.

[0051] FIGs. 14A-14C are front view s of example frame configurations including example pads.

[0052] FIGs. 15A-15D illustrate the fitting of the example frame configurations shown in FIGs. 14A-14C.

[0053] FIG. 16 is a flowchart of an example method.

[0054] FIG. 17 is a flowchart of an example method.

DETAILED DESCRIPTION

[0055] This disclosure relates to systems and methods for predicting fit of a wearable device for a user, based on image data captured by an image sensor of a computing device. Systems and methods, in accordance with implementations described herein, provide for the development of a depth map, and a three- dimensional mesh model, of a portion of the user on which the wearable device is to be worn. Systems and methods, in accordance with implementations described herein, provide for the development of a depth map and/or a three-dimensional mesh/three-dimensional model, from images captured by the image sensor of the computing device in which the image sensor does not include a depth sensor. In some implementations, the image sensor may be a front facing camera of a mobile device such as a smart phone or a tablet computing device. In some implementations, the depth map and/or the three-dimensional mesh/model may be developed from the images captured by the image sensor of the computing device. In some implementations, the depth map and/or the three-dimensional mesh/model may be developed from the images captured by the image sensor of the computing device combined with data provided by an inertial measurement unit (IMU) of the computing device.

[0056] In some implementations, fixed landmarks may be detected in a series or sequence of frames of image data captured by the image sensor of the computing device. The depth map and/or the three-dimensional mesh/model may be developed based on locations of the fixed landmarks in the series frames of image data captured by the image sensor of the computing device, alone or together with data provided by the IMU of the computing device. Development of a depth map and/or a three- dimensional mesh in this manner may allow for sizing and/or fitting of a wearable device for the user based on images captured by the user, without the need for specialized equipment and/or without assistance from a technician and/or without access to a retail establishment for the sizing and/or fitting of the wearable device. [0057] In some examples, the sizing and/or fitting of a head mounted wearable device, such as, for example, glasses, including smart glasses, may be accomplished based on the detection of fixed facial landmarks in the image data that define a nose of the user, such that a configuration of the nose of the user may be used to determine sizing and/or fitting of the head mounted wearable device. In some examples, a three- dimensional mesh, or model, of the nose, or nose area, of the user, based on one or more depth maps of the nose/nose area, may be provided to a simulator module or engine, to provide for the sizing and/or fitting of the head mounted wearable device. In some examples, the sizing and/or fitting of the head mounted wearable device may be driven by one or more characteristics associated with the nose of the user including, for example, a width at one or more portions of the nose, a slope of the nose, and the like. [0058] In some examples, the one or more characteristics may include a width of the nose at a bridge portion of the nose, at a root end thereof where a bridge portion of the head mounted wearable device would be seated when worn by the user. In some examples, the one or more characteristics may include a slope of the nose along the nasal ridge, or dorsum, extending from a root end portion, or sellion, to a tip end portion of the nose. In some examples, the one or more characteristics may include other measures including, for example, a width at an intermediate portion of the nose, a width at the ala of the nose, a slope along opposite sides of the nose, and other such measures and/or characteristics. Sizing and/or fitting of a head mounted wearable device based on detection of one or more characteristics associated with the nose/nose area of the user based on image data captured in this manner may provide for relatively accurate sizing and/or fitting of the head mounted wearable device without the use of specialized equipment and/or physical and/or virtual proctoring, and the like. Accuracy in sizing and/or fitting of the head mounted wearable device may become particularly important in head mounted wearable devices incorporating display capability, corrective lenses, and the like.

[0059] Hereinafter, systems and methods, in accordance with implementations described herein, will be described with respect to images captured by a handheld computing device for the fitting of ahead mounted wearable device, such as, for example, glasses, including smart glasses having display capability and computing capability, simply for purposes of discussion and illustration. The principles to be described herein may be applied to the sizing and/or fitting of a wearable device from images captured by an image sensor of a computing device operated by a user, for use in a variety of other scenarios including, for example, the sizing and/or fitting of other types of wearable devices (including devices having display and/or computing capabilities), the sizing and/or fitting of apparel items, and the like, which may make use of the front facing camera of the computing device operated by the user. In some situations, the principles to be described herein may be applied to other types of scenarios such as, for example, the accommodation of furnishings in a space, and the like.

[0060] The selection of wearable devices, such as head mounted wearable devices in the form of eyewear, or glasses, may rely on the determination of the physical fit, or wearable fit, to ensure that the eyewear is comfortable when worn by the user and/or is aesthetically complementary to the user. The incorporation of corrective lenses into the head mounted wearable device may rely on the determination of ophthalmic fit, to ensure that the head mounted wearable device can provide the desired vision correction. In the case of a head mounted wearable device including computing capability, for example, in the form of smart glasses including computing/processing capability and display capability⁷, selection may also rely on the determination of a display fit, to ensure that visual content is visible to the user. Existing systems for procurement of these ty pes of wearable devices do not provide for accurate fitting and customization, particularly without access to a retail establishment and/or specialized equipment and/or the assistance of a technician. That is, accurate sizing and/or fitting often relies on the user having access to a retail establishment, where samples are available for physical try on, and an optician is available to facilitate the determination of wearable fit and/or ophthalmic fit and/or aesthetic fit based on physical try-on and measurements collected using specialized equipment. In some situations, existing virtual systems that provide for online selection of a wearable device, such as eyewear, or glasses, simply superimpose an image of a selected frame on an image of the user, with only limited regard to actual physical sizing and fit of the selected frame for the user. Accuracy in sizing and fitting is particularly important in providing for the proper functionality⁷ of head mounted wearable devices including display capability and/or corrective lenses. The virtual placement of the image of the selected frame on the image of the user does not take into account user facial features which may affect the fit of the physical frames on the user. For example, variation in nose bridge height and/or nose bridge width and/or and nose slope may affect how a physical frame physically fits, and is physically positioned on the face/head of the user, affecting fit and function of the head mounted wearable device when worn by the user. Thus, these types of systems can yield inaccurate results in the selection of eyewear.

[0061] Hereinafter, systems and methods will be described with respect to the selection, sizing and/or fitting of a head mounted wearable device, simply for purposes of discussion and illustration. The principles to be described herein can be applied to the sizing and fitting of other types of wearable devices including, for example, glasses that may or may not include processing/computing/display capability⁷ and/or corrective lenses. Hereinafter, systems and methods will be described with respect to the selection, sizing and/or fitting of a head mounted wearable device based on the detection of facial landmarks defining features associated with the nose of the user, for the development of a depth map and/or a three-dimensional mesh/model of the nose, to be used in the selection and sizing/fitting of the head mounted display device for the user. The principles to be described herein may make use of other features, instead of, or in addition to, the features associated with the nose as detailed herein.

[0062] FIG. 1A is a third person view of a user in an ambient environment 10, with one or more external computing systems 11 accessible to the user via a network 12. FIG. 1A illustrates numerous different wearable devices that are operable by the user, including a first wearable device 100 in the form of glasses worn on the head of the user, a second wearable device 190 in the form of ear buds worn in one or both ears of the user, a third wearable device 195 in the form of a watch worn on the wrist of the user, and a handheld computing device 200 held by the user. In some examples, the first wearable device 100 is in the form of a pair of smart glasses including, for example, a display, one or more images sensors that can capture images of the ambient environment, audio input/output devices, user input capability. computing/processing capability and the like. In some examples, the second wearable device 190 is in the form of an ear worn computing device such as headphones, or earbuds, that can include audio input/output capability, an image sensor that can capture images of the ambient environment, computing/processing capability, user input capability and the like. In some examples, the third wearable device 195 is in the form of a smart watch or smart band that includes, for example, a display, an image sensor that can capture images of the ambient environment, audio input/output capability, computing/processing capability, user input capability and the like. In some examples, the handheld computing device 200 can include a display, one or more image sensors that can capture images of the ambient environment, audio input/output capability, computing/processing capability, user input capability, and the like, such as in a smartphone. In some examples, the example wearable devices 100, 190. 195 and the example handheld computing device 200 can communicate with each other and/or with the external computing system(s) 11 to exchange information, to receive and transmit input and/or output, and the like. The principles to be described herein may be applied to other types of wearable devices not specifically shown in FIG. 1 A.

[0063] Hereinafter, systems and methods will be described with respect to the sizing and/or fitting of a wearable device, such as, for example, one of the wearable devices 100, 190, 195 shown in FIG. 1 A, from images captured by one or more image sensors of the example handheld computing device 200 operated by the user, for purposes of discussion and illustration. Principles to be described herein may be applied to images captured by other types of computing devices. Principles to be described herein may be applied to the sizing and/or fitting of other types of wearable devices, with or without display capability, and with or without computing capability. Hereinafter, systems and methods will be described with respect to the sizing and/or fitting or a wearable device from images of the face/head of the user, together with position and/or acceleration data provided by the computing device 200, for example, for the fitting of a head mounted w earable device, simply for purposes of discussion and illustration. Principles to be described herein may be similarly used for sizing and/or fitting from images captured by a computing device, together with position/accel eration data provided by the computing device, for other purposes such as, for example, the sizing and/or fitting of other types of wearable devices including apparel, the insertion of augmented reality items into an augmented reality scene and/or a real world scene, and the like.

[0064] In some situations, a user may choose to use a computing device (such as the example handheld computing device 200 shown in FIG. 2C, or another computing device) for the virtual selection, sizing and fitting of a wearable device, such as the example first wearable device 100 in the form of glasses described above. For example, a user may use an application executing on the example computing device 200 to select glasses for virtual try on, and for the virtual sizing and fitting of selected glasses. In order to provide for the virtual sizing and/or fitting of a wearable device such as the example glasses, the user may use an image sensor of the example computing device 200 to capture images, for example a series of images, of the face/head of the user. In some examples, the images may be captured by the image sensor via an application executing on the computing device 200. In some examples, fixed features, or landmarks, may be detected within the series of images captured by the image sensor of the computing device 200. In some examples, position and/or orientation data provided by a sensor of the computing device 200 may be combined with the detection of landmarks and/or fixed features in the series of images. The combination of the detected landmarks and/or features in the series of images together with the position and/or orientation data associated with the computing device 200 as the series of images are captured, may allow- a depth map to be developed without the use of specialized equipment such as, for example a depth sensor in operation as the images are captured. A three-dimensional mesh, for example, of the face/head of the user, may be developed from the depth data collected in this manner, as the series of images is captured, and the detected landmarks and/or features in the series of images is combined with the position and/or orientation data associated with the computing device 200 as the series of images are captured. The resulting three-dimensional mesh may be processed, for example by a sizing simulator, to predict sizing and/or fitting of the wearable device, such as the example first wearable device 100 in the form of glasses. The ability to accurately predict fit in this manner may simplify the process associated with the fitting of a wearable device such as, for example the wearable device 100 in the form of glasses as described above, making such wearable device more easily accessible to a wide variety of users.

[0065] FIG. IB illustrates a user wearing the example first wearable device 100 in the form of smart glasses, or augmented reality glasses, including display capability, eye/gaze tracking capability, and computing/processing capability, with a computing device 200. in the form of a handheld computing device, such as a smart phone, held by the user. FIG. 2A is a front view, and FIG. 2B is a rear view, of the example first wearable device 100 shown in FIGs. 1A and IB. FIG. 2C is a front view of the example computing device 200 shown in FIGs. 1 A and IB.

[0066] The example head mounted wearable device 100 includes a frame 110 having rim portions 123 surrounding glass portions, or lenses 127 defining a front frame portion 120 of the frame 110. Arm portions 130 are coupled to the front frame portion 120 respective hinge portions 140. In some examples, the lenses 127 may be corrective/prescription lenses. In some examples, the lenses 127 may be an optical material including glass and/or plastic portions that do not necessarily incorporate corrective/prescription parameters. A bridge portion 129 may connect the rim portions 123 of the frame 110. The bridge portion 129 may be seated on the bridge portion of the nose of the user, proximate the root end of the nose, or sellion. In the example shown in FIGs. 2A and 2B, the wearable device 100 is in the form of a pair of smart glasses, or augmented reality' glasses, simply for purposes of discussion and illustration. The principles to be described herein can be applied to the sizing and/or fitting of a head mounted wearable device in the form of glasses that do not include the functionality typically associated with smart glasses. The principles to be described herein can be applied to the sizing and/or fitting of a head mounted wearable device in the form of glasses (including smart glasses, or eyewear that does not include the functionality typically associated with smart glasses) that include corrective/prescription lenses.

[0067] In some examples, the wearable device 100 includes a display device 104 that can output visual content, for example, at an output coupler 105, so that the visual content is visible to the user. In the example shown in FIGs. 2A and 2B, the display device 104 is provided in one of the two arm portions 130, simply for purposes of discussion and illustration. Display devices 104 may be provided in each of the two arm portions 130 to provide for binocular output of content. In some examples, the display device 104 may be a see through near eye display. In some examples, the display device 104 may be configured to project light from a display source onto a portion of teleprompter glass functioning as a beamsplitter seated at an angle (e.g., 30-45 degrees). The beamsplitter may allow- for reflection and transmission values that allow the light from the display source to be partially reflected while the remaining light is transmitted through. Such an optic design may allow a user to see both physical items in the world, for example, through the lenses 127, next to content (for example, digital images, user interface elements, virtual content, and the like) output by the display device 104. In some implementations, waveguide optics may be used to depict content on the display device 104.

[0068] The example wearable device 100. in the form of smart glasses as shown in FIGs. 2A and 2B, includes one or more of an audio output device 106 (such as, for example, one or more speakers), an illumination device 108, a sensing system 111, a control system 112, at least one processor 114, and an outward facing image sensor 116 (for example, a camera). In some examples, the sensing system 111 may include various sensing devices and the control system 112 may include various control system devices including, for example, the at least one processor 114 operably coupled to the components of the control system 112. In some examples, the control system 112 may include a communication module providing for communication and exchange of information between the wearable device 100 and other external devices. [0069] In some examples, the head mounted wearable device 100 includes a gaze tracking device 115 to detect and track eye gaze direction and movement. Data captured by the gaze tracking device 115 may be processed to detect and track gaze direction and movement as a user input. In the example shown in FIGs. 2A and 2B, the gaze tracking device 115 is provided in one of two arm portions 130, simply for purposes of discussion and illustration. In the example arrangement shown in FIGs. 2A and 2B. the gaze tracking device 115 is provided in the same arm portion 130 as the display device 104, so that user eye gaze can be tracked not only with respect to objects in the physical environment, but also with respect to the content output for display by the display device 104. In some examples, gaze tracking devices 115 maybe provided in each of the two arm portions 130 to provide for gaze tracking of each of the two eyes of the user. In some examples, display devices 104 may be provided in each of the two arm portions 130 to provide for binocular display of visual content. [0070] In some examples, the head mounted wearable device 100 can include pads 180 provided on the front frame portion 120 of the frame 110. In the example shown in FIGs. 2A and 2B, a first pad 180 is positioned on a first of the rim portions 123, at a position corresponding to where the rim portion 123 would rest on a first side of the nose of the user, and a second pad 180 is positioned on a second of the rim portions 123, at a position corresponding to where the rim portion 123 would rest on a second side of the nose of the user. In some examples, the pads 180 may provide for adjustment of a position of the frame 110 on the nose of the user, and/or may maintain a position of the frame 110 on the nose/relative to the eyes of the user. The adjustment of the position of the frame 110 and/or the maintaining of the position of the frame 110 provided by the pads 180 may maintain alignment of the eyes with the lenses 127 and/or with corrective features of the lenses 127. The adjustment of the position of the frame 110 and/or the maintaining of the position of the frame 1 10 provided by the pads 180 may help to position content output by the display device 104 within the field of view of the user, in a head mounted wearable device 100 that includes display capability. In some examples, the pads 180 may enhance user comfort. In some examples, the pads 180 may be removably coupled to the rim portions 123. This may allow the position of the frame 110 to be customized for a particular user, and/or provide for fine tuning of a position of the frame 110 for a particular user. For example, different pads 180, having different sizes and/or shapes and/or configurations, may be coupled onto the rim portions 123, to provide a type and/or level of adjustment in position of the frame 110 that best positions the frame 110 on the nose/face of a particular user.

[0071] The example wearable device 100 can include more, or fewer features than described above. The principles to be described herein are applicable to the virtual sizing and/or fitting of head mounted wearable devices including display capability and/or computing capability, i.e., smart glasses, and also to head mounted wearable devices that do not include display and/or computing capabilities, and to head mounted wearable devices with or without corrective lenses.

[0072] FIG. 2C is a front view of an example computing device, in the form of the example handheld computing device 200 shown in FIGs. 1A and IB. The example computing device 200 may include an interface device 210. In some implementations, the interface device 210 may function as an input device, including, for example, a touch surface 212 that can receive touch inputs from the user. In some implementations, the interface device 210 may function as an output device, including, for example, a display portion 214 allowing the interface device 210 to output information to the user. In some implementations, the interface device 210 can function as an input device and an output device. The example computing device 200 may include an audio output device 216, or speaker, that outputs audio signals to the user.

[0073] The example computing device 200 may include a sensing system 220 including various sensing system devices. In some examples, the sensing system devices include, for example, one or more image sensors, one or more position and/or orientation sensors, one or more audio sensors, one or more touch input sensors, and other such sensors. The example computing device 200 shown in FIG. 2C includes an image sensor 222. In the example shown in FIG. 2C, the image sensor 222 is a front facing camera. The example computing device 200 may include additional image sensors such as, for example, a world facing camera. The example computing device 200 shown in FIG. 2C includes an inertial measurement unit (IMU) 224 including, for example, one or more position sensors and/or orientation sensors and/or acceleration sensors such as, for example, an accelerometer, a gyroscope, a magnetometer, and other such sensors that can provide position and/or orientation and/or acceleration data. The example computing device 200 shown in FIG. 2C includes an audio sensor 226 that can detect audio signals, for example, for processing as user inputs. The example computing device 200 shown in FIG. 2C includes a touch sensor 228. for example corresponding to the touch surface 212 of the interface device 210. The touch sensor 228 can detect touch input signals for processing as user inputs. The example computing device 200 may include a control system 270 including various control system devices. The example computing device 200 may include a processor 290 to facilitate operation of the computing device 200. [0074] As noted above, a computing device such as the example handheld computing device 200 may be used to capture images of the user. The images may be used, together with position data and/or orientation data of the example handheld computing device 200, to develop one or more depth map(s) from which a three- dimensional mesh may be developed. The three-dimensional mesh may be provided to, for example, a sizing and/or fitting simulator for the virtual sizing and/or fitting of a wearable device such as the example head mounted wearable device 100 described above. This may allow the user to use the computing device 200 for the virtual selection and sizing/fitting of the wearable device 100, such as the glasses described above, without the use of specialized equipment, without a proctored virtual fitting, without access to a retail establishment, and the like.

[0075] Numerous different sizing and fitting measurements and/or parameters may be taken into account when selecting and/or sizing and/or fitting a wearable device, such as the example head mounted wearable device 100 shown in FIGs. 1A- 2B. for a particular user. This may include, for example, wearable fit parameters, or wearable fit measurements. Wearable fit parameters/measurements may take into account how a particular frame 110 fits and/or looks and/or feels on a particular user. Wearable fit parameters/measurements may take into consideration numerous factors such as, for example, whether the rim portions 123 and bridge portion 129 are shaped and/or sized so that the bridge portion 129 rests comfortably on the bridge of the nose of the user, whether the frame 110 is wide enough to be comfortable with respect to the temples, but not so wide that the frame 110 cannot remain relatively stationary when worn by the user, whether the arm portions 130 are sized to comfortably rest on the ears of the user, and other such comfort related considerations. Wearable fit parameters/measurements may take into account other as-wom considerations including how the frame 110 may be positioned based on the natural head pose of the user, i.e., where the user tends to naturally w ear glasses. In some examples, aesthetic fit measurements or parameters may be taken into account, such as whether the frame 110 is aesthetically pleasing to the user/compatible with the user’s facial features, and the like.

[0076] In a head mounted wearable device including display capability, display fit parameters, or display fit measurements may be taken into account in selecting and/or sizing and/or fitting the head mounted wearable device 100 for a particular user. Display fit parameters/measurements may be used to configure the display device 104 for a selected frame 110 for a particular user, so that content output by the display device 104 is visible to the user. For example, display fit parameters/measurements may facilitate calibration of the display device 104, so that visual content is output within at least a set portion of the field of view of the user. For example, the display fit parameters/measurements may be used to configure the display device 104 to provide at least a set level of gazability, corresponding to an amount, or portion, or percentage of the visual content that is visible to the user at a periphery (for example, a least visible comer) of the field of view of the user.

[0077] In an example in which the head mounted wearable device 100 is to include corrective lenses, ophthalmic fit parameters, or ophthalmic fit measurements may be taken into account in the selecting and/or sizing and/or fitting process. Some example ophthalmic fit measurements are shown in FIGs. 2D-2F. Ophthalmic fit measurements may include, for example, a pupil height PH, which may represent a distance from a center of the pupil to a bottom of the respective lens 127. Ophthalmic fit measurements may include an interpupillary distance IPD, which may represent a distance between the pupils. IPD may be characterized by a monocular pupil distance, for example, a left pupil distance LPD representing a distance from a central portion of the bridge of the nose to the left pupil, and a right pupil distance RPD representing a distance from the central portion of the bridge of nose to right pupil. Ophthalmic fit measurements may include a pantoscopic angle PA. representing an angle defined by the tilt of the lens 127 with respect to a vertical plane. Ophthalmic fit measurements may include a vertex distance V representing a distance from the cornea to the respective lens 127. Ophthalmic fit measurements may include other such parameters, or measures that provide for the selecting and/or sizing and/or fitting of a head mounted wearable device 100 including corrective lenses, with or without a display device 104 as described above. In some examples, ophthalmic fit measurements, together with display fit measurements, may provide for the output of visual content by the display device 104 within a defined three-dimensional volume such that content is within a corrected field of view of the user, and thus visible to the user.

[0078] FIG. 3 is a block diagram of an example system for sizing and/or fitting of a w earable device from images captured by a computing device. Wearable devices to be sized and/or fitted in this manner can include the various example wearable computing devices described above, as well as other types of wearable devices such as clothing, accessories and the like. [0079] The system may include a computing device 300. The computing device 300 can access additional resources 302 to facilitate the sizing and/or fitting of a wearable device. In some examples, the additional resources may be available locally on the computing device 300. In some examples, the additional resources may be available to the computing device 300 via a network 306. In some examples, some of the additional resources 302 may be available locally on the computing device 300, and some of the additional resources 302 may be available to the computing device 300 via the network 3O6.The additional resources 302 may include, for example, server computer systems, processors, databases, memory storage, and the like. In some examples, the processor(s) may include object recognition engine(s) and/or module(s), pattern recognition engine(s) and/or module(s), configuration identification engine(s) and/or modules(s), simulation engine(s) and/or module(s), sizing/fitting engine(s) and/or module(s), and other such processors.

[0080] The computing device 300 can operate under the control of a control system 370. The computing device 300 can communicate with one or more external devices 304, either directly (via wired and/or wireless communication), or via the network 306. In some examples, the one or more external devices may include another wearable computing device, another mobile computing device, and the like. In some implementations, the computing device 300 includes a communication module 380 to facilitate external communication. In some implementations, the computing device 300 includes a sensing system 320 including various sensing system components. The sensing system components may include, for example one or more image sensors 322, one or more position/ orientation sensor(s) 324 (including for example, an inertial measurement unit, an accelerometer, a gyroscope, a magnetometer and other such sensors), one or more audio sensors 326 that can detect audio input, one or more touch input sensors 328 that can detect touch inputs, and other such sensors. The computing device 300 can include more, or fewer, sensing devices and/or combinations of sensing devices.

[0081] In some implementations, the one or more image sensor(s) 322 may include, for example, cameras such as, for example, one or more forward facing cameras, one or more outward, or world facing, cameras, and the like. The one or more image sensor(s) 322 can capture still and/or moving images of an environment outside of the computing device 300. The still and/or moving images may be displayed by a display device of an output system 340, and/or transmitted externally via a communication module 380 and the network 306, and/or stored in a memory 330 of the computing device 300. The computing device 300 may include one or more processor(s) 390. The processors 390 may include various modules or engines configured to perform various functions. In some examples, the processor(s) 390 may include object recognition engine(s) and/or module(s), pattern recognition engine(s) and/or module(s), configuration identification engine(s) and/or modules(s), simulation engine(s) and/or module(s), sizing/fitting engine(s) and/or module(s). and other such processors. The processor(s) 390 may be formed in a substrate configured to execute one or more machine executable instructions or pieces of software, firmware, or a combination thereof. The processor(s) 390 can be semiconductor-based including semiconductor material that can perform digital logic. The memory 330 may include any type of storage device that stores information in a format that can be read and/or executed by the processor(s) 390. The memory 330 may store applications and modules that, when executed by the processor(s) 390, perform certain operations. In some examples, the applications and modules may be stored in an external storage device and loaded into the memory 330.

[0082] FIG. 4A illustrates the use of a computing device, such as the example handheld computing device 200 show n in FIG. 2C, to capture images for the virtual selection and/or fitting of a wearable device such as the example head mounted wearable device 100 shown in FIGs. 1A-2B. In particular, FIG. 4A illustrates the use of a computing device to capture images, using a front facing camera of the computing device, for use in the virtual selection and/or sizing and/or fitting of a wearable device. As noted above, the principles described herein can be applied to the use of other types of computing devices and/or to the selection and/or sizing and/or fitting of other types of wearable devices.

[0083] In the example shown in FIG. 4A, the user is holding the example handheld computing device 200 so that the head and face of the user is in the field of view of the image sensor 222 of the computing device 200. In particular, the head and face of the user is in the field of view of the image sensor 222 of the front facing camera of the computing device 200, so that the image sensor 222 can capture images of the head and face of the user. In some examples, images captured by the image sensor 222 are displayed to the user on the display portion 214 of the computing device 200. so that the user can verify the initial positioning of the head and face of the user within the field of view of the image sensor 222. FIG. 4B illustrates an example image frame 400 captured by the image sensor 222 of the computing device 200 during an image data capture process using the computing device 200 operated by the user as shown in FIG. 4A. The image data captured by the image sensor 222 may be processed, for example, by resources available to the computing device 200 as described above (for example, the additional resources 302 described above with respect to FIG. 3) for the virtual selection and/or sizing and/or fitting of a wearable device. In some examples, the capture of images and the accessing of the additional resources may be performed via an application executing on the computing device 200.

[0084] S\ stems and methods, in accordance with implementations described herein, may detect one or more features, or landmarks, or key points, within image data represented by a series of images, or image frames, captured in this manner. One or more algorithms may be applied to combine the one or more features and/or landmarks and/or key points, with position and/or orientation data provided bysensors such as, for example, position and/or orientation sensors included in the IMU 224 of the computing device 200, as the series of images is captured.

[0085] As shown in FIG. 4B, the image data captured by the image sensor 222 may be processed, for example, by a recognition engine of the additional resources 302, to detect and/or identify various fixed features and/or landmarks and/or key points in the image data/series of image frames captured by the image sensor 222. In the example shown in FIG. 4B, various example facial landmarks have been identified in the example image frame 400. In some examples, the example facial landmarks may represent facial landmarks that remain substantially fixed, even in the event of changes in facial expression and the like. In the example shown in FIG. 4B, the example facial landmarks include a first landmark 41 OR representing an outer comer of the right eye, and a second landmark 410L representing an outer comer of the left eye. A distance between the first landmark 410R and the second landmark 410L may represent an inter-lateral commissure distance (ILCD). The first landmark 41 OR and the second landmark 41 OR, from which the measure for ILCD are taken, may remain relatively fixed, or relatively stable, regardless of eye gaze direction, facial expression, head orientation and the like, across a series of image frames captured by the image sensor 222.

[0086] In the example shown in FIG. 4B, the example facial landmarks include a third landmark 420R representing an inner comer of the right eye, and a fourth landmark 420L representing an inner comer of the left eye. A distance between the third landmark 420R and the fourth landmark 420L may represent an inter-medial commissure distance (IMCD). The third landmark 420R and the fourth landmark 420R, from which the measurement for IMCD are taken, may remain relatively fixed, or relatively stable, regardless of eye gaze direction, facial expression, head orientation and the like, across a series of image frames captured by the image sensor 222.

[0087] In the example shown in FIG. 4B, the example facial landmarks include a fifth landmark 430R representing a pupil center of the right eye, and a sixth landmark 430L representing a pupil center of the left eye. A distance between the fifth landmark 430R and the sixth landmark 430L may represent an inter-pupillary distance (IPD). The fifth landmark 430R and the sixth landmark 430L, from which the measurement for IPD are taken, may remain relatively fixed, or relatively stable, in a situation in which user gaze is focused on a point in the distance as the series of images are captured by the image sensor 222.

[0088] In the example shown in FIG. 4B, the example facial landmarks include a seventh landmark 405L representing an ear saddle point of the right ear, and an eighth landmark 405R representing an ear saddle point of the left ear. A distance between the seventh landmark 405L and the eighth landmark 405R may be representative of a head width HW, or a width of the user's head at a portion of the head at which the head mounted wearable device 100 (for example, in the form of glasses) would be worn. The seventh landmark 405L and the eighth landmark 405R, from which the measure for head width HW are taken, may remain relatively fixed, or relatively stable, regardless of facial expression, head orientation and the like, across a series of image frames captured by the image sensor 222.

[0089] In the example shown in FIG. 4B, the example facial landmarks include a ninth landmark 415 A representing a nose bridge, or a sellion, and a tenth landmark 415B representing a nose tip. A distance between the ninth landmark 415A and the tenth landmark 415B may be representative of a nose length NL. In some examples, the ninth landmark 415 A, representing the nose bridge, or sellion, may correspond to a portion of the nose at which the bridge portion 129 of the example head mounted wearable device 100 would be seated on the nose of the user. The ninth landmark 415A and the tenth landmark 415B. from which the measure for nose length NL are taken, may remain relatively fixed, or relatively stable, regardless of facial expression, head orientation and the like, across a series of image frames captured by the image sensor 222.

[0090] In the example shown in FIG. 4B, the example facial landmarks include an eleventh landmark 425R representing a right most point of the nose, or nostril, and a twelfth landmark 425L representing a left most point of the nose, or nostril. A distance between the eleventh landmark 425R and the twelfth landmark 425L may be representative of a nose width NW. In some examples, the eleventh landmark 425R and the twelfth landmark 425L, from which the measure for nose width NW are taken, may remain relatively fixed, or relatively stable, across a series of image frames captured by the image sensor 222.

[0091] In the example shown in FIG. 4B, example fixed, or static features, or landmarks, or key points, or elements 440 are identified in a background 450 of the image data captured by the image sensor 222. In the example shown in FIG. 4B, the fixed, or static features, or landmarks, or key points, or elements 440 represent relatively clearly defined and/or clearly identifiable features, for example, clearly defined geometric features such as comers, intersections and the like, that remain fixed, or stable, across the series of image frames captured by the image sensor 222. [0092] FIGs. 5A-5F illustrate the use of a computing device, such as the example handheld computing device 200 show n in FIG. 2C, to capture image data. In particular, FIGs. 5A-5F illustrate a first series of movements of the example handheld computing device 200 to capture image data including a first series of image frames, capturing a first series of perspectives of the face and/or head of the user for use in predicting virtual sizing and/or fitting of a w earable device such as the example head mounted wearable device 100 shown in FIGs. 1, 2A and 2B.

[0093] In the example shown in FIG. 5A, the user has initiated the capture of image data for example, via an application executing on the example handheld computing device 200. In FIG. 5A, the computing device 200 is positioned so that the head and face of the user is captured within the field of view of the image sensor 222. In the example shown in FIG. 5A. the image sensor 222 is included in the front facing camera of the computing device 200, and the head and face of the user are captured within the field of view' of the front facing camera of the computing device 200. In the initial position shown in FIG. 5A, the computing device 200 is positioned substantially straight out from the head and face of the user, somewhat horizontally and vertically aligned with the head and face of the user, simply for purposes of discussion and illustration. The capture of image data by the image sensor 222 of the computing device 200 can be initiated at other positions of the computing device 200 relative to the head and face of the user.

[0094] In the positions show n in FIGs. 5B and 5C, the user has moved, for example, sequentially moved, the computing device 200 in the direction of the arrow Al. In the positions shown in FIGs. 5B and 5C, the head and face of the user remain in substantially the same position as shown in FIG. 5A. As the computing device 200 is moved from the position shown in FIG. 5A to the position shown in FIG. 5B and then to the position shown in FIG. 5C, the image sensor 222 captures, for example, sequentially captures, image data of the head and face of the user from the different positions and/or orientations of the computing device 200/image sensor 222 relative to the head and face of the user. FIGs. 5B and 5C show just two example image frames captured by the image sensor 222 as the user moves the computing device 200 in the direction of the arrow Al, while the head and face of the user remain substantially stationary. Any number of image frames may be captured by the image sensor 222 as the computing device 200 is moved in the direction of the arrow Al. Similarly, any number of image frames captured by the image sensor 222 may be analyzed and processed by the recognition engine to detect and/or identify the landmarks 405 and/or the landmarks 410 and/or the landmarks 415 and/or the landmarks 420 and/or the landmarks 425 and/or the landmarks 430 and/or the elements440 in the image frames captured as the computing device 200 is moved in this manner.

[0095] In FIGs. 5D-5F, the computing device 200 has been moved, for example, sequentially moved, in the direction of the arrow A2. In this example, as the computing device 200 is moved in the direction of the arrow A2, the head of the user remains in substantially the same position. As the computing device 200 is moved in the direction of the arrow' A2, the image sensor 222 captures image data of the head and face of the user from corresponding perspectives of the computing device 200/image sensor 222 relative to the head and face of the user. Thus, as the computing device 200 is moved in the direction of the arrow Al, and then in the direction of the arrow' A2, the image sensor 222 captures image data including the head and face of the user from the various different perspectives of the computing device 200/image sensor 222 relative to the head and face of the user. In this particular example, the position and/or orientation of head and face of the user remain substantially the same. FIGs. 5D-5F show just three example image frames captured by the image sensor 222 as the user moves the computing device 200 in the direction of the arrow A2. Any number of image frames may be captured by the image sensor 222 as the computing device 200 is moved in the direction of the arrow A2. Similarly, any number of image frames captured by the image sensor 222 may be analyzed and processed by the recognition engine to detect and/or identify the landmarks 405 and/or the landmarks 410 and/or the landmarks 415 and/or the landmarks 420 and/or the landmarks 425 and/or the landmarks 430 and/or the elements 440 in the image frames captured as the computing device 200 is moved in this manner.

[0096] FIGs. 6A-6D illustrate the use of a computing device, such as the example handheld computing device 200 shown in FIG. 2C, to capture image data. In particular, FIGs. 6A-6D illustrate a second series of movements of the example handheld computing device 200 to capture image data including a second series of image frames capturing a second series of perspectives of the face and/or head of the user. Image data captured in this manner may be used in predicting virtual sizing and/or fitting of a wearable device such as the example head mounted wearable device 100 shown in FIGs. 1, 2A and 2B.

[0097] In the example shown in FIG. 6A, the user has initiated the capture of image data for example, via an application executing on the example handheld computing device 200. In FIG. 6A, the computing device 200 is positioned so that the head and face of the user is captured within the field of view of the image sensor 222. In the example shown in FIG. 6A, the image sensor 222 is included in the front facing camera of the computing device 200, and the head and face of the user are captured within the field of view of the front facing camera of the computing device 200. In the initial position shown in FIG. 6A, the computing device 200 is positioned substantially straight out from the head and face of the user, somewhat horizontally and vertically aligned with the head and face of the user, simply for purposes of discussion and illustration. The capture of image data by the image sensor 222 of the computing device 200 can be initiated at other positions of the computing device 200 relative to the head and face of the user.

[0098] In the positions shown in 6B, the user has moved the computing device 200 in the direction of the arrow A3. In this example, movement of the computing device 200 in the direction of the arrow A3 positions the computing device 200 at the left side of the user, capturing a profile image, or a series of profile perspectives, of the head and face of the user. In the position shown in FIG 6B, the head and face of the user remain in substantially the same position as shown in FIG. 6A. simply for purposes of discussion and illustration. As the computing device 200 is moved from the position shown in FIG. 6A to the position shown in FIG. 6B, the image sensor 222 captures, for example, sequentially captures, image data of the head and face of the user from the different positions and/or orientations of the computing device 200/image sensor 222 relative to the head and face of the user. FIG. 6B illustrates just one example image frame captured by the image sensor 222 as the user moves the computing device 200 in the direction of the arrow A3, while the head and face of the user remain substantially stationary'. Any number of image frames may be captured by the image sensor 222 as the computing device 200 is moved in the direction of the arrow A3. Similarly, any number of image frames captured by the image sensor 222 may be analyzed and processed by the recognition engine to detect and/or identify the landmarks 405 and/or the landmarks 410 and/or the landmarks 415 and/or the landmarks 420 and/or the landmarks 425 and/or the landmarks 430 and/or the elements 440 in the image frames captured as the computing device 200 is moved in this manner.

[0099] In FIGs. 6C and 6D, the computing device 200 has been moved, for example, sequentially moved, in the direction of the arrow A4, from the position shown in FIG. 6B. In this example, as the computing device 200 is moved in the direction of the arrow A4, the head of the user remains in substantially the same position. In this example, movement of the computing device 200 in the direction of the arrow A4 positions the computing device 200 at the right side of the user, capturing a profile image, or a series of profile images, of the head and face of the user. As the computing device 200 is moved in the direction of the arrow A4, the image sensor 222 captures image data of the head and face of the user from corresponding perspectives of the computing device 200/image sensor 222 relative to the head and face of the user. Thus, as the computing device 200 is moved in the direction of the arrow A3, and then in the direction of the arrow A4, the image sensor 222 captures image data including the head and face of the user from the various different perspectives of the computing device 200/image sensor 222 relative to the head and face of the user. In this particular example, the position and/or orientation of head and face of the user remain substantially the same. Any number of image frames may be captured by the image sensor 222 as the computing device 200 is moved in the direction of the arrow A3 and the arrow A4, in addition to or instead of the example image frames shown in FIGs. 6A-6D. Similarly, any number of image frames captured by the image sensor 222 may be analyzed and processed by the recognition engine to detect and/or identify the landmarks 405 and/or the landmarks 410 and/or the landmarks 415 and/or the landmarks 420 and/or the landmarks 425 and/or the landmarks 430 and/or the elements 440 in the image frames captured as the computing device 200 is moved in this manner.

[00100] The image data captured by the image sensor 222 of the computing device 200 as the computing device 200 is moved as shown in FIGs. 5A-5F and/or as shown in FIGs. 6A-6D may be processed, for example, by a recognition engine accessible to the computing device 200 (for example, via the external computing systems 11 described above with respect to FIG. 1 A, or via the additional resources 302 described above with respect to FIG. 3). Landmarks and/or features and/or key points and/or elements may be detected in the image data captured by the image sensor 222 through the processing of the image data. In some examples, the detected landmarks and/or features and/or key points and/or elements may be substantially fixed, or substantially unchanging, or substantially constant. The example landmarks 405, 410, 415, 420, 425, 430 and the example elements 440, and measures associated therewith, illustrate just some example landmarks and/or elements that may be detected in the frames of image data captured by the image sensor 222.

[00101] As noted above, one example feature or measure may include the head width HW, between the seventh landmark 405R and the eighth landmark 405L representing a head width betw een the left and right ear saddle points. Another example feature or measure may include the ILCD, representing a distance between the outer comers of the eyes of the user. Another example feature or measure may include the IMCD, representing a distance between the inner comers of the eyes of the user. Another example feature or measure may include the nose length NL. Another example feature or measure may include the nose width NW. In some examples, the facial features or landmarks from which one or more of the HW. the NL, the NW, the ILCD and/or the IMCD are determined may remain substantially constant, even in the event of changes in facial expression, changes in gaze direction, intermittent blinking and the like. As noted above, IPD may remain substantially constant, provided a distance gaze is maintained. Other example landmarks or features may include various fixed elements 440 detected in the background 450, or the area surrounding the head and face of the user. In the example shown in FIGs. 5A- 5F and 6A-6D. the fixed elements 440 are geometric features detected in the background 450, or the area surrounding the user, simply for purposes of discussion and illustration. The fixed elements may include other types of elements detected in the background 450. For example, in FIGs. 5A-5F and 6A-6D, the fixed elements 440 are geometric features (lines, edges, comers and the like) detected in a repeating pattern in the background 450. and at the intersections between adjacent walls, at the intersections between the walls and the floor, at the intersections between the walls and the ceiling, and the like, simply for purposes of discussion and illustration. In some examples, other fixed elements, features and the like may be detected in the background, including, for example, features in a room such as windows, frames, furniture, and other elements having defined features that are detectable in the image data captured by the image sensor 222.

[00102] These elements having fixed contours and/or geometry in the area surrounding the head and face of the user that may be detected in the frames of image data captured by the image sensor 222. Detected features and/or landmarks, and changes in the frames of image data sequentially captured by the image sensor 222 as the computing device 200 is moved, can be correlated with position and/or orientation data provided by the position and/or orientation sensors included in the IMU 224 of the computing device 200 at positions corresponding to the capture of the image data. For example, changes in position and orientation data of detected features and/or landmarks between, for example, between a first position and/or orientation and a second position and/or orientation, can be compared to corresponding changes in position and/or orientation of the computing device.

[00103] In some examples, data provided by the position and/or orientation sensors included in the IMU 224, together with the processing and analysis of the image data, may be used to provide the user with feedback, to provide for improved collection of image data. In some examples, one or more prompts may be output to the user. These prompts may include, for example, a prompt indicating that the user repeat the image data collection sequence. These prompts may include, for example, a prompt providing further instruction as to the user’s motion of the computing device 200 during the image data collection sequence. These ty pes of prompts may provide for the collection of image data from a different perspective that may provide a more complete representation of the head and/or face of the user. These prompts may include, for example, a prompt indicating that a change in the ambient environment may produce improved results such as, for example, a change to include fixed features in the background 450, a change in illumination of the ambient environment, and the like. In some examples, the prompts may be visual prompts output on the display portion 214 of the computing device 200. In some examples, the prompts may be audible prompts output by the audio output device 216 of the computing device 200. [00104] Image data collected in this manner, and/or the fixed landmarks and/or fixed elements detected in the image data, and/or the features of measures associated with the fixed landmarks and/or fixed elements, combined with data provided by position and/or orientation sensors included in the IMU 224 of the computing device 200, may be processed by the one or more processors of the additional resources 302 accessible to the computing device 200 to predict fit of a wearable device, such as the example head mounted wearable device 100.

[00105] In particular, the fixed landmarks and/or fixed features detected in the image data and/or associated features and/or measures, combined with the position/ orientation data associated with the computing device 200, may be used to extract depth/develop a depth map. In this example, the fixed landmarks and/or fixed elements detected in the image data, combined with data provided by position and/or orientation sensors included in the IMU 224 of the computing device 200, may be processed by the one or more processors of the additional resources 302 accessible to the computing device 200 to develop one or more depth maps of the face and/or head of the user. In some examples, the depth map(s) may be processed by the one or more processors of the additional resources 302 to develop a three-dimensional mesh, or a three-dimensional model, of the face and/or head of the user. A simulation module, or a simulation engine, may process the three-dimensional mesh, or three-dimensional model, of the face/head of the user to fit the head mounted wearable device 100 on the three-dimensional mesh or model, and predict fit of the head mounted wearable device 100 on the user.

[00106] In some examples, a metric scale may be applied to determine one or more facial and/or cranial and/or ophthalmic measurements associated with the detected landmarks and/or features (for example, HW and/or NL and/or NW and/or IPD and/or IMCD and/or ILCD and/or HW and the like, as described in the example above, and/or other such measures). The determined one or more facial and/or cranial and/or ophthalmic measurements may be processed by, for example, a machine learning algorithm, to predict fit of the head mounted wearable device 100 on the user. In some examples, metric scale may be provided by. for example, an object having a known scale captured in the image data, by entry of scale parameters by the user, and the like. In some examples, in which metric scale is not otherwise provided, the data associated with the detected landmarks/features/elements and the position/ orientation data associated with the computing device 200 may be aggregated by algorithms executed by the one or more processors to determine scale. [00107] The image data captured in the manner described above, when processed by one or more fitting and/or sizing and/or simulation engines and/or modules, may provide for the prediction of fit of a wearable device, such as the head mounted wearable device 100 described above, using the computing device 200 operated by the user, without the use of specialized equipment such as a depth sensor, a pupilometer and the like, without the use of a reference object having a known scale, without access to a retail establishment, and without a proctor to supervise the capture of the image data and/or to capture the image data. Rather, the image data may be captured by the image sensor 222 of the computing device 200 operated by the user, and in particular, by the image sensor 222 included in the front facing camera of the computing device 200.

[00108] As noted above, in some examples, one or more depth maps of the face/head of the user may be generated based on a series of image frames including image data captured from different positions of the computing device 200 relative to the head and/or face of the user. The fixed landmarks and/or features and/or elements detected in the image data obtained in this manner may be tracked, and correlated with data provided by position and/or orientation sensors included in the IMU 224 of the computing device 200 to generate the one or more depth maps used to determine fit of the head mounted wearable device 100. In some examples, depth maps generated in this manner may be fused to generate a three-dimensional mesh, or a three-dimensional model, of the face/head of the user. In some examples, the fixed landmarks and/or features and/or elements detected in the image data obtained in this manner may be tracked, and correlated with data provided by position and/or orientation sensors included in the IMU 224 of the computing device 200, to determine metric scale (in a situation in which known scale is not otherwise provided). For example, changes in position and/or orientation of a detected feature or landmark can be compared to corresponding changes in position and/or orientation of the computing device 200, and correlated to determine metric scale.

[00109] In some examples, the frames of image data collected in this manner may be analyzed and processed, for example, by object and/or pattern recognition engines provided in the additional resources 302 accessible to the computing device 200, to detect the fixed landmarks and/or elements in the sequentially captured image frames. Data provided by the position and/or orientation sensors of the IMU 224 may be associated with the detected landmarks and/or elements in the sequential frames of image data. In some examples, changes in the measures associated with the fixed landmarks and/or elements, from image frame to image frame as the position and/or orientation of the computing device relative to the head/face of the user is changed and the sequential image frames are captured, may be associated with the data provisioned by the position and/or orientation sensors of the IMU 224.

[00110] This combined data may be aggregated, for example, by one or more algorithms applied by a data aggregating engine of the additional resources 302, to develop a one or more associated depth maps. In some examples, the depth map(s) may be fused to generate the three-dimensional mesh of the face/head of the user. In an example in which metric scale is not otherwise provided, the data aggregating engine may aggregate this data to associate changes in pixel distance (based on analysis of the sequential frames of image data) with changes in position/orientation data of the computing device 200 to generate an estimate of metric scale.

[00111] For example, a head width HW1 (based on the fixed facial landmarks 405R, 405L), a nose length NL1 (based on the fixed facial landmarks 415A, 415B), a nose width NW1 (based on the fixed landmarks 425R, 425L), an ILCD1 (based on the fixed facial landmarks 41 OR, 410L). and an IMCD1 (based on the fixed facial landmarks 420R, 420L), is associated with the first position shown in FIG. 5 A. Similarly, a particular position is associated with each of the detected fixed elements 440 in the background 450, and relative positions of the plurality of fixed elements 440 in the background. This is represented in FIG. A, simply for illustrative purposes, by a distance DI 1 between a first pair of the fixed elements 440, a distance D12 between a second pair of the fixed elements 440, a distance D13 between a third pair of the fixed elements 440, a distance D14 betw een a fourth pair of the fixed elements 440, and a distance D15 between a fifth pair of the fixed elements 440. A first position and a first orientation may be associated with the computing device 200, corresponding to the first position shown in FIG. 5A, based on data provided by the IMU 224. The position and orientation of the computing device 200 at the first position shown in FIG. 5A may in turn be associated with the facial landmarks 405R, 405L and the associated HW1, the facial landmarks 410R, 410L and the associated ILCD1, the facial landmarks 415A, 415B and the associated NL1, the facial landmarks 420R, 420L and the associated IMCD1, the facial landmarks 425R, 425L and associated NW1. and with the plurality of fixed elements 440 and the associated distances Dll, D12, D13, D13 and D15.

[00112] As the computing device is moved from the first position shown in FIG. 5A to the second position shown in FIG. 5B, a second position and a second orientation of the computing device 200 are associated with the computing device 200 based on data provided by the IMU 224. A motion stereo baseline can be determined based on the first position and first orientation, and the second position and second orientation of the computing device 200, together with the changes in position and/or orientation of the fixed landmarks and/or elements and associated measures. As the computing device 200 is moved relative to the head and face of the user from the first position shown in FIG. 5A to the second position shown in FIG. 5B, the image data captured by the image sensor 222 changes, so that the respective positions of the landmarks 405R, 405L, 410R, 410L, 415A, 415B, 420R, 420L, 425R, 425L and elements 440 change within the image frame. This in turn causes a change from the HW1, ILCD1, NL1, IMCD1, and NWl shown in FIG. 5A to the HW2. ILCD2, NL2, NW2, ILMD2, and NW2 shown in FIG. 5B. Similarly, this causes a change in the example distances associated with the example pairs of elements 440, from DI 1, D12, D13, D14 and DI 5 shown in FIG. 5 A, to D21, D22, D23, D24 and D25 shown in FIG. 5B.

[00113] In FIG. 5B, the relative second positions of the landmarks 405R, 405L, 410R, 410L, 415A, 415B, 420R, 420L, 425R, 425L and elements 440 (and corresponding distances HW2, ILCD2, NL2, NW2, ILMD2, NW2, D21, D22, D23, D24 and D25) can be correlated with the corresponding movement of the computing device 200 from the first position and first onentation to the second position and second orientation. That is, the known change in position and orientation of the computing device 200, from the first position/orientation to the second position/ orientation, may be correlated with a known amount of linear rotation (for example, based on gyroscope data from the IMU 224) and linear acceleration (for example, from accelerometer data from the IMU 224). Thus, the detected change in position of the landmarks 405R, 405L. 410R, 410L, 415A, 415B, 420R, 420L, 425R, 425L and elements 440 (and corresponding distances HW2. ILCD2, NL2, NW2, ILMD2, NW2, D21, D22, D23, D24 and D25) may be determined, using the detected known change in position and orientation of the computing device 200 together with an associated scale value. This data may provide a first reference source for the development of a depth map for the corresponding portion of the head/face of the user captured in the corresponding image frames.

[00114] Additional data may be obtained as the user continues to move the computing device 200 further in the direction of the arrow Al, i.e., substantially vertically in this example, from the second position and second orientation shown in FIG. 5B to the third position and third orientation shown in FIG. 5C, while the head remains substantially still. As the computing device 200 is moved relative to the head and face of the user from the second position shown in FIG. 5B to the third position shown in FIG. 5C, image data captured by the image sensor 222 changes, so that the respective positions of the landmarks 405R, 405L, 410R, 410L. 415A, 415B. 420R, 420L, 425R, 425L and elements 440 change within the image frame. This in turn causes a change from the HW2, ILCD2, NL2, IMCD2, and NW2 shown in FIG. 5B to the HW3, ILCD3, NL3, IMCD3, and NW3shown in FIG. 5C. Similarly, this causes a change in the example distances associated with the example pairs of elements 440, from D21, D22, D23, D24 and D25 shown in FIG. 5B to D31, D32, D33. D34 and D35 shown in FIG. 5C.

[00115] The relative third positions of the landmarks 405R, 405L, 410R, 410L,

415A, 415B, 420R, 420L, 425R, 425L and elements 440 (and corresponding distances HW3, ILCD3, NL3, IMCD3, NW3, D31, D32, D33, D34 and D35) can be correlated with the corresponding movement of the computing device 200. That is, the known change in position and orientation of the computing device 200, from the second position/ orientation to the third position/orientation, based on a known amount of linear rotation (for example, based on gyroscope data from the IMU 224) and linear acceleration (for example, from accelerometer data from the IMU) may provide another reference source for the development of depth map(s) for corresponding portion(s) of the head/face of the user (as well as a reference source for scale, if scale is not otherwise provided and is to be determined). The detected change in position of the landmarks 405R. 405L, 410R, 410L. 415A, 415B. 420R, 420L, 425R, 425L and elements 440 (and corresponding distances HW3, ILCD3, NL3, IMCD3, NW3, D31, D32, D33, D34 and D35) may be determined, using the detected known change in position and orientation of the computing device 200, as a baseline for the development of a second depth map for the corresponding portion of the head/face of the user captured in the corresponding image frames.

[00116] Data may continue to be obtained as the user continues to move the computing device 200. In this example, the user changes direction, and moves the computing device 200 in the direction of the arrow A2, as shown in FIGs. 5D, 5E and 5F, substantially vertically in this particular example, from the third position and third orientation shown in FIG. 5C to an example fourth position/orientation shown in FIG. 5D. an example fifth position/orientation shown in FIG. 5E. and an example sixth position/orientation shown in FIG. 5F. while the head remains substantially still. As the computing device 200 is moved relative to the head and face of the user as shown in FIGs. 5D-5F, image data captured by the image sensor 222 changes, so that the respective positions of the landmarks 405R, 405L, 410R, 410L, 415A, 415B, 420R, 420L, 425R, 425L and elements 440 change within the image frame. This in turn causes a sequential change from the HW3. ILCD3, NL3, IMCD3. NW3 shown in FIG. 5C, to the HW4/ILCD4/NL4/IMCD4/NW4, HW5/ILCD5/NL5/IMCD5/NW5, and HW6/ILCD6/NL6/IMCD6/NW6 shown in FIGs. 5D-5F, respectively. Similarly, this causes a sequential change in the example distances associated with the example pairs of elements 440, from D31, D32, D33. D34 and D35 shown in FIG. 5C, to D41/D42/D43/D44/D45 shown in FIG 5D, to D51/D52/D53/D54/D55 shown in FIG. 5E, and to D61/D62/D63/D64/D65 shown in FIG. 5F.

[00117] The relative positions of the landmarks 405R, 405L, 41 OR, 410L, 415A, 415B, 420R, 420L, 425R. 425L and elements 440 (and corresponding distances) can again be correlated with the corresponding movement of the computing device 200, with known positions and orientations of the computing device 200 as the computing device 200 is moved as shown, based on a known amount of linear rotation (for example, based on gyroscope data from the IMU 224) and linear acceleration (for example, from accelerometer data from the IMU. The detected changes in positions of the landmarks 405R, 405L, 410R, 410L, 415A, 415B, 420R, 420L, 425R, 425L and elements 440, and corresponding distances, as the computing device 200 is moved in the direction of the arrow A2 as shown In FIGs. 5D-5F, may be determined, using the detected known changes in position and orientation of the computing device 200. This data may, again, be processed by the one or more processors, to develop one or more depth maps corresponding to portions of the face/head of the user captured in the image data of the associated image frames. [00118] As shown in FIGs. 6A-6D, the user may continue to collect image data from which one or more additional depth maps may be developed, to facilitate the development of a three-dimensional mesh, or a three-dimensional model, of the face/head of the user, for the prediction of fit of the head mounted wearable device 100.

[00119] For example, as shown in FIG. 6A, as the user initiates the continued collection of image data, a head w idth HW7 (based on the fixed facial landmarks 405R, 405L). a nose length NL7 (based on the fixed facial landmarks 415 A, 415B), a nose width NW7 (based on the fixed landmarks 425R, 425L). an ILCD7 (based on the fixed facial landmarks 41 OR, 410L), and an IMCD7 (based on the fixed facial landmarks 420R, 420L), is associated with the position shown in FIG. 6A. Similarly, a particular position is associated with each of the detected fixed elements 440 in the background 450, and relative positions of the plurality of fixed elements 440 in the background. This is represented in FIG. 6A, simply for illustrative purposes, by a distance D71 between the first pair of the fixed elements 440, a distance D72 between the second pair of the fixed elements 440, a distance D73 betw een the third pair of the fixed elements 440, a distance D74 between the fourth pair of the fixed elements 440, and a distance D75 between the fifth pair of the fixed elements 440. A position and orientation may be associated with the computing device 200, corresponding to the position shown in FIG. 6A, based on data provided by the IMU 224. The position and orientation of the computing device 200 at the position shown in FIG. 6A may in turn be associated with the facial landmarks 405R, 405L and the associated HW7. the facial landmarks 41 OR, 410L and the associated ILCD7, the facial landmarks 415 A, 415B and the associated NL7, the facial landmarks 420R, 420L and the associated IMCD7, the facial landmarks 425R, 425L and associated nose width NW7, and with the plurality of fixed elements 440 captured in the background 450 and the associated distances D71, D72, D73, D73 and D75.

[00120] As the computing device is moved in the direction of the arrow A3, from the seventh position shown in FIG. 6A to an eighth position shown in FIG. 6B, an eighth position and orientation of the computing device 200 are associated with the computing device 200 based on data provided by the IMU 224. As the computing device 200 is moved relative to the head and face of the user from the seventh position shown in FIG. 6A to the eighth position shown in FIG. 6B. the image data captured by the image sensor 222 changes, so that the respective positions of the fixed facial landmarks and fixed elements in the background 450 change within the image frame. In this example, some of the fixed facial features, and fixed elements in the background, that were visible/ detectable in the seventh position shown in FIG. 6 A, are no longer visible/detectable in the eighth position shown in FIG. 6B, due to the change in position of the computing device 200 relative to the face/head of the user. In this example, based on the detectable facial features and/or elements, a nose length NL8 is determined (based on the detection of the facial landmarks 415A, 415B), and D81 and D83 are determined (based on the detection of the corresponding fixed elements 440 in the background 450). The change in position and/or orientation of the computing device 200 relative to the face/head of the user in turn causes a change from the NL7 shown in FIG. 6A to the NL8 shown in FIG. 6B. Similarly, this causes a change in the example distances associated with the example pairs of elements 440, from the distances D71 and D73 shown in FIG. 6A to the distances D81 and D83 shown in FIG. 6B.

[00121] The relative change in measures and/or distances, i.e., the change from the NL7, D71, and D73 shown in FIG. 6A to the NL8, D81 and D83 shown in FIG. 6B, can be correlated with the corresponding movement of the computing device 200 from the seventh position and orientation to the eighth position and second orientation. That is, the known change in position and orientation of the computing device 200, from the seventh position/orientation to the eighth position/orientation, may be correlated with a known amount of linear rotation (for example, based on gyroscope data from the IMU 224) and linear acceleration (for example, from accelerometer data from the IMU 224). Thus, the detected change in position of the landmarks 415A, 415B and elements 440 (and corresponding distances NL7/NL8, D71/D81, and D73/D83) may be determined, using the detected known change in position and orientation of the computing device 200 together with an associated scale value. This data may provide an additional source for the development of a depth map for the corresponding portion of the head/face of the user captured in the corresponding image frames.

[00122] Additional data may be obtained as the user moves the computing device 200 in the direction of the arrow A4, from the eighth position and orientation shown in FIG. 6B to a ninth position and orientation shown in FIG. 6C and a tenth position and orientation shown in FIG. 6D, while the head remains substantially still. As the computing device 200 is moved relative to the head and face of the user from the eighth position shown in FIG. 6B to the ninth and tenths positions shown in FIGs. 6C and 6D, image data captured by the image sensor 222 changes, so that the respective positions of the landmarks 405R, 405L, 410R, 410L, 415A, 415B, 420R, 420L, 425R, 425L and elements 440 change within the image frame.

[00123] This in turn causes a sequential change in relative positions of the fixed facial landmarks (and corresponding measures) detected in the image data of the respective image frames, and of the fixed elements 440 (and corresponding distances) detected in the background 450 in the image data of the respective image frames. This includes, for example, a change from the nose length NL7 shown in FIG. 6A, to the nose length NL8 shown in FIG. 6B, to a nose length NL9 shown in FIG. 6C, and a nose length NL10 shown in FIG. 6D. Similarly, this includes, for example, a change from the distances D71, D72, D73, D74, and D75 shown in FIG. 6A, to distances D81 and D83 shown in FIG. 6B, to distances D91, D92, D93, D94, and D95 in FIG. 6C, to distances D120 and D104 in FIG. 6D.

[00124] Detection of the fixed landmarks 415A, 415B and associated nose length NL (i.e., NL7, NL8, NL9, NL10), from the image data captured in the sequential image frames shown in FIGs. 6A-6D, may be correlated with corresponding position/orientation data associated with the computing device 200 as it is moved to capture the sequential image frames as shown. Similarly, detection of the fixed landmarks 405R, 405L, 41 OR, 410L, 420R, 420L and associated head width HW, ILCD, and IMCD (i.e., HW7, ILCD7, and IMCD7 at the seventh position shown in FIG. 6A. and HW9, ILCD9. and IMCD9 at the ninth position shown in FIG. 6C), may be correlated with corresponding position/orientation data associated with the computing device 200 at the seventh and ninth positions shown in FIGs. 6 A and 6C. Detection of the fixed elements 440 and associated distances may be similarly correlated with the corresponding position/orientation data associated with the computing device 200 at the respective positions at which the fixed elements associated with the distances are detected. For example, detection of the fixed elements 440 in the background 450 of the image data collected as the computing device 200 and the sequential image frames are captured as shown in FIGs. 6A-6D, may be correlated with the corresponding position/orientation data associated with the computing device 200, to detect changes in distances D71/D81/D91, D72/D92/D120, D73/D83/D93, D74/D94/D104, and D75/D95. Thus, the detected changes in positions of the landmarks 405R, 405L, 410R. 410L, 415A. 415B, 420R, 420L, 425R, 425L and elements 440, and corresponding distances, as the computing device 200 is moved as shown, may be determined, using the detected known changes in position and orientation of the computing device 200. This data may, again, be processed by the one or more processors, to develop one or more depth maps corresponding to portions of the face/head of the user captured in the image data of the associated image frames.

[00125] The examples shown in FIGs. 5A-5F and 6A-6D describe ten example data collection points, simply for ease of discussion and illustration. In some examples, image data and position and orientation data may be obtained at more, or fewer, points as the computing device 200 is moved. In some examples, image data and position and orientation data may be substantially continuously obtained, with corresponding depth data being substantially continuously determined.

[00126] Depth data, detected in this manner, may be aggregated, for example, by a data aggregating engine and associated algorithms available via the additional resources 302 accessible to the computing device 200. The image data, and the associated position and orientation data, may continue to be collected until the aggregated data determined in this manner provides a relatively complete data set for the development of a three-dimensional mesh/three-dimensional model of the face and/or head of the user.

[00127] Similarly, in a situation in which metric scale is not otherwise provided, this motion stereo approach may be applied to the determination of scale. Depth data, detected as described above based on comparison of fixed landmarks and/or features and/or elements in sequentially collected image data, combined with position and/or orientation data associated with the computing device 200 as the image data is collected, may be aggregated by, for example, a data aggregating engine and associated algorithms, until the aggregated data produces scale values that coalesce to provide a relatively robust, reliable determination of metric scale.

[00128] FIGs. 5A-5F and 6A-6D provide just one example of a manner in which the image data may be captured. In particular, FIGs. 5A-5F and 6A-6D provide just one example of how the image data may be captured by a user operating the computing device 200. without the need for specialized equipment and/or proctoring and/or a physical or virtual appointment with a technician for assistance. Other t pes of computing devices may be used to obtain the image data, operated in manners other than described in the above example(s).

[00129] As noted above, the one or more depth maps may be generated from the image data representing the face and/or head of the user from various different perspectives/various different positions and/or orientations of the computing device 200 relative to the face and/or head of the user. In some examples, the depth maps may be fused, or stitched together, to develop a three-dimensional mesh, representative of a three-dimensional model, of the face and/or head of the user. FIG. 7A illustrates a perspective view of an example three-dimensional mesh 700 of a face and head of a user. The example three-dimensional mesh 700 may be generated based on a series of depth maps, developed from two-dimensional image data in a series of image frames as described above, that have been stitched or fused together to generate the three-dimensional mesh 700. FIG. 7B illustrates a portion of the three-dimensional mesh 700, superimposed on the face/head of the user, including the identification of some of the example fixed facial landmarks 405R, 405L, 410R. 410L, 415A, 415B, 420R, 420L, 425R. 425L, 430R. 430L.

[00130] In some examples, the three-dimensional mesh 700, or three- dimensional model, may be provided to a simulation engine or a simulation module, to predict a fit of the wearable device (i. e.. the head mounted wearable device 100) for the user. In some examples, various metric measurements, including for example, facial and/or cranial and/or ophthalmic measurements, may be extracted from the three-dimensional model for processing in predicting fit. In some examples, these measurements may include one or more of the example head width HW, nose length NL, nose width NW. IPD, IMCD, ILCD, and/or other such measurements that can be derived based on the application of a known or determined metric scale to various fixed facial/cranial/ophthalmic landmarks. In some examples, the various measurements may be used to predict various aspects of fit associated with the head mounted wearable device 100. In some examples, the processing of the three- dimensional mesh 700 or model may predict a wearable fit. representative of how the head mounted wearable device 100 will physically fit on the face/head of the user and be worn by the user. In a situation in which the head mounted wearable device 100 is to include corrective or prescription lenses, this processing and fitting prediction may take into account ophthalmic fit. In a situation in which the head mounted wearable device 100 is to include display capability, this processing and fitting prediction may take into account display fit, so that content output by a display device of the head mounted wearable device 100 is visible to the user.

[00131] In some examples, one or more facial and/or cranial and/or ophthalmic measurements may be extracted, for example, from the three-dimensional mesh 700, to predict sizing and/or fitting of the head mounted wearable device 100 for the user based on the image data obtained as described above. In some examples, the three- dimensional mesh 700 and/or extracted facial and/or cranial and/or ophthalmic measurements may be provided to a sizing and/or fitting simulator, or simulation engine, or simulation module. In some examples, the sizing and/or fitting simulator may access a database of available head mounted wearable devices and apply a machine learnings model to select one or more head mounted wearable devices, from the available head mounted wearable devices, that are predicted to fit the user based on the three-dimensional mesh 700 and/or the extracted facial/cranial and/or ophthalmic measurements. FIG. 7C illustrates one example head mounted wearable device 750. of a plurality of head mounted wearable devices which may be considered by the simulator and/or the machine learning model, positioned on the three- dimensional mesh 700 of the face/head of the user. FIG. 7D illustrates the one example head mounted wearable device 750 positioned on the three-dimensional mesh 700, with the three-dimensional mesh 700 superimposed on the face of the user. In some examples, the simulator implementing the machine learning model may access a fit database including fit data for each of the plurality of available head mounted wearable devices. Fit scores, accumulated across a relatively large pool of users, may be accessed to provide an indication and prediction of fit for the user, based on one or more of the measurements extracted from the three-dimensional mesh 700. The database accessed by the machine learning model may include, for example, a distribution of scoring frequency for each of the plurality of available head mounted wearable devices for a range of head widths, a range of nose widths, a range of nose lengths, a range of ILCDs and/or IMCDs, and the like. These scores may be taken into consideration by the machine learning model in predicting fit for a head mounted wearable device for the user.

[00132] In some examples, the one or more head mounted wearable devices, predicted by the simulator implementing the machine learning model to be a fit for the user, may be presented to the user, for virtual try on, comparison, and the like prior to purchase. In some examples, the simulator implementing the machine learning model may predict whether a head mounted wearable that has already been selected by the user will fit the user. In some examples, the simulator may provide a fitting image 800 to the user, as shown in FIG. 8. The fitting image 800 may provide a visual indication during the virtual try on, representative of how a selected head mounted wearable device 850 will look on the face and/or head of the user.

[00133] Systems and methods, in accordance with implementations described herein, may provide a prediction of fit of the head mounted wearable device 100 for the user based on image data, obtained by the user operating the computing device 200, combined with position and/or orientation data provided by one or more sensors of the computing device 200. In the examples described above, image data of the head and face of the user is obtained by the image sensor 222 of a front facing camera of the computing device 200. In some situations, the collection of image data in this manner may pose challenges due to, for example, the relative proximity' between the image sensor 222 of the front facing camera and the head/face of the user, inherent, natural movement of the head and face of the user as the computing device 200 is moved, combined with the need for accuracy in the fitting of head mounted wearable devices. The use of static key points, or elements, or features, in the background that anchor the captured image data as the computing device 200 is moved and sequential frames of image data are captured, may increase the accuracy of the depth data derived from the image data and position/orientation data, and the subsequent three- dimensional mesh, and the fitting of the head mounted wearable device fitted based on the three-dimensional mesh and/or extracted facial/cranial/ophthalmic measurements. The collection of multiple frames of image data including the fixed facial landmarks and the static key points or features or elements in the background, and the combining of the image data with corresponding position/orientation data associated with the computing device 200 as the series of frames of image data is collected, may improve the level of accuracy in prediction of fit of the head mounted wearable device.

[00134] In the examples described above, the movement of the computing device 200 is in a substantially vertical direction, in front of the user, in a substantially horizontal direction, across the front and to the left and right side profiles of the user, while the head and face of the user remain substantially still, or static. The image data obtained through the example movement of the computing device 200 as shown in FIGs. 5A-5E and 6A-6D may provide for the relatively clear and detectable capture of the fixed facial landmarks and/or static key points/fixed elements in the background from the changing perspective of the computing device relative to the head/face of the user as the computing device 200 is moved. In some examples, systems and methods, in accordance with implementations described herein, may be accomplished using other movements of the computing device 200 relative to the user.

[00135] Systems and methods, in accordance with implementations described herein, have been presented with respect to the prediction of fit for a head mounted wearable device, simply for purposes of discussion and illustration. The principles described herein may be applied to the prediction of fit for other types of wearable devices. Similarly, systems and methods, in accordance with implementations described herein, have been presented using head width HW and/or nose length NL and/or nose width NW and/or ILCD and/or IMCD as example fixed facial measures, simply for purposes of discussion and illustration. Other facial and/or cranial and/or ophthalmic landmarks from which other facial and/or cranial and/or ophthalmic features and/or measurements may be detected may also be applied, alone, or together with these landmarks and associated measurements, to accomplish the disclosed prediction of fit.

[00136] Systems and methods, in accordance with implementations described herein, provide for the prediction of fit of a wearable device from image data and position/orientation data using a client computing device. In some implementations, systems and methods, in accordance with implementations described herein, provide for the determination of scale from the image data and position/orientation data obtained using the client computing device. Systems and methods, in accordance with implementations described herein, may provide for the prediction of fit from image data and position/orientation data without the use of a known reference object. Systems and methods, in accordance with implementations described herein, may predict fit from image data and position/orientation data without the use of specialized equipment such as. for example, depth sensors, pupilometers and the like that may not be readily available to the user. Systems and methods, in accordance with implementations described herein, may predict from image data and position/orientation data without the need for a proctored virtual fitting and/or access to a physical retail establishment. Systems and methods, in accordance with implementations described herein, may improve accessibility to the virtual selection and accurate fitting of wearable devices. The prediction of fit in this manner provides for a virtual try on of an actual wearable device to determine wearable fit and/or ophthalmic fit and/or display fit of the wearable device.

[00137] FIG. 9 is a flowchart of an example method 900 of predicting fit from image data and position/orientation data. A user operating a computing device (such as, for example, the computing device 200 described above) may initiate image capture functionality of the computing device (block 910). In some examples, the image capture functionality may be operable within an application executing on the computing device. Initiation of the image capture functionality may cause an image sensor (such as, for example, the image sensor 222 of the front facing camera of the computing device 200 described above) to capture first image data including a face and/or a head of the user (block 915). At least one fixed feature may be detected within the first image data (block 920). The at least one fixed feature may include fixed facial features and/or landmarks that remain substantially static, and/or fixed or static key points or features in a background area surrounding the head/face of the user in the first image data. A first position and orientation of the computing device may be detected (block 925) based on. for example, data provided by position/orientation sensors of the computing device at a point corresponding to capture of the first image data.

[00138] Continued operation of the image capture functionality may cause the computing device to incrementally capture second image data including the face and/or a head of the user and the at least one fixed feature (block 930, block 935), until the image capture functionality is terminated. In some examples, the image capture functionality may be terminated when it is determined, for example, within the application executing on the computing device, that a sufficient amount of image data has been captured for the development of a three-dimensional mesh/three- dimensional model of the face and/or head of the user for the purposes of predicting fit of a head mounted wearable device. Changes in the position and the orientation of the computing device may be correlated with changes in position of the at least one fixed feature detected in a current frame of image data compared to the position of the at least one fixed feature detected in a previous frame of image data (block 940). Depth data may be extracted based on the comparison of the current image frame of data to the previous image frame of data, and the respective position of the at least one fixed feature (block 945). At least one depth map of the face and/or head of the user may be generated based on the depth data extracted from the correlation of the position/orientation data of the computing device with the changes of position in the at least one fixed feature detected in the frames of image data (block 950). The depth maps may be fused, or stitched, together to develop a three-dimensional mesh, or a three-dimensional model, of the face and/or head of the user (block 955). The three- dimensional mesh, and/or measurements extracted therefrom, may be processed by a machine learning model, to predict fit of a head mounted wearable device for the user (block 960).

[00139] FIG. 10A illustrates the use of a computing device, such as the example handheld computing device 200 shown in FIG. 2C, to capture images for the virtual selection and/or sizing and/or fitting of a wearable device, such as the example head mounted wearable device 100 shown in FIGs. 1A-2B. In particular, FIG. 10A illustrates the use of a computing device to capture images, using a front facing camera of the computing device, for use in the virtual selection and/or sizing and/or fitting of a wearable device. As noted above, the principles described herein can be applied to the use of a handheld computing device such as the example handheld computing device 200 shown in FIG. 2C, as well as other types of computing devices and/or to the selection and/or sizing and/or fitting of a wearable device such as the head mounted wearable device 100 shown in FIGs. 1A-2B, as w ell as other ty pes of wearable devices.

[00140] In the example shown in FIG. 10 A, the user is holding the example handheld computing device 200 so that the head and face of the user is in the field of view of the image sensor 222 of the computing device 200. In particular, the head and face of the user is in the field of view' of the image sensor 222 of the front facing camera of the computing device 200, so that the image sensor 222 can capture images of the head and face of the user. In some examples, images captured by the image sensor 222 are displayed to the user on the display portion 214 of the computing device 200. This may allow' the user to verify the initial positioning of the head and face of the user within the field of view of the image sensor 222. FIG. 10B illustrates an example image frame 1000 captured by the image sensor 222 of the computing device 200 during an image data capture process using the computing device 200 operated by the user. The image data captured by the image sensor 222 may be processed, for example, by resources available to the computing device 200 as described above (for example, the additional resources 302 described above with respect to FIG. 3) for the virtual selection and/or sizing and/or fitting of a wearable device. In some examples, the capture of image data and the accessing of the additional resources 302 may be performed via an application executing on the computing device 200.

[00141] S\ 'stems and methods, in accordance with implementations described herein, may detect one or more features, or landmarks, or key points, within image data represented by a series of frames of image data captured in this manner. Depth data may be extracted from the series of frames of image data, to develop one or more corresponding depth maps, from which a three-dimensional mesh, or model, may be generated. In some examples, the detection of the one or more features, or landmarks, or key points in the series of frames of image data may be combined with position and/or orientation data provided by sensors such as. for example, position and/or orientation sensors included in the IMU 224 of the computing device 200, as the series of frames of image data is captured.

[00142] The image data captured by the image sensor 222 may be processed, for example, by a recognition engine of the additional resources 302, to detect and/or identify various fixed features and/or landmarks and/or key points in the image data/series of image frames captured by the image sensor 222. In the example shown in FIG. 10B, various example facial landmarks have been identified in the example image frame 1000. In the example shown in FIG. 10B, the example facial landmarks 1070 include a landmark 1070A and a landmark 1070B, corresponding to detected temple portions of the face of the user. The example facial landmarks 1070 include a landmark 1070C and a landmark 1070D corresponding to detected cheek portions of the face of the user. A landmark 1070E and a landmark 1070F correspond to outer comers of the eyes of the user. A landmark 1070G corresponds to a detected chin portion of the user.

[00143] As noted above, in some examples, the sizing and/or fitting of the head mounted wearable device 100 may be accomplished based on characteristics and/or measurements of the nose of the user. In particular, a configuration of the nose, including for example, a size, a shape, and the like of the nose, may be used to predict sizing and/or fitting of the head mounted wearable device for a particular user. In some examples, one or more characteristics of the configuration of the nose of the user may provide a relatively reliable basis for the sizing and/or fitting of the head mounted wearable device 100. Accordingly, as shown in FIG. 10B. one or more facial landmarks associated with the nose of the user may be detected in the image data. For example, a landmark 1010, corresponding to a sellion, or root end portion 1015, of the nose of the user may be detected in the image data. A landmark 1020, corresponding to a tip end portion of the nose of the user, may be detected in the image data. In some examples, landmarks 1030R and 1030L, corresponding to right and left portions of the lower end portion of the nose, defining the ala, may be detected in the image data. In some examples, landmarks 1040R and 1040L, corresponding to left and right bounds of the bridge of the nose at the root end portion 1015 of the nose, may be detected in the image data. In some examples, a distance between the landmarks 1040R and 1040L may define a width at the bridge of the nose.

[00144] Hereinafter, systems and methods, in accordance with implementations described herein, will be described with respect to the use of characteristics associated with the nose of the user, and the development of the three-dimensional mesh, or model of the nose of the user, from image data captured by the user operating a computing device such as the example handheld computing device 200. Thus, the description to follow will focus on the use of the example facial landmarks 1010 and/or 1020 and/or 1030R/1030L and/or 1040R/1040L. simply for purposes of discussion and illustration. The principles described herein can be applied to the use of more, or fewer facial landmarks associated with the nose of the user, instead of or in addition to the example facial landmarks 1010 and/or 1020 and/or 1030R/1030L and/or 1040R/1040L and/or different combinations thereof. Further, the principles described herein can be applied to the use of more, or fewer facial landmarks, and/or different combinations of facial landmarks, for the sizing and/or fitting of the head mounted wearable device 100.

[00145] FIGs. 10C and 10D illustrate the identification of characteristics, and/or measurements, associated with the nose of the user that can be determined based on landmarks identified within the image data captured by the computing device 200 operated by the user.

[00146] In some examples, a width W of the nose may be determined, for use in the sizing and/or fitting of the head mounted wearable device 100. In some examples, the width W may represent a width at a portion of the nose at which the bridge portion 129 of the head mounted wearable device 100 is seated when worn by the user. In some examples, the determination of the width W may be based on the detection of one or more facial landmarks in the image data captured by the computing device operated by the user. For example, the width W may be a width of the nose at the landmark 1010, corresponding to the sellion, at the root end portion 1015 of the nose. In some examples, the width W may represent a distance between the landmark 1040R (corresponding to the right outer portion of the nose, at the root end portion 1015 of the nose, or bridge portion of the nose) and the landmark 1040L (corresponding to the left outer portion of the nose, at the root end portion 1015 of the nose, or bridge portion of the nose) at the root end portion 1015 of the nose. In some examples, the width W of the nose, taken at the root end portion 1015 of the nose as shown, may provide a basis for the sizing and/or fitting of the head mounted wearable device 100. In some examples, the width W of the nose, and in particular the width W of the nose at the root end portion 1015, where the bridge portion 129 of the head mounted wearable device 100 would be seated when worn by the user, provides a relative accurate basis for the sizing and/or fitting of the head mounted wearable device 100. In some examples, the width W of the nose, for example, the bridge width of the nose, taken at the root end portion of the nose, may provide an initial basis and/or a singular basis for the sizing and/or fitting of the head mounted wearable device 100.

[00147] In some examples, a slope of the nose, for example, a slope of the nose along the dorsum, or nasal ridge 1025, may be determined, for use in the sizing and/or fitting of the head mounted wearable device 100. In some examples, the slope of the nose along the dorsum, or nasal ridge 1025, may provide an indication of where the bridge portion 129 of a particular frame 110 of a head mounted wearable device would be seated along the nasal ridge 1025 and/or would remain seated. In some examples, the slope of the nose along the nasal ridge 1025 may provide an indication of whether or not a particular frame 110 would remain seated at the root end portion 1015 of the nose on a particular user. In some examples, the slope S of the nose, along the nasal ridge 1025 may be determined by dividing a height H of the nose by a depth D of the nose. As show n in FIG. 10D, in some examples, the height D of the nose may correspond to a distance between the landmark 1010 at the root end portion 1015 of the nose, and the landmark 1030 (i.e.. 1030R. 1030L) at lower end portion of the nose, defining the ala.

[00148] As noted above, in some examples, image data, for example, a series of frames of image data, may be captured by the computing device 200 operated by the user, for example, via an application executing on the computing device 200. The image data may be processed, for example, by an object and/or pattern recognition engine and/or module accessible to the computing device 200, to detect the various facial landmarks and associated measurements described above. In some examples, the detection of these landmarks may be combined with movement data, such as position and/or orientation and/or acceleration data provided by the IMU 224 of the computing device 200 as the sequential frames of image data are captured, to apply scale and determine distances between the respective landmarks, such as the width W, height H. depth D, and slope S described above. One or more depth maps may be generated as the sequential frames of image data are captured, and a three- dimensional mesh may be generated from the one or more depth maps, by, for example, modeling engine or module accessible to the computing device 200. In the examples described herein, one or more depth map(s) and a three-dimensional mesh or model of the nose/nose area of the user may be generated, for the sizing and/or fitting of the head mounted wearable device 100. The three-dimensional mesh or model may be provided to a simulation module and/or engine, for the sizing and/or fitting of the head mounted wearable device 100.

[00149] FIGs. 11A-11J illustrate the use of a computing device, such as the example handheld computing device 200 shown in FIG. 2C, to capture image data. In particular, FIGs. 11A-11J illustrate a series of movements of the example handheld computing device 200 to capture image data including a series of sequentially- captured frames of image data, including a plurality of different perspectives of the face and/or head of the user, including in particular a portion of the face/head of the user including the nose. Image data captured as illustrated in the example shown in FIGs. 11A-11J may be used in predicting virtual sizing and/or fitting of a wearable device such as the example head mounted wearable device 100 shown in FIGs. 1 A- 2B.

[00150] In the example shown in FIG. 11 A, the user has initiated the capture of image data for example, via an application executing on the example handheld computing device 200. In FIG. 11 A, the computing device 200 is positioned so that the head and face of the user is captured within the field of view of the image sensor 222. In the example shown in FIG. 11 A, the image sensor 222 is included in the front facing camera of the computing device 200, and the head and face of the user are captured within the field of view of the front facing camera of the computing device 200. In the initial position shown in FIG. 11 A, the computing device 200 is positioned substantially straight out from the head and face of the user, somew hat horizontally and vertically aligned with the head and face of the user, simply for purposes of discussion and illustration. The capture of image data by the image sensor 222 of the computing device 200 can be initiated at other positions of the computing device 200 relative to the head and face of the user.

[00151] In the positions shown in FIGs. 1 IB and 11C, the user has moved, for example, sequentially moved, the computing device 200 in the direction of the arrow Al. In the positions shown in FIGs. 1 IB and 11C. the head and face of the user remain in substantially the same position as shown in FIG. 11 A. As the computing device 200 is moved from the position shown in FIG. 11 A to the position shown in FIG. 1 IB and then to the position shown in FIG. 11C, the image sensor 222 captures, for example, sequentially captures, image data of the head and face of the user from the different positions and/or orientations of the computing device 200/image sensor 222 relative to the head and face of the user. FIGs. 1 IB and 11C show just two example image frames captured by the image sensor 222 as the user moves the computing device 200 in the direction of the arrow Al, while the head and face of the user remain substantially stationary. Any number of image frames may be captured by the image sensor 222 as the computing device 200 is moved in the direction of the arrow Al. Similarly, any number of image frames captured by the image sensor 222 may be analyzed and processed by the recognition engine to detect and/or identify the example landmark 1010 and/or the example landmark 1020 the example landmarks 1030 and/or the example landmarks 1040 in the frames of image data captured as the computing device 200 is moved in this manner.

[00152] In FIGs. 1 ID-11G, the computing device 200 has been moved, for example, sequentially moved, in the direction of the arrow A2. In this example, as the computing device 200 is moved in the direction of the arrow A2, the head of the user remains in substantially the same position. As the computing device 200 is moved in the direction of the arrow' A2, the image sensor 222 captures image data of the head and face of the user from corresponding perspectives of the computing device 200/image sensor 222 relative to the head and face of the user. Thus, as the computing device 200 is moved in the direction of the arrow Al, and then in the direction of the arrow A2, the image sensor 222 captures image data including the head and face of the user from the various different perspectives of the computing device 200/image sensor 222 relative to the head and face of the user. In this particular example, the position and/or orientation of the head and face of the user remain substantially the same. FIGs. 11 D- 11 G show j ust some of the example frames of image data that may be captured by the image sensor 222 as the user moves the computing device 200 in the direction of the arrow A2. Any number of frames of image data may be captured by the image sensor 222 as the computing device 200 is moved in the direction of the arrow A2. Similarly, any number of frames of image data captured by the image sensor 222 may be analyzed and processed by the recognition engine to detect and/or identify the example landmark 1010 and/or the example landmark 1020 and/or the example landmarks 1030 and/or the example landmarks 1040 in the frames of image data captured as the computing device 200 is moved in this manner.

[00153] In the example shown in FIG. 11H, the user has moved the computing device 200. from the position shown in FIG. 11G (in which the computing device 200 is positioned substantially straight out from the head and face of the user, somewhat horizontally and vertically aligned with the head and face of the user) in the direction of the arrow A3. In this example, movement of the computing device 200 in the direction of the arrow A3 positions the computing device 200 at the left side of the user, capturing a profile image, or a senes of profile perspectives, of the head and face of the user. In the position shown in FIG 11H, the head and face of the user remain in substantially the same position as shown in FIG. 11G, simply for purposes of discussion and illustration. As the computing device 200 is moved from the position shown in FIG. 11G to the position shown in FIG. 11H, the image sensor 222 captures, for example, sequentially captures, image data of the head and face of the user, and in particular the nose of the user, from the different positions and/or orientations of the computing device 200/image sensor 222 relative to the head and face of the user. Any number of frames of image data may be captured by the image sensor 222 as the computing device 200 is moved in the direction of the arrow A3. Similarly, any number of image frames captured by the image sensor 222 may be analyzed and processed by the recognition engine to detect and/or identify' the example landmark 1010 and/or the example landmark 1020 and/or the example landmarks 1030 and/or the example landmarks 1040 in the image frames captured as the computing device 200 is moved in this manner.

[00154] In FIGs. I ll and 11 J, the computing device 200 has been moved, for example, sequentially moved, in the direction of the arrow' A4, from the position shown in FIG. 11H. In this example, as the computing device 200 is moved in the direction of the arrow A4, the head of the user remains in substantially the same position. In this example, movement of the computing device 200 in the direction of the arrow A4 positions the computing device 200 at the right side of the user, capturing a profile image, or a series of profile images, of the head and face of the user. As the computing device 200 is moved in the direction of the arrow A4, the image sensor 222 captures image data of the head and face of the user, and in particular, the nose of the user, from corresponding perspectives of the computing device 200/image sensor 222 relative to the head and face of the user. Thus, as the computing device 200 is moved in the direction of the arrow A3, and then in the direction of the arrow A4, the image sensor 222 captures image data including the head and face of the user, and in particular, the nose of the user, from the various different perspectives of the computing device 200/image sensor 222 relative to the head and face of the user. Any number of frames of image data may be captured by the image sensor 222 as the computing device 200 is moved in the direction of the arrow A3 and the arrow A4. Similarly, any number of frames of image data captured by the image sensor 222 may be analyzed and processed by the recognition engine to detect and/or identify the example landmark 1010 and/or the example landmark 1020 and/or the example landmarks 1030 and/or the example landmarks 1040 in the frames of image data captured as the computing device 200 is moved in this manner.

[00155] The image data captured by the image sensor 222 of the computing device 200 as the computing device 200 is moved as shown in FIGs. 11A-11J may be processed, for example, by a recognition engine accessible to the computing device 200 (for example, via external computing systems of the additional resources 302 described above with respect to FIG. 3). Landmarks and/or features and/or key points and/or elements may be detected in the image data captured by the image sensor 222 through the processing of the image data. In this example, the example landmarks 1010, 1020, 1030 and 1040, and measures associated therewith, illustrate just some example landmarks and/or elements that may be detected in the frames of image data captured by the image sensor 222.

[00156] As noted above, one example feature or measure may include the nose width W, between the landmarks 1040R, 1040L, representing a width of the nose, at the root end portion 1015 of the nose, where the bridge portion 129 of the head mounted wearable device 100 would be seated when worn by the user. Another example feature or measure may include the height H of the nose, representing a distance between the landmark 1010 and the landmark 1030R and/or between the landmark 1010 and the landmark 1030L. Another example feature or measure may include the depth D of the nose, representing a distance between the landmark 1020 and the landmark 1030R, and/or between the landmark 1020 and the landmark 1030L. As noted above, a slope S may be determined by dividing the nose height H by the nose depth D. The slope S may be representative of a slope of the dorsum, or nasal ridge 1025 of the nose.

[00157] In some examples, data provided by the position and/or orientation sensors included in the IMU 224, together with the processing and analysis of the image data, may be used to provide the user with feedback, to provide for improved image data capture. In some examples, one or more prompts may be output to the user. These prompts may include, for example, a prompt indicating that the user repeat the image data collection sequence. These prompts may include, for example, a prompt providing further instruction as to the user’s motion of the computing device 200 during the image data collection sequence. These ty pes of prompts may provide for the collection of image data from a different perspective that may provide a more complete representation of the head and/or face of the user. These prompts may include, for example, a prompt indicating that a change in the ambient environment may produce improved results such as, for example, a change to include fixed features in the background, a change in illumination of the ambient environment, and the like. In some examples, the prompts may be visual prompts output on the display portion 214 of the computing device 200. In some examples, the prompts may be audible prompts output by the audio output device 216 of the computing device 200.

[00158] Image data collected in this manner, and/or the fixed landmarks and/or fixed elements detected in the image data, and/or the features of measures associated with the fixed landmarks and/or fixed elements, combined with data provided by position and/or orientation sensors included in the IMU 224 of the computing device 200, may be processed by the one or more processors of the additional resources 302 accessible to the computing device 200 to predict fit of a wearable device, such as the example head mounted wearable device 100.

[00159] In some examples, the fixed landmarks and/or fixed features detected in the image data and/or associated features and/or measures, alone or in combination with the position/orientation data associated with the computing device 200, may be used to extract depth/develop a depth map. In this example, the fixed landmarks and/or fixed elements detected in the image data, combined with data provided by position and/or orientation sensors included in the IMU 224 of the computing device 200, may be processed by the one or more processors of the additional resources 302 accessible to the computing device 200 to develop one or more depth maps of the nose of the user. In some examples, the depth map(s) may be processed by the one or more processors of the additional resources 302 to develop a three-dimensional mesh, or a three-dimensional model, of the nose of the user. A simulation module, or a simulation engine, may process the three-dimensional mesh, or three-dimensional model, of the nose of the user to fit the head mounted wearable device 100 on the three-dimensional mesh or model, and predict fit of the head mounted wearable device 100 on the user.

[00160] In some examples, a metric scale may be applied to determine one or more facial and/or cranial and/or ophthalmic measurements associated with the detected landmarks and/or features (for example, nose width N and/or nose length L and/or nose height H and/or nose depth D and/or interpupillary distance IPD, and the like, as described in the example above, and/or other such measures). The determined one or more facial and/or cranial and/or ophthalmic measurements may be processed by, for example, a machine learning algorithm, to predict fit of the head mounted wearable device 100 on the user. In some examples, metric scale may be provided by, for example, an object having a known scale captured in the image data, by entry of scale parameters by the user, and the like. In some examples, in which metric scale is not otherwise provided, the data associated with the detected landmarks and/or features and/or elements and the position and/or orientation data associated with the computing device 200 may be aggregated by algorithms executed by the one or more processors to determine scale.

[00161] The image data captured in the manner described above, when processed by one or more fitting and/or sizing and/or simulation engines and/or modules, may provide for the prediction of fit of a wearable device, such as the head mounted wearable device 100 described above, using the computing device 200 operated by the user, without the use of specialized equipment such as a depth sensor, a pupil ometer and the like, without the use of a reference object having a known scale, without access to a retail establishment, and without a proctor to supervise the capture of the image data and/or to capture the image data. Rather, the image data may be captured by the image sensor 222 of the computing device 200 operated by the user, and in particular, by the image sensor 222 included in the front facing camera of the computing device 200.

[00162] As noted above, in some examples, one or more depth maps of the nose of the user may be generated based on a series of image frames including image data captured from different positions of the computing device 200 relative to the head and/or face of the user. The fixed landmarks and/or features and/or elements detected in the image data obtained in this manner may be tracked, and correlated with data provided by position and/or orientation sensors included in the IMU 224 of the computing device 200 to generate the one or more depth maps. In some examples, depth maps generated in this manner may be fused to generate a three-dimensional mesh, or a three-dimensional model, of the nose of the user.

[00163] In some examples, the frames of image data collected in this manner may be analyzed and processed, for example, by object and/or pattern recognition engines provided in the additional resources 302 accessible to the computing device 200, to detect the fixed landmarks and/or elements in the sequentially captured frames of image data. Data provided by the position and/or orientation sensors of the IMU 224 may be associated with the detected landmarks and/or elements in the sequential frames of image data. In some examples, changes in the measures associated with the fixed landmarks and/or elements, from image frame to image frame as the position and/or orientation of the computing device relative to the head/face of the user is changed and the sequential image frames are captured, may be associated with the data provisioned by the position and/or orientation sensors of the IMU 224.

[00164] This combined data may be aggregated, for example, by one or more algorithms applied by a data aggregating engine of the additional resources 302, to develop the one or more associated depth maps. In some examples, the depth map(s) may be fused to generate the three-dimensional mesh or model. In this example, the depth map(s) are fused to generate a three-dimensional mesh or model of the nose of the user. In an example in which metric scale is not otherwise provided, the data aggregating engine may aggregate this data to associate changes in pixel distance (based on analysis of the sequential frames of image data) with changes in position/orientation data of the computing device 200 to generate an estimate of metric scale.

[00165] For example, a head nose width W1 (based on the landmarks 1040R, 1040L), and a nose length NL1 (based on the landmarks 1010, 1020), are associated with the first position shown in FIG. 1 1A. A first position and a first orientation may be associated with the computing device 200, corresponding to the first position shown in FIG. 11A, based on data provided by the IMU 224. The position and orientation of the computing device 200 at the first position shown in FIG. 11 A may in turn be associated with the landmarks 1040R, 1040L and the associated Wl, and the landmarks 1010, 1020 and the associated LI.

[00166] As the computing device is moved from the first position shown in FIG. 11 A to the second position shown in FIG. 1 IB, a second position and a second orientation of the computing device 200 are associated with the computing device 200 based on data provided by the IMU 224. A motion stereo baseline can be determined based on the first position and first orientation, and the second position and second orientation of the computing device 200, together with the changes in position and/or orientation of the landmarks and/or elements and associated measures detected in the image data. As the computing device 200 is moved relative to the head and face of the user from the first position shown in FIG. 11 A to the second position shown in FIG. 1 IB, the image data captured by the image sensor 222 changes, so that the respective positions of the landmarks 1010. 1020, 1030R, 1030L, 1040R, 1040L change within the frame of image data. This in turn causes a change from the WT and LI shown in FIG. 11 A to the W2 and L2 shown in FIG. 1 IB.

[00167] In FIG. 1 IB, the relative second positions of the landmarks 1010, 1020. 1030R, 1030L, 1040R, 1040L (and corresponding distances W2 and L2) can be correlated with the corresponding movement of the computing device 200 from the first position and first orientation to the second position and second orientation. That is, the known change in position and orientation of the computing device 200, from the first position/orientation to the second position/orientation, may be correlated with a known amount of linear rotation (for example, based on gyroscope data from the IMU 224) and linear acceleration (for example, from accelerometer data from the IMU 224). Thus, the detected change in position of the landmarks 1010, 1020, 1030R, 1030L, 1040R, 1040L (and corresponding distances W2 and L2) may be determined, using the detected known change in position and orientation of the computing device 200 together w ith an associated scale value. This data may provide a first reference source for the development of a depth map for the corresponding portion of the head/face of the user captured in the corresponding image frames.

[00168] As the computing device 200 is moved from the second position shown in FIG. 11 B to the third position shown in FIG. 11 C, image data captured by the image sensor 222 changes, so that the respective positions of the landmarks 1010, 1020. 1030R, 1030L, 1040R, 1030L change , causing a change from the W2 and L2 shown in FIG. 1 IB to the W3 and L3 shown in FIG. 11C. The relative third positions of the landmarks 1010, 1020, 1030R, 1030L, 1040R, 1030L (and corresponding distances W3 and L3) can be correlated with the corresponding movement of the computing device 200 as described above to provide another reference source for the development of depth map(s) (as well as a reference source for scale, if scale is not otherwise provided and is to be determined).

[00169] Data may continue to be obtained as the user continues to move the computing device 200 in the direction of the arrow A2, as shown in FIGs. 1 ID-11G, from the third position and third orientation shown in FIG. 11 C to an example fourth position/ orientation shown in FIG. 1 ID, an example fifth position/orientation shown in FIG. 1 IE, an example sixth position/orientation shown in FIG. 1 IF, an example seventh position/orientation shown in FIG. 11G, an example eighth position/orientation shown in FIG. 11H, an example ninth position/orientation show n in FIG. 1 II, and an example tenth position/orientation shown in FIG. 11 J. As the computing device 200 is moved as shown in FIGs. 1 ID-fJ, image data captured by the image sensor 222 changes, so that the respective positions of the landmarks 1010, 1020. 1030R, 1030L, 1040R, 1040L change within the respective frames of image data. This in turn causes a sequential change from the W3 and L3 shown in FIG. 11C, to the W4/L4, W5/L5, W6/L6, W7/L7, L8, W9/L9, and LI 0 shown in FIGs. 1 1D- 11G, respectively. Continued movement of the computing device 200 in the direction of the arrow' A3 and then the arrow- A4 also for detection of changes from the D8 and H8 shown in FIG. 11H to the D10 and H10 shown in FIG. 11J. As described above, a slope S8 of the nasal ridge 1025 may be determined, corresponding to the nose height H8 and nose depth D8 shown in FIG. 11H. Similarly, a slope S10 of the nasal ridge 1025 may be determined, corresponding to the nose height H10 and nose depth D10 shown in FIG. 11 J.

[00170] The relative positions of the landmarks 1010, 1020, 1030R, 1030L, 1040R, 1040L (and corresponding distances) can again be correlated with the corresponding movement of the computing device 200, with known positions and orientations of the computing device 200 as the computing device 200 is moved as shown, based on a known amount of linear rotation (for example, based on gyroscope data from the IMU 224) and linear acceleration (for example, from accelerometer data from the IMU. This data may, again, be processed by the one or more processors, to develop one or more depth maps corresponding to portions of the face/head of the user captured in the image data of the associated image frames. In this particular example, this data is processed by the one or more processors to develop or more depth maps of the nose of the user, to facilitate the development of a three- dimensional mesh, or a three-dimensional model, of the nose of the user, for the prediction of fit of the head mounted wearable device 100.

[00171] The example shown in FIGs. 1 1 A- 11G describes a plurality of example data collection points, simply for ease of discussion and illustration. In some examples, image data and position and orientation data may be obtained at more, or fewer, points as the computing device 200 is moved. In some examples, image data and position and orientation data may be substantially continuously obtained, with corresponding depth data being substantially continuously determined. Depth data, detected in this manner, may be aggregated, for example, by a data aggregating engine and associated algorithms available via the additional resources 302 accessible to the computing device 200. The image data, and the associated position and orientation data, may continue to be collected until the aggregated data determined in this manner provides a relatively complete data set for the development of a three- dimensional mesh/three-dimensional model of the nose of the user. Similarly, in a situation in which metric scale is not otherwise provided, this motion stereo approach may be applied to the determination of scale.

[00172] FIGs. 11A-11G provide just one example of a manner in which the image data may be captured by a user operating the computing device 200, without the need for specialized equipment and/or proctoring and/or a physical or virtual appointment with a technician for assistance. Other types of computing devices may be used to obtain the image data, operated in manners other than described in the above example(s).

[00173] As noted above, the one or more depth maps may be generated from the image data captured in this manner, from various different perspectives/various different positions and/or orientations of the computing device 200 relative to the face and/or head, and in particular, the nose, of the user. In some examples, the depth maps may be fused, or stitched together, to develop a three-dimensional mesh, representative of a three-dimensional model, of the nose of the user. FIG. 12A illustrates a perspective view of an example three-dimensional mesh 1200 of the nose of the user. The example three-dimensional mesh 1200 may be generated based on a series of depth maps, developed from two-dimensional image data in a series of image frames as described above, that have been stitched or fused together to generate the three-dimensional mesh 1200. FIG. 12B illustrates the three-dimensional mesh 1200, superimposed on the nose of the user, including the identification of some of the example fixed facial landmarks.

[00174] In some examples, the three-dimensional mesh 1200, or three- dimensional model, may be provided to a simulation engine or a simulation module, to predict a fit of the wearable device (i.e., the head mounted wearable device 100) for the user. In some examples, various metric measurements may be extracted from the three-dimensional mesh 1200 for processing in predicting fit. In some examples, these measurements may include one or more of the example nose width W, slope S (determined from nose height H and nose depth D as described above), and/or other such measurements that can be derived based on the application of a known or determined metric scale to various fixed landmarks. In some examples, the various measurements may be used to predict various aspects of fit associated with the head mounted wearable device 100. In some examples, the processing of the three- dimensional mesh 1200 or model may predict a wearable fit, representative of how the head mounted wearable device 100 will physically fit on the face/head of the user, based on how the various features in the bridge portion 129 and corresponding portions of the rim portions 123 of the head mounted wearable device 100 will fit on the nose of the user. These features (nose width W, slope S, and corresponding physical features of the frame 110 of the head mounted wearable device 100) may also be used to predict how the head mounted wearable device 100 will be seated on the nose of the user, to facilitate the prediction of ophthalmic fit and/or display fit. For example, in a situation in which the head mounted wearable device 100 is to include corrective or prescription lenses, the processing of these measurements and features, and determination of how the frame 110 will be worn by the user, may allow the fitting prediction to take into account ophthalmic measurements such as pantoscopic angle, providing for the prediction of ophthalmic fit. Similarly, in a situation in which the head mounted wearable device 100 is to include display capability', this processing and fitting prediction may take into account display fit, so that content output by a display device of the head mounted wearable device 100 is visible to the user.

[00175] In some examples, the one or more measurements described above may be extracted, for example, from the three-dimensional mesh 1200, to predict sizing and/or fitting of the head mounted wearable device 100 for the user based on the image data obtained as described above. In some examples, the three-dimensional mesh 1200 and/or extracted measurements may be provided to a sizing and/or fitting simulator, or simulation engine, or simulation module. In some examples, the sizing and/or fitting simulator may access a database of available head mounted wearable devices and apply a machine learnings model to select one or more head mounted wearable devices, from the available head mounted wearable devices, that are predicted to fit the user based on the three-dimensional mesh 1200 and/or the extracted measurements. FIG. 12C illustrates one example head mounted wearable device 1250, of a plurality of head mounted wearable devices which may be considered by the simulator and/or the machine learning model, positioned on a model of the head/face of the user including the three-dimensional mesh 1200 of the nose of the user. FIG. 12D illustrates an image of the example head mounted wearable device 1250 superimposed on an image of the face of the user, with the three-dimensional mesh 1200 superimposed on the nose of the user.

[00176] In some examples, the simulator may access a fit database including fit data for each of the plurality of available head mounted wearable devices. Fit scores, accumulated across a relatively large pool of users, may be accessed to provide an indication and prediction of fit for the user, based on one or more of the measurements extracted from the three-dimensional mesh 1200. The database accessed by the machine learning model may include, for example, a distribution of scoring frequency for each of the plurality of available head mounted wearable devices for a range of nose widths, a range of nose slopes, and the like. These scores may be taken into consideration by the machine learning model in predicting fit for a head mounted wearable device for the user.

[00177] In some examples, the one or more head mounted wearable devices, predicted by the simulator implementing the machine learning model to be a fit for the user, may be presented to the user, for virtual tty on, comparison, and the like prior to purchase. In some examples, the simulator may predict whether a head mounted wearable that has already been selected by the user will fit the user. In some examples, the simulator may provide a fitting image 1300 to the user, as shown in FIG. 13. The fitting image 1300 may provide a visual indication during the virtual try- on, representative of how a selected head mounted wearable device 1350 will look on the face and/or head of the user.

[00178] As described above, in some implementations, pads 180 may be coupled to the rim portions 123 of the head mounted wearable device 100. In some examples, the pads 180 may be removably coupled to the rim portions 123. This may allow different sizes and/or shapes and/or configurations of pads 180 to be coupled to the rim portions rim portions 123 of the head mounted wearable device 100. The ability to customize a size and/or a shape and/or a configuration of pads 180 that are coupled to the rim portions 123 of the head mounted wearable device 100 may provide for adjustment of the fit of the head mounted wearable device 100. In some situations, this ability to adjust the fit using different sizes/shapes/configurations of pads 180 may expand the array of frames that will work for the user’s particular sizing needs, and thus provide the user with a wider section of frames from which to choose. In some situations, this ability⁷ to adjust the fit using different sizes/shapes/configurations of pads 180 may provide for fine tuning of ophthalmic fit and/or display fit. In some situations, this ability to adjust the fit using different sizes/shapes/configurations of pads 180 may help to maintain a desired position and/or orientation of the frame of the head mounted wearable device 100 on the nose and/or face of the user. In some situations, this ability⁷ to adjust the fit using different sizes/shapes/configurations of pads 180 may improve comfort of the head mounted wearable device 100 when worn by the user. In some situations, this ability to adjust the fit using different sizes/shapes/configurations of pads 180 may improve an appearance of the head mounted wearable device 100 on the face of the user.

[00179] In some examples, the position of the nose bridge, at the root end portion 1015 of the nose, and/or the height of the nose bridge, may provide an indication of how the head mounted wearable device head mounted wearable device 100 will be seated on the nose and/or how the head mounted wearable device 100 will be positioned on the face of the user. This may also provide an indication of whether or not some level of adjustment may improve the fit of the head mounted wearable device 100. Improvement in the fit of the head mounted wearable device 100 may include improvement in the physical fit, or wearable fit characteristics of the head mounted wearable device 100 and/or aesthetic fit characteristics of the head mounted wearable device 100. Improvement in the fit of the head mounted wearable device 100 may include improvement in the ophthalmic fit and/or display fit of the head mounted wearable device 100. [00180] In some examples, systems and methods, in accordance with implementations described herein, may use the image data captured as described above, and/or the three-dimensional mesh 1200, or model, developed from the image data, and/or the measurements extracted therefrom, to predict whether the addition of pads 180 will improve the fit (i.e., wearable fit and/or ophthalmic fit and/or display fit and/or aesthetic fit) of the head mounted wearable device 100. In some examples, the image data captured as described above, and/or the three-dimensional mesh 1200 developed from the image data, and/or the measurements extracted therefrom, may be used to predict a size and/or a shape and/or a configuration of pads 180 that can be used to improve the fit (i.e., wearable fit and/or ophthalmic fit and/or display fit and/or aesthetic fit) of the head mounted wearable device 100. Assistance provided in this manner, as part of the virtual sizing and/or fitting and/or try on process, may reduce or substantially eliminate user frustration in selecting the size/shape/configuration of pads 180 for adjustment of the head mounted wearable device 100 after product receipt. Assistance provided in this manner, as part of the virtual sizing and/or fitting and/or try on process, may provide for addition of the correct pads 180 to the head mounted wearable device 100 to provide for the desired wearable fit and/or ophthalmic fit and/or display fit and/or aesthetic fit, rather than rely ing on user trial and error after product receipt. This may further enhance the frictionless sizing and/or fitting and/or adjustment of the head mounted wearable device 100 in a virtual manner.

[00181] FIGs. 14A-14C are front views of the example frame 110 of the example head mounted wearable device 100 shown in FIGs. 1A-2B.

[00182] FIG. 14A presents a first frame configuration 110A, without the use of pads 180. When the first frame configuration 110A is worn by the user, the bridge portion 129 would be seated on the bridge portion, at the root end portion 1015 of the nose, with a contact portion 124A of the first (i.e., right, in the example arrangement shown in FIG. 14A) rim portion 123A seated on a first (i.e.. right, in the example arrangement shown in FIG. 14A) side of the nose, and a contact portion 124B of the second (i.e., left, in the example arrangement shown in FIG. 14A) rim portion 123B seated on the second (i.e., left, in the example arrangement shown in FIG. 14A) side of the nose. A bridge distance Bl extends between the contact portion 124A of the first rim portion 123 A and the contact portion 124B of the second rim portion 123B. The nose of the user may be accommodated in the area bounded by the bridge portion 129 and the contact portions 124A, 124B of the rim portions 123 A, 123B.

[00183] FIG. 15A illustrates the first frame configuration 110A fitted on the user, superimposed on the three-dimensional mesh 1200. It may be determined, based on the image data captured and processed as described above to generate the one or more depth maps and corresponding three-dimensional mesh, that the distance Bl between the contact portions 123A, 123B of the rim portions 123 is, for example, greater than the nose width W of the user. This would likely cause the bridge portion 129 of the head mounted wearable device 100 to be seated too far down on the nasal ridge, and some possible misalignment between the optical axes of the eyes of the user/the field of view of the user, and/or misalignment with the optical curvature of corrective lenses, and/or misalignment with the output coupler 105 of the display device 106. In some examples, the distance Bl between the contact portions 123A, 123B of the rim portions 123 may be adjusted through the addition of pads 180 on the rim portions 123. In some examples, the virtual sizing and/or fitting of the head mounted wearable device 100 as described above can include a prediction of fit based on the addition of pads 180 to adjust a fit of the frame 1 10 on the nose, and face, of the user, to adjust an orientation of the frame 110 on the nose, and face, of the user, and the like. The virtual sizing and/or fitting of the head mounted wearable device 100 can include a selection of one. of a plurality of different size and/or shape and/or configuration of pads, that will provide the desired sizing and/or fitting of the head mounted wearable device 100 for a particular user.

[00184] As shown in FIG. 15 A, the first frame configuration 110A is seated somewhat lower than desired on the bridge of the nose, resulting in the misalignment described above. Accordingly, in some examples, the image data and resulting depth maps and/or three-dimensional mesh 1200 and/or measurements extracted therefrom may be processed to select pads 180 which may be coupled to the rim portions 123 to provide for the desired position and/or orientation of the head mounted wearable device 100 when worn by the user.

[00185] FIG. 14B illustrates a second frame configuration HOB including pads 180B coupled to the rim portions 123 A, 123B, with a distance B2 betw een the contact portions 124A, 124B of the rim portions 123 A, 123B that is changed from the distance Bl. FIG. 15B illustrates the second frame configuration HOB, including the pads 180B. fitted on the user, superimposed on the three-dimensional mesh 1200. The first frame configuration 110A is also shown in FIG. 15B, for purposes of comparison. As shown in FIG. 15B, the second frame configuration HOB is seated somewhat higher than the first frame configuration 110A on the bridge of the nose, and somewhat closer to the root end portion 1015, or bridge of the nose.

[00186] FIG. 14C illustrates a third frame configuration HOC including pads 180C coupled to the rim portions 123 A, 123B, with a distance B3 between the contact portions 124A, 124B of the rim portions 123 A, 123B that is changed from the distance B2 and/or the distance Bl. FIG. 15C illustrates the third frame configuration 1 10C, including the pads 180C, fitted on the user, superimposed on the three- dimensional mesh 1200. The second frame configuration HOB and the first frame configuration 110A are also shown in FIG. 15C, for purposes of comparison. As shown in FIG. 15C, the third frame configuration 110C is seated somewhat higher than the second frame configuration 110B, and somewhat higher than the first frame configuration 110A, on the bridge of the nose, and closer to the root end portion 1015, or bridge of the nose.

[00187] FIG. 15D illustrates a profile view of the first frame configuration 110 A, the second frame configuration 110B, and the third frame configuration 110C, as worn by the user.

[00188] As shown in FIGs. 15C and 15D, a size and/or a shape and/or a configuration of the pads 180C included with the third frame configuration 110C may provide an improved fit of the frame 1 10 on the face/head of the user. The size and/or shape and/or configuration of the pads 180C included with the third frame configuration 110C may provide for improved alignment of the optical axis of the user with the curvature corrective lenses which may be included in the head mounted wearable device 100. That is, the addition of pads, for example as in the third frame configuration, may provide for adjustment to achieve the desired pantoscopic angle and/or the desired pantoscopic height and/or the desired vertex distance as discussed above with respect to FIGs. 2D-2F. The size and/or shape and/or configuration of the pads 180C included with the third frame configuration 110C may provide for improved alignment of the optical axis of the user with content output by the display device 106. The size and/or shape and/or configuration of the pads 180C included with the third frame configuration 110C may position the head mounted w earable device 100 on the nose/face of the user, and maintain the head mounted w earable device 100 on the nose/face of the user in a position that is more comfortable to the user and/or aesthetically pleasing. [00189] The prediction of the sizing and/or fitting of the head mounted wearable device 100. including the selection of pads 180 for the sizing and/or fitting of the head mounted wearable device 100, based on a three-dimensional mesh generated from depth maps developed from one or more frames of image data may provide for the relatively accurate virtual sizing and/or fitting of the head mounted wearable device 100. without the use of specialized equipment and/or proctoring by a technician during a physical or virtual fitting session.

[00190] S\ 'stems and methods, in accordance with implementations described herein, may provide a prediction of fit of the head mounted wearable device 100 for the user based on image data, obtained by the user operating the computing device 200, combined with position and/or orientation data provided by one or more sensors of the computing device 200. In the examples described above, image data of the head and face of the user, and in particular, the nose of the user, is obtained by the image sensor 222 of a front facing camera of the computing device 200. In some situations, the collection of image data in this manner may pose challenges due to. for example, the relative proximity between the image sensor 222 of the front facing camera and the head/face of the user, inherent, natural movement of the head and face of the user as the computing device 200 is moved, combined with the need for accuracy in the fitting of head mounted wearable devices. The use of static key points, or elements, or features, in the background that anchor the captured image data as the computing device 200 is moved and sequential frames of image data are captured, may increase the accuracy of the depth data derived from the image data and position/orientation data, and the subsequent three-dimensional mesh, and the fitting of the head mounted wearable device fitted based on the three-dimensional mesh and/or extracted measurements. The collection of multiple frames of image data and the combining of the image data with corresponding position/orientation data associated with the computing device 200 as the series of frames of image data is collected, may improve the level of accuracy in prediction of fit of the head mounted wearable device.

[00191] In the examples described above, the example movement of the computing device 200 is in a substantially vertical direction, in front of the user, and in a substantially horizontal direction, across the front and to the left and right side profiles of the user, while the head and face of the user remain substantially still, or static. The image data obtained through the example movement of the computing device 200 as shown in FIGs. 11 A-l 1 J may provide for the relatively clear and detectable capture of the example facial landmarks and/or static key points/fixed elements as the perspective of the computing device 200 changes relative to the head/face of the user. In some examples, systems and methods, in accordance with implementations described herein, may be accomplished using other movements of the computing device 200 relative to the user.

[00192] Systems and methods, in accordance with implementations described herein, provide for the prediction of fit of a wearable device from image data and position/orientation data using a client computing device. In some implementations, systems and methods, in accordance with implementations described herein, provide for the determination of scale from the image data and position/orientation data obtained using the client computing device. Systems and methods, in accordance with implementations described herein, may provide for the prediction of fit from image data and position/orientation data without the use of a known reference object. Systems and methods, in accordance with implementations described herein, may predict fit from image data and position/orientation data without the use of specialized equipment such as. for example, depth sensors, pupilometers and the like that may not be readily available to the user. Systems and methods, in accordance with implementations described herein, may predict from image data and position/orientation data without the need for a proctored virtual fitting and/or access to a physical retail establishment. Systems and methods, in accordance with implementations described herein, may improve accessibility to the virtual selection and accurate fitting of w earable devices. The prediction of fit in this manner provides for a virtual try-on of an actual w earable device to determine wearable fit and/or ophthalmic fit and/or display fit of the wearable device.

[00193] FIG. 16 is a flowchart of an example method 1600 of predicting fit from image data and position/orientation data. A user operating a computing device (such as, for example, the computing device 200 described above) may initiate image capture functionality of the computing device (block 1610). In some examples, the image capture functionality may be operable within an application executing on the computing device. Initiation of the image capture functionality may cause an image sensor (such as, for example, the image sensor 222 of the front facing camera of the computing device 200 described above) to capture first image data including at least a portion of a face and/or a head of the user (block 1620). In some examples, the image data includes a nose of the user. In some examples, the first image data is captured at a first position and a first orientation of the computing device, for example, a first position and a first orientation of the computing device relative to the head of the user. In some examples, the position and orientation of the computing device may be provided based on, for example, data provided by position/orientation sensors of the computing device at a point corresponding to capture of the first image data. At least one fixed feature may be detected within the first image data (block 1630). The at least one fixed feature may include fixed facial features and/or landmarks that remain substantially static. The at least one fixed feature may include features defining the nose of the user, such as, for example, a width of the nose at a root end portion of the nose, corresponding to a bridge portion of the nose at which a bridge portion of a head mounted wearable device would be seated. In some examples, the at least one fixed feature may be defined by two facial landmarks detected in the image data.

[00194] Continued operation of the image capture functionality may cause the computing device to incrementally capture image data including the portion of the face and/or head of the user and the at least one fixed feature (block 1640, block 1650), until the image capture functionality is terminated. In some examples, the image capture functionality may be terminated when it is determined, for example, within the application executing on the computing device, that a sufficient amount of image data has been captured for the determination of measurements associated with one or more fixed features detected in the image data (block 1660). The extracted measurements may be processed by a machine learning model, and/or a simulator, to predict fit of a head mounted wearable device for the user (block 1670).

[00195] FIG. 17 is a flowchart of an example method 1700 of simulating fit from image data and position/orientation data. A user operating a computing device (such as, for example, the computing device 200 described above) may initiate image capture functionality of the computing device (block 1710). In some examples, the image capture functionality may be operable within an application executing on the computing device. Initiation of the image capture functionality may cause an image sensor (such as, for example, the image sensor 222 of the front facing camera of the computing device 200 described above) to capture first image data including at least a portion of a face and/or a head of the user (block 1715). In some examples, the image data includes a nose of the user. At least one fixed feature may be detected within the first image data (block 1720). The at least one fixed feature may include fixed facial features and/or landmarks that remain substantially static. The at least one fixed feature may include features defining the nose of the user, such as, for example, a width of the nose at a root end portion of the nose, corresponding to a bridge portion of the nose at which a bridge portion of a head mounted wearable device would be seated. In some examples, the at least one fixed feature may be defined by two facial landmarks detected in the image data. A first position and a first orientation of the computing device may be detected (block 1725) based on, for example, data provided by position/orientation sensors of the computing device, at a position and an orientation of the computing device corresponding to capture of the first image data. [00196] Continued operation of the image capture functionality may cause the computing device to incrementally capture second image data including the portion of the face and/or head of the user and the at least one fixed feature (block 1730, block 1735), until the image capture functionality is terminated. In some examples, the image capture functionality may be terminated when it is determined, for example, within the application executing on the computing device, that a sufficient amount of image data has been captured for the development of a three-dimensional mesh/three- dimensional model of the portion of the face and/or head of the user, for example, the nose of the user, for the purposes of predicting and/or simulating fit of a head mounted wearable device. Changes in the position and the orientation of the computing device may be correlated with changes in position of the at least one fixed feature detected in a current frame of image data compared to the position of the at least one fixed feature detected in a previous frame of image data (block 1740). Depth data may be extracted based on the comparison of the current image frame of data to the previous image frame of data, and the respective position of the at least one fixed feature (block 1745). At least one depth map of the portion of the face and/or head of the user may be generated based on the depth data extracted from the correlation of the position/orientation data of the computing device with the changes of position in the at least one fixed feature detected in the frames of image data (block 1750). The at least one depth map may be at least one depth map corresponding to the nose of the user. The depth maps may be fused, or stitched, together to develop a three- dimensional mesh, or a three-dimensional model, of the portion of the face and/or head of the user (block 1755), for example, a three-dimensional mesh, or a three- dimensional model, of the nose of the user. The three-dimensional mesh, and/or measurements extracted therefrom, may be processed by a simulator and/or a machine learning model, to simulate fit of a head mounted wearable device for the user (block 1760).

[00197] A number of embodiments have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the specification.

[00198] In addition, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. In addition, other steps may be provided, or steps may be eliminated, from the descnbed flows, and other components may be added to, or removed from, the described systems. Accordingly, other embodiments are within the scope of the following claims.

[00199] Further to the descriptions above, a user may be provided with controls allowing the user to make an election as to both if and when systems, programs, or features described herein may enable collection of user information (e.g., information about a user’s social network, social actions, or activities, profession, a user’s preferences, or a user’s current location), and if the user is sent content or communications from a server. In addition, certain data may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user’s identity may be treated so that no personally identifiable information can be determined for the user, or a user’s geographic location may be generalized where location information is obtained (such as to a city, ZIP code, or state level), so that a particular location of a user cannot be determined. Thus, the user may have control over what information is collected about the user, how that information is used, and what information is provided to the user.

[00200] While certain features of the described implementations have been illustrated as described herein, many modifications, substitutions, changes and equivalents will now occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the scope of the implementations. It should be understood that they have been presented by way of example only, not limitation, and various changes in form and details may be made. Any portion of the apparatus and/or methods described herein may be combined in any combination, except mutually exclusive combinations. The implementations described herein can include various combinations and/or sub-combinations of the functions, components and/or features of the different implementations described.

Claims

WHAT IS CLAIMED IS:

1. A non-transitory computer-readable medium storing executable instructions that when executed by at least one processor of a computing device are configured to cause the at least one processor to: capture, by an image sensor of the computing device, current image data, the current image data including a head of a user; detect at least one fixed feature in the current image data; detect a change in a position and an orientation of the computing device, from a current position and a current orientation corresponding to the capture of the current image data, to a previous position and a previous orientation corresponding to the capture of previous image data including the head of the user; detect a change in a position of the at least one fixed feature between the current image data and the previous image data; correlate the change in the position and the orientation of the computing device with the change in the position of the at least one fixed feature; generate a three-dimensional model of the head of the user based on depth data extracted from the correlating of the change in position and orientation of the computing device with the change in position and orientation of the at least one fixed feature; and predict, by a machine learning model accessible to the computing device, a fit of a head mounted wearable device on the head of the user based on the three- dimensional model of the head of the user.

2. The non-transitory computer-readable medium of claim 1, wherein the at least one fixed feature includes at least two facial landmarks that are representative of a facial measurement, including at least one of: a distance between a first ear saddle point and a second ear saddle point representative of a head width of a user; a distance between an outer comer portion of a right eye and an outer comer portion of a left eye of the user; a distance between an inner comer portion of the right eye and an inner comer portion of the left eye of the user; or a distance between a pupil of the right eye and a pupil of the left eye of the user.

3. The non-transitory computer-readable medium of claim 1 or 2, wherein the at least one fixed feature includes a distance between at least two fixed elements detected in a background area surrounding the head of the user.

4. The non-transitory computer-readable medium of any one of claims 1 to 3, wherein the executable instructions cause the at least one processor to detect the change in the position and the orientation of the computing device, including: detect the previous position and the previous orientation of the computing device in response to receiving previous data provided by an inertial measurement unit of the computing device at the capture of the previous image data; detect the current position and the current orientation of the computing device in response to receiving current data provided by the inertial measurement unit of the computing device at the capture of the current image data; and determine a magnitude of movement of the computing device corresponding to the change in the position and the orientation of the computing device based on a comparison of the current data and the previous data.

5. The non-transitory computer-readable medium of claim 4, wherein the executable instructions cause the at least one processor to: associate the magnitude of the movement of the computing device to a change in a measurement associated with the at least one fixed feature; and determine depth data based on the associating.

6. The non-transitory computer-readable medium of any one of claims 1 to 5, wherein the executable instructions cause the at least one processor to: repeatedly capture image data as the computing device is moved relative to the user to capture image data from a pl urality of different positions and orientations of the computing device relative to the head of the user; correlate a plurality of changes in position and orientation of the computing device with a corresponding plurality of changes in position of the at least one fixed feature detected the image data; determine depth data as the image data is repeatedly captured from the plurality of different positions and orientations based on the correlating; and develop the three-dimensional model of the head of the user for predicting the fit of the head mounted wearable device based on the repeatedly capturing of the image data by the computing device from the plurality of different positions and orientations and the depth data determined from the repeatedly capturing of the image data.

7. The non-transitory computer-readable medium of any one of claims 1 to 7, wherein the executable instructions cause the at least one processor to: generate the three-dimensional model of the head of the user; extract at least one measurement from the three-dimensional model of the head of the user; and select a head mounted wearable device, from a plurality of available head mounted wearable devices, based on the at least one measurement, the at least one measurement including at least one of: a cranial measurement determined based on distance between two fixed facial features detected in the current image data and the previous image data; or an ophthalmic measurement determined based on a distance between two optical features detected in the current image data and the previous image data.

8. A computer-implemented method, comprising: capturing current image data, via an application executing on a computing device operated by a user, the current image data including a nose of the user captured at a current position and a current orientation of the computing device; detecting at least one fixed feature in the current image data; detecting a change in a position and an orientation of the computing device; detecting a change in a position of the at least one fixed feature between the current image data and previous image data captured at a previous position and a previous orientation of the computing device; correlating the change in the position and the orientation of the computing device with the change in the position of the at least one fixed feature; generating a three-dimensional model of the nose of the user based on depth data extracted from the correlating of the change in position and orientation of the computing device with the change in position and orientation of the at least one fixed feature: and simulating, by a simulation engine accessible to the computing device, a fit of a head mounted wearable device on a head of the user based on the three-dimensional model of the nose of the user.

9. The computer-implemented method of claim 8, wherein the at least one fixed feature includes at least two facial features that are representative of a fixed measurement associated with the nose of the user.

10. The computer-implemented method of claim 9, wherein the fixed measurement includes at least one of: a width of the nose at a root end portion of the nose; or a slope of the nose along a nasal ridge of the nose.

11. The computer-implemented method of claim 10. wherein the width of the nose is representative of a distance between a right end portion of the nose at the root end portion of the nose, and a left end portion of the nose at the root end portion of the nose.

12. The computer-implemented method of claim 1 1 , wherein the at least two facial features includes three facial features, including: a sellion at the root end portion of the nasal ridge of the nose; a tip of the nose at a distal end portion of the nasal ridge of the nose; and an ala at a lower end portion of the nose, corresponding to a first lower end portion and a second lower end portion of the nose.

13. The computer-implemented method of claim 12. wherein the fixed measurement includes: a nose height, representative of a distance between the root end portion of the nose and at least one of the first lower end portion or the second lower end portion of the nose; and a nose depth, representative of a distance between the tip of the nose and at least one of the first lower end portion or the second lower end portion of the nose, wherein the slope is the nose height divided by the nose depth.

14. The computer-implemented method of any one of claims 8 to 13, further comprising: generating, by the simulation engine, a simulated fit of the head mounted wearable device based on the simulating; selecting a pair of adjustment pads, from a plurality of adjustment pads, that are selectively couplable to the head mounted wearable device; and generating a simulated adjusted fit of the head mounted wearable device including the pair of adjustment pads.

15. The computer-implemented method of claim 14, wherein the pair of adjustment pads includes a first adjustment pad that is selectively couplable to a first rim portion of the head mounted wearable device and a second adjustment pad that is selectively couplable to a second rim portion of the head mounted wearable device.

16. The computer-implemented method of claim 14 or 15. wherein generating the simulated adjusted fit includes adjusting at least one of: a position of a bridge portion of the head mounted wearable device along a nasal ridge of the nose on the three-dimensional model of the nose; or an angular position of a front frame portion of the head mounted wearable device relative to the nasal ridge of the nose on the three-dimensional model of the nose.

17. The computer-implemented method of any one of claims 8 to 16, wherein detecting the change in the position and the orientation of the computing device includes: detecting the previous position and the previous orientation of the computing device in response to receiving previous data provided by an inertial measurement unit of the computing device at the capturing of the previous image data; detecting the current position and the current orientation of the computing device in response to receiving current data provided by the inertial measurement unit of the computing device at the capturing of the current image data; and determining a magnitude of movement of the computing device corresponding to the change in the position and the orientation of the computing device based on a comparison of the current data and the previous data.

18. The computer-implemented method of claim 17, wherein correlating the change in the position and the orientation of the computing device with the change in the position of the at least one fixed feature includes: associating the magnitude of the movement of the computing device to a change in a measurement associated with the at least one fixed feature; and determining depth data based on the associating.

19. The computer-implemented method of claim 18. further comprising: repeatedly capturing image data as the computing device is moved relative to the user to capture image data from a plurality of different positions and orientations of the computing device relative to the head of the user; correlating a plurality of changes in position and orientation of the computing device with a corresponding plurality of changes in position of the at least one fixed feature detected the image data; determining depth data as the image data is repeatedly captured from the plurality of different positions and orientations based on the correlating; and developing the three-dimensional model of the nose of the user for predicting the fit of the head mounted wearable device based on the repeatedly capturing of the image data by the computing device from the plurality of different positions and orientations and the depth data determined from the repeatedly capturing of the image data.

20. The computer-implemented method of claim 18 or 19, wherein predicting, by the simulation engine accessible to the computing device, the fit of the head mounted wearable device includes: generating the three-dimensional model of the nose of the user; extracting at least one measurement from the three-dimensional model of the head of the user; and selecting a head mounted wearable device, from a plurality of available head mounted wearable devices, based on the at least one measurement.

21. The computer-implemented method of claim 20. wherein the at least one measurement includes at least one of: a nose width based on distance between two fixed facial features detected in the current image data and the previous image data; or a nose slope determined based on nose height and a nose depth, the nose height being based on a distance between two fixed facial features detected in the current image data and the previous image data, and the nose depth being based on a distance between two fixed facial features detected in the current image data and the previous image data.

22. A system, comprising: a computing device, including: an image sensor; at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the at least one processor to: capture current image data, the current image data including a head of a user; detect at least one fixed feature in the current image data; capture previous image data, the previous image data including the head of the user; detect the at least one fixed feature in the previous image data; detect a change in a position and an orientation of the computing device, from a previous position and a previous orientation corresponding to the capture of the previous image data, to a current position and a current orientation corresponding to the capture of the current image data; detect a change in a position of the at least one fixed feature between the current image data and the previous image data; correlate the change in the position and the orientation of the computing device with the change in the position of the at least one fixed feature; generate a three-dimensional model of the head of the user based on depth data extracted from the change in position and orientation of the computing device correlated with the change in position and orientation of the at least one fixed feature; and predict a fit of a head mounted wearable device on the head of the user based on the three-dimensional model of the head of the user.

23. The system of claim 22, wherein the instructions cause the at least one processor to: generate the three-dimensional model of the head of the user; extract at least one measurement from the three-dimensional model of the head of the user; and select a head mounted wearable device, from a plurality of available head mounted wearable devices, based on the at least one measurement, the at least one measurement including at least one of: a cranial measurement determined based on distance between two fixed facial features detected in the current image data and the previous image data; or an ophthalmic measurement determined based on a distance between two optical features detected in the current image data and the previous image data.

24. The system of claim 22 or 23, wherein the at least one fixed feature includes a plurality of fixed features, including: at least one facial landmark defined by at least two fixed facial features; and at least one fixed element defined by at least two fixed key points detected in a background area surrounding the head of the user.