EP4189470A1

EP4189470A1 - Method and device for automatically determining production parameters for a pair of spectacles

Info

Publication number: EP4189470A1
Application number: EP21739241.4A
Authority: EP
Inventors: Kevin Metka; Pawel JOBKIEWICZ; Julian HOELZ
Original assignee: Tribe GmbH
Current assignee: Mister Spex Se
Priority date: 2020-07-31
Filing date: 2021-06-01
Publication date: 2023-06-07
Also published as: US20230221585A1; DE112021003994A5; WO2022022765A1

Abstract

The invention relates to a method and device for automatically determining production parameters for a pair of spectacles, which method comprises the following in one or more processors designed for data processing: capturing head image data for at least a part of the head of a spectacles wearer and determining a head parameterisation for at least a part of the head of the spectacles wearer, the head parameterisation indicating head parameters for at least the part of the head of the spectacles wearer, which parameters are relevant for the adjustment of a pair of spectacles, and the head parameters comprising at least one lens grinding parameter and at least one spectacles support parameter. The method also comprises the following: providing a spectacles parameterisation for the spectacles, the spectacles parameterisation indicating relevant spectacles parameters for adjusting the spectacles for the spectacles wearer; and performing data mapping for the head parameterisation and the spectacles parameterisation, in which for the adjustment of the spectacles for the spectacles wearer at least one spectacles parameter is adjusted according to an associated head parameter and/or at least one further spectacles parameter is determined.

Description

Method and device for automatically determining manufacturing parameters for spectacles

The invention relates to a method and a device for automatically determining production parameters for spectacles.

background

It is known to fit eyeglasses on site in the presence of an optician. Without the presence of an optician, it has hitherto not been possible to reliably determine the necessary parameters for centering the spectacle lens. Likewise, no frame adjustment is possible.

In addition, it was suggested to identify glasses online. The spectacles user is asked to place a reference object, the size of which is standardized and therefore generally known, for example a credit card, on their forehead and take a photo. Since the reference object has a standardized size, the pupillary distance can be derived using simple arithmetic. However, it cannot be guaranteed that the reference object and the pupil of the eye are on the same frontal plane. Likewise, the orthogonal angle of the reference object to the frontal plane cannot be guaranteed. In practice, this means a measurement deviation of several millimeters on average, which is unsuitable for the centering of the spectacle lens.

So far, the necessary parameters for the centering of the spectacle lens can only be determined in the presence of an optician. In particular, different devices are necessary for measuring a pupillary distance and a grinding-in height. In addition, the conventional measuring devices lead to a high measurement variance and low accuracy. This also applies to the current mobile applications.

summary

The object of the invention is to specify a method and a device for automatically determining production parameters for glasses, which in particular enables the parameters for the glasses to be determined efficiently and with high accuracy using an online method. The object is solved by a method and a device for automatically determining production parameters for glasses according to independent claims 1 and 12. Configurations are the subject matter of dependent subclaims.

According to one aspect, a method for automatically determining production parameters for glasses is created, the following being provided in one or more processors set up for data processing: capturing head image data for at least part of a head of a glasses user; Determining a head parameterization for at least the part of the head of the spectacles user, the head parameterization for an adjustment of spectacles relevant head parameters for at least the part of the head of the spectacles user and the head parameters include at least one lens grinding parameter and at least one spectacles support parameter; Providing a spectacle parameterization for the spectacles, wherein the spectacle parameterization for the adjustment of the spectacles for the spectacles user indicates relevant spectacles parameters; and carrying out a data mapping for the head parameterization and the glasses parameterization, in which case at least one glasses parameter is adjusted according to an assigned head parameter and/or at least one further glasses parameter is determined in order to adapt the glasses for the glasses user.

According to a further aspect, an apparatus for automatically determining manufacturing parameters for eyeglasses is provided, having one or more processors configured for data processing, which are configured to: receive head image data for at least part of a head of a user of glasses; Determining a head parameterization for at least the part of the head of the spectacles user, the head parameterization for an adjustment of spectacles relevant head parameters for at least the part of the head of the spectacles user and the head parameters include at least one lens grinding parameter and at least one spectacles support parameter; Providing a spectacle parameterization for the spectacles, wherein the spectacle parameterization for the adjustment of the spectacles for the spectacles user indicates relevant spectacles parameters; and carrying out a data mapping for the head parameterization and the glasses parameterization, in which case at least one glasses parameter is adjusted according to an assigned head parameter and/or at least one further glasses parameter is determined in order to adapt the glasses for the glasses user. Furthermore, the following can be provided: acquisition of RGB head image data for at least the part of the head of the spectacles user; providing calibration data indicative of a calibration of an image capture device used to capture RGB header image data; and determining the at least one lens grinding parameter using the RGB header image data and the calibration data by means of image data analysis, wherein a localization vector associated with the pupils is determined, which indicates an image pixel position for the pupils. With regard to the at least one lens grinding parameter, a horizontal distance between the pupils can be determined, for example.

The following can also be provided: providing depth image data and determining the at least one lens grinding parameter using the RGB header image data, the depth image data and the calibration data.

The following can also be provided in the method: providing reference feature data that indicate a biometric reference feature for the spectacles user, and determining the at least one lens grinding parameter using the RGB header image data, the reference feature data and the calibration data. The reference feature data can indicate, for example, a reference length measure, for example a diameter of the iris, as a biometric reference feature. The iris diameter has essentially the same characteristic size for a large number of people. It can be provided that the at least one lens grinding parameter is initially determined using the depth image data in order to then verify the result obtained in this way by means of a determination using the reference feature data.

The at least one spectacle parameter or the at least one further spectacle parameter can include a real grinding height for spectacles designed as progressive lenses, in which case the following can also be provided: determining at least one fixed point of a real spectacle frame of real spectacles, which is a transition between a spectacle lens and the spectacle frame displays, in which case a localization vector assigned to the at least one fixed point of the spectacle frame is determined, which localization vector indicates an image pixel position for the at least one fixed point of the spectacle frame; and vertically projecting a pupil mark indicative of the pupil onto the eyeglass frame.

The at least one spectacle parameter or the at least one further spectacle parameter can include a virtual grinding height for spectacles designed as varifocal spectacles, where In this case, the following can also be provided: providing a 3D model of virtual glasses, from which a glasses parameterization for the virtual glasses is determined; determining at least one fixed point of a spectacle frame of the virtual glasses, which indicates a transition between a spectacle lens and the spectacle frame, wherein a localization vector assigned to the at least one fixed point of the spectacle frame is determined, which indicates an image pixel position for the at least one fixed point of the spectacle frame; and vertically projecting a pupil mark indicative of the pupil onto the eyeglass frame

In the method, the 3D modeling of the virtual glasses can be selected from a large number of different virtual glasses, for which a respective 3D modeling is stored in a memory device. For this purpose, the respective 3D modeling (3D model data) is stored in advance in the memory device for the virtual glasses. The 3D model data can be stored in a data format suitable for 3D printing (STL, OBJ, etc.).

The 3D modeling may include, in one example, the following eyewear model data: (i) components of an eyewear - (a) front piece, bridge, cheekpieces; (b) left and right stirrup, stirrup inflection point; and (c) nose pad length and angle; and (ii) Characteristics of each component: Each pair of glasses is designed slightly differently, and the components thereby have different characteristics in terms of size and deformation.

When selecting the virtual glasses, you can choose from all characteristics for all components the one that best suits the glasses user (“cutting away the solution space”). Three components can be considered here: (i) header parameters; (ii) glasses parameters and (iii) mapping logic (“cutting away the solution space”). A cutting plane method can be used. The starting point here is the entire solution space, which includes all size combinations of the glasses components. Based on the head parameters, a size combination is then selected that fits best. "Best fits" is defined with the help of biometric and customer-relevant data, for example as follows: "Normal glasses for a man should have a width that corresponds to the width of the head. A pair of sunglasses for a woman should have a width equal to the head width + 10%.” This then gives the mapping logic. In the case of methods, the following can also be provided: determining a 3D coordinate system; mapping the head parameterization for at least part of the head of the glasses user and the glasses parameterization into the 3D coordinate system and determining one or more of the following parameters in the 3D coordinate system: horizontal interpupillary distance, face width at the pupillary level, real focal height and virtual focal height.

Based on the head parameterization, a temple length for the temples of the glasses and a bending point for the temples can be determined for the adjustment of the glasses.

The head parameters can include one or more lens grinding parameters from the following group: horizontal interpupillary distance and head width.

The head parameters may include one or more eyewear parameters from the following group: face width at pupillary level, nose width, nose attachment point, ear attachment point, distance between nose and ears, and cheek contour.

The configurations explained above in connection with the method can be provided accordingly in connection with the device.

Further exemplary embodiments are explained below with reference to figures of a drawing:

1 shows a schematic representation relating to a method for automatically determining production parameters for spectacles;

2 shows a schematic representation of a pair of spectacles with the horizontal distance between the pupils drawn in and the real grinding height drawn in;

3 shows a schematic representation relating to the determination of a position for a pupil;

4 shows a schematic representation of a pair of glasses with the real segment height drawn in;

5 shows a schematic representation of nine RGB image pixels (solid line) and four depth image pixels (dashed line) and two marked RGB image pixels (striped and checked area); 6 shows a schematic representation of four RGB image pixels (solid line) and nine depth image pixels (dashed line) and one marked depth image pixel (striped area);

7 shows a schematic representation of a canonical spectacle model with front, temple, inflexion point and inflection angle;

FIG. 8 shows a schematic representation of a face width determination; and FIG. 9 shows a schematic representation of facial feature points.

A method and a device for automatically determining production parameters for spectacles are described below using various exemplary embodiments.

In this case, according to FIG. 1, images of a head front view are recorded for a spectacle wearer 1 with the aid of a recording device 2 . The recording device 2 can be formed with a mobile terminal such as a mobile phone, tablet or laptop computer or a stationary terminal such as a desktop computer. As is usual for such end devices, the recording device 2 has a camera for recording the images and a display device (display) for outputting image data to the spectacle wearer 1 . A user interface of the recording device 2 also has an input device for receiving inputs from the spectacle wearer 1, be it via a keyboard and/or a touch-sensitive input surface.

Commercially available mobile terminals, for example mobile phones or laptop computers, are able to collect a range of environmental measurement data, for example by means of one or more sensor or recording devices: CMOS camera, infrared camera, distance sensors and point projectors.

Images can be recorded by means of the recording device 2, from which digital image information can be determined: image data (RGB information); depth data (especially distances) and calibration data (such as resolution, angle, etc.). 3D data is determined from digital image information, with the following being provided in one exemplary embodiment (cf. also further explanations below):

(i) Points of interest (POI) are detected in the image data, e.g. pupils, frames, noses, etc., up to the entire part of the head (e.g. face) (ii) These POIs are mapped to the depth data with the help of the calibration data and biometric data (especially for plausibility checks).

(iii) The necessary distances can be calculated from the “vectors” determined in this way. The mapping is done from "2D to 3D". I.e. the POI is a vector (x, y), and after the mapping there is a vector (x, y, z) taking into account the depth data.

For example, POIs are mapped to the depth image data (including calibration data). The reference feature data, which display a biometric reference feature, can also be used for a plausibility check, for example a distribution of the horizontal distance between the pupils in the population. This serves as security. For example, a warning be generated if an extraordinary pupillary distance is determined, which deviates from a typical area.In this way, a corresponding action can be initiated, for example the glasses user can be asked to repeat the measurement, for example to record image data/sensor data again.

If, in an alternative embodiment, no depth data is available, for example because the recording device 2 does not have a corresponding sensor, the POIs are mapped using a so-called reference method. As explained above, it is provided here that the iris (especially the iris diameter) be used as a reference, for example. The biometric data will continue to be used for plausibility checks.

With regard to POI, only the pupil can be relevant, but also the iris contour (or the pixel position in the image). The horizontal distance between the pupils can be determined from the iris contour, the pupil (both as pixel positions in the RGB image) and the diameter, especially when no depth image data is available.

In particular for processing the image recordings and the user inputs, the recording device is connected to a data processing device 3, be it via a wireless or wired connection that is set up to exchange data. The data processing device 3 has one or more processors for data processing and is connected to a memory device 4 .

In particular, respective spectacle data for a large number of different spectacle models are stored in the memory device 4, the spectacle data specifying characteristic data for the various spectacle models. The data processing device is optionally connected to a production device 5, which is set up to receive parameters for glasses to be produced and to produce them automatically, in particular a glasses frame or parts thereof using a 3D printer.

In the method for automatically determining manufacturing parameters for glasses, one or more of the following parameters are determined: pupillary distance (PD), real grinding height (rSH) and virtual grinding height (vSH). With regard to an adjustment of the glasses, it can be provided that a front part, nose pads and/or temples of the glasses are adjusted during the process.

In particular, the following steps can be provided, which are explained in more detail below: data collection by means of image recordings; Determining features and reference points and applying a projection methodology. Image data with light, color and distance information about the recorded object (head of the spectacle wearer 1) are important in data collection. These objects are primarily a part of the face of the glasses user and glasses models. This data from the "2D world" is projected stably and precisely into a 3D world coordinate system using multi-stage projection and filter methods. The desired final measurement data for a spectacle lens centering, a custom manufacture of spectacle frames and/or a personalized spectacle model recommendation can then be determined from position matrices.

I. Determination of Parameters

The pupillary distance (PD) is defined as the horizontal distance in millimeters (mm) between the two pupils. The center points of both pupils are used as the starting points for the measurement. The interpupillary distance is necessary for centering the lens of single vision and progressive lenses.

The real grinding height ("real grinding height" - rGH) is the vertical distance in mm of the pupil to the inner lower edge of the spectacle frame that the spectacle wearer wears during the measurement. The grinding height is necessary in order to be able to grind progressive lenses. The virtual grinding height (vGH) is the vertical distance in mm from the pupil to the lower inner edge of the virtual frame, which the user sees projected onto his face through the screen of the mobile device. The grinding height is necessary in order to be able to grind progressive lenses.

2 shows a schematic representation of a pair of glasses 20 with the horizontal distance 21 drawn in between the pupils 22 and the real grinding height 23 drawn in. a) Features and reference points

According to one exemplary embodiment, the following points are defined and determined: pixels of interest in a two-dimensional RGB image (POI) (pupil position, frame position of real glasses, frame position of virtual glasses); 3D world coordinate system; Depth data in 2D depth image and calibration data.

Pixel of interest in a two-dimensional RGB image (POI): In order to determine the parameters PD, rGH and vGH, it is necessary to determine the exact position of the pupils, for example the deepest point of the spectacle frame, the so-called box size. For this purpose, RGB images and camera calibration data (resolution of the recorded image, camera angle information) are analyzed for the respective mobile terminal (recording device 2). The pupils are determined using pupil finder methods (image analysis algorithms) and stored in a localization vector (POI). With the help of the calibration data, the pupils can be clearly localized as pixel information (x, y) in the RGB image.

In one exemplary embodiment, the pupil finder methodology provides a two-stage method. First, a cascaded finding of the pupil (so-called "Cascaded Convolutional Neural Networks") is performed: (i) finding the face; (ii) finding the eye area; (iii) finding the iris; and (iv) finding the pupil. In a further step, plausibility data for comparison (biometric information) are also provided. A plausibility check can be carried out in each step of the method, for example using the biometric data, for example according to the following scheme: Step (1) - Has the iris been found within the eye area?; Step (2) - Has the pupil been found inside the iris?; ... ; Step (n) - Is the calculated pupillary distance within a plausible range, for example 50 to 70mm?; ... This supports the stability and accuracy of the method. Fig. 3 shows a schematic representation relating to the determination of a position for a pupil 30.

For rGH it is necessary to determine the exact position of the frame. To do this, a fixed point on the frame is defined as follows:

- Frame: The relevant point on the frame is the transition between the lens and frame of the glasses (and thus the "inner point of the glasses")

- Vertical projection: Starting from the pupil found, a vertical projection onto the frame (as defined above) is carried out

Using a line finder method, the frame fixed points (left and right side) are determined and stored in a localization vector. This vector is congruent with the camera's calibration data, so the exact pixel position of the frame fixpoints is known. A spectacle frame in an image represents a line geometry. That means you choose an algorithm that specializes in finding lines (and thus the frame) - especially where does the line begin and where does it end. We currently use the Houghs Line Finder.

Fig. 4 shows a schematic representation of a pair of glasses 40 with the real segment height 41 drawn in.

The virtual glasses are provided as a modeled 3D object to determine the virtual grinding height (vGH). Here the exact dimensions are known. The lower central point of the bridge is defined as the anchor point on the glasses.

3D world coordinate system: The definition of a world coordinate system serves as a starting point. This is a Euclidean system with an anchor point at the origin. This anchor point is defined by the lens face of the RGB camera. The orientation of the coordinate axes are defined as follows:

- x-axis: Parallel to the horizontal orientation of the mobile device

- y-axis: Parallel to the vertical orientation of the mobile device

- z-axis: parallel to the camera recording direction of the mobile device Depth data in 2D depth image Advanced mobile devices provide depth information. These grey-scale images are captured synchronously with the RGB images and together with the calibration data, the depth and RGB images can be congruently transformed. The depth images contain the distance from the depth lens to the recorded object per pixel. Each RGB and depth image pair contains various calibration data that further specify the capture. It is assumed that the following quantities

are available or can be extracted by software: angles along x-y axis for POI; Angle along y-z axis for POI; Resolution RGB image and resolution depth image.

The formalization is explained in more detail below:

1. Image Pixels of Interest in 2D RGB Image (POI)

POI = Pixels of interest = position of relevant pixels (e.g. found pupil) in the RGB image imgRGß = RGB image imgDEp = depth image p _x = position of the left pupil in the RGB image = { p ₂ = position of the right pupil in the RGB image = f _x = position of left frame in RGB image = f ₂ = position of right frame in RGB image v _x = position of left virtual frame in world coordinate system = {x _Vi , y _v , z _Vi ] v ₂ = position of right virtual frame in world coordinate system = {x _V2 ,y _V2 , z _V2 ]

2. 3D world coordinate system

World coordinate system = Euclidean coordinate system with three dimensions

Anchor point = coordinate origin = camera lens origin point = (0,0,0)

3. Depth image data in 2D depth image d _pi = position of left pupil in depth image = |x _dpi ,y _dp dp ₂ = right pupil position in depth image = d _fi = left frame position in depth image = df ₂ = right frame position in depth image d _P0I = distance of the POI to the camera lens in mm

4. Calibration data axy _POi ⁼ angle along x — y — axis for POI a _yZ poi = angle along y — z — axis for POI r ^es RGB ⁼ resolution RGB image = {res _x , res _y } res _DEP = resolution depth image = { res _x , res _y }

In one embodiment, the projection methodology comprises four steps:

- Determine angles between axes in the world coordinate system

- Determine distance to POI

- Projection of 2D input images to 3D world coordinate system

- Calculation distance i) Angle between axes in world coordinate system

The following is provided for determining the angle between axes in the world coordinate system: The two angles are required for the projection into the world coordinate system. These are available in the calibration data and can be used for the projection. ii) Determine distance to POI

The following is provided for determining the distance to the POI: From the localization of the POI in the RGB image, a connection to the depth image must be established in order to determine the distance of the POI from the camera. This is done using a mapping method that takes into account the resolution of the RGB and the depth image. The resolution of RGB and depth image is usually different. A total of three cases can be distinguished:

Case 1 : resolution of RGB image and depth image match;

Case 2: RGB image resolution is greater than depth image resolution; - Case 3: The resolution of the RGB image is smaller than the resolution of the depth image.

The aim is to derive the depth information (=distance in mm) for a pixel in the RGB image that has already been found to be relevant (e.g. the pupil or the frame):

Case 1 :

The coordinates of the POI in the RGB image are projected exactly onto the coordinates in the depth image. The corresponding distance information can be determined.

Case 2:

To describe the initial situation, FIG. 5 shows a schematic representation of nine RGB image pixels 50 (delimited by a solid line) and four depth image pixels 51 (delimited by a dashed line) and two marked RGB image pixels 52 (striped and checked area).

Two cases can occur:

- The POI is completely within a depth pixel (striped area). Congruence is determined as follows: The POI is projected onto the depth image pixel with the same coordinates.

- The POI is in more than one depth pixel (checkered area). Congruence is determined as follows: The POI is projected onto the arithmetic average of the distances of all affected depth image pixels.

The corresponding distance information can be determined.

Case 3:

To describe the initial situation, FIG. 6 shows a schematic representation of the initial situation with four RGB image pixels 60 (delimited by a solid line) and nine depth image pixels 61 (delimited by a dashed line) and one marked depth image pixel (striped area). The POI always overlaps at least three depth image pixels (blue area). Congruence is determined as follows: The POI is projected onto the arithmetic average of the distances of all overlapped depth image pixels. iii) Projection of 2D input images to 3D world coordinate system

The position in the 3D world coordinate system is calculated from the pixel distance and the two angular dimensions using a Euclidean position formula. iv) Calculation distance

The distance between two points in the 3D world coordinate system is calculated using a Euclidean distance formula:

PD: The pupil distance is given in mm and is calculated from the two pupil points in the world coordinate system. rGH: The real grinding height is specified in mm and is calculated from a pupil point and a real frame point in the 3D world coordinate system. vGH: The virtual grinding height is specified in mm and is calculated from a pupil point and a virtual frame point in the 3D world coordinate system.

A possible formalization is explained in more detail below. ii) Determine distance to POI

Case 1 dpoi ⁼ depth information from the POI in the depth image at pixel position {xp _0I ,y _P0I }

case 2 case 3 iii) Projection of 2D input images to 3D world coordinate system

(x,y,z) = map(dpoi, a _xy , a _yz ) iv) Calculation distance

One or more of the following advantages can result from the different versions:

- Increase in measurement accuracy - With the proposed solution, a measurement accuracy of less than one mm can be achieved.

- Minimization of measurement variance: With the proposed solution, a measurement variance of less than two millimeters can be achieved (one standard deviation).

- All measurements can be carried out with just a commercially available mobile device

- No real glasses are necessary to determine the grinding height, a virtual fitting on the mobile device is sufficient. This enables the recommendation of suitable glasses frames, the measurement of the necessary parameters for successful lens centering and the frame adjustment without the presence of an optician. This enables a qualitatively equivalent online purchase of glasses, as well as at stationary vending machines in the sense of a self-service principle. The return rate in current online eyewear sales can be significantly reduced through better recommendation and measurement, which has a positive effect on the profitability of online eyewear sellers and protects the environment by reducing the package quantities. II. Fitting frames for a custom-made pair of glasses a) Features and reference points

The glasses include in particular the front part, the left and right temples and nose pads. A projection methodology is used to determine the optimal frame size.

The following points are determined: canonical glasses model and modification points. Canonical in this context means the definition of the eyewear components and the sizes and shape adjustments: Canonical model = {all components, size adjustments, shape adjustments}. The specific glasses model is then calculated from this finite number of combinations and retrieved from the memory. A canonical glasses model can be defined in an embodiment via the following components: front, temples (left and right), nose pads (left and right), bending point temples (left and right) and bending angle temples (left and right).

Fig. 7 shows a schematic representation of a canonical spectacles model 70 with front part 71, temple 72, bending point 73 and bending angle 74 as well as nose pad 75.

Modification points: The frame is adjusted separately for each component using the following modification points:

Front: Width of the entire front part 71. The scaling is done with aspect ratio stability. Temple: total length of temple 72, inflexion point 73 and inflexion angle 74 b) projection methodology

The projection methodology includes two steps: front part projection method and temple projection method. i) Front part projection method

Face measurement data are collected and placed on a discrete grid, from which the size of the front part 71 can be determined. In addition, "aesthetic principles" may be considered, for example as follows: (i) women tend to wear larger glasses; and (ii) the Eyebrows should be above the glasses. Also, the pupils should not regularly be in the lower half of the lens.

Face measurement data: Pupillary distance and face width are recorded. Pupillary distance is part of the above claim. Face width is defined as the total width of the recognizable face at the pupillary level. Face width can be detected using current face recognition methods.

8 shows a schematic representation relating to a face width determination. A discrete grid 80 is created along the dimensions of pupillary distance and face width 81 respectively. For this purpose, static data is collected on these variables (distribution in the population) and an equidistant grid is formed from the distribution. The front part is divided into equidistant sizes and assigned to each grid point tuple from pupil distance and face width.

Projection: The size of the front part can be derived for a grid point tuple determined from pupil distance and face width.

Table 1 : Example table for a projection, classification S,... ,XL is exemplary and projected onto a cardinal scale S, M, L classification is for illustrative purposes. Different sizes and shapes are used for each component of the glasses, which are provided with an ID number. When determining the glasses, an optimal size and shape is selected for each component.

A possible formalization is explained in more detail below:

facial measurement data

PD = pupillary distance in mm

GB = face width in mm, measured at eye level FT = front part width in mm, measured at the widest point

Discrete grid

R _PD = grid for pupil distances = {p _1; ..., p _N }, N e NR _G B = grid for face widths = {g _1; ..., g _M }, M e NR _FT = grid for front section widths = {f _1; ..., f _L }, L e N

projection ii) stirrup projection method

Facial measurement data is collected and placed on a discrete grid, from which the temple length and the bending point can be determined. Here, too, additional aesthetic principles can be taken into account. For example, the arms of women should always be a little longer, as they often put their glasses up in their hair.

Face measurement data: Two facial feature points are located: nose-set point and ear-set point. The nosepiece point and earpiece point serve as references for the contact points of the nosepads and temples. These points can be captured using facial recognition methods. Fig. 9 shows a schematic representation of facial feature points 90. In addition, the face width is determined so that a connecting line 91 of the two determined facial feature points can be shifted so that it corresponds to the natural temple position (laterally parallel to the head - from the ear attachment point to the temple). Together with the depth data from the identified facial feature points, the length of the bracket up to the bending point can then be determined.

Discrete Grid: A discrete grid is created along each dimension "length of stirrup to bend point". For this purpose, static data is collected on these variables, ie the average distribution in the population is used and an equidistant grid is formed from this distribution. The stirrups are divided into equidistant lengths and assigned to each grid point from "Length of stirrup to bending point".

Projection: For a determined grid point from "Length of the bracket to the bending point", the bracket length and the bending point can be derived.

Table 2: Example table for a projection, classification S,... , XL is exemplary and projected onto a cardinal scale

A possible formalization is explained in more detail below:

facial measurement data

NP = nosepiece point in world coordinate system

OP = ear attachment point in world coordinate system

GB = face width in mm, measured at eye level

L = distance NP — OP in the world coordinate system (calculation analogous to the previous chapter) NOA = Projected Nose - Ear - Distance

Discrete grid

RNOA ⁼ grid for projected noses - ears - distances = {n _1; n _N }, N e NR _BL = grid for stirrup lengths up to the bending point = {b _1; , b _L }, L e N

Projection n _i i— (bi ... b _QVL ), Q <L; hey N

One or more of the following advantages may result in the various implementations. It is possible to adapt a spectacle frame to an individual head shape. All you need is a standard mobile device. An automated procedure has been created (scalable). The delivery time can be shortened by combining it with 3D printing technology. In addition, wearing comfort can be significantly increased with custom-made glasses. It also eliminates the need for subsequent adjustment of the eyeglass frame to the wearer's head, for example in the nose or ear area, which in turn eliminates the need for the presence of an optician and allows for online or stationary vending machine purchases.

III. Determine a glasses recommendation

In order to determine a recommendation, all relevant input data is recorded. This includes facial analysis data, preferences about existing objects (glasses), and visual data (images of glasses). a) Features and reference points

The following points are determined: face width, portfolio and preferences of the glasses user. Face Width: The shape of the face is a relevant aspect when it comes to the fashionable fit of glasses. The width of the face is used for this and is defined as the recognizable width of the face in mm at the level of the eyes.

Portfolio: The glasses portfolio includes all relevant glasses that are available for deriving the recommendation. Each item of this portfolio contains two pieces of information: an RGB image of the glasses and a classification based on descriptive characteristics (shape, color, style, etc.).

Preferences: Preferences are a binary vector that assigns the preference (preferred, non-preferred) to each image.

A possible formalization is explained in more detail below:

1 . face width

GB = face width in mm, measured at the level of the pupils

2.Portfolio

N = number of glasses in the total portfolio bi(j) = glasses with index i, ie {1, ... , N}, contains RGB image and classification j,j = 1, ..., P PF _Ges ⁼ {t> i, ..., b _N } = total portfolio

3. Preferences

M = number of glasses with preference, M < N

Pj = preference for glasses with index j,j e {1,...,M}, pj e {0,1} b) Projection methodology

The projection methodology includes three steps: face projection method, image projection method and image preference method. i) face projection method

Facial measurement data is collected and placed on a discrete grid, from which the recommended glasses can be determined. Face measurement data: The face width is recorded. Face width is defined as the total width of the recognizable face at the pupillary level. Face width can be detected using current face recognition methods.

Discrete grid: A discrete grid is created along each face width dimension. For this purpose, static data is collected on these variables (distribution in the population) and an equidistant grid is formed from the distribution. The glasses portfolio is divided into equidistant sizes and assigned to each grid point based on the width of the face.

Projection: The recommended glasses can be derived for a grid point determined from the width of the face. ii) Image projection method

For a fixed RGB image with recognizable glasses (input image), a trained neural network is used to perform feature extraction. As a result, a suitable similarity metric is used, which compares the input image with every image in the portfolio and sorts it according to confidence.

The similarity metric is provided with a confidence level from which applies "these glasses are similar to the input image", so that a recommendation sub-portfolio can be derived. iii) Image preference method

For a set of fixed RGB images with recognizable glasses (input images from the capture device 2), a trained neural network is used to perform feature extraction. As a result, a similarity metric is used, which compares the input images with each image for existing glasses and sorts them according to confidence. A preference vector can be used as an additional input parameter, which indicates one or more preferences determined from the input images. Such a preference can concern qualitative factors for the user, for example one or more factors from the following group: sunglasses or regular glasses, color, material, brand and the like.

The similarity metric is provided with a confidence level from which applies "these glasses are similar to the input image and preferred", so that a recommendation sub-portfolio can be derived. A possible formalization is explained in more detail below:

1 . face projection method

R _G B = grid for face widths = {g _1; ..., gn _} , NGN

2. Image projection method imginput = input image with recognizable glasses

NN(img _in p _Ut ) = trained neural network with input image a = {«(!), = NN(imgi _nput ) = sorted confidence vector

3. Image preference method imginput = {imginput^ ■■■< img _inputM } = M input images with recognizable glasses Preference vector for all images j = 1, ... , M Trained neural network with input images 1, ..., M and preference vector = NN(img _in p _Ut , p) = sorted confidence vector

One or more of the following advantages can result from the different versions: Everything in one mobile device; Consideration of all relevant visual data and consideration of preferences. So far, only self-selection was possible online, but this was insufficient, since spectacle wearers do not know how their head size compares to the rest of the spectacle wearer population. In concrete terms, this means that nobody says of themselves: “I have a statistically significantly large head”. Recommendations based purely on taste are inadequate when it comes to eyewear. Head and face shape recognition can be automated without the presence of an optician, for example online or at a self-service machine, and also combined with deep learning-based preference recognition.

The features disclosed in the above description, the claims and the drawing can be important both individually and in any combination for the realization of the various embodiments.

Claims

24

Method for automatically determining production parameters for glasses, the following being provided in one or more processors set up for data processing:

- acquiring head image data for at least part of a head of a spectacle user;

- determining a head parameterization for at least the part of the head of the spectacles user, wherein

- the head parameterization for an adjustment of glasses relevant head parameters for at least the part of the head of the glasses user and

- the head parameters include at least one lens grinding parameter and at least one spectacle support parameter;

- Providing a spectacle parameterization for the spectacles, wherein the spectacle parameterization for the adjustment of the spectacles for the spectacles user shows relevant spectacle parameters; and

- Execution of a data mapping for the head parameterization and the glasses parameterization, in which case for the adjustment of the glasses for the glasses user at least one glasses parameter is adjusted according to an assigned head parameter and/or at least one further glasses parameter is determined. Method according to Claim 1, characterized in that the following is also provided:

- acquiring RGB head image data for at least the part of the head of the glasses user;

- providing calibration data indicative of a calibration of an image capture device used to capture the RGB header image data; and

- Determination of the at least one lens grinding parameter using the RGB header image data and the calibration data by means of image data analysis, wherein a localization vector assigned to the pupils is determined, which indicates an image pixel position for the pupils. Method according to Claim 2, characterized in that the following is also provided:

- Providing depth image data and

- determining the at least one lens grinding parameter using the RGB header image data, the depth image data and the calibration data. Method according to Claim 2 or 3, characterized in that the following is also provided:

- Providing reference feature data that indicate a biometric reference feature for the glasses user;

- determining the at least one lens grinding parameter using the RGB header image data, the reference feature data and the calibration data. Method according to at least one of the preceding claims, characterized in that the at least one spectacle parameter or the at least one further spectacle parameter comprises a real grinding height for spectacles designed as progressive lenses, the following also being provided:

- Determining at least one fixed point of a real spectacle frame of real spectacles, which indicates a transition between a spectacle lens and the spectacle frame, wherein a localization vector assigned to the at least one fixed point of the spectacle frame is determined, which indicates an image pixel position for the at least one fixed point of the spectacle frame; and

- vertically projecting a pupil mark indicative of the pupils onto the eyeglass frame. Method according to at least one of the preceding claims, characterized in that the at least one spectacle parameter or the at least one further spectacle parameter comprises a virtual grinding height for spectacles designed as progressive lenses, the following also being provided:

- Providing a 3D model of virtual glasses, from which a glasses parameterization for the virtual glasses is determined;

- determining at least one fixed point of a spectacle frame of the virtual spectacles, which indicates a transition between a spectacle lens and the spectacle frame, wherein a localization vector assigned to the at least one fixed point of the spectacle frame is determined, which indicates an image pixel position for the at least one fixed point of the spectacle frame; and

- vertically projecting a pupil mark indicative of the pupils onto the eyeglass frame. 7. The method as claimed in claim 6, characterized in that the 3D modeling of the virtual glasses is selected from a large number of different virtual glasses, for which a respective 3D modeling is stored in a memory device.

8. The method according to at least one of the preceding claims, characterized in that the following is also provided:

- determining a 3D coordinate system;

- mapping the head parameterization for at least part of the head of the glasses user and the glasses parameterization into the 3D coordinate system and

- Determining one or more of the following parameters in the 3D coordinate system: horizontal distance between the pupils, face width at the level of the pupils, real grinding height and virtual grinding height.

9. The method according to at least one of the preceding claims, characterized in that based on the head parameterization for the adjustment of the glasses, a temple length for temples of the glasses and a bending point for the temples are determined.

10. The method according to at least one of the preceding claims, characterized in that the head parameters include one or more lens grinding parameters from the following group: horizontal interpupillary distance and head width.

11. The method according to at least one of the preceding claims, characterized in that the head parameters include one or more parameters from the following group: face width at the level of the pupils, nose width, nose attachment point, ear attachment point, distance between nose and ears and cheek contour.

12. Apparatus for automatically determining manufacturing parameters for eyeglasses, comprising one or more processors configured for data processing, configured to:

- receiving head image data for at least part of a head of a glasses user;

- the head parameterization for an adjustment of glasses relevant head parameters for at least the part of the head of the glasses user and 27

- Execution of a data mapping for the head parameterization and the glasses parameterization, in which case for the adjustment of the glasses for the glasses user at least one glasses parameter is adjusted according to an assigned head parameter and/or at least one further glasses parameter is determined.