EP4453905A1 - Method and apparatus for determining at least one optical parameter of at least one spectacle lens - Google Patents
Method and apparatus for determining at least one optical parameter of at least one spectacle lensInfo
- Publication number
- EP4453905A1 EP4453905A1 EP22836297.6A EP22836297A EP4453905A1 EP 4453905 A1 EP4453905 A1 EP 4453905A1 EP 22836297 A EP22836297 A EP 22836297A EP 4453905 A1 EP4453905 A1 EP 4453905A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- information
- feature
- spectacle lens
- optical parameter
- modified
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Definitions
- the present invention relates to a computer-implemented method for a training of a computer-implemented method for determining at least one optical parameter of at least one spectacle lens, a computer-implemented method for determining at least one optical parameter of at least one spectacle lens, a data training apparatus for a training of a process for determining at least one optical parameter of at least one spectacle lens, a data determining apparatus for determining at least one optical parameter of at least one spectacle lens, a computer program, a trained machine-learning model, a use of a carrier element, and a method for producing at least one spectacle lens.
- WO 2017/174 791 Al discloses a method and a device for measuring the topography and/or the gradients and/or the curvature of an optically active surface of an object.
- the device allows the object to be arranged in a receiving region having a contact surface for contact with the object.
- Inside the device there is a plurality of point light sources that provide light which is reflected at the surface to be measured of an object arranged in the receiving region.
- the device comprises at least one camera having an objective assembly and an image sensor for detecting a brightness distribution which is produced on a light sensor by the light of the point light sources reflected at the surface to be measured.
- this technique learns priors over geometric transformations and regularities of the 3D world through end-to-end training from image pairs.
- the proposed method performs matching in real-time on a modem GPU and can be readily integrated into modern SfM or SLAM systems.
- WO 2021/140204 Al discloses a method for retrieving at least one optical parameter of an ophthalmic lens comprises: obtaining (10) an image of a first and second patterns by using an image capture device located at a first position; from that image, obtaining (12) a first set of data from at least apart of the first pattern that is seen through the lens by the image capture device and obtaining (14) a second set of data from at least apart of the second pattern that is seen directly i.e. outside the lens by the image capture device; retrieving (16) the at least one optical parameter by using the first and second sets of data and taking account of relative positions of the image capture device, the lens and the first and second patterns.
- WO 2020/216 788 Al or US 2021/382329 Al discloses a method, a device, and a computer program for determining at least one optical parameter of a spectacle lens, and to a method for manufacturing the spectacle lens using said at least one optical parameter, wherein the optical parameter denotes a value for a property of the spectacle lens which is adjusted during manufacture of the spectacle lens in order to achieve an intended correction of the refractive error of at least one eye of a user of the spectacle lens.
- the method comprises the following steps: a) capturing at least one image using the spectacle lens; and b) determining at least one optical parameter of the spectacle lens by image processing the at least one image, wherein the at least one image comprises an eye portion including at least one eye and/or a face portion adjacent to at least one eye of a user of the spectacle lens.
- the determination of the at least one optical parameter of the spectacle lens is carried out by using a classification or regression of at least one image statistics parameter using a machine learning method, in particular a neural network, wherein before step b) a training of the neural network takes place, whereby a trained neural network is obtained, wherein a plurality of training data sets is used for training the neural network, each training data set comprising a pair of at least one image and at least one comparison image, and at least one associated optical parameter of the lens or at least one optical effect related to the lens said spectacle lens.
- a computer-implemented method for a training of a computer- implemented method for determining at least one optical parameter of at least one spectacle lens a computer-implemented method for determining at least one optical parameter of at least one spectacle lens, a data training apparatus for a training of a process for determining at least one optical parameter of at least one spectacle lens, a data determining apparatus for determining at least one optical parameter of at least one spectacle lens, a computer program, a trained machine-learning model, use of a carrier element and a method for producing at least one spectacle lens having the features of the independent claims.
- Preferred embodiments, which can be implemented in an isolated fashion or in any arbitrary combination, are listed in the dependent claims or throughout the following description.
- the present invention relates to a computer-implemented method for a training of a computer-implemented method for determining at least one optical parameter of at least one spectacle lens, wherein the method comprises the steps of a) providing at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature, wherein the at least one modified feature is generated from at least one original feature by a view through the at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature; and b) using the at least one recorded data set as an input for a procedure for determining a deviation between the at least one recorded data set and at least one reference data set comprising information about the at least one original feature, wherein the procedure uses at least one machine learning algorithm configured to combine the information about the at least one original feature and the information about the at least one modified feature, wherein the at least one machine learning algorithm is trained by comparing the at least one machine learning algorithm
- the indicated steps may, preferably, be performed in the given order, commencing with step a) and finishing with step c). However, any or all of the indicated steps may also be repeated several times and/or preformed concurrently in part.
- At least step b) and c) may be computer-implemented.
- Step a) may also be computer-implemented.
- the term “computer-implemented method” refers to a method which involves a programmable apparatus, in particular, a computer, a computer network, or a readable medium carrying a program, whereby at least one of the steps of the method, specifically at least one of steps b) and c), and optionally also step a), are performed by using at least d computer program.
- the at least one computer program may be accessible by an apparatus, which may be adapted for performing the method via a network, such as via an in-house network or via internet.
- the present method can, thus, being performed on a programmable apparatus, which is configured for this purpose, such as by providing a computer program, which is configured for such a purpose.
- training refers to a process of determining parameters of an algorithm of a machine and deep learning model on the training data set.
- the training may comprise at least one optimization or tuning process, wherein a best parameter combination is determined.
- the procedure for determining a deviation may improve in such a way that the capability to determine the correct at least one optical value improves. In that sense the procedure is at least part of a machine learning model.
- training data sets may be used. These training data sets may comprise real data or simulated data.
- the term "determine” or any grammatical variation thereof refers to a process of generating representative results which are, typically, denoted as “data”.
- the data comprise information, which are related to at least one optical parameter of a spectacle lens designated to correct at least one refractive error of at least one eye of a person.
- a different term, such as ’’user”, “subject”, or “wearer”, may also be applicable.
- the term “optical parameter” relates to at least one property of the at least one optical lens, preferably of the at least one spectacle lens, which is capable of altering the incident light beam through the at least one optical lens, preferably the at least one spectacle lens.
- the Standard particularly Sections 3.10, 3.12 and 3.13, deals with optical parameters of spectacle lenses, in particular with regard to spherical and astigmatic properties of different kinds of spectacle lenses.
- the term “spectacle lens” refers to an optical lens, which is used for determining and/or correcting a defective vision of a wearer of the optical lens, wherein the optical lens is carried in front of the eye of the user, thereby avoiding a direct contact with the eye of a user, according to the Standard, 3.5.2.
- the optical lens has a lens body which comprises a particular shape that is configured for this purpose.
- the spectacle lens as used herein may be a unifocal lens having a single focal point.
- lens refers to a combination of a right spectacle lens which is configured to correct at least one refractive error of the right eye of a user and a left spectacle lens which is configured to correct at least one refractive error of the left eye of a user.
- spectacle refers to an element which comprises a pair of spectacle lenses and a spectacle frame, wherein each spectacle lens is received by the spectacle frame.
- step a) at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature is provided.
- data set refers to a collection of “data”.
- data refers to an item, such as a numeric item, which comprises at least one piece of information.
- information may refer to a content being comprised by the piece of data.
- the term “recorded image” refers to a picture or a photograph, in particular as taken by a camera.
- the term “providing” or any grammatical variation thereof refers to receiving a data set for being available to a purpose, such as a further procedure.
- the data set may be provided on a data carrier, such as a disc or an USB flash drive; alternatively or in parallel, the data set my be provided in electronic form via email or a remote computer program receiving the data set to the procedure.
- the provided data set may be stored before it is inputted into the procedure, or alternatively or in parallel, the data set may be inputted directly into the procedure.
- the at least one modified feature is generated from at least one original feature by a view of the at least one original feature through at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature.
- feature refers to a distinct characteristic of the recorded image or the optional reference image.
- the feature may be a coherent section within the respective image that is recognizable as it distinguishes from neighboring sections, for example in color, grey scale, image gradients or intensity differences.
- the feature may be a pattern or a part of a pattern, which may particularly be a pattern having a repeated or parallel uniform occurrence, in example a uniform repetition of a part of the pattern.
- original feature refers to the feature as it is, particularly a feature free from a refraction, more particularly a feature free from refraction caused by the at least one spectacle lens.
- modified feature refers to a feature that is influenced by a refraction, in particular a refraction caused by the at least one spectacle lens.
- influences refers to the fact that the refraction of the spectacle lens is the cause for the generation of the modified pattern, while the refraction depends on the at least one optical parameter.
- spectacle lenses having at least one different optical parameter may generate different modified patterns.
- the at least one optical parameter of the spectacle lens causes typically a distinct refraction that modifies the original feature in an unambiguous manner, which can be identified in the modified feature.
- the modification may be photometrical, particularly the modification may cause a magnification of the original feature, and/or the modification may deform the original pattern, in particular the observed coordinates of the original pattern.
- the at least one data set is used as an input for a procedure for determining a deviation between the at least one recorded data set and at least one reference data set comprising information about the at least one original feature.
- the term “input” refers to data or a data set, which is transferred to a method step or computer program or a routine of a computer program. The input may start an execution of the method step or the computer program or the routine of the computer program.
- the term “deviation” refers to a modification of the information about the modified pattern caused by the influence of the refraction, in particular due to the at least one spectacle lens, more particularly due to the at least one optical parameter of the at least one spectacle lens.
- the deviation is determined in relation to the original pattern, in particular to a segment of the original pattern that corresponds to a segment of the modified pattern.
- the term “process” refers to at least a part of the computer-implemented method.
- the procedure uses at least one machine learning algorithm, wherein the at least one machine learning algorithm is trained by comparing the at least one modified feature to the at least one original feature or the at least one original feature to the at least one modified feature, whereby the at least one optical parameter of the at least one spectacle lens is determined in the procedure.
- machine learning - o - algorithm refers to a process of applying artificial intelligence to automatically generate a statistical model.
- a machine learning algorithm configured to generate the desired model based on a large number of training data sets can, preferably, be used.
- the term “training” or grammatical variations thereof refers to a process of building a trained model, in particular determining parameters, in particular weighs, of the model.
- the training may comprise at least one optimization or tuning process, wherein a best parameter combination is determined.
- comparing refers to a process of analyzing the refraction caused by the at least one optical parameter of the spectacle lens on the original feature by investigating the deviation between the original feature and the modified feature. As a result, of the comparison the at least one optical parameter is determined.
- the procedure uses at least one machine learning algorithm configured to combine the information about the at least one original feature and the information about the at least one modified feature.
- the term “combined” or any grammatical combination thereof refers to a correlation of first pieces of information comprised by a set with second pieces of information comprised by a different set.
- a first set may be the at least one recorded data set.
- a second set may be the at least one reference data set.
- the term “correlate” refers to a mutual interaction of information, which is implemented in such a manner that the interaction generates an output.
- the machine learning algorithm may comprise nodes.
- the correlation may be implemented in such a way that information about at least a portion of the at least one modified feature in a first origin node and information about at least a part portion of the at least one original feature in a second origin node is passed to a common node and processed, in particular by a concatenation of vectors to a common piece of information.
- the common piece of information may then be comprised by the output of the second machine learning algorithm.
- steps a) and b) are repeated until a difference between the at least one optical parameter as determined for the at least one spectacle lens and the at least one optical parameter as known for the at least one spectacle lens is below a threshold.
- the steps a) and b) may be repeated using a training data set.
- the term “training data set” may refer to a data set on which the machine and deep learning model is trained.
- the term “threshold” refers to a maximal deviation between at the least one optical parameter determined in step b) and at the least one optical parameter as known.
- the one optical parameter is “known” as it may be available to computer-implemented method.
- step c) the procedure for determining the deviation between the at least one recorded data set and the at least one reference data set is obtained and applicable in the computer-implemented method for determining the at least one optical parameter of the at least one spectacle lens.
- the computer-implemented method for determining the at least one optical parameter of the at least one spectacle lens is described in the following.
- combining the information about the at least one original feature and the information about the at least one modified feature comprises correlating the information about the at least one modified feature comprised by the at least one recorded data set and the information about the at least one original feature comprised by the at least one reference data set, wherein the correlation is a mutual interaction of both information, which is implemented in such a manner that the interaction generates an output data comprising the information about the at least one modified feature and the information about the at least one original feature.
- the mutual interaction of both information may be generated by using a specific architecture of the machine learning algorithm, particularly as provided by a graph neural network and/or a recurrent neural network.
- the output data, specifically of the respective machine learning algorithm may be generated by using a machine learning algorithm that considers the information about the at least one original feature and the information about the at least one modified feature information to generate at least one common output of the machine learning algorithm.
- The, particularly second, machine learning algorithm may comprise at least one output node, wherein the at least one output node may comprise common information comprising, specifically processed, information about the original feature and, specifically processed, information about the modified feature.
- the information comprised by the at least one output node may be input the at least one input node of a further, particularly third, machine learning algorithm.
- the at least one optical parameter as known for the at least one spectacle lens may further be an input for the procedure for determining a deviation between the at least one recorded data set and at least one reference data set.
- the at least one optical parameter is also available for the computer-implemented method.
- steps a) and b) can be repeated for more than one data set. These data sets may be referred to as “training data set”.
- the present invention relates to a computer-implemented method for determining at least one optical parameter of at least one spectacle lens, wherein the method comprises the steps of: a) providing at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature, wherein the at least one modified feature is generated from at least one original feature by a view through the at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature; and d) using the at least one recorded data set as an input for a procedure for determining a deviation between the at least one recorded data set and at least one reference data set comprising information about the at least one original feature as obtained during a training phase, whereby the at least one optical parameter of the at least one spectacle lens is determined in the procedure; and e) outputting the at least one optical parameter of the at least one spectacle lens.
- the term “outputting” refers to issuing or displaying at least one piece of data, in particular of output data.
- the output data comprises information about the at least one optical parameter of the at least one spectacle lens.
- the term “training phase” refers to a training of the procedure trained by the computer-implemented method for a training of a computer-implemented method for determining at least one optical parameter of at least one spectacle lens as described herein.
- the procedure for determining a deviation between the at least one recorded data set and at least one reference data set comprising information about the at least one original feature as obtained during a training phase is used, the computer-implemented method for determining at least one optical parameter of at least one spectacle lens may require the use of the machine learning algorithm defined in the computer-implemented method for a training of a computer- implemented method for determining at least one optical parameter of at least one spectacle lens.
- the at least one reference data set may further be an input for the procedure for determining a deviation between the at least one recorded data set and at least one reference data set.
- the at least one reference data set is then available for the computer- implemented method.
- the at least one recorded data set may comprise information about the at least one modified feature received from a plurality of recorded images.
- the term “plurality” refers to an assembly of at least two units, in particular a unit of at least two recorded images.
- the at least two recorded images of the plurality of the recorded images may show the at least one modified feature from a different perspective.
- the term “perspective” refers to a point of view, particularly to a point of view onto an image, more particularly to a point of view onto the at least one recorded image.
- the information about the at least one modified feature may be comprised in the at least one recorded data set at least one of:
- the at least one coordinate of the modified feature may be determined from the at least one recorded image and/or the at least one portion of the recorded image.
- the recorded data set may comprise at least one of:
- a pointer to a database storing at least one of: o the at least one recorded image; o the at least one portion of the recorded image; or o the at least one coordinate of the modified feature.
- the information about the at least one original feature is comprised in the recorded data as at least one of:
- the reference data set comprises at least one of:
- a pointer to a database storing at least one of: o the at least one reference image; o the at least one portion of the reference image; or o the at least one coordinate of the original feature.
- the term “database” refers to an organized collection of data sets or data.
- the data sets or data is stored and may be accessed electronically.
- the database can store the reference data set.
- the data or data set in the database may be accessed multiple time.
- the data or data set may be not altered after the accessing.
- the information about the at least one original feature may be used several times, in particular for several runs of the computer-implemented method for determining at least one optical parameter of at least one spectacle lens, especially for several runs of the computer- implemented method for determining at least one optical parameter of different spectacle lenses. After determination of the information on at least one original feature, corresponding data may be stored in a database.
- the term “pointer” refers an object that stores an address of and/or within a database.
- the information about the original feature is determined from at least one of:
- the at least one reference image and/or the at least one portion of the reference image shows the at least one original feature from a viewing direction vertical from above.
- the information about the original feature is constructed from the at least one recorded image and/or the at least one portion of the recorded image from a visible segment of the at least one original feature for a segment of the at least one original feature being covered by the at least one spectacle lens in the at least one recorded image.
- the term “constructed” refers to a method for predicting a portion of the original feature, in particular a portion of a pattern having a uniform occurrence as defined above, wherein the constructed portion is not shown on the recorded image which might be covered by the at least one spectacle lens.
- the construction may be performed by geometrical consideration.
- the at least one recorded data set may have information only about a relevant segment of the at least one recorded image, wherein the relevant segment is determined via image segmentation by partitioning the at least one recoded image into the relevant segment and a negligible segment, wherein the relevant segment is a section of the at least one recorded image showing the at least one modified feature, and wherein the negligible segment is a further section of the at least one recorded image being free of the at least one modified feature.
- segment refers to a part of the at least one modified or the at least one original feature.
- image segmentation refers to a process of partitioning a digital image, in particular the recorded image, into multiple content related segments. The content related segments are chosen according to a criterion, such as color, grey-scale, image gradients or based on a reprojection error of the modified features w.r.t the original feature.
- the at least one reference data set may have only information about the at least one original feature corresponding to a corresponding segment, wherein the corresponding segment is the part of the information about the at least one original feature that corresponds to the information about the at least one modified feature comprised in the relevant segment.
- the image segmentation is performed by at least one of:
- an “image processing method” is a computer based method that performs the image segmentation. Such a method may be thresholding as well as edge or region based segmentation.
- the term “machine learning based method” is a particular image processing method that comprises at least one machine learning algorithm as defined in the above.
- the information about the at least one original feature and the information about the at least one modified feature may be adapted to one another by at least one of:
- the term “adapted” refers to an at least partial or full alignment of a first and a second parameter, wherein the first parameter is given for a first set, particularly the information about the at least one original feature, and the second parameter is given for a second set, particularly the information about the at least one modified feature.
- the alignment may be considered as partial in case a deviation of the first parameter and the second parameter is decreased but both parameters do not match, while the alignment may be considered as full in case the parameters match.
- the term Huaweiscaling“ refers to a transformation that enlarges or diminishes the information about the at least one original feature or the information about the at least one modified feature.
- the transformation may be a linear transformation.
- adjusting a segment refers to a method of aligning said segments to each other. In particular, to aligning said segments in such a way that they cover each other having the largest possible intersection.
- adjusting a perspective may refer to rotational or translational or a rotational and translational transformations, in particular of the point of view.
- the term “editing the information” may refer to modifying the information, particularly modifying the information in such a way that influences of camera parameters on the information about the original feature or the information about the modified feature are compensated for, particularly compensated for in such a way that the influence on the information about the original feature or the information about the modified feature has no impact on the determination of the at least one optical parameter using the procedure for determining the deviation between the at least one recorded data set and the at least one reference data set.
- a “camera parameter” is a parameter used in a camera model to describe a mathematical relationship between three-dimensional directions of a point, in particular a point of view, in the depicted scene from which the light comes from and the two-dimensional pixels of its projection onto the image plane.
- the at least one recorded data set may be given in pixels m m and/or in direction vectors d m and/or the at least one reference data set is given in pixels m 0 and/or in direction vectors d 0 .
- the term “pixel” or “picture cell” or “ picture element” refers to an individual color value or gray-scale value of an image, particularly a recorded image or a reference image.
- the pixels m m and/or the pixels m 0 may be two- dimensional vectors.
- the term “direction vector” refers to a vector pointing from a point of view to a pixel, particularly to a pixel of the respective pattern.
- the direction vectors d m and/or the direction vectors d 0 may be three-dimensional vectors.
- the information about the at least one original feature or the information about the at least one modified feature may be scaled by an image resolution of the at least one recorded image or the at least one reference image.
- the pixels m m and/or the pixels m 0 may be scaled by the image resolution of the at least one recorded image and/or the at least one reference image showing the original feature.
- image resolution is the detail an image holds.
- the number of pixel counts may be referred to as the image resolution.
- the image resolution may be given by a set of two positive integer numbers, with the first number being the number of pixel columns (width) and the second being the number of pixel rows (height).
- the image resolution may be given by the product of both numbers.
- the information about the at least one original feature or the information about the at least one modified feature may be scaled by normalizing the direction vectors d m and/or the direction vectors d 0 to a unitary vector length, in particular vector length of 1.
- the vector length of 1 may be given in arbitrary units.
- the section of the information about the at least one original feature section and/or the section of the information about the at least one modified feature may be adjusted by subtracting an anchor pixel m a or an anchor direction vector d a , respectively.
- anchor pixel or “anchor direction vector” refers to a common pixel or direction vector, observable in the at least one recorded image and the at least one reference image.
- the common pixel or direction vector may be a center pixel or a center direction vector.
- the perspective of the information about the at least one original feature may be adjusted by considering a camera pose of a camera that took the at least one reference image; and/or wherein the perspective of the information about the at least one modified feature is adjusted by considering a camera pose of a camera that took the at least one recorded image.
- the term “camera pose” refers to a position and an orientation of a camera with respect to a reference coordinate system, in particular the coordinate system of the information about the at least one original feature or of the information about the at least one modified feature.
- the camera pose may be given by the rotation, such as by roll, pitch, and yaw or as a 3D position, such as by x, y, z, of the camera with respect to the reference coordinate system.
- the process to determine the camera pose may weigh a part of the respective image that shows the at least one original feature above a part of the respective image that shows the at least one modified feature.
- the term “weigh” or any grammatical variation thereof refers to an evaluation of an influence of individual variables of a mathematical model, for example with regard to their importance or reliability. It results in more important and/or more reliable elements having a greater influence on the result.
- the variables may be multiplied with a parameter having a high absolute value for giving the variable more importance or reliability or a low parameter for giving the variables less importance or reliability.
- the process to determine the camera pose of the respective image may use at least one of:
- the term “random sample consensus algorithm” refers to a resampling algorithm for estimating a model within a set of measurements with outliers and gross errors.
- the term “perspective-n-point method” is a method that can cope with problem of estimating the pose of a calibrated camera given a set of n 3 dimensional points in the world and their corresponding two-dimensional projections in the image.
- Typical perspective-n-point methods are algebraic perspective-3 -point (AP3P), efficient perspective- n-point (EPnP), and sequential quadratic perspective-n-point (SQPnP).
- the random sample consensus algorithm can be used with said methods to make the solution for the camera pose more robust to outliers.
- the term “nonlinear pose optimization method” refers to an algorithm that starts with an approximate value of the camera pose and carries out a nonlinear optimization to determine the camera pose, in particular carries out an iterative least squares adjustment.
- the term “Kalman filtering” refers to an algorithm that uses a series of measurements observed over time, including statistical noise and other inaccuracies, and produces estimates of unknown variables that tend to be more accurate than those based on a single measurement alone, by estimating a joint probability distribution over the variables for each timeframe.
- the camera pose may be determined relative to the coordinate system of the respective feature.
- the at least one camera parameter may be determined by at least one of:
- the term “database” refers to an organized collection of data sets or data comprising information.
- the data or data set is stored and accessed electronically.
- the database can store the at least one camera parameter.
- the data or data set in the database may be accessed multiple time.
- the information about the at least one camera parameter may be used several times, in particular for several different runs of the a computer-implemented method for determining at least one optical parameter of at least one spectacle lens, in particular for different several runs using different spectacle lenses. After calibrating the camera the information may be stored in a database.
- the at least one camera parameter may be at least one of:
- the term “camera constant” refers to a distance of a camera coordinate system to an imaging sensor.
- the term “principal point“ refers to a point on the image plane onto which the perspective center is projected. It may also be the point from which the focal length of the lens is measured.
- the term “ distortion parameter” refers to a parameter related to at least one lens distortion, such as a radial- symmetric distortion and/or a tangential-asymmetric distortion.
- the term “aspect ratio” refers to a deviation of an actual shape of at least one camera pixel from an assumed shape, in particular a deviation of the actual shape of the at least one camera pixel from a square shape, in particular deviation of the actual shape of the at least one camera pixel due to a compression of the at least one camera pixel.
- the procedure for determining a deviation between the at least one modified feature and the at least one original feature may use a first machine learning algorithm configured to embed the information about the at least one modified feature and the information about the at least one original feature.
- a first machine learning algorithm configured to embed the information about the at least one modified feature and the information about the at least one original feature.
- the term "embedding" or grammatical variations thereof refers to a correlation of first pieces of information comprised by a set with second pieces of information comprised by the same set, wherein the second pieces of information may be different or identical with the first pieces of information. Consequently, the at least one modified feature may be correlated with itself and, thus, independent of a correlation of the information about the at least one original feature, or vice versa. The correlation generates the output of the first machine learning algorithm.
- the correlation may be implemented in such a way that information about at least a first portion of the at least one modified feature in a first origin node and information about at least a second portion of the at least one modified feature in a second origin node may be passed to a common node and processed to a common piece of information.
- the same implementation may be performed for information about the at least one original feature.
- the common piece of information may then be comprised by the output of the first machine learning algorithm.
- the common piece of information generated for the at least one modified feature and the common piece of information generated for the at least one original feature may be different from each other.
- The, particularly first, machine learning algorithm may comprise at least two output nodes, at least one first output node may comprise processed information about only the original feature and at least one second output node may comprise processed information about only the modified feature.
- the information comprised by the at least two output nodes may be input in at least two input node of a further, particularly second, machine learning algorithm.
- first or “second” or “third” are considered as a description of an element without specifying an order or a chronological sequence and without excluding a possibility that other elements of the same may be present.
- a reportedfirst“ element is different from a recruitedsecond“ element and außthird elements This holds for any possible permutation.
- the first machine learning algorithm may embed the pixels m m and/or the direction vectors d m representing the information about the at least one modified feature into feature vectors f m and the pixels m 0 or the direction vectors d 0 , respectively, representing the information about the at least one original feature into feature vectors f 0 .
- feature vector may refer to a vector having one or more dimensions.
- the first machine learning algorithm may be implemented as at least one of:
- Multi-Layer Perceptron neural network having at least one layer
- recurrent neural network having at least one layer.
- Multi-Layer Perceptron neural network or “MLP neural network” specifically refers to a class of feedforward artificial neural networks.
- a feedforward neural network is an artificial neural network wherein connections between the nodes do not form a cycle.
- layers refers to a collection of nodes.
- the Multi-Layer Perceptron neural network has an input layer comprising neurons into which information is inputted, in particular the information about the at least one modified feature or the at least one original feature.
- the Multi-Layer Perceptron neural network has an output layer comprising neurons which can be read out or output information, in particular processed information about the at least one modified feature or the at least one original feature.
- the Multi-Layer Perceptron neural network may comprise at least one hidden layer comprising neurons, which process information.
- the number of neurons may be ascending, specifically in such a way that more nodes of a destination layer receiving an information are present compared to a number of nodes of an origin layer providing the information.
- the term “convolutional neural network” refers to a class of artificial neural networks.
- a convolutional neural network includes at least one layer that performs at least one convolution.
- the at least one convolution may be performed by a hidden layer.
- the term “recurrent neural network” refers to a class of artificial neural networks where connections between nodes form a graph along a temporal sequence. The nodes are connected via directed edges, so that data comprising information can pass from a node to a connected node but not vice versa.
- the at least one machine learning algorithm configured to combine the information about the at least one original feature and the information about the at least one modified feature may be a second machine learning algorithm.
- the second machine learning algorithm is implemented as at least one of:
- the graph neural network comprises nodes for storing information and edges for passing information between the nodes;
- graph neural network describes a class of machine learning algorithm designed to perform inference on data described by graphs by providing a data structure corresponding to a structure of the graph.
- the data structure consists of two components: nodes and edges, wherein the nodes correspond to vertices of the graph structure, and wherein the edges correspond to edges of the graph structure.
- node refers to an element in the data structure used to process and store data comprising information.
- edges refers to a connection of two nodes along which data comprising information can pass. A node may thus be considered as an element in the data structure in which at least to edges meet. An image may be considered as a special graph.
- the second machine learning algorithm may comprise an attention mechanism weighing the edges.
- the term “weighing” refers to the evaluation of individual influencing variables of a mathematical model, for example with regard to their importance or reliability. Consequently, more important or more reliable elements may have a greater influence on the result.
- the edges may be weighed directly, in particular by multiplying the information passing from one edge to another edge by a trainable attention parameter.
- an input into the second machine learning algorithm may store the information about the at least one modified feature in a first type of the nodes, wherein an input into the second machine learning algorithm may store the information about the at least one original feature in a second type of nodes. The information about the modified feature may be input into the first type of nodes.
- the information about the original feature may be input into the second type of nodes.
- the information about the at least one modified feature may be inputted into the second machine learning algorithm and stored in the first type of the nodes are the feature vectors f m and/or wherein the information about the at least one original feature may be inputted into the second machine learning algorithm and stored in the second type of the nodes are the feature vectors f 0 .
- the term “inputting” or grammatical variations thereof refers to entering an input, in particular input data comprising information.
- the information passed between the nodes may be passed iteratively, wherein in a first iterative step the information passed between the nodes may be passed between the nodes of the same type and in a second step the information passed between the nodes may be passed between the nodes of a different type.
- iteratively describes that an iteration process is repeating the same or similar actions several times in order to approach a solution or a specific goal. The same or similar action may also be referred to as iterative steps. In each iterative step,
- - information stored in an origin node may be passed via the edges to all connected nodes;
- the passed information may be weighed with the attention parameter:
- the correlated information may be stored in the connected nodes.
- all nodes of one type may be involved.
- Each node of one type may be connected via so called self-edges to each other node of the same type, particularly.
- Each node of one type may further be connected to each other node of the other type, particularly via so called cross-edges.
- the edges may be undirected so that the information passed between a first node and a second node that are connected via an edge can be passed from the first node to the second node and vice versa. All edges of the graph neural network may be undirected.
- the procedure for determining a deviation between the at least one modified feature and the at least one original feature may use a third machine learning algorithm configured to perform a regression for determining the at least one optical parameter.
- regression refers to a statistical analysis tool that aims to determine a relationship between an input, in particular input data or input dataset, and a statistical model to determine output parameters.
- the statistical model may be fitted onto the input data.
- the input data may not change during the regression.
- the input may be the output of the at least one or second machine learning algorithm configured to combine the information about the at least one original feature and the information about the at least one modified feature.
- the input may further be the output of the second machine learning algorithm, in particular the graph neural network.
- the statistical model may be represented by the third machine learning algorithm.
- the output may be the at least one optical parameter of the spectacle lens.
- the third machine learning algorithm may be implemented as at least one of:
- Multi-Layer Perceptron neural network having at least two layers; in particular wherein the at least two layers have an same number of neurons;
- the term “fullysupport-vector machine model“ refers to a supervised learning model having associated learning algorithms that analyze data for classification and regression analysis.
- the term “random forest method“ refers to an ensemble learning method for classification, regression and other tasks that operates by constructing a multitude of decision trees at training time.
- the term “bulboosting algorithm“ refers to an ensemble metaalgorithm for primarily reducing bias, and also variance in supervised learning, and a family of machine learning algorithms that convert weak learners to strong ones.
- the at least two layers may comprise an output layer generating the output, wherein the output layer may have a number of neurons that may equal a number of the least one optical parameter.
- the output layer may output the determined at least one optical parameter.
- Each neuron of the output layer may output a different optical parameter.
- the input for the procedure for determining the deviation between the at least one modified feature and the at least one original feature may be an input for the first machine learning algorithm.
- an output of the first machine learning algorithm may be an input for the second machine learning algorithm.
- an output of the second machine learning algorithm may be an input for the third machine learning algorithm.
- the term “input” refers to data or a data set comprising information which is provided to a method step or computer program or a routine of the computer program. The input may start the execution of the method step or the computer program or the routine of the computer program.
- output refers to data or a data set comprising information outputted by the method step or the computer program or the routine of the computer program, in particular after internal calculation.
- the output of the second machine learning algorithm may be generated by correlating information stored in the nodes of the second machine learning algorithm. This step may be referred to as “graph readout”.
- the information may be correlated in such a way that it may serve as an input for the third machine learning algorithm.
- the information may be combined in such a way that the number on individual combined information meets the number of input parameters.
- the information may be correlating by at least one of
- the term “combined” refers to a combination of node data comprising the information.
- the third machine learning algorithm may output the at least one optical parameter.
- the at least one optical parameter may be at least one of:
- cylinder axis refers to a direction of the principal meridian of a lens whose vertex power is chosen for reference.
- all method steps can be performed by using a single processing device, such as a computer, especially a stand-alone computer, or an electronic communication unit, specifically a smartphone, a tablet, a personal digital assistant, or a laptop.
- the single processing device may be configured to exclusively perform at least one computer program, in particular at least one line of computer program code configured to execute at least one algorithm, as used in at least one of the methods according to the present invention.
- the computer program as executed on the single processing device may comprise all instructions causing the computer to carry out the at least one of the methods according to the present invention.
- At least one method step can be performed by using at least one remote processing device, especially selected from at least one of a server or a cloud computer, which is not located at the site of the user when executing the at least one method step.
- the computer program may comprise at least one remote portion to be executed by the at least one remote processing device to carry out the at least one method step.
- the computer program may comprise at least one interface configured to forward and/or receive data to and/or from the at least one remote portion of the computer program.
- the present invention relates to a data training apparatus for a training of a process for determining at least one optical parameter of at least one spectacle lens
- the data training apparatus is configured to carry out a computer-implemented method for a training of a computer-implemented method for determining at least one optical parameter of at least one spectacle lens comprising the steps: a) providing at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature, wherein the at least one modified feature is generated from at least one original feature by a view through the at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature; and b) using the at least one recorded data set as an input for a procedure for determining a deviation between the at least one recorded data set and at least one reference data set comprising information about the at least one original feature, wherein the procedure uses at least one machine learning algorithm configured to combine the information about
- the present invention relates to a data determining apparatus for determining at least one optical parameter of at least one spectacle lens, wherein the data determining apparatus is configured to carry out a computer-implemented method for determining at least one optical parameter of at least one spectacle lens comprising the steps of a) providing at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature, wherein the at least one modified feature is generated from at least one original feature by a view through the at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature; and d) using the at least one recorded data set as an input for a procedure for determining a deviation between the at least one recorded data set and at least one reference data set comprising information about the at least one original feature as obtained during a training phase, whereby the at least one optical parameter of the at least one spectacle lens is determined in the procedure; and e) outputting
- the term “data apparatus”, in particular the “data training apparatus” or “the data determining apparatus” refers to an apparatus which is designated for determining the data related to refractive values from the at least one input file, which may, preferably, be provided to the processing device by at least one input interface, and to provide the data related to the refractive values, such as for further processing, preferably by at least one input interface, in particular by using a system as described below in more detail.
- the processing device may comprise at least one of an integrated circuit, in particular an application-specific integrated circuit (ASIC), or a digital processing device, in particular at least one of a digital signal processor (DSP), a field programmable gate array (FPGA), a microcontroller, a microcomputer, a computer, or an electronic communication unit, specifically a smartphone, a tablet, a laptop.
- ASIC application-specific integrated circuit
- DSP digital signal processor
- FPGA field programmable gate array
- microcontroller e.g., a microcontroller
- microcomputer e.g., a computer
- an electronic communication unit specifically a smartphone, a tablet, a laptop.
- Further components may be feasible, in particular at least one of a data acquisition unit, a preprocessing unit, or a data storage unit.
- the processing device may, preferably, be configured to perform at least one computer program, in particular at least one computer program performing or supporting the determining of the data related to the refractive values from the at least one input file, wherein the processing of the data be performed in a consecutive, a parallel, or a combined fashion.
- the present invention relates to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method as described.
- computer program product refers to the program as a tradable product.
- the product may generally exist in an arbitrary format, such as in a paper format, or on a computer-readable data carrier.
- the computer program product may be distributed over a data network, such as the internet.
- the present invention relates to a trained machine-learning model in accordance with the method as described.
- the term “trained machinelearning model” refers to a model for determining at least one optical parameter of at least one spectacle lens which was trained by using at least one training dataset, also denoted as “training data”.
- the training may have involved the determination of the at least one optical parameter of at least one spectacle lens by using at least one machine learning algorithm as described elsewhere herein.
- the present invention related to a use of a carrier element in performing the method as described herein, wherein the carrier element comprises a surface showing at least one original feature.
- the term “surface” refers to a visible surface area of the carrier element.
- the surface may further comprise a marked lay-on area for at least one spectacle having the at least one spectacle lens.
- the term “lay-on area” refers to an area that is provided on the surface of the carrier element for placing the at least one spectacle, in particular the frame of the spectacle. The lay-on area may be rectangular; however, a different form may also be feasible.
- the carrier element may be at least one of
- the term “board” refers to a plane object.
- the board may be an aluminum board or a hard polymer board.
- the term “screen” refers to an electronic visual display device.
- the present invention relates to a method for producing at least one spectacle lens, wherein the producing of the at least one spectacle lens comprises processing at least one lens blank by using data related to refractive values as determined by carrying out a step of a) providing at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature, wherein the at least one modified feature is generated from at least one original feature by a view through the at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature; and d) using the at least one recorded data set as an input for a procedure for determining a deviation between the at least one recorded image data set and at least one reference image data set comprising information about the at least one original feature as obtained during a training phase, whereby the at least one optical parameter of the at least one spectacle lens is determined in the procedure; and e) outputting the at least one optical parameter of the at least one
- the computer-implemented method for a training of a computer- implemented method for determining at least one optical parameter of at least one spectacle lens exhibit the following advantages of a more efficient and more reliable approach for determining the at least one optical parameter.
- the information about the at least one reference image may be known, in particular the at least one coordinate of the original feature may be known. In this case, only one recorded image is required and it is not necessary to capture a reference image.
- the errors in the determined at least one optical parameter may then be reduced as the information about the at least one reference image may be more accurate. This may be because the information about the at least one reference image may be determined by an algorithm. Another reason may be that the information about the at least one reference image may be given in a resolution higher than the resolution of the at least one recorded image. The effort to determine the at least one optical parameter is thereby reduced. Also the information about the at least one reference image may be determined statistically.
- the at least one original feature may be a pattern
- this pattern may be optimized in such a way that the at least one optical parameter may be determined more reliably, in particular with a higher accuracy.
- the terms “have”, “comprise” or “include” or any arbitrary grammatical variation thereof are used in a non-exclusive way. Thus, these terms may refer to both a situation in which, besides the feature introduced by these terms, no further features are present in the entity described in this context and to a situation in which one or more further features are present.
- the expressions “A has B”, “A comprises B” and “A includes B” may both refer to a situation in which, besides B, no other element is present in A (i.e. a situation in which A solely and exclusively consists of B) and to a situation in which, besides B, one or more further elements are present in entity A, such as element C, elements C and D or even further elements.
- the terms “preferably”, “more preferably”, “particularly”, “more particularly”, or similar terms are used in conjunction with optional features, without restricting alternative possibilities.
- features introduced by these terms are optional features and are not intended to restrict the scope of the claims in any way.
- the invention may, as the skilled person will recognize, be performed by using alternative features.
- features introduced by "in an embodiment of the invention” or similar expressions are intended to be optional features, without any restriction regarding alternative embodiments of the invention, without any restrictions regarding the scope of the invention and without any restriction regarding the possibility of combining the features introduced in this way with other features of the invention.
- Embodiment 1 A computer-implemented method for a training of a computer- implemented method for determining at least one optical parameter of at least one spectacle lens, wherein the method comprises the steps of: a) providing at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature, wherein the at least one modified feature is generated from at least one original feature by a view through the at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature; b) using the at least one recorded data set as an input for a procedure for determining a deviation between the at least one recorded data set and at least one reference data set comprising information about the at least one original feature, wherein the procedure uses at least one machine learning algorithm configured to combine the information about the at least one original feature and the information about the at least one modified feature, wherein the at least one machine learning algorithm is trained by comparing the at least one modified feature to the at least one
- Embodiment 2 The method according to the preceding embodiment, wherein combining the information about the at least one original feature and the information about the at least one modified feature comprises correlating the information about the at least one modified feature comprised by the at least one recorded data set and the information about the at least one original feature comprised by the at least one reference data set, wherein correlating comprises a mutual interaction of both information, which is implemented in such a manner that the interaction generates an output.
- Embodiment s The method according to any of the preceding embodiments, wherein the at least one optical parameter of at least one spectacle lens is further an input for the procedure for determining a deviation between the at least one recorded data set and at least one reference data set.
- Embodiment 4 The method according to any of the preceding embodiments, wherein steps a) and b) are repeated for more than one data set.
- a computer-implemented method for determining at least one optical parameter of at least one spectacle lens comprising the steps of: a) providing at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature, wherein the at least one modified feature is generated from at least one original feature by a view through the at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature; d) using the at least one recorded data set as an input for a procedure for determining a deviation between the at least one recorded data set and at least one reference data set comprising information about the at least one original feature as obtained by any one of the preceding embodiments, whereby the at least one optical parameter of the at least one spectacle lens is determined in the procedure; and
- Embodiment 6 The method according to any of the preceding embodiments, wherein the at least one reference data set is further an input for the procedure for determining a deviation between the at least one recorded data set and at least one reference data set.
- Embodiment 7 The method according to any one of the preceding embodiments, wherein the at least one recorded data set comprises information about the at least one modified feature received from a plurality of the recorded images.
- Embodiment 8 The method according to the preceding embodiment, wherein at least two recorded images the at least one modified feature from a different perspective.
- Embodiment 9 The method according to any one of the preceding embodiments, wherein the information about the at least one modified feature is comprised in the at least one recorded data set at least one of:
- Embodiment 10 The method according to any one of the preceding embodiments, wherein the recorded data set comprises at least one of:
- a pointer to a database storing at least one of: o the at least one recorded image; o the at least one portion of the recorded image; or o the at least one coordinate of the modified feature.
- Embodiment 11 The method according to any one of the preceding embodiments, wherein the information about the at least one original feature is comprised in the recorded data as at least one of:
- Embodiment 12 The method according to any one of the preceding embodiments, wherein the reference data set comprises at least one of:
- a pointer to a database storing at least one of: o the at least one reference image; o the at least one portion of the reference image; or o the at least one coordinate of the original feature.
- Embodiment 13 The method according to any one of the preceding embodiments, wherein the information about the original feature is determined from at least one of:
- Embodiment 14 The method according to the preceding embodiment, wherein the at least one reference image and/or the at least one portion of the reference image shows the at least one original feature from a viewing direction vertical from above.
- Embodiment 15 The method according to any one of the two preceding embodiments, wherein the information about the original feature is constructed from the at least one recorded image and/or the at least one portion of the recorded image from a visible segment of the at least one original feature for a segment of the at least one original feature being covered by the at least one spectacle lens in the at least one recorded image.
- Embodiment 16 The method according to any one of the preceding embodiments, wherein the at least one recorded data set has only information about the at least one modified feature received from a relevant segment of the at least one recorded image, wherein the relevant segment is determined via image segmentation by partitioning the at least one recoded image into the relevant segment and a negligible segment, wherein the relevant segment is a section of the at least one recorded image showing the at least one modified feature, and wherein the negligible segment is a further section of the at least one recorded image being free of the at least one modified feature.
- Embodiment 17 The method according to the preceding embodiment, wherein the at least one reference data set has only information about the at least one original feature corresponding to a corresponding segment, wherein the corresponding segment is the part of the information about the at least one original feature that corresponds to the information about the at least one modified feature comprised in the relevant segment.
- Embodiment 18 The method according to any one of the two preceding embodiments, wherein the image segmentation is performed by at least one of:
- a geometric procedure in particular by at least one of: o investigating at least one back projection error in the at least one recorded image or in the at least one reference image; or o defining a plurality of segments of the recorded image showing the at least one original feature and/or the at least one modified feature; and comparing the at least one shown original feature and/or the at least one shown modified feature to a known original feature and determining a deviation between the known original feature and at least one shown original feature and/or the at least one shown modified feature for a plurality of segments; and identifying the relevant segment as the possible segment, wherein the deviation is largest.
- Embodiment 19 The method according to any one of the preceding embodiments, wherein the information about the at least one original feature and the information about the at least one modified feature are adapted to one another by at least one of:
- Embodiment 20 The method according to any one of the preceding embodiments, wherein the at least one recorded data set is given in at least one of pixels m m and/or direction vectors d m , wherein the at least one reference data set is given in at least one of pixels m 0 or direction vectors d 0 .
- Embodiment 21 The method according to any one of the two preceding embodiments, wherein the information about the at least one original feature or the information about the at least one modified feature is scaled by an image resolution of the at least one recorded image or the at least one reference image.
- Embodiment 22 The method according to the preceding embodiment, wherein the pixels m m and/or the pixels m 0 are scaled by the image resolution of the at least one recorded image and/or the at least one reference image showing the original feature.
- Embodiment 23 The method according to the preceding embodiment 13, wherein the information about the at least one original feature or the information about the at least one modified feature is scaled by normalizing the direction vectors d m and/or the direction vectors d 0 to a unitary vector length, in particular vector length of 1.
- Embodiment 24 The method according to any one of the five preceding embodiments, wherein the segment of the information about the at least one original feature section and/or the section of the information about the at least one modified feature is adjusted by subtracting an anchor pixel m a or an anchor direction vector d a .
- Embodiment 25 The method according to any one of the six preceding embodiments, wherein the perspective of the information about the at least one original feature section is adjusted by considering a camera pose of a camera that took the at least one reference image; and/or wherein the perspective of the information about the at least one modified feature is adjusted by considering a camera pose of a camera that took the at least one recorded image.
- Embodiment 26 The method according to the preceding embodiment, wherein the process to determine the camera pose weighs a part of the respective image that shows the at least one original feature above a part of the respective image that shows the at least one modified feature.
- Embodiment 27 The method according to the two preceding embodiments, wherein the process to determine the camera pose of the respective image uses at least one of:
- Embodiment 28 The method according to any one of the three preceding embodiments, wherein the camera pose is determined relative to the coordinate system of the respective feature.
- Embodiment 29 The method according to any one of the ten preceding embodiments, wherein for editing the information about the at least one original feature or the information about the at least one modified feature by considering at least one camera parameter the at least one camera parameter is determined by at least one of:
- Embodiment 30 The method according to any one of the preceding eleven embodiments, wherein the at least one camera parameter is at least one of:
- Embodiment 31 The method according to any one of the preceding embodiments, wherein the procedure for determining a deviation between the at least one modified feature and the at least one original feature uses a first machine learning algorithm configured to embed the information about the at least one modified feature and the information about the at least one original feature.
- Embodiment 32 The method according to the preceding embodiment, wherein the first machine learning algorithm embeds the pixels m m and/or the direction vectors d m representing the information about the at least one modified feature into feature vectors f m and the pixels m 0 or the direction vectors d 0 respectively, representing the information about the at least one original feature into feature vectors f 0 .
- Embodiment 33 The method according to any one of the two preceding embodiments, wherein the first machine learning algorithm is implemented as at least one of
- Multi-Layer Perceptron neural network having at least two layers, in particular wherein the at least two layers have an ascending number of neurons starting from the input;
- recurrent neural network having at least one layer.
- Embodiment 34 The method according to any one of the preceding embodiments, wherein the at least one machine learning algorithm configured to combine the information about the at least one original feature and the information about the at least one modified feature is a second machine learning algorithm.
- Embodiment 35 The method according to the preceding embodiment, wherein the second machine learning algorithm is implemented as at least one of
- the graph neural network comprises nodes for storing information and edges for passing information between the nodes;
- Embodiment 36 The method according to the preceding embodiment, wherein the second machine learning algorithm comprises an attention mechanism weighing the edges.
- Embodiment 37 The method according to any one of the two preceding embodiments, wherein an input into the second machine learning algorithm stores the information about the at least one modified feature in a first type of the nodes, wherein an input into the second machine learning algorithm stores the information about the at least one original feature in a second type of nodes.
- Embodiment 38 The method according to the preceding embodiment, wherein the information about the at least one modified feature inputted into the second machine learning algorithm and stored in the first type of the nodes are the feature vectors f m and/or wherein the information about the at least one original feature inputted into the second machine learning algorithm and stored in the second type of the nodes are the feature vectors f 0 .
- Embodiment 39 The method according to any one of the two preceding embodiments, wherein the information passed between the nodes is passed iteratively, wherein in a first iterative step the information passed between the nodes is passed between the nodes of the same type and in a second step the information passed between the nodes is passed between the nodes of a different type.
- Embodiment 40 The method according to any one of the five preceding embodiments, wherein the edges are undirected so that the information passed between a first node and a second node that are connected via an edge can be passed from the first node to the second node and vice versa.
- Embodiment 41 The method according to any one of the preceding embodiments, wherein the procedure for determining a deviation between the at least one modified feature and the at least one original feature uses a third machine learning algorithm configured to perform a regression for determining the at least one optical parameter.
- Embodiment 42 The method according to the preceding embodiment, wherein the third machine learning algorithm is implemented as at least one of:
- Multi-Layer Perceptron neural network having at least two layers; in particular wherein the at least two layers have an same number of neurons;
- Embodiment 43 The method according to the preceding embodiment, wherein the at least two layers comprise an output layer generating the output, wherein the output layer has a number of neurons that equals a number of the least one optical parameter.
- Embodiment 44 The method according to any one of the thirteen preceding embodiments, wherein the input for the procedure for determining the deviation between the at least one modified feature and the at least one original feature is an input for the first machine learning algorithm.
- Embodiment 45 The method according to any one of the eleven preceding embodiments, wherein an output of the first machine learning algorithm is an input for the second machine learning algorithm.
- Embodiment 46 The method according to any one of the five preceding claims, wherein an output of the second machine learning algorithm is an input for the third machine learning algorithm.
- Embodiment 47 The method according to any one of the thirteen preceding embodiments, wherein the output of the second machine learning algorithm is generated by correlated information stored in the nodes of the second machine learning algorithm.
- Embodiment 48 The method according to the preceding embodiment, wherein the information is correlated by at least one of:
- Embodiment 49 The method according to any one of the eight preceding embodiments, wherein the third machine learning algorithm outputs the at least one optical parameter.
- Embodiment 50 The method according to any one of the preceding embodiments, wherein the at least one optical parameter is at least one of:
- Embodiment 51 A data training apparatus for a training of a process for determining at least one optical parameter of at least one spectacle lens, wherein the data training apparatus is configured to carry out a computer-implemented method for a training of a computer-implemented method for determining at least one optical parameter of at least one spectacle lens comprising the steps: a) providing at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature, wherein the at least one modified feature is generated from at least one original feature by a view through the at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature; b) using the at least one recorded data set as an input for a procedure for determining a deviation between the at least one recorded data set and at least one reference data set comprising information about the at least one original feature, wherein the procedure uses at least one machine learning algorithm configured to combine the information about the at least one original feature
- Embodiment 52 The data training apparatus according to the preceding embodiment, wherein the data training apparatus is configured to carry out the method according to any one of embodiments 1 to 4 or 6 to 50.
- Embodiment 53 A data determining apparatus for determining at least one optical parameter of at least one spectacle lens, wherein the data determining apparatus is configured to carry out a computer-implemented method for determining at least one optical parameter of at least one spectacle lens comprising the steps of a) providing at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature, wherein the at least one modified feature is generated from at least one original feature by a view through the at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature; d) using the at least one recorded data set as an input for a procedure for determining a deviation between the at least one recorded data set and at least one reference data set comprising information about the at least one original feature as obtained by any one of the preceding method embodiments, whereby the at least one optical parameter of the at least one spectacle lens is determined in the procedure; and e) outputting the at least
- Embodiment 54 A data determining apparatus according to the preceding embodiment, wherein the processing device is configured to carry out the method of any one of embodiments 6 to 50.
- Embodiment 55 A computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any one of the preceding method embodiments.
- Embodiment 56 A trained machine-learning model trained in accordance with the method of any of embodiments 1 to 4 or 6 to 50.
- Embodiment 57 Use of a carrier element for performing the method of any one of the preceding method embodiments, wherein the carrier element comprises a surface showing at least one original feature.
- Embodiment 58 Use of a carrier element according to the preceding embodiment, wherein the surface further comprises a marked lay-on area for at least one spectacle having the at least one spectacle lens.
- Embodiment 59 Use of a carrier element according to any of the two preceding embodiments, wherein the carrier element is at least one of:
- Embodiment 60 A method for producing at least one spectacle lens, wherein the producing of the at least one spectacle lens comprises processing at least one lens blank by using data related to refractive values as determined by carry out a step of a) providing at least one recorded data set comprising information about the at least one modified feature received from at least one recorded image showing at least one part of the spectacle lens and at least one modified feature, wherein the at least one modified feature is generated from at least one original feature by a view through the at least one part of the spectacle lens, wherein at least one optical parameter of at the least one spectacle lens influences generating the at least one modified feature; d) using the at least one recorded data set as an input for a procedure for determining a deviation between the at least one recorded data set comprising information about the at least one modified feature and at least one reference data set comprising information about the at least one original feature as obtained by any one of the preceding method embodiments, whereby the at least one optical parameter of the at least one spectacle lens is determined in the procedure; and e) outputting the at least one optical parameter
- Figure 1 illustrates a recorded image of a modified feature and an original feature
- Figure 2 illustrates an implementation of a procedure for determining a deviation between the modified feature and the original feature to determine at least one optical parameter of a spectacle lens
- Figures 3a and 3b each illustrates a further implementation of the procedure
- Figure 4 illustrates an image editing routing for a data set inputted into the procedure.
- Figure 1 shows two recorded images 100 that each show a spectacle 110 of a person having a frame 112 and a spectacle lens 114.
- the recorded images 100 on the right side is discussed . This discussion applies correspondingly to the recorded images 100 on the left side.
- the frame 112 is placed on a marked lay-on area 124 of a visible surface 122 of a carrier element 120.
- the depicted carrier element 120 is a board.
- the carrier element 120 may be a piece of paper or a screen.
- An original feature 130 is printed on the surface 122 of the carrier element 120.
- This original feature 130 is modified by a view through the spectacle lens 114 in such a manner that the recorded image 100 shows a modified feature 132.
- the modified feature 132 is generated by the refraction of the spectacle lens 114.
- the refraction depends on the at least one optical parameter 140 of the spectacle lens 114, wherein the optical parameter 140 may be a spherical power or a cylinder power and a cylinder axis.
- the at least one optical parameter 140 can be determined by comparing the modified feature 132 to the original feature 130.
- the spectacle lens 114 depicted in the recorded image 100 on the left side of Figure 1 has the following optical parameter 140: +2 sphere.
- the spectacle lens 114 in the recorded image 100 on the right side has the following optical parameter 140: -8 sphere; 2 cylinder, 30° cylinder axis.
- the refraction of the spectacle lenses 114 causes typically an unambiguous modified pattern 130 from which the at least one optical parameter 140 can be determined.
- the at least one optical parameter 140 of the spectacle lens 114 is determined using a trained procedure 200 for determining a deviation between the modified feature 132 and the original feature 130 running as a computer program on a data determining apparatus 400.
- the procedure 200 may be considered as a trained machine learning model.
- the procedure 200 is depicted in Figure 2.
- a data set 250 is used as an input for the machine-learning model 200.
- the dataflow is generally depicted by arrows, which point from an output to an input.
- the data set 250 comprises at least a recorded data set 252 comprising information about the at least one modified feature 132.
- the recorded data set 252 is determined from the recorded image 100, as explained later in Figure 4.
- the data set 250 may optionally further comprise a reference data set 254 comprising information about the at least one original feature 130.
- the at least one recorded data set may be given in of two-dimensional pixels m m or, alternatively, in three-dimensional direction vectors d m .
- the at least one reference data set may be given in two-dimensional pixels m 0 or three-dimensional direction vectors d 0 , respectively.
- the recorded data set 252 may comprise information about the at least one modified feature 132 determined from two recorded images 100, showing the modified pattern 132 from different perspectives.
- the procedure 200 comprises a first machine learning algorithm 210 configured to embed the information about the modified feature 132 and the information about the at least one original feature 130 independently for one another.
- the data set 250 is inputted into the first machine learning algorithm 210.
- the correlating involves generating an first output 214 and a second out 216, which considers an interaction between different parts of the modified feature 132 or different parts of the original feature 130.
- the first machine learning algorithm 210 may be implemented as a Multi-Layer Perceptron neural network with four layers 212, wherein the layers 212 have an ascending number of neurons 218 starting from the input.
- the first machine learning algorithm 210 generates feature vectors f m and feature vectors f 0 .
- the feature vectors f m represent information about the at least one modified feature 132.
- the feature vectors f m are comprised in the first output 214.
- the feature vectors f m represent information about the at least one original feature 130.
- the feature vectors f 0 are comprised in the second output 216.
- the feature vectors f m may be generated by embedding the two dimensional pixels m m using the first machine learning algorithm 210 and, independent form this, the feature vectors f 0 may be generated by embedding the two dimensional pixels m 0 using the first machine learning algorithm 210.
- the first output 214 and the second output 216 is inputted into a second machine learning algorithm 220.
- the second machine learning algorithm 220 combines the information about the at least one original feature 130 and the information about the at least one modified feature 132. In that sense, information comprised in the first output 214 is correlated with the information comprised in the second output 216 and a common output is generated, which depends on both information.
- the second machine learning algorithm 220 may be implemented as a graph neural network comprising two types of nodes 222, 224 for storing information and two types of edges 226, 228 for passing data comprising information between the nodes 222, 224.
- the first output 214 comprising information about the modified feature 132 is inputted into the first type of nodes 222 and the second output 216 comprising information about the original feature 130 is inputted into the second type of nodes 224. Therefore, the feature vectors f m , are stored in the first type of nodes 222 and the feature vectors f 0 are stored in the second type of the nodes 224.
- Data comprising information is passed between the nodes 222, 224 via the edges 226, 228 in an iterative manner.
- the data is passed between the nodes 222, 224 of the same type via self-edges 226 and in a second iterative step the information is passed between the nodes 222, 224 of a different type via cross edges 228.
- each node 222, 224 is connected via self-edges 226 with all node 222, 224 of the same type.
- each node 222, 224 is connected via-cross edges 228 with all nodes 222, 224 of the other type.
- the edges 226, 228 are undirected so that the information passed between a first node 222, 224 and a second node 222, 224 that are connected via one of the edges 226, 228 can be passed from the first node 222, 224 to the second node 222, 224 and vice versa.
- the second machine learning algorithm 220 comprises an attention mechanism weighing the edges 222, 224.
- the output of the second machine learning algorithm 220 is generated by correlating the information stored in the nodes 222, 224.
- the information may be correlated by calculating a sum and/or calculating a mean value and/or determining a maximum value and/or determining a minimal value.
- the procedure 200 further comprises a third machine learning algorithm 230 configured to perform a regression on the data read out from the second machine learning algorithm 220 for determining the at least one optical parameter 140.
- the output of the second machine learning algorithm 220 is inputted into the third machine learning algorithm 230.
- the third machine learning algorithm 230 may be implemented as a Multi-Layer Perceptron neural network.
- the depicted Multi-Layer Perceptron neural network has three layers 232, 234.
- the three layers 232, 234 have the same number of neurons 236.
- an output layer 234 that generates the output of the third machine learning algorithm 230 has a number neurons 236 that equals a number of the least one optical parameter 140 as each neuron 236 of the output layer outputs a different optical parameter 140.
- the output of the third machine learning algorithm 230 is the least one optical parameter 140.
- Figure 3a shows a different architecture of the procedure 200, which comprises only the second machine learning algorithm 220.
- the data set 250 is inputted in the second machine learning algorithm 220 and the at least one optical parameter 140 is read out from the second machine learning algorithm 220.
- the procedure 200 comprises the second machine learning algorithm 220 and the third machine learning algorithm 230.
- the data set 250 is inputted into the second machine learning algorithm 220 and the at least one optical parameter 140 is read out from the third machine learning algorithm 230.
- the procedure 200 and in particular the respective first machine learning algorithm 210 and/or second machine learning algorithm 220 and/or third machine learning algorithm 230 are trained to a trained machine learning model on a data training apparatus 410.
- the data set 250 provided for purpose of this training further comprises the at least one optical parameter 140 as known for the at least one spectacle lens. Therefore, trainable parameter of the procedure 200 are optimized until a difference between the at least one optical parameter 140 as determined by the procedure 200 and the at least one optical parameter 140 as known is below a threshold, in particular until a parameter set is determined for which the difference is minimal.
- the steps a) and b) may be repeated for more than one training data set.
- the recorded data set 252 may be processed by an image editing routing 300 as depicted in Figure 4.
- the reference data set 254 may be processed by the image editing routing 300.
- the reference image is inputted into the image editing routing 300.
- the recorded image 100 may be processed as recorded data set 252, wherein the recorded data set 252 comprises the pixels m m .of the recorded image 100.
- the camera pose of a camera that took the at least one recorded image 100 may be determined.
- a part of the recorded image 100 that shows the at least one original feature 130 is weigh above a part that shows the at least one modified feature 132 by using a perspective-n-point method in combination with a random sample consensus algorithm.
- the camera pose is determined relative to the coordinate system of the recorded image 100.
- a camera to world transformation may be determined, wherein T is the camera to world transformation , R is the camera to world rotation and X is the position of the camera in the world coordinate system.
- At least one camera parameter of the camera may be determined. Therefore the at least one camera parameter is queried from a database; alternatively the camera may be calibrated.
- the camera parameter may be a camera constant and/or a principal-point location and/or at least one distortion parameter and/or an aspect ratio and/or a skew of a sensor axis. Accordingly, a camera imaging function TI is known. Any influence of the at least one camera parameter on the recorded image 100 may be compensated for.
- the recoded image 100 is partitioned in a relevant segment 322 and a negligible segment 324.
- the relevant segment 322 is a section of the recorded image 100 showing the at least one modified feature 132 due to the view through the spectacle lens 114
- the negligible segment 324 is a different section of the at least one recorded image 100 being free of the at least one modified feature 132.
- the at least one recorded data set 252 has only information about the relevant segment 322 of the at least one recorded image.
- m m is a two-dimensional pixel of the recorded image
- TI is the camera imaging function
- N is a function that normalizes the direction vectors d m , in particular to a vector length of 1.
- the direction vectors d m are given the coordinate system of the camera.
- a transformation into the coordinate system of the at least one original feature 130 may be performed using the function wherein R is the camera to world rotation.
- the reference data set 254 may be prepared by querying information about the original feature 130 from a database.
- the information about the original feature 130 may be determined from the recorded image 100, wherein the part of the at least one original feature 130 covered by the at least one spectacle lens 114 in the at least one recorded image 100 is extrapolated from the at least one original feature 130.
- the information about the original feature 130 may be determined by using a reference image showing the at least one original feature 130. This reference image may show the at least one original feature 130 from a viewing direction vertical from above. Any influence of a camera parameter on the reference image may also be compensated for.
- the camera parameters may be determined in analogy to the determination of the at least one camera parameter of the camera. As a result, the three-dimensional coordinates of all observed board points X corresponding to the information about the original feature 130 are known.
- the perspective of the information comprised in the reference data set 254 may further be adjusted to the perspective of the information comprised in the recorded data set 252 by considering the camera pose.
- the information comprised in the recorded data set 252 may further be edited in such a manner that it comprises only information of a corresponding segment 326 that corresponds to the relevant segment 322.
- m 0 n R w c * X + T w c , wherein it is the camera imaging function, T ⁇ , is the world to camera transformation, is the world to camera rotation and X are the known three-dimensional board points.
- the reference data set 254 may comprise the two-dimensional pixels m 0 and/or three-dimensional direction vectors d 0 .
- the information comprised in the recorded data set 254 may be scaled by the resolution of the recorded image 100.
- the information comprised in the reference data set 254 may be scaled to the scale of the information comprised in the recorded data set 252. In that sense, the reference data set 254 may be scaled by the resolution of the recorded image 100 and/or the reference image.
- the relevant segment 322 may be adjusted by subtracting an anchor pixel m a or an anchor direction vector d a , respectively.
- the corresponding segment 326 is adjusted to the relevant segment 322.
- an anchor pixel m a or an anchor direction vector d a may be subtracted from the information about the corresponding segment 334, respectively.
- the anchor pixel m a may be a center pixel or the anchor direction vector d a may be a center direction vector of the relevant segment or the corresponding segment, respectively.
- the anchor pixel m a may be calculated as wherein nt; refers to the two-dimensional pixels m 0 or m m and N is the total count of the respective pixels.
- the anchor direction vector d a may be calculated as wherein d t refers to the two-dimensional pixels d 0 or d m and N is the total count of the respective pixels.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Ophthalmology & Optometry (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Eyeglasses (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP21216738.1A EP4202869A1 (en) | 2021-12-22 | 2021-12-22 | Method and apparatus for determining at least one optical parameter of at least one spectacle lens |
| PCT/EP2022/087238 WO2023118296A1 (en) | 2021-12-22 | 2022-12-21 | Method and apparatus for determining at least one optical parameter of at least one spectacle lens |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| EP4453905A1 true EP4453905A1 (en) | 2024-10-30 |
Family
ID=79018663
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP21216738.1A Withdrawn EP4202869A1 (en) | 2021-12-22 | 2021-12-22 | Method and apparatus for determining at least one optical parameter of at least one spectacle lens |
| EP22836297.6A Withdrawn EP4453905A1 (en) | 2021-12-22 | 2022-12-21 | Method and apparatus for determining at least one optical parameter of at least one spectacle lens |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| EP21216738.1A Withdrawn EP4202869A1 (en) | 2021-12-22 | 2021-12-22 | Method and apparatus for determining at least one optical parameter of at least one spectacle lens |
Country Status (2)
| Country | Link |
|---|---|
| EP (2) | EP4202869A1 (en) |
| WO (1) | WO2023118296A1 (en) |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102016106535B4 (en) | 2016-04-08 | 2019-03-07 | Carl Zeiss Ag | Apparatus and method for measuring a surface topography |
| EP3730998A1 (en) | 2019-04-23 | 2020-10-28 | Carl Zeiss Vision International GmbH | Determination of at least one optical parameter of a spectacle lens |
| US12442718B2 (en) * | 2020-01-09 | 2025-10-14 | Essilor International | Method and system for retrieving an optical parameter of an ophthalmic lens |
-
2021
- 2021-12-22 EP EP21216738.1A patent/EP4202869A1/en not_active Withdrawn
-
2022
- 2022-12-21 WO PCT/EP2022/087238 patent/WO2023118296A1/en not_active Ceased
- 2022-12-21 EP EP22836297.6A patent/EP4453905A1/en not_active Withdrawn
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023118296A1 (en) | 2023-06-29 |
| EP4202869A1 (en) | 2023-06-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210012093A1 (en) | Method and apparatus for generating face rotation image | |
| US11651581B2 (en) | System and method for correspondence map determination | |
| JP5830546B2 (en) | Determination of model parameters based on model transformation of objects | |
| Zheng et al. | Single-image vignetting correction from gradient distribution symmetries | |
| US20060285741A1 (en) | Direct vision sensor for 3D computer vision, digital imaging, and digital video | |
| JP2024507089A (en) | Image correspondence analysis device and its analysis method | |
| JPH10124658A (en) | Image distortion correction method of camera using neural network | |
| CN116228780B (en) | Silicon wafer defect detection method and system based on computer vision | |
| CN118972696A (en) | Automatic lens switching method and system based on intelligent light sensitivity test | |
| CN120374870B (en) | Vision-based traditional Chinese medicine defect detection method | |
| CN116958419A (en) | A binocular stereo vision three-dimensional reconstruction system and method based on wavefront coding | |
| CN119722531A (en) | An integrated image restoration method and system based on image mask modeling | |
| Woodford et al. | Large scale photometric bundle adjustment | |
| CN115147709A (en) | A 3D reconstruction method of underwater target based on deep learning | |
| JP6482130B2 (en) | Geometric verification apparatus, program and method | |
| CN120807827B (en) | A binocular 3D scanning system and method with texture mapping function | |
| CN112329845A (en) | Method and device for replacing paper money, terminal equipment and computer readable storage medium | |
| CN120540321A (en) | Robot visual servo control system and control method based on dense tracking | |
| CN119048696B (en) | Dynamic self-adaptive 3D model generation method based on deep learning | |
| CN120125750A (en) | FP-XFeat feature detector, 3D reconstruction method and system for typical rescue scenarios | |
| EP4453905A1 (en) | Method and apparatus for determining at least one optical parameter of at least one spectacle lens | |
| CN119863396A (en) | Depth constraint-based three-dimensional Gaussian camera motion blur removal method | |
| Yan et al. | DAN: Distortion-aware Network for fisheye image rectification using graph reasoning | |
| CN112612956B (en) | Panoramic view-based art design method and platform | |
| CN120778034B (en) | A three-dimensional measurement method for non-Lambertian metallic surfaces |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
| PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
| 17P | Request for examination filed |
Effective date: 20240703 |
|
| AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC ME MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
| 17Q | First examination report despatched |
Effective date: 20241118 |
|
| GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
| DAV | Request for validation of the european patent (deleted) | ||
| DAX | Request for extension of the european patent (deleted) | ||
| GRAJ | Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted |
Free format text: ORIGINAL CODE: EPIDOSDIGR1 |
|
| INTG | Intention to grant announced |
Effective date: 20250213 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
| GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
| INTC | Intention to grant announced (deleted) | ||
| INTG | Intention to grant announced |
Effective date: 20250403 |
|
| STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
| 18D | Application deemed to be withdrawn |
Effective date: 20250805 |