EP4256307A1

EP4256307A1 - Computer-implemented method, computer program, data processing system, and device for determining the reflectance behavior of a surface of an object, and storage medium with instructions stored thereon for determining the reflectance behavior of a surface of an object

Info

Publication number: EP4256307A1
Application number: EP23704170.2A
Authority: EP
Inventors: Tobias NOELL
Original assignee: Carl Zeiss Vision International GmbH
Current assignee: Carl Zeiss Vision International GmbH
Priority date: 2022-02-23
Filing date: 2023-02-14
Publication date: 2023-10-11
Also published as: US11928851B2; US20230386172A1; CN116940827A; WO2023161083A1; EP4235149A1

Abstract

The invention relates to computer-implemented method for determining the reflectance behavior of surface (7) of an object (6), having the following steps: • - providing multiple images of the object (6), said images differing in terms of the capturing direction and/or the illumination direction during the capturing process, • - generating data sets with entries, each of which describes a reflectance value derived from the images, an assigned capturing direction, and an assigned illumination direction, each data set being assigned to a respective point of the surface of the object (6), and • - ascertaining missing entries of the data sets on the basis of entries of already generated data sets, wherein a degree of similarity to the reflectance values derived from the images is taken into consideration, an average value or a weighted average value of the reflectance values derived from the images is calculated, or the reference values derived from the images are subjected to a smoothing process.

Description

Computer-implemented method. computer program. Data processing system and device for determining the reflectance behavior of a surface of an object, and storage medium with instructions stored thereon for determining the reflectance behavior of a surface of an object

Description

The present invention relates to a computer-implemented method, a computer program, a data processing system and a device for determining the reflectance behavior of a surface of an object (in particular a spectacle lens, a spectacle frame or spectacles) and a non-volatile computer-readable storage medium with instructions stored thereon for determining the reflectance behavior of a surface of an object (especially a spectacle lens, a spectacle frame or a pair of glasses).

The visual appearance of surfaces is largely determined by how incoming light interacts with this surface. Possible interactions are, for example, diffuse scattering (with rough surfaces), directed radiation (with reflective surfaces) or penetration (with transparent surfaces).

For opaque surfaces, this interaction or reflection behavior can be described mathematically using a bidirectional reflectance distribution function (BRDF) (reference [1]: NICODEMUS, FRED E. "Directional reflectance and emissivity of an opaque surface". Applied Optics 4.7 1965, pages 767 -773). The BRDF gives the amount of reflected light in each possible observation direction for each possible light incidence angle, i.e. it is a four-dimensional function (two parameters for light incidence direction and two parameters for observation direction). In the case of general surfaces, the reflectance behavior also changes over the surface and is therefore also different for each surface point. One speaks of an SVBRDF (“Specially Varying” BRDF) with six parameters (two additional parameters for the identification of the respective point on the surface).

In order to measure the SVBRDF, a surface must be illuminated from different directions and observed from different directions (or recorded from different directions). Unfavorably, the reflectance behavior, for example for reflective materials, can change significantly even for the smallest changes in angle, so that the parameter space has to be scanned closely for a measurement and an extremely large number of light positions and directions of observation are required. In order to be able to measure an SVBRDF with a manageable small number of measurements, two concepts have been used in practice up to now:

1) In the case of dichroic separation, it is assumed that the reflectance behavior of the entire surface consists partly of a “diffuse” and partly of a “specular” term. Strong changes over the surface are allowed for the diffuse term, since it can also be determined from a few angle configurations. The reflecting term, on the other hand, is divided over the entire surface, ie it is either constant or only minor changes are assumed (literature [2]: GEORGHIDIADES, ATHINODOROS S., "Recovering 3-D shape and reflectance from a small number of photographs". Proceedings of the 14th Eurographics workshop on rendering Eurographics Association, 2003, ^pages 230-240, reference [3]: YU, YIZHOU et al., "Inverse global illumination: Recovering reflectance models of real scenes from photographs". annual conference on Computer graphics and interactive techniques, ACM Press/Addison-Wesley-Publishing Co. 1999, pp. 215-224).

However, this method cannot be used to describe surfaces with strongly changing reflective behavior, which is generally very limiting.

2) The vast majority of methods assume that a surface consists of only a very small number of basic or basic materials (Reference [4]: LENSCH, HENDRIK et al. "Image-based reconstruction of spatial appearance and geometric detail" ACM Transactions and Graphis (TOG) 22.2, 2003, pages 234-257, reference [5]: GOLDMANN, DAN B et al., "Shape and spatially-varying brdfs from photometric stereo".Pattem Analysis and Maching Intelligence, IEEE Transactions on 32.6, 2010, pages 1060-1071, reference [6]: HOLROYD, MIACHAEL, LAWRENCE, JASON, and ZICKLER, TODD "A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance".ACM Transactions on Graphics (TOG) 29.4 , 2010, page 99, reference [7]: Giljoo Nam, Joo Ho Lee, Diego Gutierrez, and Min H. Kim “Practical SVBRDF acquisition of 3D objects with unstructured flash photography” ACM Trans Graph 37.6, Article 267, December 2018 .

Reference [8]: NIELSEN, JANNIK BOLL "On Practical Sampling of Bidirectional Reflectance". PhD thesis, January 1, 2016 (2016-01-01), XP055885310. Also already shows a method for determining the reflectance behavior of an object, in which several images of the object are provided, from which the reflectance value is derived and tabulated. Missing entries in the data records are added based on entries from data records that have already been created. TUNWATTANAPONG, BOROM et al. "Acquiring reflectance and shape from continuous spherical harmonic illumination" ACM Transactions on Graphics, Vol. 32, No. 4, July 21, 2013, pages 1-12, DOI: 10. 1145/2461912.2461944, shows a device for detecting the spatially varying reflection properties of 3D objects. Here, the 3D objects are observed under continuous spherical-harmonic lighting conditions. The lighting setup used consists of a rotating arc with controllable LEDs. This creates a full 3D model and a set of composite reflectance maps.

US 2014/0268160 A1 shows a device for measuring surface orientation maps of an object. In this case, the object is illuminated by a light source with a controllable illumination field, while images of the object are recorded by cameras. From these images, the object's reflection properties are then extracted, including an albedo, a reflection vector, a roughness, and/or anisotropy parameters of a specular reflection lobe.

AUDET, SAMUEL et al. "A user-friendly method to geometrically calibrate projector-camera systems" COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, 2009. CVPR WORKSHOPS 2009. IEEE COMPUTER SOCIETY CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 06/20/2009, pages 47-54, ISBN: 978-1-4244-3994-2, shows a method of calibrating a projector-camera system using both a printed pattern and a projection pattern. To simplify the calibration process, these markers are used simultaneously and are superimposed.

WEINMANN, MICHAEL et al. "Advances in geometry and reflectance acquisition (course notes)", 20151102; 1077952576-1077952576, 11/2/2015, pages 1-71, DOI: 10.1145/2818143.2818165 and US 2016/275720 A1 show various calibration methods, including using reflecting balls.

WEYRICH, TIM et al. "Principles of appearance acquisition and representation", SIGGRAPH 2008 Class Notes, 1077952576-1077952576, 08/11/2008, pages 1-119, XP058345154, DOI:

10.1145/1401132.1401234 already teaches the use of a color table for color calibration of a camera in addition to calibration using reflecting balls.

A behavior measured at one point can therefore be transferred to another point on the surface if both points are assigned to the same base material. This significantly reduces the number of images required for the SVBRDF measurement. The base materials are usually assigned based on the color of the surface, ie it is derived from it assumed that two spots of similar color have the same base material. The main difficulty with these methods is recognizing the correct number of base materials and their specific extents and making the appropriate mapping from surface point to base material. Each base material and its associated extent can be referred to as a segment of the surface, so that this approach can also be referred to as a segmentation problem, in which the different segments of the surface are to be determined and as few images as possible are to be recorded for the determined segments for the SVBRDF measurement.

Based on this and in particular based on the method of the cited reference [8], which is considered to be the closest prior art, it is the object of the invention to develop a computer-implemented method for determining the reflectance behavior of a surface of an object in such a way that an improved determination is possible . Furthermore, a computer program, a data processing system and a device for determining the reflectance behavior of a surface of an object and a storage medium with instructions stored thereon for determining the reflectance behavior of a surface of an object are to be provided.

The invention is defined in the independent claims. Advantageous configurations are specified in the dependent claims.

In the computer-implemented method for determining the reflectance behavior of a surface of an object, multiple images of the object are provided, with the images differing in a recording direction and/or in an illumination direction during the recording. Data sets are created with entries that each describe a reflectance value derived from the images and an associated recording direction and an associated illumination direction, each data set being associated with a point on the surface of the object. Furthermore, missing entries for data sets for points on the surface are determined based on entries from data sets already created. Determining the missing entries can include calculating, combining and/or other steps to be performed. According to the invention, a measure of similarity to the reflectance values derived from the images can be taken into account when determining missing entries in the data sets. Alternatively or additionally, when missing entries in the data sets are determined, a mean value calculation or a weighted mean value calculation of the reflectance values derived from the images can be carried out. As a further alternative or in addition, when missing entries in the data sets are determined, the reflectance values derived from the images can be smoothed. Thus, from relatively few images, data sets can first be created in which the reflectance values are derived from the images. These are therefore measured reflectance values. The determined missing entries of the data sets are then based on the already created data sets (or their entries) and they are therefore not measured reflectance values, but, for example, calculated entries of the data sets.

This advantageously means that just a few images are sufficient to achieve a very good determination of the reflectance behavior of the surface of the object. In addition, the complex and error-prone segmentation as in the previously known procedure is not necessary, so that a very good determination of the reflectance behavior of the surface of the object can also be achieved. In this way, for example, the reflectance values can be determined for surface points that may not have been recorded or were not illuminated in a specific configuration and are therefore not present on the recordings. The step of adding missing entries in the data records is therefore preferably carried out without a segmentation step.

Reflectance is understood here in particular as the degree of reflection and therefore the ratio of the reflected radiant power to the incident radiant power. Reflectance value is understood here in particular as a value that describes the reflectance. Reflectance behavior is understood here in particular as how the light is reflected and/or scattered by the surface of the object. In other words, the reflectance behavior describes the optical impression that a person gains when looking at the surface due to the reflecting, scattering and/or absorbing properties of the surface.

Illumination direction is understood here in particular as the angle of incidence (relative to a predetermined coordinate system) at which the surface of the object is illuminated with light during the recording. The recording direction (related to the predetermined coordinate system) is understood here in particular to mean the angle at which the recording of the surface of the object takes place.

When determining missing entries in the data sets for points on the surface of the object (which is also referred to below as determining further data sets for points on the surface of the object) based on entries in data sets that have already been created, a measure of similarity to the reflectance values derived from the images can be taken into account. The measure of similarity can take into account at least one selected from the group of similarity of the angular configuration, spatial distance and photometric similarity. The similarity of the angular configuration can in particular involve the deviation in the angle of the recording direction and/or the deviation in the angle of the illumination direction. The spatial distance can in particular be the distance over the surface (geodetic distance). The photometric similarity can in particular be the similarity of the reflectance values derived from the images. The smaller the respective difference, the more likely the similarity is assumed.

Furthermore, when determining missing entries in the data sets for points on the surface of the object, based on entries in data sets that have already been created, a mean value calculation or a weighted mean value calculation of the reflectance values derived from the images can be carried out.

The calculation of the mean can in particular be the calculation of the arithmetic mean. Another parameter can be taken into account in the weighted mean calculation, such as the distance to the point on the surface for which the further data set is to be calculated. The greater the distance, the smaller the influence of an already known reflectance value s.

Incorrect values can be better avoided by calculating the mean value, since individual incorrect reflectance values are less taken into account.

Furthermore, the reflectance values derived from the images can be smoothed. This means in particular that the measured values are averaged in order to be able to better correct undesired outliers. One can also say that the derived reflectance values are subjected to low-pass filtering for smoothing.

The step of providing the multiple images of the object can include the step of illuminating and taking the picture. In this case, the images can have a plurality of pixels, which are then assigned to corresponding image points and a combination of recording direction and illumination direction.

In the computer-implemented method according to the invention for determining the reflectance behavior of a surface of an object, the data sets can be parameterized in such a way that a four-dimensional angle component with two parameters for the illumination direction and two parameters for the recording direction and a two-dimensional spatial component with two parameters for the corresponding point on the surface of the object is present. A discretization of this function is achieved by the data sets. Furthermore, a continuously spatially varying bidirectional reflectance distribution function can be calculated based on the data sets. This means in particular that the function is continuous. In particular, photo-realistic representations of the object can be created in this way.

With the method according to the invention (including its development), the reflectance behavior of the surface of the object can be determined using the data sets.

The object can in particular be a spectacle lens, a spectacle frame or spectacles.

In the computer-implemented method according to the invention, at least one of the sequences of steps a) or b) can be carried out for the calibration. In step a), at least a first calibration image of a calibration object with a plurality of distinguishable markers at positions that are known relative to one another is evaluated, at least a second calibration image of the calibration object, which was illuminated with a pattern when the image was recorded, is evaluated and the calibration is then based on carried out the evaluations of the first and second calibration image(s). In step b), at least a first calibration image of a calibration object is evaluated with a plurality of distinguishable spheres at positions that are known relative to one another, with the center points of the spheres being determined, and the calibration is based on the evaluation(s) of the first calibration image(s). ) carried out. With such a calibration, an improved determination of the reflectance behavior of the surface of the object is made possible.

Alternatively or additionally, at least a fourth calibration recording of a calibration object can be evaluated in the computer-implemented method for a calibration. This calibration object has at least one non-repetitive pattern, whereby characteristic features of the surface of the calibration object are specified, a variable size, several calibration bodies with different sizes, a color chart for color calibration of cameras for generating the recordings, or characteristic features with known scaling and/or or known relative distances to each other.

Furthermore, alternatively or additionally, in the computer-implemented method in a step III) for calibrating at least one camera-projector combination, a calibration object can be recorded with at least one camera of the at least one camera-projector combination(s). In a step IV) one or more patterns on the calibration object is projected with at least one of the projectors camera-projector combination (s), which of the at least one camera of the camera-projector combination(s) is recorded. In a step V), based on the recordings of the at least one camera of the camera-projector combination(s), a calibration of the at least one camera of the camera-projector combination(s) and/or of the at least one projector of the camera-projector Combination(s) carried out. It is provided here that the relative positioning of the calibration object for at least one camera of the camera-projector combination be changed after performing step sequences III) and IV) and then perform step sequences III) and IV) again. Alternatively or additionally, a camera response curve of the at least one camera of the camera-projector combination(s) can be calibrated based on recordings with different exposure times.

In the computer-implemented method, a bidirectional reflectance distribution function can be used to create the data sets, which describes the amount of reflected light of an exiting beam for each possible observation direction as a BRDL value for each point on the surface of the object for possible light incidence angles of an incident beam, see above that the reflectance distribution function is a six-dimensional puncture, which has a four-dimensional angular component with two parameters for the light incidence direction and two parameters for the observation direction, and a two-dimensional spatial component with two parameters for the corresponding point on the surface of the object.

This parameterization enables a good description of the reflectance behavior of the object's surface.

In the computer-implemented method, a bidirectional reflectance distribution function (BRDF) can also be used, which describes the amount of reflected light of an emerging beam in each possible observation direction as a BRDF value for each point on the surface of the object for possible light incidence angles of an incident beam, so that the reflectance distribution function (SVBRDF) is a six-dimensional function that has a four-dimensional angle part with two parameters for the direction of light incidence and two parameters for the direction of observation as well as a two-dimensional spatial part with two parameters for the corresponding point on the object surface, the four-dimensional angle part of the reflectance distribution function using a Rusinkiewicz Parameterization can be parameterized in such a way that the transformed coordinate axes are polar angles and azimuth angles of a mean vector and polar angles and azimuth angles of a differential vector, with the mean vector being the mean vector between the incident and emerging beam and the differential vector being the difference between the incident beam and the mean vector. The space spanned by the transformed coordinate axes is then discretized by means of a Coordinate lattice structure carried out with lattice points, the lattice spacing for the coordinate axis polar angle of the central vector being smaller than for the remaining transformed coordinate axes. Furthermore, a reflectance measurement is carried out for each grid point of the surface of the object to measure the BRDF value. A BRDF table is constructed for each grid point using the measured BRDF values from the data sets, and the BRDF tables for unmeasured surface points are completed by calculating the BRDF values of the unmeasured surface points from the measured BRDF values , in order to describe the reflectance behavior of the surface of the object. This means that even surfaces with strongly changing reflectance behavior can be measured and the reflectance behavior of the surface can be described, with the frequently erroneous segmentation being able to be completely dispensed with. It is therefore not necessary to recognize a number of base materials or to determine this in any way. There is also no corresponding assignment of a surface point to a specific base material. In particular, this is achieved by the step of calculating the BRDF values of the unmeasured points of the surface.

A computer program is also provided for determining the reflectance behavior of a surface of an object based on a plurality of provided images of the object, which differ in a recording direction and/or illumination direction of the recording, the computer program comprising instructions which, when executed on a computer cause the computer to create data sets with entries each describing a reflectance value derived from the images and an associated recording direction and an associated illumination direction, each data set being associated with a point on the surface of the object. Furthermore, the instructions cause the computer to determine missing entries in the data sets for points on the surface of the object based on entries in data sets that have already been created. Determining the missing entries can include calculating, combining and/or other steps to be performed. According to the invention, a measure of similarity to the reflectance values derived from the images can be taken into account when determining missing entries in the data sets. Alternatively or additionally, when missing entries in the data sets are determined, a mean value calculation or a weighted mean value calculation of the reflectance values derived from the images can be carried out. As a further alternative or in addition, when missing entries in the data sets are determined, the reflectance values derived from the images can be smoothed.

A very good determination of the reflectance behavior of a surface of the object can thus be achieved in an advantageous manner with only a few images. There is also a non-transitory computer-readable storage medium with instructions stored thereon for determining the reflectance behavior of a surface of an object based on a plurality of provided images of the object, which differ in a recording direction and/or illumination direction during recording, provided, the instructions, if they are executed on a computer, cause the computer to create data sets with entries that each describe a reflectance value derived from the images and an associated recording direction and an associated illumination direction, each data set being associated with a point on the surface of the object. Furthermore, the instructions cause the computer to determine missing entries in the data sets for points on the surface of the object based on entries in data sets that have already been created. Determining the missing entries can include calculating, combining and/or other steps to be performed. According to the invention, a measure of similarity to the reflectance values derived from the images can be taken into account when determining missing entries in the data sets. Alternatively or additionally, when missing entries in the data sets are determined, a mean value calculation or a weighted mean value calculation of the reflectance values derived from the images can be carried out. As a further alternative or in addition, when missing entries in the data sets are determined, the reflectance values derived from the images can be smoothed.

A very good determination of the reflectance behavior of the surface of the object can thus advantageously be achieved with just a few images.

Furthermore, a data processing system is provided for determining the reflectance behavior of a surface of an object based on a plurality of provided images of the object, which differ in a recording direction and/or illumination direction during recording, the data processing system comprising a processor and a memory. The processor is designed to create data sets with entries based on instructions from a computer program stored in the memory, each of which describes a reflectance value derived from the images and an associated recording direction and an associated illumination direction, with each data set being associated with a point on the surface of the object . Furthermore, the processor is designed, based on the instructions of the computer program stored in the memory, to determine missing entries in the data sets for points on the surface of the object based on entries in data sets that have already been created. Determining the missing entries can include calculating, combining and/or other steps to be performed. According to the invention, a measure of similarity to the reflectance values derived from the images can be taken into account when determining missing entries in the data sets. Alternatively or additionally, when determining missing entries in the data records, a mean value calculation or a weighted averaging of the reflectance values derived from the images can be performed. As a further alternative or in addition, when missing entries in the data sets are determined, the reflectance values derived from the images can be smoothed.

A device for determining the reflectance behavior of a surface of an object is also provided, the device comprising a control unit and a detection unit, the detection unit being set up to record a plurality of images of the object which are in a recording direction and/or an illumination direction during the recording distinguish, and wherein the control unit is set up to create data sets with entries that each describe a reflectance value derived from the images and an associated recording direction and an associated illumination direction, each data set being associated with a point on the surface of the object, and missing entries of Records for points of the object determined based on entries already created datasets. The reflectance behavior of the surface of an object can be determined using the data sets created in this way. For this purpose, the control unit is set up to take into account a measure of similarity to the reflectance values derived from the images. Alternatively or additionally, the control unit is also set up to carry out a mean value calculation or a weighted mean value calculation of the reflectance values derived from the images. As a further alternative or in addition, the control unit is also set up to smooth the reflectance values derived from the images.

With the method according to the invention, even surfaces with strongly changing reflectance behavior can be measured and the reflectance behavior of the surface can be described, with the frequently erroneous segmentation being able to be completely dispensed with. In concrete terms, it is not necessary to recognize a number of base materials or to determine them in any way. There is also no corresponding assignment of a surface point to a specific base material. This is achieved in particular by the step of calculating further data sets.

A method for determining the reflectance behavior of a surface of an object is also provided, which has the following steps:

Providing a bidirectional reflectance distribution function (BRDF), which describes the amount of reflected light of an emerging beam in each possible observation direction as a BRDF value for each point on the surface of the object for possible light incidence angles of an incident ray, so that the reflectance distribution function (SVBRDF) a is a six-dimensional function that has a four-dimensional angular component with two parameters for the direction of light incidence and two parameters for the direction of observation and a two-dimensional spatial component with two parameters for the corresponding point on the object surface. The following steps are carried out further:

Parameterization of the four-dimensional angular component of the reflectance distribution function (SVBRDF) using a Rusinkiewicz parameterization such that the transformed coordinate axes are polar angle (0 _ft ) and azimuth angle ( _ft ) of a center vector (/l) and polar angle (0 _d ) and azimuth angle ( _d ) of a difference vector ( d) are present, where the mean vector (h) is the mean vector between the incident ray and the emerging ray and the difference vector (d) is the difference between the incident ray and mean vector (h), discretizing the space spanned by the transformed coordinate axes using a coordinate grid structure with grid points, whereby the grid spacing for the coordinate axis polar angle (0 _ft ) of the mean vector (h) can be smaller than for the rest of the transformed coordinate axes 0 _d , _d ) ,

performing a reflectance measurement for each grid point at a predetermined number of points on the surface of the object to measure the BRDF value, the predetermined number of points on the surface being less than the total number of points describing the surface of the object and being different at the different grid points creating a BRDF table for each grid point with the measured BRDF values and completing the BRDF tables for the unmeasured points of the surface by determining (e.g. calculating) the BRDF values of the unmeasured points of the surface from the measured BRDF values to describe the reflectance behavior of the object's surface.

In particular, the data records can be combined into the BRDF tables.

When calculating the BRDF values of the unmeasured points on the surface from the measured BRDF values, a degree of similarity to the measured points can be taken into account.

At least one selected from the group of similarity of the angular configuration, spatial distance (e.g. over the surface) and photometric similarity can be taken into account as a measure of similarity.

In the case of the similarity of the angular configuration, in particular, the similarity increases as the distance decreases; distance is, and for photometric similarity in particular, similarity increases with decreasing distance.

An averaging calculation or a weighted averaging calculation can be performed in calculating the BRDF values of the unmeasured point of the surface from the measured BRDF values.

Furthermore, the measured BRDF values can be smoothed when creating the BRDF tables for each grid point with the measured BRDF values.

Furthermore, based on the completed BRDF tables, a continuous bidirectional reflectance distribution function can be calculated.

To calculate the continuous bidirectional reflectance distribution function, a multivariate interpolation (that is, an interpolation for a function with several variables) or a parameter estimation of the parameterized reflectance distribution function can be performed.

A device for determining the reflectance behavior of a surface of an object is also provided, the device comprising a control unit and a detection unit, the control unit providing a bidirectional reflectance distribution function (BRDF) which, for each point on the surface of the object to be measured, for possible angles of incidence of light incident beam describes the amount of reflected light of an emerging beam in each possible observation direction as a BRDF value, so that the reflectance distribution function (SVBRDF) is a six-dimensional function that has a four-dimensional angular component with two parameters for the light incidence direction and two parameters for the observation direction and a two-dimensional spatial component with two parameters for the corresponding point on the object surface, the control unit parameterizing the four-dimensional angular component of the reflectance distribution function (SVBRDF) using Rusinkiewicz parameterization in such a way that the transformed coordinate axes are polar angles (0^) and azimuth angles (^) of a center vector (h) and polar angle (0^) and azimuth angle ( ^) of a difference vector (d), where the mean vector (h) is the mean vector between the incident ray and the outgoing ray and the difference vector (d) is the difference between the incident ray and the mean vector ( h) and the space spanned by the transformed coordinate axes is discretized by means of a coordinate grid structure with grid points, with the grid spacing for the coordinate axis Polar angle (0^) of the mean vector (h) is less than for the rest of the transformed coordinate axes 0 _d , _d ), wherein the detection unit performs a reflectance measurement for each grid point at a predetermined number of points on the surface of the object to be measured to measure the BRDF value, the predetermined number of points on the surface being smaller than the total number of points for the description of the surface of the object to be measured and can be different for the different grid points, with the control unit creating a BRDF table for each grid point with the measured BRDF values and the BRDF tables for the non-measured points of the surface by determining (e.g. calculating ) of the BRDF values of the unmeasured points of the surface are completed from the measured BRDF values in order to describe the reflectance behavior of the surface of the object to be measured.

It goes without saying that the features mentioned above and those still to be explained below can be used not only in the specified combinations, but also in other combinations or on their own, without departing from the scope of the present invention.

The device according to the invention can be further developed in the same way as the method according to the invention (this also applies vice versa).

The invention is explained in more detail below using exemplary embodiments with reference to the accompanying drawings, which also disclose features that are essential to the invention. These embodiments are provided for illustration only and are not to be construed as limiting. For example, a description of an embodiment having a plurality of elements or components should not be construed as implying that all of those elements or components are necessary for implementation. Rather, other embodiments may include alternative elements and components, fewer elements or components, or additional elements or components. Elements or components of different exemplary embodiments can be combined with one another unless otherwise stated. Modifications and variations that are described for one of the embodiments can also be applicable to other embodiments. To avoid repetition, the same or corresponding elements in different figures are denoted by the same reference symbols and are not explained more than once. From the figures show:

1 shows a schematic view of an embodiment of the device according to the invention, 2 shows a schematic representation to explain the spherical coordinates for the incident ray 9 and the emerging ray 12, and

FIG. 3 shows a representation according to FIG. 2 to explain the Rusinkiewicz parameterization that was carried out.

In the exemplary embodiment of a 3D digitizing system 1 shown in FIG. 1 (which can also be referred to as a measuring system 1 or as a device 1 for determining the reflectance behavior of a surface of an object), it comprises a control unit 2, an illumination unit 3 that can be positioned in space, a detection unit 4 that can be positioned in space and a holder 5 on which an object 6 to be digitized is positioned. The holder 5 can be designed, for example, as a rotatable glass plate.

The 3D digitizing system 1 is designed in particular in such a way that it can be used to determine a description of the reflectance behavior of the surface 7 of the object 6 .

The lighting unit 3 has a light source 8, which can direct a light beam 9 (incident beam) onto the surface 7 of the object 6, and a first movement unit 10, which changes the position of the light source 8 in space (and thus the direction of the light beam 9 and the point of impact of the light beam 9 on the surface 7) can adjust and change. The detection unit 4 includes a detector 11 for detecting the light beam 12 reflected by the surface 7 and a second movement unit 13 with which the position of the detector 11 in space can be set and changed. The holder 5 is rotatable, as indicated by the arrow 14. The surface 7 of the object 6 can thus be illuminated at different points with different angles and the reflection or scattering can be detected at different angles.

The control unit 2 can carry out all the calculation and evaluation steps described below. However, it is also possible that the control unit 2 only controls the measurement steps described below and transmits the data obtained thereby to a separate calculation unit 15, as indicated by dashed lines in FIG. 1, and the calculation unit 15 carries out the calculation and evaluation steps. The calculation unit 15 can be a computer with a processor, the same applies to the control unit 2.

In particular for opaque surfaces 7, this interaction or reflectance behavior can be described mathematically using a bidirectional reflectance distribution function (BRDF) (reference [1]). The BRDF gives the amount of reflected light (BRDF value) in every possible observation direction for every possible angle of incidence of light. So it is fundamental a four-dimensional function with two parameters for the direction of light incidence and two parameters for the direction of observation. Since it is generally not constant for surfaces 7 but can also differ from the location on the surface 7 (i.e. from the surface point), the BRDF has two additional parameters for identifying the respective location on the surface 7, so that the bidirectional Reflectance distribution function is a six-dimensional function, which is a four-dimensional angular component with two parameters for the direction of light incidence and two parameters for the direction of observation, and a two-dimensional spatial component with two parameters for the corresponding points on the object surface. In this case, the bidirectional reflectance distribution function can also be referred to as a spatially varying bidirectional reflectance distribution function (e.g. specially varying BRDF or SVBDF).

In order to be able to describe the reflectance behavior of the surface 7 of the object 6 well, even if the surface 7 has a locally strongly varying reflectance behavior, the four-dimensional angle component is parameterized in the six-dimensional bidirectional reflectance distribution function using the Rusinkiewicz parameterization in such a way that the transformed coordinate axes are polar angles 0h and Azemuth angle h of a mean vector h and polar angles 0d and azemuth angle <t> _(i a difference vector d exist. The mean vector h is the mean vector between incident ray 9 and outgoing ray 12 and the difference vector d is the difference of incident ray 9 and mean vector h ( the mean vector h as well as the difference vector d are each given here in bold type and can also be given as h and d.) The Rusinkiewicz parameterization is described in the article "A New Change of Variables for Efficient BRDF Representation" Szymon M. Rusinkiewicz, Stanford University described in detail, which can be downloaded from https://www.cs.princeton.edu/~smr/papers/brdf_change_of_variables/, for example, and was presented in 1998 at the Eurographics Workshop on Rendering.

This parameterization is based on a reflection, shown schematically in Fig. 2, of the incident light beam 9 (which runs in the opposite direction to the arrow shown) at a point on the surface 7, which leads to a reflected light beam 12 and is shown in Fig. 2 in spherical coordinates, in Connection with Fig. 3 clarifies, since in Fig. 3 the same reflection is shown in the Rusinkiewicz parameterization.

In FIG. 2, the incident light beam 9 is clearly described with the spherical coordinates 0i and <t>, based on the surface normal n and the surface tangent t. The reflected light beam 12 has the angles 0 ₀ and <t> ₀ . After the Rusinkiewicz parameterization, this reflection is described by the transformed coordinate axes 0h, h, 0 _d , O _d . Further details result from the following description of the vectors.

0^ and 0^ are the spherical coordinates of the mean vector h in the t-n-b system and the two rotations for d bring the mean vector h to the north pole and are the spherical coordinates 0[ and O; of the incident ray 9 in this transformed system (further details can be found in reference [1] and in the Rusinkiewicz article “A New Change of Variables for Efficient BRDF Representation”).

According to the invention, it has been found that the BRDF values of many materials occurring in practice show a strong change in the 0h direction and a smaller change along the other coordinate axes 0 _d , h and O _d . Therefore, the space spanned by the Rusinkiewicz parameterization is discretized by means of a lattice structure, with a smaller lattice spacing being chosen along the 0h direction than for the directions of the other coordinate axes, since greater changes in the reflectance behavior are to be expected along the 0h direction. The result is an "angle table" that contains a number of angle configurations from the direction of light incidence and the direction of observation that are important for the reflectance behavior. The number of entries is regulated by the fineness of the discretization of the space. As shown in the example table of angles below, the angle θh is varied in 10° increments so that there are nine ranges. No subdivision is performed for the angles Oh, 0<i and O _d , so that there is a single range for each. This results in an angle table with 9 * 1 * 1 * 1= 9 entries, as shown below.

angle table:

The associated BRDF values of the corresponding surface points are now sought for each of the nine entries in the angle table and are assigned to these entries. This combination of discrete angles (corresponding to the entries from the angle table) with the associated BRDF values is also referred to below as the BRDF table. A BRDF table is given schematically for the index value 2 according to the angle table, whereby to simplify the description, 25 points of the surface 7 with the coordinates K1 and K2 are assumed and measured values are only available if none entered in the corresponding "Value" column.

Since each point of the surface 7 thus provides a BRDF value in one of the BRDF tables, not all BRDF tables will be completely filled after the measurement, as is shown schematically for the BRDF table with index = 2, since in in practice only a small number of characteristic angular configurations are measured per surface point.

Therefore, based on the measured BRDF values, the unmeasured BRDF values are determined and the BRDF tables are thus completed. Here, this problem can be treated as a "scattered data interpolation" problem. A basic idea is that unknown (unmeasured) BRDF values per surface point can be calculated using other known (measured) BRDF values can be determined from the remaining BRDF tables if it can be assumed that the surface 7 reacts in a similar way.

Therefore, a measure of similarity to all other unknown BRDF values of all BRDF tables is determined for each known BRDF value based on the following consideration:

1) Similarity of angular configuration. What is the distance of the angular configuration to the unknown entry? A low distance indicates a high similarity.

2) Spatial distance is taken into account. So how far away is the unknown entry on surface 7? The lower the distance, the higher the similarity.

3) Photometric similarity is taken into account. How similar are the respective BRDF tables of the respective entries. A low distance indicates a high similarity.

Unknown BRDF values are thus determined using known BRDF values (e.g. by a weighted average or other interpolation techniques) which have a high degree of similarity with respect to the previous three considerations. It is also possible that measured BRDF values can be corrected in a similar way, e.g. to perform smoothing. In this way the unknown BRDF values are filled in and thus the entire SVBRDF is determined without having to determine a number of base materials.

The BRDF table filled for Index = 2 is shown below:

Each row of a BRDF table can also be used as a data record DS with at least one entry (e.g.

Reflectance value and an associated recording direction and an associated

Illumination direction) are referred to, each data set being assigned to a point on the surface 7 of the object 6. A continuous SVBRDF can then be determined based on the felled BRDF tables. This is possible, for example, by multivariate interpolation or parameter estimation of a parameterized BRDF model.

Claims

patent claims

1. Computer-implemented method for determining the reflectance behavior of a surface

(7) an object (6), with the steps:

- providing several images of the object (6) which differ in a recording direction and/or in a lighting direction during the recording,

- Creation of data sets with entries that each describe a reflectance value derived from the images and an associated recording direction and an associated illumination direction, each data set being associated with a point on the surface (7) of the object (6), and

- Determining missing entries in the data sets based on entries from data sets that have already been created, characterized in that when determining missing entries in the data sets, a measure of similarity to the reflectance values derived from the images is taken into account.

2. Computer-implemented method for determining the reflectance behavior of a surface

(7) an object (6), with the steps:

- Determination of missing entries in the data sets based on entries of data sets that have already been created, characterized in that when determining missing entries in the data sets, a mean value calculation or a weighted mean value calculation of the reflectance values derived from the images is carried out.

3. Computer-implemented method for determining the reflectance behavior of a surface

(7) an object (6), with the steps:

- Creation of data sets with entries that each describe a reflectance value derived from the images and an associated recording direction and an associated illumination direction, each data set being associated with a point on the surface (7) of the object (6), and - Determining missing entries in the data sets based on entries from data sets that have already been created, characterized in that the reflectance values derived from the images are smoothed when determining missing entries in the data sets.

4. Computer-implemented method according to claim 2 or 3, characterized in that a measure of similarity to the reflectance values derived from the images is taken into account when determining missing entries in the data sets.

5. Computer-implemented method according to claim 1 or 4, characterized in that at least one selected from the group of similarity of the angular configuration, spatial distance over the surface (7) and photometric similarity is taken into account as a measure of similarity.

6. Computer-implemented method according to claim 5, characterized in that in the case of the similarity of the angular configuration, the similarity increases as the distance decreases, in the case of the spatial distance the similarity increases the lower the distance is, and in the case of the photometric similarity the similarity decreases as the distance decreases distance increases.

7. Computer-implemented method according to claim 1 or according to one of claims 3 to 6, characterized in that when determining missing entries in the data sets, a mean value calculation or a weighted mean value calculation of the reflectance values derived from the images is carried out.

8. Computer-implemented method according to claim 1 or 2 or according to one of claims 4 to 7, characterized in that when missing entries in the data sets are determined, the reflectance values derived from the images are smoothed.

9. Computer-implemented method according to one of the above claims, characterized in that the data sets are parameterized in such a way that there is a four-dimensional angle component with two parameters for the illumination direction and two parameters for the recording direction.

10. Computer-implemented method according to any one of the above claims, characterized in that the object (6) is at least one selected from a group comprising a spectacle lens, a spectacle frame and spectacles.

11. Computer-implemented method according to one of the above claims, characterized in that for the calibration at least one of the step sequences a) evaluating at least one first calibration recording of a calibration object with a plurality of distinguishable markers at positions that are known relative to one another,

evaluating at least one second calibration shot of the calibration object, which was illuminated with a pattern when the shot was taken, and

Carrying out the calibration based on the evaluation of the first and second calibration image(s), or b) evaluating at least a third calibration image of a calibration object with a plurality of distinguishable spheres at known positions relative to one another, the centers of the spheres being determined, and

Carrying out the calibration based on the evaluation(s) of the third calibration recording(s).

12. Computer-implemented method according to one of the above claims, characterized in that at least a fourth calibration recording of a calibration object is evaluated for a calibration, the calibration object having at least one of the following properties:

- the calibration object has a non-repetitive pattern, whereby characteristic features of the surface of the calibration object are specified,

- the calibration object has a variable size,

- the calibration object has several calibration bodies with different sizes,

- the calibration object has a color chart for color calibration of cameras to generate the recordings, and

- the calibration object has characteristic features with known scaling and/or known relative distances to one another.

13. Computer-implemented method according to any one of the above claims, characterized in that

III) for a calibration of at least one camera-projector combination, a calibration object is recorded with at least one camera of at least one camera-projector combination(s),

IV) at least one of the projectors of the camera-projector combination(s) projects one or more patterns onto the calibration object, which are recorded by the at least one camera of the camera-projector combination(s),

V) based on the recordings of the at least one camera of the camera-projector combination(s), a calibration of the at least one camera of the camera-projector combination

combination(s) and/or the at least one projector of the camera-projector combination(s) is carried out, preferably at least one of the following sequences of steps being carried out:

- Changing the relative positioning of the calibration object for at least one camera of the camera-projector combination after performing the sequence of steps III) and IV) and then performing the sequence of steps III) and IV) again, and

- Carrying out a calibration of a camera response curve of the at least one camera of the camera-projector combination(s) based on recordings with different exposure times.

14. Computer-implemented method according to one of the above claims, characterized in that to create the data sets, a bidirectional reflectance distribution function (BRDF), which for each point on the surface (7) of the object (6) for possible light incidence angles of an incident beam (9), the Amount of reflected light of an emerging beam (12) in each possible observation direction as a BRDF value is provided, so that the reflectance distribution function (BRDF) is a six-dimensional function that has a four-dimensional angular component with two parameters for the light incidence direction and two parameters for has the direction of observation and a two-dimensional local component with two parameters for the corresponding point on the surface (7) of the object (6).

15. Computer-implemented method according to claim 14, characterized by the following steps:

- Parameterization of the four-dimensional angular component of the reflectance distribution function (SVBRDF) using a Rusinkiewicz parameterization such that the transformed coordinate axes are polar angle (0 _ft ) and azimuth angle ( _ft ) of a mean vector (h) and polar angle (0 _d ) and azimuth angle ( _d ) of a difference vector ( d) are present, the mean vector (h) being the mean vector between the incident ray (9) and the emerging ray (12) and the difference vector (d) being the difference between the incident ray (9) and mean vector (h),

- Discretization of the space spanned by the transformed coordinate axes using a coordinate grid structure with grid points, the grid spacing for the coordinate axis polar angle (0^) of the mean vector (h) being smaller than for the remaining transformed coordinate axes ( _ft , 0 _d , _d ).

16. Computer program for determining the reflectance behavior of a surface (7) of an object (6) based on a plurality of provided images of the object (6) which differ in a recording direction and/or lighting direction during recording, the computer program comprising instructions which, when run on a computer, cause the computer to - to create data sets with entries that each describe a reflectance value derived from the images and an associated recording direction and an associated illumination direction, each data set being associated with a point on the surface (7) of the object (6), and

- to determine missing entries in the data sets based on entries in data sets that have already been created, characterized in that when determining missing entries in the data sets, a measure of similarity to the reflectance values derived from the images is taken into account.

17. Computer program for determining the reflectance behavior of a surface (7) of an object (6) based on a plurality of provided images of the object (6), which differ in a recording direction and/or lighting direction during the recording, the computer program comprising instructions which, when run on a computer, cause the computer to

- to create data sets with entries that each describe a reflectance value derived from the images and an associated recording direction and an associated illumination direction, each data set being associated with a point on the surface (7) of the object (6), and

- to determine missing entries in the data sets based on entries in data sets that have already been created, characterized in that when missing entries in the data sets are determined, a mean value calculation or a weighted mean value calculation of the reflectance values derived from the images is carried out.

18. Computer program for determining the reflectance behavior of a surface (7) of an object (6) based on a plurality of provided images of the object (6), which differ in a recording direction and/or illumination direction during recording, the computer program comprising instructions which, when run on a computer, cause the computer to

- to determine missing entries in the data sets based on entries in data sets that have already been created, characterized in that when missing entries in the data sets are determined, the reflectance values derived from the images are smoothed.

19. Data processing system for determining the reflectance behavior of a surface (7) of an object (6) based on a plurality of provided images of the object (6), which are in a Differentiating the recording direction and/or the direction of illumination when recording, the data processing system comprising a processor and a memory, the processor being designed to be based on instructions from a computer program stored in the memory

20. Data processing system for determining the reflectance behavior of a surface (7) of an object (6) based on a plurality of provided images of the object (6), which differ in a recording direction and/or illumination direction when recording, the data processing system having a processor and a memory includes, wherein the processor is adapted to be based on instructions of a computer program stored in the memory

21. Data processing system for determining the reflectance behavior of a surface (7) of an object (6) based on a plurality of provided images of the object (6), which differ in a recording direction and/or lighting direction during recording, the data processing system having a processor and a memory includes, wherein the processor is adapted to be based on instructions of a computer program stored in the memory

- to determine missing entries of the data sets based on entries of already created data sets, characterized in that when missing entries in the data sets are determined, the reflectance values derived from the images are smoothed.

22. Non-transitory computer-readable storage medium with instructions stored thereon for determining the reflectance behavior of a surface (7) of an object (6) based on a plurality of provided images of the object (6), which differ in a recording direction and/or lighting direction during recording, wherein the Instructions, when executed on a computer, cause the computer to

23. Non-transitory computer-readable storage medium with instructions stored thereon for determining the reflectance behavior of a surface (7) of an object (6) based on a plurality of provided images of the object (6), which differ in a recording direction and/or lighting direction during recording, wherein the Instructions, when executed on a computer, cause the computer to

24. Non-transitory computer-readable storage medium with instructions stored thereon for determining the reflectance behavior of a surface (7) of an object (6) based on a plurality of provided images of the object (6), which differ in a recording direction and/or lighting direction during recording, wherein the Instructions, when executed on a computer, cause the computer to

- to create data sets with entries that each describe a reflectance value derived from the images and an associated recording direction and an associated illumination direction, each data set being associated with a point on the surface (7) of the object (6), and - to determine missing entries in the data sets based on entries in data sets that have already been created, characterized in that when missing entries in the data sets are determined, the reflectance values derived from the images are smoothed.

25. Device (1) for determining the reflectance behavior of a surface (7) of an object (6), the device (1) comprising a control unit (2) and a detection unit (4), the detection unit (4) being set up for recording several images of the object (6), which differ in a recording direction and/or an illumination direction during recording, and wherein the control unit (2) is set up to create data sets with entries that each have a reflectance value derived from the images and an associated describe the recording direction and an associated direction of illumination, each data set being associated with a point on the surface (7) of the object (6), the control unit (2) being set up to determine missing entries in the data sets based on entries in data sets that have already been created, characterized in that that the control unit (2) for determining the missing entries in the data sets is also set up to take into account a measure of similarity to the reflectance values derived from the images.

26. Device (1) for determining the reflectance behavior of a surface (7) of an object (6), the device (1) comprising a control unit (2) and a detection unit (4), the detection unit (4) being set up for recording several images of the object (6), which differ in a recording direction and/or an illumination direction during recording, and wherein the control unit (2) is set up to create data sets with entries that each have a reflectance value derived from the images and an associated describe the recording direction and an associated direction of illumination, each data set being associated with a point on the surface (7) of the object (6), the control unit (2) being set up to determine missing entries in the data sets based on entries in data sets that have already been created, characterized in that that the control unit (2) is also set up to determine the missing entries in the data sets, to carry out an average calculation or a weighted average calculation of the reflectance values derived from the images.

27. Device (1) for determining the reflectance behavior of a surface (7) of an object (6), the device (1) comprising a control unit (2) and a detection unit (4), the detection unit (4) being set up for recording several images of the object (6), which are in one Differentiating a recording direction and/or an illumination direction during recording, and wherein the control unit (2) is set up to create data sets with entries that each describe a reflectance value derived from the images and an associated recording direction and an associated illumination direction, each data set having a Point of the surface (7) of the object (6) is assigned, the control unit (2) being set up to determine missing entries in the data records based on entries in data records that have already been created, characterized in that the control unit (2) to determine the missing entries in the data sets is also set up to carry out a smoothing of the reflectance values derived from the images.