WO2025252691A1

WO2025252691A1 - Surface property determination

Info

Publication number: WO2025252691A1
Application number: PCT/EP2025/065248
Authority: WO
Inventors: Christian Lennartz
Original assignee: TrinamiX GmbH
Current assignee: TrinamiX GmbH
Priority date: 2024-06-07
Filing date: 2025-06-03
Publication date: 2025-12-11
Anticipated expiration: 2026-12-07

Abstract

The present invention is in the field of surface property determination. It relates to a method for determining a surface property of an object comprising: a) illuminating a surface of the object with coherent light, b) recording image data of the surface under illumination with an event camera, c) determining a surface property using the image data, and d) outputting the surface property.

Description

Surface Property Determination

Description

The present invention is in the field of surface property determination. In particular, it relates to a method for determining a surface property, a use of the surface property for authenticating an object, a system for determining a surface property, a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method for determining a surface property.

Background

The determination of surface properties plays an important role in different areas. For example, the determination of surface materials can be a powerful tool to authenticate a person by distinguishing a real face from a spoofing mask as disclosed in WO 2023/156315 A1 . The detection of a surface property can also be useful for product authentication as disclosed in WO 2023/156460 A1. However, distinguishing similar surface properties reliably imposes high demands towards camera hardware and image processing.

US 2022/0253519 A1 discloses a face authentication system including an event detector detecting a change in luminance of a pixel. However, for demanding applications, there is a demand for further increase of security.

US 2016/0106327 A1 discloses an apparatus detecting a change in a blood flow based on a sensed change of the laser speckle.

It was therefore an object of the present disclosure to provide surface property determination with high accuracy, high speed and moderate cost.

Summary

In one aspect it relates to a method for determining a surface property of an object comprising: a) illuminating a surface of the object with coherent light, b) recording image data of the surface under illumination with an event camera, c) determining a surface property using the image data, and d) outputting the surface property.

In another aspect it relates to a use of the surface property of any one of the preceding claims for authenticating an object.

In another aspect it relates to a system for determining a surface property of an object comprising: a) a projector configured to illuminate a surface of the object with coherent light, b) an event camera configured to record image data of the surface under illumination, c) a processor configured to determine a surface property using the image data, and d) an output configured to output the surface property. In another aspect it relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method for determining a surface property of an object comprising: a) illuminating a surface of the object with coherent light, b) recording image data of the surface under illumination with an event camera, c) determining a surface property using the image data, and d) outputting the surface property.

By using an event camera for recoding the surface under illumination with coherent light it is possible to resolve the quick changes of the speckles of the coherent light or movements, even very quick and short movements. Any static part of the scene produces very little image data. In this way, high spatial and temporal resolution can be obtained for the illuminated surface with moderate image data size. This increases the accuracy of the surface property determination, i.e. subtle differences in surface properties can be distinguished. At the same time, a faster determination is possible with the same hardware. This enables surface determination in high-speed applications, for example in industrial quality control. In addition, disturbances from motion blur, for example in mobile devices which cannot be held perfectly still, can be decreased.

The term "surface property” may refer to property of the outer layer of an object which is exposed to the environment, for example air. The outer layer may have a thickness according to the penetration depth of light into the object, for example 100 pm or 1 mm. The surface property includes the material or the chemical composition, the surface texture, like the surface roughness or smoothness. The surface property may further relate to temporal and/or local changes of the surface. The term "temporal” changes may refer to changes over time, for example periodic changes, for example within a time period of 1 pis to 10 s or 10 pis to 1 s. The term "local” may refer to geometric size below the size of the image or below the size of the object, for example 10 pm to 10 cm, or 100 pm to 1 cm, such as 0.5 mm to 5 mm. Examples for temporal and/or local changes include local movements within the surface like swelling, pulsing, vibrations, perfusions, fluctuations or waves. In case the object is a human, a surface property may be a condition measure or a vital sign, i.e. a surface property specific for the skin of a human such as blood perfusion in the skin.

The term "object” may refer to any object which can be measured with light. An object can be a living body, for example a human, or a non-living object. An object may refer to a complete object or a small piece thereof, for example a sample extracted from the object. In case of a living body, the object may refer to a body part, for example face or hand.

The surface is illuminated with coherent light. The term "light” may refer to electromagnetic radiation in one or more of the infrared, the visible and the ultraviolet spectral range. Herein, the term "ultraviolet spectral range”, generally, refers to electromagnetic radiation having a wavelength of 1 nm to 380 nm, preferably of 100 nm to 380 nm. Further, in partial accordance with standard ISO-21348 in a valid version at the date of this document, the term "visible spectral range”, generally, refers to a spectral range of 380 nm to 760 nm. The term "infrared spectral range” (IR) generally refers to electromagnetic radiation of 760 nm to 1000 pm, wherein the range of 760 nm to 1.5 pm is usually denominated as "near infrared spectral range” (NIR) while the range from 1.5 p to 15 pm is denoted as "mid infrared spectral range” (MidlR) and the range from 15 pm to 1000 pm as "far infrared spectral range” (FIR). Preferably, light used for the typical purposes of the present invention is light in the infrared (IR) spectral range, more preferred, in the near infrared (NIR) and/or the mid infrared spectral range (MidlR), especially the light having a wavelength of 1 m to 5 pm, preferably of 1 pm to 3 pm.

The term "coherent light” may refer to light which is able to exhibit interference effects due to a consistent and well- defined phase relationship between its oscillating waves. Coherent light may also include partially coherent light, i.e. light with a non-perfect mutual phase relationship.

The term "illuminate” may refer to the process of exposing at least one element to light. The term "projector” may refer to a device configured for generating or providing light in the sense of the above-mentioned definition. The projector may be a pattern projector, a floodlight projector or both either at the same time or the projector may repeatedly switch from illuminating patterned light to floodlight.

The term "pattern projector” may refer to a device configured for generating or providing at least one light pattern, in particular at least one infrared light pattern. The term "light pattern” may refer to at least one pattern comprising a plurality of light spots. The light spot may be at least partially spatially extended. At least one spot or any spot may have an arbitrary shape. In some cases a circular shape of at least one spot or any spot may be preferred. The spots may be arranged by considering a structure of a display comprised by a device that is further comprising the optoelectronic apparatus. Typically, an arrangement of an OLED-pixel-structure of the display may be considered. The term "infrared light pattern” may refer to a light pattern comprising spots in the infrared spectral range. The infrared light pattern may be a near infrared light pattern. The infrared light may be coherent. The infrared light pattern may be a coherent infrared light pattern.

The pattern projector may be configured for emitting light at a single wavelength, e.g. in the near infrared region. In other embodiments, the pattern projector may be adapted to emit light with a plurality of wavelengths, e.g. for allowing additional measurements in other wavelengths channels.

The infrared light pattern may comprise at least one regular and/or constant and/or periodic pattern such as a triangular pattern, a rectangular pattern, a hexagonal pattern or a pattern comprising further convex tilings. For example, the infrared light pattern is a hexagonal pattern, preferably a hexagonal infrared light pattern, preferably a 2/5 hexagonal infrared light pattern.

Using a periodical 2/5 hexagonal pattern can allow distinguishing between artefacts and usable signal.

At least one of the infrared light spots may be associated with a beam divergence of 0.2° to 0.5°, preferably 0.1 ° to 0.3°. The term "beam divergence” may refer to at least one measure of an increase in at least one diameter and/or at least one diameter equivalent, such as a radius, with a distance from an optical aperture from which the beam emerges. The measure may be an angle or an angle equivalent. In the context of the present invention, typically, a beam divergence may be determined at 1/e².

The pattern projector may comprise at least one pattern projector configured for generating the infrared light pattern. The pattern projector may comprise at least one emitter, in particular a plurality of emitters. The term "emitter” may refer to at least one arbitrary device configured for providing at least one light beam. The light beam may generate the infrared light pattern. The emitter may comprise at least one element selected from the group consisting of at least one laser source such as at least one semi-conductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separate confinement heterostructure laser, at least one quantum cascade laser, at least one distributed Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one Indium Arsenide laser, at least one Gallium Arsenide laser, at least one transistor laser, at least 50 one diode pumped laser, at least one distributed feedback lasers, at least one quantum well laser, at least one interband cascade laser, at least one semiconductor ring laser, at least one vertical cavity surface emitting laser (VCSEL); at least one non-laser light source such as at least one LED or at least one light bulb. For example, the pattern projector comprises at least one least one VCSEL, preferably a plurality of VCSELs. The plurality of VCSELs may be arranged in at least one array, e.g. comprising a matrix of VCSELs. The VCSELs may be arranged on the same substrate, or on different substrates. The term "vertical-cavity surface-emitting laser” may refer to a semiconductor laser diode configured for laser beam emission perpendicular with respect to a top surface. Examples for VCSELs can be found e.g. in en.wikipedia.org/wikiA/erticalcavity_surface-emitting_laser. VCSELs are generally known to the skilled person such as from WO 2017/222618 A. Each of the VCSELs is configured for generating at least one light beam. The plurality of generated spots may be associated with the infrared light pattern. The VCSELs may be configured for emitting light beams at a wavelength range from 800 to 1000 nm. For example, the VCSELs may be configured for emitting light beams at 808 nm, 850 nm, 940 nm, and/or 980 nm. Preferably the VCSELs emit light at 940 nm, since terrestrial sun radiation has a local minimum in irradiance at this wavelength, e.g. as described in CIE 085-1989 „Solar spectral Irradiance”.

The pattern projector may comprise at least one optical element configured for increasing, e.g. duplicating, the number of spots generated by the pattern projector. The pattern projector, particularly the optical element, may comprises at least one diffractive optical element (DOE) and/or at least one metasurface element. The DOE and/or the metasurface element may be configured for generating multiple light beams from a single incoming light beam. Further arrangements, particularly comprising a different number of projecting VCSEL and/or at least one different optical element configured for increasing the number of spots may be possible. Other multiplication factors are possible. For example, a VCSEL or a plurality of VCSELs may be used and the generated laser spots may be duplicated by using at least one DOE.

The pattern projector may be configured to illuminated patterned light comprising less than 4000 light beams, preferably less than 3000 light beams, more preferably less than 2000 light beams, most preferably less than 1000 light beams. For example, the patterned light may comprise 100 to 4000 light beams or 200 to 3000 light beams or 300 to 2000 light beams or 500 to 1000 light beams.

The pattern projector may comprise at least one transfer device. The term "transfer device”, also denoted as "transfer system” may refer to one or more optical elements which are adapted to modify the light beam, particularly the light beam used for generating at least a portion of the infrared light pattern, such as by modifying one or more of a beam parameter of the light beam, a width of the light beam or a direction of the light beam. The transfer device may comprise at least one imaging optical device .The transfer device specifically may comprise one or more of: at least one lens, for example at least one lens selected from the group consisting of at least one focus-tunable lens, at least one aspheric lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multilens system; at least one holographic optical element; at least one meta optical element. Specifically, the transfer device comprises at least one refractive optical lens stack. Thus, the transfer device may comprise a multi-lens system having refractive properties.

The pattern projector may be configured for emitting modulated or non-modulated light. In case a plurality of emitters is used, the different emitters may have different modulation frequencies, e.g. which can be used for distinguishing the light beams.

The light beam or light beams generated by the pattern projector may propagate parallel to an optical axis. The pattern projector may comprise at least one reflective element, preferably at least one prism, for deflecting the illuminating light beam onto the optical axis. As an example, the light beam or light beams, such as the laser light beam, and the optical axis may include an angle of less than 10°, preferably less than 5° or even less than 2°. Other embodiments, however, are feasible. Further, the light beam or light beams may be on the optical axis or off the optical axis. As an example, the light beam or light beams may be parallel to the optical axis having a distance of less 10 than 10 mm to the optical axis, preferably less than 5 mm to the optical axis or even less than 1 mm to the optical axis or may even coincide with the optical axis.

The term "flood projector” may refer to at least one device configured for providing substantially continuous spatial illumination. The flood projector may illuminate a measurement area, such as a user, a portion of the user and/or a face of the user, with a spatially constant or essentially constant illumination intensity. The term "flood light” may refer to substantially continuous spatial illumination, in particular diffuse and/or uniform illumination. The flood light has a wavelength in the infrared range, in particular in the near infrared range. The flood projector may comprise at least one least one VCSEL, preferably a plurality of VCSELs. The term "substantially continuous spatial illumination” may refer to uniform spatial illumination, wherein areas of non-uniform are possible.

A relative distance between the flood projector and the pattern projector may be below 3.0 mm. The relative distance between the flood projector and the pattern projector may be below 2.5 mm, preferably below 2.0 mm. The pattern projector and the flood projector may be combined into one module. For example, the pattern projector and the flood projector may be arranged on the same substrate, in particular having a minimum relative distance. The minimum relative distance may be defined by a physical extension of the flood projector and the pattern projector. Arranging the pattern projector and the flood projector having a relative distance below 3.0 mm can result in decreased space requirement of the two projectors. In particular, said projectors can even be combined into one module. Such a reduced space requirement can allow reducing the transparent area(s) in a display necessary for operation of the projectors) behind the display.

In an embodiment, the pattern projector and the flood projector may comprise at least one VCSEL, preferably a plurality of VCSELs. The pattern projector may comprise a plurality of first VCSELs mounted on a first platform. The flood projector may comprise a plurality of second VCSELs mounted on a second platform. The second platform may be beside the first platform. The optoelectronic apparatus may comprise a heat sink. Above the heat sink a first increment comprising the first platform may be attached. Above the heat sink a second increment comprising the second platform may be attached. The second increment may be different from the first increment. Thus, the first platform may be more distant to the optical element configured for increasing, e.g. duplicating, the number of spots. The second platform may be closer to the optical element. The beam emitted from the second VCSEL may be defocused and thus, form overlapping spots. This leads to a substantially continuous illumination and, thus, to flood illumination. The projector may be positioned such that it can illuminate light through the transparent display. Hence, light emitted by the projector crosses the transparent display before it impinges on the person. From the person's view, the projector is placed behind the transparent display.

Image data of the surface under illumination is recorded with an event camera. The term "event camera” may refer to an imaging sensor that responds to local changes in brightness. An event camera may also be referred to as neuro- morphic camera, silicon retina or dynamic vision sensor. An event camera may comprise pixels which independently respond to changes in brightness as they occur. Each pixel may store a reference brightness level, and continuously compare it to the current light intensity. If the difference in light intensity exceeds a preset threshold, that pixel may reset its reference level and generate data, for example a discrete data packet that contains the pixel address and timestamp. The packet may also contain the polarity, for example an indictor indicating an increase or a decrease of a brightness change, or an instantaneous measurement of the illumination level. An event camera may output an asynchronous stream of data packets triggered by changes in scene illumination. Event camera may include temporal contrast sensors, contrast detection sensors and dynamic vision sensors.

The event camera may comprise an array of pixels, for example an array of CMOS or CCD pixels. The event camera may comprise a readout circuit configured to generate data packages for each pixel. The readout circuit may, for example, comprise a subthreshold MOS based logarithmic photocurrent-to-voltage converter, an asynchronous deltamodulation (ADM) or "level-crossing" sampler, a voltage comparator, a logic with ADM control, and an interface and state-logic to a read-out periphery.

The event camera may have a dynamic range of 20 to 130 dB, for example 40 to 100 dB or 80 to 120 dB. The event camera may have a field of view between 10°x10° and 75°x75°, preferably 55°x65°. The event camera may have a spatial resolution below 2 megapixels, for example 0.1 to 1.5 megapixels, such as 0.3 to 1 .0 megapixels. The event camera may have an equivalent framerate of 50 000 to 300 000 frames per second.

The event camera may comprise further elements, such as one or more optical elements, e.g. one or more lenses. As an example, the optical sensor may be a fix-focus camera, having at least one lens which is fixedly adjusted with respect to the camera. Alternatively, however, the camera may also comprise one or more variable lenses which may be adjusted, automatically or manually. Other cameras, however, are feasible.

The event camera may be configured to switch from an event mode to a conventional mode, i.e. the pixel values are read out at a predefined point in time, for example all at the same time, also referred to as global shutter, or row after row, also referred to as rolling shutter. The event camera may be configured to record image data comprising pattern images in event mode and flood images in conventional mode. Parts of the pixels of the event camera may be read out in a conventional way, for example with a global shutter, and from the remaining pixels events are read out. For example, 20 to 50 %, such as one third or one fourth, of the pixels are read according to a global shutter. The pixels read out according to a global shutter may record a flood image while the pixels from which events are read out may record pattern images. Alternatively, an event camera and a conventional camera may be used. The event camera may record pattern images and the conventional camera may record flood images. In this way, static scene features can be recorded while retaining high resolution for quickly changing features such as the speckle pattern. The term "image data” may refer to a data structure comprising one or more than one image or data from which one or more than one images can be generated. Image data may comprise multiple data packets, wherein each data packet represents an event of a pixel of the event camera. A data packet may comprise an identifier for the pixel causing the event and a timestamp. A data packet may further comprise a polarity indicator, i.e. an indicator indicating an increase or a decrease of the light intensity. A data packet may further comprise a value for the light intensity measured at the event. The data packets may be combined to generate an image, for example by integrating all events for each pixel within a time frame. Such time frame may be 0.1 to 10 ms, such as 0.5 to 3 ms, for example 1 ms. Image data may comprise pattern images or flood images. Image data may comprise pattern images and flood images.

The term "pattern image” may refer to an image generated by the camera while illuminating the infrared light pattern, e.g. on an object and/or a user. The pattern image may comprise an image showing a user, in particular at least parts of the face of the user, while the user is being illuminated with the infrared light pattern, particularly on a respective area of interest comprised by the image. The pattern image may be generated by imaging and/or recording light reflected by an object and/or user which is illuminated by the infrared light pattern. The pattern image showing the user may comprise at least a portion of the illuminated infrared light pattern on at least a portion the user. For example, the illumination by the pattern illumination source and the imaging by using the optical sensor may be synchronized, e.g. by using at least one control unit of the optoelectronic apparatus.

The term "flood image” may refer to an image generated by the camera while illumination source is illuminating infrared flood light, e.g. on an object and/or a user. The flood image may comprise an image showing a user, in particular the face of the user, while the user is being illuminated with the flood light. The flood image may be generated by imaging and/or recording light reflected by an object and/or user which is illuminated by the flood light. The flood image showing the user may comprise at least a portion of the flood light on at least a portion the user. For example, the illumination by the flood illumination source and the imaging by using the optical sensor may be synchronized, e.g. by using at least one control unit of the optoelectronic apparatus.

The projector and/or the event camera may be placed behind a transparent display. The term "display” may refer to an arbitrary shaped device configured for displaying an item of information. The item of information may be arbitrary information such as at least one image, at least one diagram, at least one histogram, at least one graphic, text, numbers, at least one sign, or an operating menu. The display may be or may comprise at least one screen. The display may have an arbitrary shape, e.g. a rectangular shape. The display may be a front display of a device.

The display may be or may comprise at least one organic light-emitting diode (OLED) display. The term "organic light emitting diode” may refer to a light-emitting diode (LED) in which an emissive electroluminescent layer is a film of organic compound configured for emitting light in response to an electric current. The OLED display may be configured for emitting visible light. The display, particularly a display area, may be covered by glass. In particular, the display may comprise at least one glass cover.

The transparent display may be at least partially transparent. The term "at least partially transparent” may refer to a property of the display to allow light, in particular of a certain wavelength range, e.g. in the infrared spectral region, in particular in the near infrared spectral region, to pass at least partially through. For example, the display may be semitransparent in the near infrared region. For example, the display may have a transparency of 20 % to 50 % in the near infrared region. The display may have a different transparency for other wavelength ranges. For example, the display may have a transparency of > 80 % for the visible spectral range, preferably > 90 % for the visible spectral range. The transparent display may be at least partially transparent over the entire display area or only parts thereof. Typically, it is sufficient if only those parts of the display area are at least partially transparent trough which light needs to pass from the projector or to the camera.

The display may comprise a display area. The term "display area” may refer to an active area of the display, in particular an area which is activatable. The display may have additional areas such as recesses or cutouts. The display may have a first area associated with a first pixel per inch (PPI) value and a second area associated with a second PPI value. The first PPI value may be lower than the second PPI value, preferably first PPI value is equal to or below 400 PPI, more preferably the second PPI value may be equal to or higher than 300 PPI. The first PPI value may be associated with the at least one continuous area being at least partially transparent.

The event camera may be positioned such that it can receive light from the object through the transparent display. Light reflected or refracted from the person firstly crosses the transparent display before it impinges on the event camera. From the object's view, the event camera may be placed behind the transparent display.

The system for determining a surface property comprises a processor. The processor may be a logic circuitry configured for performing basic operations of a computer or system, and/or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor may be configured for processing basic instructions that drive the computer or system. As an example, the processor may comprise at least one arithmetic logic unit (ALU), at least one floating-point unit (FPU), such as a math co-processor or a numeric co-processor, a plurality of registers, specifically registers configured for supplying operands to the ALU and storing results of operations, and a memory, such as an L1 and L2 cache memory. In particular, the processor may be a multi-core processor. Specifically, the processor may be or may comprise a central processing unit (CPU). Additionally or alternatively, the processor may be or may comprise a microprocessor, thus specifically the processor's elements may be contained in one single integrated circuitry (IC) chip. Additionally or alternatively, the processor may be or may comprise one or more application-specific integrated circuits (ASICs) and/or one or more field-programmable gate arrays (FPGAs) and/or one or more tensor processing unit (TPU) and/or one or more chip, such as a dedicated machine learning optimized chip, or the like. The processor specifically may be configured, such as by software programming, for performing one or more evaluation operations. At least one or any component of a computer program configured for performing the authentication process may be executed by the processing device. Alternatively or in addition, the processor may be or may comprise a connection interface. The connection interface may be configured to transfer data from the device to a remote device; or vice versa. At least one or any component of a computer program configured for performing the authentication process may be executed by the remote device.

The processor may be configured, such as by software programming, for performing one or more evaluation operations. At least one or any component of a computer program configured for performing the authentication process may be executed by the processing device. Alternatively or in addition, the processor may be or may comprise a connection interface. The connection interface may be configured to transfer data from the device to a remote device; or vice versa. At least one or any component of a computer program configured for performing the authentication process may be executed by the remote device. The surface property may be a material, so determining a surface property may mean determining material data using the image data. Such determination may be accomplished by a material model configured to receive image data as input and output material data. A material model may be a deterministic material model, a data-driven material model or a hybrid material model. The deterministic material model, preferably, reflects physical phenomena in mathematical form, e.g., including first-principles material models. A deterministic material model may comprise a set of equations that describe an interaction between the material and the patterned electromagnetic radiation. A data- driven material model may be a classification material model. A hybrid material model may be a classification material model comprising at least one machine-learning architecture with deterministic or statistical adaptations and material model parameters. Statistical or deterministic adaptations may be introduced to improve the quality of the results since those provide a systematic relation between empiricism and theory. In an embodiment, the data-driven material model may be a classification material model. The classification material model may comprise at least one machine-learning architecture and material model parameters. For example, the machine-learning architecture may be or may comprise one or more of: linear regression, logistic regression, random forest, piecewise linear, nonlinear classifiers, support vector machines, naive Bayes classifications, nearest neighbors, neural networks, convolutional neural networks, generative adversarial networks, support vector machines, or gradient boosting algorithms or the like. In the case of a neural network, the material model can be a multi-scale neural network or a recurrent neural network (RNN) such as, but not limited to, a gated recurrent unit (GRU) recurrent neural network or a long short-term memory (LSTM) recurrent neural network. The data-driven material model may be parametrized according to a training data set. The data-driven material model may be trained based on the training data set. Training the material model may include parametrizing the material model. The term training may also be denoted as learning. The term specifically may refer to a process of building the classification material model, in particular determining and/or updating parameters of the classification material model. Updating parameters of the classification material model may also be referred to as retraining. Retraining may be included when referring to training herein. In an embodiment, the training data set may include at least one image and material information.

Extracting material data from the image data with a data-driven material model may comprise providing the image data to a data-driven material model. Additionally or alternatively, extracting material data from the image with a data- driven material model may comprise may comprise generating an embedding associated with the image based on the data-driven material model. An embedding may refer to a lower dimensional representation associated with the image such as a feature vector. Feature vector may be suitable for suppressing the background while maintaining the material signature indicating the material data. In this context, background may refer to information independent of the material signature and/or the material data. Further, background may refer to information related to biometric features such as facial features. Material data may be determined with the data-driven material model based on the embedding associated with the image. Additionally or alternatively, extracting material data from the image by providing the image to a data-driven material model may comprise transforming the image into material data, in particular a material feature vector indicating the material data. Hence, material data may comprise further the material feature vector and/or material feature vector may be used for determining material data.

A surface property of a person may be a condition measure. The processor may be configured to determine a condition measure of the person using the image data. The term "condition measure” may refer to a measure suitable for determining the condition of a person. A condition of a person may be a physical and/or mental condition. A physical condition may be associated with physical stress level, fatigue, excitation, suitability of performing a certain task of a person or a medical condition. A mental condition may be associated with mental stress level, attentiveness, concentration level, excitation, suitability of performing a certain task of a person or the like. Such a certain task may require concentration, attention, wakefulness, calming or similar characteristics of the person. Condition measures may indicate a condition of a person. Condition measures may be one or several of the following: heart rate, blood pressure, respiratory rate. In some embodiments, the condition of a person may be critical corresponding to a high value of the condition measure and the condition of a person may be non-critical corresponding to a low value of the condition measure. Followingly, the critical condition measure according to these embodiments may be equal or lower than a threshold and a non-critical condition measure may be lower than a threshold. In other embodiments, the condition of a person may be critical corresponding to a low value of the condition measure and the condition of a person may be non-critical corresponding to a high value of the condition measure. Followingly the critical condition measure according to these embodiments may be equal or higher than a threshold and a non-critical condition measure may be lower than a threshold. A critical condition measure may be associated with a high stress level, low attentiveness, low concentration level, high fatigue, high excitation.

The condition measure of a person may be determined based on the motion of a body fluid, preferably blood, most preferably red blood cells. The motion of body fluids is not constant over time but changes due to activity of parts of the person, e.g. the heart. Such a change in motion may be determined based on a change in feature contrast over time. A high difference between values of feature contrast at different points in time may be associated with a fast change in motion. A low difference between values of feature contrast at different points in time may be associated with a slow change in motion. The change in motion of a body fluid, preferably blood, may be periodically associated with a corresponding motion frequency. Accordingly, the feature contrast may change periodically with the corresponding motion frequency. The motion frequency may correspond to the length of a period associated with the periodic change in feature contrast. In some embodiments, half of a period may be comprised in the at least two images. In other embodiments, one or several periods may be comprised in the at least two images. Preferably, pattern feature associated with the same part of a person may be used for determining the condition of a person. This is advantageous due to the fact that the blood perfusion and thus, the feature contrast across different parts of the body varies. In some embodiments, at least one condition measure may be determined based on the feature contrast.

A feature contrast may represent a measure for a contrast of an intensity distribution within an area of a pattern, in particular within the area of a pattern feature. Additionally or alternatively, feature contrast may refer to a measure for a contrast associated with a pattern feature. The feature contrast may be determined of the at least two pattern features. The feature contrast may indicate at least two feature contrast values, wherein the first of the at least two feature contrast values may be associated with the first of the at least two images and/or the first of the at least two pattern features and the second of the at least two feature contrast values may be associated with the second of the at least two images and/or the second of the at least two pattern features. In particular, the first of the at least two pattern features, in particular the at least one first pattern feature may be associated with the first of the at least two images, in particular the at least one first image. In particular, the second of the at least two pattern features, in particular the at least one second pattern feature may be associated with the second of the at least two images, in particular the at least one second image. The feature contrast may be determined of the first pattern feature of the at least two images and for the second pattern feature of the at least two images. The feature contrast may be determined separately for the at least two pattern features of the at least two images.

The feature contrast may indicate and/or may comprise determining at least two feature contrast values associated with the at least two pattern features. Determining the feature contrast of the at least two pattern features may include determining a first feature contrast value associated with the first pattern feature and determining a second feature contrast value associated with the second pattern feature. A feature contrast value may be determined by determining the ratio of a standard deviation of a pattern feature intensity and a mean of the pattern feature intensity. Pattern feature intensity may comprise intensity values associated with the corresponding pattern feature.

In particular, a feature contrast value K over an area of the pattern may be expressed as a ratio of standard deviation cr to the mean pattern feature intensity <l>, i.e.,

Feature contrast values are generally distributed between 0 and 1 . Feature contrast may be determined based on at least one pattern feature. Fol lowingly, at least two feature contrast values may be determined based on the at least two pattern features.

The complete pattern of the image may be used for determining the feature contrast. Alternatively, a section of the pattern may be used for determining the feature contrast. The section of the pattern, preferably, represents a smaller area of the pattern than an area of the pattern. The area may be of any shape. The section of the pattern may be obtained by cropping the image. The feature contrast may be different for different parts of an object. Different parts of the object may correspond to different parts of the image. Followi ngly , the feature contrast may be different for different parts of a image.

The image data may comprise an image set, i.e. a set of multiple images, for example at least two images. The set of images may be generated at different points in time. In some embodiments, at least one or more than one feature contrast values may be determined of the at least one pattern feature. A feature contrast value may correspond to a numerical value of a feature contrast.

The image set may comprise a time series. The time series may comprise images separated by a constant time interval associated with an imaging frequency or changing time intervals. Preferably, the time series is constituted such that the imaging frequency is at least twice the motion frequency. This is known as the Nyquist theorem. For higher resolution more images than at least required by the Nyquist theorem may be received.

For the determination of the condition measure, an indication of an interval between the different points in time where the at least two images are generated is received. The indication of the interval comprises measure(s) suitable for determining the time between the different points in time where the at least two images are generated.

A frequency is a reciprocal value of the length of a period. The length of a period may be determined by the interval between two images comprising a share of the period of the heart beating or the heart cycle. In a normal human at rest the heart beats between 60 and 80 times per minute corresponding to a resting heart rate of 60 to 80 beats per minute (bpm). The resting heart rate may be lower, e.g. if the human is sportive or suffers from bradycardia. In situation where the human is active, the heart rate may increase up to 230 bpm. Animals may have heart rates ranging from 6 to 1000 bpm. The images may be generated depending on the expected heart rate of the object examined. The interval between the images may be chosen to be up to 10 seconds. In the case of a human, the interval may be chosen up to 2 seconds. Followingly, the imaging frequency may be chosen to be at least 12 images per minute or at least 60 images in the case of a human. In an example, the method may be used for determining the heart rate of a human. For this purpose, an imaging frequency of 60 images per minute may be chosen. As the feature contrast is determined, one may recognize that the imaging frequency may be too low. In such a case, the imaging frequency may be increased such that the condition measure may be determined. Alternatively, the imaging frequency for imaging a human may be chosen to be a high frequency such as 460 images per minute. A heart rate may be determined based on the at least two images and an indication of the interval between the at least two different points in time indicating an interval of 0.13 seconds. In the example, the human may have a heart rate in the range of 60 to 80 bpm. Followingly, the imaging frequency may be adjusted according to expected and/or predetermined condition measures.

In an example, the interval may comprise a half, a full, double length of a period or the like. The interval may be between at least two images. Followingly, the at least two images may be separated by a half, a full, double length of a period or the like. In the exemplary case of three images indication of one or two different intervals may be received. If more than two images are received, the indication of the interval may comprise an indication of an interval between the first and the second image and/or an interval between the first and the third image (or every other image if more than three images may be received) and/or an interval between the second and the third image (or every other image if more than three images may be received). This applies accordingly to other scenarios with a different amount of images as the skilled person will recognize. Measures for the indication of the interval may be at least two points in time corresponding to the different points in time where the at least two images are generated and/or the time that passed between the different points in time and/or an imaging frequency associated with the generation of the images. The at least two points in time may be determined based on a timestamp of the at least two images. The imaging frequency may comprise a selected value. The imaging frequency may be selected based on the expected condition measure, e.g. an expected heart rate. Alternatively, the image frequency of a video may be used to determine an imaging frequency. An expected heart rate may comprise a heart rate associated with the object monitored. In some embodiments, estimation of condition measure may be used to select the imaging frequency. An estimation of condition measure may take the living species and its surrounding into account.

The image data set may comprise at least one first image and at least one second image. The at least one first image and the at least one second image may be generated at different points in time, in particular at at least two different points in time. The at least one first image and the at least one second image may be generated while the object is illuminated by patterned coherent electromagnetic radiation. The at least one first image may show at least one first pattern feature, preferably at least two first pattern features, formed by illuminating at least a part of the object by the patterned coherent electromagnetic radiation. The at least one second image may show at least one second pattern feature, preferably at least two second pattern features, formed by illuminating at least a part of the object by the patterned coherent electromagnetic radiation. In particular, the at least one first pattern feature and the at least one second pattern feature may be associated with the same body part of the object.

A feature contrast may be determined of the at least one first pattern feature and the at least one second pattern feature. The feature contrast may indicate a first feature contrast value associated with the at least one first pattern feature and a second feature contrast value associated with the at least one second pattern feature. A condition measure may be determined based on the feature contrast and the indication of the at least one interval by providing the feature contrast of the at least one first pattern feature and the at least one second pattern feature and the indication of the at least one interval between the generation of the at least one first image and the at least one second image to a data-driven model, wherein the data-driven model is parametrized on a training data set including historical feature contrasts indicating a plurality of first feature contrast values associated with a plurality of first pattern features and a plurality of second feature contrast values associated with the a plurality of second pattern features, a plurality of historical indications of the at least one interval and a plurality of historical condition measures.

The condition measure of the object may be determined based on the feature contrast and the indication of the at least one interval by providing the feature contrast of the at least two pattern features and the indication of the at least one interval between the generation of the at least two pattern images to a data-driven model, wherein the data- driven model is parametrized on a training data set including historical feature contrasts, historical indications of the at least one interval and historical condition measures. The feature contrast may indicate at least two feature contrast values associated with the at least two pattern features.

The condition measure of the object may be determined based on the first feature contrast, the second feature contrast and the indication of the interval. This may be achieved by providing the first feature contrast, the second feature contrast and the indication of the interval to a data-driven model, wherein the data-driven model may be parametrized based on a training data set including historical first feature contrasts, historical second feature contrasts, historical indications of the at least one interval and historical condition measures.

Providing the feature contrast may include providing a first feature contrast value and a second feature contrast value. The data-driven model may be parametrized based on the training data set to provide and/or output a condition measure based on being provided with the feature contrast of the at least two pattern features and the indication of the at least one interval between the generation of the at least two pattern images. The data-driven model may be trained based on the training data set including historical feature contrasts, historical indications of the at least one interval and historical condition measures to provide and/or output a condition measure based on being provided with the feature contrast of the at least two pattern features and the indication of the at least one interval between the generation of the at least two pattern images. The condition measure of the object may be determined based on the feature contrast and the indication of the at least one interval by providing the feature contrast of the at least two pattern features and the indication of the at least one interval between the generation of the at least two pattern images to a data-driven model, wherein the data-driven model is trained on a training data set including historical feature contrasts, historical indications of the at least one interval and historical condition measures. The data-driven model may receive the feature contrast and the indication of the at least one interval at an input layer and/or may provide a condition measure based on having received the feature contrast and the indication of the at least one interval at an input layer. The data-driven model may comprise at least one machine learning architecture, in particular a deep learning architecture. For example, the data-driven model may be a neural network such as a CNN, in particular a 3D CNN, or a transformer. Further, the data-driven model may be a transformer network.

The at least two pattern features may be associated with the at least two images. Preferably, a first pattern feature of the at least two pattern features may be shown in the first image of the at least two images and a second pattern feature of the at least two pattern features may be shown in the second image of the at least two images. A feature contrast may be determined of the at least two pattern features including the first pattern feature and the second pattern feature. In particular, determining a feature contrast of the at least two pattern features including the first pattern feature and the second pattern feature may include determining a first feature contrast including a first feature contrast value of the first pattern feature of the first image of the at least two images and a second feature contrast value of the second pattern feature of second first image of the at least two images. Determining a condition measure of the object based on the feature contrast and the indication of the at least one interval may include determining a condition measure based on a first feature contrast value of the first pattern feature of the first image of the at least two images and a second feature contrast value of the second pattern feature of second first image of the at least two images.

In some embodiments, the condition measure may be determined using an algorithm that may implement a mechanistic model or a data-driven model. The mechanistic model, preferably, reflects physical phenomena in mathematical form, e.g., including first-principle models. A mechanistic model may comprise a set of differential equations that describe an interaction between the object and the coherent electromagnetic radiation thereby resulting in a specific condition measure. In particular, flow of a fluid and/or the geometry of the object may be represented by the mechanistic model. The mechanistic model may comprise relations between the at least two images, the indication of the point in time and the condition measure. For this purpose, the relations may be suitable for determining the time interval between the at least two images and determining a motion frequency corresponding to the beats per time unit (heart rate). Based on the required input to the mechanistic model leading to a feature contrast as determined from the pattern of the at least two images, an associated condition measure can be determined with the mechanistic model. Including a pulse wave analysis into the mechanistic model may be suitable for determining the blood pressure as another condition measure. To do so, the velocity of the pulse wave may be determined. This information may be comprised in the at least two images and the indication about an interval. The absorption and reflection behaviour of the part of the object may indicate the aspiration level comprised in the at least one pattern feature of the images. Oxygen-rich blood absorbs and thus, reflects light differently than oxygen-poor blood. Other condition measures may be determined by deploying relations between pattern features and the condition measure.

Preferably, the data-driven model may a parametrized classification model. The classification model may comprise at least one machine-learning architecture and model parameters. For example, the machine-learning architecture may be or may comprise one or more of: linear regression, logistic regression, random forest, piecewise linear, nonlinear classifiers, support vector machines, naive Bayes classifications, nearest neighbours, neural networks, convolutional neural networks, generative adversarial networks, support vector machines, or gradient boosting algorithms or the like. In the case of a neural network, the model can be a multi-scale neural network or a recurrent neural network (RNN) such as, but not limited to, a gated recurrent unit (GRU) recurrent neural network or a long short-term memory (LSTM) recurrent neural network. The term "training”, also denoted learning, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, with-out limitation, to a process of building the classification model, in particular determining and/or updating parameters of the classification model. The classification model may be at least partially data-driven. For example, the classification model may be based on experimental data, such as data determined by illuminating a plurality of objects such as humans and recording the images. For example, the training may comprise using at least one training dataset, wherein the training data set comprises images, e.g. of a plurality of humans with known condition measures. For example, if the neural network is a feedforward neural network such as a CNN, a backpropagation-algorithm may be applied for training the neural network. In case of a RNN, a gradient descent algorithm or a backpropagation-through-time algorithm may be employed for training purposes.

The surface property may be the surface roughness. The term "surface roughness” may refer to the lateral and/or vertical extent of surface features. The surface roughness may be evaluated based on the surface roughness measure. The surface roughness measure may quantify the surface roughness. The surface roughness measure may refer to a measure suitable for quantifying the surface roughness. Surface roughness measure may be related to the speckle pattern. For example, surface roughness measure may include at least one of a fractal dimension, speckle size, speckle contrast, speckle modulation, roughness exponent, standard deviation of the height associated with surface features, lateral correlation length, average mean height, root mean square height or a combination thereof. Preferably, the surface roughness measure may be suitable for describing the vertical and lateral surface features. Surface roughness measure may comprise a value associated with the surface roughness measure. Surface roughness measure may refer to a term of a quantity for measuring the surface roughness and/or to the values associated with the quantity for measuring the surface roughness.

The fractal dimension may be determined based on the Fourier transform of the image data and/or the inverse of the Fourier transform of the image data. In particular, the fractal dimension may be determined based on the slope of a linear function fitted to a double logarithmic plot of the power spectral density versus a frequency obtained by Fourier transform. Speckle size may refer to the spatial extent of one or more speckles. Where the speckle size may refer to the spatial extent of more than one speckle, the speckle size may be determined based on an average of more than one speckle sizes and/or a weighting of the more than one speckle sizes. Speckle contrast may refer to a measure for the standard deviation of at least a part of the image data in relation to the mean intensity of at least the part of the image data. The speckle modulation may refer to a measure for the intensity fluctuation associated with the speckles in at least a part of the image data. Roughness exponent, standard deviation of the height associated with surface features, lateral correlation length or a combination thereof may be determined based on the autocorrelation function associated with the double logarithmic plot of the power spectral density versus a frequency obtained by Fourier transform.

Determining the surface roughness based on the speckle pattern may refer to determining the surface roughness based on a distribution of a plurality of speckles in the image data. Determining the surface roughness measure based on the distribution of the plurality of speckles in the image data may refer to determining the distribution of the plurality of speckles in the image data. Determining the distribution of the speckles may comprise determining at least one of fractal dimension associated with the image data, speckle size associated with the image data, speckle contrast associated with the image data, speckle modulation associated with the image data, roughness exponent associated with the image data, standard deviation of the height associated with surface features associated with the image data, lateral correlation length associated with the image data, average mean height associated with the image data, root mean square height associated with the image data or a combination thereof. Additionally or alternatively, determining the surface roughness measure may comprise determining at least one of fractal dimension associated with the image data, speckle size associated with the image data, speckle contrast associated with the image data, speckle modulation associated with the image data, roughness exponent associated with the image data, standard deviation of the height associated with surface features associated with the image data, lateral correlation length associated with the image data, average mean height associated with the image data, root mean square height associated with the image data or a combination thereof.

As the speckles are caused by the irregularities of the surface, the speckles reflect the roughness of the surface. Fol- lowingly, determining the surface roughness measure based on the speckles in the image data utilizes the relation between the speckle distribution and the surface roughness. Thereby, a low-cost, efficient and readily available solution for surface roughness evaluation apart from fixed medical or cosmetic context is enabled. Determining the surface roughness measure may be based on the distribution of the speckles in the image data. Determining the surface roughness measure based on the distribution of the speckles in the image data may comprise determining at least one of a size distribution of the speckles, a power spectral density associated with the image data, a fractal dimension associated with the image data, a speckle contrast, a speckle modulation or a combination thereof. Additionally or alternatively, determining the surface roughness measure based on the distribution of the speckles in the image data may comprise providing the image data to a model, in particular a data-driven model, wherein the data-driven model may be parametrized and/or trained based on a training data set comprising one or more image data and one or more corresponding surface roughness measure.

One or more than one surface properties may be determined of an object. A plurality of surface properties may be determined for different parts of the object. A map of the object may be generated containing a plurality of surface properties of the object. The map may associate the surface properties of the object with the location on the object. The map may have the same resolution as the image from which the surface property is determined. In this case, each pixel of the map may correspond to a pixel in the image from which a surface property may be determined. The map may have a lower resolution than the image from which the surface property is determined. In this case, each pixel of the map may correspond to a group of pixels in the image from which a surface property may be determined. The object may be illuminated with patterned coherent light. The map may comprise surface property values for the pattern features. The map may comprise a plurality of values for the speckle contrast or the rate of change of the speckles.

The surface property may be used for authenticating an object. The present disclosure therefore further relates to a method for authentication an object comprising: a) illuminating a surface with coherent light, b) recording image data of the surface under illumination with an event camera, c) determining a surface property using the image data, and d) authenticating the object using the surface property.

The present disclosure further relates to a method for authentication an object comprising: a) illuminating a surface with coherent light, b) recording image data of the surface under illumination with an event camera, c) determining a surface property for a plurality of locations on the object using the image data, and d) generating a map associating the determined surface property value with the location on the object, and e) authenticating the object using the map.

Authenticating the object may be performed by comparing the map with a predetermined map, for example a map stored in memory from an enrollment process or a map stored in memory comprising values expected for an authentic object.

The disclosure further relates to an authenticating system for authentication an object comprising: a) a projector configured to illuminate a surface of the object with coherent light, b) an event camera configured to record image data of the surface under illumination, c) a processor configured to determine a surface property using the image data and authentication the object using the surface property. The disclosure further relates to an authenticating system for authentication an object comprising: a) a projector configured to illuminate a surface of the object with coherent light, b) an event camera configured to record image data of the surface under illumination, c) a processor configured to determining a surface property for a plurality of locations on the object using the image data, generating a map associating the determined surface property value with the location on the object, and authenticating the object using the map.

The method of authentication an object may comprise identifying the object, for example based on the flood image. Particularly therefore, the processor may forward data to a remote device. Alternatively or in addition, the processor may perform the identification of the user based on the flood image, particularly by running an appropriate computer program having a respective functionality. The term "identifying” may refer to identity check and/or verifying an identity of the user. The identifying of the user may comprise analyzing the flood image. The analyzing of the flood image may comprise performing a face verification of the imaged face to be the user's face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user's face, with a template. Determining if the imaged face is the face of the user may comprise identifying the user, in particular determining if the imaged face corresponds to at least one image of the user's face stored in at least one memory, e.g. of the device.

The analyzing may comprise one or more of the following: a filtering; a selection of at least one region of interest; a formation of a difference image between the flood image and at least one offset; an inversion of flood image; a background correction; a decomposition into color channels; a decomposition into hue; saturation; and brightness channels; a frequency decomposition; a singular value decomposition; applying a Canny edge detector; applying a Laplacian of Gaussian filter; applying a Difference of Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Scharr operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high-pass filter; applying a low-pass filter; applying a Fourier transformation; applying a Radon-transfor- mation; applying a Hough-transformation; applying a wavelet-transformation; a thresholding; creating a binary image. The region of interest may be determined manually by a user or may be determined automatically, such as by recognizing the user within the image. In particular, the analyzing of the flood image may comprise using at least one image recognition technique, in particular a face recognition technique. An image recognition technique comprises at least one process of identifying the user in an image. The image recognition may comprise using at least one technique selected from the technique consisting of: color-based image recognition, e.g. using features such as hue, saturation, and value (HSV) or red, green, blue (RGB); template matching, for example as illustrated on https://www.mathworks.com/help/vision/ug/pattern-matching.html; image segment and/or blob analysis e.g. using size, color, or shape; machine learning and/or deep learning e.g. using at least one convolutional neural network.

The neural network may be trained by the user, such as in a training procedure, in which the user is indicated to take at least one or a plurality of pictures showing himself.

The analyzing of the flood image may comprise determining a plurality of facial features. The analyzing may comprise comparing, in particular matching, the determined facial features with template features. The template features may be features extracted from at least one template. The template may be or may comprise at least one image generated in an enrollment process, e.g. when initializing the seat belt monitoring system. Template may be an image of an authorized user. The template features and/or the facial feature may comprise a vector. Matching of the features may comprise determining a distance between the vectors. The identifying of the user may comprise comparing the distance of the vectors to a least one predefined limit, wherein the user is successfully identified in case the distance is smaller than or equal to the predefined limit at least within tolerances. The user declining and/or rejected otherwise.

For example, the image recognition may comprise using at least one model, in particular a trained model comprising at least one face recognition model. The analyzing of the flood image may be performed by using a face recognition system, such as FaceNet, e.g. as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. The trained model may comprises at least one convolutional neural network. For example, the convolutional neural network may be designed as described in M. D. Zeller and R. Fergus, "Visualizing and understanding convolutional networks”, CoRR, abs/1311.2901, 2013, or C. Szegedy et al., "Going deeper with convolutions”, CoRR, abs/1409.4842, 2014. For more details with respect to convolutional neural network for the face recognition system reference is made to Florian Schroff, Dmitry Kalenichenko, James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. As training data labelled image data from an image database may be used. Specifically, labeled faces may be used from one or more of G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, "Labeled faces in the wild: A database for studying face recognition in unconstrained environments”, Technical Report 07-49, University of Massachusetts, Amherst, October 2007, the Youtube® Faces Database as described in L. Wolf, T. Hassner, and I. Maoz, "Face recognition in unconstrained videos with matched background similarity”, in IEEE Conf, on CVPR, 2011, or Google® Facial Expression Comparison dataset. The training of the convolutional neural network may be performed as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832.

The processor may be configured to correct image artifacts caused by diffraction of the light when passing the transparent display. The term "correct” may mean partially or fully remove the artifacts or tag them so they can be excluded from further processing, in particular from determine if the imaged person is an authorized person. Correcting image artifacts may take into account the information about the transparent display, in particular the dimensions of the pixels or the distance of repeating features to each other. This information can facilitate identifying artifacts as diffraction patterns can be calculated and compared to the image. Correcting image artifacts may comprise identifying reflection features, sorting them by brightness and selecting the locally brightest features. For determining a distance around a feature in the image which qualifies as local, the information of the transparent display may be used, in particular a distance in the image by which a light beam may be displaced by diffraction on the transparent display may be calculated based on the information about the transparent display. This method can be particularly useful for pattern images. Further details are disclosed in WO 2021/105265 A1.

The processor may be configured to determine the quality of the image from the camera. Determination of the quality of the image can mean determining the brightness of the image, in particular determining if the brightness of the image is within a predetermined range. This predetermined range may be selected such that image recognition yields optimum results. The processor may generate a signal indicative of the brightness level of the image. Such signal may be use, for example by a controller of the projector, to adjust the illumination power of the projector. The signal may also be used by a controller of the camera to adjust the camera settings according to the signal indicative of the brightness level and/or trigger the camera to generate a new image. Determination of the quality of the image can also mean determining the head position of the person, in particular determining the angle of the face of the person relative to the camera. It may be determined if the angle of the face of the person relative to the camera is within a predetermined range. This predetermined range may be selected such that image recognition yields optimum results. The processor may generate a signal indicative of the head position of the person. Such signal may be used, for example by a controller of the camera to trigger the camera to generate a new image. The signal may also be used to inform the user to turn the head, for example by displaying such information on the transparent display.

The processor may be configured for outsourcing at least one step of the authentication process, such as the identification of the user, and/or at least one step of the validation of the authentication process, such as the consideration of the material data, to a remote device, specifically a server and/or a cloud server. The seat belt monitoring system and the remote device may be part of a computer network, particularly the internet. The seat belt monitoring system may transmit the generated data and/or data associated to an intermediate step of the authentication process and/or its validation to the remote device. In such a scenario, the processor may be and/or may comprise a connection interface configured for transmitting information to the remote device. Data generated by the remote device used in the authentication process and/or its validation may further be transmitted to the seat belt monitoring system. This data may be received by the connection interface comprised by the seat belt monitoring system. The connection interface may specifically be configured for transmitting or exchanging information. In particular, the connection interface may provide a data transfer connection, e.g. Bluetooth, NFC, or inductive coupling. As an example, the connection interface may be or may comprise at least one port comprising one or more of a network or internet port, a USB-port, and a disk drive.

The processor may be configured for using a facial recognition authentication process operating on the pattern image and/or extracted material data. The processor may be configured for extracting material data from the pattern image.

The authentication process may be validated based on the extracted material data. The validating based on the extracted material data may comprise determining if the extracted material data corresponds a desired material data. Determining if extracted material data matches the desired material data may be referred to as validating. Allowing or declining the user and/or object to perform at least one operation on the device that requires authentication based on the material data may comprise validating the authentication or authentication process. Validating may be based on material data and/or image. Determining if the extracted material data corresponds a desired material data may comprise determining a similarity of the extracted material data and the desired material data. Determining a similarity of the extracted material data and the desired material data may comprise comparing the extracted material data with the desired material data. Desired material data may refer to predetermined material data. In an example, desired material data may be skin. It may be determined if material data may correspond to the desired material data. In the example, material data may be non-skin material or silicon. Determining if material data corresponds to a desired material data may comprise comparing material data with desired material data. A comparison of material data with desired material data may result in a allowing and/or declining the user and/or object to perform at least one operation that requires authentication. In the example, skin as desired material data may be compared with non-skin material or silicon as material data and the result may be declination since silicon or non-skin material may be different from skin.

The authentication process or its validation may include generating at least one feature vector from the material data and matching the material feature vector with associate reference template vector for material. The authentication unit may be configured for authenticating the user in case the user can be identified and/or if the material data matches the desired material data. The device may comprise at least one authorization unit configured for allowing the user to perform at least one operation on the device, e.g. unlocking the device, in case of successful authentication of the user or declining the user to perform at least one operation on the device in case of non-suc- cessful authentication. Thereby, the user may become aware of the result of the authentication.

The processor may be configured to use the identification of the object for the determination of the condition measure. Reference data for a particular object may be stored in memory which may be used to determine the condition measure when this particular person has been identified. For example, the average blood perfusion rate or the average breath rate for a particular object may be stored which may be used to determine a medical condition or the stress level of the particular object when this object has been identified.

The processor may be configured to use the identification of the object for the determination of the control signal. Object-specific personal data or preference settings may be stored in memory which may be used to generate a control signal when this particular object has been identified. The preference settings may have been manually entered or they may have been gathered from previous events, for example feedback from the object or an analysis of various sensors indicating the reaction of the object to the action triggered by the control signal. For example, a person may suffer from a disease which may have an impact on the object's ability to drive a vehicle, for example epilepsy. If this object is identified and the condition measure indicates an epileptic fit, the control signal may be directed to trigger the vehicle automation system to bring the vehicle into a save mode, for example by driving on a break-down lane and stopping the vehicle.

The system for determining a surface property comprises an output to output the control signal. The control signal may be output to a storage device, for example a hard disc, a memory device, such as RAM or a flash memory. The control signal may be output to a computer system, for example the board computer of the vehicle or the controller of the functionality the control signal is designed to address.

The system for determining a surface property is suitable for various purposes. The system for determining a surface property may be part of an authentication system which may be integrated into vehicles including cars, motorcycles, buses, trucks, trains or even airplanes. It may be attached to a vehicle, or it may be integrated as component or as part of a component of a vehicle, for example as part of a display in the dashboard, an entertainment control system, or load speakers. It can be places at various places, for example it may be integrated into the steering wheel, besides a speed gauge, in the center of a dashboard, the A pillar, a side door, in a mirror or in a window.

The system for determining a surface property may be part of an authentication system which may be integrated into consumer electronic products, for example smartphones, laptops, tablets, smartwatches, fitness trackers, televisions, home theater systems, digital cameras, headphones, gaming consoles, virtual reality headsets, portable speakers, e- readers, drones, smart home devices such as smart bulbs, smart thermostats, and smart locks, portable chargers, digital voice assistants, streaming devices, wearable fitness devices, portable DVD players, GPS devices, or digital photo frames.

The present invention further relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to the present invention. The term "computer-readable data medium" may refer to any suitable data storage device or computer readable memory on which is stored one or more sets of instructions (for example software) embodying any one or more of the methodologies or functions described herein. The instructions may also reside, completely or at least partially, within the main memory and/or within the processor during execution thereof by the computer, main memory, and processing device, which may constitute computer-readable storage media. The instructions may further be transmitted or received over a network via a network interface device. Computer-readable data medium include hard drives, for example on a server, USB storage device, CD, DVD or Blue-ray discs. The computer program may contain all functionalities and data required for execution of the method according to the present invention or it may provide interfaces to have parts of the method processed on remote systems, for example on a cloud system.

Brief Description of the Figures

Figure 1 illustrates the hardware elements of the system for determining a surface property.

Figure 2 illustrates an example for pixel logic in an event camera.

Figure 3 illustrates an example of an embodiment for the method for determining a surface property of an object.

Figure 4 illustrates an example for a face authentication method.

Figure 5 illustrates another example for a face authentication method.

Description of Embodiments

Figure 1 illustrates the hardware elements of the system for determining a surface property. The system for determining a surface property 100 a projector 101, an event camera 102 and a processor 103. These components may be mounted on a printed circuit board providing the communication lines and electricity from a battery or an interface to an electricity supply. The projector 101 may project light 120 to a person 110. The projector 101 may comprise a VCSEL array and optics. The light may be patterned light, for example a periodic dot pattern such as a hexagonal dot pattern. The light may have a wavelength of 940 nm. The event camera 102 may be a temporal contrast camera, for example an event camera which is sensitive at or around 940 nm. The light 120 may impinge on object 110. Light 130 may be reflected to the camera 102 which generates an image in the optical range matching the wavelength emitted by projector 102, for example in the infrared range. The image data may comprise a grayscale image, i.e. each pixel contains only the total intensity information, or an RGB image, i.e. different pixels indicate the intensity in a particular wavelength.

The image may be passed to processor 103. The processor 103 may be a microcontroller, i.e. containing memory and IO controller functionalities or it may be a CPU which is connected to memory and IO controllers. The processor 103 may determine a material or a condition measure, for example the pulse frequency or the blood pressure of the person 110. The processor 103 may be configured to authenticate the person 110 using the surface property, for example by comparing the surface property to a reference obtained from local or remote memory. The system 100 may be communicatively coupled to or integrated into an electronic product 140, for example a smartphone or the board computer system of a car. The electronic product 140 may comprise an unlock mechanism using the surface property provided by the system 100.

Figure 2 illustrates an example for pixel logic in an event camera. In figure 2a, an example for a circuit diagram for a pixel of an event camera is illustrated. A photodiode 201 may generate an electric current which may be stored in a capacitor 202 leading to an increase in voltage dependent on the illumination intensity. A voltage comparator 203 may compare the voltage of the capacitor 202 with a reference voltage, for example the voltage of the last event. If the difference exceeds a threshold, the voltage comparator 203 may output a signal indicating a new event. An ana- log-to-digital converter 204 may convert such signal into pixel data. The pixel data may be combined with a timestamp to generate image data for that pixel. The voltage comparator 203 may store the current voltage of the capacitor 202 as new reference voltage.

Figure 2b illustrates an example for a corresponding voltage diagram. The voltage 210 is drawn against the time 220. The curve 211 demonstrates the voltage of the capacitor 202. Every time this voltage changes by a threshold 212, the voltage comparator 203 may trigger an event, so the corresponding timestamps 221, 222, 223, and 224 can be saved. A timestamp combined with an identifier of the pixel having caused the event may form a data packet. Image data may comprise all data packets within a certain time period. These packets may be converted into a gray scale image, for example by counting the number of events for each pixel and assigning it a gray scale value dependent on the number of events.

Figure 3 illustrates an example of an embodiment for the method for determining a surface property of an object. The object, for example the face of a human, may be illuminated 301 with coherent light, for example with near infrared light, for example with a wavelength of 940 nm. The illuminated light may be patterned, for example a hexagonal dot pattern. Image data may be recorded with an event camera 302. The events recorded by the event camera may be combined to generate images, for example by adding for each pixel all events within a time frame of 1 ms. The image data may be used to determine a surface property of the object 303. The surface property may be the material the object is made of. The determination may be achieved by a material classification model which receives the image data and outputs a material classifier, for example the probability if the object is out of skin. The material classification model may be a convolutional neural network (CNN) which is trained with labelled images. The surface property may be output 304, for example to a memory or to a second process on the processor. For example, the second process may be a program to authenticate the object 305 by using the surface property.

Figure 4 illustrates an example for a face authentication method. A face may be illuminated with patterned coherent light 401, for example with a hexagonal dot pattern. Such light may be generated with a projector comprising VCSEL arrays emitting light with about 940 nm. While the face is illuminated with patterned light, image data may be recorded with an event camera in event mode 402, i.e. the image data comprises events when the recorded intensity registered by a pixel changes by more than a threshold value. The image data may be processed to yield a pattern image 403, for example by integrating the events within a time frame, for example during 1 ms. The pattern image 403 may be used to classify which material the recorded face is composed of 404. For example, a classifier model such as a trained CNN may output a probability if the face is made of skin.

The face may then be illuminated with flood light 411, for example by switching the projector from patterned mode to flood mode or by switching off a pattern projector and switching on a flood projector. While the face is under flood illumination, the event camera may be switched to global shutter mode, i.e. the intensity values of all pixels are read out at a certain point in time. The event camera in global shutter mode may record an image 412, namely a flood image 413. The flood image 413 may be analyzed to identify the person 414. This may be achieved by converting the flood image 413 into a feature vector, for example with a trained CNN, and determine the similarity of the feature vector with feature vectors of a database. Alternatively, the feature vector may be compared with a reference feature vector, for example the reference saved for the person who claims to be the person of the face under illumination.

The identified person 414 and the classified material 404 may be compared to a reference 420. Such comparison may involve determining if the similarity of the feature vector from the flood image 413 to the reference feature vector is sufficiently high, for example exceeds a preset value, and at the same time if the probability of the material classification exceeds a preset threshold. If both is the case, the face may be authenticated 422, for example by outputting a corresponding signal which may be used to unlock a device or confirm a payment. If this is not the case, the face may be rejected 423.

Figure 5 illustrates another example for an authentication method. The object, for example a person, may be illuminated as described for figure 4 to yield an image of the person under pattern illumination 501 . The image 501 may be used to determine for some or all of the reflection patterns a speckle contrast. The resulting values may be associated with the position of the pattern feature in the image 501 to yield a speckle contrast map 502. The map 502 may be compared with a reference speckle contrast map 503, for example stored in memory from an enrollment process. If there is sufficient similarity between map 502 and map 503, the object may be authenticated 505. Otherwise, the object my be rejected 506.

The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the claims.

Any steps presented herein can be performed in any order. The methods disclosed herein are not limited to a specific order of these steps. It is also not required that the different steps are per-formed at a certain place or in a certain computing node of a distributed system, i.e. each of the steps may be performed at different computing nodes using different equipment/data processing.

As used herein ..determining" also includes ..initiating or causing to determine", "generating" also includes ..initiating and/or causing to generate" and "providing” also includes "initiating or causing to determine, generate, select, send and/or receive”. "Initiating or causing to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

In the claims as well as in the description the word "comprising” does not exclude other elements or steps and the indefinite article "a” or "an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation. In the claims as well as in the description the word "comprising” or "including” or similar wording does not exclude other elements or steps and shall not be construed limiting to the elements or steps lined out. The indefinite article "a” or "an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation or further elements may be included.

Providing in the scope of this disclosure may include any interface configured to provide data. This may include an application programming interface, a human-machine interface such as a display and/or a software module interface. Providing may include communication of data or sub-mission of data to the interface, in particular display to a user or use of the data by the receiving node, entity or interface.

Various units, circuits, entities, nodes or other computing components may be described as "con-figured to” perform a task or tasks. Configured to shall recite structure meaning "having circuitry that” performs the task or tasks on operation. The units, circuits, entities, nodes or other computing components can be configured to perform the task even when the unit/circuit/component is not operating. The units, circuits, entities, nodes or other computing components that form the structure corresponding to "configured to” may include hardware circuits and/or memory storing program instructions executable to implement the operation. The units, circuits, entities, nodes or other computing components may be described as performing a task or tasks, for convenience in the description. Such descriptions shall be interpreted as including the phrase "configured to.” Any recitation of "configured to” is expressly intended not to invoke 35 U.S.C. § 112(f) interpretation.

In general, the methods, apparatuses, systems, computer elements, nodes or other computing components described herein may include memory, software components and hardware components. The memory can include volatile memory such as static or dynamic random-access memory and/or nonvolatile memory such as optical or magnetic disk storage, flash memory, programmable read-only memories, etc. The hardware components may include any combination of combinatorial logic circuitry, clocked storage devices such as flops, registers, latches, etc., finite state machines, memory such as static random-access memory or embedded dynamic random-access memory, custom designed circuitry, programmable logic arrays, etc.

Any disclosure and embodiments described herein relate to the methods, the systems, apparatuses, devices, chemicals, materials, computer program elements lined out above and vice versa. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples and vice versa. All terms and definitions used herein are understood broadly and have their general meaning.

Claims

Claims:

1 . A method for determining a surface property of an object comprising: a) illuminating a surface of the object with coherent light, b) recording image data of the surface under illumination with an event camera, c) determining a surface property using the image data, and d) outputting the surface property.

2. The method according to claim 1, wherein the surface property is the material or the surface texture.

3. The method according to claim 1 , wherein the surface property is a local movement within the surface.

4. The method according to claim 3, wherein the object is a person and wherein the surface property is the blood perfusion in the skin of the person.

5. The method according to any of the claims 1 to 4, wherein the coherent light has a wavelength of 800 nm to 1000 nm.

6. The method according to any of the claims 1 to 5, wherein the light comprises a hexagonal pattern.

7. The method according to any of the claims 1 to 6, wherein the image data comprises data packets, wherein a data packet represents a pixel event, and wherein images are determined by combining data packets within a time frame of 0.5 to 3 ms.

8. Use of the surface property of any one of the preceding claims for authenticating an object.

9. Use according to claim 8, wherein a map comprising a plurality of surface properties associated with the location on the object is determined for different parts of the object and wherein the map is used for authenticating the object.

10. A system for determining a surface property of an object comprising: a) a projector configured to illuminate a surface of the object with coherent light, b) an event camera configured to record image data of the surface under illumination, c) a processor configured to determine a surface property using the image data, and d) an output configured to output the surface property.

11. The system according to claim 10, wherein the projector is configured to illuminate patterned light and flood light.

12. The system according to claim 10, wherein the event camera is configured to record image data in event mode when the object is illuminated with patterned light and in global shutter mode when the object is illuminated with flood light.

13. The system according to any of the claims 9 to 12, wherein the projector and the event camera are placed behind a transparent display.

14. The system according to any of the claims 9 to 13, wherein the system is configured to be integrated into a consumer electronic product.

15. A non-transient computer-readable medium including instructions that, when executed by one or more proces- sors, cause the one or more processors to perform a method for determining a surface property of an object comprising: a) illuminating a surface of the object with coherent light, b) recording image data of the surface under illumination with an event camera, c) determining a surface property using the image data, and d) outputting the surface property.