EP4740123A1 - Secure face authentication - Google Patents

Abstract

A method for authenticating a user of a device comprising an illumination source, the method comprising: a) receiving a visible light image showing the object while the object is illuminated with visible light, b) receiving an IR pattern image showing the object while the object is illuminated with an infrared light pattern emitted from the illumination source, c) determining if a material indicator associated with the object shown in the IR pattern image corresponds to a living organism, in particular by providing the IR pattern image to an IR data-driven model parametrized based on an IR training data set comprising at least one training IR pattern image and at least one training material indicator, wherein the IR data-driven model provides a material indicator based on the IR pattern image, d) determining if the object shown in the visible light image corresponds to an authorized user, and, e1) allowing the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to an authorized user and determining that the material indicator corresponds to a living subject, or e2) declining the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to an unauthorized object and/or the material indicator corresponding to a non-living subject.

Description

SECURE FACE AUTHENTICATION

TECHNICAL FIELD

The invention relates a device and/or a system for authenticating a user, a method for authenticating an user, a use of a device, a computer program element, a non-transitory computer-readable data medium

TECHNICAL BACKGROUND

An object can be authenticated based on an IR image. Such an IR image is generated while the object is illuminated with flood IR illumination. This IR image is then provided to recognition models trained for detecting and recognizing an authorized user. These detection and recognition models are trained with large datasets comprising a plurality of visible light images. Hence, the performance ef face authentication algorithms operating on IR images is decreased.

SUMMARY

In an aspect, the disclosure relates to a method for authenticating a user of a device comprising an illumination source, the method comprising: a) receiving a visible light image showing the object while the object is illuminated with visible light, b) receiving an IR pattern image showing the object while the object is illuminated with an infrared light pattern emitted from the illumination source, c) determining if a material indicator associated with the object shown in the IR pattern image corresponds to a living organism, in particular by providing the IR pattern image to a IR data-driven model parametrized based on an IR training data set comprising at least one training IR pattern image and at least one training material indicator, wherein the IR data-driven model provides a material indicator based on the IR pattern image, d) determining if the object shown in the visible light image corresponds to an authorized user, and, e1) allowing the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to an authorized user and determining that the material indicator corresponds to a living organism, or e2) optionally, declining the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to an unauthorized object and/or the material indicator corresponding to a non-living organism.

In another aspect, the disclosure relates to a method for authenticating a user of a device comprising an illumination source, the method comprising: a) receiving a visible light image showing the object while the object is illuminated with visible light, b) receiving an IR pattern image showing the object while the object is illuminated with an infrared light pattern emitted from the illumination source, c) determining if a material indicator associated with the object shown in the IR pattern image corresponds to a living organism, in particular by providing the IR pattern image to an IR data-driven model parametrized based on an IR training data set comprising at least one training IR pattern image and at least one training material indicator, wherein the IR data-driven model provides a material indicator based on the IR pattern image, d) determining if the object shown in the visible light image corresponds to an authorized user, and, e) allowing the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to an authorized user and determining that the material indicator corresponds to a living organism, otherwise declining the object to access at least one function of the device.

In an aspect, the disclosure relates to a method for authenticating a user of a device comprising an illumination source, the method comprising: a) receiving a visible light image showing the object while the object is illuminated with visible light, b) receiving an IR pattern image showing the object while the object is illuminated with an infrared light pattern emitted from the illumination source, c) determining if a material indicator associated with the object shown in the IR pattern image corresponds to a living organism, in particular by providing the IR pattern image to an IR data-driven model parametrized based on an IR training data set comprising at least one training IR pattern image and at least one training material indicator, wherein the IR data-driven model provides a material indicator based on the IR pattern image, d) determining if the object shown in the visible light image corresponds to an authorized user, and, e1) allowing the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to the authorized user and determining that the material indicator corresponds to the living organism, or e2) optionally, declining the object to access at least one function of the device based on determining that the object shown in the visible light image does not correspond to the authorized object and/or the material indicator does not correspond to the living organism.

In another aspect, it relates to a device for authenticating a user, the device comprising: a) an interface configured for receiving a visible light image showing the object while the object is illuminated with visible light and receiving an IR pattern image showing the object while the object is illuminated with an infrared light pattern emitted from the illumination source, a processor configured for determining if a material indicator associated with the object corresponds to a living organism, in particular by providing the IR pattern image to an IR data-driven model parametrized to output a material indicator in response to being provided with the IR pattern image based on an IR training data set comprising at least one training IR pattern image and at least one material indicator, and, determining if the object shown in the visible light image corresponds to an authorized object, and, allowing the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to an authorized user and determining that the material indicator corresponds to a living organism, optionally, declining the object to access at least one function of the device based on determining that the object shown in the visible light image does not correspond to the authorized object and/or the material indicator does not correspond to the living organism.

In another aspect, it relates to a device for authenticating a user, the device comprising: a) an interface configured for receiving a visible light image showing the object while the object is illuminated with visible light and receiving an IR pattern image showing the object while the object is illuminated with an infrared light pattern emitted from the illumination source, b) a processor configured for determining if a material indicator associated with the object corresponds to a living organism, in particular by providing the IR pattern image to an IR data-driven model parametrized to output a material indicator in response to being provided with the IR pattern image based on an IR training data set comprising at least one training IR pattern image and at least one material indicator, and, determining if the object shown in the visible light image corresponds to an authorized object, and, allowing the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to an authorized user and determining that the material indicator corresponds to a living organism, optionally otherwise declining the object to access at least one function of the device.

In another aspect, it relates to a use of a device as described herein for authenticating a user.

In another aspect, it relates to a computer program element with instructions, which when executed on a processing device is configured to carry out the steps of the method as described herein.

According to another aspect, a non-transitory computer-readable data medium storing a computer program including instructions for executing steps of the method according to the first aspect or an embodiment of the first aspect is provided.

Any disclosure, embodiments and examples described herein relate to the methods, the systems, apparatuses, chemical products and computer elements lined out above and below. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples. EMBODIMENTS

There is a need to allow an improved authentication process which requires low-cost hardware while providing a high security.

An object of the present disclosure is to provide a method, a non-transitory computer-readable data medium, a computer program element, a device and/or a system and a use of a device for improving the accuracy of recognition and the security of authentication processes.

The method, the computer program element, the non-transitory computer-readable data medium, the device and/or the system and the use of the device of the present disclosure allows for a secure authentication of an authorized user by determining whether the object presented is a spoofing object or a real user based on the liveness of the object. Further, the invention provides an improved recognition of an authorized user by using visible light images for recognition. Usually, recognition models used for 2D authentication are trained and optimized for visible light images. Followingly, recognition based on visible light images corresponds better to the training of the models used for recognition than IR image-based recognition. Consequently, the error rate of recognition based on visible light images is lower than for IR image-based recognition. Thus, by using visible light images for recognition the accuracy of the recognition process is increased and the security of access providing processes based on such recognition process is improved. Furthermore, the generation of visible light image involves low-cost standard hardware. Users usually desire visible light images in their devices and thus, cameras suitable for generating visible light images are integrated in such devices. Therefore, generating visible light images includes components already present in a large number of devices. This comes with the benefit of saving space in devices and saving resources for hardware to be integrated.

In the following, embodiments of the present disclosure will be outlined by ways of examples. It is to be understood that the present disclosure is not limited to said embodiments and/or examples.

In an embodiment, an illumination source may comprise at least one light source. The illumination source may comprise a plurality of light sources. The illumination source may be suitable for illuminating an object and/or emitting light. The illumination source may comprise for example at least one laser source and/or at least one incandescent lamp and/or at least one semiconductor light source. A semiconductor light source may be for example a lightemitting diode such as an organic and/or inorganic light-emitting diode and/or a laser diode. Laser source may comprise at least one laser diode. For example, the light source, in particular the laser source and/or the semiconductor light source may comprise one or more VCSELs and/or a VCSEL array comprising a plurality VCSELs. Hence, laser source may be for example a semiconductor light source. The illumination source may be suitable for emitting infrared (IR) light. Infrared light may be in the range of 750 nm to 1000 pm. Infrared light may comprise near infrared light, mid infrared light and far infrared light. Near infrared light may be in the range of 750 nm to values smaller than 3000 nm. Mid-infrared light may be in the range of 3 pm to values smaller than 15 pm. Far infrared light may be in the range of 15 pm to 1000 pm. The illumination source may be suitable for emitting an infrared light pattern. An infrared light pattern may comprise and/or may be associated with a plurality of infrared light beams, in particular at least two. Infrared light pattern may be infrared light. In particular, the infrared light pattern may be at least one of a near infrared light pattern, a mid-infrared light pattern and/or a far infrared light pattern. The plurality of infrared light beams may form a pattern when illuminating a surface. A pattern may refer to an arbitrary known or pre-determined arrangement comprising at least one arbitrarily shaped light spot. Light spot may refer to a contiguous area illuminated with light. The pattern may comprise at least one light spot. The light spot may be an arbitrary shaped symbol. In particular, the light spot may be or may refer to a light spot. The pattern may comprise an arrangement of periodic or non-periodic light spots. The pattern can be at least one of the following: at least one quasi random pattern; at least one Sobol pattern; at least one quasiperiodic pattern; at least one point pattern, in particular a pseudo-random point pattern; at least one line pattern; at least one stripe pattern; at least one checkerboard pattern; at least one triangular pattern; at least one rectangular pattern; at least one hexagonal pattern or a pattern comprising further convex tilings.

Further, the illumination source may comprise one or more optical elements. An optical element may refer to a device suitable for generating a multitude of light spots and/or the number of light beams emitted by one or more light sources. Additionally or alternatively, an optical element may refer to a device suitable for diverting the one or more light beams and/or light spots emitted by the light source. Optical element may comprise a mirror, a lens, a diffractive optical element, a refractive optical element, a metasurface element or a combination thereof. The metasurface element may refer to an optical element comprising a microstructure and being suitable for multiplicating the number of incident light beams and/or light spots, e.g. multiplicating the number of light beams emitted by the one or more light sources.

In an embodiment, processor may refer to an arbitrary logic circuitry configured to perform basic operations of a computer or system, and/or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor, or computer processor may be configured for processing basic instructions that drive the computer or system. It may be a semi-conductor based processor, a quantum processor, or any other type of processor configures for processing instructions. As an example, the processor may be or may comprise a Central Processing Unit ("CPU"). The processor may be a ("GPU”) graphics processing unit, (“TPU”) tensor processing unit, ("CISC") Complex Instruction Set Computing microprocessor, Reduced Instruction Set Computing ("RISC") microprocessor, Very Long Instruction Word ("VLIW") microprocessor, or a processor implementing other instruction sets or processors implementing a combination of instruction sets. The processing means may also be one or more special-purpose processing devices such as an Application-Specific Integrated Circuit ("ASIC"), a Field Programmable Gate Array ("FPGA"), a Complex Programmable Logic Device ("CPLD"), a Digital Signal Processor ("DSP"), a network processor, or the like. The methods, systems and devices described herein may be implemented as software in a DSP, in a micro-controller, or in any other side-processor or as hardware circuit within an ASIC, CPLD, or FPGA. It is to be understood that the term processor may also refer to one or more processing devices, such as a distributed system of processing devices located across multiple computer systems (e.g., cloud computing), and is not limited to a single device unless otherwise specified. The processor may also be an interface to a remote computer system such as a cloud service. The processor may include or may be a secure enclave processor (SEP). An SEP may be a secure circuit configured for processing the IR pattern image and/or the visible light image. A "secure circuit" is a circuit that protects an isolated, internal resource from being directly accessed by an external circuit. The processor may be an image signal processor (ISP) and may include circuitry suitable for processing images, in particular images with personal and/or confidential information.

In an embodiment data storage may refer to a memory. As an example, memory may be a physical system memory which may be volatile, non-volatile, or a combination thereof. The memory may include non-volatile mass storage such as physical storage media. The memory may be a computer-readable storage media such as RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, non-magnetic disk storage such as solid-state disk or any other physical and tangible storage medium which can be used to store desired program code means in the form of computer-executable instructions or data structures and which can be accessed by the computing system. Moreover, the memory may be a computer-readable media that carries computer- executable instructions (also called transmission media). Further, upon reaching various computing system components, program code means in the form of computer-executable instructions or data structures can be transferred automatically from transmission media to storage media (or vice versa). For example, computer-executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface module (e.g., a "NIC”), and then eventually transferred to computing system RAM and/or to less volatile storage media at a computing system. Thus, it should be understood that storage media can be included in computing components that also (or even primarily) utilize transmission media.

In an embodiment, an interface may be a shared boundary between at least two components of a processing unit. Interface may be a part of a processing unit. Interface may allow for exchange for information across the at least two components. Processing unit may comprise at least one processor. At least two components of a processing unit may correspond to a decentralized computing environment, a distributed computing environment, a centralized computing environment, a system including a plurality of devices such as a computer, laptop, smartphone, database or the like. Interface may be a network interface or a user interface. User interface may be an interface to a user, wherein the user may input information and/or the user interface may be used for providing information to the user. Network interface may be a virtual network interface.

In an embodiment, IR pattern image shows the object while the object is illuminated with an infrared light pattern emitted from the illumination source. The IR pattern image may show at least one pattern and/or at least two light spots and/or at least one pattern comprising at least two light spots. The IR pattern image may indicate a spatial intensity distribution of brightness associated with the infrared light pattern. The IR pattern image may show a projection of the infrared light pattern onto the object. The IR pattern image may show light reflected from the object under illumination of the infrared pattern light. The IR pattern image may show light reflected from the object being illuminated with the infrared pattern light. In an embodiment, visible light image shows the object while the object is illuminated with visible light. The visible light image may indicate a spatial intensity distribution of brightness associated with the visible light. The visible light image may show a projection of the visible light onto the object. The visible light image may show light reflected from the object under illumination of the visible light. The visible light image may show light reflected from the object being illuminated with the visible light. Visible light may refer to light in the range between 380 nm and 750 nm. The visible light image may correspond to an authorized user if the visible light image matches an image of the authorized user. The visible light image may correspond to an authorized user if the visible light image shows the authorized user. In an example, the visible light image may be a RGB image and/or a monochrome image. Additionally or alternatively, the image may be a 1 -bit image, 2-bit image, 4-bit image, 8-bit image, 16-bit image, 32-bit image or the like

In an embodiment, object may refer to a three-dimensional physical entity. Object may include living organisms such as humans and animals. It may be determined if the object shown in the visible light image corresponds to an authorized user. Object may be a user, preferably an authorized user. Authorized user may be a user with a right to access at least one function of the device. Unauthorized object may comprise an unauthorized user and/or an object. Unauthorized object may lack a right to access at least one function of the device. Object may refer to a living organism and/or a non-living organism. Living organism may refer to an authorized user and/or a non-authorized user. Non-living organism may refer to an authorized object and/or a non-authorized object. Authorized object may refer to a spoofing object.

In an embodiment, imaging unit may be suitable for generating an image. First imaging unit may be suitable for generating an IR pattern image. Second imaging unit may be suitable for generating a visible light image. Imaging unit may be a camera. Imaging unit may be a device having at least one imaging element configured for recording or generating an IR pattern image and/or a visible light image. The camera may be a digital camera. As an example, the camera may comprise at least one camera chip, such as at least one CCD chip and/or at least one CMOS chip configured for recording images. The camera may be or may comprise at least one infrared camera and/or an RGB camera. Furthermore, the camera, besides the at least one camera chip or imaging chip, may comprise further elements, such as one or more optical elements, e.g. one or more lenses. The first imaging unit may be different from the second imaging unit. The first imaging unit may be a camera different from another camera being the second imaging unit.

In an embodiment, device may be an electronic device. Device may be a mobile electronic device. Device may be a smart device, in particular a smart mobile electronic device. Device may be for example a smartphone, a smartwatch, a computer or the like. Device may be a display device. Device may comprise a display. Display may be suitable for displaying an indication that a user was allowed to access or was declined to access. Display may display an indication that a user was allowed to access or was declined to access.

In an embodiment, determining if the object shown in the visible light image corresponds to an authorized user may comprise authenticating the object. In particular, determining if the object shown in the visible light image corresponds to an authorized user may comprise determining if the object shown in the visible light image corresponds to an image of an authorized user. The visible light image may correspond to an authorized user if the visible light image matches an image of the authorized user. Authenticating the object may comprise performing an authentication process. Authentication process may include action performed for authenticating a user. Authentication process may include comparing the visible light image with information related to an enrolled user. Enrolled user may be an authorized user. User may be authorized to access. Information related to an enrolled user may comprise a template. Template may comprise template features. Template may be suitable for representing an authorized user. For example, authentication process may be a face authentication process. In the case of a face authentication process, the template may be a facial template and/or the template feature may be template facial feature and/or object feature may be a facial feature.

In an embodiment, object feature may be a lower-dimensional representation of the visible light image. Template feature may be a lower-dimensional representation of a template image. Template image may be a visible light image showing the authorized user. In particular, template image may be generated during an enrollment process of an authorized user.

In an embodiment, authentication process may comprise obtaining object feature from the visible light image, matching the object feature with the template feature and/or providing an authentication result based on the matching of the object feature with the template feature. Matching the object feature with the template feature may comprise generating a matching score. Object feature may be obtained by reducing the dimensionality of the visible light image.

In an embodiment, matching score may indicate the similarity between the visible light image and the template, in particular between the object feature and the template feature. Matching score may be determined by providing the object feature and the template feature to a model, in particular a visible light data-driven model. The visible light data-driven model may be parametrized and/or trained based on a visible light training data set comprising a plurality of visible light images, in particular a plurality of object features, and a plurality of templates, in particular a plurality of template features. The visible light training data set may further comprise an indication on the matching of the templates with the corresponding visible light images, in particular the matching of the template features with the corresponding object features. Matching of the templates with the corresponding visible light images, in particular the matching of the template features with the corresponding object features, may be indicated by a plurality of Lables, e.g. 0 referring to a non-matching and 1 referring to a matching.

In an embodiment, visible light data-driven model may receive the visible light image and may generate the object feature based on the visible light image. The visible light data-driven model may generate the object feature based on the visible light image by reducing the dimensionality of the visible light image. For example, an encoder architecture may be used for reducing the dimensionality of the visible light image. For example, an encoder and/or an encoder architecture may comprise a convolutional neural network, one or more dimensionality reduction layers, one or more pooling layers or the like. Object feature may comprise at least one object feature vector. Template feature may comprise at least one template feature vector. Template feature vector may be indicative of a feature associated with the template, in particular a template of an authorized user. Object feature vector and/or template feature vector may comprise a n-dimensional vector. Vector may comprise at least one numerical value, preferably n numerical values. Matching score may indicate the distance between the object feature vector and the template vector. In an example, a matching score may be obtained by determining the vector product of the feature vector and the template vector. Vector product may be a dot product.

In an example, function of a device may include displaying information, processing information, providing information and/or receiving information.

In an embodiment, determining if a material indicator associated with the object corresponds to a living object may comprise determining if a material indicator associated with the object corresponds to a living object by providing the IR pattern image to an IR data-driven model parametrized based on an IR training data set comprising at least one training IR pattern image and at least one training material indicator, wherein the IR data-driven model provides a material indicator based on the IR pattern image. The IR data-driven model may provide a material indicator based on the IR pattern image. Material indicator may be received from the IR data-driven model.

In an embodiment, data-driven model may be parametrized based on a training data set. The data-driven model may be trained based on a training data set. A training data set may comprise at least one input and at least one desired output. Data-driven model may represent a correlation between the desired output and the at least one input. The term training may also be denoted as learning. The term specifically may refer, without limitation, to a process of building the data-driven model, in particular determining and/or updating parameters of the data-driven model. Updating parameters of the data-driven model may also be referred to as retraining. Retraining may be included when referring to training herein. During the training the data-driven model may adjust to achieve best fit with the training data, e.g. relating the at least on input value with best fit of the at least one desired output value. For example, if the neural network is a feedforward neural network such as a CNN, a backpropagation-algorithm may be applied for training the neural network. In case of a RNN, a gradient descent algorithm may be employed for training purposes. Gradient descent algorithm may use gradient for updating parameters. Gradient may be obtained by backpropagation. Thus, gradient descent algorithm may be based on backpropagation. Training a data-driven model may include or may refer without limitation to calibrating the model. The data-driven model may be a classification model. The data-driven model may comprise at least one machine-learning architecture and model parameters. For example, the machine-learning architecture may be or may comprise one or more of: linear regression, logistic regression, random forest, piecewise linear, nonlinear classifiers, support vector machines, naive Bayes classifications, nearest neighbours, neural networks, convolutional neural networks, generative adversarial networks, support vector machines, or gradient boosting algorithms or the like. In the case of a neural network, the model can be a multi-scale neural network or a recurrent neural network (RNN) such as, but not limited to, a gated recurrent unit (GRU) recurrent neural network, vision transformer or a long short-term memory (LSTM) recurrent neural network. In an embodiment, the IR data-driven model may represent a correlation between the material indicator and the IR pattern image. The IR data-driven model may obtain the correlation between material indicator and the IR pattern image based on an IR training data set comprising a plurality of IR pattern images and a plurality of material indicators. The IR data-driven model may be parametrized based on an IR training data set. The IR data-driven model may be trained based on a IR training data set. The IR training data set may comprise a plurality of IR pattern images and a plurality of material indicators. Training the model may include parametrizing the model. The IR data-driven model may be parametrized and/or trained to provide a material indicator based on the IR pattern image, in particular receiving the IR pattern image. Providing the material indicator based on the IR pattern image may comprise determining the material indicator based on the IR pattern image. Providing the material indicator based on the IR pattern image may comprise mapping the IR pattern image to the material indicator. The IR data-driven model may be parametrized and/or trained to receive the IR pattern image. The IR data-driven model may receive the IR pattern image at an input layer or via a model loss function. A material indicator may be determined by providing the IR pattern image to an IR data- driven model parametrized based on an IR training data set comprising at least one training IR pattern image and at least one training material indicator, wherein the IR data-driven model provides a material indicator based on the IR pattern image.

In an embodiment, IR data-driven model may be an IR classification model. The IR classification model may be suitable for classifying the IR pattern image according to the material indicator. Material indicator can be for example an indication of the type of material or an indication whether the material associated with the object corresponds to skin.

In an embodiment, material indicator may indicate the material type, or the material class associated with object shown in the IR pattern image. Material indicator may refer to material indicator as a measure and/or to a value associated with the measure. For example, when referred to determining a material indicator it is to be understood that a value is determined. This value exemplarily corresponds to the measure material indicator and will be used for processing such as calculations if it is indicated that a material indicator value is used.

In an embodiment, the template, template feature and/or the template feature vector may be received and/or provided by at least one of a database or a data storage. Data storage may be external and/or internal data storage. External data storage may be connected wirelessly to a processor configured for determining if a material indicator associated with the object corresponds to a living object and/or determining if the object shown in the visible light image corresponds to an authorized object and/or allowing the object to access at least one function of the device or declining the object to access at least one function of the device.

The template, template feature and/or the template feature vector may be transmitted through a wired or wireless connection such as one of ethernet, Bluetooth, USB, LAN, WLAN and the like. By doing so, the template is received up to date and the authentication is readily available to new and/or changed authorized users. In an embodiment, the template feature and/or the template feature vector may be stored locally, eg on the device of the user. This is advantageous since the template does not need to be retrieved just before authenticating. Hence, the invention does not rely on a connection to a source that can provide a template.

In an embodiment, determining that the object shown in the visible light image corresponds to an authorized user may comprise determining that the visible light image corresponds to an image of an authorized user. The visible light image corresponding to an authorized user may be a result of the visible light image matching an image of the authorized user. The visible light image corresponding to an authorized user may be a result of the visible light showing the authorized user.

In an embodiment, determining that the object shown in the visible light image corresponds to an unauthorized object may comprise determining that the visible light image corresponds to an image of an object independent of an authorized user. In particular, determining that the object shown in the visible light image corresponds to an unauthorized user. The visible light image corresponding to an unauthorized object may be a result of the visible light image mismatching an image of the authorized user. The visible light image corresponding to an authorized user may be a result of the visible light image showing the authorized user. Identifying a spoofing attempt provides the advantage of increasing the security of authentication.

In an embodiment, the infrared light may be within a range between 380 nm and 780 nm and/or wherein the infrared light may be within a range between 750 nm and 1000 m, in particular in a range between 780 nm and 1000 nm. In particular, the infrared light may be in the range between 900 nm and 1000 nm. More preferably, the infrared light may be in the range between 930 nm and 950 nm. Infrared light is not visible by the human eye and hence, the user is not distracted by unexpected illumination. Furthermore, the invention can be used in the darkness without providing a danger to the human eye. Especially, a wavelength within the range between 900 nm and 1000 nm is advantageous since the spectrum of the sun is naturally of lower intensity between 900 nm and 1000 nm. This leads to a lower disturbance because of sunlight.

In an embodiment, determining if a material indicator associated with the object corresponds to a living object may comprise determining if at least a part of the object shown in the IR pattern image comprises at least partially of skin and wherein the material indicator corresponds to a living organism if the object comprises at least partially of skin or wherein the material indicator corresponds at least partially to a non-living organism if the object comprises material other than skin. Skin can be detected on the presented object if the object is a living organism. The living organism comprises skin. Determining that the object shown in the IR pattern image corresponds to a living organism may be referred to as detecting the presence of skin. Detecting the presence of skin in the IR pattern image may result in allowing the object to access at least one function of the device. Skin has unique characteristics when interacting with light, in particular IR light. The reason for this is the structure of skin with different layers of skin types. In an embodiment, the IR light may be coherent IR light. Coherent IR light interacts characteristically with the material of the object, in particular with skin. Skin is perfused by blood. This blood is constantly moving underneath the surface of the blood. Due to the wavelength of infrared light, the infrared light penetrates deeply into the skin and is scattered by the moving blood particles. Usually, coherent IR light produces speckles when illuminating an uneven or rough surface such as skin. These speckles become blurred when the infrared light is reflected by moving blood particles. Consequently, a blurring of the speckles formed by illuminating skin with coherent infrared light occurs. This blurring is very characteristic for human skin of a living body. Reproducing a moving fluid such as blood underneath a surface of an object constitutes a task with a very high effort. Hence, using coherent infrared light provides an extraordinary reliable test to the liveness of an object presented in an image.

In an embodiment, the method may further comprise and/or the processor of the system may be further configured for declining the object to access at least one function of the device based on determining that the object shown in the visible light image does not correspond to the authorized object and/or the material indicator does not correspond to the living organism.

In an embodiment, determining if at least a part of the object shown in the IR pattern image comprises at least partially of skin may include determining the material indicator and comparing the material indicator with a reference material indicator, and wherein the object shown in the IR pattern image comprises at least partially of skin if the determined material indicator is within a range specified by the reference material indicator or wherein the object comprises material other than skin if the determined material indicator is outside of a range specified by the reference indicator. Reference material indicator may comprise a threshold value. A threshold value may specify a range of values equal or below the threshold values. Alternatively, the threshold value may specify a range of values equal or above the threshold value. Reference material indicator may comprise and/or specify a range associated with at least two threshold values. Reference material indicator may be indicative of a range of material indicators being associated with skin. A reference material indicator may be widely applicable or may be selected depending on the use case. The reference material indicator may be selected based on the surrounding while generating the IR pattern image. In particular, the reference material indicator may be selected based on the exposure conditions while generating the IR pattern image. Hence, the reference material indicator provides an easy and robust way of providing a possibility for adjustment of criteria for determining the material of the object to be skin. The criteria for determining that the object comprises of skin or not may depend on the conditions of the surrounding. For example, under sunlight illumination the determination may be disturbed, and stricter criteria may be necessary. In such situations, adjusting the hurdle for determining that the object comprises at least partially of skin or not is advantageous.

In an embodiment, the method may further include generating the IR pattern image by a first camera while the object is illuminated with an infrared light pattern emitted from the illumination source and/or generating the visible light image generated by a second camera while the object is illuminated with visible light. The first camera may be different from the second camera. In particular, the first camera may be located differently from the second camera. In an example, the first camera may comprise an IR camera and/or the second camera may comprise a RGB camera. In an embodiment, the second camera is a RGB camera and/or a selfie camera of the device. In an example, the second camera may be a selfie camera of a mobile electronic device. The second camera may already be present in the device. For example, in a smartphone the second camera may be a selfie camera. When it comes to integration of hardware into mobile electronic devices, it has always to be considered whether there is space inside the housing of the mobile electronic device. A RGB camera or a selfie camera are essentials in a mobile electronic device such as a smartphone due to the user's wishes. Followingly, this hardware can be expected to be present in a smartphone. Thus, using such hardware is advantageous since already present hardware is used and more space within the mobile electronic device is available.

In an embodiment, the method may further include illuminating the object with an infrared light pattern emitted from the illumination source.

In an embodiment, the device may further include an illumination source configured for illuminating the object with an infrared light pattern.

In an embodiment, the device may comprise a first camera and/or a second camera. The first camera may be configured for generating the IR pattern image and/or the second camera may be configured for generating the visible light image.

In an embodiment, wherein the first camera and/or the second camera may be covered at least partially by a display, and wherein the device is a display device comprising the display. By doing so, the display area usable for displaying information and with this, the degree of usable device surface is increased.

In an embodiment, determining if a material indicator associated with the object corresponds to a living organism may precede determining if the object shown in the visible light image corresponds to an authorized user. Determining if a material indicator associated with the object corresponds to a living organism may be faster and/or may require less computational resources than determining if the object shown in the visible light image corresponds to an authorized user. Hence, time and resources that would be used for determining if the object shown in the visible light image corresponds to an authorized user can be saved if the object can be declined based on the materials assessment only.

In an embodiment, the first camera and/or the second camera may be connected to a trusted execution environment via a secure channel. The processor may be a trusted execution environment. The trusted execution environment may be configured for determining if a material indicator associated with the object corresponds to a living organism, in particular by providing the IR pattern image to an IR data-driven model parametrized to output a material indicator in response to being provided with the IR pattern image based on an IR training data set comprising at least one training IR pattern image and at least one material indicator, and/or, determining if the object shown in the visible light image corresponds to an authorized object, and/or, allowing the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to an authorized user and determining that the material indicator corresponds to a living organism, otherwise, declining the object to access at least one function of the device. Secure channel may refer to a channel to the trusted execution environment. A secure channel may provide an encrypted way of communication between two entities, in particular between the data providing entity, eg the first camera and/or the second camera, and the data receiving entity, eg the processor. The trusted execution environment may be designed as described in GlobalPlatform Technology, TEE System Architecture, Version 1.2, Public Release November 2018, document reference: GPD_SPE_009. The term "trusted execution environment” specifically may refer, without limitation, to an execution environment comprise at least one security capability and fulfills at least one security requirement. The trusted execution environment may be configured for protecting assets within the trusted execution environment from general software attacks. The trusted execution environment may be configured for defining rigid safeguards as to data and functions that a program can access. The trusted execution environment may be configured for resisting a set of defined threats. Multiple technologies are known for implementing a trusted execution environment and the level of security achieved may vary accordingly. Trusted execution environment may be for example a secure enclave processor. By doing so, tampering of image data can be circumvented and it can be ensured that only properly generated images are used for authentication.

In an embodiment, the method may further include darkening at least a part of the display covering the illumination source during illumination and/or the first camera during generating the IR pattern image and/or the second camera during generating the visible light image.

In an embodiment, the device may further comprise a display and the display may be configured for darkening at least a part of the display covering the illumination source during illumination and/or the first camera during generating the IR pattern image and/or the second camera during generating the visible light image.

Darkening at least a part of the display may refer to operating a plurality of pixels associated with an area, in particular an area associated with the first camera, the illumination source, the second camera or a combination thereof, to a pixel brightness smaller than 50% of the maximum pixel brightness, preferably smaller than 30 % of the maximum pixel brightness, more preferably smaller than 20 % of the maximum pixel brightness. A pixel is the smallest part of a display. Each pixel may be controlled separately. Pixels may comprise a monochrome scheme or a color scheme, e.g. RBG scheme. In the monochrome case, the pixel brightness may be lower than 50 % of the full pixel brightness. For displaying colors in the RGB scheme, the pixel may comprise a red, a green and a blue part. By controlling the value for each part, a specific color can be selected. In this case, each part may be controlled such that the brightness is regulated to be lower than 100% of the red, green or blue value for full brightness. In particular, the full brightness of a pixel may be 50 % or less. In some embodiments, the red, green or blue value may be 50 % or less of the red, green or blue value for full brightness. In some embodiments, the red, green and blue value may be in average 50 % or less of the full brightness. At least darkened area of the display may be an area where the pixels are turned off fully, a monochromatic area or a multicolor area. A monochromatic area may be an area where the pixels comprised in the area all share the same values for the brightness of the pixel, e.g. the same red, blue and green values or the same degree of full brightness of the pixels. A multicolor area may be an area where not all pixels comprised in the area share the same values for the brightness of the pixel, e.g. at least two pixels differing in the red, blue and/or green values or at least two pixels differing in the degree of full brightness of the pixels. Multicolor area may be an area where more than one color is displayed. Multicolor area may represent a pattern and/or a color gradient. A color gradient can be considered as a color change. The color change may be from a specific color to a brighter or darker color. During the change the pixel brightness may be changed to higher values of the full brightness of the pixel for changing to brighter colors or lower values of the full brightness of the pixel for changing to darker colors. In this way, any display artifacts caused by a projector, are less obvious or fully absent if the display displays at least darkened parts where the display area may cover the illumination source during illumination and/or an camera during image generation at least partially.

In an embodiment, the visible light may be emitted independently from the illumination source of the device. Additionally or alternatively, the visible light may be ambient light. Additionally or alternatively, the visible light may be emitted from a display of the device being a display device and comprising a display. Additionally or alternatively, the visible light is emitted from a flashlight of the device. Visible light may be emitted by the surrounding of the device. Visible light may be emitted from natural illumination source and/or artificial illumination source. By doing so, no further illumination source needs to be integrated in a device deploying secure authentication as presented herein. Less required hardware lowers the cost for manufacturing, saves time on integration, saves spaces within a device and saves resources such as energy and material. Thus, this contributes to a lower power consumption of a device which is especially important when the device is a battery-powered device.

In an embodiment, the display provides feedback associated with the visible light image and/or the IR pattern image in response to allowing or declining the object to access at least one function of the device. Feedback may be provided visually. For example, feedback may include text, photo and/or drawings. Providing feedback about the authentication process enables a better interaction with the user and decreases the chance for maloperation by the user. Consequently, the process efficiency is increased and ultimately, resources used for the authentication can be saved including energy, computational effort and time.

In an embodiment, determining if the object shown in the visible light image corresponds to an authorized user may be based on image recognition, in particular face recognition. Image recognition, in particular face recognition, may refer to providing the visible light image to the visible light data-driven model and/or receiving an indication that the object corresponds to the authorized user from the visible light image. Preferably, determining if the object shown in the visible light image corresponds to an authorized user may comprise determining that the object shown in the visible light image corresponds to an authorized user by providing the visible light image to the visible light data- driven model and/or by receiving an indication that the object corresponds to the authorized user from the visible light data-driven model. In an embodiment, template feature may be received, in particular for determining if the object corresponds to an authorized user. Template feature may be received from a database. Database may be wired or wirelessly connected to a processor configured for performing image recognition. Template feature may be obtained in an enrollment process associated with an authorized user. Enrollment process may comprise generating a template image. Preferably template image may be referred to as a visible light template image. Template image may be suitable for obtaining template feature. Template features may be obtained from the template image. Template feature may be a lower-dimensional representation of the template image. Template feature may be obtained based on a machine learning architecture such as an encoder.

In an embodiment, performing image recognition may comprise determining at least one object feature based on the visible light image and matching the at least one object feature with at least one template feature. By doing so, the amount of data used for object recognition is reduced since the template feature and the object feature are lowerdimensional representations of the visible light image and the template image. This saves resources such as energy and computational effort. Further, the requirements for hardware components used for authentication may be reduced and thus, production costs can be lowered.

In an embodiment, object feature may refer to a feature representing at least a part of the object, in particular at least a part of the human body. Part of a human body may be at least one of face, hand, finger, foot, leg, eye, iris, nose, mouth, eyebrow, ear, chin, forehead, arm, torso or the like. Preferably, template feature and/or object feature may be a facial feature. Facial feature may be a feature associated with an object representing a face. Example for facial feature may comprise at least one of the following: the nose, the eyes, the eyebrows, the mouth, the ears, the chin, the forehead, wrinkles, irregularities such as scars, cheeks including cheekbones or the like. Template feature and/or object feature may be a vector. Vector may comprise at least one numerical value.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following, the present disclosure is further described with reference to the enclosed figures. The same reference numbers in the drawings and this disclosure are intended to refer to the same or like elements, components, and/or parts.

Fig. 1 illustrates an example embodiment of a device for authenticating a user.

Fig. 2 illustrates an example embodiment of a method for authenticating a user.

Fig. 3 illustrates an example embodiment of a system for authenticating a user.

DETAILED DESCRIPTION

- start of figure description The following embodiments are mere examples for implementing the method, the system or application device disclosed herein and shall not be considered limiting.

Figure 1 illustrates an example embodiment of a device 100 for authenticating a user. The device 100 may comprise an interface and a processor configured for performing the method as described within the context of Fig. 2. The interface may be an interface to a processor. The device 100 may a mobile electronic device. The device 100 may comprise a housing 120. Device 100 may be suitable for displaying information and/or performing actions the user of the device may want to be accessible by user-selected humans. This may only involve the user of the device 100 or the user and humans the user trusts. The device 100 may be owned by the user. The device 100 may further comprise a first image generation 150 unit for generating an IR pattern image and/or a second image generation 160 unit for generating a visible light image. The device may comprise an illumination source 140 suitable for emitting an infrared light pattern. The display 130 of the device may cover at least partially the first camera 150 and/or the second camera 160 and/or the illumination source 140. The display 130 may be an OLED and/or a QLED display. The display 130 may cover at least 90 % of at least one surface of the device 100. The display 130 and/or the flashlight may emit visible light. Further, visible light may be emitted by the surrounding of the device 100 from natural illumination sources and/or artificial illumination sources. Natural illumination sources may be illumination sources occurring in nature independent of humanity. Artificial illumination sources may be illumination source generated by humans.

The interface may be a communication interface. The interface may connect data generating parts of the device such as an camera with the processor 170. Interface may be wired to at least one camera 150, 160 and/or to at least one processor 170. Interface may comprise or may be connected to a secure channel. Interface may receive the IR pattern image and the visible light image. Interface may transmit the visible light image and the IR pattern image to the processor 170. Processor 170 may be a secure execution environment such as a SEP. Processor 170 may not be tampered from outside. Processor 170 may determine if the object in the visible light image and/or the IR pattern image corresponds to a living authorized user. Based on determining that the object presented in at least one image corresponds to a living authorized user, a signal indicating that the object is allowed to access at least one function of the device may be provided, e.g. via an interface to a control unit of the device.

Figure 2 illustrates an example embodiment of a method for authenticating a user. The method may constitute an authentication process. The authentication process may start by receiving an IR pattern image and/or by receiving a visible light image. The visible light image and the IR pattern image may be generated independently from each other. For example, the visible light image may be generated with the selfie camera of a smartphone. The IR light pattern may be emitted from an illumination source. Hence, the first camera for generating the IR pattern image may be connected to the illumination source. Preferably, the first camera and the illumination source may be connected via a control unit. The generation and/or the receiving of the IR patten image and/or the visible light image may be initiated by an action of the user. For example, the user may request to unlock a device, to be granted access to a location and/or to information or to pay. Other embodiments are, however, feasible. In another example, generation and/or the receiving of the IR patten image and/or the visible light image may be initiated based on establishing a connection between two devices, e.g. by establishing a Bluetooth connection or NFC. The authentication process may comprise an image recognition and a liveness detection. Image recognition may determine if the object shown in the visible light image corresponds to an authorized user 240. Liveness detection may determine if a material indicator associated with the object corresponds to a living organism 230. Liveness detection may be performed before image recognition. This can save time and resources that would have been used for the image recognition although the object does not correspond to a living organism and can thus not be an authorized user but is a spoofing object. Based on determining that the object shown in the visible light image corresponds to an authorized user in 240 and determining that a material indicator associated with the object corresponds to a living organism in 230, the object may be allowed to access at least one function of the device 250a. This may include approving a user's payment, removing an access restriction to a location, providing information or unlocking a device. Based on determining that the object shown in the visible light image corresponds to an unauthorized user in 240 or determining that a material indicator associated with the object corresponds to a non-living organism in 230, the object may be declined to access at least one function of the device 250b. Declining or allowing the object may result in providing feedback to the object. The feedback may indicate successful authentication or unsuccessful authentication. In the case of unsuccessful authentication, the feedback may include suggestion for another authentication trial. These suggestions may include suggestions on the position of the object or offering an alternative for authentication independent from the current authentication process. The authentication process may be carried out by the device as described within the context of Fig. 1 and/or by the system as described within the context of Fig. 3.

Figure 3 illustrates an example embodiment of a system for authenticating a user. This system may be used for allowing or declining an object from accessing a public location and/or resource. For example, access to a company area, access into a mobility system such as car sharing or public transport, access to books in a library or the like may be requested by the user. A first camera 310 may generate an IR pattern image. Based on the IR pattern image it can be determined if the object shown in the IR pattern image corresponds to a living organism. For this purpose, the IR pattern image is transmitted via an interface 330 configured for receiving the IR pattern image to a processor 340. A second camera 320 may generate a visible light image. Based on the visible light image it can be determined if the object shown in the visible light image corresponds to an authorized user. For this purpose, the visible light image is transmitted via an interface 330 configured for receiving the visible light image to a processor 340. The processor 340 may perform mathematical operations to arrive at a result. The result may indicate whether the object presented in the images is the authorized user or whether the object is unauthorized. This result may be transmitted via the interface 330 to a control unit 360. The control unit 360 may be suitable for generating a signal controlling an access action. Based on the signal from the control unit 360, the authorized user may be allowed to access at least one function of a device. The device which the authorized user is allowed to access a function of may be a part of or may be connected to the system for authenticating a user. The system may comprise the device as described within the context of Fig. 1 and/or may be configured for performing the method as described within the context of Fig. 2.

The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the claims.

Any steps presented herein can be performed in any order. The methods disclosed herein are not limited to a specific order of these steps. It is also not required that the different steps are performed at a certain place or in a certain computing node of a distributed system, i.e. each of the steps may be performed at different computing nodes using different equipment/data processing.

As used herein ..determining" also includes ..initiating or causing to determine", "generating" also includes ..initiating and/or causing to generate" and "providing” also includes "initiating or causing to determine, generate, select, send and/or receive”. "Initiating or causing to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

In the claims as well as in the description the word "comprising” does not exclude other elements or steps and the indefinite article "a” or "an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

Any disclosure and embodiments described herein relate to the methods, the systems, devices, the computer program element lined out above and vice versa. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples and vice versa.

All terms and definitions used herein are understood broadly and have their general meaning.

Claims

Claims

1 . A method for authenticating a user of a device comprising an illumination source, the method comprising: a) receiving a visible light image showing the object while the object is illuminated with visible light, b) receiving an IR pattern image showing the object while the object is illuminated with an infrared light pattern emitted from the illumination source, c) determining if a material indicator associated with the object shown in the IR pattern image corresponds to a living organism, in particular by providing the IR pattern image to an IR data- driven model parametrized based on an IR training data set comprising at least one training IR pattern image and at least one training material indicator, wherein the IR data-driven model provides a material indicator based on the IR pattern image, d) determining if the object shown in the visible light image corresponds to an authorized user, and, e) allowing the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to an authorized user and determining that the material indicator corresponds to a living organism, otherwise declining the object to access at least one function of the device.

2. The method according to any one of the preceding claims, wherein determining if a material indicator associated with the object corresponds to a living object comprises determining if at least a part of the object shown in the IR pattern image comprises skin and wherein the material indicator corresponds to a living organism if the object comprises skin.

3. The method according to claim 3, wherein determining if at least a part of the object shown in the IR pattern image comprises skin includes determining the material indicator and comparing the material indicator with a reference material indicator, and wherein the object shown in the IR pattern image comprises skin if the determined material indicator is within a range specified by the reference material indicator.

4. The method according to any one of the preceding claims, further including generating the IR pattern image by a first camera and/or generating the visible light image generated by a second camera.

5. The method according to claim 4, wherein the first camera and/or the second camera are covered at least partially by a display, and wherein the device is a display device comprising the display.

6. The method according to claim 4 and 5, further including darkening at least a part of the display covering the illumination source during illumination and/or the first camera during generating the IR pattern image and/or the second camera during generating the visible light image.

7. The method according to claim 4 to 6, wherein the second camera is a RGB camera and/or a selfie camera of the device.

8. The method according to any one of the preceding claims, wherein the visible light is emitted independently from the illumination source of the device.

9. The method according to any one of the preceding claims, wherein the visible light is ambient light and/or, wherein the visible light is emitted from a display of the device being a display device and comprising a display and/or wherein the visible light is emitted from a flashlight of the device.

10. The method according to any one of the preceding claims, wherein determining if the object shown in the visible light image corresponds to an authorized user is based on image recognition.

11. The method according to claim 10, wherein performing image recognition comprises determining at least one object feature based on the visible light image and matching the at least one object feature with at least one template feature.

12. The method according to any one of the preceding claims, wherein the display provides feedback associated with the visible light image and/or the IR pattern image in response to allowing or declining the object to access at least one function of the device.

13. A device for authenticating a user, the device comprising: a) an interface configured for receiving a visible light image showing the object while the object is illuminated with visible light and receiving an IR pattern image showing the object while the object is illuminated with an infrared light pattern emitted from the illumination source, b) a processor configured for determining if a material indicator associated with the object corresponds to a living organism, in particular by providing the IR pattern image to an IR data- driven model parametrized to output a material indicator in response to being provided with the IR pattern image based on an IR training data set comprising at least one training IR pattern image and at least one material indicator, and, determining if the object shown in the visible light image corresponds to an authorized object, and, allowing the object to access at least one function of the device based on determining that the object shown in the visible light image corresponds to an authorized user and determining that the material indicator corresponds to a living organism, otherwise, declining the object to access at least one function of the device.

14. Use of a device according to claim 13 for authenticating a user.

15. A computer program element with instructions, which when executed on a processing device is configured to carry out the steps of the method of any one of claims 1 to 12.

5