EP4666267A1

EP4666267A1 - Authentication system and method for vehicles

Info

Publication number: EP4666267A1
Application number: EP24701964.9A
Authority: EP
Inventors: Christian Hess; Muhammad Muneeb HASSAN
Original assignee: TrinamiX GmbH
Current assignee: TrinamiX GmbH
Priority date: 2023-02-15
Filing date: 2024-01-26
Publication date: 2025-12-24
Also published as: KR20250149984A; WO2024170253A1; CN120677513A

Abstract

A method for accessing a vehicle and/or a function of the vehicle, the method comprising: - illuminating the object with patterned infrared illumination; - generating at least one pattern image of the object while the object is being illuminated by the patterned infrared illumination; - determining if the object corresponds to a living human based the at least one pattern image; and - allowing the object to access the vehicle and/or the function of the vehicle based on de- termining that the object corresponds to a living human.

Description

Authentication System and Method for Vehicles

Technical field

The invention relates to a non-transient computer-readable medium, a method for accessing a vehicle and/or a function of the vehicle, a method for performing in-car payment, a system for accessing a vehicle and/or a function of the vehicle, a system for performing in-car payment, a vehicle, use of a system for accessing a vehicle and/or a function of the vehicle, use of a system for performing in-car payment, computer-readable medium, a method for verifying the qualification of a driver of a vehicle, a system for verifying the qualification of a driver of a vehicle.

Technical Background

Vehicles should only be started or accessed by people that are authorized by the owner of the vehicle to drive. Commonly available vehicles can be opened and started with keys independent of the human using the keys. Examples for people who should not be able to control a vehicle are children or thieves. Hindering these people to control a vehicle requires a reliable and accurate authentication of the people that are authorized by the owner of the vehicle to drive.

Hence, it is desired to authenticate qualified people for entering, starting and controlling of vehicles.

Summary

Any disclosure, embodiments and examples described herein relate to the methods, the systems, apparatuses, chemical products and computer elements lined out above and below. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples.

In an aspect, the disclosure relates to a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of the preceding claims referring to a method.

In another aspect, it relates to a system for performing in-car payment, the system comprising: an input configured for receiving a request for payment, an illumination source configured for illuminating an object with patterned infrared illumination; an image generation unit configured to generate at least one pattern image of the object while the object is being illuminated by the patterned infrared illumination; a processor configured for determining if the object corresponds to a living human based on the at least one pattern image; and releasing the payment based on determining that the object corresponds to a living human.

In another aspect, it relates to a method for verifying the qualification of a driver of a vehicle, the method comprising: illuminating the driver with patterned infrared illumination while the driver is using a function of a vehicle; generating at least one pattern image of the driver while the driver is being illuminated by the patterned infrared illumination; determining if the driver is qualified for continuing the use of the function of the vehicle based the at least one pattern image; and allowing or declining the continuation of the use of the function of the vehicle based on determining that the driver is qualified for continuing the use of the function of the vehicle.

In another aspect, it relates to a system for verifying the qualification of a driver of a vehicle, the system comprising: an illumination source configured for illuminating the driver with patterned infrared illumination while the driver is using a function of a vehicle; an image generation unit configured for generating at least one pattern image of the driver while the object is being illuminated by the patterned infrared illumination; a processor configured for determining if the driver is qualified for continuing the use of the function of the vehicle based on the at least one pattern image; and allowing or declining the continuation of the use of the function of the vehicle based on determining that the object corresponds to a living human.

In another aspect, it relates to a computer-readable medium with instructions, which when executed on a processing device is configured to carry out the steps of the method as disclosed herein.

In another aspect, it relates to a method for accessing a vehicle and/or a function of the vehicle, the method comprising: triggering to illuminate an object with patterned infrared illumination; triggering to generate at least one pattern image of the object while the object is being illuminated by the patterned infrared illumination; determining if the object corresponds to a living human based the at least one pattern image; and allowing the object to access the vehicle and/or the function of the vehicle based on determining that the object corresponds to a living human.

In another aspect, it relates to a method for determining a condition of a driver, the method comprising: triggering to illuminate the driver with patterned infrared illumination; triggering to generate at least two pattern images of the object while the object is being illuminated by the patterned infrared illumination, wherein the at least two pattern images are generated at at least two different points in time, providing an indication of at least one interval between at least two different points in time determining a condition measure associated with a condition of the driver based on the at least two pattern images and the indication of the at least one interval. providing the condition measu re.

In another aspect, it relates to a use of the system as disclosed herein. In another aspect, it relates to a system for accessing a vehicle and/or a function of the vehicle, the system comprising: an input configured for receiving a request for payment, an illumination source configured for illuminating the object with patterned infrared illumination, an image generation unit configured for generating at least one pattern image of the object while the object is being illuminated by the patterned infrared illumination; a processor configured for determining if the object corresponds to a living human based the at least one pattern image; and allowing the object to access the vehicle and/or the function of the vehicle based on determining that the object corresponds to a living human.

In another aspect, it relates to a method for accessing a vehicle and/or a function of the vehicle, the method comprising: illuminating the object with patterned infrared illumination; generating at least one pattern image of the object while the object is being illuminated by the patterned infrared illumination; determining if the object corresponds to a living human based the at least one pattern image; and allowing the object to access the vehicle and/or the function of the vehicle based on determining that the object corresponds to a living human.

Embodiments

To allow only qualified people to enter, start or to control a vehicle, the qualified people are authenticated reliably and accurately. Usually, authentication can be spoofed by spoofing objects like a mask, an image or the like. Relying on keys for opening and starting a vehicle entails the chance for abusive use of the keys by e.g., a thief. Commonly available authentication strongly relies on a high-quality images of the person and requires the person to behave in a certain way such as positioning themselves in relation to a camera. Therefore, there is a need for a reliable, secure, and flexible authentication. The presented invention provides a fast and user-friendly way to allow only qualified persons to access, start and/or to control a vehicle. This is achieved by verifying the legitimacy of the user, ie determining if a real human or a spoofing object is presented. For this purpose, the object, ie real human or spoofing object, is illuminated by patterned light. The interaction of patterned light interacts with the object depends on the material of the object. Hence, a spoofing object made of silicon provides a different pattern image than a living human. Followingly, determining if the object corresponds to a living human based the at least one pattern image allows to distinguish between spoofing objects such as masks and real human. Since the invention does not only depend on a 2D image, it comes with the additional benefit of allowing the user to move more freely. This in turn increases the flexibility of the user while authenticating, decreases the number of authentication attempts and thus, enables a frictionless opening, starting and controlling of the vehicle. In the following, embodiments of the present disclosure will be outlined by ways of examples. It is to be understood that the present disclosure is not limited to said embodiments and/or examples.

In an embodiment, determining if the object corresponds to an authorized person based on the flood image may be referred to as authenticating the object based on the flood image. Authenticating the object based on the flood image may be validated based on the extracted material data.

In an embodiment, person, in particular authorized person and/or user, may refer to an enrolled person and/or a registered person. Enrolled person may be a person having undergone an enrolment process. Enrollment process may be a process for generating a template, in particular a template suitable for comparison with the recorded image of the person with the camera placed behind the transparent display.

In an embodiment, the illuminating and/or the recording may be initiated and/or triggered by a request for authentication, preferably a request for payment. In particular, the illuminating and/or the recording may be initiated and/or triggered by a payment terminal. Preferably, the illuminating and/or the recording may be initiated and/or triggered by at least one of the person, the device associated with the person, an application of the device associated with the person or a combination thereof. Device associate with the person may refer to a mobile electronic communication device such as a smartphone.

In an embodiment, an occupant may be illuminated by a patterned infrared illumination from an illumination source. The illumination source may comprise a projector. The projector may be suitable for emitting and/or projecting patterned infrared illumination. The projector may comprise a metasurface element and/or array of vertical cavity surface-emitting lasers (VCSEL). The metasurface element may be suitable for replicating a beam, in particular for replicating emitted patterned infrared illumination. The VCSEL may emit patterned infrared illumination. The projector may be suitable for emitting patterned infrared illumination and/or flood illumination. Flood illumination may be suitable for illuminating a continuous area. In particular, flood illumination may have substantially constant irradiance. Patterned infrared illumination may be suitable for illuminating at least two continuous areas. The continuous areas may be of arbitrary shape. The continuous areas may be a light spot of arbitrary size. Patterned infrared illumination may be suitable for projecting at least two light spots.

In an embodiment, at least one pattern image may be generated with an image generation unit. The image generation unit may be a camera. The term “camera” may refer to at least one unit of the optoelectronic apparatus configured for generating at least one image. The image may be generated via a hardware and/or a software interface, which may be considered as the camera. The camera may comprise at least one optical sensor, in particular at least one pixelated optical sensor. The camera may comprise at least one CMOS sensor and/or at least one CCD chip. For example, the camera may comprise at least one CMOS sensor, which may be sensitive in the infrared spectral range. The term “image” may refer to data recorded by using the optical sensor, such as a plurality of electronic readings from the CMOS or CCD chip. The image may comprise raw image data or may be a pre-processed image. For example, the pre-processing may comprise applying at least one filter to the raw image data and/or at least one background correction and/or at least one background subtraction. The camera may have a field of view between 10°x10° and 75°x75°, preferably 55°x65°. The camera may have a resolution below 2 MP, preferably between 0.3 MP and 1.5 MP. The camera may comprise further elements, such as one or more optical elements, e.g. one or more lenses. As an example, the optical sensor may be a fix-focus camera, having at least one lens which is fixedly adjusted with respect to the camera. Alternatively, however, the camera may also comprise one or more variable lenses which may be adjusted, automatically or manually. Other cameras, however, are feasible.

In an embodiment, a request for payment may be associated with a payment account, a to-be- purchased good or service, a vendor or service provider, an amount of money or the like. A service may be an entertainment service and/or navigating service. A request for payment may be triggered by the occupant, an app store associated with the vehicle, a toll station and/or a gas station. A request for payment may be triggered by establishing a connection between the vehicle and a device. This device may be an electronic device, in particular a mobile electronic device. The device may be suitable for connecting with at least a part of the vehicle. The device may be suitable for establishing a connection with at least a part of the vehicle by means of near-field communication, radio-frequency identification, Bluetooth, LAN, WLAN, ethernet or the like. The vehicle may be suitable for establishing a connection with a device by means of near- field communication, radio-frequency identification, Bluetooth, LAN, WLAN, ethernet or the like. The device may be integrated into the road, a pillar, a sign, a fuel dispenser or the like.

In an embodiment, a request for payment and/or the illuminating may be triggered by a driver, an occupant, an app store associated with the vehicle, a toll station and/or a gas station.

In an embodiment, an input may comprise of one or more of serial or parallel interfaces or ports, USB, Centronics Port, FireWire, HDMI, Ethernet, Bluetooth, RFID, Wi-Fi, USART, or SPI, or analogue interfaces or ports such as one or more of ADCs or DACs, or standardized interfaces or ports to further devices. In an embodiment, an interface may be a shared boundary between at least two components of a processing unit. The interface may be a part of a processing unit. The interface may allow for exchange for information across the at least two components. The processing unit may comprise at least one processor. At least two components of a processing unit may correspond to a decentralized computing environment, a distributed computing environment, a centralized computing environment, a system including a plurality of devices such as a computer, laptop, smartphone, database or the like. The interface may be a network interface or a user interface. The user interface may be an interface to a user, wherein the user may input information and/or the user interface may be used for providing information to the user. The network interface may be a virtual network interface.

In an embodiment, processor may refer to an arbitrary logic circuitry configured to perform basic operations of a computer or system, and/or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor, or computer processor may be configured for processing basic instructions that drive the computer or system. It may be a semi-conductor based processor, a quantum processor, or any other type of processor configures for processing instructions. As an example, the processor may be or may comprise a Central Processing Unit ("CPU"). The processor may be a (“GPU”) graphics processing unit, (“TPU”) tensor processing unit, ("CISC") Complex Instruction Set Computing microprocessor, Reduced Instruction Set Computing ("RISC") microprocessor, Very Long Instruction Word ("VLIW') microprocessor, or a processor implementing other instruction sets or processors implementing a combination of instruction sets. The processing means may also be one or more special-purpose processing devices such as an Application-Specific Integrated Circuit ("ASIC"), a Field Programmable Gate Array ("FPGA"), a Complex Programmable Logic Device ("CPLD"), a Digital Signal Processor ("DSP"), a network processor, or the like. The methods, systems and devices described herein may be implemented as software in a DSP, in a micro-controller, or in any other side-processor or as hardware circuit within an ASIC, CPLD, or FPGA. It is to be understood that the term processor may also refer to one or more processing devices, such as a distributed system of processing devices located across multiple computer systems (e.g., cloud computing), and is not limited to a single device unless otherwise specified. The processor may also be an interface to a remote computer system such as a cloud service. The processor may include or may be a secure enclave processor (SEP). An SEP may be a secure circuit configured for processing the spectra. A "secure circuit" is a circuit that protects an isolated, internal resource from being directly accessed by an external circuit. The processor may be an image signal processor (ISP) and may include circuitry suitable for processing images, in particular images with personal and/or confidential information.

In an embodiment, determining if the object corresponds to a living human may comprise extracting material data from the pattern image. Material data may be extracted from the pattern image. Material data may indicate the type of material. In particular, material data may indicate whether the object associated with the pattern image comprises at least partially of skin. Material data may be associated with the object, in particular with the object shown in the pattern image. Extracting material data from the pattern image may comprise generating the material type and/or data derived from the material type. Preferably, extracting material data may be based on the pattern image, more preferably the partial image. Material data may be extracted by using a model. Extracting material data may include providing the pattern image to a model and/or receiving material data from the model. Providing the pattern image to a model may comprise and may be followed by receiving the pattern image at an input layer of the model. Model may be a data-driven model.

In an embodiment, accessing a vehicle may comprise at least one of controlling to unlock the vehicle, opening the vehicle, in particular a door of the vehicle, removing a blocking element or the like.

In an embodiment, function of the vehicle may comprise at least one of starting of the vehicle, controlling a part of the vehicle such as light, speed, gear, display, heating, connection such as internet connection or the like.

In an embodiment, data-driven model may comprise a convolutional neural network and/or a least partially an encoder decoder structure such as an autoencoder. Other examples for generating a representation may be FFT, wavelets, deep learning, like CNNs, energy-based models, normalizing flows, GANs, vision transformers, or transformers used for natural language processing, autoregressive image modelling. GANs, Autoregressive Image Modeling, Normalizing Flows, Deep Autoencoders, Deep Energy-Based Models, Vision Transformers. Supervised or unsupervised schemes may be applicable to generate representation, also embedding in e.g. Euclidian space. In another embodiment, extracting material data may include providing the pattern image to a model and/or receiving material data from the model.

In another embodiment, data-driven model may be trained according to a training data set including at least one pattern image and material data. In another embodiment, the data-driven model may be parametrized according to a training data set including at least one pattern image and material data. The data-driven model may be parametrized according to a training data set including at least one pattern image and material data. The data-driven model may be parametrized according to a training data set to receive the pattern image and provide material data based on the received pattern image. The data-driven model may be trained according to a training data set to receive the pattern image and provide material data as output based on the received pattern image. The training data set may comprise at least one pattern image and material data, preferably material data associated with the at least one pattern image. The pattern image may comprise a representation of the pattern image. The representation may be a lower dimensional representation of the pattern image. The representation may comprise at least a part of the data or the information associated with the pattern image. Representations of a pattern image may comprise a feature vector. In an embodiment, determining a representation, in particular a lowerdimensional representation may be based on principal component analysis (PCA) mapping or radial basis function (RBF) mapping. Determining a representation may also be referred to as generating a representation. Generating a representation based on PCA mapping may include clustering based on features in the pattern image and/or partial image. Additionally, or alternatively, generating a representation may be based on neural network structures suitable for reducing dimensionality. Neural network structures suitable for reducing dimensionality may comprise an encoder. In an example, neural network structure may be an autoencoder. In an example, neural network structure may comprise a convolutional neural network (CNN). CNN may comprise at least one convolutional layer and/or at least one pooling layer. CNNs may reduce the dimensionality of a partial image and/or a pattern image by applying a convolution, e.g. based on a convolutional layer, and/or by pooling. Applying a convolution may be suitable for selecting a feature related to material information of a pattern image, in particular a partial image.

In an embodiment, extracting material data from the pattern image with a data-driven model may comprise providing the pattern image to a data-driven model. Additionally or alternatively, extracting material data from the pattern image with a data-driven model may comprise generating an embedding associated with the pattern image based on the data-driven model. An embedding may refer to a lower dimensional representation associated with the pattern image such as a feature vector. The feature vector may be suitable for suppressing the background while maintaining the material signature indicating the material data. In this context, background may refer to information independent of the material signature and/or the material data. Further, background may refer to information related to biometric features such as facial features. Material data may be determined with the data-driven model based on the embedding associated with the pattern image. Additionally or alternatively, extracting material data from the pattern image by providing the pattern image to a data-driven model may comprise transforming the pattern image into material data, in particular a material feature vector indicating the material data. Hence, material data may comprise further the material feature vector and/or material feature vector may be used for determining material data. In an embodiment, determining if the object corresponds to a living human based on the at least one pattern image may be referred to as validating. Releasing the payment may be based on validating. Validating may be based on determining that the object corresponds to a living human. The validation based on the extracted material data may comprise determining if the extracted material data corresponds a desired material data. Desired material data may refer to predetermined material data. Predetermined material data may be material data associated with the authorized user.

In an embodiment, the payment may be further released based on determining that the object corresponds to an authorized user. Determining that the object corresponds to an authorized user may comprise determining if the object corresponds to an authorized user. Determining if the object corresponds to an authorized person based on the flood image may comprise matching the flood image with a template. Matching the flood image with a template may comprise determining a matching score between the flood image and a template. Template may be generated during an enrolment process of the authorized user. Templated may be associated with the authorized user. Determining a matching score of the template and the flood image may comprise determining a similarity of the template and the flood image. The template may comprise and/or may be represented with a template feature. The template feature may be a template vector. The flood image may comprise and/or may be represented by an image feature. The image feature may be a feature vector. Determining a similarity between the template and the flood image may comprise determining a distance between the template vector and the feature vector. The feature vector may be a facial feature vector.

In an embodiment, authenticating the object may be a facial authentication. The feature vector may be a facial feature vector. Template may show, may include and/or may represent a face of an authorized user.

In an embodiment, the illumination and/or the generation may be triggered based on a timer, an event caused by the using of the function of the vehicle, a change of conditions related to driving the vehicle, a request from an occupant and/or a request from governmental institution. An event may be for example an accident. This is beneficial since this enables a frictionless or passive verification of the driver’s qualification to continue the use of the function of the vehicle.

In an embodiment, determining if the driver is qualified for continuing the use of the function of the vehicle based on the at least one image may include at least one of: extracting material data based on the pattern image, determining a blood perfusion measure based on a pattern image, determining a head pose based on a flood image and/or a pattern image, determining an occlusion based on a flood image and/or a pattern image, in particular an occlusion of a driver’s eye.

In an embodiment, the pattern image may be generated within the vehicle. The illumination source and/or the image generation unit may be located within the vehicle. Prior to generating the pattern image, the authorized user may have been allowed to access the vehicle while the authorized user was outside of the vehicle. Thus, a secure authentication of the object has been performed while the authorized user was outside of the vehicle. An event can occur after authenticating the authorized user outside of the vehicle. Thus, an additional authentication before starting the engine and controlling it increases the security and hinders thieves, children or other unauthorized users from controlling the vehicle.

In an embodiment, extracting material data may comprise determining a similarity between the extracted material data and the desired material data. Determining a similarity between the extracted material data and the desired material data may comprise comparing the extracted material data with desired material data. In an example, desired material data may be skin. It may be determined if material data may correspond to the desired material data. In the example, material data may be non-skin material or silicon. Determining if material data corresponds to a desired material data may comprise comparing material data with desired material data. A comparison of material data with desired material data may result in a allowing and/or declining the object to perform at least one operation that requires authentication. In the example, skin as desired material data may be compared with non-skin material or silicon as material data and the result may be declination since silicon or non-skin material may be different from skin.

In an embodiment, infrared light may comprise of near infrared light, midinfrared light and/or far- infrared light. Infrared light may be in the range of 750 nm to 1000 pm. Near infrared light may be in the range of 780 nm to 3000 nm excluding the value of 3000 nm. Mid infrared light may be in the range of 3 pm to 15 pm excluding the value of 15 pm. Far infrared light may be in the range of 15 pm to 1000 pm. The patterned infrared illumination may comprise coherent patterned infrared illumination. Visible illumination may be in the range of 380 nm to 750 nm excluding the value of 750 nm as being comprised in the range.

In an embodiment, the patterned infrared illumination may be coherent patterned infrared illumination and determining if the object corresponds to a living human may comprise determining a blood perfusion measure based on the pattern image. Determining a blood perfusion measure may comprise determining a speckle contrast of the pattern image and determining a blood perfusion measure based on the determined speckle contrast. A speckle contrast may represent a measure for a mean contrast of an intensity distribution within an area of a speckle pattern. In particular, a speckle contrast K over an area of the speckle pattern may be expressed as a ratio of standard deviation o to the mean speckle intensity <l>, i.e. ,

Speckle contrast may comprise a speckle contrast value. Speckle contrast values may be distributed between 0 and 1. The blood perfusion measure is determined based on the speckle contrast. Thus, the vital sign measure may depend on the determined speckle contrast. If the speckle contrast changes, the blood perfusion measure derived from the speckle contrast may change accordingly. A blood perfusion measure may be a single number or value that may represent a likelihood that the object is a living subject. Preferably, for determining the speckle contrast, the complete pattern image may be used. Alternatively, for determining the speckle contrast, a section of the pattern image may be used. The section of the pattern image, preferably, represents a smaller area of the pattern image than an area of the complete pattern image. The section of the pattern image may be obtained by cropping the pattern image. In an embodiment, a data-driven model may be used for determining a blood perfusion measure. Data-driven model be parametrized and/or trained based on a training data set. Training data set may comprise a pattern image and a blood perfusion measure. Data-driven model may be parametrized and/or trained based on the training data set to output a blood perfusion measure based on receiving a pattern image.

In an embodiment, the patterned infrared illumination may be emitted by an illumination source and the illumination source may be covered at least partially by a transparent display and/or wherein the at least one pattern image may be generated with an image generation unit and the image generation unit may be covered at least partially by a transparent display and/or wherein the flood image may be generated with an image generation unit and the image generation unit may be covered at least partially by a transparent display.

In an embodiment, a data-driven model may be used for determining if the object corresponds to a living human based the at least one pattern image and/or determining if the object corresponds to an authorized user based on the flood image. Data-driven model used for determining if the object corresponds to a living human based the at least one pattern image may be parametrized and/or trained based on a training data set. The training data set for determining if the object corresponds to a living human may comprise a pattern image and a corresponding label indicating if the object corresponds to a living human. Data-driven model may be parametrized and/or trained based on the training data set to output a label indicating if the object corresponds to a living human based on receiving a pattern image. Data-driven model used for determining if the object corresponds to an authorized user based on the flood image may be parametrized and/or trained based on a training data set. The training data set for determining if the object corresponds to an authorized user may comprise a flood image and a corresponding label indicating if the if the object corresponds to an authorized user.

In an embodiment, determining if the object corresponds to a living human based the at least one pattern image may comprise providing the pattern image to a material data-driven model, wherein the material data-driven model is parametrized and/or trained based on historical pattern images and corresponding indications if the object associated with the historical pattern images corresponds to a living human. The material data-driven model may be parametrized and/or trained to provide indications if objects associated with pattern images correspond to living humans. In an embodiment, determining if the object corresponds to an authorized user based on the flood image may comprise providing the flood image to an authentication data-driven model wherein the material data-driven model is parametrized and/or trained based on historical flood images and corresponding indications if the object associated with the historical flood images may correspond to an authorized user. The authentication data-driven model may be parametrized and/or trained to provide indications if objects associated with flood images correspond to authorized users.

In an embodiment, a flood image associated with the object may be received and/or generated. Generating a flood image may include capturing flood light from the object. Flood image may show at least a part of a contour of the object. In an embodiment, the flood image may be associated with the visible illumination and/or the flood infrared illumination.

In an embodiment, the infrared illumination may be within a range between 750 nm and 1000 nm. In an embodiment, it may be determined if the object corresponds to an authorized user based on the flood image, and the payment may be further released based on determining that the object corresponds to an authorized user based on the flood image.

In an embodiment, object may be a spoofing object, a user of the vehicle, an owner of the vehicle, an authorized person and/or an occupant of the vehicle. The object may be determined to be an authorized person. Prior to determining that the object may be an authorized person, the term object is used to include the possibility of spoofing object such as masks, images or the like as being presented in an authentication process.

In an embodiment, a head pose of the occupant may be detected based on the flood image and the payment may be further released based on the head pose of the occupant. Detecting a head pose may include determining a pitch angle and/or a yaw angle based on the flood image. Contour of the object may show the head pose. Contour may indicate the position of a part of a object in relation to another part of the object. The distance between these two parts of the object may indicate a head pose. For example, contour may indicate the position of the nose and the position of the eyes. If a driver would look substantially towards the camera located eg in the dashboard by looking on the road, the distance between the right eye and the nose is similar to the distance between the left eye and the nose. If the driver would turn to the left side, the distance between the right eye and the nose may be significantly higher than the distance between the left eye and the nose or the left eye may not be visible anymore. Thus, the contour of the driver’s face includes information about the head pose of the driver. Based on determining the head pose, the payment may be released. The payment may be released based on the determined head pose if the determined head pose corresponds to an allowed head pose. An allowed head pose may enable determining if the object corresponds to a living human based the at least one pattern image and/or determining if the object corresponds to an authorized user based on the flood image. An allowed head pose may be defined by a range of pitch angles and/or yaw angles. Determining that the pitch angles and yaw angles associated with the flood image correspond to the defined by a range of pitch angles and/or yaw angles may result in releasing the payment.

In an embodiment, releasing the payment may be followed by carrying out the transaction associated with the payment and/or enabling the use of a product associated with the request for payment, providing the product associated with the request for payment and/or the providing the service associated with the request for payment.

In an embodiment, the occupant may be a driver of the vehicle.

In an embodiment, the pattern image may show the object, in particular it may show the object under illumination with patterned infrared light.

In an embodiment, the system for accessing a vehicle and/or a function of the vehicle may be integrated into the exterior of the vehicle and into the interior of the vehicle. By doing so, a secure passive access and passive starting is enabled without the driver of the car noticing. Further, the driver is relieved of the burden to unlock and start the vehicle on its own. This is also known as frictionless access. The same technical effect is achieved when the method is carried out for accessing the vehicle and accessing a function of the vehicle.

In an embodiment, the pattern image may be generated while the object may have been illuminated by patterned infrared light and outside of the vehicle. A second pattern image may be received in response to allowing the object to access the vehicle based on the pattern image generated outside of the vehicle. The second pattern image may have been generated while the object may have been illuminated by patterned infrared light and within the vehicle. The second pattern image may have been generated by an image generation unit covered by a transparent display. The second pattern image may show an occupant under illumination with patterned infrared light. It may be determined if the occupant may be a living human based on the second pattern image. Based on determining that the occupant may be a living human, the occupant is allowed to access a function of the vehicle such as starting the engine. This embodiment shows how frictionless access can be enabled without compromising on safety.

In an embodiment, determining that the object corresponds to a living human may comprise determining that the object associated with the pattern image is blood-perfused and/or determining that the object comprises of skin.

In an embodiment, the image generation unit and/or the projector may be covered at least partially with a transparent display. By doing so, it is not obvious from the outside that authentication is performed. Since it is not visible to a potential impostor, the chance for fraud is reduced. The transparent display is at least partially transparent. The term “at least partially transparent” may refer to a property of the display to allow light, in particular of a certain wavelength range, e.g. in the infrared spectral region, in particular in the near infrared spectral region, to pass at least partially through. For example, the display may be semitransparent in the near infrared region. For example, the display may have a transparency of 20 % to 50 % in the near infrared region. The display may have a different transparency for other wavelength ranges. For example, the display may have a transparency of > 80 % for the visible spectral range, preferably > 90 % for the visible spectral range. The transparent display may be at least partially transparent over the entire display area or only parts thereof. Typically, it is sufficient if only those parts of the display area are at least partially transparent trough which light needs to pass from the projector or to the camera. In particular, the transparent display may be at least partially transparent where the projector and/or the camera are covered by the display.

In an embodiment, the patterned infrared illumination may be emitted by an illumination source and the pattern image may be generated by an image generation unit, wherein the illumination source and/or the image generation unit may be covered at least partially by a transparent display. The display may comprise a display area. The term “display area” may refer to an active area of the display, in particular an area which is activatable. The display may have additional areas such as recesses or cutouts. The display may have a first area associated with a first pixel per inch (PPI) value and a second area associated with a second PPI value. The first PPI value may be lower than the second PPI value, preferably first PPI value is equal to or below 400 PPI, more preferably the second PPI value may be equal to or higher than 300 PPI. The first PPI value may be associated with the at least one continuous area being at least partially transparent. The transparent display may be attached to a vehicle. It can be places at various places, for example on the outside of a vehicle or the inside. When placed outside of the vehicle, it may be integrated into a vehicle body, a door, a window, a mirror or in between windows, for example in the B pillar of a car. When placed inside the vehicle, it may be integrated into the steering wheel, replacing a speed gauge, in the center of a dashboard, in a mirror or in a window.

In an embodiment, providing the determined condition measure may include determining if the determined condition measure may correspond to a target condition measure and allowing the driver to control at least one function of the vehicle in response to determining that the determined condition measure may correspond to the target condition measure.

In an embodiment, determining a condition measure associated with a condition of the driver based on the at least two pattern images and the indication of the at least one interval may comprise providing the at least two pattern images and the indication of the at least one interval to a condition model. The condition model may be a data-driven model, in particular based on a statistical distribution between pattern images, indications on intervals and condition measures.

The condition model may be parametrized and/or trained based on historical pattern images, historical indications on intervals and historical condition measures. The condition model may be parametrized and/or trained to determine condition measures in response to being provided with at least two pattern images and at least one indication of the interval.

A condition measure may be a measure suitable for determining the condition of a living organism. A condition of a living organism may be a physical and/or mental condition. A physical condition may be associated with physical stress level, fatigue, excitation, suitability of performing a certain task of a living organism or the like. A mental condition may be associated with mental stress level, attentiveness, concentration level, excitation, suitability of performing a certain task of a living organism or the like. Such a certain task may require concentration, attention, wakefulness, calming or similar characteristics of the living organism. Examples for such a task can be controlling machinery, vehicle, mobile device or the like, operating on another living species, activities relating to sports, playing games, tasks in an emergency case, making decisions or the like. Condition measures indicate a condition of a living organism. Condition measures may be one or several of the following: heart rate, blood pressure, aspiration level or the like. In some embodiments, the condition of a living organism may be critical corresponding to a high value of the condition measure and the condition of a living organism may be non-critical corresponding to a low value of the condition measure. Followingly, the critical condition measure according to these embodiments may be equal or lower than a threshold and a non-critical condition measure may be lower than a threshold. In other embodiments, the condition of a living organism may be critical corresponding to a low value of the condition measure and the condition of a living organism may be non-critical corresponding to a high value of the condition measure. Followingly the critical condition measure according to these embodiments may be equal or higher than a threshold and a non-critical condition measure may be lower than a threshold. A critical condition measure may be associated with a high stress level, low attentiveness, low concentration level, high fatigue, high excitation, low suitability of performing a certain task of the living organism or the like. A non- critical condition measure may be associated with a low stress level, high attentiveness, high concentration level, low fatigue, low excitation, high suitability of performing a certain task of the living organism or the like.

The condition measure of a living organism may be determined based on the motion of a body fluid, preferably blood, most preferably red blood cells. The motion of body fluids is not constant over time but changes due to activity of parts of the living organism, e.g the heart. Such a change in motion may be determined based on a change in feature contrast over time. A high difference between values of feature contrast at different points in time may be associated with a fast change in motion. A low difference between values of feature contrast at different points in time may be associated with a slow change in motion. The change in motion of a body fluid, preferably blood, may be periodically associated with a corresponding motion frequency. Accordingly, the feature contrast may change periodically with the corresponding motion frequency. The motion frequency may correspond to the length of a period associated with the periodic change in feature contrast. In some embodiments, half of a period may be comprised in the at least two reflection images. In other embodiments, one or several periods may be comprised in the at least two reflection images. Preferably, pattern feature associated with the same part of a living organism may be used for determining the condition of a living organism. This is advantageous due to the fact that the blood perfusion and thus, the feature contrast across different parts of the body varies. In some embodiments, at least one condition measure may be determined based on the feature contrast.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following, the present disclosure is further described with reference to the enclosed figures. The same reference numbers in the drawings and this disclosure are intended to refer to the same or like elements, components, and/or parts.

Figure 1 shows the elements of the system for accessing a vehicle and/or a function of the vehicle placed behind a transparent display.

Figures 2 and 3 illustrate potential placements of the system for accessing a vehicle and/or a function of the vehicle.

Figure 4 shows how the driver is allowed to access the vehicle and/or a function of the vehicle. Further Figure 4 shows an embodiment of how in-car payment can be conducted.

DETAILED DESCRIPTION Start of figure description

The following embodiments are mere examples for implementing the method, the system or application device disclosed herein and shall not be considered limiting.

Figure 1 illustrates the elements of the system for accessing a vehicle and/or a function of the vehicle 100 which is attached to a vehicle. It contains the transparent display 101 which allows light 120 to pass from the illumination source 102 to a person 110. The light may be infrared light, in particular patterned infrared light, which is invisible to the person. The transparent display 101 may only be transparent at the positions at which light 120, 130 passes. T ransparent may mean that at least 30 % or at least 50 % of the incident light passes through the transparent display 101. The transparent display 101 further allows reflected light 130 which was reflected by the person 110 to pass to the camera 103. The light 120 may impinge on the face of the person, but it may also impinge on the whole head including hair, the upper part of the body including head neck and shoulders or even the complete body. The camera 103 generates an image in the optical range matching the wavelength emitted by projector 102, for example in the infrared range. The image may be a grayscale image, i.e. each pixel contains only the total intensity information, or an RGB image, i.e. different pixels indicate the intensity in a particular wavelength. The image is passed to processor 104. The processor 104 may be a microcontroller, i.e. containing memory and IO controller functionalities or it may be a CPU which is connected to memory and IO controllers. The processor 104 determines if the person is an authorized person and/or if the person is a real human. Determining if the person is authorized may involve vectorizing the image into features. Such feature vector may be compared to a stored template. If the difference between the feature vector and the stored template is below a predefined threshold, the processor may determine that the person in the vehicle is authorized. The processor may further determine if the image really shows a human rather than a spoofing mask. This may be accomplished by classifying the material of the face in the image by evaluating reflection characteristics in the reflected light. Another option to determine if the pattern image shows a real human is to detect blood perfusion, eg in the facial region. If no skin or blood perfusion is detected, the processor may determine that the person in front of the transparent display is not authorized or that a spoofing object is presented. The processor may generate a signal 140 indicating that the person in the vehicle is authorized if the authorized user is presented. The signal 140 may be forwarded to a controller for unlocking of the vehicle, starting an engine, granting access to the board computer or effecting a secure payment, for example via a wireless communication interface.

Figure 2 indicates potential positions where the system for accessing a vehicle and/or a function of the vehicle may be attached to a vehicle on the outside, for example a car 200. The system for accessing a vehicle and/or a function of the vehicle may be placed between the windows in the B pillar 201. Alternatively or additionally, the system for accessing a vehicle and/or a function of the vehicle may be placed in the side mirror 202. The system for accessing a vehicle and/or a function of the vehicle may be integrated in a B pillar and inside the vehicle to enable a frictionless access. Frictionless access includes accessing the vehicle without having the user to unlock the vehicle and starting the engine of the vehicle without having the user to start it. Figure 3 indicated potential positions where the system for accessing a vehicle and/or a function of the vehicle may be attached on the inside of a vehicle. The figure shows a dashboard 301 and the windshield of a car as seen from the inside of the car. The system for accessing a vehicle and/or a function of the vehicle may be integrated into the interior mirror 302. This may be particularly useful if the mirror functionality is only mimicked by a display which displays the rear view recorded by a camera. Another possibility is space behind the steering wheel 303 where the gauges such as the speed gauge are typically placed. Furthermore, the system for accessing a vehicle and/or a function of the vehicle may also be integrated into the steering wheel 304. The center console 305 is another option as replacing traditional controls with a display become more and more popular. In an embodiment, space behind the steering wheel 303 and the center console may be combined in a continuous display.

Figure 4 shows a system for accessing a vehicle and/or a function of the vehicle in the inside of a car. The system for accessing a vehicle and/or a function of the vehicle 401 is placed behind the steering wheel into the dashboard. Light rays 402 are emitted onto the face of the driver 403. The light reflected by the person 403 may pass through the system for accessing a vehicle and/or a function of the vehicle 401 where it is recorded by a camera which generates an image which is analyzed by a processor to determine if the person is an authorized person and/or if the object associated with the pattern image is a living human.

The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure and the claims.

Any steps presented herein can be performed in any order. The methods disclosed herein are not limited to a specific order of these steps. It is also not required that the different steps are performed at a certain place or in a certain computing node of a distributed system, i.e. each of the steps may be performed at different computing nodes using different equipment/data processing.

As used herein ..determining" also includes ..initiating or causing to determine", “generating" also includes ..initiating and/or causing to generate" and “providing” also includes “initiating or causing to determine, generate, select, send and/or receive”. “Initiating or causing to perform an action” includes any processing signal that triggers a computing node or device to perform the respective action.

In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

Any disclosure and embodiments described herein relate to the methods, the systems, devices, the computer program element lined out above and vice versa. Advantageously, the benefits provided by any of the embodiments and examples equally apply to all other embodiments and examples and vice versa.

All terms and definitions used herein are understood broadly and have their general meaning.

Claims

1. A method for accessing a vehicle and/or a function of the vehicle, the method comprising: illuminating an object with patterned infrared illumination; generating at least one pattern image of the object while the object is being illuminated by the patterned infrared illumination; determining if the object corresponds to a living human based the at least one pattern image; and allowing the object to access the vehicle and/or the function of the vehicle based on determining that the object corresponds to a living human.

2. A method for performing in-car payment, the method comprising: receiving a request for payment, illuminating an object with patterned infrared illumination; generating at least one pattern image of the object while the object is being illuminated by the patterned infrared illumination; determining if the object corresponds to a living human based the at least one pattern image; and releasing the payment based on determining that the object corresponds to a living human.

3. The method according to claim 2, wherein the request for payment and/or the illuminating is triggered by a driver, an occupant, an app store associated with the vehicle, a toll station and/or a gas station.

4. The method according to claim 1, the method further comprising receiving and/or generating a flood image associated with the object and detecting a head pose of the object based on the flood image and wherein the object is allowed to access the vehicle and/or a function of the vehicle based on the head pose.

5. The method according to claims 2 and 3, the method further comprising receiving and/or generating a flood image associated with the object and detecting a head pose of the object based on the flood image and wherein the payment is further released based on the head pose of the object.

6. The method according to claims 2, 3 and 5, the method further comprising receiving and/or generating a flood image associated with the object and determining if the object corresponds to an authorized user based on the flood image, and wherein the payment is further released based on determining that the object corresponds to an authorized user based on the flood image.

7. The method according to claims 1 and 4, the method further comprising receiving and/or generating a flood image associated with the object and determining if the object corresponds to an authorized user based on the flood image, and wherein the object is allowed to access the vehicle and/or a function of the vehicle based on determining that the object corresponds to an authorized user based on the flood image.

8. The method according to any one of the preceding claims, wherein the patterned infrared illumination is emitted by an illumination source and the illumination source is covered at least partially by a transparent display and/or wherein the at least one pattern image is generated with an image generation unit and the image generation unit is covered at least partially by a transparent display and/or wherein the flood image is generated with an image generation unit and the image generation unit is covered at least partially by a transparent display.

9. The method according to claim 2 to 6, wherein releasing the payment is followed by carrying out the transaction associated with the payment and/or enabling the use of a product associated with the request for payment, providing the product associated with the request for payment and/or the providing the service associated with the request for payment.

10. The method according to any one of the preceding claims, wherein a data-driven model is used for determining if the object corresponds to a living human based the at least one pattern image and/or determining if the object corresponds to an authorized user based on the flood image.

11. The method according to any one of the preceding claims, wherein determining if the object corresponds to a living human based on the at least one pattern image comprises extracting material data from the pattern image and/or determining if the object corresponds to a living human based on the at least one pattern image comprises determining a blood perfusion measure, wherein the patterned infrared illumination is coherent patterned infrared illumination.

12. The method according to any one of the preceding claims, wherein the infrared illumination is within a range between 750 nm and 1100 nm.

13. A system for accessing a vehicle and/or a function of the vehicle, the system comprising: an input configured for receiving a request for payment, an illumination source configured for illuminating the object with patterned infrared illumination, an image generation unit configured for generating at least one pattern image of the object while the object is being illuminated by the patterned infrared illumination; a processor configured for determining if the object corresponds to a living human based the at least one pattern image; and allowing the object to access the vehicle and/or the function of the vehicle based on determining that the object corresponds to a living human.

14. A vehicle comprising the system according to claim 13.

15. Use of a system according to claim 13 for authenticating a user for in-car payment, for vehicle access and/or for starting a vehicle.