Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/18—Eye characteristics, e.g. of the iris
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
  • G06F3/013—Eye tracking input arrangements
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T7/00—Image analysis
  • G06T7/20—Analysis of motion
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/10—Image acquisition
  • G06V10/12—Details of acquisition arrangements; Constructional details thereof
  • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
  • G06V10/141—Control of illumination
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/20—Image preprocessing
  • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/40—Extraction of image or video features
  • G06V10/54—Extraction of image or video features relating to texture
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
  • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
  • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/18—Eye characteristics, e.g. of the iris
  • G06V40/193—Preprocessing; Feature extraction
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/40—Spoof detection, e.g. liveness detection
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/40—Spoof detection, e.g. liveness detection
  • G06V40/45—Detection of the body part being alive
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/20—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only
  • H04N23/21—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only from near infrared [NIR] radiation

Definitions

• the present invention relates to a Driver Monitoring System, DMS, configured to detect spoofing attempts.
• DMS Driver Monitoring System
• biometric authentication factors As technology has increased the ability to accurately determine identity based on various biometrics, they have become increasingly popular as authentication factors.
• the accuracy of identification using biometrics means that it can ensure authentication and non-repudiation, while the fact that biometrics are intrinsically tied to the person ensures availability without imposed burdens such as remembering passwords, or carrying physical tokens, such as keys.
• Flowever, with new authentication factors come new potential attack vectors that need consideration for security to be guaranteed.
• One attack vector could be spoofing, that is presenting falsified biometric data that mimics that of an authorized user. For example, if visual facial features are used as authentication factor, then one could spoof facial features by presenting a picture of the authorized person, or by wearing a mask that looks like them.
• the anti-spoofing systems be readily available, and seamlessly functionable in daily situations, for instance within vehicles, where visually-based authentication of a user, e.g. face-recognition, is utterly beneficial. Additionally, it is desirable that integrating these anti spoofing systems into already existing hardware be possible at a minimum cost.
• a Driver Monitoring System configured to detect spoofing attempts
• the system comprising: an image sensor module, for capturing a sequence of image frames of a portion of a target; a light source, operable to emit IR pulses, the IR pulses having a pulse width, and being temporally separated by a separation time duration; a controller configured to control the light sources and the image sensor, so to capture each consecutive image of the sequence in a different lighting condition, the different lighting conditions corresponding to the IR pulses of the light sources; a processor, and a processor operable software implemented in the processor.
• the software comprises: a motion analyzer, operable to analyze a motion in said sequence of images, and output a motion-based result; a first neural network, trained to extract at least one type of information from an at least two channel image created by stacking a set of at least two consecutive images from the image sequence, and output a spatial result; a fusion program, operable to merge the motion-based and the spatial results, and to output a final result, the final result indicating if the target is live or fake.
• the first neural network is trained to perform 3D shape analysis to extract the at least one type of information, so that said spatial result distinguishes between 2D and 3D shapes.
• the first neural network may exploit the 3D information provided by the different lighting conditions of the set at least two consecutive images, and may extract a combination of texture and shape information.
• the image sensor module, and the light source may be integrated parts of a DMS system.
• the images may be taken from at least a portion of the target, say for instance the eye region of a driver, or the entire face of the driver.
• the fusion program is a classifier, which outputs the final result based on the motion-based result and/or the spatial result.
• the final result should be understood as an indication whether images acquired by the DMS relate to an actual living person (live) or not (fake).
• the fusion program may be a type classifier, for example but not limited to a rule-based classifier, or alternatively, a neural network classifier, which may be trained on pre-registered data.
• the classifier may determine a result based on the output of any or both motion analyzer and first neural network. For instance, a basic rule-based classifier may simply forward both or either of the motion-based result, and spatial result as the final result. Alternatively, if both the motion-based result and the spatial result determine that the target is “fake”, the fusion program may output a final result of “fake”, otherwise if the motion-based result and/or the spatial result determine that the target is live, the fusion program may output a final result of “live”.
• the fusion program ensures that the advantage of both types of anti-spoofing systems; i.e. by means of the motion analyzer, and the first neural network are brought together and exploited within one compact, low-cost system.
• the motion analyzer may be a tracker, such as an eye tracker, such that the outputted motion-based result may be dependent on results of previous image frames which may have been analyzed prior to the image frames of the sequence.
• a 3D surface may be at least partially reconstructed based on multiple 2D images captured in different lighting conditions, which may typically be referred to as photometric stereo. This is due to the fact that the amount of light reflected from a surface may vary with the angular relationship between the surface, the light source, and the camera.
• a method for detecting spoof attempts using a driver monitoring system, DMS, comprising steps of: a) emitting IR pulses, the pulses having a pulse width, and being temporally separated by a time duration, b) acquiring a sequence of image frames of at least a portion of a target such that each consecutive image of the sequence is acquired in a different lighting condition, the different lighting conditions corresponding to the IR pulses; c) analysing a motion in the sequence of image frames, and outputting a motion-based result; and d) choosing at least one set of at least two images from the sequence of image frames, such that the two images are taken consecutively, e) determining a region of interest in the set of at least two images, f) creating an at least two channel image by appending (or stacking) said at least two images (in a depth-wise manner), g) feeding said at least two channel image to a first neural network, h) extracting at least one type of information,
• DMS driver monitoring systems
• eye tracking for traffic safety purposes
• spoofing fraud such as the use of neural networks
• Step e may be carried out based on the motion-based results, from which feature locations, including the location of the region of interest may be extracted.
• Step e may be referred to as frame preparation, in which consecutive image frames may be for example cropped, rotated, translated, etc. to include the region of interest in a pre-determined size.
• each appended image will add an additional channel.
• the image frames may be acquired by an NIR image sensor module, while the IR pulses may be emitted by at least one light source.
• Steps c through k and the alternative and/or additional steps mentioned below may all be carried out by a processor comprising a processor implemented software.
• the image sensor module may be referred to as a camera assembly, or camera for short.
• image sensor module and camera are used interchangeably, and unless stated otherwise, are meant to refer to the same component.
• the light source is configured to emit IR light continuously.
• the separation time duration is arranged to be zero.
• the first neural network may only have the possibility of extracting texture information, and not 3D information.
• the IR light source may comprise at least two IR light sources separated by a distance from the camera, which is in turn positioned at a certain distance from the target.
• two light sources spatially separated by a first distance, and arranged to emit IR light alternatingly. It may be that the light sources may be positioned at the furthest distance from one another, such as on either side of the camera.
• one light source may be positioned at the front of the camera; closer to the target, while the other is positioned on one side of the camera.
• the spatial separation may allow for obtaining distinct lighting conditions in which the images will be acquired in.
• the pulse width of the IR pulses from each of the light sources are equal.
• the pulse width of the IR pulses from each of the light sources are not equal.
• the sequence of images may be taken in different lighting conditions that exclusively correspond to the pulse width of the IR pulses.
• the latter may be a configuration best fitted for embodiments with more than one light source.
• one of the different lighting conditions corresponds to a non-illuminated condition between IR pulses.
• some of the images may be acquired during the temporal separation between the pulses so that specific lighting condition is a non-illumination by the light source.
• the ambient lighting may provide one of the different lighting conditions.
• the consecutive images from the sequence of images preferably have a short temporal distance between them, in order to reduce unwanted significant target movements between them, so that the network may focus on discriminating between live and fake targets.
• the network may be trained to handle any existing motion between the frames, by for instance geometric alignment of the images if the target, and/or the camera, and/or a feature such as an eyelid has moved in between consecutive frames.
• the eye tracking program is configured to detect and compare a reflection against a retina of the target in said sequence of images and feed an outcome to the fusion program.
• lighting conditions in which the appearance of the target’s pupil may differ may be created and compared.
• the motion analyzer may not necessarily, or explicitly detect facial motions per se, but may detect the existence or non-existence of a reflection off the retina in different lighting conditions, for instance, the motion analyzer may detect whether the pupil is dark or bright.
• This embodiment may be highly beneficial in detecting spoofs such as 2D prints of a target, or reoccurring video loops in a fast and efficient way.
• the information about the pupil being light or dark may be fed as input to the first neural network. More information about alternative embodiments where information from the motion analyzer may be fed to the first neural network as input can be found further in the text.
• the first neural network is trained to perform texture information analysis to extract the at least one type of information, so that the spatial result is a classification of the target being live, or texturally fake, or spoof, for example a portion of a 2D photograph, a fake object, or a replaying video loop.
• the fake object may be a mask, a fake human head, such as a doll head, or a mannequin.
• the neural network may be trained to recognize texture information such as color, lines, silhouettes, etc. and differentiate between the texture and/or shape of a live target and a fake target.
• the first neural network may be trained to discriminate between “live” or “fake” on an intermediate level.
• the set of at least two consecutive images may for this purpose be combined together and fed to the first neural network.
• the first neural network may extract texture information from the fed stacked images.
• the motion-based and spatial results are both fed as direct input to the fusion program, to output the final result by directly merging said motion-based and spatial results.
• the first neural network additionally functions as the fusion program so that said spatial result is said final result, and/or wherein the motion-based result, i.e. the detected and analyzed motion information from the motion analyzer, is fed as input into the first neural network.
• the first neural network is additionally trained to operate on the information from the motion data analysis to extract information on whether the target is live or fake.
• the spatial result and the final result may be the same, and the same as the final outputted result from the spoof-detection system.
• at least a subset of the motion-based result is used for determining the region of interest in the set of at least two images, for instance to crop the image frames of the set of at least two images and extract the face region of the target.
• the motion analyzer additionally functions as the fusion program so that said spatial result outputted from the first neural network is fed as input into the motion analyzer.
• the motion analyzer is additionally trained to operate on the information from the first neural network to extract information on whether the target is live or fake.
• the motion-based result and the final result may be the same, and the same as the final outputted result from the spoof-detection system.
• a DMS system it may of course be that more than one sequence of image frames is captured of at least a portion of a target by said NIR image sensor module, consecutively. This may be well in line with the video acquisition functionality of a DMS system.
• the motion analyzer is operable to analyze a motion in each sequence of said plurality of sequence of images, and output more than one motion-based result, each motion-based result corresponding to each analyzed sequence of images.
• the first neural network may be operable to extract at least one type of information from a set of at least two consecutive images from each of said more than one image sequences, and output more than one spatial result, each spatial result corresponding to each sequence of images.
• the system may benefit from a more reliable spoof detection, due to a larger number of spoof analysis. It may be that the latter is performed with a given frequency throughout the duration of usage, e.g. driving.
• the processor implemented software further comprises a second neural network trained to perform scene dynamics analysis to extract a second type of information from a plurality of image sequences, and output an iteration result, the iteration result being a second motion-based result, and wherein the iteration result outputted by the second neural network is fed into the fusion program.
• a plurality of sequence of image frames are acquired, other information useful for spoof detection may be extracted by analysing the target over an extended period of time. For this, image frames of each sequence of images may be appended together into one stack, such that each stack of the multiple stack corresponds to one sequence of images. This may add yet another layer of analysis to the spoof detection, in which inconsistencies or repetitions may be detected, and output the iteration result and fed to the fusion program.
• a recurrent neural network architecture may be utilized as the second neural network, to which the multiple stack may be fed.
• the second neural network is trained to analyze scene dynamics to extract the second type of information.
• Scene dynamics may for example refer to the ambient environment of the target, which may in turn refer to the in-cabin environment of a vehicle, and/or the outer environment for example visible through the vehicle window. This may be beneficial for detecting spoofing attempts such as 2D prints of a target presented to the camera, in which the scenery never changes. Additionally, or alternatively, and more importantly, it may be utterly useful in detecting replaying video loops, in which the scenery changes are recurrent over an extension of time, which only one layer of spoof detection, i.e. motion analysis, together with the first neural network analysis (the motion-based and spatial results) may not be capable of detecting.
• the second neural network is operable independent of the motion analyzer and/or the first neural network.
• An embodiment includes a further step of calculating a confidence level of at least one of said motion-based result, said spatial result, said iteration result and/or said final result. This may be beneficial to account for possible errors in the system, measurements and/or analysis.
• Figure 1 a and 1 b schematically show a DMS system configured to detect spoofing attempts.
• Figure 2 shows a block diagram of an embodiment of the anti-spoofing system.
• Figure 3 shows a block diagram of an embodiment of the anti-spoofing system.
• Figure 4 shows a flowchart of a method for detecting spoofing attempts.
• Figure 1a schematically demonstrates a DMS system 1 configured to detect spoofing attempts, i.e. to determine if images acquired by the DMS relate to an actual living person and not e.g. to some artifact like a photo or face mask.
• the system includes a camera 10 and two infra-read (IR) light sources 20a, 20b.
• the camera 10 generally may comprise an image sensor that includes an active region, the active region including a plurality of pixels operable to sense radiation substantially in an IR part of the spectrum, an optical assembly 11 disposed over the active region of the image sensor, the optical 11 assembly controlled by a controller 30 and configured to consecutively expose the image sensor to a sequence of exposures, corresponding to the sequence of images
• the optical assembly 11 may comprise a shutter, a read-out circuit configured to acquire output signals from the plurality of pixels, one or more image processors configured to generate the sequence of images based on the output signals from the plurality of pixels.
• the camera 10 further includes an IR or near-IR (NIR) filter 13.
• the filter 13 may be configured to be withdrawable, for instance in the case of capturing images during the temporal separation of the time duration between the IR pulses.
• the light sources 20a, 20b may be solid state light sources, such as a LED.
• the light sources 20a, 20b are LEDs configured to emit light with a light spectrum concentrated in a 50 nm band centered around 850 or 940 nm (NIR).
• the filter 13 may be a band pass filter, e.g. an interference filter.
• the filter 13 may be configured to have a pass-band substantially corresponding to the light emission spectrum of the light sources 20a, 20b.
• the filter 13 should have a pass-band of around 825 - 875 nm, or 915 - 965 nm.
• the combination of narrow band illumination and narrow pass-band filtering makes the image acquisition system less sensitive to surrounding light, e.g. sunlight.
• FIGS 1 a and 1 b relative distances of some of the latter components from one another and/or from a target 50, 9 are schematically demonstrated.
• the system 1 is considered to be installed within a vehicle (not shown), positioned such that the camera 10 is situated facing the target, which is the drivers face 50, and/or the drivers eye region 9, (for short referred to as the driver)at a distance of Dc, while the two IR light sources 20a, 20b are situated on either side of the camera, separated by a length of L from one another, a lateral distance of la, and lb from the camera 10, and at a distance of D1 and D2 from the driver 50, 9.
• the two light sources 20a, 20b are positioned on the same lateral plane with respect to the camera, and/or the driver 50, 9.
• the vertical distance between the camera 10, and the light sources 20a, 20b, is Lc.
• the two light sources 20a, 20b are positioned non-symetrically with relation to the camera 10 and/or target 50, 9.
• the camera 10 is operable to capture a sequence of image frames from the target 50, 9, while the IR light sources 20a, 20b are configured to emit pulses of IR light alternatingly.
• the camera 10, and the IR sources 20a, 20b are electronically connected to a controller 30.
• the controller 30 is configured to control the acquisition of the images frames such that they correspond with the pulse width of the IR pulses emitted from either of the light sources 20a, and 20b, and/or from the temporal separation between each consecutive pulse.
• the light sources 20a, 20b are triggered by pulses which have a duty cycle selected to provide sufficient illumination for acquiring one image frame.
• either one or both light sources 20a, 20b may be activated for an image frame.
• the light sources 20a, 20b are controlled by a pulsed signal with a duty cycle to ensure that the pulse width corresponds to the time required for the camera 10 to acquire one image. The camera 10 is then synchronized with this pulsed signal.
• the controller 30 is inturn electronically connected to a processor 40, comprising a processor implemented software.
• the system also has a memory, such as a RAM memory, e.g. storing software to be executed by the processing circuitry 15.
• FIGS 2 and 3 demonstrate example embodiments 42, 44 of how the processor implemented software functions.
• a sequence of consecutive images 100 is configured to be captured by the camera 10 from the target 50, 9 such that each image of the sequence corresponds to a different lighting condition.
• the sequence 100 may be a selected sequence of image frames of a video clip captured by the camera 10 of the DMS system 1.
• the reason behind acquiring the sequence of images in different lighting conditions is that a 3D surface may be at least partially reconstructed based on multiple 2D images captured in different lighting conditions. It may be that the reconstruction in and of itself is not utilized for the purposes of this invention, but the neural network 210 may extract 3D information through the reconstruction process which may be useful for discriminating a live target 50, 9, from fake.
• the sequence 100 is arranged to be fed to a motion analyzer 200, such as a motion analyzer in a DMS system, which in turn is configured to detect, and analyze motion events, for instance detect and analyze facial movement, such as for instance in the eyes, and/or mouth area of the target 50, 9, and/or to estimate a head pose and a gaze of the target 50, 9, and estimate for example drowsiness, and to output an exsistence, or non- exsistence of natural motions of the target 50.
• a motion analyzer 200 such as a motion analyzer in a DMS system, which in turn is configured to detect, and analyze motion events, for instance detect and analyze facial movement, such as for instance in the eyes, and/or mouth area of the target 50, 9, and/or to estimate a head pose and a gaze of the target 50, 9, and estimate for example drowsiness, and to output an exsistence, or non- exsistence of natural motions of the target 50.
• the motion analyzer 200 may compare images in the sequence with one another and detect if the target 50, 9 demonstrates reasonable and/or natural movements.
• the motion-based result may then be determining the existence or the non-existence of those movements.
• This may be implemented in several ways, by for instance, the processor implemented software calculating a confidence level, meaning that for instance, the processor may give a score of in the case that it detects reasonable movements with certainty e.g. 0, while if absolutely no reasonable movements are detected a score of e.g. 1 is given.
• the score that an analyzed sequence of images receive may lay between 0 and 1.
• a predefined threshold between 0 and 1 , for which below this threshold the outputted motion-based result may be the existence of reasonable and/or natural motion, hence no spoof detection, while for scores above the threshold value the outputted motion-based result may be non-existence of reasonable and/or natural movements, meaning spoof is detected.
• a set 110 of at least two consecutive images from the sequence 100, which have been acquired in different lighting conditions, is arranged to be selected by the processor implemented software.
• the images of the set 110 are appended, in other words, are stacked together, and fed to a first neural network, for example a convolutional neural network 210 for texture and 3D shape analysis by the processor implemented software.
• the image stack or in other words the two-channel image provides the material from which the first neural network 210 can extract 3D information, and/or the possibility to form an at least partial 3D reconstruction of the target 50, 9 from the different lighting conditions of each of the 2D images of the set 110.
• the first neural network 210 may classify the data into: Live, 2D prints, or replaying video loops, or masks, or any other type of spoofing attempt after analysis.
• the outputs from the motion analyzer 200 and the first neural network 210 is arranged to be fed to a fusion program 230, which may be an antispoofing classifier information fusion program, and are merged to output a final result of the target 50 being “live” or “fake”.
• a fusion program 230 which may be an antispoofing classifier information fusion program
• a Convolutional Neural Network may be used for example, in order to extract and analyze information such as, but not limited to texture information.
• texture analysis one may refer to the ability to discriminate between natural looking features, and artifacts. In the eye openings in some facial masks for example, double edges may be detectable, as there may be some inconsistency between the texture of the mask openings and the underlying real eye.
• Images taken in different lighting conditions may reveal different information from the target 50, 9.
• the set of at least two consecutive images 110 are arranged to be stacked onto one another, such that they create an at least two channel image.
• this it is meant to convey that, the information of each image frame of the set 110 is combined in a depth-wise manner so to provide an at least partial 3D construction of the target 50, 9.
• the neural network 210 is trained.
• frames are extracted from recordings made in a data collection, and prepared such that, for example, consecutive image frames may be cropped, rotated, translated, etc. to include the region of interest in a pre-determined size.
• Frames extracted from spoof recordings may be labeled 0, while frames extracted from live recordings may be labeled 1 .
• the frames are then divided so that all recordings of randomly chosen subjects are used as training data, and the rest as validation data.
• Frames from mask recordings are divided so that frames with some of the masks were used for training, while frames with other masks are used for validation.
• a learning rate scheduler may be used to iteratively update the rate.
• the determination and outputting the spatial result may be implemented in several ways, by for instance, the processor implemented software calculating a confidence level, meaning that for instance, the processor may give a score of e.g. 1 in the case that it detects texture discrepancies, and or distinguished 2D from 3D with certainty, while if absolutely no texture discrepancies are detected, and/or 3D structure of the target is determined with certainty a score of e.g. 0 is given.
• the score that an analyzed sequence of images receives may lay between 0 and 1.
• the outputted spatial result may be the existence of texture discrepancies, and/or 2D structures of the target, hence spoof is detected, while for scores below the threshold value the outputted spatial result may be minimal texture discrepancies, or 3D structure of the target, hence no spoofing attempt has occurred.
• Calculating a confidence level may be beneficial when accounting for errors in the system for example, which may be disregarded.
• an additional layer of spoof detection is depicted. Note that, the spoof detection may function independent of this additional layer, and the existence of this layer may add an extra degree of spoof detection to the readily explained embodiment 42.
• the additional layer comprises capturing multiple sequences of image frames 101, 103, 105 from the target 50.
• Each of the images in each of the sequences 101 , 103, 105 are arranged to be taken at different lighting conditions. These sequences may be arranged to be acquired temporally consecuent to one another as it is with video acquisition.
• the images in each sequence 101 are arranged to be captured temporally consecuent to one another as it is with video acquisition.
• 103, 105 are arranged to be combined with one another, resulting in a multiple stack 150 of images.
• This multiple stack 150 is arranged to then be fed as input to a second neural network 270 by the processor implemented software.
• the second neural network may be for example, a recurrent neural netwok (RNN), in which inconsistencies or repetitions may be detected, and analyzed and an iteration result will be outputted.
• RNN recurrent neural netwok
• the output is then fed to the fusion program 230, 250, configured to merge the iteration result with the other outputs and to output a final result of the target 50 being “live” or “fake”.
• the second neural network 270 may be trained prior to use.
• the determination and outputting of the final result can be implemented in several ways, by for instance, the processor implemented software calculating a confidence level at the final layer.
• the sequence of images 100 captured from the target 50 is arranged to be fed to the motion analyzer 200 to detect, and analyze motion events, and output an exsistence, or non-exsistence of natural motions of the target 50.
• the output from the motion analyzer 200 is arranged to be fed as input together with the combined set of at least two selected consecutive images 110 from the sequence 100, to a first neural network 250, which not only performs texture and 3D shape analysis on the inputted data, but will also perform fusion, to output a final result of the target 50 being “live” or “fake”.
• an additional layer of spoof detection is depicted. Note that, again the spoof detection may function independent of this additional layer, and the existence of this layer may add an extra degree of spoof detection to the readily explained embodiment 44,.
• the additional layer is configured to perform the same steps mentioned for that of embodiment 42, and therefore reference is made to that description.
• Figure 4 shows a flowchart of a method for detecting spoofing attempts using the system in Fiigures 1-3.
• the light sources 20a, 20b emit IR pulses with a given time duration, and a given temporal separation.
• step S2 simultaneously with each IR pulse and/or in the separation of the time duration in between each consecutive IR pulse, the camera 10 acquires a sequence of images 100 of the target 50, 9 in different conditions created by the alternating IR pulses. Therefore, steps S1 and S2 are required to be carried out in conjunction with one another.
• the pulse width of the IR pulses, hence the image acqusition time of the camera is set to 350 ps.
• each IR pulse is started 500 ps into the period, and is ended after 350 ps. Therefore, an effective frame rate of 60 frames per second will be achieved for the mentioned two-flash configuration.
• an effective frame rate of 30 frames pers second may be reached.
• the motion analyzer 200 of the processor implemented software analyzes the sequence of images 100 for detecting existence of natural and/or reasonable motions of the target 50, 9.
• the motion analyzer 200 calculates a confidence level in S4 in order to detect the existence or non existence of the mentioned motions, and to compensate for possible error of the system , and/or measurements.
• S4 is not an essensial step for performing the method as shown in the example embodiment of Fiigure 4, and may be neglected, or alternatively, replaced with other measures.
• the motion analyzer 200 outputs a Motion-based result.
• a set of at least two consecutive images 110 acquired at different lighting conditions are selected from the sequence of images 100.
• a region of interest is determined in the set of at least two images 110.
• This region of interest could for example be the eye region of the target 50, 9, or the entire face region.
• This step may be referred to as frame preparation, in which consecutive image frames may be for example cropped, rotated, translated, etc. to include the region of interest in a pre-determined desired size.
• the frame preperation e.g. cropping is typically done based on the motion-based results.
• the tracker may provide for example facial feature coordinates which may be used to determine the crop. Alternatively, other methods, such as utilizing a face detector may be used for achieving the latter, which would then render the antispoofing feature independent of the motion analyzer.
• the images of the set of at least two consecutive images 110 are stacked onto one another in a depth-wise manner as previously explained, in order to create an at least two channel image.
• each consecutive image taken in different lighting conditions of the set will add a channel. For instance if more than two consecutive images, for example three of four images taken in different lighting conditions are chosen for the set of images 110, and stacked onto one another, then a three- or four-channel image will be created.
• the stack is fed to a first neural network that will preform classifaction of the set by analyzing its texture information and/or distinguishing 2D from 3D structures.
• the first neural network calculates a confidence level in S10 for the existence of texture discrepancies, and/or 2D structures of the target, and to compensate for possible error of the system , and/or measurements.
• S10 is not an essensial step for performing the method as shown in the example embodiment of fiigure 4, and may be neglected, or alternatively, replaced with other measures.
• it is not necessary that calculating the confidence level is a post processing step.
• the confidence level may be computed by the first neural network itself, meaning that S10 and S9 may be combined into one step.
• the first neural network outputs a spatial result.
• the motion- based result of S5 may be fed as input to the neural network S9 together with the image stack.
• the motion-based result of S5 can be fed together with the spatial result to a fusion program in S12, which will merge the input information, possibly calculate a confidence level in S13 and output a final result in S14 classifying the target 50 as “live” or “fake”.
• S21 - S26 An independent set of steps S21 - S26 is visible on the left side of the flowchart shown in Figure 4, which adds an extra layer of spoof detection to the method.
• multiple sequences of image frames 101 , 103, 105 are captured from the target 50, 9 Such that each of the images in each of the sequences 101 , 103, 105 are taken at different lighting conditions. These sequences acquired temporally consequent to one another.
• S22 the images in each sequence 101 , 103, 105 are stacked onto one another, resulting in a multiple stack 150 of images in S23. This multiple stack 150 is then fed as input to a second neural network 270 in S24.
• the second neural network may be for example, a recurrent neural netwok, in which inconsistencies or repetitions may be detected, and analyzed, and a confidence level may be calculated as shown in S25, and an iteration result will be outputted in S26.
• the present invention by no means is limited to the preferred embodiments described above. On the contrary, many modifications and variations are possible within the scope of the appended claims.
• the details of the DMS may be different than those disclosed herein.
• the pulse width and pulse separation may be adjusted based on the implementation, as may the number of frames or sets of frames used as input to the neural networks.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Physics & Mathematics (AREA)
• Human Computer Interaction (AREA)
• Health & Medical Sciences (AREA)
• General Health & Medical Sciences (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Evolutionary Computation (AREA)
• Ophthalmology & Optometry (AREA)
• Databases & Information Systems (AREA)
• Medical Informatics (AREA)
• Software Systems (AREA)
• Computing Systems (AREA)
• Artificial Intelligence (AREA)
• General Engineering & Computer Science (AREA)
• Toxicology (AREA)
• Signal Processing (AREA)
• Image Analysis (AREA)
• Collating Specific Patterns (AREA)
• Traffic Control Systems (AREA)

Priority Applications (4)

Applications Claiming Priority (2)

Publications (1)

Family

ID=71409349

Family Applications (1)

Country Status (6)

Cited By (4)

Families Citing this family (3)

Citations (3)

Family Cites Families (12)

• 2020
  • 2020-07-01 EP EP20183434.8A patent/EP3933666B1/en active Active
• 2021
  • 2021-07-01 JP JP2022581602A patent/JP7692942B2/ja active Active
  • 2021-07-01 WO PCT/EP2021/068192 patent/WO2022003107A1/en not_active Ceased
  • 2021-07-01 CN CN202180047255.1A patent/CN115968487A/zh active Pending
  • 2021-07-01 US US18/013,966 patent/US12505698B2/en active Active
  • 2021-07-01 KR KR1020237002828A patent/KR20230043848A/ko active Pending

Patent Citations (3)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events