WO2018185574A1

WO2018185574A1 - Apparatus and method for documents and/or personal identities recognition and validation

Info

Publication number: WO2018185574A1
Application number: PCT/IB2018/051349
Authority: WO
Inventors: Salvatore MAFODDA; Manuel Alejandro RUIZ GARCIA
Original assignee: Lending Solution S.R.L.
Priority date: 2017-04-04
Filing date: 2018-03-02
Publication date: 2018-10-11
Also published as: IT201700037053A1

Abstract

The present invention relates to the context of the personal identification procedures, validation of identity documents, etc. In particular, the present invention proposes to offer a process and a semi-automatic and secure apparatus for recognition and validation of personal identities, identity documents or both at once.

Description

APPARATUS AND METHOD FOR DOCUMENTS AND/OR PERSONAL IDENTITIES RECOGNITION AND VALIDATION

DESCRIPTION

Background

In general, the recognition and validation tasks under study require a heterogeneous variety of attributes to be analysed, in particular:

• recognition and validation of identity documents may rely in the analysis of optical, visual, physical and electronic data.

• Recognition of a person's identity may rely, among others, in his or her face, fingerprints, identity document, distinctive cultural or linguistic features of an ethnicity.

However, in both cases, a robust and highly reliable outcome can be achieved by exploiting exclusively visual, optical and electronic data when available.

For example, very often on airports, banks and police offices identity documents are only controlled by accurate visual analysis and the person identity is almost verified only by direct comparison with the identity document's photo. Very often, during the document's examination, only the following features are analysed:

• printing substrate features (paper and polymer).

• ink features.

• printing processes and distinctive features thereof. • security features such as: surface data which can be analysed by using UV and/or IR light sources: filigrees, holograms, security drawings, fluorescent fibres).

• physical features of the document (assembly and production).

• biographical data.

• confidential coded information inside an incorporated chip, magnetic tapes, etc., and machine readable zones (MRZ).

Features which can be traced to different inks, distinctive features of printing processes, biographical data, MRZ codes and bar codes can be analysed up to a high level of accuracy and reliability by pure visual and optical means, while smart chips can be read through standard NFC communication interface and protocol.

Technical problem solved by the invention

The object of the present invention is to provide a robust, reliable and fast mechanism for the completion of the above tasks, based exclusively on the accurate analysis of a restricted set of attributes of the identity document and/or face of the document's holder.

This is obtained through a process as defined in claim 1.

The present invention further relates to a multi-spectral lighting device as defined in claim 13.

Still, the present invention further relates to a system for implementing a process, as defined in claim 11.

Additional features of the present invention are defined in the corresponding depending claims.

The advantages, together with the features and the use modes of the present invention, will result evident from the following detailed description of preferred embodiments thereof, shown by way of example and not for limitative purposes. Brief description of the figures

The drawings shown in the enclosed figures will be referred to, wherein:

• figure 1 is a block diagram exemplifying a system according to the present invention;

• figure 2 is a block diagram exemplifying a lighting system according to the present invention;

• figure 3 is a block diagram exemplifying the functionalities of a mobile client device according to the present invention;

• figure 4 is a block diagram exemplifying the functionalities of a server device according to the present invention;

• figure 5 is a block diagram exemplifying the attribute acquisition and extraction module;

• figure 6 is a diagram of a multi-spectral lighting device according to the invention;

• figure 7 is a block diagram of the states of the lighting device according to the invention;

• figure 8 is a block diagram exemplifying the process for detecting a document;

• figure 9 shows some examples for detecting documents performed according to the invention;

• figure 10 is a block diagram exemplifying the process for detecting the contours (or edges);

• figure 11 shows the effects of anti-noise filtering on the detection of contours and the detection of reflection areas;

• figure 12 shows the comparison between detections of contours performed for different colour channels;

• figure 13 shows by way of example the detection of angles and the determination of contour; and

• figures 14 and 15 relate to the phase for identifying the user and the used mobile device. Detailed description of possible embodiments of the invention

The present invention will be described hereinafter with reference to the above-mentioned figures.

As already mentioned, one of the objectives is to be able to use a restricted set of attributes for the recognition and validation of identity documents and of the face of the document's holder. Such set is built upon the hypothesis that the recognition and validation procedures can be robustly and reliably completed by using exclusively visual, optical and electronica data, which can be successfully extracted by using a device capable of capturing both medium or high resolution multi-spectral images (with different light sources, thereamong ultraviolet (UV) and/or infrared (IR) sources) and information coded electronically by standard such Near Field Communication (NFC).

Under the term "multispectral image" in the present description the image of a subject is meant lighted by a multi-spectral source and which, by said lighting, can emit a fluorescence.

Based on this last remark, the main objectives of the present invention are the following:

• to provide a device for the multi-spectral lighting and reading electronically coded data which can be easily attached and controlled by any modern smartphone, to transform such mobile device in a medium-level, fast and reliable forensic document examination device.

• To provide a computer-vision process for the detection, recognition and automatic validation of identity document and face of the document's holder.

• To provide a sophisticated back-end mechanism for validating and correcting any result obtained by recognition and validation automated processes, since the robustness of any computer-vision method is subjected to a predefined set of working conditions, that cannot be always verified in arbitrary application setting. • To define a networked client-server architecture, comprising client devices of "fat" type, for the acquisition, extraction, partial analysis and encoding of all necessary data for recognition and validation purposes, together with any other environmental information that can improve the procedure reliability (date, time, GPS coordinates, etc).

• To provide a software framework wholly based upon suitable mechanisms for the protection of end users' sensible data and privacy, such as: encrypted communication channels, robust data encoding, secure data exchanging protocols and without storing any sensible data that is not strictly necessary to any recognition or validation task.

In general terms, a system according to the present invention, for documents and/or personal identities recognition and validation, thus comprises:

• a mobile client device comprising:

O means for acquiring data;

O first means for processing acquired data;

O data transreceiving means; and

O user interface means.

• a remote server apparatus comprising:

O data transreceiving means;

O second means for processing data received from said mobile client device; and

O means for filing, updating and recovering processed data, wherein said first and second processing means are programmed to implement the processes according to the invention.

As to automatic recognition and validation of a person's identity by means of face biometrical recognisers, this type of technology requires a device for acquiring images with medium or high resolution and a filing device keeping a database of known identities. Therefore, the framework proposed according to the invention provides a server side for filing, updating and recovering data, and a mobile device capable of collecting, analysing and transmitting biometric data and identity document data.

The most innovative aspects of the present invention are the exploitation of the detecting and processing capabilities of a modern smartphone, its means for acquiring images and/or other types of data, to implement a document examination portable device, in case improved thanks to the introduction of a multi-spectral lighting device and two software interfaces (an app for the mobile device and server side services) expanding the device capabilities to a scalable, reliable and secure networking system for recognition and verification of documents and faces.

Such components, as shown in figure 1 , can be summarized as follows:

• Multi -spectral lighting device: it corresponds to a ring of LED lights which can be easily attached to a smartphone, capable of flashing and emitting visible light and/or UV light and/or IR light, triggered and controlled (light intensity, turning-on delay with respect to the triggering signal and turning-on duration) via hardware through an USB interface. As shown in figure 2, this communication interface corresponds to a general USB serial one, commonly known as USB-UART, and as consequence the device can be connected and interfaced to any smartphone supporting the USB On-The-Go (OTG) technology.

• App for mobile device: core functionalities of the present invention are assigned to this component. This application is responsible for handling all user interaction, data acquisition (interfacing with sensors and lighting devices), partial data analysis (or attribute extraction), partial data/attribute verification, data/attribute encoding/decoding, encoded data/attribute transmission and reception through secure and encrypted services. In figure 3 it is possible to display the multiple interactions between the different units defining the application. It is noted that the application itself represents a bridge between the detection devices and the offered services. • Server side services: all services are provided by a server infrastructure, as shown in figure 4. The services can be divided in four major application user interfaces (APIs). The first one, defined by the user validation block, is responsible for the credentials of any remote user or back-end operator; unauthorized persons cannot access any service of the platform, not be able to operate by means of the remote application. The second one, represented by the data storage unit, is a virtual interface between the physical databases and the back-end operator or remote App. The third one, represented by the attribute extraction unit, represents another virtual interface both for demanding algorithms as to the required computation power which cannot be implemented inside the mobile app, and for attribute extraction or for external services providing specific attribute extraction algorithms not implemented inside the present invention. Last, but not least, is the attribute and data correction/validation tool, available exclusively to the back-end operators.

Multi -spectral lighting device

As previously stated, most activities of an identity document examination process generally can be performed by visual, optical and electronic data analysis. The detection and acquisition of all information generally requires specialized devices. Such specialization can be divided in terms of the detection type, which can be both active and passive. The acquisition of electronic data is a passive measurement since available data can be extracted, for example, by a NFC reader without any external dependence.

Image capturing is a passive measurement too, since in any indoor or outdoor urban environment has a light source when direct sunlight is missing. On the contrary, capturing UV and/or IR images for document examination requires suitable light sources and, depending on the examination, specific sensors too (all image sensors, regardless the underlying manufacturing technology, perceive a wider light spectrum than the visible. However, any visible light capturing device, like the camera module for smartphones, protects its sensor with different layers of coating filters that block wavelengths beyond the visible spectrum).

However, in the first two phases of identity documentation examination, the UV and/or IR light sources can be employed to induce fluorescence emission of some document materials. On one hand, UV induces visible fluorescence which under intense illumination can be directly captured by any camera module. On the other hand, visible and IR light induce IR fluorescence. Depending upon the manufacturing of the module camera (in particular, on the quality of the IR cut-off filter), this kind of fluorescence can be also acquired by using intense illumination together with a proper IR filter. Modern smartphones are provided with high-resolution cameras and NFC readers that can be properly exploited to perform a reliable visual and electronic data acquisition. Then, by providing suitable light sources and filters, one can exploit the smartphone camera module to acquire also optical data beyond the visible spectrum.

To this purpose the present invention intends to provide an external lighting device to allow smartphones the acquisition of fluorescent emissions produced by identity documents under UV and/or IR intense illumination. Moreover, by using a single sensor, the multi-spectral images need to be captured sequentially (one light source at a time), therefore a fundamental requirement for the lighting device is the ability to communicate with the smartphone to synchronize a particular light emission with the data acquisition process.

Hereinafter in the description, then, some main functionalities of the invention will be referred to, as herein reported:

• Development of a multi-spectral lighting device for smartphones, comprising intense UV and IR light sources and the necessary filtering mechanism.

• Development of an electronic controller for the lighting device, able to communicate with the smartphone through standard USB serial communications (USB-UART), to synchronize the capturing process with the external light emission.

• Design of image processing algorithms to improve the acquisition outcomes and to implement the partial analysis and validation of security features automatically.

With reference to figure 6, the lighting device is divided into two main components: a lighting ring and a controller.

The ring is an electronic card including the light sources and all analog circuits necessary to drive them. The light sources are LEDs selected from LEDs in the ultra-violet (UV), LEDs in the infra-red (IR) and/or white LEDs, regularly positioned along its surface. The ring-shaped card can be easily attached to and detached from the controller, which represents the only mechanical support of the ring.

The programmable electronic controller includes all control and logic circuits to drive the ring and to communicate with command interface. This command, control and communication interface corresponds to a general USB serial one, commonly known as USB-UART. As a consequence, it can be connected to, and interfaced with, any smartphone supporting the USB On-The- Go (OTG) technology.

The lighting device is able to simultaneously flash any combination of light sources after receiving a triggering signal, based upon a user defined state. The device state is defined by flash duration, turning-on delay (after the arrival of the triggering signal) and lighting configuration.

Under lighting the light intensity and power state (on/off) of each LED during ring turning-on is meant. Each type of light source has its own lighting configuration. LEDs producing white light share the same light intensity and power state; as a consequence, all white lights are turned-on altogether, each one by producing the same light intensity. Differently, IR and/or UV LED intensity and power state can be set independently; that is, each UV or IR LED can be, or cannot be, activated with a particular light intensity, independently from the state of the other UV or IR LEDs. For any light source type, the device is able to operate at least 16 different discrete levels of light intensities.

The device state commands and the triggering signals are sent by the user to the controller through the command interface. The ring states can be defined by a set of commands and retrieved by a GET command, while the TRIGGER command flashes the lights based upon a previously defined state. Such interface also accepts the following commands:

• ON: it flashes the lights based upon a previously defined state, but ignores the flash duration parameter, by keeping lights powered indefinitely.

• OFF: it turns off the lights when the ring LEDs are powered or in pause.

• TOGGLE: it turns off the lights if they are turned-on or it turns-on them again in case they had been previously turned-off by the same command. In case of a turning-on due to the TRIGGER command, the new turning-on starts again with remaining timeout (that is, if the device state has been configured with a turning-on time T and the ring has been active for a time t<T before receiving the TOGGLE command, then upon the second reception of this command the ring will be activated again for a time equal to T-t).

• STATUS: it returns the state of the lights, which can be 0 (off), -1 (on) or a positive integer indicating the remaining flashing time in milliseconds.

Each parameter of the state can be set independently from the other ones, but all parameters should be explicitly set before any other command is sent to the device: after turning-on, the ring cannot be triggered or turned-on if not all parameters have been correctly set at least once. During operation, the state parameters can be updated at any time, also during a flashing phase. When lights are active after receiving the TRIGGER command, the ongoing operation state is not updated; on the contrary when they are activated after receiving the ON command, the lighting configuration should be updated in real time. Moreover, while flashing, subsequent commands of TRIGGER or ON type are ignored. A qualitative state diagram of the device is shown in figure 7. To drive the light sources, at least according to a possible embodiment, a rechargeable lithium battery was included into the device. However, also embodiments without battery are to be provided. In these cases, the power supply is obtained directly from the mobile device thereto it is connected. When the device is idle, the battery assumes a passive role and gets charged by the smartphone through the USB connection. This occurs in background and the charging current is limited by the device itself, to minimize the power resources extracted from the mobile device.

Advantageously, the lighting device could be also equipped with a RFID reader, configured to be able to be interfaced with, and used by the mobile client device. This is particularly useful when, as mobile device, one wanted to use a smartphone not having a RFID sensor. In these cases, through the lighting device, the smartphone could however acquire reading of RFID tags to implement the method according to the invention.

Automatic detection of identity documents

Prior to data acquisition, it is necessary to verify if an identity document is really present or not in a determined image frame (acquired by video or photo). Within computer vision, the identity document detection corresponds to a particular case of the problem of object detection. With respect to the general problem, there are several additional constraints that however simplify the solution to the problem (for example, planar surface, regular patterns and layouts, standard IR and/or UV responses, etc). All these constraints provide specific indications that can be exploited to process a solution. However, to the purpose of the present invention, it is possible to limit the analysis to the subset of visual and optical indications of photometric origin (for example, the document planarity is a piece of data of geometrical origin which can be analysed visually).

Therefore, one of the problems faced and solved by the present invention can be stated as: given a multi-spectral image, in particular a live video frame (or a photo) • To determine if there is an identity document inside thereof (detection problem);

• To identify the document type (classification problem);

• To extract the identity document from the image (rectification) and to identify and extract all attributes or features of interest (problem of extracting data and attributes contained therein).

Based upon the proposed architecture, the present invention also provides to measure the acquisition quality, data to guarantee an optimum extraction of the attributes contained therein, and to reduce to the minimum the computational resources requested to the client devices.

Hereinafter within the present invention one will clarify what is meant under "all features of interest" that one wants to extract from the identity document. The analysis and validation of identity documents depend exclusively upon the regulations for manufacturing the document itself; such regulations usually define a long list of constraints relating all production aspects, from the selection of the materials to the design of the document layout. To the purpose of the present invention, although without this is considered to be limiting, the analysis can be limited to the identity document layout, to some unique elements (like patterns and logos, the type of used font, possible RFID tags and additional environmental data that may help the validation process. Therefore, the features of interest are related to:

1. Identity document's owner biographical data (full name, citizenship, date of birth, place of birth, gender, age, height, etc.) and photo, if present in the document;

2. Document self-data (number, country of issue, issue date, expiration date, etc.);

3. RFID tags if available, for example readable through NFC;

4. Bar codes and MRZ, if any; 5. Logos, patterns, textures, font types and layout of the identity document needed for document's validation.

6. Multi-spectral imaging of the identity document.

7. Live acquisition of the face of the document's holder (there is a clear distinction between the document's owner and holder, since the uniqueness of the identity between the two has to be validated).

8. Environmental setting (time, date and GPS coordinates).

The geolocalization of the process for extracting the features of interest (and then of the smartphone performing such process) both at time of capturing the identity documents and of live face capture, allows to verify that the geographical place in which the document has been photographed (both sides) corresponds to the place of the face capture of the presumed identity document's owner.

Putting a time-stamp (date and time) further allows to certify that the identification has occurred in that place (as already said both for the identity document capture and for the live face capture) and in that moment.

It is further possible to define reliability thresholds, and once exceeded them, the use of other biometrical procedure can be provided such as fingerprint on smartphone and digital graphometric signature, still preferably geolocalized and with time-stamp. Such additional procedures optionally can be provided even then for example the comparison with the live photo or with the photo already existing on DB has not exceeded the reliability threshold or other requirement defined in advance. Moreover, advantageously, the process according to the invention can even provide that in case the automatic recognition systems (of photo, of biometric data and/or of graphometric signature) do not provide an acceptable result in terms of reliability, or however by the operator's decision, the latter could perform a step of visual verification, with the purpose of confirming the automatic recognition, or discarding it in case of doubts. Optionally, the system itself can be designed so as to request to the person who has to be identified, to perform determined motions with his/her head and/or eyes during video acquisition. In this way, the unconditioned blinking reflex is stimulated, which through eyes' recognition in the image and pupils' tracking, can be used as parameter for identifying a real live capture. The request made by the system could also not be an explicit motion request, on the contrary for example some messages could be shown on the screen in different positions so as to force the eyes' motion and cause the reflex without asking it explicitly to the involved person.

As shown in figure 5, according to a possible embodiment of the invention, it is possible to divide the identity document acquisition and attribute extraction process into five specific tasks:

T2.1. Identity document detection. The goal of this activity is to extract and rectify the image area representing the document, while verifying and ensuring the quality of the image acquisition (determining each time if a reflective element covers an image area contained inside the document, or invalidating video frames captured while the device mobile is moving). This task is achieved through the combination of different computer vision and image processing algorithms based upon the estimation of the identity documents' contours.

With reference to figure 8, the goal of the upper transitory cycle is to determine (that is recognize) the identity document type, which will be used as reference type for the main cycle.

Then, in this cycle the video sequence is analysed to determine a reference image by enabling the identification of the examined identity document type.

Once the identity document type is known, the main processing cycle is performed, and the video sequence is analysed with the purpose of obtaining a high-quality and stable image of the identity document (that is a primary image). The identity document type contained inside such primary image has to be the same as the one existing inside the reference image. Should the primary image satisfy this and the rest of stability criteria, then the subsequent processing unit is activated for an additional analysis of such image.

Consequently, the reference image role is the (quick and robust) identification of the identity document type. While the primary image role is to provide a high-quality representation of the identity document for the reliable and accurate extraction of the attributes.

Figure 9 includes, by way of example, some detections of different Italian documents.

T2.2. Document type recognition. This task is devoted to the identification of the identity document class, by means of automatic learning techniques-based classification.

The possible "identity document types" are mostly defined by regulations; for example, a complete list can be found in PRADO database [Council of the European Union. Public register of authentic travel and identity documents online], iFADO [Council of the European Union. Intranet false and authentic documents online] or EdisonTD [Central Intelligence Division of The National Police of the Netherlands. Edison travel documents database].

In relation to the recognition and classification techniques, first of all it is to be said that, in general terms, the problem of recognizing an identity document by using image classification techniques represents a particular instance of the wider problem of image-based document classification. Such last activity is well documented in the scientific literature in the field, see for example the articles:

a) M. Diligenti, P. Frasconi, and M. Gori. Hidden tree markov models for document image classification. IEEE Transactions on Pattern Analysis Machine Intelligence (TPAMI), 25(4):519-523, 2003 b) G. Meng, N. Zheng, Y. Song, and Y. Zhang. Document images retrieval based on multiple features combination. In 9th International Conference on Document Analysis and Recognition (ICDAR), pages 143-147, 2007.

c) J. Kumar and D. S. Doermann. Unsupervised classification of structurally similar document images. In 12th International Conference on Document Analysis and Recognition (ICDAR), pages 1225-1229, 2013.

Then, obviously, any method providing a solution for such recognition task could exploit, combine or extend techniques already known for image- based document classification.

By only way of example, two approaches commonly known for image- based identity document classification can be synthetically shown hereinafter.

A first example is included in the work:

M. Simon, E. Rodner, and J. Denzler.

Fine-grained classification of identity document types with only one example.

In 14th IAPR International Conference on Machine Vision

Applications (MVA), pages 126-129, 2015.

According to these authors, aligned (scanned) images of identity documents are analysed to determine the issue country, the document type and its version. To this purpose a carrier of features is produced, given by the connection of different standard features, by coding both photometric information and layout (that is spatial information). Therefore, different combinations of features are evaluated and, with a pure empiric approach, the most promising configuration is identified to classify its own set of data, constituted by 74 categories of identity and travel documents coming from 35 countries, by using a Support-Vector-Machine (SVM) with linear kernel and one-vs-all strategy for a multi-class classification.

A second example can be found in the work:

L. de las Heras, O. Ramos Terrades, J. Llados, D. Fernandez-Mota, and C. Cahero.

Use case visual Bag-of-Words techniques for camera-based identity document classification.

In 13th International Conference on Document Analysis and

Recognition (ICDAR), pages 721-725, 2015.

These authors have proposed a method for classifying identity document images captured with mobile devices. The classification is performed by means of a SVM trained with an image histogram representation. Such histogram representations are computed in terms of Bags-of-Visual-Words (BOVW) model, defined by a codebook of 1000 words. Their set of data is constituted by more than 3500 images from 129 different classes of identity documents.

Both such methods are based upon standard features, classifiers and experimental strategies used in the general context of the image-based classification.

T2.3. Textual attributes extraction. In this task all textual attributes (biographical and/or identity document self-data) are extracted, including MRZ areas if any, through the implementation of an optical character recognition (OCR) technology.

T2.4. Validation attributes extraction. The present activity is related to the extraction of specific items related to the document's manufacture, that can be used to analyse (by automatic or assisted means) the validity of the document. Also, this task is responsible for collecting all environmental data which can be measured by means of the mobile device, capturing a multi-spectral image of the identity document, and extracting any electronic attribute available by NFC. This task implements different artificial vision (computer-vision) algorithms, image processing and automatic learning for the detection and recognition of the different visual and optical attributes distinctive of the identity document.

T2.5. Face detection, recognition and validation. The objective of this task is to validate the identity of the document's owner and holder through automatic face recognition algorithms, by using a live video capture of the document's holder face.

Each goal or task can be wholly or partially implemented on the mobile device or on the remote server.

The identity document detection (task T2.1) and the recognition of the document type (task T2.2) are two highly-related and complementary tasks, in the sense that the results of each one depend upon the results of the other one. This is the reason why the first design choice is to determine whenever the presence of an identity document has to be determined prior the contour detection or vice- versa. Generally, the document detection is a particular instance of the object detection problem. Classical approaches for identifying objects inside an image are based upon maps with pixel-wise annotations (like bounding boxes or any other contour shape) for direct training of some learning model, capable of providing for each image pixel a measure or probability of being part of the target object. The main drawback of this approach is that this kind of label maps are expensive to be collected. However, in our case, the implementation of such label maps under controlled experimental conditions can be easily performed by the automatic identification of the document's contours on video data. In particular, strong geometrical cues (rectangular shape, aspect ratio, etc.) are exploited to generate reliable document proposals based on edges. As a consequence, the design choice of the present invention is to implement an object (identity document) proposal generator based upon edges calculated in the input images. Then, any identity document proposal will be validated through a pre- trained automatic learning model capable to determine if the proposed image region includes or does not include an identity document. The goal of task T2.2 is precisely to train this automatic learning model for identity document recognition.

Having identified the identity document's class, it is then possible to define specific contour detection strategies for a more accurate document analysis. For example, an Italian identity card requires the extraction of two adjacent contour regions instead of one, like the rest of identity documents. Then, instead of performing a blind search over the image to determine the identity document's contours, the interaction with the user (real-time preview of the video frame) is exploited to provide a feedback to the user based upon a simple identity document template overlay, therefore the contour search can be reduced to specific image areas based upon simple binary masking.

When the document's contours are found, a high-resolution photo is taken to proceed with the complete processing procedure and then, in case of successful identity document detection, identification and extraction, provide the image data to tasks T2.3, T2.4 and T2.5, as shown in figure 5. Then, practically:

• on the video sequence a document tracking and the identification of its typology are performed.

• when the document is recognized, a photo thereof is taken;

• when the photo stability (smartphone not moved during capture and focusing good quality) and the presence of the document therein are verified, one proceeds with the subsequent document analysis.

A complete block diagram of T2.1 , summarizing the above-described procedures, is shown in figure 8. This task has been achieved through the following procedure:

1. Video and photo acquisition. Such procedures are evidently device- dependent and rely in the API available for the hardware interfacing of the mobile device. In general, the image acquisition should be handled through a callback mechanism, to have the CPU available for any other computation while waiting for the next image or video frame. It is assumed that these blocks are responsible for the video/image frame downsizing to improve computation performances of the other processing blocks (the modern mobile devices provide very high-resolution images, however such data level is not strictly necessary for the processing procedure according to the present invention) and, in case of photo acquisition, to keep a temporary copy of the full resolution image for a possible subsequent document attribute extraction.

Video and photo video acquisition stability tests. Although there are robust approaches for image stability analysis (optical flow, point spread function estimation, etc.) these approaches are computationally demanding and require GPGPU acceleration. A better alternative is to exploit the inertial measurement units present on all modern mobile devices, to obtain an accurate sensor motion estimation during capture. This method is fast, accurate and completely depends on mobile hardware and the API available for interfacing it.

Contour detection. Apparent object contours are generated from a topological connection of image's edges, that is continuous locus of pixels with sharp brightness variations. The adjective "apparent" is used since the edges extracted from a two-dimensional image of a three-dimensional scene can be either viewpoint dependent (occlusions) or viewpoint independent (surfaces, textures and shapes, not related to the scene geometry). Therefore, accurate contour estimation depends upon the ability to discriminate between both types of edges, which is impossible under general conditions. In reality, it is preferred to define precise experimental conditions and a set of assumptions to yield a reliable edge discrimination. Hereinafter it is assumed that:

(a) only one identity document is present on each image frame.

(b) there are no other rectangular objects inside the image. (c) background has smooth brightness variations.

(d) background colour is perceptively different from one of the identity document.

(e) At most one corner of the document's boundary is occluded.

In general, any set of pixels with brightness discontinuities can be identified based upon the image gradient. However, as a linear operator, the image gradient is a high-pass filter and then highly susceptible to noise. As a consequence, any edge computation should be performed after some noise filtering procedure.

Figure 10 shows a possible complete contour detection process. It is to be meant that such description is provided by way of example, but not as limiting example. According to the present invention in fact different methodologies and algorithms could be adopted. According to such implementation the process comprises five processing modules, that is:

3.1. Noise filtering. Standard filtering on image processing corresponds to Gaussian smoothing. However linear noise filtering does not keep the edges contained in images, by reducing the effectiveness of algorithms detecting the same. To overcome this limitation a non-linear Meanshift filter was tried. Meanshift is an iterative data clustering algorithm that assigns to each image's pixel the mean value of a set of adjacent pixels, defined by a metric distance based upon spatial and chromatic bounds. In Figure 11 (a) there is an example of possible Gaussian filtering with respect to Meanshift. In general, in terms of accuracy, a (linear) Gaussian filter could never perform filtering better than (non-linear) Meanshift one.

There are two potential limitations of this approach in computational terms: computational complexity and iterative nature of algorithm. The first limitation can be mitigated with a parallel implementation of the algorithm because computations are pixel-wise independent; of course, possible target devices are reduced to those having GPGPU hardware and adequate software interfaces to exploit such capabilities. The second limitation is less restrictive since for a stable video acquisition, the mean values can be computed in one iteration by frame, without degrading the quality of estimates, requiring only a negligible increment in the software implementation complexity.

According to the present invention a particular type of noise source is considered, given by the specular reflections produced by the wrong placement of the identity document with respect to the light source and capture device. The implementation according to the invention of the non-linear filter takes into account the identification of image regions that may be damaged by such visual artefact, allowing the rejection of candidate boundaries that contain such compromised regions. Figure 11 (b) shows an image example with a very complex and large reflection spot, detected inside the document boundaries. It is important to note that, around the edges of the reflection spot, the gradient computation is not performed by the implemented algorithms of the present invention.

3.2. Edge detection. Canny algorithm is the most widely known and used technique for edge computation. Edges area computed based upon image gradient intensity and orientation, by comparing the relative strength and directional coherence between neighbouring pixels. This method is reliable, fast and requires few parameters setting. Unfortunately, this algorithm is limited to images with one single channel with light intensity, condition that may be too restrictive for applications of interest in this case. For the edge detection, apart from the assumptions (a)-(e) described previously it is assumed that:

(f) The edge detection should be independent from light conditions;

(g) The edges should not be detected in presence of images with poor chromatic content (for example, grayscale photocopies).

The first additional assumption is introduced to reduce the complexity of the image processing task and to increase the method robustness. On the contrary, the second one to completely discard identity documents photocopied in grey tones. In the present invention both problems are solved by analysing the chromatic components of different colour spaces in which the image frame is transformed. A multi-channel probabilistic version of Canny algorithm is proposed, that requires two different input images: one single channel bearing luminance values and one multi-channel image bearing chromatic information. The second image is used to augment the pixel's neighbourhood during the gradient computation, by allowing a robust rejection of incoherent or noisy pixels from the luminance image. As final step, a probabilistic test for line patterns is performed by using a region growing approach driven by the gradient orientation in the nearby pixel, to exclude curvilinear edges prior to the contour computation. Figure 12 shows different results which justify our procedure choice. It is to be noted that the luminance component L of space of LUV colours is completely full of noise, but it allows more accurate estimation of the document edges. Besides, the chromatic components are almost noise-free, but one single channel does not provide a complete boundary description. Moreover, considering the mean value among all chromatic channels, the luminance channel noise can be wholly rejected, but the end outcome results to be strongly degraded. At last, by fusing edges, no improvement in the result is obtained.

. Contour computation. To this purpose a standard implementation of the topological algorithm of Suzuki e Abe is used. According to a possible embodiment of the present invention, only the subset of external contours is considered. Moreover, candidate contours covering small or degenerated (shapes whose ration between the area and the perimeter length is outside predefined bounds) image shapes are discarded for further computations.

. Boundary line fitting. Once contours have been computed, it is to be determined whenever a contour belongs or does not belong to the document's boundary. The idea is to determine all linear contour segments and, then, to identify the subset of lines defining the document's shape. By definition, a contour is constituted by a series of unordered points, describing a geometrical locus inside the image, together with the hierarchy description based upon some of its topological relations with respect to the other contours. For geometrical computations, however, it is worthwhile to work with ordered sets of points. For this reason, in the present invention, instead of processing raw contour points, the convex envelope (the smallest convex set covering the contour shape) of each contour is considered to identify all linear segments within the set of computed contours; it is important to note that the convex envelope contains only few points of the original contour, by reducing considerably the processing time.

The first step is to pre-process the convex envelope shape by filtering the points that are very close to its immediate neighbours, this is particularly useful for curvilinear segments, where a high density of points is necessary to define the covering envelope shape. After that, each envelope segment defined between two subsequent vertexes is considered as candidate line. Subsequent candidate lines are then merged into one single line based upon the following heuristic rule:

• If the inner angle between subsequent candidate lines is less than some predefined threshold and the angular variance between all inner segments is below a certain predefined threshold, then the candidate lines are merged into one single line.

• Otherwise, one line is stored if and only if its length is greater than a predefined lower bound.

In the last step, each line identified based upon the convex envelope, is interpolated based upon the original contour points; lines with a small number of supporting inliers are rejected. If at least four lines are found, one proceeds with next processing unit. In figure 13 there is a dummy example of the line fitting procedure. 3.5. Corner computation. This is a standard and quite easy procedure, consisting only of the computation of the intersection point between two subsequent candidate lines obtained in the previous step. In homogeneous coordinates, given two candidate lines 11 and 12, the intersection point can be computed by the cross product: p 12 = 11 ^χ 12. The intersection points outside the image contour bounds (or in case of masked contour detection, the bounds given by the mask template) are rejected. If at the end of the procedure only four points are obtained, the document has been detected, otherwise the detection fails.

Document type identification. As mentioned previously, the actual block does not represent a specific procedure of task T2.1 , but of task 2.2. The document type identification is implemented through a supervised automatic learning technique. In the present invention the document recognition is made both before document image rectification and after it, to provide an accurate and reliable estimation of the document type, in fact, in case both estimations differ the image frame is discarded and the detection process is started again.

User interaction. During the contour detection step, when the document type has been detected, a translucent shape containing the detected document type layout is overlaid on the video frame. This allows to improve the document detection, through the user feedback, since he/she tries to align the identity document image with such shape. It is to be noted that even after the document type identification, the search for the document contours limits to the proximity of edges of the region limited by the translucent shape, to reduce to the minimum the required computational efforts. In figure 8 it is observed that the contour detection implemented on the whole image is performed only during the transitory phase required for the identity document type recognition (cycle on top left in the diagram in figure).

As clearly illustrated in figure 8, the recognition mechanism is defined by the processing cycle higher in figure, which is active until reaching a successful recognition. This involves that the identity document type recognition is performed if and only if an identity document proposal has been detected within the image.

Moreover, when a successful recognition is obtained, the processing procedure flow goes from the processing cycle on top in figure (defined as transitory loop) to the lower cycle of the processing procedure (as shown in figure 8). In the lower processing cycle, the contour detection is limited to a limited (or masked) region surrounding the edges of the document template, then the identity document type identification can be performed successfully only, and uniquely after detection. Mobile device identification

Advantageously, according to the present invention a step for identifying the used mobile device can be further provided, so as to guarantee that the device is used by the same user who has activated the identification service. To this purpose, with reference to figures 14 and 15, when the application residing on the mobile device (App) is started for the first time the Server service stores the univocal code which identifies the smartphone (UUID/UDID) and associates it to the user account (for example characterized by username/password). Subsequently, in order to be authorized to use the services according to the present invention, it will be necessary to perform a two-step login procedure. In particular, as shown in figure 14, the user will enter his/her credentials (username/password) by means of the App on the mobile device and these, together with the UUID/UDID code of the smartphone will be sent to the Server. The Server will verify the validity thereof and, if the credentials are valid, the user will receive a temporary password (OTP) via SMS.

Then, as illustrated in figure 15, the user, still by means of the App, could re-forward to the server the OTP received via SMS together with the univocal UUID/UDID code of the mobile device in use in that moment. The server then could verify both the OTP validity and the correspondence of the univocal UUID/UDID code; if such server validation procedure is successful it will return a token in form of GUID the expiration thereof could be renewed automatically for a limited period of time within a working session (typically lasting 15 minutes).

The present invention has been sofar described with reference to preferred embodiments thereof. It is to be meant that each one of the technical solutions implemented in the preferred embodiments, herein described by way of example, can be advantageously combined, differently from what described, with the other ones, to create additional embodiments, belonging to the same inventive core and all however within the protective scope of the herebelow reported claims.

Claims

1. A process for documents and/or personal identities recognition and validation, comprising the steps of:

- acquiring, by a mobile device, a video sequence catching a document to be recognized and validated;

- processing each single frame of said video sequence for extracting a reference image of the document to be recognized;

- classifying said document to be recognized on the basis of said reference image;

- acquiring a digital primary image corresponding to said reference image;

- extracting from said primary image one or more predetermined attributes of the document;

- associating to said attributes a time-stamp related to the capture time and a set of geographical coordinates related to the place of capture;

- validating said document on the basis of said classification and said extracted attributes,

wherein said one or more attributes is selected among:

• document data, such as: number, country of issue, issue date, expiration date;

• data stored on RFID tags;

• data reported as barcodes and/or MRZ zones;

• data related to logos, patterns, textures, font types and/or document layout,

and wherein said video sequence is acquired by lighting the document by a lighting device according to one of claims 13 to 17

and said processing of the video sequence frames is performed by multi-spectral techniques.

2. The process according to claim 1 , further comprising a phase of recognition of a face reproduced in said reference image.

3. The process according to the preceding claim, wherein said recognition phase comprises a step of comparing said face with face images stored in a database.

4. The process according to claim 2 or 3, wherein said recognition phase comprises a step of acquiring an image of the face of the document holder, so as to compare it with the face image reproduced on the document.

5. The process according to the preceding claim, wherein the image of the face of the document holder is extracted from a video sequence recorded while the holder is induced to perform predetermined face movements.

6. The process according to anyone of claims 2 to 5, further comprising a step of acquiring and verifying biometric data and/or a graphometric signature, when said face recognition results to be performed with a reliability index diverging from a predefined threshold.

7. The process according to the preceding claim, wherein said step of acquiring biometric data comprises acquiring a fingerprint.

8. The process according to claim 6 or 7, wherein said step of acquiring comprises associating the acquired data to a timestamp related to the time of capture and a set of geographical coordinates related to the place of capture.

9. The process according to anyone of the preceding claims, further comprising an identification phase of the user and of the used mobile device, comprising a two-step authentication by an OTP temporary password.

10. The process according to the preceding claim, wherein said device is a smartphone and said two-step authentication comprises using of user credentials and the verification of a univocal code identifying the smartphone.

11. A system for documents and/or personal identities recognition and validation, comprising:

- a mobile client device comprising:

o means for acquiring data;

o first means for processing acquired data;

o data transreceiving means; and

o user interface means,

- a remote server apparatus comprising:

o data transreceiving means; and

o second means for processing data received from said mobile client device,

wherein said first and second processing means are programmed so as to implement a process according to anyone of claims 1 to 10.

12. The system according to claim 14, further comprising a lighting device according to anyone of claims 13 to 18.

13. A multi-spectral lighting device, comprising:

a plurality of light sources;

a first support for said light sources;

a main body housing:

o interface and connection means with external apparatuses;

o power and driving means of said light sources; and

o programmable control means for managing said interface and connection means and said power and driving means, wherein said light sources comprise: • one or more white LED emitting in the visible;

• one or more UV LED emitting in the ultra-violet;

• one or more IR LED emitting in the infra-red,

and further comprising a RFID reader, configured to be interfaced with, and used by, an external apparatus.

14. The device according to the preceding claim, wherein said first support is shaped as a ring.

15. The device according to claims 13 and 14, wherein the white LEDs, the UV LEDs and/or the IR LEDs are alternate on said first ring support.

16. The device according to anyone of claims 13 to 15, wherein said interface and connection means are of USB type.

17. The device according to anyone of claims 13 to 16, wherein said programmable control means is configured so as to control said power and driving means for allowing a selective on/off switching of said light sources.