WO2024025502A1

WO2024025502A1 - Mobile automated fingerprint identification system and method

Info

Publication number: WO2024025502A1
Application number: PCT/TR2023/050724
Authority: WO
Inventors: Seref SAGIROGLU; Bilgehan ARSLAN
Original assignee: Sagiroglu Seref; Arslan Bilgehan
Priority date: 2022-07-28
Filing date: 2023-07-24
Publication date: 2024-02-01

Abstract

The invention relates to the method and system that performs biometric identification/verification using the finger images it collected contactless by means of the new data acquisition method it incorporates. In particular, the invention relates to a method and system that collects fingerprint images using a new contactless data acquisition method, enables the finger images it collects to be used for biometric identification/verification with the new data acquisition method it offers, can convert the collected finger images into fingerprints using enhancement methods, extract characteristic features for biometric identification/verification from the enhanced fingerprint image, compares the template consisting of the characteristic features of the fingerprint with the templates of other fingerprint samples registered in the database, and identifies potential matching candidate(s) as a result of the comparison.

Description

MOBILE AUTOMATED FINGERPRINT IDENTIFICATION SYSTEM AND METHOD

Technical field of the invention

The invention relates to a mobile automated fingerprint identification/verification system that enables the development and collection of finger images with contactless method, enhancement of finger images to be converted into fingerprints, extraction of biometric characteristics from the enhanced fingerprint, use of this collected fingerprint for identification or different purposes, and management of all steps from a single centre, and to the method thereof.

In particular, the invention relates to a fingerprint-based portable biometric identification/verification method and system that incorporates a new contactless fingerprint acquisition method, develops and collects fingerprint images with this method, enhances the fingerprint images it collects, extracts the characteristic features from the enhanced image, compares these features with the features of the fingerprint samples registered in databases, and matches them.

State of the Art

The most common procedures for detecting a fingerprint characteristic pattern require the finger to be placed or rolled over the scanner, reader, sensor, etc. device plate or pressing or rolling the finger surface onto a surface such as paper card, film, etc., by painting it with a dyestuff such as ink. The process of contacting the finger surface painted with a dyestuff on a paper card and transferring the print on this card to digital media with the help of a device such as a scanner is called offline fingerprint acquisition or offline sensing. Today, most civilian applications and law enforcement databases accept live scan digital images obtained by directly sensing the finger surface with an electronic fingerprint scanner. This method is called online sensing. In the online sensing process, no ink or any colorant needs to be applied to the finger surface. All that needs to be done is to bring the finger into contact with a data capture device such as a sensor, scanner, etc. to ensure that the fingerprint is transferred to the digital environment. In general, approaches used to detect and record the characteristic pattern on the finger surface work using the contact-based data collection method. Contact-based means that an individual has to press or roll their finger on a shooting (filming) surface, and these pressure-requiring actions cause deformation caused by the deterioration of the elastic structure of the fingerprint. In order to prevent this deformation caused by pressure, new methods have been developed in which fingerprints can be collected contactless.

In order for the fingerprint collected using any data acquisition method to be usable in biometric identification/verification, the fingerprint must be converted into a suitable format for comparison and matching. This conversion process is referred to as image preprocessing. Image preprocessing consists of a series of interventions in which many methods are parametrically adjusted according to the specifications of the image to be processed, and more than one method is applied to the image in appropriate order. Image processing steps that should be applied to fingerprint is shaped according to which data acquisition method is used and the structure of the collected data. For this reason, when a new data acquisition method is recommended, the effective usability of the collected data in biometric identification/verification should also be evaluated.

The fingerprint, for which the preprocessing steps are completed, is now ready to extract the characteristic features that can distinguish the individual. These characteristic features are defined as minutiae points or Level-2 features. In addition, in some specific cases arising from external factors such as injury, disease, or in cases arising from some innate features, characteristic data that cannot be classified as minutiae points, but that are distinctive, are also observed in the fingerprint. Biometric identification/verification is performed by using these minutiae points and characteristics that are not classified as minutiae points but have distinctive features. For this reason, feature extraction is performed using algorithms for extracting these features from the developed and enhanced fingerprint template.

After extracting the feature from the developed and enhanced fingerprint template, these features should be compared with other sample templates that are potentially to match. If the system in which the fingerprint is used is designed only for verification, the sample registered by the system and the newly collected sample are compared. However, if the system is designed for identification, a specific sample group registered by the system and the newly collected sample are compared. If the similarity rate measured as a result of the comparison made in both identification and verification exceeds the threshold level set for the biometric identification/verification system, these samples are labelled as possible similar templates.

In the state of the art, a biometric identification/verification system in which fingerprint is used as a distinguishing element is designed to consist of basically five components that are capable of processing data. These are data acquisition, feature extraction, matching, data storage and decision making. Fingerprints are collected by taking into account the fulfilment of certain requirements related to distinctiveness within the framework of certain standards and specifications, in order for the identification/verification system to work properly and correctly over the five components specified. Because the content of the collected biometric fingerprint and the amount of data it contains can directly affect the performance of the identification/verification system, and for this reason, the data acquisition component is the most critical component. The quality of the biometric fingerprint data coming from the data acquisition component directly affects the working performance of the other components. Since the data collected in the data acquisition component will vary according to the technology used, other components are designed to extract meaningful and usable information from this data.

The discovery of fingerprint acquisition approaches and the completion of the development of these methods and their use with their current versions has been possible with the cooperation of different disciplines and the adaptation of technological developments to existing methods. Academic studies on fingerprint acquisition and commercial software/hardware developed are carried out to complement or improve the deficiencies of the techniques and methods already developed, but many different problems arising from the application of the methods, both contact-based and contactless methods still used in fingerprint acquisition, could not be overcome.

In the contact-based fingerprint acquisition application, which is applied by placing the finger of the individual on the image sensing plate or surface of devices such as a scanner, reader or rolling on this plate/surface, or pressing or rolling the finger on a paper surface after a colouring (paint) substance is applied to the finger surface; problems arising from (i) data loss and/or erroneous data formation in the fingerprint image due to the deformation in the elastic structure of the finger caused by the pressure formed as a result of pressing the finger against the plate/surface in order to capture the ridge-valley pattern on the finger surface, (ii) variance in the number and type of final characteristics in two different prints of the same fingerprint and/or variance in the false characteristics obtained from noise due to the fact that the parameters such as the position of the finger/the pressure applied/the noise rate from the device, etc. are not exactly the same although the same data acquisition process is used, (iii) deformations caused by excessive dry/ moist/ wet finger surface, (iv) skin deformations caused by cuts/wounds on the finger surface, etc., (v) the residual fingerprints of the previous individual on the device used for data acquisition are mixed with the fingerprint of the next individual, (vi) noise from the data acquisition device, (vii) performance degradation and durability loss of the data acquisition device under heavy use, and (viii) location problems, such as improper finger placement on the data acquisition device are encountered.

In contactless fingerprint acquisition method, which can collect fingerprints without making any contact by using special data acquisition mechanisms and customized image capture devices in order to eliminate the problems existing in contact-based fingerprint acquisition methods, (i) problems arising from data acquisition mechanisms used for contactless acquisition, (ii) problems arising from the environment in which data is collected in the contactless acquisition process, (iii) problems arising from the necessity of processing steps that require too many guiding/auxiliary instructions for the data acquisition process, (iv) problems arising from the quality and adequacy of the collected fingerprint image, (v) problems arising from the necessity of using complex software and high-cost hardware in the data acquisition process and (vi) the complexity of the fingerprint image acquisition process and implementation difficulties in the methods used for contactless acquisition are encountered.

The patent file numbered "TR201506503” and titled "Contact-based, Contactless, Biometric, Identity Access Device Containing All Kinds of Electronic Payment Functions" in the state of the art has been examined. In the abstract of the invention that is the subject of the application, the following information is given: “This invention relates to Contact-based, Contactless, Biometric, Identity Access Device Containing All Kinds of Electronic Payment Functions, comprising upper cabinet, display bezel, colour touch screen, contactless reader, touch screen capacitive connector, touch screen light connector, SAM card slot, chip card reader area for serving, fingerprint and finger vein reader frame, USB port of palm (palm) vein reader, fingerprint and finger vein reader module, HDMI connection cable connector, external power supply, Ethernet interface, plastic cover of SAM card slot, bottom cabinet, USB type B, mini USB, security port on main board, security port on main board, fingerprint and finger vein reader device connection connector, main board bottom security cover, contactless antenna connector, PCB firewall, chip card reader area for service area, lock cover of SAM card slot, main board, keyboard illuminated protective area, functional password/PIN keys, LCD connector, push button battery, micro HDMI, battery, SIM card slot 1 , SIM card slot, charging module and SAM card slot.”

The patent file numbered "TR201513220" and titled "Biometric Authentication Device with Eft-Pos Feature” in the state of the art has been examined. In the abstract of the invention that is the subject of the application, the following information is given: “This invention relates to a Payment Device capable of Biometric Authentication; comprising upper body, keyboard frame, screen protection frame, touch screen film, LCD screen, LCD screen holder, LCD screen connector, touch film connector, main board, smart card reader for customer, smart card reader for service, button battery, protective mesh wall, contactless antenna connector, tamper, tamper, lower body, SIM/SAM cover, fingerprint reader, USB connector, fingerprint reader connector, contactless antenna, sandbox PCB security cover, contactless antenna connector, SIM reader, SAM reader, keyboard, keyboard holder, fingerprint reader faceplate, mini HDMI connector, battery, dock connector, and hidden SAM”

Patent document no “EP2645302” and titled “Noncontact biometric identification device” in the state of the art is reviewed. In the invention that is the subject of the application, a sensor that detects the movement of the living body part in the reader area and collects the recording data, a processor that organizes a certain number of data in a comparison range, a contactless biometric identification system that selects the data as recording data, which will operate on the comparison range, not overlapping and have the maximum radius, and the working method thereof is mentioned.

Patent document no “SE1950970” and titled “Single-feature fingerprint recognition” in the state of the art is reviewed. The invention, which is the subject of the application, relates to the fingerprint identification system and method. With the touch of the finger on the sensor screen, it creates and records fingerprint data by means of an identifier that recognizes the fingerprint.

In the state of the art, it is seen that in the existing fingerprint acquisition methods, not all of the internal problems arising from the application of the methods can be overcome. Especially in scanning, touching and swiping processes used in contactbased fingerprint collection methods, the print on the finger surface should be taken to the scanning screen, reader or paper and examined. At this stage, the amount and direction of the pressure applied for the formation of the fingerprint causes the pattern formed by the ridge lines to not be taken properly due to the deterioration in the elastic structure of the finger surface. In this context, losses and errors occur in the fingerprint data received. In contactless fingerprint collection methods, in addition to the problems arising from the data collection mechanisms used and the environment where the fingerprint is collected, problems such as the cost of software and hardware used in contactless fingerprint collection and processing of the collected data, the complexity of the application instructions in the collection process affect the quality, adequacy and collectability of the collected fingerprint data.

The aim of the invention

The most important aim of the invention is to offer a fingerprint biometrics-based portable identification/verification system that can provide the same service as employee identification/verification systems using fingerprints collected using contactbased data acquisition methods and incorporates a new contactless fingerprint acquisition method that will eliminate the disadvantages of contact-based acquisition methods. The invention consists of a device developed to collect fingerprints from the individual with the contactless method, methods and a system that extracts fingerprints from the collected finger images, develops, enhances, compares and matches these fingerprints. Another object of the invention is to offer a new method of contactless fingerprint sensing and development alternative to contact-based methods in order not to be affected by all kinds of deformation problems caused by contact (deformation caused by pressure, deformation caused by finger surface and skin problems, deformation caused by the use of data acquisition device) due to the nature of contact-based fingerprint acquisition methods.

Another object of the invention is to standardize the environmental conditions where fingerprints are collected, which are the points to be considered during the data collection phase in contactless fingerprint acquisition methods, to arrange and design the equipment/device/hardware used and all kinds of data acquisition components to meet the requirements of the collected fingerprint image, to be able to collect fingerprints that have sufficient criteria for biometric identification/verification regardless of environmental conditions, to present a new data acquisition mechanism developed to control the factors affecting the data acquisition process and to standardize all the steps performed.

Another aim of the invention is to provide a new low-cost, compact, portable and contactless fingerprint data collection method that does not require expertise to use, as an alternative to high-cost, non-portable and special-installation data acquisition mechanisms of contactless data acquisition methods.

One of the aims of the invention is to present a new data acquisition method that can be used practically in scenarios where only certain parts of a cadaver or a corpse are present and/or data acquisition methods that require the state of being alive (requiring the use of pressure, temperature, etc.) cannot be used.

Another aim of the invention is to provide a biometric identification/verification system that ensures that all components that make up the biometric system are coordinated with each other, and that standardises the whole process, allowing the system to be managed from a single centre and be portable.

Contact-based data acquisition approaches are preferred in biometric identification/verification systems used in the state of the art, and biometric identification/verification systems are always negatively affected by the disadvantages of using contact-based approaches. Due to the inherent problems of contact-based acquisition methods, there is always some data loss or erroneous data collection in the fingerprints collected. By means of the invention, a method and device that does not contain the disadvantages of contact-based acquisition methods, can be used effectively in all biometric identification/verification systems, and where more and more accurate characteristic features can be extracted, are collected contactless.

Another aim of the invention is to control the data collection environment, which is an important element for contactless data acquisition approaches. Many academic studies on contactless approaches and prototypes developed for the implementation of these approaches use data acquisition mechanisms and imaging devices that are highly functional, costly, require professionalism to use, have complex working mechanisms, and are not portable, or noisy fingerprint data affected by the ambient conditions and collected with simple (ordinary, classical) imaging devices without using a specialized mechanism to collect data. In the invention, the most optimal setting of many factors such as the environment in which the finger is displayed, the background, the type and intensity of the light source used, the resolution of the display device used, the depth of field, focus, etc. provides the opportunity to obtain better quality fingerprint images. In addition, the data acquisition (collection) device and method presented within the scope of the developed invention have a much lower cost, easier use compared to the data acquisition mechanisms used in other contactless data acquisition approaches and can collect images with the same quality as the already developed devices with high functionality/complex/high costs. In addition, the data acquisition device and method in the invention are designed in such a way that it can obtain data of sufficient quality even when used with display devices with ordinary features.

Another aim of the invention is to provide a new fingerprint sensing and development method that can collect biometric characteristic data from non-viable body surfaces in cases where it is necessary to collect biometric data from a cadaver or a body part containing a finger surface, where fingerprint collection methods that are designed to work on living individuals and that use factors such as pressure, temperature, etc. while data acquisition are ineffective. The invention provides a fingerprint-based biometric identification/verification system with mobility that eliminates, by means of its mobility feature, the problem of running biometric identification/verification systems, which require fixed installation and includes specialized data acquisition and transfer components/methods, over certain centres/locations and that the identification/verification process is carried out only through these centres/locations.

Complex process steps that require high functionality and special equipment are not used in the development of both software and hardware components used in the design of the fingerprint-based biometric identification/verification system presented within the scope of the invention. Thanks to the use of a data acquisition mechanism that does not require a fixed infrastructure especially in the data acquisition component and the high quality of the collected data, a low-cost fingerprint data collection method and fingerprint-based biometric identification/verification system have been developed without the need to use complex development, enhancement, feature extraction, comparison and matching approaches.

Due to the fact that the fingerprint sample is recorded without the need for assistant personnel in the data acquisition process and the data collection/storage/transmission stages are carried out very quickly and practically due to the use of mobile devices in the data acquisition process, a more practical data collection model and an easy-to- use biometric identification/verification system and method are presented in the invention.

Another aim of the invention is to introduce a new contactless data acquisition method that will ensure that every component that makes up the invention is designed to be easily integrated into existing biometric-based identification/verification infrastructures and that the biometric fingerprint sample collected by the data acquisition component in the invention can work effectively in existing biometric identification/verification systems.

In the state of the art, the stages of collecting fingerprints, developing, improving, feature extraction, comparison and matching in criminal-victim-crime relations, in which criminal evaluations are made, are carried out in company with both technological equipment (computers, software tools, etc.) and field experts. At these stages, different field experts work to fulfil different tasks. In this context, another aim of the invention is to provide a system that manages the fingerprint collection, development, enhancement, comparison and matching steps, which are carried out by different field experts and using different equipment, from a single centre and will contribute to the standardisation of the working mechanism of all these steps.

Description of drawings:

FIGURE -1 is the drawing showing the device and data acquisition mechanism that is the subject of the invention, used to collect fingerprints.

FIGURE -2 is the drawing showing the operation flow of the fingerprint identification system that is the subject of the invention.

Reference numbers:

100. Mobile Device

110. Connector

120. Collection and Development Module

121. Realisation of contactless fingerprint acquisition process

122. Collection of candidate image

123. Evaluation of candidate image

124. ROI determination

125. Transmitting the ROI image determined from the candidate image to the enhancement module via the server

130. Enhancement Module

131. Editing the contrast of the image

132. Orientation estimation of ridge lines 133. Frequency estimation of ridge lines

134. Filtering

135. Binarization

136. Thinning

137. Evaluation of the image

138. Completing the enhancement process

140. Feature Extraction Module

141. Minutiae point extraction

142. Extracting false characteristics

143. Minutiae point map evaluation

144. Completion of the characteristic determination process

150. Matching Module

151. Evaluation of characteristic features

152. Comparison of the characteristic features with the characteristic features of the samples registered in the database

153. Calculation of match scores

154. Sensing of matches with scores exceeding the threshold value

155. Examination of similar matches

156. Transmission of matched samples to mobile device

160. Control Module

161. Evaluation of candidate image according to data collection criteria 162. Deciding (Judging) on the suitability of candidate images

163. Evaluation of the image according to data enhancement criteria

164. Deciding on the adequacy of the level of enhancement

165. Evaluation of the minutiae point map transmitted by the feature extraction module by the control module by looking at the feature extraction criteria defined in the control module.

166. Deciding on the adequacy of the minutiae point map

167. Manually checking for similar matches

200. Darkroom box

210. Darkroom box upper part

220. Darkroom box lower part

230. Lighting Ring

240. Macro lens

250. Power supply

Description of the invention

In the general sense, the invention comprises a system comprising modules (collection and development module (120), enhancement module (130), feature extraction module (140), matching module (150), and control module (160)), in which have methods for the developing, contactless collecting, enhancing, transferring of the fingerprint to the central server, comparing of the fingerprint with fingerprint databases and sending the matching samples to the user as a result of comparison, mobile device (100), the connector (110) connecting the mobile device (100) with the darkroom box (200), the macro lens (240) used to better capture the details in the fingerprint, darkroom box upper part (210) and darkroom box lower part (220) of the darkroom box (200), designed to collect fingerprints, and the lighting ring (230) used to illuminate the finger surface and the power supply (250) where the batteries used to power the LEDs in the dark room box are placed.

All parts are made of non-reflective Bakelite type hard material and their thickness is 5mm. There is a space with a radius of 2 cm positioned in the middle of the darkroom box upper part (210), and this annular space corresponds to the camera located on the rear surface of the mobile device (100). In cases where the resolution of the lens of the camera of the mobile device (100) is insufficient, the macro lens (240) is attached to this space located in the darkroom box upper part (210), thereby increasing the resolution. In the lighting ring (230), a total of 6 yellow illuminated LEDs are positioned at equal distances from each other, and the wavelength of the yellow LEDs used in the device is between 570-590 nm. By means of these yellow LEDs in the lighting ring (230), after making the ridge lines in the fingerprint more prominent, macro lenses (240) with +1 , +4, +8, +10 close-up feature were used to observe the details on the finger surface. These macro lenses (240) regulate the minimum focal distance of the lens of the mobile device (100) used to view the fingerprint, allowing it to get even closer to the fingerprint. Thus, the details of the fingerprint can be drilled down and more characteristic features can be captured. Rechargeable batteries in the power supply (250) are used for the LEDs inside the device to work. By means of the button on the power supply (250), the LEDs are turned on and off.

The imaging process of the fingerprint developed using the darkroom box (200) is performed with the mobile device (100), and the collected trace is transmitted to the server for comparison in the fingerprint database by the mobile device (100). Fingerprint samples that show similarity as a result of the comparison are transmitted to the user by the mobile device (100). For this reason, the mobile device (100) acts as a client in the client-server model. The server, on the other hand, provides the execution of the fingerprint identification/verification system and the methods in the system with its collection and development module (120), enhancement module (130), feature extraction module (140), matching module (150), and control module (160). The connector (110) is used to connect the mobile device (100) to the darkroom box (200). The darkroom box (200) consisting of the darkroom box upper part (210) and the darkroom box lower part (220) is used to better capture the details of the finger image by providing suitable environmental conditions, comprising lighting ring (230), macro lens (240) and power supply (250). The lighting ring (230) is used to provide sufficient illumination and to obtain the contrast value during the imaging of the finger surface. The macro lens (240) is used to collect the image of the enhanced fingerprint with higher resolution. The power supply (250) provides the energy necessary for the lighting ring (230) to operate.

There are conditions that must be met during the use of the device that collects and develops the fingerprint in a contactless way and the method involved in the enhancement, feature extraction and matching processes. Fingerprint imaging device is designed for viewing the fingerprint from above. For this reason, the finger surface and the device are positioned at an angle of 90 degrees. The method and system responsible for fingerprint enhancement, matching, feature extraction and control of all these works based on client-server architecture. The mobile device (100) provides the client task and can be connected to the server either wired or wirelessly. For this reason, if it is desired to get instant results in the place where the fingerprint is collected, the necessary communication infrastructure must be available for the mobile device (100) to access the server. The designed fingerprint image capture device can perform its task in conditions of temperature, humidity, pressure, etc., where the functions of the LED lamps and 9-volt battery in the mobile device (100) and the power supply (250) can work properly.

The invention is a system that works with client-server architecture and performs biometric identification/verification by using fingerprints collected by contactless method. In this system, there is a contactless data acquisition device specially developed to collect the fingerprint image. The invention consists of a device consisting of interlocking parts and developed to collect contactless fingerprints (Figure-1 ) and a method using five system modules developed for biometric identification/verification (Figure-2).

The fingerprint identification/verification method, which is the subject of the invention, comprises a collection and development module (120), enhancement module (130), feature extraction module (140), matching module (150) and control module (160), and the system created by these modules is run on a server. In general, this method and system comprises the process steps of: - Collecting the candidate fingerprint image (122) with the data acquisition mechanism in the mobile device (100) and the darkroom box (200),

- Transmitting the images collected by the mobile device (100) to the server,

- Transmitting the collected candidate image to the control module (160) by the collection and development module (120) for evaluation (123),

- Evaluation of the candidate image (161 ) that is transmitted by the collection and development module (120) by the control module (160) according to the data collection criteria defined in the control module (160),

- Judging the suitability of the collected candidate images (162) by the control module (160),

- Transmission of a command by the control module (160) to the collection and development module (120) for the next step, ROI determination (124) (Region of Interest to be used in feature extraction) if the candidate image is suitable,

- Transmission of a command by the control module (160) to the collection and development module (120) to perform the contactless fingerprint collection process (121 ) by running the data acquisition mechanism connected to the mobile device from the beginning if the candidate image is not suitable,

- Determining the ROI (124) from the fingerprint image of the collection and development module (120) with the positive feedback received from the control module (160),

- Transmitting (125) the determined ROI image over the server to the enhancement module (130),

- Editing (131 ) of the contrast value in the image transmitted from the collection and development module (120) to the enhancement module (130) by the enhancement module (130), then estimating the orientation of the ridge lines (132) and frequency of the ridge lines (133) in the image,

- Applying filtering (134), binarization (135) and thinning (136) processes to the contrast-adjusted fingerprint image by the enhancement module (130),

- Transmission of the image to the control module (160) by the enhancement module (130) for the evaluation (137) of the fingerprint image that has been enhanced and made ready for feature extraction,

- Transmission a command by the control module (160) to the enhancement module (130) to continue processing if the image meets the enhancement criteria and a decision is made about the adequacy of the enhancement level (164) as a result of the evaluation made in the control module (160),

- Transmission of a command by the control module (160) to the collection and development module (120) to perform the contactless fingerprint collection process (121 ) by re-running the data acquisition process mechanism connected to the mobile device if the image does not meet the enhancement criteria,

- Completion of the enhancement process (138) and transmission of the image to the feature extraction module (140) by the enhancement module (130) after receiving positive feedback from the control module (160),

- Extraction of the minutiae points (141 ) by the feature extraction module (140) from the image coming from the enhancement module (130) and then extraction of the false characteristics in these extracted minutiae points (142),

- Transmission of the minutiae point map by the feature extraction module (140) obtained from the fingerprint image to the control module (160) for evaluation (143),

- Evaluation (165) of the minutiae point map transmitted by the feature extraction module (140) by the control module (160) by looking at the feature extraction criteria defined in the control module (160),

- Transmission of a command to the feature extraction module (140) by the control module (160) to continue processing, if, as a result of the evaluation made in the control module (160), it is decided (166) that the minutiae point map is sufficient,

- Transmission of a command by the control module (160) to the collection and development module (120) to perform the contactless fingerprint acquisition process (121 ) by running the data acquisition mechanism connected to the mobile device from the beginning, in case of false/insufficient characteristics in the minutiae point map,

- Completion of the characteristic determination process (144) and transmission of the image to the matching module (150) by the feature extraction module (140) upon the result of the evaluation from the control module (160),

- Evaluation (151 ) of the characteristics transmitted from the feature extraction module (140) of the matching module (150) and then comparison of these characteristics with the characteristics of the samples registered in the database - Calculation of match scores (153) and detection of matches with scores exceeding the threshold value (154) upon the result of comparison,

- Transmission of the fingerprint matches (155) that exceed the threshold value to the control module (160) by the matching module (150) to perform a more detailed similarity examination,

- Transmission of the results the control module (160) has obtained to the matching module (150), after completing the process of manually checking (167) for similar matches, and

- Transmission of the matched samples to the mobile device (156) by the matching module (150) in line with the feedback received from the control module (160).

The details of the modules included in the method running on the server through which the fingerprint image taken by using the data acquisition mechanism in the mobile device (100) and the dark room box (200) is transmitted are as follows.

The collection and development module (120) has three tasks, and these tasks are defined as developing the finger image by providing suitable environmental conditions, digitizing and recording the developed image, and finally sending the collected image to the central servers.

Collection and development module (120) consists of the mobile device (100) used to capture, record and transmit the finger image to the server, the lighting ring (230) used to make the characteristic features on the finger surface and the details in the ridgevalley pattern more evident, the macro lens (240) and the darkroom box (200) used to assist the camera of the mobile device (100) to record more detailed the ridge-valley pattern on the finger surface and the external characteristics outside this pattern, and to detect more specific details. There is a mechanism designed to fix the finger position and angle in the darkroom box (200), which is specially designed to regulate the data acquisition environment conditions and to guide the individual during the fingerprint collection stage. By means of this mechanism, positioning errors that may occur during the data acquisition phase are prevented by directing the individual.

The device, which consists of a dark room box (200) and a data acquisition mechanism, works in an integrated manner with the mobile device (100). This device includes a lighting ring (230), a macro lens (240), and a curved chamber that acts as a guide for positioning the finger image. The finger image is recorded using the mobile device (100) and the device in the darkroom box (200) integrated into the mobile device (100). Afterwards, the image is sent to the control module (160) to evaluate whether it meets the desired criteria. If the image conforms to the specified standards, the next step is taken, otherwise the finger imaging process is repeated. If the collected image is deemed appropriate by the control module (160), the region of interest is cropped based on the core point in the image. Afterwards, the cropped image is transmitted to the server and transferred to the enhancement module (130).

The enhancement module (130) is developed to convert the finger image collected with the collection and development module (120) into a format usable by a biometric system. The tasks of this module are to increase image contrast, reduce noise, highlight ridge lines, combine broken ridge lines, fill gaps in ridge-valley pattern, increase finger image interpretability, prepare image for feature extraction and reduce data size.

The enhancement module (130) consists of steps containing methods used for segmentation, normalisation, and filtering. Fingerprint image is expressed by two different regions as background and foreground. The foreground is defined as the region containing the fingerprint ridge lines and important characteristics, and the background is defined as the region outside the boundaries of the image. Segmentation is a step applied to distinguish the foreground region from the background region. This step is applied to reduce the data size, shorten the image processing process, and perform accurate feature extraction. By means of the darkroom box (200), which is designed to prevent the data acquisition phase from being affected by environmental conditions, there is no problem of separating finger images from complex backgrounds. In this separation process, segmentation is easily performed by using the pixel density variable determined based on the background with the adaptive threshold value method. In the images obtained from the collection and development module (120), problems such as low contrast, gaps/disconnection in the ridge-valley pattern, noise arising from the data acquisition stage are eliminated in the normalisation stage. At this stage, the image is enhanced by changing the pixel values in the image without disturbing the structure of the ridge lines in the fingerprint. Normalisation of the collected finger images was performed using the contrast limited adaptive histogram equalisation method. In this way, the contrast value between the elevation and trough structures in the ridge-valley pattern has been increased as much as possible and the brightness level of the image has been enhanced so that this pattern can be observed better. In other words, a certain level of clarity has been given to the image. Finger images obtained from the collection and development module (120) also have deformations such as gaps/disconnection, defects caused by porosity and/or noise in the ridge-valley pattern that cannot be corrected by pixel enhancement. Spatial filtering method was used for the restoration of these deformations.

In the enhancement module (130), contrast values of the image are regulated so that the ridge-valley pattern on the finger surface can be highlighted. Then, the orientations and frequencies of the ridge lines, which are evident by the contrast editing, are calculated for later use in the filtering step. These defects are tried to be eliminated with spatial field filters by using the orientations and frequencies of the ridge lines in order to eliminate the gaps/disconnection in the ridge lines that make up the fingerprint, the deficiencies caused by the insufficient tonal difference in the ridge-valley pattern, etc. Afterwards, the ridge lines are subjected to the process of thinning and binarization, and a skeleton image of the ridge-valley pattern is obtained. The enhanced image is transmitted to the control module (160) to measure the usability of the enhanced finger image in a biometric system. If the control module (160) decides that the finger image has sufficiently enhanced, it transmits this decision to the enhancement module (130). Finally, the enhancement module (130) transmits the enhanced image to the feature extraction module (140).

The finger image collected with the mobile device (100) is now transformed into a fingerprint after the data enhancement step. At this stage, it is necessary to determine the distinguishing elements for the biometric system to be able to identify /verify from the fingerprint. The feature extraction module (140) has been developed to detect minutiae points that will be used to identify the individual in the enhanced fingerprint. The task of this module is to convert the developed and enhanced fingerprint into a suitable format for feature extraction (position and zoom adjustments), to extract the characteristic features to be used for biometric identification/verification, to determine the location/position of these characteristics and to determine their type, and to create a map of the minutiae points in the ridge-valley pattern.

The feature extraction module (140) is designed to detect biometric distinguishing elements from fingerprints collected and developed by the data acquisition mechanism, and enhanced by the enhancement module. The operation of the feature extraction module (140) consists of sequential steps. Minutiae point comparison was performed during the biometric identification/verification process. Although filtering processes are carried out in order not to remove false characteristics at the ends of the fingerprint, the place where the fingerprint ends can be labelled as the minutiae point. The presence of such false characteristics is checked in the feature extraction module (140) and cleared if any. Afterwards, other minutiae points are then looked into again. The fingerprint image with the minutiae points marked is transmitted to the control module (160) to evaluate whether there are errors caused by the failure of the image collection and development and/or enhancement processes. If the control module (160) decides that the minutiae points are correct and sufficient, it forwards this decision to the feature extraction module (140). Finally, the minutiae points verified by the control module (160) are forwarded by the feature extraction module (140) to the matching module (150).

The tasks of the matching module (150), in which the data obtained from the feature extraction module (140) come in the fourth stage are to compare the collected fingerprints with the registered samples in the database, calculate the similarity scores in the compared samples, list the matches with similar characteristics and transmit them to the control module (160), and finally to transmit the samples that are deemed to be matched by the control module (160) to the mobile device (100), which is the client.

The components of the matching module (150) are the matching methods used to compare the collected fingerprint and calculate similarity scores, and the fingerprint database to be compared. This database is the place where the samples to be compared are recorded. The minutiae point map from the feature extraction module (140) and the minutiae point maps of the registered samples in the database are compared with the matching methods. The distinctive characteristics of the fingerprint and the relative positions of these characteristics form a template. This template is essentially minutiae point map that summarises the fingerprint. When comparing two fingerprints, the matching rates of these maps are observed. The characteristics found in two samples of a finger are never the same due to the nature of the data acquisition process. For this reason, the system is expected to be tolerant to some extent when mapping the minutiae point map. This tolerance is defined as the threshold value of the matching module (150). Considering these concepts, the operation stages of the matching module (150) are, respectively, to compare the minutiae points marked with the feature extraction module (140) with the minutiae point templates of other registered examples in the database in a 1 *N manner, and to calculate a match score between the pairs being compared as a result of the comparison with each sample in the database. This score is related to the number of similarity points (the number of matching features) and the similarity captured in the maps created by these minutiae points (template similarity). An increase in the similarity relationship means that the probability of two samples belonging to the same individual increases. Matches with scores exceeding the similarity threshold for the invention are listed. In some cases, although fingerprint matches show high similarity, the dissimilar parts are sufficient to distinguish the two prints. In case of this and similar scenarios, similar matches are re-evaluated by the control module (160). Similar matches are forwarded to the mobile device (100) by the matching module (150) after the control module (160) has acknowledged them.

The last module, the control module (160), has been developed to overcome the disadvantages of fingerprint identification/verification systems. This module comprises methods developed to control the outputs obtained from collection and development module (120), enhancement module (130), the feature extraction module (140) and the matching module (150) and to produce a definite result about the successful or unsuccessful completion of the processing steps performed in these modules. The control module (160), designed to compare the minimum expected result from each module with the result obtained from the module, comprises a method designed using the criteria determined to evaluate the results obtained from the image collection and development (120), enhancement (130), feature extraction (140) and matching (150) modules. These criteria have been specially selected in order to evaluate the performance of the modules and by using the combination of these criteria, the control module (160) can make a decision. The threshold values determined for these criteria were calculated by taking the average of the results obtained as a result of the tests performed using data sets with different characteristics. The control module (160) compares the output obtained from each module connected to the system with the predetermined evaluation criteria defined in the control module (160), and decides which module should work again, and if a module successfully completes its task, it decides that the output obtained from that module is suitable to be transmitted to the next module, and ensures that the fingerprint data processed on it proceeds in the system.

Control module (160) has evaluation criteria for evaluating results from other modules. While the control module (160) determines whether the finger image coming out of the collection and development module (120) is suitable for insertion into the enhancement module (130); it uses a method that comprises evaluation criteria created by using predetermined threshold values for the quality, resolution, pixel density, sharpness/blurring, size of the image and the location/position of the finger surface during the imaging process of the collected finger image. While the control module (160) determines whether the finger image emerging from the enhancement module (130) is suitable for insertion into the feature extraction module (140), a method comprising evaluation criteria created by using predetermined threshold values for the frequency and orientation of the ridge lines in the ridge-valley pattern is used. In addition, it performs a manual assessment of whether there is an obvious problem with the ridge-valley pattern. While the control module (160) is determining whether the fingerprint coming out of the feature extraction module (140) is suitable for insertion into the matching module (150), a method that includes the evaluation criteria created by using predetermined threshold values for the detected characteristic amount (the amount of minutiae point) and the distinctiveness of the detected characteristics is used. The matching module (150) sends fingerprints that are similar to the collected fingerprint and the samples registered in the database to the control module (160). In the control module (160), a method comprising evaluation criteria created by using the similarity rate of the fingerprint match(es) that show similarity as a result of the comparison of the collected fingerprints and the fingerprints in the database, the number of registered fingerprints that are similar to the collected fingerprint (total number of samples with a possible match probability) and predetermined threshold values for the distinctiveness of the matching characteristics is used. The threshold values of the evaluation criteria in the control module (160) were determined as a result of the measurements made on the fingerprints collected with the collection and development module (120).

The control module (160) is the module where all critical decisions are made while the method and system are running. The output produced by each module is reevaluated in the control module (160). Evaluations are made in accordance with the criteria in the control module (160) in order to detect all undesirable situations beforehand. While the relations of the modules with each other are unidirectional, the relation of all modules with the control module (160) is bidirectional. The control module (160) has tasks that it undertakes for each module. The control module (160) checks the suitability of the candidate images from the collection and development module (120). Candidate images are evaluated by looking at the criteria (given above) such as resolution, blur, positioning etc. in the control module (160). If there is no obvious error in the collected finger image in this evaluation, the control module (160) commands the collection and development module (120) to transmit the collected image to the enhancement module (130). If the finger image is not properly collected, the control module (160) decides to restart the collection and development module (120). The candidate image transmitted by the control module (160) to the enhancement module (130) is enhanced in both pixel-wise and contextual filtering-wise. The enhanced finger image is evaluated a second time in the control module (160). Here, the control module (160) specifically evaluates the areas that cannot be enhanced and looks for reasons (deformations on the finger surface, illumination intensity, resolution of the lens used, finger size/position) why these areas cannot be enhanced. In addition, regardless of the result in the collection and development module (120), the control module (160) checks whether an increase in performance is observed by increasing/decreasing the sensitivity values of the running functions by changing the parametric values of the techniques and methods used in the enhancement module (130) in order to understand whether there is a situation arising from the special condition of the finger displayed. After these checks, if the control module (160) decides that the result obtained from the enhancement module (130) is not sufficient, it decides to run the collection and development module (120) again and collect the finger image again. If the control module (160) determines that the enhancement is sufficient, the enhanced fingerprint image is transmitted to the feature extraction module (140). The minutiae points detected by the feature extraction module (140) are also evaluated by the control module (160). Extracted features may be false as a result of an unsuccessful enhancement process. Within these and similar possible possibilities, the control module (160) checks by observing whether the distinguishing points are Level-2 characteristics (minutiae points) and other distinguishing elements. If the control module (160) detects errors in the characteristics included in the minutiae point map, it transmits a decision to the collection and development module (120) to collect the finger image again. If no such problem is observed, the decision module (160) decides that the minutiae point map presented by the feature extraction module (140) is suitable for transmitting to the matching module (150). The final task of the control module (160) is to compare the similarity of the fingerprint samples in the database transmitted by the matching module (150) and exceeding the matching score, with the collected sample. Overlapping and non-overlapping parts are manually examined from the minutiae point maps of both the collected image and the image in the database. After this review, the list of similar samples is eventually transmitted to the matching module (150).

As a result, within the scope of the invention, a new data acquisition device and method that can collect fingerprints with contactless approaches using a mobile device (100) has been developed, solving the difficulties encountered in the previous art. Also, within the scope of the invention, a portable biometric identification/verification system to standardise the stages of fingerprint development, collection, enhancement, feature extraction, comparison and matching has been designed, and a device and method that enables the collection of a fingerprint that can be used in biometric identification/verification systems with a mobile device (100) has been developed. In the control module (160), which is designed to evaluate the results obtained from these stages in the development, collection, enhancement, feature extraction and matching stages of the contactless collected fingerprint and to make a decision about which step to repeat and/or which stage to return to as a result of this evaluation, new methods have been developed using the criteria mentioned above.

Claims

CLAIMS A biometric identification/verification system that collects fingerprints with contactless data acquisition method, is portable and can use contactless fingerprints, comprising

• client-server that performs biometric evaluation of fingerprint, comprising the collection and development module (120), which develops the finger image by providing suitable environmental conditions, digitises and records the developed image, and sends the collected image to the central server; the enhancement module (130) that separates the background in the finger image obtained from the collection and development module (120) from the image, regulates contrast, reduces noise, accentuates ridge lines, connects broken ridge lines, fills gaps in the ridge-valley pattern, increases the interpretability of the finger image, and prepares the image for feature extraction, and reduces the data size by clearing any unwanted data from the finger image as a result of these operations; the feature extraction module (140) that converts the developed and enhanced fingerprint into a suitable format for feature extraction, extracts the characteristic features to be used for biometric identification/verification from the fingerprint, determines the location, position and type of these characteristics, creates the map of the minutiae points in the ridge-valley pattern; the matching module (150) that compares the collected fingerprints with the registered samples in the database, calculates the similarity scores of each match being compared, lists the matches with similar characteristics and transmits them to the control module (160), and transmits the possible matching candidate fingerprints, which are deemed to be similar to the registered samples in the database, to the client mobile device (100); and to the control module (160) that controls the operation mechanism of the collection and development module (120), enhancement module (130), feature extraction module (140) and matching module (150) and decides which module or process step needs to be reworked, and decides that if a module successfully completes its task, the output obtained is suitable for transmission to the next module,

• The mobile device (100) that realizes the process of contactless fingerprint imaging together with the darkroom box (200), collects finger images, transmits the collected images to the central server so that they can be compared in databases, transfers the matching results sent from the server to the user, and acts as a client in a client-server architecturebased biometric identification/verification system,

• A connector (110) that enables the mobile device (100) to be connected to the darkroom box (200), and

• A darkroom box (200) that provides suitable environmental conditions to better capture the details of the finger image consisting of darkroom box upper part (210), darkroom box lower part (220), macro lenses (240) with close-up feature in order to observe the details on the finger surface, which make the ridge lines in the fingerprint more prominent, a lighting ring (230) that provides lighting during imaging of the finger surface and obtaining sufficient contrast value, and a power supply (250) that provides the energy necessary for the operation of the lighting ring. Portable biometric identification/verification system using contactless fingerprints according to Claim 1 , comprising a mobile device (100) with processor, display screen, camera and internet access, which acts as a client in a client-server system. Portable biometric identification/verification system using contactless fingerprints according to Claim 1 , wherein the mobile device (100) may be a phone, tablet or computer. Portable biometric identification/verification system using contactless fingerprints according to Claim 1 , wherein both the data acquisition device and the biometric identification/verification system can be portable. Portable biometric identification/verification system using contactless fingerprints according to Claim 1 , comprising the enhancement module (130) that performs segmentation, normalisation and filtering steps on the finger image collected using the collection and development module (120). Portable biometric identification/verification system using contactless fingerprints according to Claim 1 , comprising feature extraction module (140) that converts the developed and enhanced fingerprint to the appropriate format by making position and zoom adjustments and extracts Level-2 characteristics (minutiae point) used for biometric identification/verification after this conversion. Portable biometric identification/verification system using contactless fingerprints according to Claim 1 , comprising client-server to ensure the performance of the processes of:

- Collecting the candidate fingerprint image (122) with the data acquisition mechanism in the mobile device (100) and the darkroom box (200),

- Transmitting the images collected by the mobile device (100) to the server,

- Evaluation of the candidate image (161 ), that is transmitted by the collection and development module (120), by the control module (160) according to the data collection criteria defined in the control module (160),

- Transmission of a command by the control module (160) to the collection and development module (120) for the next step, ROI determination (124), if the candidate image is suitable,

- Determining the ROI (124) from the fingerprint image of the collection and development module (120) with the positive feedback received from the control module (160), - Transmitting (125) the determined ROI image over the server to the enhancement module (130),

- Editing (131 ) of the contrast value in the image transmitted from the collection and development module (120) to the enhancement module (130) by the enhancement module (130), then estimating the orientation of the ridge lines (132) and estimating the frequency of the ridge lines (133) in the image,

- Transmission of the image by the enhancement module (130) to the control module (160) for the evaluation of the image (137),

- Evaluation of the incoming image by the control module (160) according to data enhancement criteria (163),

- Extraction of the minutiae points (141 ) by the feature extraction module (140) from the image coming from the enhancement module (130) and then elimination of the false characteristics in these extracted minutiae points (142),

- Transmission of the minutiae point map by the feature extraction module (140) obtained from the fingerprint image to the control module (160) for evaluation (143), - Evaluation (165) the minutiae point map transmitted by the feature extraction module (140) by the control module (160) by looking at the feature extraction criteria defined in the control module (160),

- Completion of the characteristic sensing process (144) and transmission of the image to the matching module (150) by the feature extraction module (140) upon the result of the evaluation from the control module (160),

- Evaluation (151 ) of the characteristics transmitted from the feature extraction module (140) of the matching module (150) and then comparison of these characteristics with the characteristics of the samples registered in the database (152),

- Calculation of match scores (153) and detection of matches with scores exceeding the threshold value (154) upon the result of comparison,

- Transmission of the matched samples to the mobile device (156) by the matching module (150) in line with the feedback received from the control module (160). Fingerprint identification/verification system according to Claim 7, comprising the control module (160) to ensure the evaluation processes of the criteria of - Quality of collected finger image

- Resolution of the collected finger image

- Pixel density of the collected finger image

- Sharpness/blur of the collected finger image

- Size of finger image collected

- Location/position of the finger surface in the collected image. Fingerprint identification/verification method according to Claim 7, comprising the control module (160) to ensure the processes for evaluating criteria for performing a manual check of

- Frequency of ridge lines in a ridge-valley pattern

- Orientation of ridge lines in a ridge-valley pattern

- Whether the ridge-valley pattern has obvious problems. Fingerprint identification/verification system according to Claim 7, comprising the control module (160) to ensure the evaluation processes of the criteria of

- Detected characteristic amount

- The distinctiveness of the detected characteristics. Fingerprint identification/verification system according to Claim 7, comprising the control module (160) to ensure the evaluation processes of the criteria of

- Similarity rate of fingerprint matching with collected data

- Number of registered fingerprints matching the collected data

- Distinctiveness of matching characteristics.