US10558841B2 - Method and apparatus for recognizing fingerprint ridge point - Google Patents

Method and apparatus for recognizing fingerprint ridge point Download PDF

Info

Publication number
US10558841B2
US10558841B2 US15/841,055 US201715841055A US10558841B2 US 10558841 B2 US10558841 B2 US 10558841B2 US 201715841055 A US201715841055 A US 201715841055A US 10558841 B2 US10558841 B2 US 10558841B2
Authority
US
United States
Prior art keywords
pixel
target pixel
ridge
fingerprint
pixels
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US15/841,055
Other versions
US20180101710A1 (en
Inventor
Hongfa WANG
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Assigned to TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED reassignment TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: WANG, Hongfa
Publication of US20180101710A1 publication Critical patent/US20180101710A1/en
Application granted granted Critical
Publication of US10558841B2 publication Critical patent/US10558841B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12—Fingerprints or palmprints
    • G06V40/1347—Preprocessing; Feature extraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12—Fingerprints or palmprints
    • G06V40/1347—Preprocessing; Feature extraction
    • G06V40/1359—Extracting features related to ridge properties; Determining the fingerprint type, e.g. whorl or loop
    • G06K9/001—
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G06K9/00013—
    • G06K9/0008—
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/30—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12—Fingerprints or palmprints
    • G06V40/1347—Preprocessing; Feature extraction
    • G06V40/1353—Extracting features related to minutiae or pores
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/12—Fingerprints or palmprints
    • G06V40/1365—Matching; Classification
    • G06V40/1376—Matching features related to ridge properties or fingerprint texture

Definitions

  • the present disclosure relates to the field of image recognition technologies, and in particular, to a method and an apparatus for recognizing fingerprint ridge points.
  • Fingerprints are unique, convenient to be checked, and durable over the life of an individual, and therefore have become a primary means for biometric identification.
  • fingerprint recognition has becomes a very important technology of identity authentication.
  • a fingerprint consists of ridges on the epidermis at the end of a person's finger. Two fingerprints usually have same general features, but minutia features of the two fingerprints would not be identical.
  • Fingerprint ridges are not continuous, smooth or straight, but instead they are often terminal, bifurcated or bent. These ridge endings, bifurcations points, and bending points are referred to as “feature points”.
  • Fingerprint recognition is a technology of performing identity authentication by comparing feature points of different fingerprints.
  • a method for recognizing a fingerprint ridge point is particularly provided.
  • One aspect of the present disclosure provides a method for recognizing a fingerprint ridge point.
  • the method includes receiving an input fingerprint image; performing orientation field estimation on the fingerprint image; identifying a pixel in the fingerprint image as a target pixel; determining a ridge direction of the target pixel, obtained according to the orientation field estimation, and a normal direction perpendicular to the ridge direction; defining a rectangular region by using the target pixel as a central point, a length of the rectangular region being along the ridge direction, a width of the rectangular region being along the normal direction of perpendicular to the ridge, and the rectangular region comprising multiple pixel rows arranged in the normal direction; calculating a sum of grayscale values of pixels in each pixel row; and determining that the target pixel is a fingerprint ridge point when the sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.
  • the apparatus includes: a module for receiving a fingerprint image, configured to: receive an input fingerprint image, and perform orientation field estimation on the fingerprint image; a module for determining a pixel direction, configured to: traverse a pixel in the fingerprint image as a target pixel, and determine a ridge direction of the target pixel according to the orientation field estimation, and a normal direction perpendicular to the ridge direction; a module for selecting a reference pixel, configured to define a rectangular region by using the target pixel as a central point, a length of the rectangular region being along the ridge direction, a width of the rectangular region being along the normal direction of perpendicular to the ridge, and the rectangular region comprising multiple pixel rows arranged in the normal direction; and a module for determining a fingerprint ridge point, configured to: calculate a sum of grayscale values of pixels in each pixel row, and determine that the target pixel is a fingerprint ridge point when a sum of grayscale
  • Another aspect of the present disclosure provides a computer readable media containing program instructions, when executed by one or more processor, the programs instructions perform a method for recognizing a fingerprint ridge point.
  • the method includes: performing orientation field estimation on a fingerprint image; identifying a pixel in the fingerprint image as a target pixel; determining a ridge direction of the target pixel, obtained according to the orientation field estimation, and a normal direction perpendicular to the ridge direction; defining a rectangular region by using the target pixel as a central point, a length of the rectangular region being along the ridge direction, a width of the rectangular region being along the normal direction of perpendicular to the ridge, and the rectangular region comprising multiple pixel rows arranged in the normal direction; calculating a sum of grayscale values of pixels in each pixel row; and determining that the target pixel is a fingerprint ridge point when the sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.
  • pixels in the fingerprint image are traversed.
  • a rectangular region is defined by using a traversed pixel as a central point. Lengthwise and widthwise extending directions of the rectangular region are respectively corresponding to a ridge direction and a normal direction of a target pixel, and the rectangular region includes multiple pixel rows arranged in the normal direction.
  • a sum of grayscale values of pixels in each pixel row is calculated, and it is determined that the target pixel is a fingerprint ridge point when a sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.
  • both a grayscale value of a traversed pixel and a grayscale value of a pixel at an adjacent position of the traversed pixel are used as references, so that incorrectly determined noise is effectively eliminated, and recognition accuracy is improved.
  • FIG. 1 is a flowchart of a method for recognizing a fingerprint ridge point according to an embodiment
  • FIG. 2 is a schematic diagram of defining a rectangular region by using a target pixel as a central point according to an embodiment
  • FIG. 3 is a schematic diagram of a grayscale value of each pixel in a pixel row in a rectangular region according to an embodiment
  • FIG. 4 is a schematic diagram of a received fingerprint image according to an embodiment
  • FIG. 5 is a schematic diagram of a binarized fingerprint ridge image has been performed according to an embodiment
  • FIG. 6 is a schematic diagram of the definition of a ridge distance according to an embodiment
  • FIG. 7 is a flowchart of a process of calculating a weighted sum as a sum of grayscale values of pixels in a pixel row according to an embodiment
  • FIG. 8 is a schematic diagram of an apparatus for recognizing a fingerprint ridge point according to an embodiment.
  • FIG. 9 is a schematic structural diagram of a computer device for performing the foregoing method for recognizing a fingerprint ridge point according to an embodiment.
  • Detection of feature points in fingerprint recognition often depends on detection of fingerprint ridges. That is, the feature points are obtained by analyzing the fingerprint ridges. A collected fingerprint image is binarized to obtain fingerprint ridges according to the fingerprint image. Often, black pixels are used for fingerprint ridge points, and white pixels are used for background regions on two sides of fingerprint ridges, thereby facilitating recognition of feature points of a fingerprint.
  • the binarization methods usually include a local binarization method based on Gabor filters and the like.
  • a method based on Gabor filters When a method based on Gabor filters is used, local directions and ridge frequencies need to be accurately calculated. However, it is relatively difficult to accurately calculate local directions and ridge frequencies when preprocessing of some low-quality fingerprint images.
  • a method for recognizing a fingerprint ridge point may be implemented by using a computer, and may be performed on a computer system that is based on the von Neumann architecture.
  • the computer program may be a fingerprint recognition program.
  • the computer system may be a terminal device such as a smartphone, a tablet computer, a palmtop computer, a notebook computer or a personal computer on which a fingerprint sensor is installed.
  • the method includes the following steps.
  • Step S 104 Identify a pixel in the fingerprint image as a target pixel, and determine a ridge direction, of the target pixel according to orientation field estimation and a normal direction perpendicular to the ridge direction.
  • the fingerprint image may be collected by using a camera or other fingerprint image capturing devices.
  • a user may put a finger on a glass panel of a fingerprint sensor, and an image collection device located below the glass panel may obtain a fingerprint image of the finger.
  • a fingerprint image photographed by another device may be received, and fingerprint recognition can be performed on the obtained fingerprint image.
  • illumination normalization is first performed on the fingerprint image.
  • An objective of illumination normalization is to eliminate the impact of light on the fingerprint image during collection of the fingerprint image. That is, during collection of the fingerprint image, due to different light source locations or light intensities, some regions of the collected fingerprint image are relatively bright and some other regions are relatively dark, or the entire fingerprint image is relatively bright or dark. After illumination normalization is performed, the relatively bright or dark regions may be corrected, thereby eliminating the impact of light on the collected fingerprint image.
  • illumination normalization may be implemented by using a method based on a feature subspace or a correction method based on histogram equalization.
  • Orientation field estimation of the fingerprint image may be used to determine direction information of a ridge that passes through each pixel in the fingerprint image.
  • the direction information of each point may be calculated based on gradient distribution of grayscale values of pixels. For example, if a difference between a grayscale value of a pixel and a grayscale value of a pixel adjacent to the pixel on the left or right side is relatively small, and a difference between the grayscale value of the pixel and a grayscale value of a pixel adjacent to the pixel on the upper or lower side is relatively large, a ridge direction is a horizontal direction from the left side to the right side of the pixel, and a normal direction perpendicular to the ridge direction is a vertical direction from the upper side to the lower side of the pixel.
  • Step S 106 Define a rectangular region by using the target pixel as a central point, where lengthwise and widthwise extending directions of the rectangular region are respectively corresponding to the ridge direction and the normal direction of the target pixel, and the rectangular region includes multiple pixel rows arranged in the normal direction.
  • a fingerprint image includes multiple pixels.
  • the pixel A is identified as a target pixel.
  • a direction formed by connecting pixels A 1 , A 2 , A, A 3 , and A 4 is a ridge direction of the pixel A obtained by performing orientation field estimation on the pixel A.
  • a direction formed by connecting E, D, C, B, F, G, H, and I is a normal direction of the pixel A obtained by performing orientation field estimation on the pixel A.
  • the ridge direction of the pixel A is perpendicular to the normal direction of the pixel A.
  • a rectangular region defined by using the pixel A as a central point is a rectangular region formed by using E 1 , E 4 , 14 , and I 1 as vertices.
  • Four sides of the rectangular region are: a long side formed by connecting pixels E 1 , D 1 , C 1 , B 1 , F 1 , G 1 , H 1 , and I 1 , a short side formed by connecting pixels E 1 , E 2 , E, E 3 , and E 4 , a long side formed by connecting pixels E 4 , D 4 , C 4 , B 4 , F 4 , G 4 , H 4 , and I 4 , and a short side formed by connecting pixels I 1 , I 2 , E, I 3 , and I 4 .
  • multiple pixel rows arranged in the normal direction of the pixel A in the rectangle includes:
  • a first pixel row E 1 , E 2 , E, E 3 , and E 4 ;
  • a second pixel row D 1 , D 2 , D, D 3 , and D 4 ;
  • a third pixel row C 1 , C 2 , C, C 3 , and C 4 ;
  • a ninth pixel row I 1 , I 2 , I, I 3 , and I 4 .
  • Step S 108 Calculate a sum of grayscale values of pixels in each pixel row, and determine that the target pixel is a fingerprint ridge point when a sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.
  • grayscale values of pixels included in the rectangle (E 1 , E 4 , I 4 , I 1 ) in FIG. 2 , and a sum of grayscale values of each pixel row is calculated:
  • the sum of grayscale values of the pixels in the fifth pixel row is 57, and the fifth pixel row includes a target pixel A. That is, the sum of grayscale values corresponding to the pixel row that includes the target pixel A is the minimum of the sums of grayscale values corresponding to all the pixel rows. Therefore, it is determined that the target pixel A is a fingerprint ridge point.
  • the pixels A 1 , A 2 , A 3 , and A 4 in the fifth pixel row are all pixels in the ridge direction of the target pixel A. Pixels in another pixel row, for example, B 1 , B 2 , B, B 3 , and B 4 in the fourth pixel row, are also in the ridge direction, and form a pixel row that is adjacent and parallel to the fifth pixel row in the normal direction. If the sum of grayscale values of the pixels in the fifth pixel row is less than the sum of grayscale values of the pixels in the fourth pixel row, it indicates that the color of the pixels in the fifth pixel row is generally darker than the color of the pixels in the fourth pixel row, so that the fifth pixel row better meets a feature of a fingerprint ridge. Therefore, a candidate fingerprint ridge formed by the fifth pixel row is definitely chosen over a candidate fingerprint ridge formed by the fourth pixel row as a fingerprint ridge.
  • the target pixel A can be determined as a fingerprint ridge point in the fingerprint ridge.
  • the rectangular region is defined, and the sums of grayscale values of the pixel rows are calculated and compared. Therefore, during determining of a fingerprint ridge point, not only the grayscale value of the target pixel A is used as a determining basis, but also a grayscale value of a pixel adjacent to the target pixel A in the ridge direction is used as a reference. For example, if a grayscale value of the target pixel A is relatively large but the grayscale values of A 1 , A 2 , A 3 , and A 4 are relatively small, a general sum of grayscale values of the fifth pixel row is relatively small. This indicates that the target pixel A may be noise data, but the target pixel A may still be determined as a fingerprint ridge point. Therefore, interference from noise is eliminated, and recognition accuracy is improved.
  • step S 104 to step S 108 are sequentially performed until pixels in the entire fingerprint image are traversed
  • fingerprint ridge points included in the fingerprint image can be determined.
  • Grayscale values of fingerprint ridge points are set to a relatively large value (for example, 255), and grayscale values of pixels that are not fingerprint ridge points in the fingerprint image are set to a relatively small value (for example, 0), so that the fingerprint image is binarized, thereby facilitating the following recognition process of features included in the fingerprint image.
  • FIG. 4 shows an input fingerprint image
  • FIG. 5 is a binarized image generated by traversing all pixels in the input fingerprint image.
  • a grayscale value of a fingerprint ridge point is 255
  • a grayscale value of a non-fingerprint ridge point is 0 (that is, color negative processing is performed).
  • the length of a long side of a rectangle may be set in advance.
  • the step of defining a rectangular region by using the target pixel as a central point is: calculating an average ridge distance of the fingerprint image; and setting a first quantity of the pixel rows according to the average ridge distance.
  • a distance between a ridge and an adjacent furrow of a fingerprint is usually defined as a fingerprint ridge distance.
  • the length between the center of a ridge and the center of a furrow is usually calculated as the ridge distance. It is assumed that there are n ridges whose widths are respectively w 1 , . . . , w n in the fingerprint image, and the ridges are defined separated by n- 1 furrows whose widths are v 1 , . . . , v n .
  • the average ridge distance (the ridge distance for short hereinafter) of the fingerprint image refers to an average value of ridge distances in a particular region.
  • the size of a ridge distance depends on a structure of the fingerprint and the resolution of image collection. In the fingerprint image, when a ridge distance is larger, it indicates that ridges there are sparser. In contrast, when a ridge distance is smaller, it indicates that ridges there are denser.
  • the average ridge distance may be obtained by estimating ridge frequency.
  • Methods for estimating the ridge distance include a geometric method and a spectral analysis method.
  • a method proposed by Yilong Yin may be used, and in the method, a ridge distance is estimated based on a statistics window.
  • a method proposed by En Zhu may be used, and in the method, local regions are projected on a curve that passes through the centers of the regions, where tangential directions of all points are perpendicular to the ridge direction. In this way, the average ridge distance of the entire image is calculated.
  • the setting a first quantity of the pixel rows according to the average ridge distance may be specifically: setting the first quantity of the pixel rows to be less than a quantity of pixels corresponding to the average ridge distance.
  • the defined rectangular region extends greatly in the normal direction of the target pixel A.
  • the rectangular region may extend across two ridges. Consequently, pixels in a paralleled fingerprint ridge affect the determining of the target pixel A.
  • the defined rectangular region may not cover multiple ridges in the normal direction of the target pixel A. Therefore, interference from another ridge is eliminated, and the accuracy of a binarized image is improved.
  • a second quantity of pixels included in each pixel row may be set according to a quantity of the pixel rows.
  • a value of the second quantity may be 1 ⁇ 3 to 2 ⁇ 3 of that of the first quantity.
  • a quantity of pixels selected in the ridge direction of the target pixel may be less than pixels selected in the normal direction of the target pixel. That is, a side, extending in the normal direction of the target pixel, of the rectangular region is the long side of the rectangular region, and a side, extending in the ridge direction of the target pixel, of the rectangular region is the short side of the rectangular region.
  • the length of the short side may be selected to be between 1 ⁇ 3 and 2 ⁇ 3 of the length of the long side.
  • the second quantity is set to be relatively large, determining of a target pixel at a position where a fingerprint ridge bends sharply is easily interfered with by another fingerprint ridge point in the bent ridge. If the second quantity is set to be relatively small, a quantity of adjacent reference pixels on which the determining of the target pixel that is incorrectly collected as noise depends is reduced. Therefore, the target pixel that is incorrectly collected as the digital noise is easily ignored. If the value of the second quantity is set to be 1 ⁇ 3 to 2 ⁇ 3 of that of the first quantity, a balance can be reached between the first quantity and the second quantity, so that the accuracy of binarization is improved.
  • a corresponding weight coefficient is set for each pixel in the pixel row, and a weighted sum is calculated according to weight coefficients as the sum of grayscale values of pixels in the pixel row.
  • the method includes the following steps.
  • Step S 202 Traverse pixel rows, and identify a traversed pixel row as a target pixel row.
  • Step S 204 Determine grayscale values of pixels in the traversed pixel row that is used as the target pixel row and corresponding weight coefficients.
  • Step S 206 Multiply the grayscale values of the pixels in the traversed pixel row by the corresponding weight coefficients, and add the products to obtain a weighted sum.
  • the sum of grayscale values of the first pixel row E 1 , E 2 , E, E 3 , and E 4 is: 79 ⁇ b 1 +45 ⁇ b 2 +83 ⁇ b 3 +84 ⁇ b 2 +51 ⁇ b 1 .
  • An appropriate weight coefficient is set in advance for a pixel at each position in the pixel row, so that proportions accounted for by the grayscale value of each pixel in the calculated sum of grayscale values of the pixel row may be changed, and the pixels at some positions in the pixel row may have a greater impact on the sum of grayscale values of the entire pixel row.
  • the weight coefficients of the pixels in the target pixel row decrement with a distance from a central pixel in the target pixel row.
  • the weight coefficient b 1 corresponding to E 1 and E 4 may be set to be less than the weight coefficient b 2 corresponding to E 2 and E 3 .
  • the grayscale value of E has a relatively strong impact on determining of whether E is a fingerprint ridge point. Therefore, the weight coefficient b 2 corresponding to E 2 and E 3 may be set to be less than the weight coefficient b 3 corresponding to E.
  • the weight coefficient of each pixel in the target pixel row is set in a manner in which weight coefficients decrement with a distance from the central pixel in the target pixel row.
  • a grayscale value of a pixel that is relatively close to the central pixel in the target pixel row may have a relatively large impact on the sum of grayscale values of the target pixel row, and a grayscale value of a pixel that is relatively far from the central pixel in the target pixel row is enabled to have a relatively small impact on the sum of grayscale values of the target pixel row, thereby improving determining accuracy.
  • a grayscale value of the target pixel may be determined. It is determined whether the grayscale value of the target pixel is greater than a threshold. If the grayscale value is greater than the threshold, it is determined that the target pixel is not a fingerprint ridge point.
  • pre-estimation may be performed on the fingerprint image to directly determine that a pixel that clearly has a relatively large grayscale value is not a fingerprint ridge point. Therefore, the step of defining a rectangular region is omitted, thereby increasing a recognition speed.
  • an apparatus for recognizing a fingerprint ridge point includes a module for receiving a fingerprint image 102 , a module for acquiring a pixel direction 104 , a module for selecting a reference pixel 106 , and a module for determining a fingerprint ridge point 108 .
  • the module for receiving a fingerprint image 102 is configured to: receive an input fingerprint image, and perform orientation field estimation on the fingerprint image.
  • the module for acquiring a pixel direction 104 is configured to: traverse a pixel in the fingerprint image as a target pixel, and determine a ridge direction, obtained according to orientation field estimation, of the target pixel and a normal direction perpendicular to the ridge direction.
  • the module for selecting a reference pixel 106 is configured to: define a rectangular region by using the target pixel as a central point, where lengthwise and widthwise extending directions of the rectangular region are respectively corresponding to the ridge direction and the normal direction of the target pixel, and the rectangular region includes multiple pixel rows arranged in the normal direction.
  • the module for determining a fingerprint ridge point 108 is configured to: calculate a sum of grayscale values of pixels in each pixel row, and determine that the target pixel is a fingerprint ridge point when a sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.
  • the module for selecting a reference pixel 106 is further configured to: calculate an average ridge distance of the fingerprint image, and set a first quantity of the pixel rows according to the average ridge distance.
  • the module for selecting a reference pixel 106 is further configured to set, according to a quantity of the pixel rows, a second quantity of pixels included in each pixel row, where a value of the second quantity is 1 ⁇ 3 to 2 ⁇ 3 of that of the first quantity.
  • the module for determining a fingerprint ridge point 108 is further configured to: traverse pixel rows, and identify a traversed pixel row as a target pixel row; determine grayscale values of pixels in the traversed pixel row that is used as the target pixel row and corresponding weight coefficients; and multiply the grayscale values of the pixels in the traversed pixel row by the corresponding weight coefficients, and add the products to obtain a weighted sum.
  • the apparatus for recognizing a fingerprint ridge point further includes a pre-filtering module 110 , configured to: determine a grayscale value of the target pixel; determine whether the grayscale value of the target pixel is greater than a threshold; and if the grayscale value is greater than the threshold, determine that the target pixel is not a fingerprint ridge point.
  • a pre-filtering module 110 configured to: determine a grayscale value of the target pixel; determine whether the grayscale value of the target pixel is greater than a threshold; and if the grayscale value is greater than the threshold, determine that the target pixel is not a fingerprint ridge point.
  • pixels in the fingerprint image are traversed.
  • a rectangular region is defined by using a traversed pixel as a central point. Lengthwise and widthwise extending directions of the rectangular region are respectively corresponding to a ridge direction and a normal direction of a target pixel, and the rectangular region includes multiple pixel rows arranged in the normal direction.
  • a sum of grayscale values of pixels in each pixel row is calculated, and it is determined that the target pixel is a fingerprint ridge point when a sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.
  • both a grayscale value of a traversed pixel and a grayscale value of a pixel at an adjacent position of the traversed pixel are used as references, so that incorrectly determined noise is effectively eliminated, and recognition accuracy is improved.
  • FIG. 9 shows a computer system 10 that is based on the von Neumann architecture and performs the foregoing method for recognizing a fingerprint ridge point.
  • the computer system 10 may be a terminal device such as a smartphone, a tablet computer, a palmtop computer, a notebook computer or a personal computer.
  • the computer system 10 may include an external input interface 1001 , a processor 1002 , a memory 1003 , and an output interface 1004 that are connected through a system bus.
  • the external input interface 1001 may optionally include at least one of a mouse 10012 , a keyboard 10014 , a touchpad 10016 or a fingerprint sensor interface 10018 .
  • the memory 1003 may include an external memory 10032 (for example a hard disk, an optical disk or a floppy disk) and an internal memory 10034 .
  • the output interface 1004 may include devices such as a display screen 10042 and a speaker 10044 .
  • the method is performed based on a computer program.
  • a program file of the computer program may be stored in the external memory 10032 of the computer system 10 and is loaded to the internal memory 10034 when being run.
  • the program file may then be compiled into machine code, and the machine code is transmitted to the processor 1002 and executed by the processor 1002 .
  • a logic module for receiving a fingerprint image 102 a module for acquiring a pixel direction 104 , a module for selecting a reference pixel 106 , and a module for determining a fingerprint ridge point 108 may be formed in the computer system 10 , and implement their the functions as described in relation to FIG. 8 .
  • input parameters may all be received by using the external input interface 1001 , transmitted to and cached in the memory 1003 , and then input to the processor 1002 to be processed.
  • Result data of processing may be cached in the memory 1003 for subsequent processing, or may be transmitted to the output interface 1004 for output.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Collating Specific Patterns (AREA)
  • Image Input (AREA)
  • Image Analysis (AREA)

Abstract

Embodiments of the present disclosure disclose a method for recognizing a fingerprint ridge point. The method includes performing orientation field estimation on a fingerprint image; identifying a pixel in the fingerprint image as a target pixel; determining a ridge direction of the target pixel according to the orientation field estimation, and a normal direction perpendicular to the ridge direction; defining a rectangular region by using the target pixel as a central point, a length of the rectangular region being along the ridge direction, a width of the rectangular region being along the normal direction of perpendicular to the ridge, and the rectangular region comprising multiple pixel rows; calculating a sum of grayscale values of pixels in each pixel row; and determining that the target pixel is a fingerprint ridge point when the sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.

Description

RELATED APPLICATION
This application claims priority to PCT application No. PCT/CN2016/079584, filed on Apr. 18, 2016, which claims priority to Chinese Patent Application No. 201510566721.3, entitled “METHOD AND APPARATUS FOR RECOGNIZING FINGERPRINT RIDGE POINT” filed on Sep. 8, 2015. Both applications are incorporated by reference in the entirety.
FIELD OF THE TECHNOLOGY
The present disclosure relates to the field of image recognition technologies, and in particular, to a method and an apparatus for recognizing fingerprint ridge points.
BACKGROUND OF THE DISCLOSURE
Fingerprints are unique, convenient to be checked, and durable over the life of an individual, and therefore have become a primary means for biometric identification. After centuries of development, fingerprint recognition has becomes a very important technology of identity authentication. A fingerprint consists of ridges on the epidermis at the end of a person's finger. Two fingerprints usually have same general features, but minutia features of the two fingerprints would not be identical. Fingerprint ridges are not continuous, smooth or straight, but instead they are often terminal, bifurcated or bent. These ridge endings, bifurcations points, and bending points are referred to as “feature points”. Fingerprint recognition is a technology of performing identity authentication by comparing feature points of different fingerprints.
SUMMARY
To improve accuracy of fingerprint recognition by obtaining better a binarized fingerprint image based on Gabor filters, a method for recognizing a fingerprint ridge point is particularly provided.
One aspect of the present disclosure provides a method for recognizing a fingerprint ridge point. The method includes receiving an input fingerprint image; performing orientation field estimation on the fingerprint image; identifying a pixel in the fingerprint image as a target pixel; determining a ridge direction of the target pixel, obtained according to the orientation field estimation, and a normal direction perpendicular to the ridge direction; defining a rectangular region by using the target pixel as a central point, a length of the rectangular region being along the ridge direction, a width of the rectangular region being along the normal direction of perpendicular to the ridge, and the rectangular region comprising multiple pixel rows arranged in the normal direction; calculating a sum of grayscale values of pixels in each pixel row; and determining that the target pixel is a fingerprint ridge point when the sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.
Another aspect of the present disclosure provides an apparatus for recognizing a fingerprint ridge point. The apparatus includes: a module for receiving a fingerprint image, configured to: receive an input fingerprint image, and perform orientation field estimation on the fingerprint image; a module for determining a pixel direction, configured to: traverse a pixel in the fingerprint image as a target pixel, and determine a ridge direction of the target pixel according to the orientation field estimation, and a normal direction perpendicular to the ridge direction; a module for selecting a reference pixel, configured to define a rectangular region by using the target pixel as a central point, a length of the rectangular region being along the ridge direction, a width of the rectangular region being along the normal direction of perpendicular to the ridge, and the rectangular region comprising multiple pixel rows arranged in the normal direction; and a module for determining a fingerprint ridge point, configured to: calculate a sum of grayscale values of pixels in each pixel row, and determine that the target pixel is a fingerprint ridge point when a sum of grayscale values corresponding to a pixel row that comprises the target pixel is the minimum.
Another aspect of the present disclosure provides a computer readable media containing program instructions, when executed by one or more processor, the programs instructions perform a method for recognizing a fingerprint ridge point. The method includes: performing orientation field estimation on a fingerprint image; identifying a pixel in the fingerprint image as a target pixel; determining a ridge direction of the target pixel, obtained according to the orientation field estimation, and a normal direction perpendicular to the ridge direction; defining a rectangular region by using the target pixel as a central point, a length of the rectangular region being along the ridge direction, a width of the rectangular region being along the normal direction of perpendicular to the ridge, and the rectangular region comprising multiple pixel rows arranged in the normal direction; calculating a sum of grayscale values of pixels in each pixel row; and determining that the target pixel is a fingerprint ridge point when the sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.
By means of the foregoing method and the apparatus for recognizing a fingerprint ridge point, during binarization of a fingerprint image, pixels in the fingerprint image are traversed. A rectangular region is defined by using a traversed pixel as a central point. Lengthwise and widthwise extending directions of the rectangular region are respectively corresponding to a ridge direction and a normal direction of a target pixel, and the rectangular region includes multiple pixel rows arranged in the normal direction. A sum of grayscale values of pixels in each pixel row is calculated, and it is determined that the target pixel is a fingerprint ridge point when a sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum. Compared with a method of using a filter or threshold filtering, both a grayscale value of a traversed pixel and a grayscale value of a pixel at an adjacent position of the traversed pixel are used as references, so that incorrectly determined noise is effectively eliminated, and recognition accuracy is improved.
BRIEF DESCRIPTION OF THE DRAWINGS
To describe the technical solutions of the embodiments of the present disclosure or the existing technology more clearly, the following briefly introduces the accompanying drawings required for describing the embodiments or the existing technology. Apparently, the accompanying drawings in the following description show only some embodiments of the present disclosure, and a person of ordinary skill in the art may still derive other drawings from these accompanying drawings without creative efforts.
FIG. 1 is a flowchart of a method for recognizing a fingerprint ridge point according to an embodiment;
FIG. 2 is a schematic diagram of defining a rectangular region by using a target pixel as a central point according to an embodiment;
FIG. 3 is a schematic diagram of a grayscale value of each pixel in a pixel row in a rectangular region according to an embodiment;
FIG. 4 is a schematic diagram of a received fingerprint image according to an embodiment;
FIG. 5 is a schematic diagram of a binarized fingerprint ridge image has been performed according to an embodiment;
FIG. 6 is a schematic diagram of the definition of a ridge distance according to an embodiment;
FIG. 7 is a flowchart of a process of calculating a weighted sum as a sum of grayscale values of pixels in a pixel row according to an embodiment;
FIG. 8 is a schematic diagram of an apparatus for recognizing a fingerprint ridge point according to an embodiment; and
FIG. 9 is a schematic structural diagram of a computer device for performing the foregoing method for recognizing a fingerprint ridge point according to an embodiment.
 DESCRIPTION OF EMBODIMENTS
The following clearly and completely describes the technical solutions in the embodiments of the present disclosure with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are some of the embodiments of the present disclosure rather than all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.
Detection of feature points in fingerprint recognition often depends on detection of fingerprint ridges. That is, the feature points are obtained by analyzing the fingerprint ridges. A collected fingerprint image is binarized to obtain fingerprint ridges according to the fingerprint image. Often, black pixels are used for fingerprint ridge points, and white pixels are used for background regions on two sides of fingerprint ridges, thereby facilitating recognition of feature points of a fingerprint.
Further, the binarization methods usually include a local binarization method based on Gabor filters and the like. When a method based on Gabor filters is used, local directions and ridge frequencies need to be accurately calculated. However, it is relatively difficult to accurately calculate local directions and ridge frequencies when preprocessing of some low-quality fingerprint images.
In one embodiment of the present disclosure, a method for recognizing a fingerprint ridge point is provided. The method may be implemented by using a computer, and may be performed on a computer system that is based on the von Neumann architecture. The computer program may be a fingerprint recognition program. The computer system may be a terminal device such as a smartphone, a tablet computer, a palmtop computer, a notebook computer or a personal computer on which a fingerprint sensor is installed.
Specifically, as shown in FIG. 1, the method includes the following steps.
Step S102: Receive an input fingerprint image, and perform orientation field estimation on the fingerprint image.
Step S104: Identify a pixel in the fingerprint image as a target pixel, and determine a ridge direction, of the target pixel according to orientation field estimation and a normal direction perpendicular to the ridge direction.
In this embodiment, the fingerprint image may be collected by using a camera or other fingerprint image capturing devices. A user may put a finger on a glass panel of a fingerprint sensor, and an image collection device located below the glass panel may obtain a fingerprint image of the finger. In another embodiment, a fingerprint image photographed by another device may be received, and fingerprint recognition can be performed on the obtained fingerprint image.
In one embodiment, after the fingerprint image is collected, illumination normalization is first performed on the fingerprint image. An objective of illumination normalization is to eliminate the impact of light on the fingerprint image during collection of the fingerprint image. That is, during collection of the fingerprint image, due to different light source locations or light intensities, some regions of the collected fingerprint image are relatively bright and some other regions are relatively dark, or the entire fingerprint image is relatively bright or dark. After illumination normalization is performed, the relatively bright or dark regions may be corrected, thereby eliminating the impact of light on the collected fingerprint image. In this embodiment, illumination normalization may be implemented by using a method based on a feature subspace or a correction method based on histogram equalization.
Orientation field estimation of the fingerprint image may be used to determine direction information of a ridge that passes through each pixel in the fingerprint image. In this embodiment, the direction information of each point may be calculated based on gradient distribution of grayscale values of pixels. For example, if a difference between a grayscale value of a pixel and a grayscale value of a pixel adjacent to the pixel on the left or right side is relatively small, and a difference between the grayscale value of the pixel and a grayscale value of a pixel adjacent to the pixel on the upper or lower side is relatively large, a ridge direction is a horizontal direction from the left side to the right side of the pixel, and a normal direction perpendicular to the ridge direction is a vertical direction from the upper side to the lower side of the pixel.
Step S106: Define a rectangular region by using the target pixel as a central point, where lengthwise and widthwise extending directions of the rectangular region are respectively corresponding to the ridge direction and the normal direction of the target pixel, and the rectangular region includes multiple pixel rows arranged in the normal direction.
As shown in FIG. 2, a fingerprint image includes multiple pixels. When a pixel A is traversed, the pixel A is identified as a target pixel. A direction formed by connecting pixels A1, A2, A, A3, and A4 is a ridge direction of the pixel A obtained by performing orientation field estimation on the pixel A. A direction formed by connecting E, D, C, B, F, G, H, and I is a normal direction of the pixel A obtained by performing orientation field estimation on the pixel A. The ridge direction of the pixel A is perpendicular to the normal direction of the pixel A.
In this embodiment, a rectangular region defined by using the pixel A as a central point is a rectangular region formed by using E1, E4, 14, and I1 as vertices. Four sides of the rectangular region are: a long side formed by connecting pixels E1, D1, C1, B1, F1, G1, H1, and I1, a short side formed by connecting pixels E1, E2, E, E3, and E4, a long side formed by connecting pixels E4, D4, C4, B4, F4, G4, H4, and I4, and a short side formed by connecting pixels I1, I2, E, I3, and I4.
As shown in FIG. 2, multiple pixel rows arranged in the normal direction of the pixel A in the rectangle (E1, E4, I4, I1) includes:
a first pixel row: E1, E2, E, E3, and E4;
a second pixel row: D1, D2, D, D3, and D4;
a third pixel row: C1, C2, C, C3, and C4;
. . .
a ninth pixel row: I1, I2, I, I3, and I4.
Step S108: Calculate a sum of grayscale values of pixels in each pixel row, and determine that the target pixel is a fingerprint ridge point when a sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.
As shown in FIG. 3, grayscale values of pixels included in the rectangle (E1, E4, I4, I1) in FIG. 2, and a sum of grayscale values of each pixel row is calculated:
a sum of grayscale values of the first pixel row E1, E2, E, E3, and E4 is 79+45+83+84+51=342;
a sum of grayscale values of the second pixel row D1, D2, D, and D3 is 42+80+49+97+96=364;
a sum of grayscale values of the third pixel row C1, C2, C, C3, and C4 is 43+46+88+50+58=285;
. . .
a sum of grayscale values of the fifth pixel row A1, A2, A, A3, and A4 is 12+11+7+23+4=57;
. . .
a sum of grayscale values of the ninth pixel row I1, I2, I, I3, and I4 is 98+54+48+49+53=302.
By means of comparison, it may be learned that, the sum of grayscale values of the pixels in the fifth pixel row is 57, and the fifth pixel row includes a target pixel A. That is, the sum of grayscale values corresponding to the pixel row that includes the target pixel A is the minimum of the sums of grayscale values corresponding to all the pixel rows. Therefore, it is determined that the target pixel A is a fingerprint ridge point.
The pixels A1, A2, A3, and A4 in the fifth pixel row are all pixels in the ridge direction of the target pixel A. Pixels in another pixel row, for example, B1, B2, B, B3, and B4 in the fourth pixel row, are also in the ridge direction, and form a pixel row that is adjacent and parallel to the fifth pixel row in the normal direction. If the sum of grayscale values of the pixels in the fifth pixel row is less than the sum of grayscale values of the pixels in the fourth pixel row, it indicates that the color of the pixels in the fifth pixel row is generally darker than the color of the pixels in the fourth pixel row, so that the fifth pixel row better meets a feature of a fingerprint ridge. Therefore, a candidate fingerprint ridge formed by the fifth pixel row is definitely chosen over a candidate fingerprint ridge formed by the fourth pixel row as a fingerprint ridge. The target pixel A can be determined as a fingerprint ridge point in the fingerprint ridge.
In addition, in this embodiment, the rectangular region is defined, and the sums of grayscale values of the pixel rows are calculated and compared. Therefore, during determining of a fingerprint ridge point, not only the grayscale value of the target pixel A is used as a determining basis, but also a grayscale value of a pixel adjacent to the target pixel A in the ridge direction is used as a reference. For example, if a grayscale value of the target pixel A is relatively large but the grayscale values of A1, A2, A3, and A4 are relatively small, a general sum of grayscale values of the fifth pixel row is relatively small. This indicates that the target pixel A may be noise data, but the target pixel A may still be determined as a fingerprint ridge point. Therefore, interference from noise is eliminated, and recognition accuracy is improved.
As described above, after step S104 to step S108 are sequentially performed until pixels in the entire fingerprint image are traversed, fingerprint ridge points included in the fingerprint image can be determined. Grayscale values of fingerprint ridge points are set to a relatively large value (for example, 255), and grayscale values of pixels that are not fingerprint ridge points in the fingerprint image are set to a relatively small value (for example, 0), so that the fingerprint image is binarized, thereby facilitating the following recognition process of features included in the fingerprint image. For example, as shown in FIG. 4 and FIG. 5, FIG. 4 shows an input fingerprint image, FIG. 5 is a binarized image generated by traversing all pixels in the input fingerprint image. A grayscale value of a fingerprint ridge point is 255, and a grayscale value of a non-fingerprint ridge point is 0 (that is, color negative processing is performed).
Further, in this embodiment, the length of a long side of a rectangle may be set in advance. Specifically, the step of defining a rectangular region by using the target pixel as a central point is: calculating an average ridge distance of the fingerprint image; and setting a first quantity of the pixel rows according to the average ridge distance.
As shown in FIG. 6, a distance between a ridge and an adjacent furrow of a fingerprint is usually defined as a fingerprint ridge distance. From the perspective of technology implementation, the length between the center of a ridge and the center of a furrow is usually calculated as the ridge distance. It is assumed that there are n ridges whose widths are respectively w1, . . . , wn in the fingerprint image, and the ridges are defined separated by n-1 furrows whose widths are v1, . . . , vn. As shown in FIG. 6, the ridge distance is calculated as follows:
r i =w i/2+v i +w i+1/2.
The average ridge distance (the ridge distance for short hereinafter) of the fingerprint image refers to an average value of ridge distances in a particular region. Generally, the size of a ridge distance depends on a structure of the fingerprint and the resolution of image collection. In the fingerprint image, when a ridge distance is larger, it indicates that ridges there are sparser. In contrast, when a ridge distance is smaller, it indicates that ridges there are denser. Alternatively, the average ridge distance may be obtained by estimating ridge frequency.
Methods for estimating the ridge distance include a geometric method and a spectral analysis method. A method proposed by Yilong Yin may be used, and in the method, a ridge distance is estimated based on a statistics window. Alternatively, a method proposed by En Zhu may be used, and in the method, local regions are projected on a curve that passes through the centers of the regions, where tangential directions of all points are perpendicular to the ridge direction. In this way, the average ridge distance of the entire image is calculated.
In this embodiment, after the average ridge distance is estimated (the average ridge distance may be denoted by using a quantity of pixels), the setting a first quantity of the pixel rows according to the average ridge distance may be specifically: setting the first quantity of the pixel rows to be less than a quantity of pixels corresponding to the average ridge distance.
Referring to FIG. 2, FIG. 3, and FIG. 6, if the first quantity of the pixel rows is set to be relatively large, the defined rectangular region extends greatly in the normal direction of the target pixel A. In this case, the rectangular region may extend across two ridges. Consequently, pixels in a paralleled fingerprint ridge affect the determining of the target pixel A. After the first quantity of the pixel rows is set as the quantity of the pixels corresponding to the average ridge distance, the defined rectangular region may not cover multiple ridges in the normal direction of the target pixel A. Therefore, interference from another ridge is eliminated, and the accuracy of a binarized image is improved.
Correspondingly, in this embodiment, after the step of setting a quantity of the pixel rows according to a ridge interval length, a second quantity of pixels included in each pixel row may be set according to a quantity of the pixel rows. A value of the second quantity may be ⅓ to ⅔ of that of the first quantity.
That is, when the rectangular region is defined according to the target pixel, a quantity of pixels selected in the ridge direction of the target pixel may be less than pixels selected in the normal direction of the target pixel. That is, a side, extending in the normal direction of the target pixel, of the rectangular region is the long side of the rectangular region, and a side, extending in the ridge direction of the target pixel, of the rectangular region is the short side of the rectangular region. The length of the short side may be selected to be between ⅓ and ⅔ of the length of the long side.
If the second quantity is set to be relatively large, determining of a target pixel at a position where a fingerprint ridge bends sharply is easily interfered with by another fingerprint ridge point in the bent ridge. If the second quantity is set to be relatively small, a quantity of adjacent reference pixels on which the determining of the target pixel that is incorrectly collected as noise depends is reduced. Therefore, the target pixel that is incorrectly collected as the digital noise is easily ignored. If the value of the second quantity is set to be ⅓ to ⅔ of that of the first quantity, a balance can be reached between the first quantity and the second quantity, so that the accuracy of binarization is improved.
In one embodiment, during calculation of a sum of grayscale values of pixels in each pixel row, instead of using an algebraic sum method to calculate the sum of grayscale values of pixels in the pixel row, a corresponding weight coefficient is set for each pixel in the pixel row, and a weighted sum is calculated according to weight coefficients as the sum of grayscale values of pixels in the pixel row. Specifically, as shown in FIG. 7, the method includes the following steps.
Step S202: Traverse pixel rows, and identify a traversed pixel row as a target pixel row.
Step S204: Determine grayscale values of pixels in the traversed pixel row that is used as the target pixel row and corresponding weight coefficients.
Step S206: Multiply the grayscale values of the pixels in the traversed pixel row by the corresponding weight coefficients, and add the products to obtain a weighted sum.
For example, as shown in FIG. 3, for the first pixel row E1, E2, E, E3, and E4, if a weight coefficient corresponding to E1 and E4 is b1, a weight coefficient corresponding to E2 and E3 is b2, and a weight coefficient corresponding to E is b3, the sum of grayscale values of the first pixel row E1, E2, E, E3, and E4 is:
79×b 1+45×b 2+83×b 3+84×b 2+51×b 1.
An appropriate weight coefficient is set in advance for a pixel at each position in the pixel row, so that proportions accounted for by the grayscale value of each pixel in the calculated sum of grayscale values of the pixel row may be changed, and the pixels at some positions in the pixel row may have a greater impact on the sum of grayscale values of the entire pixel row.
In one embodiment, the weight coefficients of the pixels in the target pixel row decrement with a distance from a central pixel in the target pixel row.
In the foregoing example, compared with E1 and E4, E2 and E3 have a relatively strong impact on determining of whether E is a fingerprint ridge point. Therefore, the weight coefficient b1 corresponding to E1 and E4 may be set to be less than the weight coefficient b2 corresponding to E2 and E3. Compared with E2 and E3, the grayscale value of E has a relatively strong impact on determining of whether E is a fingerprint ridge point. Therefore, the weight coefficient b2 corresponding to E2 and E3 may be set to be less than the weight coefficient b3 corresponding to E.
The weight coefficient of each pixel in the target pixel row is set in a manner in which weight coefficients decrement with a distance from the central pixel in the target pixel row. In this case, a grayscale value of a pixel that is relatively close to the central pixel in the target pixel row (that is, the pixel and the target pixel are both in the normal direction of the target pixel) may have a relatively large impact on the sum of grayscale values of the target pixel row, and a grayscale value of a pixel that is relatively far from the central pixel in the target pixel row is enabled to have a relatively small impact on the sum of grayscale values of the target pixel row, thereby improving determining accuracy.
In one embodiment, after the step of identifying a pixel in the fingerprint image as a target pixel, a grayscale value of the target pixel may be determined. It is determined whether the grayscale value of the target pixel is greater than a threshold. If the grayscale value is greater than the threshold, it is determined that the target pixel is not a fingerprint ridge point.
That is, pre-estimation may be performed on the fingerprint image to directly determine that a pixel that clearly has a relatively large grayscale value is not a fingerprint ridge point. Therefore, the step of defining a rectangular region is omitted, thereby increasing a recognition speed.
To improve accuracy of fingerprint recognition, in an embodiment, as shown in FIG. 8, an apparatus for recognizing a fingerprint ridge point is further provided. The apparatus includes a module for receiving a fingerprint image 102, a module for acquiring a pixel direction 104, a module for selecting a reference pixel 106, and a module for determining a fingerprint ridge point 108.
The module for receiving a fingerprint image 102 is configured to: receive an input fingerprint image, and perform orientation field estimation on the fingerprint image.
The module for acquiring a pixel direction 104 is configured to: traverse a pixel in the fingerprint image as a target pixel, and determine a ridge direction, obtained according to orientation field estimation, of the target pixel and a normal direction perpendicular to the ridge direction.
The module for selecting a reference pixel 106 is configured to: define a rectangular region by using the target pixel as a central point, where lengthwise and widthwise extending directions of the rectangular region are respectively corresponding to the ridge direction and the normal direction of the target pixel, and the rectangular region includes multiple pixel rows arranged in the normal direction.
The module for determining a fingerprint ridge point 108 is configured to: calculate a sum of grayscale values of pixels in each pixel row, and determine that the target pixel is a fingerprint ridge point when a sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum.
In this embodiment, the module for selecting a reference pixel 106 is further configured to: calculate an average ridge distance of the fingerprint image, and set a first quantity of the pixel rows according to the average ridge distance.
In this embodiment, the module for selecting a reference pixel 106 is further configured to set, according to a quantity of the pixel rows, a second quantity of pixels included in each pixel row, where a value of the second quantity is ⅓ to ⅔ of that of the first quantity.
In this embodiment, the module for determining a fingerprint ridge point 108 is further configured to: traverse pixel rows, and identify a traversed pixel row as a target pixel row; determine grayscale values of pixels in the traversed pixel row that is used as the target pixel row and corresponding weight coefficients; and multiply the grayscale values of the pixels in the traversed pixel row by the corresponding weight coefficients, and add the products to obtain a weighted sum.
In this embodiment, as shown in FIG. 8, the apparatus for recognizing a fingerprint ridge point further includes a pre-filtering module 110, configured to: determine a grayscale value of the target pixel; determine whether the grayscale value of the target pixel is greater than a threshold; and if the grayscale value is greater than the threshold, determine that the target pixel is not a fingerprint ridge point.
By means of the method and the apparatus for recognizing a fingerprint ridge point, during binarization of a fingerprint image, pixels in the fingerprint image are traversed. A rectangular region is defined by using a traversed pixel as a central point. Lengthwise and widthwise extending directions of the rectangular region are respectively corresponding to a ridge direction and a normal direction of a target pixel, and the rectangular region includes multiple pixel rows arranged in the normal direction. A sum of grayscale values of pixels in each pixel row is calculated, and it is determined that the target pixel is a fingerprint ridge point when a sum of grayscale values corresponding to a pixel row that includes the target pixel is the minimum. Compared with a method of using a filter or threshold filtering, both a grayscale value of a traversed pixel and a grayscale value of a pixel at an adjacent position of the traversed pixel are used as references, so that incorrectly determined noise is effectively eliminated, and recognition accuracy is improved.
In an embodiment, as shown in FIG. 9, FIG. 9 shows a computer system 10 that is based on the von Neumann architecture and performs the foregoing method for recognizing a fingerprint ridge point. The computer system 10 may be a terminal device such as a smartphone, a tablet computer, a palmtop computer, a notebook computer or a personal computer. Specifically, the computer system 10 may include an external input interface 1001, a processor 1002, a memory 1003, and an output interface 1004 that are connected through a system bus. The external input interface 1001 may optionally include at least one of a mouse 10012, a keyboard 10014, a touchpad 10016 or a fingerprint sensor interface 10018. The memory 1003 may include an external memory 10032 (for example a hard disk, an optical disk or a floppy disk) and an internal memory 10034. The output interface 1004 may include devices such as a display screen 10042 and a speaker 10044.
In this embodiment, the method is performed based on a computer program. A program file of the computer program may be stored in the external memory 10032 of the computer system 10 and is loaded to the internal memory 10034 when being run. The program file may then be compiled into machine code, and the machine code is transmitted to the processor 1002 and executed by the processor 1002. In this way, a logic module for receiving a fingerprint image 102, a module for acquiring a pixel direction 104, a module for selecting a reference pixel 106, and a module for determining a fingerprint ridge point 108 may be formed in the computer system 10, and implement their the functions as described in relation to FIG. 8. In addition, in a process of performing the method for recognizing a fingerprint ridge point, input parameters may all be received by using the external input interface 1001, transmitted to and cached in the memory 1003, and then input to the processor 1002 to be processed. Result data of processing may be cached in the memory 1003 for subsequent processing, or may be transmitted to the output interface 1004 for output.
What is disclosed above is merely preferred embodiments of the present disclosure, and certainly is not intended to limit the protection scope of the present disclosure. Therefore, equivalent variations made in accordance with the claims of the present disclosure shall fall within the scope of the present disclosure.

Claims (17)

What is claimed is:
1. A method for recognizing a fingerprint ridge point, the method comprising:
receiving a fingerprint image;
performing orientation field estimation on the fingerprint image;
identifying a target pixel in the fingerprint image;
determining a ridge line including the target pixel and extending in a ridge direction, according to the orientation field estimation, and a normal line extending in a normal direction perpendicular to the ridge direction;
defining a rectangular region by using the target pixel as a central point, a length of the rectangular region being along the ridge direction, a width of the rectangular region being along the normal direction, and the rectangular region including a first number of pixel rows arranged in the normal direction, wherein the defining includes:
calculating an average ridge distance of the fingerprint image;
determining the first number of pixel rows according to the average ridge distance;
calculating a sum of grayscale values of pixels in each of the first number of pixel rows; and
determining that the target pixel is a fingerprint ridge point when the sum of grayscale values corresponding to a pixel row that includes the target pixel is the smallest among the first number of pixel rows.
2. The method for recognizing a fingerprint ridge point according to claim 1, the method further comprises:
setting, according to the first number of pixel rows, a second number of pixels included in each of the first number of pixel rows, wherein the second number is ⅓ to ⅔ of the first number.
3. The method for recognizing a fingerprint ridge point according to claim 1, wherein calculating the sum of grayscale values of pixels in each of the first number of pixel rows comprises:
traversing the first number of pixel rows to identify a traversed pixel row as a target pixel row;
determining grayscale values of pixels in the traversed pixel row that is used as the target pixel row and corresponding weight coefficients; and
multiplying the grayscale values of the pixels in the traversed pixel row by the corresponding weight coefficients to obtain a weighted sum.
4. The method for recognizing a fingerprint ridge point according to claim 3, wherein values of weight coefficients of the pixels in the target pixel row decrease as distances from pixels to a central pixel in the target pixel row increase.
5. The method for recognizing a fingerprint ridge point according to claim 1, wherein after the step of identifying the target pixel in the fingerprint image, the method further comprises:
determining a grayscale value of the target pixel;
determining whether the grayscale value of the target pixel is greater than a threshold; and
if the grayscale value is greater than the threshold, determining that the target pixel is not the fingerprint ridge point.
6. An apparatus for recognizing a fingerprint ridge point, the apparatus comprising a memory and a processor coupled to the memory, the processor being configured to:
receive a fingerprint image;
perform orientation field estimation on the fingerprint image;
identify a target pixel in the fingerprint image;
determine a ridge line including the target pixel and extending in a ridge direction, according to the orientation field estimation, and a normal line extending in a normal direction perpendicular to the ridge direction;
define a rectangular region by using the target pixel as a central point, a length of the rectangular region being along the ridge direction, a width of the rectangular region being along the normal direction, and the rectangular region including a first number of pixel rows arranged in the normal direction, wherein the step to define the rectangular region includes
calculating an average ridge distance of the fingerprint image;
determining the first number of pixel rows according to the average ridge distance;
calculate a sum of grayscale values of pixels in each of the first number of pixel rows; and
determine that the target pixel is a fingerprint ridge point when the sum of grayscale values corresponding to a pixel row that includes the target pixel is the smallest among the first number of pixel rows.
7. The apparatus for recognizing a fingerprint ridge point according to claim 6, wherein the processor is further configured to:
set, according to the first number of pixel rows, a second number of pixels included in each of the first number of pixel rows, wherein the second number is ⅓ to ⅔ of the first number.
8. The apparatus for recognizing a fingerprint ridge point according to claim 6, wherein the processor is further configured to:
traverse the first number of pixel rows to identify a traversed pixel row as a target pixel row;
determine grayscale values of pixels in the traversed pixel row that is used as the target pixel row and corresponding weight coefficients; and
multiply the grayscale values of the pixels in the traversed pixel row by the corresponding weight coefficients to obtain a weighted sum.
9. The apparatus for recognizing a fingerprint ridge point according to claim 6, wherein the processor is further configured to:
determine a grayscale value of the target pixel;
determine whether the grayscale value of the target pixel is greater than a threshold; and
if the grayscale value is greater than the threshold, determine that the target pixel is not the fingerprint ridge point.
10. A non-transitory computer readable media containing program instructions, when executed by one or more processor, the programs instructions perform a method for recognizing a fingerprint ridge point, the method comprising:
performing orientation field estimation on a fingerprint image;
identifying a target pixel in the fingerprint image;
determining a ridge line including the target pixel and extending in a ridge direction, according to the orientation field estimation, and a normal line extending in a normal direction perpendicular to the ridge direction;
defining a rectangular region by using the target pixel as a central point, a length of the rectangular region being along the ridge direction, a width of the rectangular region being along the normal direction, and the rectangular region including a first number of pixel rows arranged in the normal direction, wherein the defining includes:
calculating an average ridge distance of the fingerprint image;
determining the first number of pixel rows according to the average ridge distance;
calculating a sum of grayscale values of pixels in each of the first number of pixel rows; and
determining that the target pixel is a fingerprint ridge point when the sum of grayscale values corresponding to a pixel row that includes the target pixel is the smallest among the first number of pixel rows.
11. The non-transitory computer readable medium according to claim 10, the method further comprises:
setting, according to the first number of pixel rows, a second number of pixels included in each of the first number of pixel rows, wherein the second number is ⅓ to ⅔ of the first number.
12. The non-transitory computer readable medium according to claim 10, wherein calculating the sum of grayscale values of pixels in each of the first number of pixel rows comprises:
traversing the first number of pixel rows to identify a traversed pixel row as a target pixel row;
determining grayscale values of pixels in the traversed pixel row that is used as the target pixel row and corresponding weight coefficients; and
multiplying the grayscale values of the pixels in the traversed pixel row by the corresponding weight coefficients to obtain a weighted sum.
13. The non-transitory computer readable medium according to claim 12, wherein values of weight coefficients of the pixels in the target pixel row decrease as distances from pixels to a central pixel in the target pixel row increase.
14. The computer readable medium according to claim 10, wherein after identifying the target pixel in the fingerprint image, the method further comprises:
determining a grayscale value of the target pixel;
determining whether the grayscale value of the target pixel is greater than a threshold; and
if the grayscale value is greater than the threshold, determining that the target pixel is not the fingerprint ridge point.
15. The method according to claim 1, further comprising:
subjecting the fingerprint image to illumination normalization to reduce impact of light.
16. The apparatus according to claim 6, wherein the processor is further configured to:
subject the fingerprint image to illumination normalization to reduce impact of light.
17. The non-transitory computer readable media according to claim 10, wherein the method further comprises:
subjecting the fingerprint image to illumination normalization to reduce impact of light.
US15/841,055 2015-09-08 2017-12-13 Method and apparatus for recognizing fingerprint ridge point Active 2036-08-09 US10558841B2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201510566721 2015-09-08
CN201510566721.3 2015-09-08
CN201510566721.3A CN106503609B (en) 2015-09-08 2015-09-08 Fingerprint ridge point identification method and device
PCT/CN2016/079584 WO2017041488A1 (en) 2015-09-08 2016-04-18 Fingerprint ridge point recognition method and apparatus

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/079584 Continuation WO2017041488A1 (en) 2015-09-08 2016-04-18 Fingerprint ridge point recognition method and apparatus

Publications (2)

Publication Number Publication Date
US20180101710A1 US20180101710A1 (en) 2018-04-12
US10558841B2 true US10558841B2 (en) 2020-02-11

Family

ID=58239186

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/841,055 Active 2036-08-09 US10558841B2 (en) 2015-09-08 2017-12-13 Method and apparatus for recognizing fingerprint ridge point

Country Status (6)

Country Link
US (1) US10558841B2 (en)
EP (1) EP3349140B1 (en)
JP (1) JP6487576B2 (en)
KR (1) KR101939936B1 (en)
CN (1) CN106503609B (en)
WO (1) WO2017041488A1 (en)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107341482B (en) * 2017-07-13 2020-07-10 海信视像科技股份有限公司 Fingerprint identification method and device and computer readable storage medium
CN109426795A (en) * 2017-09-05 2019-03-05 比亚迪股份有限公司 Fingerprint identification method and device
US11610430B2 (en) * 2018-06-19 2023-03-21 Sony Corporation Information processing apparatus, wearable device, information processing method, and program
CN109976615B (en) * 2019-03-28 2021-03-12 Oppo广东移动通信有限公司 Fingerprint image processing method and related device
KR102223320B1 (en) * 2019-08-13 2021-03-05 숭실대학교산학협력단 Apparatus and method for determining status of vehicle LED lighting based on comparison of adjacent grayscale
JP6927611B1 (en) * 2020-03-16 2021-09-01 Ｎｅｃプラットフォームズ株式会社 Feature extraction device, feature extraction method, and program
CN113408416B (en) * 2021-06-18 2023-04-07 展讯通信（上海）有限公司 Fingerprint frequency estimation method and device and fingerprint information extraction method and device
CN113361448A (en) * 2021-06-23 2021-09-07 浙江中正智能科技有限公司 Method for detecting interference straight line in fingerprint image
CN115690857A (en) * 2021-07-28 2023-02-03 比亚迪半导体股份有限公司 Method and device for detecting fingerprint peeling, fingerprint input method and device and chip
KR102614078B1 (en) * 2021-12-22 2023-12-15 주식회사 라이프사이언스테크놀로지 Apparatus and method of personal identification using biometric signal
CN114445867B (en) * 2022-02-21 2024-07-23 厦门天马微电子有限公司 Fingerprint identifier and display device
CN114663927A (en) * 2022-03-25 2022-06-24 合肥维信诺科技有限公司 Fingerprint image processing method, device, storage medium and electronic device
CN115953808A (en) * 2022-12-14 2023-04-11 昆山国显光电有限公司 Fingerprint direction field calculation method and device, computer equipment and storage medium
FR3146740B1 (en) * 2023-03-17 2025-02-21 Idemia Identity & Security France method of estimating an inter-ridge distance relating to a fingerprint
CN116935493B (en) * 2023-09-13 2024-01-02 成都市青羊大数据有限责任公司 Education management system
CN117496560B (en) * 2023-12-29 2024-04-19 深圳市魔力信息技术有限公司 Fingerprint line identification method and device based on multidimensional vector

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6233348B1 (en) * 1997-10-20 2001-05-15 Fujitsu Limited Fingerprint registering apparatus, fingerprint identifying apparatus, and fingerprint identifying method
KR100391182B1 (en) 2000-09-25 2003-07-16 주식회사 디토정보기술 The method of a fingerprint minutia extraction using direct valley following
US20040081360A1 (en) * 2002-10-28 2004-04-29 Lee Shih-Jong J. Fast pattern searching
US20050207627A1 (en) * 2004-03-22 2005-09-22 Lg Electronics Inc. Fingerprint recognition system and method
US20050264878A1 (en) * 2003-06-17 2005-12-01 Cross Match Technologies, Inc. System for print imaging with prism illumination optics
US20060153431A1 (en) * 2003-01-06 2006-07-13 Tatsunobu Ando Fingerprint matching device and method, recording medium, and program
CN101079102A (en) 2007-06-28 2007-11-28 中南大学 Fingerprint identification method based on statistic method
EP2104059B1 (en) 2008-03-14 2011-01-12 NEC Corporation Apparatus and method for processing image
CN101996321A (en) 2009-08-24 2011-03-30 北京易创科技有限公司 Fingerprint recognition pretreatment method and device
CN102510330A (en) 2011-11-02 2012-06-20 杭州电子科技大学 Novel fuzzy vault method based on fingerprint characteristic data and matching algorithm
US20150063699A1 (en) * 2013-08-30 2015-03-05 Konica Minolta Laboratory U.S.A., Inc. Line segmentation method applicable to document images containing handwriting and printed text characters or skewed text lines
US9542585B2 (en) * 2013-06-06 2017-01-10 Apple Inc. Efficient machine-readable object detection and tracking
US20180018497A1 (en) * 2015-02-13 2018-01-18 Byd Company Limited Method and device for calculating line distance
US20180089484A1 (en) * 2015-03-31 2018-03-29 Nec Corporation Biological pattern information processing device, biological pattern information processing method, and program

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2833313B2 (en) * 1992-01-10 1998-12-09 富士通株式会社 Authenticator for fingerprint feature points
JP2725599B2 (en) * 1994-06-21 1998-03-11 日本電気株式会社 Ridge direction extraction device
JP4020202B2 (en) * 2003-06-16 2007-12-12 三菱電機株式会社 Striped pattern extraction system and striped pattern extraction method
JP4403513B2 (en) * 2005-08-09 2010-01-27 日本電気株式会社 Fingerprint ridge recognition device, fingerprint ridge recognition method, and program
CN100470579C (en) * 2007-09-21 2009-03-18 清华大学 Deterministic coding method and system in fingerprint identification
JP5273801B2 (en) * 2009-05-13 2013-08-28 Ｎｅｃソフト株式会社 Fingerprint feature point determination system, fingerprint feature point determination method, fingerprint feature point determination program

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6233348B1 (en) * 1997-10-20 2001-05-15 Fujitsu Limited Fingerprint registering apparatus, fingerprint identifying apparatus, and fingerprint identifying method
KR100391182B1 (en) 2000-09-25 2003-07-16 주식회사 디토정보기술 The method of a fingerprint minutia extraction using direct valley following
US20040081360A1 (en) * 2002-10-28 2004-04-29 Lee Shih-Jong J. Fast pattern searching
US20060153431A1 (en) * 2003-01-06 2006-07-13 Tatsunobu Ando Fingerprint matching device and method, recording medium, and program
US20050264878A1 (en) * 2003-06-17 2005-12-01 Cross Match Technologies, Inc. System for print imaging with prism illumination optics
US20050207627A1 (en) * 2004-03-22 2005-09-22 Lg Electronics Inc. Fingerprint recognition system and method
CN101079102A (en) 2007-06-28 2007-11-28 中南大学 Fingerprint identification method based on statistic method
EP2104059B1 (en) 2008-03-14 2011-01-12 NEC Corporation Apparatus and method for processing image
CN101996321A (en) 2009-08-24 2011-03-30 北京易创科技有限公司 Fingerprint recognition pretreatment method and device
CN102510330A (en) 2011-11-02 2012-06-20 杭州电子科技大学 Novel fuzzy vault method based on fingerprint characteristic data and matching algorithm
US9542585B2 (en) * 2013-06-06 2017-01-10 Apple Inc. Efficient machine-readable object detection and tracking
US20150063699A1 (en) * 2013-08-30 2015-03-05 Konica Minolta Laboratory U.S.A., Inc. Line segmentation method applicable to document images containing handwriting and printed text characters or skewed text lines
US20180018497A1 (en) * 2015-02-13 2018-01-18 Byd Company Limited Method and device for calculating line distance
US20180089484A1 (en) * 2015-03-31 2018-03-29 Nec Corporation Biological pattern information processing device, biological pattern information processing method, and program

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Korean Intellectual Property Office (KIPO) Office Action 1 for 10-2017-7035024 dated Apr. 14, 2018 11 Pages (including translation).
Ratha et al., Adaptive flow orientation based feature extraction in fingerprint images, Feb. 12, 1995. *
The World Intellectual Property Organization (WIPO) International Search Report for PCT/CN2016/079584 dated Jul. 20, 2016 5 Pages (including translation).

Also Published As

Publication number Publication date
EP3349140A4 (en) 2019-04-24
US20180101710A1 (en) 2018-04-12
JP2018524663A (en) 2018-08-30
JP6487576B2 (en) 2019-03-20
EP3349140B1 (en) 2023-06-28
WO2017041488A1 (en) 2017-03-16
CN106503609B (en) 2019-07-16
KR101939936B1 (en) 2019-01-17
EP3349140A1 (en) 2018-07-18
KR20180003595A (en) 2018-01-09
CN106503609A (en) 2017-03-15

Similar Documents

Publication Publication Date Title
US10558841B2 (en) Method and apparatus for recognizing fingerprint ridge point
CN112418150B (en) Palm vein image evaluation method, palm vein image evaluation device, computer equipment and storage medium
US11874907B2 (en) Method for enhancing fingerprint image, identifying fingerprint and starting-up application program
US9633269B2 (en) Image-based liveness detection for ultrasonic fingerprints
US8401250B2 (en) Detecting objects of interest in still images
US11475707B2 (en) Method for extracting image of face detection and device thereof
CN110287791B (en) Screening method and system for face pictures
WO2020244071A1 (en) Neural network-based gesture recognition method and apparatus, storage medium, and device
CN106203326B (en) A kind of image processing method, device and mobile terminal
WO2022237397A1 (en) Method and apparatus for evaluating image authenticity, computer device, and storage medium
US11869271B2 (en) Electronic apparatus and method of processing fingerprint image
WO2016192213A1 (en) Image feature extraction method and device, and storage medium
TWI727514B (en) Fingerprint identification method, device, storage medium, and terminal
US10990796B2 (en) Information processing apparatus, image processing method and recording medium on which image processing program is recorded
CN106778631B (en) A design method of heterogeneous iris classifier for fast filtering of fake irises during iris recognition preprocessing
EP3125194A1 (en) Biometric authentication device, biometric authentication method, and program
CN112801034A (en) Finger vein recognition device
KR101750897B1 (en) A Biometric Method using Thumb feature
US10825144B2 (en) Image processing apparatus and image processing method
CN112861756A (en) Finger vein identification method and system

Legal Events

Date Code Title Description
AS Assignment

Owner name: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED, CHINA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WANG, HONGFA;REEL/FRAME:044390/0407

Effective date: 20171207

Owner name: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED, CHI

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WANG, HONGFA;REEL/FRAME:044390/0407

Effective date: 20171207

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: AWAITING TC RESP., ISSUE FEE NOT PAID

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4