CN109815850B - Iris image segmentation and positioning method, system and device based on deep learning - Google Patents

Iris image segmentation and positioning method, system and device based on deep learning Download PDF

Info

Publication number
CN109815850B
CN109815850B CN201910002663.XA CN201910002663A CN109815850B CN 109815850 B CN109815850 B CN 109815850B CN 201910002663 A CN201910002663 A CN 201910002663A CN 109815850 B CN109815850 B CN 109815850B
Authority
CN
China
Prior art keywords
iris
image
map
segmentation
boundary
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910002663.XA
Other languages
Chinese (zh)
Other versions
CN109815850A (en
Inventor
孙哲南
谭铁牛
王财勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Automation of Chinese Academy of Science
Original Assignee
Institute of Automation of Chinese Academy of Science
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Automation of Chinese Academy of Science filed Critical Institute of Automation of Chinese Academy of Science
Priority to CN201910002663.XA priority Critical patent/CN109815850B/en
Publication of CN109815850A publication Critical patent/CN109815850A/en
Application granted granted Critical
Publication of CN109815850B publication Critical patent/CN109815850B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

The invention belongs to the field of mode recognition, computer vision and image processing, and particularly relates to a method, a system and a device for iris image segmentation and positioning based on deep learning, aiming at solving the problem of low iris recognition accuracy in an uncontrollable scene. The method comprises the following steps: acquiring an iris image to be processed; generating four mapping maps by adopting a multitask neural network model, wherein the four mapping maps respectively correspond to a pupil center, an iris inner boundary, an iris outer boundary and an iris segmentation mask; processing the iris segmentation mask mapping map by adopting threshold segmentation to complete iris segmentation; predicting the pupil center position according to the geometric relation between the pupil center and the iris mask; and denoising and calculating the mapping map by using the geometric relationship among the pupil, the iris and the sclera to obtain the inner and outer circle parameters of the iris so as to complete the iris positioning. The invention can effectively divide and position the iris image collected in the non-controllable environment, lays a good foundation for subsequent normalization and identification, and improves the accuracy of iris identification in the non-controllable environment.

Description

Iris image segmentation and positioning method, system and device based on deep learning
Technical Field
The invention belongs to the field of pattern recognition, computer vision and image processing, and particularly relates to an iris image segmentation and positioning method, system and device based on deep learning.
Background
With the rise of artificial intelligence, biometric identification technologies such as face identification, iris identification and fingerprint identification have received great attention, wherein the iris identification technology is considered to be one of the most stable, most accurate and most reliable verification methods, and therefore is widely applied to the fields of intelligent unlocking, border control, bank finance, access control and attendance checking and the like.
In the iris recognition system, iris segmentation and positioning are in the initial part of the whole processing flow, so the accuracy of the iris segmentation and positioning directly influences the precision of subsequent processing. The iris segmentation means extracting effective iris texture pixels, eliminating noise, and finally outputting a binary segmentation mask. Iris localization refers to accurately locating the inner and outer circular boundary parameters of an iris. The result of the iris localization is used in the iris normalization operation; and mask segmentation of the iris will be related to the processing of noisy regions of the iris image.
Conventional iris segmentation positioning methods are generally combined and may be collectively referred to as iris segmentation. Typical segmentation methods can be divided into two broad categories: firstly, an edge-based method needs to respectively position the inner and outer edges of an iris, upper and lower eyelids, remove eyelash occlusion and the like to obtain an isolated iris area; and the other is a pixel-based method, which directly distinguishes iris pixels from non-iris pixels according to appearance characteristics, such as color, texture and the like, near the pixel points. Generally speaking, these methods usually rely on a large amount of prior knowledge, and many intermediate operations, and the process is complex, and is often only suitable for iris images with clear inner and outer boundaries of the iris. With the wide-range popularization of remote iris recognition, mobile-end iris recognition and the like, the acquired iris images often contain various noises such as specular reflection, oblique eyes, blurring, glasses shielding and the like due to illumination, target motion and distance change, and the traditional method cannot well process the images.
Therefore, in order to effectively perform accurate preprocessing operation on the iris image acquired in the non-controllable scene, a new, accurate and efficient iris image segmentation and positioning method needs to be developed urgently, so that the requirements of users are met, and the iris identification precision in the non-controllable scene is effectively improved.
Disclosure of Invention
In order to solve the above problem in the prior art, that is, the problem of low iris recognition accuracy in an uncontrollable scene, the present invention provides an iris image segmentation method based on deep learning, which includes:
step S10, acquiring an iris image to be processed as a first iris image;
step S20, inputting the first iris image into a trained full convolution coding and decoding multitask neural network model for forward propagation to obtain a first pupil center mapping map, a first iris inner boundary mapping map, a first iris outer boundary mapping map and a first iris segmentation mask mapping map;
step S30, processing the first iris segmentation mask map by threshold segmentation to obtain a binary iris segmentation mask map and finish iris segmentation;
the trained full-convolution coding and decoding multitask neural network model is obtained in the following mode:
step S21, acquiring an iris image as a second iris image, marking the second iris image and acquiring a third iris image; integrating an attention mechanism, and constructing a full-convolution encoding and decoding multitask neural network;
and step S22, inputting the third iris image into the full-convolution encoding and decoding multitask neural network for training to obtain a trained full-convolution encoding and decoding multitask neural network model.
In some preferred embodiments, step S21 "mark second iris image" includes the steps of:
step S211, marking effective iris pixels in the second iris image by adopting a binary code label; the valid iris pixels are marked as 1, and the rest parts are marked as 0; using two ellipses closest to the inner and outer boundaries of the iris as marks of the inner and outer boundaries of the iris; the boundary point is marked as 1, and the rest parts are marked as 0;
the effective iris pixel does not contain noise such as mirror reflection and non-iris areas such as hair, eyelashes, pupils and sclera.
Step S212, marking the marked center of the inner elliptical boundary of the iris as the center of the pupil, and obtaining an iris image comprising an effective iris distinguishing mark, inner and outer boundary marks of the iris and a pupil center mark as a third iris image; the pupil center of the iris image is marked as 1, and the rest parts are marked as 0.
In some preferred embodiments, the fully convolutional codec multitasking neural network includes an encoding layer and a decoding layer.
The encoding layer encodes features through a plurality of successive convolution, ReLu, and pooling operations, and successively reduces the resolution of the map of the original input image to obtain a low-resolution map.
The decoding layer performs jump connection through a layer corresponding to the decoding layer, performs resolution recovery on the low-resolution mapping image after the coding layer is continuously reduced by using bilinear upsampling, and outputs four mapping images with the same size as the original input, corresponding to a pupil center mapping image, an iris inner boundary mapping image, an iris outer boundary mapping image and an iris segmentation mask mapping image.
In some preferred embodiments, the step S22 of inputting the third iris image into the fully-convolutional codec multitask neural network for training includes:
step S221, inputting the third iris image into the full-convolution encoding and decoding multitask neural network, and obtaining a second pupil center mapping map, a second iris inner boundary mapping map, a second iris outer boundary mapping map and a second iris segmentation mask mapping map through forward propagation.
Step S222, calculating errors of the second pupil center mapping map, the second iris inner boundary mapping map, the second iris outer boundary mapping map, the second iris segmentation mask mapping map and the third iris image, and updating parameters of the full-convolution encoding and decoding multitask neural network by using a back propagation algorithm and a random gradient descent method based on the calculated total loss function value.
And step S223, repeating the step S222 until the total loss function calculation value reaches a preset condition, and obtaining the trained full-convolution encoding and decoding multitask neural network model.
In some preferred embodiments, the overall loss function includes a focus loss for the pupil center, a cross-entropy loss for the balance of the inner and outer boundaries of the iris, and a cross-entropy loss for the segmentation;
the focal loss L for the pupil centerpupilComprises the following steps:
Figure GDA0002620910390000041
Figure GDA0002620910390000042
wherein, alpha and gamma are hyper-parameters; p ═ Pj,j=1,...,|X|},pjThe probability that the jth pixel predicted by the multitask network is the pupil center, and | X | is the number of pixels of the iris image;
Figure GDA0002620910390000043
Figure GDA0002620910390000044
for the label of the jth pixel in the iris image, 1 is expressed as the true iris-pupil center, and 0 is not.
The cross entropy loss L for the balance of the inner and outer boundaries of the irisedgeComprises the following steps:
Figure GDA0002620910390000045
wherein the content of the first and second substances,
Figure GDA0002620910390000046
Figure GDA0002620910390000047
representing the probability that the jth pixel in the iris image belongs to the inner iris boundary or the outer iris boundary, wherein k is 1 for the inner iris boundary, k is 2 for the outer iris boundary, 1 is represented as the real iris boundary, and 0 is not; beta is the proportion of non-edge pixels in the iris image, and the non-edge pixels are pixels except the inner boundary and the outer boundary of the iris.
The cross entropy loss for segmentation is LsegComprises the following steps:
Figure GDA0002620910390000051
wherein S ═ { S ═ Sj,j=1,...|X|},sjRepresenting the probability that the jth pixel in the iris image belongs to the real iris texture;
Figure GDA0002620910390000052
Figure GDA0002620910390000053
a label indicating that the jth pixel in the iris image belongs to the real iris texture.
In another aspect of the present invention, an iris image positioning method based on deep learning is provided, including:
adopting the steps S10-S20 of the iris image segmentation method based on the deep learning, acquiring a first pupil center mapping map, a first iris inner boundary mapping map, a first iris outer boundary mapping map and a first iris segmentation mask mapping map, and executing the following steps:
step B10, obtaining the predicted position of the pupil center according to the geometric relationship between the first pupil center mapping map and the first iris segmentation mask mapping map;
b20, denoising the first iris inner and outer boundary mapping map based on the preset geometric relationship among the pupil, the iris and the sclera to obtain a first iris inner and outer circle radius parameter set and a first iris inner and outer circle center initial coordinate;
and step B30, acquiring fitting points of the inner and outer contours of the iris by using a Viterbi algorithm based on the denoised inner and outer boundary mapping chart of the first iris, the first inner and outer circle radius parameter set and the initial coordinates of the center of the inner and outer circles of the first iris, and fitting the inner and outer circles by using a least square circle fitting algorithm to obtain inner and outer circle parameters of the iris so as to complete iris positioning.
In some preferred embodiments, in step B10, "obtaining the predicted position of the pupil center according to the geometric relationship between the first pupil center map and the first iris segmentation mask map" includes:
and performing threshold segmentation and connected region analysis on the first pupil center mapping map and the first iris segmentation mask mapping map, searching a connected sub region of the pupil center closest to the connected sub region of the maximum area segmentation mask, and taking the center point of the connected sub region as the finally predicted pupil center.
In some preferred embodiments, in step B20, "denoising the first inner and outer iris boundary map based on a preset geometric relationship among the pupil, the iris, and the sclera, to obtain a first inner and outer iris radius parameter set and a first inner and outer iris center start coordinate", includes the steps of:
step B21, generating a limited area by taking the predicted position of the pupil center as a circle center and the maximum distance from the pupil center to the largest-area connected subregion of the segmentation mask as a radius;
step B22, deleting the iris outer boundary mapping map connected region inside and outside the limited region to obtain a refined iris outer boundary; deleting the connected region of the iris inner boundary mapping map outside the limited region to obtain a refined iris inner boundary;
and step B23, taking the maximum and minimum distance from the pupil center to the refined inner and outer iris boundaries as a first inner and outer iris radius parameter set, wherein the pupil center coordinate is the initial coordinate of the first inner and outer iris center.
The invention provides an iris image segmentation and positioning system based on deep learning, which comprises an input module, a multitask neural network module, a threshold segmentation module, a pupil position prediction module, a denoising module, a fitting module and an output module, wherein the input module is used for inputting an iris image;
the input module is configured to acquire an iris image to be tested as a first iris image;
the multitask neural network module is configured to process the first iris image to obtain a first pupil center mapping map, a first iris inner boundary mapping map, a first iris outer boundary mapping map and a first iris segmentation mask mapping map;
the threshold segmentation module is configured to process the first iris segmentation mask map by threshold segmentation to obtain a binary iris segmentation mask map and complete iris segmentation;
the pupil position prediction module is configured to obtain a predicted position of a pupil center according to a geometric relationship between the first pupil center mapping map and the first iris segmentation mask mapping map;
the denoising module is configured to denoise the first iris inner and outer boundary mapping map based on a preset geometric relationship among the pupil, the iris and the sclera to obtain a first iris inner and outer circle radius parameter set and a first iris inner and outer circle center initial coordinate;
the fitting module is configured to obtain fitting points of inner and outer contours of the iris by using a Viterbi algorithm based on the denoised first iris inner and outer boundary mapping map, the first iris inner and outer circle radius parameter set and the first iris inner and outer circle center initial coordinates, and fit the inner and outer circles by using a least square circle fitting algorithm to obtain iris inner and outer circle parameters so as to complete iris positioning;
and the output module is configured to output the binary iris segmentation mask map and the inner and outer iris circle parameters.
In a fourth aspect of the present invention, a storage device is provided, in which a plurality of programs are stored, the programs being suitable for being loaded and executed by a processor to implement the iris image segmentation and positioning method based on deep learning described above.
In a fifth aspect of the present invention, a processing apparatus is provided, which includes a processor, a storage device; the processor is suitable for executing various programs; the storage device is suitable for storing a plurality of programs; the program is suitable to be loaded and executed by a processor to realize the iris image segmentation and positioning method based on deep learning.
The invention has the beneficial effects that:
(1) compared with the prior art, the method is based on the deep learning framework, can accurately and robustly segment and position the iris image acquired in the uncontrolled scene, meets the requirements of users, can improve the accuracy of iris recognition, and has great production practice significance.
(2) Based on a deep learning framework, the multi-task network provided by the invention effectively captures the geometric correlation of the multi-modal eye structure, and meanwhile, the attention mechanism provided by the invention is combined with an ASPP module of deep Lab or a pyramid pooling module of PSPNet, so that the network can capture the characteristics of the optimal discrimination and finally output accurate segmentation masks and other modal structures. The post-processing operation provided subsequently can effectively eliminate the interference of noise and accurately position the inner and outer circles of the iris.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is a flow chart of the iris image segmentation and localization method based on deep learning of the present invention;
FIG. 2 is a schematic diagram of an iris image acquired in a non-controllable scene according to an embodiment of the iris image segmentation and localization method based on deep learning of the present invention;
FIG. 3 is a diagram of a multitasking network structure of an embodiment of the deep learning-based iris image segmentation and positioning method of the present invention;
FIG. 4 is a schematic structural diagram of an ASPP module integrating an attention mechanism and a Deeplab according to an embodiment of the iris image segmentation and localization method based on deep learning;
FIG. 5 is a schematic structural diagram of an integrated attention mechanism and PSPNet pyramid pooling module according to an embodiment of the deep learning-based iris image segmentation and localization method of the present invention;
FIG. 6 is a schematic diagram of four mapping maps outputted by a multitask network and comparing with an original iris image according to an embodiment of the iris image segmentation and positioning method based on deep learning of the present invention;
FIG. 7 is a flowchart of the method for iris image segmentation and localization based on deep learning according to the embodiment of the present invention for predicting the pupil center point according to the geometric relationship between the pupil center and the iris mask;
FIG. 8 is a flowchart of the method for segmenting and positioning an iris image based on deep learning according to the embodiment of the present invention, wherein the inner and outer boundary mapping maps are denoised to obtain the range parameters of the inner and outer circles of the iris;
FIG. 9 is a schematic diagram of the iris mask, inner and outer circles and the original iris image in the embodiment of the iris image segmentation and positioning method based on deep learning.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
The invention provides an iris image segmentation and positioning method based on deep learning, which comprises two parts: the first part is a coding layer decoding layer multitask network based on full convolution, an attention mechanism and an ASPP module of deep Lab or a pyramid pooling layer of PSPNet are integrated in a coding layer of the network, and a segmented mask can be output at the stage to complete a segmentation task; the second part is an efficient post-processing method for iris localization. When the method is applied to low-quality iris images, noise interference can be effectively eliminated, a segmented mask can be provided, parameters of an inner circle and an outer circle can also be provided, and a good foundation is laid for a subsequent iris identification process.
The invention discloses an iris image segmentation method based on deep learning, which comprises the following steps:
step S10, acquiring an iris image to be processed as a first iris image;
step S20, inputting the first iris image into a trained full convolution coding and decoding multitask neural network model for forward propagation to obtain a first pupil center mapping map, a first iris inner boundary mapping map, a first iris outer boundary mapping map and a first iris segmentation mask mapping map;
step S30, processing the first iris segmentation mask map by threshold segmentation to obtain a binary iris segmentation mask map and finish iris segmentation;
the trained full-convolution coding and decoding multitask neural network model is obtained in the following mode:
step S21, acquiring an iris image as a second iris image, marking the second iris image and acquiring a third iris image; integrating an attention mechanism, and constructing a full-convolution encoding and decoding multitask neural network;
and step S22, inputting the third iris image into the full-convolution encoding and decoding multitask neural network for training to obtain a trained full-convolution encoding and decoding multitask neural network model.
In order to more clearly describe the iris image segmentation method based on deep learning of the present invention, the following describes each step in the embodiment of the method in detail with reference to fig. 1.
The iris image positioning method based on deep learning of the embodiment of the invention comprises the steps of S10-S30, and the steps are described in detail as follows:
step S10, an iris image to be processed is acquired as a first iris image.
Step S20, inputting the first iris image into the trained full convolution coding and decoding multitask neural network model for forward propagation to obtain a first pupil center mapping map, a first iris inner boundary mapping map, a first iris outer boundary mapping map and a first iris segmentation mask mapping map.
The trained full-convolution coding and decoding multitask neural network model is obtained in the following mode:
step S21, acquiring an iris image as a second iris image, marking the second iris image and acquiring a third iris image; and (4) integrating an attention mechanism, and constructing a full-convolution encoding and decoding multitask neural network.
Step S211, marking effective iris pixels in the second iris image by adopting a binary code label; the valid iris pixels are marked as 1, and the rest parts are marked as 0; using two ellipses closest to the inner and outer boundaries of the iris as marks of the inner and outer boundaries of the iris; the boundary points are marked as 1 and the remaining positions are marked as 0.
The effective iris pixel does not contain noise such as mirror reflection and non-iris areas such as hair, eyelashes, pupils and sclera.
Step S212, marking the marked center of the inner elliptical boundary of the iris as the center of the pupil, and obtaining an iris image comprising an effective iris distinguishing mark, inner and outer boundary marks of the iris and a pupil center mark as a third iris image; the pupil center of the iris image is marked as 1, and the rest parts are marked as 0.
And step S22, inputting the third iris image into the full-convolution encoding and decoding multitask neural network for training to obtain a full-convolution encoding and decoding multitask neural network model.
The full-convolution encoding and decoding multitask neural network comprises an encoding layer and a decoding layer.
The encoding layer encodes features through a plurality of successive convolution, ReLu, and pooling operations, and successively reduces the resolution of the map of the original input image to obtain a low-resolution map.
In the embodiment of the invention, in order to extract the most judgment characteristic in the coding layer, a pyramid pooling module of ASPP or PSPNet of deep Lab is combined with an attention mechanism.
The decoding layer performs jump connection through a layer corresponding to the decoding layer, performs resolution recovery on the low-resolution mapping image after the coding layer is continuously reduced by using bilinear upsampling, and outputs four mapping images with the same size as the original input, corresponding to a pupil center mapping image, an iris inner boundary mapping image, an iris outer boundary mapping image and an iris segmentation mask mapping image.
As shown in fig. 4 and fig. 5, the full-convolution codec multitask network according to the embodiment of the present invention is based on the existing VGG-16 model, the full connection layer is removed, the remaining layer is taken as the coding layer of the multitask network, and an attention module is integrated at the final stage of the coding layer. The attention module extracts multi-scale context information through a context module, obtains a 3D attention mapping graph with the range between [0 and 1] by utilizing a convolution layer and sigmoid operation, and finally multiplies the attention mapping graph by an iris image mapping graph at the last stage of the coding layer pixel by pixel and connects the attention mapping graph with the iris image mapping graph to obtain the final refined characteristics.
The context module is an ASPP module of deep Lab or a pyramid pooling module of PSPNet.
By doing so, the encoding layer extracts the most intrinsic, most discriminative features. And then, jumping connection is carried out on the feature maps of the coding layer by utilizing an up-sampling operation, and the feature maps are continuously amplified until four output maps with the same size as the original input maps are finally obtained. As shown in fig. 3, it is a specific implementation of the full-convolution codec multitasking network.
Step S221, inputting the third iris image into the full-convolution encoding and decoding multitask neural network, and obtaining a second pupil center mapping map, a second iris inner boundary mapping map, a second iris outer boundary mapping map and a second iris segmentation mask mapping map through forward propagation.
Step S222, calculating errors of the second pupil center mapping map, the second iris inner boundary mapping map, the second iris outer boundary mapping map, the second iris segmentation mask mapping map and the third iris image, and updating parameters of the full-convolution encoding and decoding multitask neural network by using a back propagation algorithm and a random gradient descent method based on the calculated total loss function value.
And step S223, repeating the step S222 until the total loss function calculation value reaches a preset condition, and obtaining the trained full-convolution encoding and decoding multitask neural network model.
The total loss function includes focus loss for the pupil center, cross-entropy loss for the balance of the inner and outer boundaries of the iris, and cross-entropy loss for segmentation.
The focal loss L for the pupil centerpupilAs shown in formula (1):
Figure GDA0002620910390000121
Figure GDA0002620910390000122
the definition is shown in formula (2):
Figure GDA0002620910390000123
wherein, alpha and gamma are hyper-parameters; p ═ Pj,j=1,...,|X|},pjThe probability that the jth pixel predicted by the multitask network is the pupil center, and | X | is the number of pixels of the iris image;
Figure GDA0002620910390000124
Figure GDA0002620910390000125
for the label of the jth pixel in the iris image, 1 is expressed as the true iris-pupil center, and 0 is not.
The cross entropy loss L for the balance of the inner and outer boundaries of the irisedgeAs shown in formula (3):
Figure GDA0002620910390000131
wherein the content of the first and second substances,
Figure GDA0002620910390000132
Figure GDA0002620910390000133
representing the probability that the jth pixel in the iris image belongs to the inner iris boundary or the outer iris boundary, wherein k is 1 for the inner iris boundary, k is 2 for the outer iris boundary, 1 is represented as the real iris boundary, and 0 is not; beta is the proportion of non-edge pixels in the iris image, and the non-edge pixels are pixels except the inner boundary and the outer boundary of the iris.
The cross entropy loss for segmentation is LsegAs shown in formula (4):
Figure GDA0002620910390000134
wherein S ═ { S ═ Sj,j=1,...|X|},sjRepresenting the jth of the iris imageProbability that a pixel belongs to a true iris texture;
Figure GDA0002620910390000135
Figure GDA0002620910390000136
a label indicating that the jth pixel in the iris image belongs to the real iris texture.
Fig. 6 shows the result of inputting a full-convolution codec multitask neural network model for a single iris image according to an embodiment of the present invention.
The iris image positioning method based on deep learning of the second embodiment of the present invention includes:
by adopting the steps S10-S20 of the iris image segmentation method based on deep learning, a first pupil center mapping map, a first iris inner boundary mapping map, a first iris outer boundary mapping map and a first iris segmentation mask mapping map are obtained, and the following steps are executed:
step B10, obtaining the predicted position of the pupil center according to the geometric relationship between the first pupil center mapping map and the first iris segmentation mask mapping map;
b20, denoising the first iris inner and outer boundary mapping map based on the preset geometric relationship among the pupil, the iris and the sclera to obtain a first iris inner and outer circle radius parameter set and a first iris inner and outer circle center initial coordinate;
and step B30, acquiring fitting points of the inner and outer contours of the iris by using a Viterbi algorithm based on the denoised inner and outer boundary mapping chart of the first iris, the first inner and outer circle radius parameter set and the initial coordinates of the center of the inner and outer circles of the first iris, and fitting the inner and outer circles by using a least square circle fitting algorithm to obtain inner and outer circle parameters of the iris so as to complete iris positioning.
In order to more clearly describe the iris image positioning method based on deep learning of the present invention, the following will describe each step in the embodiment of the method of the present invention in detail with reference to fig. 1.
The iris image positioning method based on deep learning of the embodiment of the invention comprises the steps B10-B30, and the steps are described in detail as follows:
and step B10, acquiring the predicted position of the pupil center according to the geometric relation between the first pupil center mapping map and the first iris segmentation mask mapping map.
And performing threshold segmentation and connected region analysis on the first pupil center mapping map and the first iris segmentation mask mapping map, searching a connected sub region of the pupil center closest to the connected sub region of the maximum area segmentation mask, and taking the center point of the connected sub region as the finally predicted pupil center. Fig. 7 is a diagram illustrating the pupil center position prediction according to the embodiment of the present invention.
And B20, denoising the first iris inner and outer boundary mapping map based on the preset geometric relation among the pupil, the iris and the sclera to obtain a first iris inner and outer circle radius parameter set and a first iris inner and outer circle center initial coordinate. Fig. 8 is a schematic flow chart illustrating denoising of inner and outer boundaries of an iris and obtaining an approximate range of inner and outer circles of the iris according to an embodiment of the present invention.
And step B21, generating a limited area by taking the predicted position of the pupil center as a circle center and the maximum distance from the pupil center to the largest-area connected subregion of the segmentation mask as a radius.
Step B22, deleting the connected region of the iris outer boundary mapping chart inside and outside the limited region to obtain a refined iris outer boundary; deleting the connected region of the iris inner boundary mapping map outside the limited region to obtain the refined iris inner boundary.
And step B23, taking the maximum and minimum distance from the pupil center to the refined inner and outer iris boundaries as a first inner and outer iris radius parameter set, wherein the pupil center coordinate is the initial coordinate of the first inner and outer iris center.
And step B30, acquiring fitting points of the inner and outer contours of the iris by using a Viterbi algorithm based on the denoised inner and outer boundary mapping chart of the first iris, the first inner and outer circle radius parameter set and the initial coordinates of the center of the inner and outer circles of the first iris, and fitting the inner and outer circles by using a least square circle fitting algorithm to obtain inner and outer circle parameters of the iris so as to complete iris positioning.
Fig. 9 is a schematic diagram showing the result of iris image segmentation and positioning after all the steps of the present invention are completed.
The iris image collected in the uncontrolled scene is degraded in image quality due to illumination, target motion, distance change and the like, and includes various noises such as specular reflection, motion/defocus blur, frame blur and the like, as shown in fig. 2, the degraded iris image makes segmentation and positioning of the iris difficult, and further affects the accuracy of iris recognition.
The iris image segmentation and positioning system based on deep learning in the third embodiment of the invention comprises an input module, a multitask neural network module, a threshold segmentation module, a pupil position prediction module, a denoising module, a fitting module and an output module;
the input module is configured to acquire an iris image to be tested as a first iris image;
the multitask neural network module is configured to process the first iris image to obtain a first pupil center mapping map, a first iris inner boundary mapping map, a first iris outer boundary mapping map and a first iris segmentation mask mapping map;
the threshold segmentation module is configured to process the first iris segmentation mask map by threshold segmentation to obtain a binary iris segmentation mask map and complete iris segmentation;
the pupil position prediction module is configured to obtain a predicted position of a pupil center according to a geometric relationship between the first pupil center mapping map and the first iris segmentation mask mapping map;
the denoising module is configured to denoise the first iris inner and outer boundary mapping map based on a preset geometric relationship among the pupil, the iris and the sclera to obtain a first iris inner and outer circle radius parameter set and a first iris inner and outer circle center initial coordinate;
the fitting module is configured to obtain fitting points of inner and outer contours of the iris by using a Viterbi algorithm based on the denoised first iris inner and outer boundary mapping map, the first iris inner and outer circle radius parameter set and the first iris inner and outer circle center initial coordinates, and fit the inner and outer circles by using a least square circle fitting algorithm to obtain iris inner and outer circle parameters so as to complete iris positioning;
and the output module is configured to output the binary iris segmentation mask map and the inner and outer iris circle parameters.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the system described above may refer to the corresponding process in the foregoing method embodiments, and will not be described herein again.
It should be noted that, the iris image segmentation and positioning system based on deep learning provided in the foregoing embodiment is only illustrated by the division of the above functional modules, and in practical applications, the functions may be allocated to different functional modules according to needs, that is, the modules or steps in the embodiments of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
A storage device according to a fourth embodiment of the present invention stores a plurality of programs, and the programs are suitable for being loaded and executed by a processor to implement the iris image segmentation and positioning method based on deep learning.
A processing apparatus according to a fifth embodiment of the present invention includes a processor, a storage device; a processor adapted to execute various programs; a storage device adapted to store a plurality of programs; the program is suitable to be loaded and executed by a processor to realize the iris image segmentation and positioning method based on deep learning.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Those of skill in the art would appreciate that the various illustrative modules, method steps, and modules described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the software modules, method steps may be located in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The iris image segmentation and positioning method, system and device based on deep learning can be applied to subway import and export systems of remote iris recognition.
The invention can be applied to remote iris recognition scenes. Firstly, a user walks into an acquisition area of a subway entrance and exit system to acquire iris images. Because the distance is long and the user is in motion, the acquired iris image is subjected to various noises such as motion blur, occlusion, oblique eyes and the like. The subway import and export system can effectively segment effective iris textures and position the inner circle and the outer circle of the iris by using the iris segmentation and positioning method, system and device based on deep learning, so that iris normalization and feature extraction are performed, the result is compared with the iris image registered in the system, and the rapid import and export of users are realized. The system can greatly improve the passing rate of subway users, and save the personnel expenditure and the misjudgment condition.
The iris image segmentation and positioning method, system and device based on deep learning can also be applied to iris unlocking of smart phones.
The iris unlocking can be widely applied to smart phones, a user can conveniently and quickly access the mobile phone device, complicated processes such as password input are omitted, the whole process is safe, and the threat of password loss or password theft is avoided. The intelligent mobile phone user utilizes the built-in or external near-infrared or visible light camera to automatically shoot the human eye image, but the obtained image has a lot of noise interference, such as mirror reflection, motion blur and the like.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can fall into the protection scope of the invention.

Claims (11)

1. An iris image segmentation method based on deep learning is characterized by comprising the following steps:
step S10, acquiring an iris image to be processed as a first iris image;
step S20, inputting the first iris image into a trained full convolution coding and decoding multitask neural network model for forward propagation to obtain a first pupil center mapping map, a first iris inner boundary mapping map, a first iris outer boundary mapping map and a first iris segmentation mask mapping map;
step S30, processing the first iris segmentation mask map by threshold segmentation to obtain a binary iris segmentation mask map and finish iris segmentation;
the trained full-convolution coding and decoding multitask neural network model is obtained in the following mode:
step S21, acquiring an iris image as a second iris image, marking the second iris image and acquiring a third iris image; integrating an attention mechanism, and constructing a full-convolution encoding and decoding multitask neural network;
and step S22, inputting the third iris image into the full-convolution encoding and decoding multitask neural network for training to obtain a trained full-convolution encoding and decoding multitask neural network model.
2. The iris image segmentation method based on deep learning of claim 1 wherein step S21 "mark the second iris image" includes the steps of:
step S211, marking effective iris pixels in the second iris image by adopting a binary code label; using two ellipses closest to the inner and outer boundaries of the iris as marks of the inner and outer boundaries of the iris;
step S212, marking the marked center of the inner elliptical boundary of the iris as the center of the pupil, and obtaining an effective iris distinguishing mark image, an inner and outer boundary mark image of the iris and an iris image of the pupil center mark image as a third iris image.
3. The iris image segmentation method based on deep learning of claim 1 is characterized in that the full convolutional encoding and decoding multitasking neural network comprises an encoding layer and a decoding layer;
the coding layer codes features through a plurality of continuous convolution, ReLu and pooling operations, and continuously reduces the resolution of a mapping map of an original input image to obtain a low-resolution mapping map;
the decoding layer performs jump connection through a layer corresponding to the decoding layer, performs resolution recovery on the low-resolution mapping image after the coding layer is continuously reduced by using bilinear upsampling, and outputs four mapping images with the same size as the original input, corresponding to a pupil center mapping image, an iris inner boundary mapping image, an iris outer boundary mapping image and an iris segmentation mask mapping image.
4. The iris image segmentation method based on deep learning of claim 1 wherein step S22, inputting the third iris image into the full-convolution codec multitask neural network for training, comprises the steps of:
step S221, inputting the third iris image into the full-convolution encoding and decoding multitask neural network, and obtaining a second pupil center mapping map, a second iris inner boundary mapping map, a second iris outer boundary mapping map and a second iris segmentation mask mapping map through forward propagation;
step S222, calculating errors of the second pupil center mapping map, the second iris inner boundary mapping map, the second iris outer boundary mapping map, the second iris segmentation mask mapping map and the third iris image, and updating parameters of the fully-convolutional encoding and decoding multitask neural network by using a back propagation algorithm and a random gradient descent method based on the calculated total loss function value;
and step S223, repeating the step S222 until the total loss function calculation value reaches a preset condition, and obtaining the trained full-convolution encoding and decoding multitask neural network model.
5. The deep learning based iris image segmentation method of claim 4, wherein the total loss function comprises focus loss for pupil center, cross entropy loss for balance of inner and outer boundaries of iris and cross entropy loss for segmentation;
the focal loss L for the pupil centerpupilComprises the following steps:
Figure FDA0002620910380000031
Figure FDA0002620910380000032
wherein, alpha and gamma are hyper-parameters; p ═ Pj,j=1,...,|X|},pjThe probability that the jth pixel predicted by the multitask network is the pupil center, and | X | is the number of pixels of the iris image;
Figure FDA0002620910380000033
Figure FDA0002620910380000034
a label of the jth pixel in the iris image, wherein 1 represents the center of a real iris pupil, and 0 is not;
the cross entropy loss L for the balance of the inner and outer boundaries of the irisedgeComprises the following steps:
Figure FDA0002620910380000035
wherein the content of the first and second substances,
Figure FDA0002620910380000036
Figure FDA0002620910380000037
representing the probability that the jth pixel in the iris image belongs to the inner iris boundary or the outer iris boundary, wherein k is 1 for the inner iris boundary, k is 2 for the outer iris boundary, 1 is represented as the real iris boundary, and 0 is not; beta is the proportion of non-edge pixels in the iris image, and the non-edge pixels are pixels except the inner boundary and the outer boundary of the iris;
the cross entropy loss for segmentation is LsegComprises the following steps:
Figure FDA0002620910380000038
wherein S ═ { S ═ Sj,j=1,...|X|},sjRepresenting the probability that the jth pixel in the iris image belongs to the real iris texture;
Figure FDA0002620910380000039
Figure FDA00026209103800000310
a label indicating that the jth pixel in the iris image belongs to the real iris texture.
6. An iris image positioning method based on deep learning is characterized by comprising the following steps:
using steps S10-S20 of the deep learning based iris image segmentation method as claimed in any one of claims 1-5, acquiring a first pupil center map, a first iris inner boundary map, a first iris outer boundary map and a first iris segmentation mask map, and performing the following steps:
step B10, obtaining the predicted position of the pupil center according to the geometric relationship between the first pupil center mapping map and the first iris segmentation mask mapping map;
b20, denoising the first iris inner and outer boundary mapping map based on the preset geometric relationship among the pupil, the iris and the sclera to obtain a first iris inner and outer circle radius parameter set and a first iris inner and outer circle center initial coordinate;
and step B30, acquiring fitting points of the inner and outer contours of the iris by using a Viterbi algorithm based on the denoised inner and outer boundary mapping chart of the first iris, the first inner and outer circle radius parameter set and the initial coordinates of the center of the inner and outer circles of the first iris, and fitting the inner and outer circles by using a least square circle fitting algorithm to obtain inner and outer circle parameters of the iris so as to complete iris positioning.
7. The method as claimed in claim 6, wherein in step B10, the method for obtaining the predicted pupil center position according to the geometric relationship between the first pupil center map and the first iris segmentation mask map includes:
and performing threshold segmentation and connected region analysis on the first pupil center mapping map and the first iris segmentation mask mapping map, searching a connected sub region of the pupil center closest to the connected sub region of the maximum area segmentation mask, and taking the center point of the connected sub region as the finally predicted pupil center.
8. The method as claimed in claim 6 or 7, wherein in step B20, based on the preset geometric relationship among pupil, iris and sclera, the first iris inner and outer boundary map is denoised to obtain the first iris inner and outer circle radius parameter set and the first iris inner and outer circle center initial coordinates, and the steps are as follows:
step B21, generating a limited area by taking the predicted position of the pupil center as a circle center and the maximum distance from the pupil center to the largest-area connected subregion of the segmentation mask as a radius;
step B22, deleting the iris outer boundary mapping map connected region inside and outside the limited region to obtain a refined iris outer boundary; deleting the connected region of the iris inner boundary mapping map outside the limited region to obtain a refined iris inner boundary;
and step B23, taking the maximum and minimum distance from the pupil center to the refined inner and outer iris boundaries as a first inner and outer iris radius parameter set, wherein the pupil center coordinate is the initial coordinate of the first inner and outer iris center.
9. An iris image segmentation and positioning system based on deep learning is characterized by comprising an input module, a multitask neural network module, a threshold segmentation module, a pupil position prediction module, a denoising module, a fitting module and an output module;
the input module is configured to acquire an iris image to be tested as a first iris image;
the multitask neural network module is configured to process the first iris image to obtain a first pupil center mapping map, a first iris inner boundary mapping map, a first iris outer boundary mapping map and a first iris segmentation mask mapping map;
the threshold segmentation module is configured to process the first iris segmentation mask map by threshold segmentation to obtain a binary iris segmentation mask map and complete iris segmentation;
the pupil position prediction module is configured to obtain a predicted position of a pupil center according to a geometric relationship between the first pupil center mapping map and the first iris segmentation mask mapping map;
the denoising module is configured to denoise the first iris inner and outer boundary mapping map based on a preset geometric relationship among the pupil, the iris and the sclera to obtain a first iris inner and outer circle radius parameter set and a first iris inner and outer circle center initial coordinate;
the fitting module is configured to obtain fitting points of inner and outer contours of the iris by using a Viterbi algorithm based on the denoised first iris inner and outer boundary mapping map, the first iris inner and outer circle radius parameter set and the first iris inner and outer circle center initial coordinates, and fit the inner and outer circles by using a least square circle fitting algorithm to obtain iris inner and outer circle parameters so as to complete iris positioning;
and the output module is configured to output the binary iris segmentation mask map and the inner and outer iris circle parameters.
10. A storage device having stored thereon a plurality of programs, characterized in that the programs are adapted to be loaded and executed by a processor to implement the method according to any of claims 1-8.
11. A treatment apparatus comprises
A processor adapted to execute various programs; and
a storage device adapted to store a plurality of programs;
wherein the program is adapted to be loaded and executed by a processor to perform:
the method of any one of claims 1-8.
CN201910002663.XA 2019-01-02 2019-01-02 Iris image segmentation and positioning method, system and device based on deep learning Active CN109815850B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910002663.XA CN109815850B (en) 2019-01-02 2019-01-02 Iris image segmentation and positioning method, system and device based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910002663.XA CN109815850B (en) 2019-01-02 2019-01-02 Iris image segmentation and positioning method, system and device based on deep learning

Publications (2)

Publication Number Publication Date
CN109815850A CN109815850A (en) 2019-05-28
CN109815850B true CN109815850B (en) 2020-11-10

Family

ID=66603779

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910002663.XA Active CN109815850B (en) 2019-01-02 2019-01-02 Iris image segmentation and positioning method, system and device based on deep learning

Country Status (1)

Country Link
CN (1) CN109815850B (en)

Families Citing this family (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110458016A (en) * 2019-07-08 2019-11-15 北京虹安翔宇信息科技有限公司 A kind of processing method of the iris recognition image based on high pass chip
CN110490083A (en) * 2019-07-23 2019-11-22 苏州国科视清医疗科技有限公司 A kind of pupil accurate detecting method based on fast human-eye semantic segmentation network
CN110458833B (en) * 2019-08-15 2023-07-11 腾讯科技(深圳)有限公司 Medical image processing method, medical device and storage medium based on artificial intelligence
CN110738663A (en) * 2019-09-06 2020-01-31 上海衡道医学病理诊断中心有限公司 Double-domain adaptive module pyramid network and unsupervised domain adaptive image segmentation method
CN110738107A (en) * 2019-09-06 2020-01-31 上海衡道医学病理诊断中心有限公司 microscopic image recognition and segmentation method with model migration function
CN111161276B (en) * 2019-11-27 2023-04-18 天津中科智能识别产业技术研究院有限公司 Iris normalized image forming method
CN110991415B (en) * 2019-12-21 2023-04-25 武汉中海庭数据技术有限公司 Structural target high-precision segmentation method, electronic equipment and storage medium
CN111588469B (en) * 2020-05-18 2021-02-02 四川大学华西医院 Ophthalmic robot end effector guidance and positioning system
CN111738122B (en) * 2020-06-12 2023-08-22 Oppo广东移动通信有限公司 Image processing method and related device
CN111938567B (en) * 2020-07-09 2021-10-22 上海交通大学 Deep learning-based ophthalmologic parameter measurement method, system and equipment
CN112287872B (en) * 2020-11-12 2022-03-25 北京建筑大学 Iris image segmentation, positioning and normalization method based on multitask neural network
CN112651328B (en) * 2020-12-23 2022-09-13 浙江中正智能科技有限公司 Iris segmentation method based on geometric position relation loss function
CN112668472B (en) * 2020-12-28 2021-08-31 中国科学院自动化研究所 Iris image feature extraction method, system and device based on federal learning
CN113536968B (en) * 2021-06-25 2022-08-16 天津中科智能识别产业技术研究院有限公司 Method for automatically acquiring boundary coordinates of inner and outer circles of iris
CN113253850A (en) * 2021-07-05 2021-08-13 中国科学院西安光学精密机械研究所 Multitask cooperative operation method based on eye movement tracking and electroencephalogram signals
CN113537111A (en) * 2021-07-26 2021-10-22 南京信息工程大学 Iris segmentation method based on double-branch deep convolutional network
CN113706469B (en) * 2021-07-29 2024-04-05 天津中科智能识别产业技术研究院有限公司 Iris automatic segmentation method and system based on multi-model voting mechanism
CN113706470B (en) * 2021-07-29 2023-12-15 天津中科智能识别产业技术研究院有限公司 Iris image segmentation method and device, electronic equipment and storage medium
CN113689385B (en) * 2021-07-29 2023-10-20 天津中科智能识别产业技术研究院有限公司 Automatic iris inner and outer circle boundary segmentation method, device, equipment and storage medium
CN114638879A (en) * 2022-03-21 2022-06-17 四川大学华西医院 Medical pupil size measuring system
CN116110113A (en) * 2022-11-15 2023-05-12 南昌航空大学 Iris recognition method based on deep learning
CN115797632B (en) * 2022-12-01 2024-02-09 北京科技大学 Image segmentation method based on multi-task learning
CN117316437B (en) * 2023-11-29 2024-03-08 首都医科大学附属北京安定医院 Pain level prediction method, system and equipment based on pupil change

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105303185A (en) * 2015-11-27 2016-02-03 中国科学院深圳先进技术研究院 Iris positioning method and device
CN106778664B (en) * 2016-12-29 2020-12-15 天津中科智能识别产业技术研究院有限公司 Iris image iris area segmentation method and device

Also Published As

Publication number Publication date
CN109815850A (en) 2019-05-28

Similar Documents

Publication Publication Date Title
CN109815850B (en) Iris image segmentation and positioning method, system and device based on deep learning
CN108009520B (en) Finger vein identification method and system based on convolution variational self-encoder network
CN110852160B (en) Image-based biometric identification system and computer-implemented method
CN110580445B (en) Face key point detection method based on GIoU and weighted NMS improvement
Nakamura et al. Scene text eraser
CN111274916B (en) Face recognition method and face recognition device
CN111160269A (en) Face key point detection method and device
CN112446270A (en) Training method of pedestrian re-identification network, and pedestrian re-identification method and device
CN103942577A (en) Identity identification method based on self-established sample library and composite characters in video monitoring
CN106845440A (en) A kind of augmented reality image processing method and system
CN112287872B (en) Iris image segmentation, positioning and normalization method based on multitask neural network
CN114821682B (en) Multi-sample mixed palm vein identification method based on deep learning algorithm
CN110348289A (en) A kind of finger vein identification method based on binary map
CN111209873A (en) High-precision face key point positioning method and system based on deep learning
CN111178130A (en) Face recognition method, system and readable storage medium based on deep learning
Fuentes-Hurtado et al. A hybrid method for accurate iris segmentation on at-a-distance visible-wavelength images
CN111027637A (en) Character detection method and computer readable storage medium
CN111666813B (en) Subcutaneous sweat gland extraction method of three-dimensional convolutional neural network based on non-local information
CN115829975A (en) Palm vein image quality detection method, system, medium and electronic device
Zhou et al. A robust pupil localization via a novel parameter optimization strategy
Liu et al. Fingerprint pore extraction
AU2021106149A4 (en) Gaze based interface to assist prospective programmers with the learning environment
Vlachos et al. Finger vein segmentation in infrared images using supervised and unsupervised clustering algorithms
An et al. A DenseNet‐based feature weighting convolutional network recognition model and its application in industrial part classification
Amandi et al. Long distance iris recognition

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant