CN106778664B - Iris image iris area segmentation method and device - Google Patents
Iris image iris area segmentation method and device Download PDFInfo
- Publication number
- CN106778664B CN106778664B CN201611242734.6A CN201611242734A CN106778664B CN 106778664 B CN106778664 B CN 106778664B CN 201611242734 A CN201611242734 A CN 201611242734A CN 106778664 B CN106778664 B CN 106778664B
- Authority
- CN
- China
- Prior art keywords
- iris
- image
- neural network
- convolutional neural
- iris image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 230000011218 segmentation Effects 0.000 title claims abstract description 57
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 88
- 238000012549 training Methods 0.000 claims abstract description 48
- 238000002372 labelling Methods 0.000 claims abstract description 18
- 238000005070 sampling Methods 0.000 claims description 24
- 230000004927 fusion Effects 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 12
- 210000000744 eyelid Anatomy 0.000 claims description 11
- 210000000720 eyelash Anatomy 0.000 claims description 10
- 210000004209 hair Anatomy 0.000 claims description 10
- 210000001747 pupil Anatomy 0.000 claims description 9
- 210000003786 sclera Anatomy 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 238000011478 gradient descent method Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 4
- 210000000554 iris Anatomy 0.000 description 233
- 238000013528 artificial neural network Methods 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 9
- 238000005457 optimization Methods 0.000 description 6
- 210000002265 sensory receptor cell Anatomy 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000003708 edge detection Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 210000001508 eye Anatomy 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 238000005034 decoration Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/193—Preprocessing; Feature extraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
Abstract
The invention discloses a segmentation method of an iris region in an iris image, which comprises the following steps: the first step is as follows: establishing a multi-scale convolutional neural network; the second step is that: pre-labeling a pre-selected preset key point in the iris image; the third step: inputting the iris image labeled with preset key points into the convolutional neural network, and training the convolutional neural network to ensure that a model of the convolutional neural network is converged; the fourth step: inputting the iris image to be tested, which needs to be subjected to iris region segmentation, into the trained convolutional neural network to obtain a binary code image of the iris image to be tested, namely the iris region to be segmented finally. The iris region segmentation method and device in the iris image disclosed by the invention can be used for timely and accurately segmenting the iris region in the iris image acquired in a non-controllable scene, meet the requirement of a user on iris segmentation and improve the working efficiency of the user.
Description
Technical Field
The invention relates to the technical field of pattern recognition, computer vision and the like, in particular to a segmentation method and a segmentation device for iris regions in iris images.
Background
At present, with the continuous development of human science and technology, iris recognition technology is more and more popularized in daily life of people, iris recognition is a biological feature recognition method for identifying identity by utilizing the invariance, uniqueness and the like of the texture of an iris, and the iris recognition technology is successfully applied to the fields of national security, border control, bank finance, access control and attendance checking, mobile terminals and the like. In the aspect of artificial intelligence research and public safety application, the iris identification technology is always a leading edge and hot technology and has a great position.
For iris recognition technology, in practical applications, many technical challenges are encountered. Especially, for an uncontrollable scene (i.e., a complex and uncontrollable scene) such as a distant scene and a scene (e.g., a motion scene) in which people are not completely matched, as shown in fig. 1, the iris image of the collected people has illumination and distance changes, and thus has characteristics of low resolution, high noise, oblique eyes, blur, occlusion, and the like, so that it is particularly difficult to accurately segment the iris region in the iris image, and thus accuracy and timeliness of iris recognition of people are affected. The iris segmentation is intended to effectively segment an iris region that is located between the black pupil and the white sclera of the eyeball and is not blocked by various noises, and has a direct influence on the accuracy of iris recognition as a preprocessing step of iris recognition.
Currently, in order to segment iris regions in iris images, typical iris segmentation methods can be divided into two main categories: edge detection based methods and pixel point based methods. The method based on edge detection needs to detect the inner and outer edges of the iris, the upper and lower eyelids and other edges respectively so as to determine the effective area of the iris. However, in an uncontrollable scene, the iris boundary is more fuzzy, or the iris is distorted and deformed and is not a circle any more, so that the method cannot effectively solve the iris segmentation in the uncontrollable scene. Another iris segmentation method is a pixel-based method, and this method utilizes the apparent characteristics of the iris, such as texture, color, etc., to determine whether each pixel belongs to an effective iris region, thereby effectively avoiding the above-mentioned defects. However, such methods often require preprocessing for light, blur, and the like, and it is often difficult to design effective features to classify pixel points.
Therefore, there is an urgent need to develop a technology that can timely and accurately segment the iris region in the iris image collected in an uncontrollable scene, meet the requirements of users on iris segmentation, improve the working efficiency of users, save precious time of people, and effectively ensure the accuracy of iris recognition on the iris image.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a method and an apparatus for segmenting an iris region in an iris image, which can timely and accurately segment the iris region in the iris image acquired in an uncontrollable scene, meet the requirements of a user on iris segmentation, improve the work efficiency of the user, save precious time of the user, effectively ensure the accuracy of iris recognition on the iris image, and have great significance in production practice.
Therefore, the invention provides a segmentation method of an iris region in an iris image, which comprises the following steps:
the first step is as follows: establishing a multi-scale convolutional neural network, wherein the convolutional neural network comprises a convolutional layer and a sampling layer which are used for sequentially processing the input image;
the second step is that: pre-labeling a pre-selected preset key point in the iris image;
the third step: inputting the iris image labeled with preset key points into the convolutional neural network, and training the convolutional neural network to ensure that a model of the convolutional neural network is converged;
the fourth step: inputting the iris image to be tested, which needs iris region segmentation, into the trained convolutional neural network to obtain a binary code image of the iris image to be tested, wherein the binary code image is the iris region which is segmented finally.
Wherein the second step specifically comprises: and respectively labeling the preset key points of the pixels belonging to the iris and the non-iris pixels in the preselected iris image.
The preset key points of the pixels belonging to the iris in the iris image comprise a black pupil and a white sclera; the preset key points of the non-iris pixels comprise hair, eyelids, eyelashes and light spots.
Wherein, the pixels of the iris and the non-iris pixels are respectively marked as binary codes 0 and 1.
Wherein the third step specifically comprises:
inputting the iris image labeled with preset key points into the convolutional neural network, training the convolutional neural network by using a random gradient descent method, calculating the class of each pixel of the input iris image through a forward propagation algorithm, performing convolutional layer parameter replacement by calculating the error between the class and the real class labeled in advance of the iris image and using a back propagation method, training the model of the convolutional neural network until the model converges, and finishing the training process.
In addition, the present invention also provides a segmentation apparatus for an iris region in an iris image, comprising:
the network establishing unit is used for establishing a multi-scale convolutional neural network, and the convolutional neural network comprises a convolutional layer and a sampling layer which are used for sequentially processing the input image;
the system comprises a pre-key point marking unit, a network training unit and a network processing unit, wherein the pre-key point marking unit is used for marking a pre-set key point in a pre-selected iris image in advance and then outputting the pre-set key point to the network training unit;
the network training unit is respectively connected with the network establishing unit and the pre-key point marking unit and is used for inputting the iris image marked with the pre-key points into the convolutional neural network established by the network establishing unit and training the convolutional neural network so as to lead the model of the convolutional neural network to be converged;
a division execution unit: and the network training unit is connected with the iris image to be tested, which needs iris region segmentation, and the iris image to be tested is input into the convolutional neural network trained by the network training unit to obtain a binary code image of the iris image to be tested, wherein the binary code image is the iris region which is segmented finally.
The pre-key point marking unit specifically comprises: and the preset key points are used for respectively labeling the pixels belonging to the iris and the non-iris pixels in the preselected iris image.
The preset key points of the pixels belonging to the iris in the iris image comprise a black pupil and a white sclera; the preset key points of the non-iris pixels comprise hair, eyelids, eyelashes and light spots.
Wherein, the pixels of the iris and the non-iris pixels are respectively marked as binary codes 0 and 1.
Wherein, the network training unit is specifically: the method is used for inputting the iris image labeled with the preset key points into the convolutional neural network, training the convolutional neural network by using a random gradient descent method, calculating the class of each pixel of the input iris image through a forward propagation algorithm, replacing convolutional layer parameters by calculating the error between the class of each pixel and a real class labeled in advance of the iris image through a back propagation method, training the model of the convolutional neural network until the model converges, and finishing the training process.
Compared with the prior art, the iris image segmentation method and device provided by the invention can be used for timely and accurately segmenting the iris region in the iris image acquired under the uncontrollable scene, meet the requirement of a user on iris segmentation, improve the working efficiency of the user, save precious time of people, effectively ensure the accuracy rate of iris identification on the iris image and have great production practice significance.
Drawings
FIG. 1 is a schematic diagram of an iris image acquired in an uncontrolled scene;
FIG. 2 is a flowchart of a segmentation method for iris region in iris image according to the present invention;
FIG. 3 is a schematic structural diagram of each component of a constructed convolutional neural network in a segmentation method of an iris region in an iris image according to the present invention;
fig. 4 is a schematic structural diagram of each component of an exemplary embodiment of a constructed convolutional neural network in a segmentation method of an iris region in an iris image according to the present invention;
FIG. 5 is a schematic diagram showing the comparison between iris regions obtained by segmentation and original iris images after a plurality of iris images are segmented by using the segmentation method for iris regions in iris images provided by the present invention;
fig. 6 is a block diagram illustrating a structure of a device for segmenting iris regions in an iris image according to the present invention.
Detailed Description
In order that those skilled in the art will better understand the technical solution of the present invention, the following detailed description of the present invention is provided in conjunction with the accompanying drawings and embodiments.
Fig. 2 is a flowchart of a segmentation method for iris regions in an iris image according to the present invention.
Referring to fig. 2, the iris region segmentation method in the iris image provided by the present invention can fuse local detail information and global structure information when segmenting the iris region, so that iris pixels are accurately positioned, and the method specifically includes the following steps:
the first step is as follows: establishing a multi-scale convolutional neural network, wherein the convolutional neural network comprises a convolutional layer and a sampling layer which are used for sequentially processing the input image;
the second step is that: pre-labeling a pre-selected preset key point in the iris image;
the third step: inputting the iris image labeled with preset key points into the convolutional neural network, and training the convolutional neural network to ensure that a model of the convolutional neural network is converged;
the fourth step: inputting the iris image to be tested, which needs iris region segmentation, into the trained convolutional neural network to obtain a binary code image of the iris image to be tested, wherein the binary code image is the iris region which is segmented finally.
It should be noted that, in practical applications, the iris segmentation technique has many challenges, especially in uncontrollable scenes, such as iris images acquired at long distances and in motion, and there are noise interferences, such as illumination and distance variation, high noise, low resolution, occlusion, and blur. The iris image shown in fig. 1 is acquired in an uncontrollable scene, so that the iris segmentation is difficult due to the image quality, and the accuracy of iris identification is directly influenced.
The two traditional iris segmentation methods are respectively based on an edge detection method and a pixel point-based method. The two methods have respective limitations in processing iris images acquired under an uncontrollable scene. The model is used for iris segmentation based on the multi-scale full-convolution neural network, and the model fuses local detail information and global structure information so that iris pixels are accurately positioned.
For the present invention, in the first step, the convolutional neural network is composed of a convolutional layer and a sampling layer, and in order to fuse the global information and the local information of the input image, several output results and the output result of the last layer may be received from the convolutional neural network for fusion, as shown in fig. 3. Since the neural receptor fields output from the deeper layers of the network gradually increase, the neural receptor fields in the later layers express more global information, and the neural receptor fields in the earlier layers express local information. Therefore, for the invention, in order to make the image model established by the convolutional neural network more robust, the invention uses a multi-scale fusion model to fuse the shallow local information and the deep global information for optimization.
In the present invention, the convolutional neural network consists of only two types of layers: convolutional layers and sampling layers. The convolutional layer is used for performing convolution operation on corresponding data (such as an input iris image), and the sampling layer (only a downsampling layer is used in the network) is used for performing downsampling on the input data, specifically, performing interval sampling on pixel values.
Wherein the convolutional layer formula is as follows:
(Yk)ij＝(Wk*x)i,j+bk；
wherein, Y is an output image, x is an input image, w is a weight, b is a bias term, coordinates of (i, j) pixel points, k is an index of the image (k image), and the use range of w and b is initialized by random numbers from 0 to 1.
In a specific implementation, for a convolutional layer and a sampling layer included in the convolutional neural network, the output of the previous layer is the input of the next layer.
In particular, the invention designs a multi-scale full convolution neural network, automatically learns the input iris image and the labeled image (namely the iris image labeled with the preset key points), and performs convolution operation for many times so as to fit the input iris image to the labeled image (namely the iris image labeled with the preset key points) as much as possible.
Referring to fig. 3, in the present invention, a typical network structure of the convolutional neural network is: based on the existing model of VGG-19layers, the model is formed by only combining a convolutional layer and a downsampling layer, and then five output results of five downsampling layers of an image input layer, pool3, pool6, pool9, pool12 and pool16 are fused and output in an image fusion output layer from front to back, as shown in fig. 4, the detailed network connection of the convolutional neural network is specifically realized.
For the present invention, in the second step, the second step is specifically: and manually labeling the preselected iris image, and labeling preset key points (including hair, eyelids, eyelashes, light spots and the like) which respectively belong to the pixels of the iris and the non-iris pixels in the preselected iris image.
In a specific implementation, the preselected iris image may be an iris image which is manually preselected by the user according to the present invention, has high definition and is easy to perform image recognition (the preset key point is clear). For the present invention, the pre-selected pictures need more diversified data, keeping richness.
Specifically, the preset key points of the pixels belonging to the iris in the iris image comprise a black pupil and a white sclera; the preset key points of the non-iris pixels comprise hair, eyelids, eyelashes and light spots; the pixels of the iris and the non-iris pixels may be labeled as 0 and 1, respectively, for example, i.e., a binary code labeling is implemented.
For the present invention, in the third step, the third step specifically is: inputting the iris image labeled with preset key points into the convolutional neural network, training the convolutional neural network by using a random gradient descent method, calculating the class of each pixel of the input iris image through a forward propagation algorithm, calculating the error (namely, the result obtained by performing a series of convolution and sampling operations on one input image) between the iris image and a pre-labeled real class (wherein, the pixel and the non-iris pixel of the iris can be respectively labeled as 0 and 1, and labeled by binary codes) of the iris image, performing convolutional layer parameter replacement by using a backward propagation method, training the model of the convolutional neural network until the model converges, and completing the training process (namely, the optimization process of convolutional layer parameters).
In the present invention, it should be noted that the standard of model convergence of the convolutional neural network is the loss calculated by the loss function, and does not decrease with the number of iterations any more.
In the invention, the marked iris image has the category of real category, the specific category marking can be that the iris pixels (including black pupil and white sclera) are marked as 1, the rest non-iris pixels (including hair, eyelid, eyelash and light spot) are marked as 0, and the real result is the manual marking result.
For the invention, the training process of the convolutional neural network is an optimization process of convolutional layer parameters, the training aims at optimizing the convolutional layer parameters, and the aim is that for an input iris image, after the convolutional neural network processing, the binarization labeling result of the category of each pixel of the input iris image can be directly obtained.
In the present invention, the output image of the image fusion output layer (fusion layer) of the convolutional neural network needs to be the same as the original input image size, and in the third step, an error calculation needs to be performed for each output image of the plurality of down-sampling layers and the iris image labeled with the key point, and the loss function loss is as follows:
where x (p, q) is the coordinates of each pixel in the image and the loss at the point of pixel (p, q) is calculated as J. Theta is a parameter of the model. 1{ y (p, q) ═ j } denotes that when y (p, q) ═ j, the expression is equal to 1, and otherwise is 0.
For the present invention, in the fourth step, the result of the specific image segmentation can be as shown in fig. 5.
Based on the method for segmenting the iris region in the iris image provided by the invention, referring to fig. 6, the device for segmenting the iris region in the iris image provided by the invention comprises:
a network establishing unit 601, configured to establish a multi-scale convolutional neural network, where the convolutional neural network includes a convolutional layer and a sampling layer that sequentially process an input image;
a pre-key point labeling unit 602, configured to label a pre-selected preset key point in an iris image in advance, and output the pre-selected preset key point to the network training unit 603;
a network training unit 603, connected to the network establishing unit 601 and the pre-key point labeling unit 602, respectively, and configured to input the iris image labeled with the pre-key point into the convolutional neural network established by the network establishing unit 601, and train the convolutional neural network, so that a model of the convolutional neural network converges;
the division execution unit 604: and the network training unit 603 is connected to input the iris image to be tested, which needs to be subjected to iris region segmentation, into the convolutional neural network trained by the network training unit 603, so as to obtain a binary code pattern of the iris image to be tested, wherein the binary code pattern is the iris region to be segmented finally.
In the present invention, the network establishing unit 601, the pre-key point marking unit 602, the network training unit 603, and the dividing execution unit 604 may be a central processing unit CPU, a digital signal processor DSP, or a single-chip microcomputer MCU respectively mounted on a main board of the apparatus of the present invention.
In the present invention, the network establishing unit 601, the pre-keypoint marking unit 602, the network training unit 603, and the division executing unit 604 may be individually configured devices or may be integrally configured together.
It should be noted that, in practical applications, the iris segmentation technique has many challenges, especially in uncontrollable scenes, such as iris images acquired at long distances and in motion, and there are noise interferences, such as illumination and distance variation, high noise, low resolution, occlusion, and blur. The iris image shown in fig. 1 is acquired in an uncontrollable scene, so that the iris segmentation is difficult due to the image quality, and the accuracy of iris identification is directly influenced.
The two traditional iris segmentation methods are respectively based on an edge detection method and a pixel point-based method. The two methods have respective limitations in processing iris images acquired under an uncontrollable scene. The model is used for iris segmentation based on the multi-scale full-convolution neural network, and the model fuses local detail information and global structure information so that iris pixels are accurately positioned.
For the present invention, in the network establishing unit 601, the convolutional neural network is composed of a convolutional layer and a sampling layer, and in order to fuse the global information and the local information of the input image, a plurality of output results and the output result of the last layer may be received from the convolutional neural network for fusion, as shown in fig. 3. Since the neural receptor fields output from the deeper layers of the network gradually increase, the neural receptor fields in the later layers express more global information, and the neural receptor fields in the earlier layers express local information. Therefore, for the invention, in order to make the image model established by the convolutional neural network more robust, the invention uses a multi-scale fusion model to fuse the shallow local information and the deep global information for optimization.
In the present invention, the convolutional neural network consists of only two types of layers: convolutional layers and sampling layers. The convolutional layer is used for performing convolution operation on corresponding data (such as an input iris image), and the sampling layer (only a downsampling layer is used in the network) is used for performing downsampling on the input data, specifically, performing interval sampling on pixel values.
Wherein the convolutional layer formula is as follows:
(Yk)ij＝(Wk*x)i,j+bk；
wherein, Y is an output image, x is an input image, w is a weight, b is a bias term, coordinates of (i, j) pixel points, k is an index of the image (k image), and the use range of w and b is initialized by random numbers from 0 to 1.
In a specific implementation, for a convolutional layer and a sampling layer included in the convolutional neural network, the output of the previous layer is the input of the next layer.
In particular, the invention designs a multi-scale full convolution neural network, automatically learns the input iris image and the labeled image (namely the iris image labeled with the preset key points), and performs convolution operation for many times so as to fit the input iris image to the labeled image (namely the iris image labeled with the preset key points) as much as possible.
Referring to fig. 3, in the present invention, a typical network structure of the convolutional neural network is: based on the existing model of VGG-19layers, the model is formed by only combining a convolutional layer and a downsampling layer, and then five output results of five downsampling layers of an image input layer, pool3, pool6, pool9, pool12 and pool16 are fused and output in an image fusion output layer from front to back, as shown in fig. 4, the detailed network connection of the convolutional neural network is specifically realized.
For the present invention, in the pre-keypoint tagging unit 602, the pre-keypoint tagging unit 602 specifically includes: the system is used for labeling the preselected iris image (specifically, the user can manually label the image through the preselected key point labeling unit 602), and labeling the preselected iris image with the preset key points (including hair, eyelids, eyelashes, light spots and the like) belonging to the pixels of the iris and the non-iris pixels respectively.
In a specific implementation, the preselected iris image may be an iris image which is manually preselected by the user according to the present invention, has high definition and is easy to perform image recognition (the preset key point is clear). For the present invention, the pre-selected pictures need more diversified data, keeping richness.
Specifically, the preset key points of the pixels belonging to the iris in the iris image comprise a black pupil and a white sclera; the preset key points of the non-iris pixels comprise hair, eyelids, eyelashes and light spots; the pixels of the iris and the non-iris pixels may be labeled as 0 and 1, respectively, for example, i.e., a binary code labeling is implemented.
For the present invention, in the network training unit 603, the network training unit 603 specifically includes: the method is used for inputting the iris image labeled with preset key points into the convolutional neural network, training the convolutional neural network by using a random gradient descent method, calculating the class of each pixel of the input iris image through a forward propagation algorithm, calculating the error between the iris image and a true class (for example, the iris pixel and the non-iris pixel can be respectively labeled as 0 and 1, and labeled by binary codes) labeled in advance by calculating the error (namely, a result obtained by performing a series of convolution and sampling operations on one input image), performing convolutional layer parameter replacement by using a backward propagation method, training the model of the convolutional neural network until the model converges, and completing a training process (namely, an optimization process of convolutional layer parameters).
In the present invention, it should be noted that the standard of model convergence of the convolutional neural network is the loss calculated by the loss function, and does not decrease with the number of iterations any more.
In the invention, the marked iris image has the category of real category, the specific category marking can be that the iris pixels (including black pupil and white sclera) are marked as 1, the rest non-iris pixels (including hair, eyelid, eyelash and light spot) are marked as 0, and the real result is the manual marking result.
For the invention, the training process of the convolutional neural network is an optimization process of convolutional layer parameters, the training aims at optimizing the convolutional layer parameters, and the aim is that for an input iris image, after the convolutional neural network processing, the binarization labeling result of the category of each pixel of the input iris image can be directly obtained.
In the present invention, the output image of the image fusion output layer (fusion layer) of the convolutional neural network needs to be the same as the original input image size, and in the third step, an error calculation needs to be performed for each output image of the plurality of down-sampling layers and the iris image labeled with the key point, and the loss function loss is as follows:
where x (p, q) is the coordinates of each pixel in the image and the loss at the point of pixel (p, q) is calculated as J. Theta is a parameter of the model. 1{ y (p, q) ═ j } denotes that when y (p, q) ═ j, the expression is equal to 1, and otherwise is 0.
For the present invention, the result of the specific image segmentation performed by the segmentation performing unit 604 can be as shown in fig. 5.
In order to better understand the technical solution of the present invention, the following embodiments are further described.
Example 1
The invention provides a segmentation method and a segmentation device for iris regions in iris images based on a multi-scale full convolution neural network, which can be applied to an airport entry and exit management system for remote iris recognition.
The invention can be applied to long-distance recognition scenes. With the wide application of iris recognition technology, the demand of remote iris recognition systems is more and more extensive. The remote iris recognition system does not require excessive matched acquisition of the user, only needs to approach an acquisition area, and can automatically acquire the iris picture of the user by utilizing the technical scheme provided by the invention. Because the distance is far away, the user often moves, so the acquired iris image has poor imaging quality and the situations of oblique eyes, shielding and the like. By using the segmentation method of the iris region in the iris image based on the multi-scale full convolution neural network provided by the invention, the airport entry and exit management system can effectively segment the iris region, and then the iris region is compared with the iris image registered in the system, so that the rapid clearance process is realized. The system can greatly improve the flow of the airport and reduce the personnel expenditure and the condition of manual misidentification and misjudgment.
Example 2
The invention provides a multi-scale full convolution neural network-based iris image iris area segmentation method and device, which can be applied to a security system based on iris recognition.
The invention can be widely applied to the scenes of identity authentication and identification by using the iris. For example, a face theft case occurs in a city, and criminals are recorded by the monitoring system at the scene of the crime. Although the criminal suspect is blinded, the eye area of the criminal suspect still leaks out. The police effectively segments the iris area by the iris segmentation method of the multi-scale full convolution neural network. The criminal suspect is found out immediately by extracting the characteristics of the effective iris area and comparing the characteristics with the irises in the police database. With the assistance of the security system based on iris recognition, the police can lock the criminal suspect in a few hours.
Therefore, for the segmentation method and the segmentation device for the iris region in the iris image, which are provided by the invention, the method and the device are based on the multi-scale full convolution neural network, have important significance for improving the accuracy of iris segmentation, and the beneficial effects are embodied in the following aspects:
1. according to the invention, the deep convolutional neural network is used in iris segmentation for the first time, so that the most effective characteristics for iris segmentation can be obtained through automatic learning without manual participation;
2. the invention is an end-to-end method, which saves the pretreatment process of traditional iris segmentation and can directly obtain the effective iris area of the input image;
3. compared with the common deep convolution neural network, the method has the advantages that the full convolution network is used, so that the operation times are effectively reduced, and the segmentation speed is obviously improved;
4. the invention uses a multi-scale network structure, integrates global information and local information, and has more robust and accurate segmentation result;
due to the advantages, the method can segment the low-quality iris image acquired in the uncontrolled scene, and effectively improves the accuracy, robustness and usability of iris segmentation.
In summary, compared with the prior art, the invention provides a segmentation method and device for iris regions in iris images, which can timely and accurately segment iris regions in iris images acquired under an uncontrollable scene, meet the requirements of users on iris segmentation, improve the working efficiency of users, save precious time of people, effectively ensure the accuracy of iris identification on iris images, and have great production practice significance.
By using the technology provided by the invention, the convenience of work and life of people can be greatly improved, and the living standard of people is greatly improved.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.
Claims (4)
1. A segmentation method of an iris region in an iris image is characterized by comprising the following steps:
the first step is as follows: establishing a multi-scale convolutional neural network, wherein the convolutional neural network comprises a convolutional layer and a sampling layer which are used for sequentially processing the input image;
the second step is that: pre-labeling a pre-selected preset key point in the iris image;
the third step: inputting the iris image labeled with preset key points into the convolutional neural network, and training the convolutional neural network to ensure that a model of the convolutional neural network is converged;
the fourth step: inputting the iris image to be tested, which needs iris region segmentation, into the trained convolutional neural network to obtain a binary code image of the iris image to be tested, wherein the binary code image is the iris region which is segmented finally;
the network structure of the convolutional neural network is as follows: based on the existing model of VGG-19layers, the model is formed by only combining a convolution layer and a down-sampling layer, and then five output results of five down-sampling layers, namely a pool3, a pool6, a pool9, a pool12 and a pool16, are respectively output after being fused in an image fusion output layer from front to back;
the second step is specifically as follows: respectively labeling preset key points of pixels belonging to the iris and non-iris pixels in the preselected iris image;
the preset key points of the pixels belonging to the iris in the iris image comprise a black pupil and a white sclera; the preset key points of the non-iris pixels comprise hair, eyelids, eyelashes and light spots;
the third step is specifically as follows:
inputting the iris image labeled with preset key points into the convolutional neural network, training the convolutional neural network by using a random gradient descent method, calculating the class of each pixel of the input iris image through a forward propagation algorithm, performing convolutional layer parameter replacement by calculating the error between the class and a real class labeled in advance of the iris image and using a back propagation method, training a model of the convolutional neural network until the model converges, and finishing the training process;
in the third step, error calculation is carried out on each output image of the plurality of downsampling layers and the iris image marked with the key point, and the loss function loss is as follows:
where x (p, q) is the coordinates of each pixel in the image and the loss at the point of pixel (p, q) is calculated as J; theta is a parameter of the model; 1{ y (p, q) ═ j } denotes that when y (p, q) ═ j, the expression is equal to 1, and otherwise is 0.
2. The method of claim 1, wherein the pixels of the iris and the non-iris pixels are labeled as binary codes 0 and 1, respectively.
3. An iris image iris region segmentation apparatus, comprising:
the network establishing unit is used for establishing a multi-scale convolutional neural network, and the convolutional neural network comprises a convolutional layer and a sampling layer which are used for sequentially processing the input image;
the system comprises a pre-key point marking unit, a network training unit and a network processing unit, wherein the pre-key point marking unit is used for marking a pre-set key point in a pre-selected iris image in advance and then outputting the pre-set key point to the network training unit;
the network training unit is respectively connected with the network establishing unit and the pre-key point marking unit and is used for inputting the iris image marked with the pre-key points into the convolutional neural network established by the network establishing unit and training the convolutional neural network so as to lead the model of the convolutional neural network to be converged;
a division execution unit: the network training unit is connected with the iris image to be tested, which needs iris region segmentation, and the iris image to be tested is input into the convolutional neural network which is trained by the network training unit to obtain a binary code image of the iris image to be tested, wherein the binary code image is the iris region which is segmented finally;
the network structure of the convolutional neural network is as follows: based on the existing model of VGG-19layers, the model is formed by only combining a convolution layer and a down-sampling layer, and then five output results of five down-sampling layers, namely a pool3, a pool6, a pool9, a pool12 and a pool16, are respectively output after being fused in an image fusion output layer from front to back;
the pre-key point marking unit specifically comprises: the preset key points are used for respectively marking the pixels which respectively belong to the iris and the non-iris pixels in the preselected iris image;
the preset key points of the pixels belonging to the iris in the iris image comprise a black pupil and a white sclera; the preset key points of the non-iris pixels comprise hair, eyelids, eyelashes and light spots;
the network training unit is specifically as follows: the system comprises a convolutional neural network, a random gradient descent method, a forward propagation algorithm and a back propagation algorithm, wherein the convolutional neural network is used for inputting the iris image labeled with preset key points into the convolutional neural network, training the convolutional neural network by using the random gradient descent method, calculating the class of each pixel of the input iris image by using the forward propagation algorithm, calculating the error between the class and the real class labeled in advance of the iris image, and training a model of the convolutional neural network by using the back propagation method until the model converges to finish the training process;
in the third step, error calculation is carried out on each output image of the plurality of downsampling layers and the iris image marked with the key point, and the loss function loss is as follows:
where x (p, q) is the coordinates of each pixel in the image and the loss at the point of pixel (p, q) is calculated as J; theta is a parameter of the model; 1{ y (p, q) ═ j } denotes that when y (p, q) ═ j, the expression is equal to 1, and otherwise is 0.
4. The apparatus of claim 3, wherein the pixels of the iris and the non-iris pixels are labeled as binary codes 0 and 1, respectively.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611242734.6A CN106778664B (en) | 2016-12-29 | 2016-12-29 | Iris image iris area segmentation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611242734.6A CN106778664B (en) | 2016-12-29 | 2016-12-29 | Iris image iris area segmentation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106778664A CN106778664A (en) | 2017-05-31 |
| CN106778664B true CN106778664B (en) | 2020-12-15 |
Family
ID=58923725
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611242734.6A Active CN106778664B (en) | 2016-12-29 | 2016-12-29 | Iris image iris area segmentation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106778664B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112801067A (en) * | 2021-04-13 | 2021-05-14 | 北京万里红科技股份有限公司 | Method for detecting iris light spot and computing equipment |
Families Citing this family (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10628919B2 (en) * | 2017-08-31 | 2020-04-21 | Htc Corporation | Image segmentation method and apparatus |
| CN108230335B (en) * | 2017-12-29 | 2020-12-01 | 北京数字精准医疗科技有限公司 | Multispectral fluorescence segmentation method and device |
| CN108335303B (en) * | 2018-01-28 | 2021-01-08 | 浙江大学 | Multi-scale palm skeleton segmentation method applied to palm X-ray film |
| CN108765422A (en) * | 2018-06-13 | 2018-11-06 | 云南大学 | A kind of retinal images blood vessel automatic division method |
| CN109087327B (en) * | 2018-07-13 | 2021-07-06 | 天津大学 | Thyroid nodule ultrasonic image segmentation method of cascaded full convolution neural network |
| CN110738212B (en) * | 2018-07-19 | 2022-11-08 | 北京四维图新科技股份有限公司 | Method and device for acquiring convolutional network model and processing equipment |
| US10957048B2 (en) * | 2018-09-05 | 2021-03-23 | Htc Corporation | Image segmentation method, apparatus and non-transitory computer readable medium of the same |
| CN109815850B (en) * | 2019-01-02 | 2020-11-10 | 中国科学院自动化研究所 | Iris image segmentation and positioning method, system and device based on deep learning |
| CN111507334B (en) * | 2019-01-30 | 2024-03-12 | 中国科学院宁波材料技术与工程研究所 | Instance segmentation method based on key points |
| CN110008851B (en) * | 2019-03-15 | 2021-11-19 | 深兰科技（上海）有限公司 | Method and equipment for detecting lane line |
| CN110059589B (en) * | 2019-03-21 | 2020-12-29 | 昆山杜克大学 | Iris region segmentation method in iris image based on Mask R-CNN neural network |
| CN110321844B (en) * | 2019-07-04 | 2021-09-03 | 北京万里红科技股份有限公司 | Fast iris detection method based on convolutional neural network |
| CN110688951B (en) * | 2019-09-26 | 2022-05-31 | 上海商汤智能科技有限公司 | Image processing method and device, electronic equipment and storage medium |
| CN110866507A (en) * | 2019-11-20 | 2020-03-06 | 北京工业大学 | Method for protecting mobile phone chatting content based on iris recognition |
| CN111161276B (en) * | 2019-11-27 | 2023-04-18 | 天津中科智能识别产业技术研究院有限公司 | Iris normalized image forming method |
| CN111027464B (en) * | 2019-12-09 | 2023-07-18 | 大连理工大学 | Iris recognition method for jointly optimizing convolutional neural network and sequence feature coding |
| CN111507208B (en) * | 2020-03-30 | 2021-06-25 | 中国科学院上海微系统与信息技术研究所 | Identity verification method, device, equipment and medium based on sclera identification |
| CN112287872B (en) * | 2020-11-12 | 2022-03-25 | 北京建筑大学 | Iris image segmentation, positioning and normalization method based on multitask neural network |
| CN112651328B (en) * | 2020-12-23 | 2022-09-13 | 浙江中正智能科技有限公司 | Iris segmentation method based on geometric position relation loss function |
| CN113706469B (en) * | 2021-07-29 | 2024-04-05 | 天津中科智能识别产业技术研究院有限公司 | Iris automatic segmentation method and system based on multi-model voting mechanism |
| CN113689385B (en) * | 2021-07-29 | 2023-10-20 | 天津中科智能识别产业技术研究院有限公司 | Automatic iris inner and outer circle boundary segmentation method, device, equipment and storage medium |
| CN113688874A (en) * | 2021-07-29 | 2021-11-23 | 天津中科智能识别产业技术研究院有限公司 | Method and system for automatically segmenting iris region in human eye iris image |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105303185A (en) * | 2015-11-27 | 2016-02-03 | 中国科学院深圳先进技术研究院 | Iris positioning method and device |
| CN105512624A (en) * | 2015-12-01 | 2016-04-20 | 天津中科智能识别产业技术研究院有限公司 | Smile face recognition method and device for human face image |
| CN105608450A (en) * | 2016-03-01 | 2016-05-25 | 天津中科智能识别产业技术研究院有限公司 | Heterogeneous face identification method based on deep convolutional neural network |
-
2016
- 2016-12-29 CN CN201611242734.6A patent/CN106778664B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105303185A (en) * | 2015-11-27 | 2016-02-03 | 中国科学院深圳先进技术研究院 | Iris positioning method and device |
| CN105512624A (en) * | 2015-12-01 | 2016-04-20 | 天津中科智能识别产业技术研究院有限公司 | Smile face recognition method and device for human face image |
| CN105608450A (en) * | 2016-03-01 | 2016-05-25 | 天津中科智能识别产业技术研究院有限公司 | Heterogeneous face identification method based on deep convolutional neural network |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度反卷积网络的虹膜定位方法研究;徐霄等;《集成技术》;20160131;第57-67页 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112801067A (en) * | 2021-04-13 | 2021-05-14 | 北京万里红科技股份有限公司 | Method for detecting iris light spot and computing equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106778664A (en) | 2017-05-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106778664B (en) | Iris image iris area segmentation method and device | |
| CN111460962B (en) | Face recognition method and face recognition system for mask | |
| CN101142584B (en) | Method for facial features detection | |
| CN109815850A (en) | Iris segmentation and localization method, system, device based on deep learning | |
| CN109145742B (en) | Pedestrian identification method and system | |
| CN110210276A (en) | A kind of motion track acquisition methods and its equipment, storage medium, terminal | |
| US20210034840A1 (en) | Method for Recognzing Face from Monitoring Video Data | |
| CN105608450A (en) | Heterogeneous face identification method based on deep convolutional neural network | |
| CN105117692A (en) | Real-time face identification method and system based on deep learning | |
| WO2016010721A1 (en) | Multispectral eye analysis for identity authentication | |
| CN102324025A (en) | Human face detection and tracking method based on Gaussian skin color model and feature analysis | |
| KR20080025610A (en) | Iris recognition method and apparatus for thereof | |
| CN101201893A (en) | Iris recognizing preprocessing method based on grey level information | |
| CN108537143B (en) | A kind of face identification method and system based on key area aspect ratio pair | |
| CN107066969A (en) | A kind of face identification method | |
| CN103902962A (en) | Shielding or light source self-adaption human face recognition method and device | |
| CN107169479A (en) | Intelligent mobile equipment sensitive data means of defence based on fingerprint authentication | |
| CN105022999A (en) | Man code company real-time acquisition system | |
| Reddy et al. | A robust scheme for iris segmentation in mobile environment | |
| CN111259757B (en) | Living body identification method, device and equipment based on image | |
| CN103034851A (en) | Device and method of self-learning skin-color model based hand portion tracking | |
| CN106156739B (en) | A kind of certificate photo ear detection and extracting method based on face mask analysis | |
| CN108520529A (en) | Visible light based on convolutional neural networks and infrared video method for tracking target | |
| CN113920591A (en) | Medium-distance and long-distance identity authentication method and device based on multi-mode biological feature recognition | |
| Liu et al. | Implementation System of Human Eye Tracking Algorithm Based on FPGA. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 300457 unit 1001, block 1, msd-g1, TEDA, No.57, 2nd Street, Binhai New Area Economic and Technological Development Zone, Tianjin Patentee after: Tianjin Zhongke intelligent identification Co.,Ltd. Address before: 300457 No. 57, Second Avenue, Economic and Technological Development Zone, Binhai New Area, Tianjin Patentee before: TIANJIN ZHONGKE INTELLIGENT IDENTIFICATION INDUSTRY TECHNOLOGY RESEARCH INSTITUTE Co.,Ltd. |
|
| CP03 | Change of name, title or address |