WO2020258981A1 - 基于眼底图像的身份信息处理方法及设备 - Google Patents

基于眼底图像的身份信息处理方法及设备 Download PDF

Info

Publication number
WO2020258981A1
WO2020258981A1 PCT/CN2020/083625 CN2020083625W WO2020258981A1 WO 2020258981 A1 WO2020258981 A1 WO 2020258981A1 CN 2020083625 W CN2020083625 W CN 2020083625W WO 2020258981 A1 WO2020258981 A1 WO 2020258981A1
Authority
WO
WIPO (PCT)
Prior art keywords
fundus image
feature vector
dimensional feature
fundus
distance
Prior art date
Application number
PCT/CN2020/083625
Other languages
English (en)
French (fr)
Inventor
和宗尧
熊健皓
付萌
朱小沛
赵昕
和超
张大磊
Original Assignee
上海鹰瞳医疗科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海鹰瞳医疗科技有限公司 filed Critical 上海鹰瞳医疗科技有限公司
Priority to US17/619,875 priority Critical patent/US11893831B2/en
Priority to EP20833250.2A priority patent/EP3992844A4/en
Publication of WO2020258981A1 publication Critical patent/WO2020258981A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31—User authentication
    • G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/14—Vascular patterns
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/197—Matching; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/50—Maintenance of biometric data or enrolment thereof
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing

Definitions

  • the invention relates to the field of image information processing, in particular to a method and equipment for processing identity information based on fundus images.
  • Fundus images also called retinal images
  • retinal images taken by a fundus camera can reflect human tissues such as the macula, optic disc, and blood vessels. Because the blood vessel direction, bifurcation point, and the shape of the optic disc are all different from person to person, the fundus image has a strong uniqueness and generally does not change greatly with people's age.
  • fundus images can be used for identity recognition, and for this, it is necessary to establish an identity information database based on fundus images.
  • a key issue is what information is stored.
  • Most documents refer to fingerprint or facial recognition technology, and use computer vision to identify and extract some so-called shape feature information of key points, and store this information in the database for subsequent follow-up Comparison, but the situation of the fundus is very different from the situation of fingerprints and faces. It is difficult to determine the so-called key points, and it is also difficult to find the corresponding key points in the two fundus images, so its practicality is very poor. Therefore, some existing technologies adopt a method of directly storing a user's fundus image, that is, the fundus image is directly used as user identity information.
  • the currently obtained user identity information When faced with a database with a large amount of user identity information, for the currently obtained user identity information, first determine whether the user’s identity information has been stored in the database. If the fundus image is stored in the database, it needs to be stored in the database. All of the fundus images are compared with the current fundus images, which is very inefficient.
  • the present invention provides an identity information comparison method based on fundus images, including:
  • the neural network is obtained by training using triad sample data.
  • the triad sample data includes a first fundus image sample, a second fundus image sample, and a third fundus image sample, wherein the The second fundus image sample and the first fundus image sample are fundus images of the same person, and the third fundus image sample and the first fundus image sample are fundus images of different people.
  • the neural network extracts the first fundus image sample, the second fundus image sample, and the multi-dimensional feature vector of the third fundus image sample, respectively, according to the extraction
  • the obtained three multi-dimensional feature vectors calculate the first distance between the second fundus image sample and the first fundus image sample, and calculate the second distance between the third fundus image sample and the first fundus image sample, And obtain a loss value according to the first distance and the second distance, thereby adjusting the parameters of the neural network according to the loss value.
  • the adjusting the parameters of the neural network according to the loss value includes:
  • the loss value is fed back to the neural network, so that the parameter is adjusted according to the loss value to reduce the first distance and increase the second distance until the first distance is greater than the second distance Less than the preset value.
  • judging whether there is a pre-stored multi-dimensional feature vector matching the currently obtained multi-dimensional feature vector in the database according to the matching result including:
  • the fundus image includes a left eye fundus image and a right eye fundus image
  • the database is used to store user data, wherein each group of user data includes a first pre-stored multi-dimensional feature vector corresponding to the left eye and The second pre-stored multi-dimensional feature vector of the right eye; the multi-dimensional feature vector includes a first multi-dimensional feature vector corresponding to the fundus image of the left eye and a second multi-dimensional feature vector corresponding to the fundus image of the right eye.
  • the fundus image includes a left eye fundus image and a right eye fundus image
  • the database is used for storing user data, wherein each group of user data includes a pre-stored multi-dimensional feature vector
  • the obtaining a multi-dimensional feature vector used to represent the user identity includes:
  • the fundus image is a left eye fundus image or a right eye fundus image
  • the database is used to store user data, wherein each group of user data includes the first pre-stored multi-dimensional feature vector corresponding to the left eye or corresponds to The second pre-stored multi-dimensional feature vector of the right eye.
  • the present invention also provides an identity verification method based on fundus images, including:
  • the identification information comparison method based on the fundus image is used to determine whether there is a pre-stored multi-dimensional feature vector matching the multi-dimensional feature vector of the fundus image in the database, and thus the confirmation of the user's identity is completed.
  • the present invention also provides a method for storing identity information based on fundus images, including:
  • the currently obtained multi-dimensional feature vector is stored in the database as the user's identity information.
  • the present invention also provides a method for training fundus image recognition model, including:
  • training data includes a first fundus image sample, a second fundus image sample, and a third fundus image sample, wherein the second fundus image sample and the first fundus image sample are fundus images of the same person;
  • the third fundus image sample and the first fundus image sample are fundus images of different people;
  • the parameters of the fundus image recognition model are adjusted according to the loss value.
  • using a fundus image recognition model to recognize the first fundus image sample, the second fundus image sample, and the third fundus image sample to obtain a loss value includes:
  • the loss value is obtained according to the first distance and the second distance.
  • the adjusting the parameters of the fundus image recognition model by using the loss value includes:
  • the parameter is adjusted according to the loss value to reduce the first distance and increase the second distance until the first distance is less than a preset value than the second distance.
  • the present invention also provides a device for comparing identity information based on fundus images, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores the An instruction executed by a processor, the instruction being executed by the at least one processor, so that the at least one processor executes the above-mentioned method for comparing identity information based on fundus images.
  • the present invention also provides an identity verification device based on fundus images, which includes: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores data that can be processed by the one processor; The instructions are executed by the at least one processor, so that the at least one processor executes the above-mentioned fundus image-based identity verification method.
  • the present invention also provides an identity information storage device based on fundus images, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores the storage device that can be used by the one An instruction executed by the processor, the instruction being executed by the at least one processor, so that the at least one processor executes the foregoing method for storing identity information based on fundus images.
  • the present invention also provides a fundus image recognition model training device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores the memory that can be used by the one processor
  • the executed instruction is executed by the at least one processor, so that the at least one processor executes the above-mentioned fundus image recognition model training method.
  • a neural network is first used to convert a user's fundus image into a multi-dimensional feature vector for expressing the user's identity, and the characteristics of the neural network are used to extract personal characteristics of the user
  • Related abstract feature information by comparing the multi-dimensional feature vector during the comparison, it can be judged whether there is data matching the current user in the database.
  • the database does not need to store fundus images, nor does it need to obtain a new one every time. Re-identify the pre-stored fundus images for all fundus images, which can improve the efficiency of identity information comparison operations.
  • the neural network provided by the present invention is trained through triple training data and the corresponding loss function, which can reduce the distance between the feature vectors extracted by the neural network for different fundus images of the same eye, and increase the distance between the feature vectors for different eyes.
  • the neural network After training, the neural network has a sufficiently small distance for the feature vectors extracted multiple times for the same fundus image, and the distance between the feature vectors of other fundus images is large enough, and this information has Certain uniqueness, the feature vector extracted by the neural network for fundus images can be used as user identity information.
  • FIG. 1 is a flowchart of an identity information comparison method in an embodiment of the present invention
  • FIG. 2 is a flowchart of a method for storing identity information in an embodiment of the present invention
  • FIG. 3 is a schematic diagram of using neural network to extract identity information in an embodiment of the present invention.
  • FIG. 4 is another schematic diagram of using neural network to extract identity information in an embodiment of the present invention.
  • Figure 5 is a flowchart of a neural network training method in an embodiment of the present invention.
  • Figure 6 is an image block in the fundus image
  • Fig. 7 is a segmentation result for the image block shown in Fig. 6;
  • Figure 8 is an image of fundus blood vessels.
  • the embodiment of the present invention provides a method for storing identity information based on fundus images.
  • the method can be executed by electronic devices such as a computer or a server. As shown in Figure 1, the method includes the following steps:
  • S1 Use neural network to recognize fundus images and obtain multi-dimensional feature vectors for representing the user's identity.
  • the neural network extracts feature information when recognizing images.
  • the neural network focuses on different content and the extracted feature information is also different.
  • the neural network will extract corresponding feature information (usually a multi-dimensional feature vector) for the category to which the fundus image belongs, and then classify according to the feature information.
  • the neural network used is configured to perform extraction of a multi-dimensional feature vector used to represent the user's identity, rather than performing a certain classification or image segmentation task.
  • the multi-dimensional feature vector extracted by the neural network from the fundus image should be different.
  • the multi-dimensional feature vector extracted each time should be Are the same (or roughly the same, similar).
  • the neural network described in this application may specifically be a deep convolutional network (Convolutional Neural Network, CNN), which can standardize the extracted multi-dimensional feature vector by setting an appropriate loss function and using a back propagation algorithm (BP).
  • CNN Deep convolutional Network
  • BP back propagation algorithm
  • a trained CNN model can obtain feature vectors from a fundus image, and these feature vectors are generally high-dimensional vectors.
  • S2 Compare the obtained multi-dimensional feature vector with each pre-stored multi-dimensional feature vector in the database. According to the location of the database, it can be divided into GPU-based database and CPU-based database.
  • the multi-dimensional feature vectors pre-stored in the database may also be multi-dimensional feature vectors extracted from other fundus images by using the neural network in step S1.
  • Euclidean Distance measures the absolute distance between points in a multi-dimensional space. This is a preferred way to judge when the data is dense and continuous. Since the calculation is based on the absolute value of the characteristics of each dimension, the Euclidean metric needs to ensure that the indicators of each dimension are at the same scale level.
  • Mahalanobis Distance is a distance based on sample distribution. For example, for two normal distribution populations, their mean values are a and b respectively, but the variances are different. Among them, the sample point A has a greater probability of which distribution belongs to in the distribution space, then A belongs to the distribution.
  • the distance is calculated to measure the similarity of the two feature vectors.
  • the distance is preferably Euclidean distance.
  • the angle between each pre-stored multi-dimensional feature vector and the currently obtained multi-dimensional feature vector can be calculated separately, and it is also feasible to measure the similarity between the two.
  • the distance between the two is less than the set threshold, it can be judged that the two have a sufficiently high similarity, indicating that the two are matched.
  • a neural network is first used to convert a user's fundus image into a multi-dimensional feature vector for expressing the user's identity, and the characteristics of the neural network are used to extract the personal characteristics of the user.
  • Related abstract feature information by comparing the multi-dimensional feature vector during the comparison, it can be judged whether there is data matching the current user in the database.
  • the database does not need to store fundus images, nor does it need to obtain a new one every time. Re-identify the pre-stored fundus images for all fundus images, which can improve the efficiency of identity information comparison operations.
  • each feature vector stored in the database is a fundus image from a certain eye.
  • step S1 the fundus image includes the left eye fundus image and the right eye fundus image
  • the neural network is used to identify the two fundus images respectively, and obtain the corresponding left eye fundus image.
  • each group of data stored in the database includes two pre-stored multi-dimensional feature vectors, and the corresponding comparison can be performed in step S2.
  • this result can be accepted in some application scenarios. For example, this result can be accepted when creating or updating a database; however, some application scenarios cannot allow this situation, such as when performing authentication.
  • this solution performs unified processing on the user's binocular fundus images, and the recognized fundus images include the left eye fundus image and the right eye fundus image.
  • the neural network respectively recognizes the binocular fundus images, and outputs a first multi-dimensional feature vector corresponding to the left eye fundus image and a second multi-dimensional feature vector corresponding to the right eye fundus image. Then the two feature vectors are merged in multiple ways. For example, two 1024-dimensional feature vectors can be connected into a 2048-dimensional feature vector.
  • the pre-stored data in the database is a combined multi-dimensional feature vector, which is the result of pre-merging two feature vectors.
  • the currently merged dimensional feature vector is compared with the multi-dimensional feature vector pre-stored in the database, and then it is determined whether there is matching data according to the comparison result. In this embodiment, only one comparison is performed. Yes, you can determine whether the eyes match.
  • the following introduces a method for storing identity information based on fundus images.
  • the method uses the above comparison scheme to establish and manage an identity information database. As shown in FIG. 2, the method includes the following steps:
  • S1A Acquire a fundus image of the user.
  • the fundus image may be any or all of the eyes.
  • the neural network is used to identify fundus images and obtain multi-dimensional feature vectors that represent the user's identity. For details, refer to the above step S1, which will not be repeated here.
  • step S3A compare the obtained multi-dimensional feature vector with each pre-stored multi-dimensional feature vector in the database. For details, refer to the above step S2, which will not be repeated here.
  • step S4A judges whether there is a pre-stored multi-dimensional feature vector matching the currently obtained multi-dimensional feature vector in the database.
  • step S5A is executed, otherwise, step S6A is executed.
  • the initial database can be empty. If there is no pre-stored multi-dimensional feature vector, there is no matching content; the initial database can also be pre-imported with multiple pieces of data, which use individuals as the basic module, such as each user
  • the data includes the user's name and its multi-dimensional feature vector (obtained based on the user's fundus image), and may also include personal information such as the user's gender, age, and degree of myopia.
  • the system can prompt whether it is necessary to add the currently obtained multi-dimensional feature vector as a new data to the original database. If yes, it will issue Apply for memory, and when the application is successful, add it to the original database. Further, the system may also prompt whether to continue to input related supplementary information, such as personal information such as the user's gender, age, myopia degree, etc., and store it in association with the corresponding multi-dimensional feature vector after the information is input.
  • related supplementary information such as personal information such as the user's gender, age, myopia degree, etc.
  • S6A Use the currently obtained multi-dimensional feature vector to replace the pre-stored multi-dimensional feature vector in the database that matches it. Specifically, when a match with the current multi-dimensional feature vector is found in the database, the system can prompt whether it is necessary to use the currently obtained multi-dimensional feature vector to overwrite the existing data, and if yes, overwrite. This step is not a necessary operation. As a replacement scheme, when a match with the current multi-dimensional feature vector is found in the database, the system can only prompt to indicate that the current user data already exists, and no replacement operation is necessary.
  • a neural network is first used to convert a user's fundus image into a multi-dimensional feature vector for expressing the user's identity, and the characteristics of the neural network are used to extract the personal characteristics related to the user
  • the database does not need to store fundus images. It is also unnecessary to use neural networks to identify existing fundus images when storing each user's identity information, thereby improving the efficiency of storing user identity information.
  • each piece of user information in the database established according to this solution may include user name, multi-dimensional feature vector (the first multi-dimensional feature vector of the left eye and/or the second multi-dimensional feature vector of the right eye or two The result of the merger), user gender, age, myopia, etc.
  • the multi-dimensional feature vector sum represents the user's identity information.
  • step S4A determines whether the identity information of the current user exists in the database.
  • step S6A replaces step S5A with determining that the current user is not a known user, thereby not allowing it to perform subsequent operations
  • step S6A replaces step S6A with determining that the current user is a known user, thereby allowing it to perform subsequent operations.
  • the subsequent operations can be It is to unlock or log in the electronic device to realize identity verification or authentication and so on.
  • the present invention also provides a device for comparing identity information based on fundus images, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores the information that can be used by the one processor
  • the executed instruction is executed by the at least one processor, so that the at least one processor executes the above-mentioned method for comparing identity information based on fundus images.
  • the present invention also provides an identity information storage device based on fundus images, including: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores the memory that can be executed by the one processor The instructions are executed by the at least one processor, so that the at least one processor executes the above-mentioned method for storing identity information based on fundus images.
  • the present invention also provides an identity verification device based on fundus images, including: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores a memory that can be executed by the one processor Instructions, the instructions are executed by the at least one processor, so that the at least one processor executes the above-mentioned fundus image-based identity verification method.
  • the following describes how to obtain a model for generating feature vectors (the neural network in the above embodiment) with reference to FIGS. 3 to 5.
  • the characteristics required to clearly extract the features are: optimizing the intra-class distance to make the intra-class distance more compact; optimizing the inter-class distance to make the distance between classes more distinguishable.
  • the intra-class distance refers to the distance between the sample points of the same type; the inter-class distance (inter-class) refers to the distance between different classes.
  • Figure 3 is a training framework of a softmax-based classification model.
  • the framework of this classification model can be applied to the training and extraction of fundus image identity features, and on this basis, it provides multiple auxiliary losses. Functions are used to improve the identity characteristics of fundus images.
  • a fundus image Imgi enters a deep convolutional network to obtain a multi-dimensional feature vector xi, and then passes through a fully connected layer and then passes through softmax to obtain a score vector.
  • Softmax loss Softmax loss:
  • N is the batch size
  • n is the number of categories
  • xi is the feature vector of the i-th fundus image of a batch
  • yi is the true identity classification (label) of this fundus image
  • Wj is the weight W of the final fully connected layer.
  • j column vector Indicates the probability value of predicting xi as its true identity classification yi
  • bj is the bias.
  • the joint supervision of softmax loss (Ls) and center loss (Lc) is used to train the deep convolutional network for discriminative feature learning. Center loss can effectively characterize changes within a class.
  • the scalar ⁇ is used to balance Ls and Lc; when ⁇ takes different values, the distribution of the feature vector is different.
  • xi is the feature vector of the ith image in a batch
  • yi is the identity label of the user to which the fundus image xi belongs
  • Cyi ⁇ Rd represents the center of the feature vector of the yith user identity classification label
  • d represents the dimension of the feature vector.
  • the scalar ⁇ [0,1] can be used to control the learning rate of the center.
  • the feature of this solution is that without the need to re-complex the training set, it can ensure that the features of different categories can be separated, while minimizing the difference in internal categories.
  • the n-channel softmax layer is introduced after the deep convolutional network structure to obtain the recognition module used to classify the input fundus image identity category, where n is the number of user identity categories and is defined as the recognition loss function ( Identification Loss):
  • cross-entropy loss (cross-entropy loss)
  • f is the feature vector
  • t is the user identity category corresponding to the feature vector f
  • ⁇ id is the parameter of the softmax layer
  • pi is the target probability distribution.
  • the fundus image pair is input to the deep convolutional neural network to obtain the image characteristics, and the recognition loss is calculated for the fundus image pair respectively, that is, the user identity classification is performed on the two input fundus images, and the fundus image pair is verified at the same time.
  • m (margin) defines the interval between classes, only when the distance of the image pair with different identities is within m distance, it will have an effect on the loss value, which is an effective constraint
  • the distance between classes makes the distance between classes more separable.
  • the embodiment of the present invention also provides a model training method based on triple data.
  • the method includes the following steps:
  • the training data includes a first fundus image sample, a second fundus image sample, and a third fundus image sample.
  • the training data used in this embodiment is three-tuple sample data, where the second fundus image sample and the first fundus image sample are the fundus images of the same eye of the same person; the third fundus image sample and the first fundus image sample Fundus images for different people.
  • the data set is first prepared.
  • the data set may be composed of fundus images of n eyes, where each eye corresponds to m fundus images, that is, there are n*m fundus images in the data set.
  • each fundus image can be trimmed first. Since the original image of the captured fundus image has more black backgrounds, the fundus image may be trimmed first. The large black pixels in the background are removed, and the fundus images are cropped to the smallest rectangle that can encompass the entire circular fundus.
  • all fundus images can be cropped into a unified format, for example, the size is unified to 224*224 pixels, and the input image format during model training and recognition can be unified 224*224 pixels and RGB. Fundus image of the color channel.
  • S2B Use the neural network to identify the first fundus image sample, the second fundus image sample, and the third fundus image sample to obtain the loss value.
  • Use the preset loss function to calculate the first distance between the second fundus image sample and the first fundus image sample, and calculate the second distance between the third fundus image sample and the first fundus image sample. According to the first distance and The second distance gets the loss value.
  • the neural network performs feature extraction on the above three fundus images respectively, and obtains three multi-dimensional feature vectors, which are respectively recorded as: Then calculate versus The first distance between, calculate versus The second distance between.
  • the first distance and the second distance are Euclidean distances.
  • the following loss function relation can be used for calculation:
  • ⁇ represents a preset value
  • the preset value is the minimum interval between the first distance and the second distance.
  • the neural network In actual training, a large amount of the above-mentioned triple sample data needs to be used, so that the neural network continuously adjusts the parameters until the loss function converges.
  • the distance between Anchor and positive In the process of neural network transmission loss, the distance between Anchor and positive should be reduced, and the distance between Anchor and Negative should be increased, so that there is a minimum interval ⁇ between the first and second distances.
  • the training data can be enhanced before training.
  • the data enhancement process can use rotation, translation, magnification, and principal component transformation (PCA) color enhancement.
  • PCA principal component transformation
  • each fundus image can generate multiple fundus images using random enhancement parameters.
  • the format of the fundus image after data enhancement can adopt a unified fundus image of 224*224 pixels and three color channels of RGB.
  • the order of processing is not limited.
  • the distance between the feature vectors extracted by the neural network from different fundus images of the same eye can be gradually reduced by training with the triple training data and the corresponding loss function. And increase the distance between the feature vectors extracted for the fundus images of different eyes. After training, the distance between the feature vectors extracted multiple times for the same fundus image is small enough, and the distance between the feature vectors of other fundus images The distance is large enough, that is, the information has a certain uniqueness, so the feature vector extracted by the neural network for the fundus image can be used as the user identity information.
  • the fundus image in order to further eliminate interference image information that is not related to fundus recognition and improve the recognition ability of the neural network, before training, the fundus image may also be segmented to obtain fundus feature images as training data.
  • the fundus image can be divided into multiple image blocks.
  • the size of the image block is set according to the size of the fundus image. In most cases, the size of the divided image block should be significantly smaller than the size of the entire fundus image. For example, the size of the fundus image is 1000*1000 (pixels), and the size of the divided image block is 100*100 (pixels).
  • the segmentation model can be a neural network such as FCN, SegNet, DeepLab, etc. Before using the segmentation model, you should use sample data to train it. It has certain semantic segmentation capabilities, which can be specifically trained using sample image blocks with artificially labeled blood vessel regions.
  • the segmentation model will extract the features of the blood vessel image in the image block, and form segmented image blocks based on the extracted features, highlight the blood vessel image in it, there are many specific highlighting methods, such as using various pixel values that are significantly different from the background to express the blood vessels Location and so on.
  • the segmentation model used in this embodiment outputs a binary image, which uses two pixel values to express separately Background and blood vessel images, visually highlight the position of blood vessels. Using segmented image blocks to splice the fundus blood vessel image to obtain the image shown in FIG. 8, and then use the image shown in FIG. 8 as the training data.
  • similar methods can also be used to extract other features such as: optic disc, macula, and retina.
  • optic disc macula
  • retina Through the extraction of fundus features, interference image information that is not related to fundus identification can be greatly eliminated, and the recognition performance of the model can be significantly improved.
  • the fundus feature image There may also be high-level indirect features (or abstract features) in the fundus feature image, such as the position and direction of blood vessel bifurcation points, the position and direction of blood vessel intersections, and blood vessel vector diagrams. After obtaining the original fundus image, the above indirect features can also be extracted from it as training data.
  • the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
  • a computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
  • the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing functions specified in a flow or multiple flows in the flowchart and/or a block or multiple blocks in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Evolutionary Computation (AREA)
  • Ophthalmology & Optometry (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Computer Security & Cryptography (AREA)
  • Epidemiology (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Computer Hardware Design (AREA)
  • Vascular Medicine (AREA)
  • Public Health (AREA)
  • Primary Health Care (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Eye Examination Apparatus (AREA)
  • Image Analysis (AREA)
  • Collating Specific Patterns (AREA)

Abstract

本发明提供基于眼底图像的身份信息处理方法及设备，其中一种身份信息比对方法包括：利用神经网络识别眼底图像，获得用于表示用户身份的多维特征向量；将获得的多维特征向量与数据库中的各个预存多维特征向量进行比对；根据比对结果判断所述数据库中是否已存有与当前获得的多维特征向量相匹配的预存多维特征向量。

Description

基于眼底图像的身份信息处理方法及设备 技术领域
本发明涉及图像信息处理领域，具体涉及一种基于眼底图像的身份信息处理方法及设备。
背景技术
通过眼底相机所拍摄的眼底图像(也称为视网膜图像)能够体现如黄斑、视盘、血管等人体组织。由于其血管走向、分叉点、视盘的形状都具备因人而异的特点，因此眼底图像具有很强的唯一性，并且一般不会随着人的年龄增长而发生很大的变化。
现有技术认为眼底图像可以用于身份识别，为此则需要建立基于眼底图像的身份信息数据库。在建立数据库时，一个关键问题是存储什么信息，大多数文献参考指纹或面部识别技术，通过计算机视觉手段识别并提取一些所谓关键点的形状特征信息，并将这些信息存储到数据库中以供后续比对，但是眼底的情况与指纹和面部的情况很不相同，很难确定所谓的关键点，也很难找到两张眼底图像中相应的关键点，所以其实用性很差。因此一些现有技术采用直接存储用户眼底图像的方式，也即将眼底图像直接作为用户身份信息。
面对具有大量用户身份信息的数据库时，对于当前获得的用户身份信息，首先要确定数据库中是否已经存储了该用户的身份信息，如果数据库中存储的是眼底图像，则需要将数据库中已经存储的全部眼底图像分别与当前的眼底图像分别进行比对，这种方式效率很低。
发明内容
有鉴于此，本发明提供一种基于眼底图像的身份信息比对方法，包括：
利用神经网络识别眼底图像，获得用于表示用户身份的多维特征向量；
将获得的多维特征向量与数据库中的各个预存多维特征向量进行比对；
根据比对结果判断所述数据库中是否已存有与当前获得的多维特征向量相匹配的预存多维特征向量。
可选地，所述神经网络是利用三元组样本数据进行训练得到的，所述三元组样本数据包括第一眼底图像样本、第二眼底图像样本以及第三眼底图像样本，其中，所述第二 眼底图像样本与所述第一眼底图像样本为同一人的眼底图像，所述第三眼底图像样本与所述第一眼底图像样本为不同人的眼底图像。
可选地，在所述神经网络的训练过程中，所述神经网络分别提取所述第一眼底图像样本、所述第二眼底图像样本以及所述第三眼底图像样本的多维特征向量，根据提取到的三个多维特征向量计算所述第二眼底图像样本与所述第一眼底图像样本的第一距离，以及计算所述第三眼底图像样本与所述第一眼底图像样本的第二距离，并根据所述第一距离和所述第二距离得到损失值，从而根据所述损失值调整所述神经网络的参数。
可选地，所述根据所述损失值调整所述神经网络的参数包括：
将所述损失值反馈至所述神经网络，使其根据所述损失值调整所述参数以减小所述第一距离增大所述第二距离直至所述第一距离比所述第二距离小于预设值。
可选地，根据别对结果判断所述数据库中是否已存有与当前获得的多维特征向量相匹配的预存多维特征向量，包括：
分别计算各个预存多维特征向量与当前获得的多维特征向量的距离；
根据所述距离判断各个预存多维特征向量是否与当前获得的多维特征向量向匹配。
可选地，所述眼底图像包括左眼眼底图像和右眼眼底图像，所述数据库用于存储用户数据，其中每一组用户数据分别包括对应于左眼的第一预存多维特征向量和对应于右眼的第二预存多维特征向量；所述多维特征向量包括对应于左眼眼底图像的第一多维特征向量和对应于右眼眼底图像的第二多维特征向量。
可选地，所述眼底图像包括左眼眼底图像和右眼眼底图像，所述数据库用于存储用户数据，其中每一组用户数据分别包括一个预存多维特征向量；
所述获得用于表示用户身份的多维特征向量包括：
获取所述神经网络输出的对应于左眼眼底图像的第一多维特征向量和对应于右眼眼底图像的第二多维特征向量；
将所述第一多维特征向量和所述第二多维特征向量进行合并得到表示用户身份的多维特征向量。
可选地，所述眼底图像为左眼眼底图像或者右眼眼底图像，所述数据库用于存储用户数据，其中每一组用户数据分别包括对应于左眼的第一预存多维特征向量或者对应于右眼的第二预存多维特征向量。
本发明还提供一种基于眼底图像的身份验证方法，包括：
获取用户的眼底图像；
利用上述基于眼底图像的身份信息比对方法判断数据库中是否已存有与所述眼底图像的多维特征向量相匹配的预存多维特征向量，并由此完成对所述用户身份的确认。
本发明还提供一种基于眼底图像的身份信息存储方法，包括：
获取用户的眼底图像；
利用上述基于眼底图像的身份信息比对方法判断数据库中是否已存有与所述眼底图像的多维特征向量相匹配的预存多维特征向量；
当所述数据库中不存在与当前获得的多维特征向量相匹配的预存多维特征向量时，将当前获得的多维特征向量存储到所述数据库中作为所述用户的身份信息。
本发明还提供一种眼底图像识别模型训练方法，包括：
获取训练数据，所述训练数据包括第一眼底图像样本、第二眼底图像样本以及第三眼底图像样本，其中，第二眼底图像样本与所述第一眼底图像样本为同一人的眼底图像；所述第三眼底图像样本与所述第一眼底图像样本为不同人的眼底图像；
利用眼底图像识别模型对所述第一眼底图像样本、第二眼底图像样本和所述第三眼底图像样本进行识别得到损失值；
根据所述损失值调整所述眼底图像识别模型的参数。
可选地，利用眼底图像识别模型对所述第一眼底图像样本、第二眼底图像样本和所述第三眼底图像样本进行识别得到损失值包括：
计算所述第二眼底图像样本与所述第一眼底图像样本的第一距离；
计算所述第三眼底图像样本与所述第一眼底图像样本的第二距离；
根据所述第一距离和所述第二距离得到所述损失值。
可选地，所述利用所述损失值调整所述眼底图像识别模型的参数包括：
将所述损失值反馈至所述眼底图像识别模型；
根据所述损失值调整所述参数以减小所述第一距离而增大所述第二距离，直至所述第一距离比所述第二距离小于预设值。
相应地，本发明还提供一种基于眼底图像的身份信息比对设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行上述基于眼底图像的身份信息比对方法。
相应地，本发明还提供一种基于眼底图像的身份验证设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个 处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行上述基于眼底图像的身份验证方法。
相应地，本发明还提供一种基于眼底图像的身份信息存储设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行上述基于眼底图像的身份信息存储方法。
相应地，本发明还提供一种眼底图像识别模型训练设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行上述眼底图像识别模型训练方法。
根据本发明提供的基于眼底图像的身份信息比对方法和设备，首先利用神经网络将用户的眼底图像转换为用于表达用户身份的多维特征向量，利用神经网络的特点，提取到与用户个人特征相关的抽象特征信息，在比对时通过比对多维特征向量即可判断数据库中是否已有与当前用户相匹配的数据，在本方案中数据库不需要存储眼底图像，也不必在每获得一个新的眼底图像时都重新识别预存的眼底图像，由此可以提高身份信息比对操作的效率。
本发明提供的神经网络通过三元组训练数据和相应的损失函数进行训练，可以减小神经网络在针对同一只眼睛的不同眼底图像所提取的特征向量间的距离，并增大针对不同眼睛的眼底图像所提取的特征向量间的距离，经过训练后，该神经网络针对同一张眼底图像多次提取的特征向量的距离足够小，并且与其它眼底图像的特征向量的距离足够大，该信息具有一定的唯一性，由此可以将该神经网络针对眼底图像提取的特征向量作为用户身份信息。
附图说明
为了更清楚地说明本发明具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明实施例中的身份信息比对方法的流程图；
图2为本发明实施例中的身份信息存储方法的流程图；
图3为本发明实施例中的一种使用神经网络提取身份信息的示意图；
图4为本发明实施例中的另一种使用神经网络提取身份信息的示意图；
图5为本发明实施例中的神经网络训练方法的流程图；
图6为眼底图像中的一个图像块；
图7为针对图6所示图像块的分割结果；
图8为眼底血管图像。
具体实施方式
下面将结合附图对本发明的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
此外，下面所描述的本发明不同实施方式中所涉及的技术特征只要彼此之间未构成冲突就可以相互结合。
本发明实施例提供一种基于眼底图像的身份信息存储方法，该方法可以由计算机或服务器等电子设备执行。如图1所示该方法包括如下步骤：
S1，利用神经网络识别眼底图像，获得用于表示用户身份的多维特征向量。神经网络在识别图像时会提取特征信息，对于不同的任务，神经网络所关注的内容不同，提取的特征信息也不相同。例如在执行分类任务时，神经网络将针对眼底图像所属的类别提取相应的特征信息(一般是多维特征向量)，然后再根据特征信息进行分类。
在本实施例中，所使用的神经网络被配置为执行提取用于表示用户身份的多维特征向量，而不是执行某种分类或者图像分割任务。针对不同的人(用户)，该神经网络对其眼底图像所提取的多维特征向量应当是不同的，而对于同一个人的同一只眼，在进行多次识别时其每次提取的多维特征向量应当是相同的(或者大致相同、相似)。本申请所述神经网络具体可以是深度卷积网络(Convolutional neural network，CNN)，通过设置适当的损失函数，使用反向传播算法(back propagation，BP)，来规范提取的多维特征向量。一张眼底图像经过训练好的CNN模型可以得到特征向量，这些特征向量一般都是高维向量。
为了使该神经网络能够提取到预期的内容，应当在此前进行训练，训练方法有多种，对于不同的训练方法所使用的训练数据不相同，具体将在下文中进行介绍。
S2，将获得的多维特征向量与数据库中的各个预存多维特征向量进行比对。按照数 据库建立位置，可以分为基于GPU建立数据库和基于CPU建立数据库。数据库中预存的多维特征向量也可以是利用步骤S1中的神经网络对其它眼底图像所提取的多维特征向量。
比对两个特征向量的方式有多种，比对结果用于表示二者的相似度。关于多维向量的相似度，可以基于欧式距离、余弦相似度、标准化欧式距离等方式进行判断。欧式距离(Eucledian Distance)衡量的是多维空间中各个点之间的绝对距离，当数据很稠密并且连续时，这是一种优选的判断方式。由于计算是基于各维度特征的绝对数值，所以欧氏度量需要保证各维度指标在相同的刻度级别。
在一些特定场景也可以使用马氏距离(Mahalanobis Distance)，马氏距离是基于样本分布的一种距离。例如两个正态分布的总体，它们的均值分别为a和b，但方差不同，其中样例点A有在分布空间中属于哪个分布的概率更大，则A属于该分布。
由于建立特征空间使用的约束距离的不同，所以在比对结果时应该采取不同比对方式。
实际应用时，神经网络针对同一个人在不同时间和不同环境下拍摄的眼底图像所提取的多维特征向量通常不是完全相同的，因此采取计算距离的方式来衡量两个特征向量的相似度具有一定的容错性，所述距离优选为欧氏距离。
在另一个可选的实施例中，可分别计算各个预存多维特征向量与当前获得的多维特征向量的夹角，以此来衡量二者的相似度也是可行的。
S3，根据比对结果判断数据库中是否已存有与当前获得的多维特征向量相匹配的预存多维特征向量。对于不同的比对方式，其比对结果的内容不同，例如可以根据距离判断各个预存多维特征向量是否与当前获得的多维特征向量向匹配、根据夹角判断各个预存多维特征向量是否与当前获得的多维特征向量向匹配。
以距离为例，当二者的距离小于设定阈值时，即可判定二者具有足够高的相似度，表示二者是相匹配的。
根据本发明实施例提供的基于眼底图像的身份信息比对方法，首先利用神经网络将用户的眼底图像转换为用于表达用户身份的多维特征向量，利用神经网络的特点，提取到与用户个人特征相关的抽象特征信息，在比对时通过比对多维特征向量即可判断数据库中是否已有与当前用户相匹配的数据，在本方案中数据库不需要存储眼底图像，也不必在每获得一个新的眼底图像时都重新识别预存的眼底图像，由此可以提高身份信息比 对操作的效率。
本方案可以用于对用户的一只眼睛的眼底图像进行处理，也即步骤S1中识别的眼底图像是左眼眼底图像或者右眼眼底图像。相应地，在数据库中存储的各个特征向量是出自某一只眼睛的眼底图像。
本方案可以扩展为对用户的双眼眼底图像进行处理，也即在步骤S1中眼底图像包括左眼眼底图像和右眼眼底图像，使用神经网络分别对这两个眼底图像进行识别，得到对应于左眼眼底图像的第一多维特征向量和对应于右眼眼底图像的第二多维特征向量。相应地，数据库中存储的每一组数据分别包括两个预存多维特征向量，在步骤S2中可以进行相应地比对。
对双眼分别比对时，可能出现一只眼睛的特征向量与数据库中的数据相匹配，另一只眼睛的特征向量与数据库中的数据不匹配的情况，有些应用场景下可以接受这种结果，例如在建立或者更新数据库时则可以接受这种结果；但有些应用场景则不能允许这种情况，比如在进行身份验证时则不应出现这种情况。
在一个优选的实施例中，本方案针对用户的双眼眼底图像进行统一的处理，识别的眼底图像包括左眼眼底图像和右眼眼底图像。在步骤S1中神经网络分别针对双眼眼底图像进行识别，并输出对应于左眼眼底图像的第一多维特征向量和对应于右眼眼底图像的第二多维特征向量。然后将这两个特征向量合并，合并方式有多重，例如可以将两个1024维的特征向量连接成一个2048维的特征向量。
相应地，数据库的预存数据是一个合并的多维特征向量，是预先将两个特征向量合并的结果。在步骤S2中进行比对时，将当前合并的维特征向量与数据库中预存的多维特征向量进行比对，进而根据比对结果判断是否存在匹配的数据，在本实施例中，只进行一次比对即可确定双眼是否匹配。
下面介绍一种基于眼底图像的身份信息存储方法，该方法利用上述比对方案来建立和管理身份信息数据库，如图2所示该方法包括如下步骤：
S1A，获取用户的眼底图像，根据上述介绍，眼底图像可以是双眼中的任一眼底图像或者全部。
S2A，利用神经网络识别眼底图像，获得用于表示用户身份的多维特征向量。具体参照上述步骤S1，此处不再赘述。
S3A，将获得的多维特征向量与数据库中的各个预存多维特征向量进行比对。具体参照上述步骤S2，此处不再赘述。
S4A，根据比对结果判断数据库中是否已存有与当前获得的多维特征向量相匹配的预存多维特征向量。当数据库中不存在与当前获得的多维特征向量相匹配的预存多维特征向量时执行步骤S5A，否则执行步骤S6A。
需要说明的是，初始的数据库可以为空，无预存多维特征向量，则不存在匹配的内容；初始的数据库也可以预先被导入了多条数据，这些数据以个人作为基本模块，例如每一条用户数据包括用户姓名及其多维特征向量(基于该用户的眼底图像获得)，此外还可以包括用户性别、年龄、近视度数等个人信息。
如果经过逐一比对确定此数据库中不存在与当前用户的多维特征向量相匹配的预存多维特征向量，则表示此数据库中没有存储当前用户的身份信息；否则表示此数据库中已经存储了当前用户的身份信息。
S5A，将当前获得的多维特征向量存储到数据库中。具体地，当一个多维特征向量在原有的数据库中找不到时，系统可以进行提示，是否需要把当前获得的多维特征向量作为一个新的数据加入到原有数据库中，如果选是，则发出申请内存，当申请成功之后，将其加入原有的数据库中。进一步地，系统还可以提示，是否继续输入相关的补充信息，比如用户性别、年龄、近视度数等个人信息，当输入了这些信息后将其与相应的多维特征向量关联存储。
S6A，利用当前获得的多维特征向量替换数据库中与其相匹配的预存多维特征向量。具体地，当在数据库中找到与当前的多维特征向量相匹配时，系统可以进行提示，是否需要使用当前获得的多维特征向量覆盖已有数据，如果选是则进行覆盖。此步骤并非必须的操作，作为替换方案，当在数据库中找到与当前的多维特征向量相匹配时系统可以只进行提示，表示当前的用户数据已存在，而不必进行替换操作。
根据本发明实施例提供的基于眼底图像的身份信息存储方法，首先利用神经网络将用户的眼底图像转换为用于表达用户身份的多维特征向量，利用神经网络的特点，提取到与用户个人特征相关的抽象特征信息，在存储时通过比对多维特征向量即可判断数据库中是否已有与当前用户相匹配的数据，进而对多维特征向量向量进行存储，在本方案中数据库不需要存储眼底图像，也不必在存储每一个用户身份信息时都使用神经网络识别已有的眼底图像，由此可以提高存储用户身份信息的效率。
作为示例性的说明，根据本方案建立的数据库中的每一条用户信息可包括用户姓名、多维特征向量(左眼的第一多维特征向量和/或右眼的第二多维特征向量或者二者的合并结果)、用户性别、年龄、近视度数等等。其中的多维特征向量和表示用户的身 份信息。
上述实施例介绍的是建立和管理数据库的过程，在此基础上还可以做出一些变化从而得到身份验证方案，具体地，根据步骤S4A的结论，即数据库中是否存有当前用户的身份信息，从而可以执行相应的身份判定操作。例如将步骤S5A替换为判定当前用户不是已知用户，从而不允许其执行后续的操作、将步骤S6A替换为判定当前用户是已知用户，从而允许其执行后续的操作，所述后续的操作可以是对电子设备解锁或者进行登录等等，从而实现身份验证或者鉴权等等。
本发明还提供一种基于眼底图像的身份信息比对设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行上述基于眼底图像的身份信息比对方法。
本发明还提供一种基于眼底图像的身份信息存储设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行上述基于眼底图像的身份信息存储方法。
本发明还提供一种基于眼底图像的身份验证设备，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行上述基于眼底图像的身份验证方法。
下面结合附图3-图5介绍如何获得一个产生特征向量的模型(上述实施例中的神经网络)。明确提取的特征需要具备的特点是：优化类内距离，使得类内距离更加紧凑；优化类间距离，使得类间距离可以区分的更开。其中，类内距离(intra-class)是指同一类各模式样本点间的距离；类间距离(inter-class)是指不同类间的距离。为了达到此目的并且提取更佳的特征向量，有以下可选的实施方式。
作为第一种可选的实施方式，图3是一种基于softmax的分类模型的训练框架，此分类模型的框架可以应用于眼底图像身份特征训练以及提取，并且在此基础上提供多种辅助损失函数用于改善眼底图像身份特征的特性。一张眼底图像Imgi进入深度卷积网络，得到多维特征向量xi，再通过一个全连接层后经过softmax，得到评分向量(score vector)。关于损失函数softmax loss：
Figure PCTCN2020083625-appb-000001
其中N是batch size，n是类别数量，xi表示一个batch的第i个眼底图像的特征向量，yi表示这张眼底图像的真实身份分类(标签)，Wj表示最后全连接层的权重W的第j列的向量，
Figure PCTCN2020083625-appb-000002
表示将xi预测为其真实身份分类yi的概率值，bj为偏置量。
例如定义一个n＝3的分类模型(1，2，3)，其中类别1，2，3代表三个用户的身份标签，眼底图像Imgi得到的评分向量si为{0.1,0.2,0.7},则判断Imgi为标签为3的用户的眼底图像，此Imgi对损失值的贡献为L1＝-log(0.7)＝0.3567，但如果得到的评分向量si为{0.1,0.5,0.4}，此Imgi对损失值的贡献为L1＝-log(0.4)＝0.9163。由此可知，模型将眼底图像预测为正确的身份类别概率值大，则对损失值的贡献小，反之则对损失值贡献较大。使用此损失函数训练的模型，提取可以进行正确分类的特征向量。
在保证不同类别的特征可以分离的同时，最大程度地减少内部类别的差异，对于身份识别任务是极其关键的一点。为了达到这个目的，可以设置如下损失函数：
Figure PCTCN2020083625-appb-000003
采用softmax loss(Ls)和center loss(Lc)的联合监督来训练深度卷积网络进行判别性特征学习。Center loss可以有效的表征类内的变化，标量λ用来平衡Ls和Lc；当λ取不同的值时，特征向量的分布不同。
训练时，每次迭代输入一批量(mini-batch)的眼底图像，m为每次迭代输入图片数量。xi是一个batch中第i个图像的特征向量，yi是眼底图像xi所属用户的身份标签，Cyi∈Rd表示第yi个用户身份分类标签的特征向量的中心，d表示特征向量的维度。训练过程中，随着每次迭代，每个用户标签的特征向量中心Cyi会进行更新，通过对相应分类标签的特征求平均值来计算中心Cyi。其次，为了避免少量错误贴标的样本引起的大扰动，可使用标量α∈[0,1]来控制中心的学习率。本方案特点在于不需要对训练集进行重新的复杂组合的情况下，就能在保证不同类别的特征可以分离的同时，最大程度地减少内部类别的差异。
作为第二种可选的实施方式，在采用softmax loss(Ls)和contrastive loss(Lc)的联合监督来训练深度卷积网络进行判别性特征学习的基础上，不仅考虑分类的准确率，还加入类间距离m(margin)，扩大决策边界，使得正样本间具有更高的相似度，而负样本间有更低的相似度。
具体如图4所示，在深度卷积网络结构后引入n通道的softmax层得到识别模块用于对输入眼底图像进行身份类别分类，其中n为用户身份类别的数目，并定义为识别损失函数(Identification Loss)：
Figure PCTCN2020083625-appb-000004
上式实际为交叉熵损失函数(cross-entropy Loss)，其中f是特征向量，t是特征向量f对应的用户身份类别，θ id是softmax层的参数，pi是目标概率分布，其中对于用户身份类别t，pt＝1，其余pi＝0，
Figure PCTCN2020083625-appb-000005
是预测的概率分布。在训练时，由于此网络联合了下述的验证损失函数(verification Loss)，实际为对比损失(contrastive Loss)，在使用以下损失函数进行训练前，应首先将训练集中的眼底图像进行两两组对，成为图像对(image pairs)，并对各组图像对是否为同一用户的身份特征进行标注，对输入图像对的特征向量(fi，li),(fj，lj)，如果li＝lj，则标注标签yij＝0；否则，标注标签yij＝1；
Figure PCTCN2020083625-appb-000006
在训练过程中，将眼底图像对输入深度卷积神经网络后得到图像特征，对眼底图像对分别计算识别损失，即对两张输入眼底图像进行用户身份分类，同时对眼底图像对进行验证，验证它们是否为同一张眼底图像，当输入眼底图像对为同一身份，即yij＝0，它们的特征向量为(fi，fj)，则Verif(fi，fj，yij，θve)＝1/2||fi-fj||22；当输入眼底图像对不是同一身份，即yij＝1，它们的特征向量为(fi，fj)，则Verif(fi，fj，yij，θve)＝1/2(max(0，m-||fi-fj||2))2，其中特征向量(fi，fj)的相似距离使用的是L2归一化。L1/L2归一化或余弦相似度，m(margin)定义了类间间隔，只有当非同身份的图像对的距离在m距离内，才对损失值产生作用，这一项有效的约束了类间距离，使得类间距离更加可分。
作为第三种可选的实施方式，如图5所示本发明实施例还提一种基于三元组数据的模型训练方法，该方法包括如下步骤：
S1B，获取训练数据，训练数据包括第一眼底图像样本、第二眼底图像样本以及第三眼底图像样本。本实施例中所使用的训练数据是三元组样本数据，其中第二眼底图像样本与第一眼底图像样本为同一个人的同一只眼的眼底图像；第三眼底图像样本与第一眼底图像样本为不同的人的眼底图像。
在具体的实施例中，首先准备数据集。作为举例，数据集可以由n只眼睛的眼底图像组成，其中每只眼睛对应m张眼底图片，即数据集中共有n*m张眼底图像。获得训练数据时，首先在n*m张眼底图像中随机选取一张眼底图像，称之为Anchor；然后选一张与Anchor同眼的另一张眼底图像，称为Positive(记为x_p)；然后选一张与Anchor不同眼的眼底图像，称为和Negative(记为x_n)，由此获得一组训练数据(Anchor，Positive，Negative)。
在利用训练数据对神经网络训练之前可以先对眼底图像进行预处理，以使得训练的神经网络在进行眼底图像识别时更为精确。具体的，可以先对每个眼底图像进行剪裁处理，由于拍摄的眼底图像原图具有较多的黑色背景，可以先对眼底图像进行裁边处理。移除背景中大片的黑色像素，眼底图像均被裁剪到最小的能够包含整个圆形眼底的矩形。
在一个具体的实施例中，所有眼底图像可以均裁剪为统一格式，例如，尺寸被统一到224*224像素，模型训练和识别时输入的图片格式可以采用统一的224*224像素和RGB三个颜色通道的眼底图像。
S2B.利用神经网络对第一眼底图像样本、第二眼底图像样本和第三眼底图像样本进行识别得到损失值。利用预设的损失函进行损失值计算第二眼底图像样本与第一眼底图像样本的第一距离，以及计算第三眼底图像样本与第一眼底图像样本的第二距离，并根据第一距离和第二距离得到损失值。
具体的，神经网络分别对上述三张眼底图像进行特征提取，得到三个多维特征向量分别记为：
Figure PCTCN2020083625-appb-000007
然后可计算
Figure PCTCN2020083625-appb-000008
与
Figure PCTCN2020083625-appb-000009
之间的第一距离、计算
Figure PCTCN2020083625-appb-000010
与
Figure PCTCN2020083625-appb-000011
之间的第二距离。在本实施例中第一距离和第二距离为欧式距离。
利用第一距离和第二距离计算损失值，具体可以采用如下损失函数关系式进行计算：
Figure PCTCN2020083625-appb-000012
其中，α表示预设值，该预设值为第一距离与第二距离之间的最小间隔。+表示[]内的值大于0时取该值为损失值，[]内的值小于0时，损失为0。
S3B.根据损失值调整神经网络的参数。即利用损失值为基准进行反向传播更新神经网络的参数。
在实际训练时需要使用大量的上述三元组样本数据，使得神经网络不断调整参数，直至损失函数收敛。在神经网络传递损失的过程中，要使得Anchor和positive的距离变小，而Anchor和Negative的距离变大，最终让第一距离和第二距离之间有一个最小的间隔α。
为了提高神经网络的鲁棒性，在训练前可对训练数据进行数据增强。数据增强过程可以使用旋转、平移、放大和主成分变换(PCA)颜色增强，通过数据增强每个眼底图像可以生成多张使用随机增强参数的眼底图像。例如，通过数据增强后的眼底图像的格式可以采用统一的224*224像素和RGB三个颜色通道的眼底图像。实际操作中可以先对眼底图像进行裁剪，再对裁剪后的眼底图像进行数据增强，也可以先对眼底图像进行数据增强，再对进过数据增强后的眼底图像进行裁剪，对于两种数据预处理的顺序不做限定。
根据本发明实施例提供的神经网络训练方法，通过三元组训练数据和相应的损失函数进行训练，可以逐渐减小神经网络在针对同一只眼睛的不同眼底图像所提取的特征向量间的距离，并增大针对不同眼睛的眼底图像所提取的特征向量间的距离，经过训练后，该神经网络针对同一张眼底图像多次提取的特征向量的距离足够小，并且与其它眼底图像的特征向量的距离足够大，也即该信息具有一定的唯一性，由此可以将该神经网络针对眼底图像提取的特征向量作为用户身份信息。
在一个可选的实施例中，为了进一步排除与眼底识别不相关的干扰图像信息，提升神经网络的识别能力，在进行训练之前，还可以对眼底图像进行分割，得到眼底特征图像作为训练数据。
在获取到眼底图像后，可以利用计算机视觉算法或机器学习算法对眼底特征进行提取，例如通过利用分割神经网络对眼底图像中的眼底特征进行提取，得到包含眼底特征置信度的概率图或二值化图像。如图6所示，可以将眼底图像划分为多个图像块，图像 块的尺寸根据眼底图像的尺寸进行设定，对于多数情况，划分的图像块的尺寸应当明显小于整个眼底图像的尺寸。例如眼底图像的尺寸为1000*1000(像素)，所划分出的图像块的尺寸是100*100(像素)。
利用预设的分割模型分别针对各个图像块中的血管影像进行分割得到分割图像块；分割模型具体可以是FCN、SegNet、DeepLab等神经网络，在使用分割模型之前应当使用样本数据对其进行训练使其具备一定的语义分割能力，具体可使用人工标记了血管区域的样本图像块训练得到。
分割模型会提取图像块中血管影像的特征，并根据提取的特征形成分割图像块，在其中凸显出血管影像，具体的凸显方式有多种，例如采用明显不同于背景的各种像素值表达血管所在的位置等等。
将图6所示的图像块输入分割模型，可以得到如图7所示的分割图像块，在这一实施例中所使用的分割模型输出的是二值图像，它采用两种像素值分别表达背景和血管影像，直观地凸显血管位置。利用分割图像块拼接出眼底血管图像，得到图8所示的图像，然后将图8所示的图像作为训练数据。
作为可选的实施例，还可以采用类似方法提取其他特征例如：视盘、黄斑、和视网膜等特征。通过对眼底特征的提取，可以极大的排除与眼底身份识别不相关的干扰图像信息，显著提升模型识别性能。
眼底特征图像中也可以存在高级的非直接特征(或称为抽象特征)，例如血管分叉点位置和方向、血管交叉点位置和方向、血管向量图等。在获取原始的眼底图像后，也可以从其中提取上述非直接特征作为训练数据。
本领域内的技术人员应明白，本发明的实施例可提供为方法、系统、或计算机程序产品。因此，本发明可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本发明可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本发明是参照根据本发明实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的 指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
显然，上述实施例仅仅是为清楚地说明所作的举例，而并非对实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其它不同形式的变化或变动。这里无需也无法对所有的实施方式予以穷举。而由此所引伸出的显而易见的变化或变动仍处于本发明创造的保护范围之中。

Claims (17)

  1. 一种基于眼底图像的身份信息比对方法，其特征在于，包括：
    利用神经网络识别眼底图像，获得用于表示用户身份的多维特征向量；
    将获得的多维特征向量与数据库中的各个预存多维特征向量进行比对；
    根据比对结果判断所述数据库中是否已存有与当前获得的多维特征向量相匹配的预存多维特征向量。
  2. 根据权利要求1所述的方法，其特征在于，所述神经网络是利用三元组样本数据进行训练得到的，所述三元组样本数据包括第一眼底图像样本、第二眼底图像样本以及第三眼底图像样本，其中，所述第二眼底图像样本与所述第一眼底图像样本为同一人的眼底图像，所述第三眼底图像样本与所述第一眼底图像样本为不同人的眼底图像。
  3. 根据权利要求2所述的方法，其特征在于，在所述神经网络的训练过程中，所述神经网络分别提取所述第一眼底图像样本、所述第二眼底图像样本以及所述第三眼底图像样本的多维特征向量，根据提取到的三个多维特征向量计算所述第二眼底图像样本与所述第一眼底图像样本的第一距离，以及计算所述第三眼底图像样本与所述第一眼底图像样本的第二距离，并根据所述第一距离和所述第二距离得到损失值，从而根据所述损失值调整所述神经网络的参数。
  4. 根据权利要求3所述的方法，其特征在于，所述根据所述损失值调整所述神经网络的参数包括：
    将所述损失值反馈至所述神经网络，使其根据所述损失值调整所述参数以减小所述第一距离增大所述第二距离直至所述第一距离比所述第二距离小于预设值。
  5. 根据权利要求1-4中任一项所述的方法，其特征在于，根据别对结果判断所述数据库中是否已存有与当前获得的多维特征向量相匹配的预存多维特征向量，包括：
    分别计算各个预存多维特征向量与当前获得的多维特征向量的距离；
    根据所述距离判断各个预存多维特征向量是否与当前获得的多维特征向量向匹配。
  6. 根据权利要求1-4中任一项所述的方法，其特征在于，所述眼底图像包括左眼眼底图像和右眼眼底图像，所述数据库用于存储用户数据，其中每一组用户数据分别包括对应于左眼的第一预存多维特征向量和对应于右眼的第二预存多维特征向量；所述多维特征向量包括对应于左眼眼底图像的第一多维特征向量和对应于右眼眼底图像的第二多维特征向量。
  7. 根据权利要求1-4中任一项所述的方法，其特征在于，所述眼底图像包括左眼眼底图像和右眼眼底图像，所述数据库用于存储用户数据，其中每一组用户数据分别包括一个预存多维特征向量；
    所述获得用于表示用户身份的多维特征向量包括：
    获取所述神经网络输出的对应于左眼眼底图像的第一多维特征向量和对应于右眼眼底图像的第二多维特征向量；
    将所述第一多维特征向量和所述第二多维特征向量进行合并得到表示用户身份的多维特征向量。
  8. 根据权利要求1-4中任一项所述的方法，其特征在于，所述眼底图像为左眼眼底图像或者右眼眼底图像，所述数据库用于存储用户数据，其中每一组用户数据分别包括对应于左眼的第一预存多维特征向量或者对应于右眼的第二预存多维特征向量。
  9. 一种基于眼底图像的身份验证方法，其特征在于，包括：
    获取用户的眼底图像；
    利用权利要求1-8中任一项所述的方法判断数据库中是否已存有与所述眼底图像的多维特征向量相匹配的预存多维特征向量，并由此完成对所述用户身份的确认。
  10. 一种基于眼底图像的身份信息存储方法，其特征在于，包括：
    获取用户的眼底图像；
    利用权利要求1-8中任一项所述的方法判断数据库中是否已存有与所述眼底图像的多维特征向量相匹配的预存多维特征向量；
    当所述数据库中不存在与当前获得的多维特征向量相匹配的预存多维特征向量时，将当前获得的多维特征向量存储到所述数据库中作为所述用户的身份信息。
  11. 一种眼底图像识别模型训练方法，其特征在于，包括：
    获取训练数据，所述训练数据包括第一眼底图像样本、第二眼底图像样本以及第三眼底图像样本，其中，第二眼底图像样本与所述第一眼底图像样本为同一人的眼底图像；所述第三眼底图像样本与所述第一眼底图像样本为不同人的眼底图像；
    利用眼底图像识别模型对所述第一眼底图像样本、第二眼底图像样本和所述第三眼底图像样本进行识别得到损失值；
    根据所述损失值调整所述眼底图像识别模型的参数。
  12. 根据权利要求11所述的方法，其特征在于，利用眼底图像识别模型对所述第一眼底图像样本、第二眼底图像样本和所述第三眼底图像样本进行识别得到损失值包括：
    计算所述第二眼底图像样本与所述第一眼底图像样本的第一距离；
    计算所述第三眼底图像样本与所述第一眼底图像样本的第二距离；
    根据所述第一距离和所述第二距离得到所述损失值。
  13. 如权利要求12所述的方法，其特征在于，所述利用所述损失值调整所述眼底图像识别模型的参数包括：
    将所述损失值反馈至所述眼底图像识别模型；
    根据所述损失值调整所述参数以减小所述第一距离而增大所述第二距离，直至所述第一距离比所述第二距离小于预设值。
  14. 一种基于眼底图像的身份信息比对设备，其特征在于，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行如权利要求1-8中任意一项所述的基于眼底图像的身份信息比对方法。
  15. 一种基于眼底图像的身份验证设备，其特征在于，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行如权利要求9所述的基于眼底图像的身份验证方法。
  16. 一种基于眼底图像的身份信息存储设备，其特征在于，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行如权利要求10所述的基于眼底图像的身份信息存储方法。
  17. 一种眼底图像识别模型训练设备，其特征在于，包括：至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器执行如权利要求11-13中任一项所述的眼底图像识别模型训练方法。
PCT/CN2020/083625 2019-06-28 2020-04-08 基于眼底图像的身份信息处理方法及设备 WO2020258981A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US17/619,875 US11893831B2 (en) 2019-06-28 2020-04-08 Identity information processing method and device based on fundus image
EP20833250.2A EP3992844A4 (en) 2019-06-28 2020-04-08 IDENTITY INFORMATION PROCESSING METHOD AND DEVICE BASED ON FUNDUS IMAGES

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910578335.4 2019-06-28
CN201910578335.4A CN110263755B (zh) 2019-06-28 2019-06-28 眼底图像识别模型训练方法、眼底图像识别方法和设备

Publications (1)

Publication Number Publication Date
WO2020258981A1 true WO2020258981A1 (zh) 2020-12-30

Family

ID=67923118

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/083625 WO2020258981A1 (zh) 2019-06-28 2020-04-08 基于眼底图像的身份信息处理方法及设备

Country Status (4)

Country Link
US (1) US11893831B2 (zh)
EP (1) EP3992844A4 (zh)
CN (1) CN110263755B (zh)
WO (1) WO2020258981A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111178221A (zh) * 2019-12-24 2020-05-19 珠海格力电器股份有限公司 身份识别方法及装置
CN113011253A (zh) * 2021-02-05 2021-06-22 中国地质大学（武汉） 基于ResNeXt网络的人脸表情识别方法、装置、设备及存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110263755B (zh) * 2019-06-28 2021-04-27 上海鹰瞳医疗科技有限公司 眼底图像识别模型训练方法、眼底图像识别方法和设备
CN110717907A (zh) * 2019-10-06 2020-01-21 浙江大学 一种基于深度学习的手部肿瘤智能检测方法
CN111178403B (zh) * 2019-12-16 2023-10-17 北京迈格威科技有限公司 训练属性识别模型的方法、装置、电子设备及存储介质
CN111553436B (zh) * 2020-04-30 2024-03-22 上海鹰瞳医疗科技有限公司 训练数据生成方法、模型训练方法及设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102496007A (zh) * 2011-12-02 2012-06-13 陈中山 人体身份识别仪
CN106022317A (zh) * 2016-06-27 2016-10-12 北京小米移动软件有限公司 人脸识别方法及装置
CN108229539A (zh) * 2016-12-13 2018-06-29 安讯士有限公司 用于训练神经网络的方法、计算机程序产品和装置
CN109390053A (zh) * 2017-08-02 2019-02-26 上海市第六人民医院 眼底图像处理方法、装置、计算机设备和存储介质
US20190130566A1 (en) * 2015-04-06 2019-05-02 IDx, LLC Systems and methods for feature detection in retinal images
CN110263755A (zh) * 2019-06-28 2019-09-20 上海鹰瞳医疗科技有限公司 眼底图像识别模型训练方法、眼底图像识别方法和设备
CN110427804A (zh) * 2019-06-18 2019-11-08 中山大学 一种基于二次迁移学习的虹膜身份验证方法

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8457366B2 (en) * 2008-12-12 2013-06-04 At&T Intellectual Property I, L.P. System and method for matching faces
CN102254189A (zh) * 2011-08-16 2011-11-23 武汉大学 一种基于云模型的人脸表情识别方法
WO2014074178A1 (en) * 2012-11-08 2014-05-15 The Johns Hopkins University System and method for detecting and classifying severity of retinal disease
CN103605962A (zh) * 2013-11-19 2014-02-26 四川长虹电器股份有限公司 基于类内类间距离的人脸验证方法
WO2018013200A1 (en) * 2016-07-14 2018-01-18 Magic Leap, Inc. Deep neural network for iris identification
CN106228142B (zh) * 2016-07-29 2019-02-15 西安电子科技大学 基于卷积神经网络和贝叶斯决策的人脸验证方法
WO2018095994A1 (en) * 2016-11-22 2018-05-31 Delphinium Clinic Ltd. Method and system for classifying optic nerve head
CN107247940A (zh) * 2017-06-18 2017-10-13 天津大学 一种面部识别和聚类的联合嵌入方法
CN109101913A (zh) * 2018-08-01 2018-12-28 北京飞搜科技有限公司 行人重识别方法和装置
CN109614613B (zh) * 2018-11-30 2020-07-31 北京市商汤科技开发有限公司 图像的描述语句定位方法及装置、电子设备和存储介质
CN109902665A (zh) * 2019-03-28 2019-06-18 北京达佳互联信息技术有限公司 相似人脸检索方法、装置及存储介质
US11298017B2 (en) * 2019-06-27 2022-04-12 Bao Tran Medical analysis system
CN110414423A (zh) * 2019-07-25 2019-11-05 上海鹰瞳医疗科技有限公司 身份识别方法及设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102496007A (zh) * 2011-12-02 2012-06-13 陈中山 人体身份识别仪
US20190130566A1 (en) * 2015-04-06 2019-05-02 IDx, LLC Systems and methods for feature detection in retinal images
CN106022317A (zh) * 2016-06-27 2016-10-12 北京小米移动软件有限公司 人脸识别方法及装置
CN108229539A (zh) * 2016-12-13 2018-06-29 安讯士有限公司 用于训练神经网络的方法、计算机程序产品和装置
CN109390053A (zh) * 2017-08-02 2019-02-26 上海市第六人民医院 眼底图像处理方法、装置、计算机设备和存储介质
CN110427804A (zh) * 2019-06-18 2019-11-08 中山大学 一种基于二次迁移学习的虹膜身份验证方法
CN110263755A (zh) * 2019-06-28 2019-09-20 上海鹰瞳医疗科技有限公司 眼底图像识别模型训练方法、眼底图像识别方法和设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3992844A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111178221A (zh) * 2019-12-24 2020-05-19 珠海格力电器股份有限公司 身份识别方法及装置
CN113011253A (zh) * 2021-02-05 2021-06-22 中国地质大学（武汉） 基于ResNeXt网络的人脸表情识别方法、装置、设备及存储介质
CN113011253B (zh) * 2021-02-05 2023-04-21 中国地质大学（武汉） 基于ResNeXt网络的人脸表情识别方法、装置、设备及存储介质

Also Published As

Publication number Publication date
US20220358791A1 (en) 2022-11-10
EP3992844A4 (en) 2023-05-24
CN110263755A (zh) 2019-09-20
CN110263755B (zh) 2021-04-27
US11893831B2 (en) 2024-02-06
EP3992844A1 (en) 2022-05-04

Similar Documents

Publication Publication Date Title
WO2020258981A1 (zh) 基于眼底图像的身份信息处理方法及设备
US10776470B2 (en) Verifying identity based on facial dynamics
Tome et al. Facial soft biometric features for forensic face recognition
Martinez et al. Local evidence aggregation for regression-based facial point detection
US20190102603A1 (en) Method and apparatus for determining image quality
CN112232117A (zh) 一种人脸识别方法、装置及存储介质
WO2021174819A1 (zh) 人脸遮挡检测方法及系统
US9684850B2 (en) Biological information processor
CN111758116B (zh) 脸部图像识别系统、识别器生成装置、识别装置与系统
KR20170016231A (ko) 사용자 인증을 위한 멀티-모달 퓨전 방법 및 사용자 인증 방법
JP2022532177A (ja) 偽造された顔の認識方法、装置、および非一時的コンピュータ可読記憶媒体
US10229311B2 (en) Face template balancing
CN114270417A (zh) 能够更新注册人脸模板的人脸识别系统及方法
JP5755046B2 (ja) 画像認識装置、画像認識方法及びプログラム
JP2009093490A (ja) 年齢推定装置及びプログラム
Kant A multimodal approach to improve the performance of biometric system
TWI325568B (en) A method for face varification
CN113468925B (zh) 一种遮挡人脸识别方法、智能终端及存储介质
JP2006338686A (ja) 顔類似度算出方法および装置
CN115795355A (zh) 一种分类模型训练方法、装置及设备
Cai et al. Robust facial expression recognition using RGB-D images and multichannel features
CN111401112B (zh) 人脸识别方法和装置
KR101501410B1 (ko) 눈 위치 검출 방법
CN111428670A (zh) 人脸检测方法、装置、存储介质及设备
Zee et al. Comparative Graph Model for Facial Recognition Systems

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20833250

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2020833250

Country of ref document: EP