WO2020001084A1 - 一种在线学习的人脸识别方法 - Google Patents

一种在线学习的人脸识别方法 Download PDF

Info

Publication number
WO2020001084A1
WO2020001084A1 PCT/CN2019/078474 CN2019078474W WO2020001084A1 WO 2020001084 A1 WO2020001084 A1 WO 2020001084A1 CN 2019078474 W CN2019078474 W CN 2019078474W WO 2020001084 A1 WO2020001084 A1 WO 2020001084A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
sample
feature vector
tested
reference feature
Prior art date
Application number
PCT/CN2019/078474
Other languages
English (en)
French (fr)
Inventor
陆生礼
庞伟
周世豪
向家淇
李宇峰
范雪梅
Original Assignee
东南大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 东南大学 filed Critical 东南大学
Publication of WO2020001084A1 publication Critical patent/WO2020001084A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items

Definitions

  • the invention discloses a face recognition method for online learning, which belongs to the technical field of computational reckoning, and particularly relates to the field of computer vision technology for face recognition.
  • Face recognition technology has been widely used in access control, security inspection, monitoring, etc. Its main task is to distinguish different individuals in the database and reject individuals outside the database. In practical applications, the features of a person's appearance will be affected by dressing, expressions, and changes due to posture, lighting, and the front picture of the same person will also change over time. In order to increase the robustness of the algorithm, it is necessary to update the model in specific situations during the recognition process. The traditional method is to re-collect the samples and train again, which is time-consuming and difficult to operate. We hope that the face recognition device can adjust the model by itself and adapt to the changes of the data set at runtime. Therefore, an online learning method that is simple and effective is urgently needed.
  • Existing online learning methods compare the shallow features of a face (such as Haar features, LBP features) to identify and track a given face in a video.
  • the target face is distinguished from one or more surrounding faces, and only a few samples need to be discerned; at the same time, the facial features change little during the short period of time included in the video, so the image
  • the shallow features can represent the facial features to a certain extent.
  • tasks such as face access control and time attendance need to distinguish a database containing hundreds of people. Over a long period of time, everyone's appearance will change, and shallow features are difficult to handle such complex tasks.
  • Deep neural network improves the recognition of the model, but the training of the network consumes a lot of computing resources and time.
  • the model trained on the offline server needs to be re-imported into the face recognition device.
  • the neural network structure is fixed , When adding / deleting members, it is necessary to train again, which brings inconvenience to practical applications.
  • a simple and accurate online learning method is needed.
  • the object of the present invention is to address the shortcomings of the background art described above, and provide an online learning method for face recognition.
  • the model training and update are implemented on the terminal device, which solves the existing problems.
  • Face recognition technology requires technical retraining when the data set changes.
  • Establish external data sets Establish external data sets based on public face databases of research institutions or self-collected data.
  • the face databases can choose public databases such as CASIA-WebFace, VGG-FACE; or they can be captured on the network by themselves Take pictures of public figures.
  • Each picture should include an identification that indicates which individual the picture belongs to. You should collect as many individuals as possible, each individual containing as many samples as possible, while reducing the number of mislabeled samples in the data set. Increasing the number of samples and the number of categories will improve the training accuracy, and will not change the structure of the face feature extractor or increase the training difficulty;
  • a local member set U ⁇ u 1 , u 2 , ..., u m ⁇ composed of m individuals, and take n corresponding face samples ⁇ x i1 for each member u i in U , x i2 , ..., x in ⁇ , preferably, the face samples should be photos with normal lighting and natural expressions.
  • Training model Use a convolutional neural network as a feature extractor.
  • the input of the neural network is a color picture.
  • the output of the neural network is the category to which the picture belongs.
  • the length of the classification layer is equal to the number of categories in the external data set.
  • the loss function can use softmaxloss. The reason is that the neural network is trained with an external data set, because the number and type of samples in the external data set are much larger than the local data set, which is conducive to the neural network to learn better features.
  • the loss function is continuously reduced with the back propagation of the error. The accuracy rate keeps increasing. When the loss function converges and no longer decreases, save the convolutional neural network model and use the l-dimensional vector connected to the classification layer as the feature vector of the input picture.
  • 2 , d represents the similarity between the two features. The larger d, the larger the feature gap, and further, when d is large enough, the two features can be considered to belong to different individuals, and the reference vector closest to y in S is found. And distance i * argmin i ⁇ ⁇ 1,2, ..., m ⁇ d i , set the similarity threshold ⁇ , if Output Otherwise output u represents the identity of the person predicted by the model;
  • u T represents the true identity of the person being tested, provided by the candidate himself
  • u represents the predicted identity of the person being tested
  • y T represents the true feature vector corresponding to the person being tested in the reference feature space
  • y represents the feature extractor Feature vector extracted from picture x
  • ⁇ represents the learning rate of the model's error correction amplitude, ⁇ (0,1)
  • At least one dense connection block is added to the network.
  • Each dense connection block contains at least two convolutional layers connected in sequence.
  • the feature maps output by all convolutional layers are stitched as input feature maps to the next convolutional layer.
  • the feature maps output by each densely connected block are down-sampled and transmitted to the input end of the next densely connected block.
  • the color face image of the input convolutional neural network is processed by multiple equal-step convolutional layers and downsampling layers to obtain the feature map of the first densely connected block, and the feature map output from the last densely connected block is again Convolution operation and mean pooling operation are performed to obtain the feature vector input to the classification layer.
  • this application also provides a face recognition method without retraining the model after adding / removing members.
  • new members provide their own true identity tags u k after completing the face recognition process
  • the present application also provides a terminal device for implementing the above-mentioned face recognition method.
  • the device includes: a memory, a processor, and a computer program stored on the memory and run on the processor.
  • the processor executes the program, the following steps are implemented: : Use the external data set to train a face feature extractor, extract the reference features corresponding to each member in the local data set to form the reference feature space, compare the feature vector and reference feature of the sample to be tested to determine the most similar to the feature vector of the sample to be tested Reference feature.
  • the identity of the member to which the reference feature most similar to the feature vector of the sample to be tested belongs is the identity of the sample to be tested; otherwise, the identity of the sample to be tested is returned.
  • the reference feature space is updated according to the difference between the predicted feature vector of the sample to be tested and its corresponding real feature vector in the reference feature space.
  • the present invention proposes a method for dynamically updating a face recognition model at a terminal, adding or deleting members.
  • This method realizes the face by flexibly adjusting the reference feature space extracted from the local data set to adapt to the changes in the data set.
  • the offline updating of the recognition model is simple in operation and small in calculation amount, and can better adapt to changes in facial features over time, and is particularly suitable for situations that require frequent changes of members;
  • the present invention implements feature extraction through a densely connected convolutional neural network.
  • a densely connected layer is formed by connecting several synchronizing convolutional layers.
  • the input feature map of the next convolution layer is made after stitching, which strengthens feature reuse, improves network performance, reduces the number of parameters and calculations, is more robust, and has a wider range of applications.
  • FIG. 1 is a flowchart of face recognition by this method.
  • Figure 2 is an example of a face cut sample from a data set.
  • FIG. 3 is a flowchart of online learning of the present invention.
  • FIG. 4 is a schematic structural diagram of a dense connection block.
  • FIG. 1 shows a flowchart of a face recognition method according to the present invention.
  • the face recognition method includes the following five steps.
  • Step 1 Establish an external data set:
  • the CASIA-WebFace database is used as the external data set.
  • Figure 2 shows a sample sample of the processed CASIA-WebFace database.
  • the face frame should fit the person more closely.
  • all pictures are scaled to the input size of the convolutional neural network. If external data sets are obtained from other data sets, it is also necessary to follow the processing method in which the face frame closely fits the edge of the face and the picture meets the input picture size requirements of the neural network.
  • Step 2 Establish a local data set: take pictures of the faces of ten people, and take multiple face sample pictures of each person with different expressions and poses.
  • Step 3 Set up a convolutional neural network: use the external data set as a sample set to train a face feature extractor:
  • This application relates to a more efficient convolutional neural network.
  • the input of the neural network is 160 * A 160-pixel color face picture.
  • the color face picture first passes through three convolution layers with a step size of 1 and a down-sampling layer to obtain a feature map of 80 * 80.
  • the feature map of 80 * 80 is then input to the first
  • the dense connection block is used as the input feature map of the first dense connection block.
  • the dense connection block contains three convolutional layers.
  • the input feature map is first input to convolutional layer 1.
  • the input feature map is concatenated with the output feature map of convolutional layer 1 and input to convolutional layer 2.
  • the convolutional layer 1 and convolutional layer 2 The output feature map is spliced and input to the convolution layer 3. Downsample the output feature map of convolution layer 3 to 40 * 40 and input the next dense connection block, repeat the same operation. After three densely connected blocks, the size of the feature map becomes 20 * 20, and the feature map of 20 * 20 then passes through two convolution layers with a step size of 2 to obtain 64 3 * 3 feature maps and 64 3 * 3 feature maps.
  • Feature map input mean pooling layer to obtain 64-dimensional feature vectors.
  • the category of the training picture is output at the classification layer, and the error is calculated and back-propagated.
  • the features of the picture to be tested are output at the feature layer, and the neural network is trained until the loss function converges. Is h (x).
  • 2 to find the reference feature vector closest to y in S And distance i * argmin i ⁇ ⁇ 1,2, ..., m ⁇ d i , set the similarity threshold ⁇ , if Output Otherwise, the output A larger ⁇ represents a more relaxed judgment criterion, and a more relaxed judgment criterion tends to regard the test subject as a member of the local data set; a smaller ⁇ does the opposite.
  • the first error correction method is directed to a situation in which a test candidate whose identity is a local member is mistakenly identified as another member in the local member set.
  • the error of the true feature vector y T enhances the similarity between the predicted feature vector y of the sample to be tested and the true feature vector y T corresponding to the sample in the reference feature space, and reduces the reference feature vector corresponding to the wrong identity Similarity to the predicted feature vector y of the sample to be tested.
  • the second error correction method is directed to the case where the test candidate whose identity is a local member is incorrectly identified as a non-local member, by learning the predicted feature vector y of the test sample and the true feature vector y corresponding to the sample in the reference feature space
  • the error of T enhances the similarity between the predicted feature vector y of the sample to be tested and the true feature vector y T corresponding to the sample in the reference feature space.
  • the third error correction method is for the case where the testee of a non-local member is mistakenly identified as a local member, by learning the predicted feature vector y of the sample to be tested and the true feature vector y T corresponding to the sample in the reference feature space. Error, reducing the reference feature vector corresponding to the wrong identity Similarity to the predicted feature vector y of the sample to be tested.
  • the face recognition method provided in this application may be implemented on a terminal device, the device including at least one memory including a update member button, a delete member button, an input module, a computer software program storing the above-mentioned face recognition method, and a processor.
  • the input module may be a card swiping device or a keyboard for a test subject to input his or her identity tag.
  • the system suspends video streaming and saves the current input picture x and the prediction result.
  • the device may further include a permission obtaining module.
  • the invention also provides a simple way to add / remove members.
  • the system suspends video streaming and removes the information of the member to be deleted from the local individual set U and the reference feature space S.
  • the administrator is given permission to add / remove members through the device's Get Permissions module.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Collating Specific Patterns (AREA)

Abstract

一种在线学习的人脸识别方法，属于计算推算的技术领域，尤其涉及人脸识别的计算机视觉技术领域。该方法利用外部数据集训练人脸特征提取器，提取本地数据集中各成员对应的参考特征以构成参考特征空间，对比待测试样本的特征向量和参考特征以确定与待测试样本的特征向量最相似的参考特征，在与待测试样本的特征向量最相似的参考特征满足阈值要求时，以与待测试样本的特征向量最相似的参考特征所属成员的身份为待测试样本的身份，否则，返回待测试样本身份识别失败的消息，根据待测试样本的预测特征向量与其在参考特征空间中对应的真实特征向量的差异更新参考特征空间，适应人脸特征随时间推移发生的变化，尤其适合频繁变更成员的场合。

Description

一种在线学习的人脸识别方法 技术领域
本发明公开了一种在线学习的人脸识别方法，属于计算推算的技术领域，尤其涉及人脸识别的计算机视觉技术领域。
背景技术
人脸识别技术已经广泛运用于门禁、安检、监控等方面，其主要任务是区分数据库中的不同个体并拒绝数据库之外的个体。在实际应用中，人的相貌特征会受到装扮、表情的影响且因姿势、光照而变化，同一个人的正面图片也会随时间的推移而出现不同。为增加算法的鲁棒性，在识别过程中，有必要在特定情况下更新模型。传统的方法是重新收集样本再次训练，这种做法费时费力，难以操作。我们希望人脸识别设备在运行时可以自行调整模型且适应数据集的变化，因此，迫切需要一种操作简单且效果良好的在线学习方法。
现有的在线学习方法通过提取人脸的浅层特征(如：Haar特征、LBP特征)进行比对，在视频中识别并跟踪给定的人脸。在这种应用场景下，把目标人脸和周围的一个或多个人脸区分开，只需要辨别很少的样本；同时，在视频包含的小段时间内，人脸特征变化较小，因此，图像的浅层特征可以在一定程度上表征人脸特征。但是，人脸门禁、考勤等任务需要分辨包含数百人的数据库，在相当长的一段时间内，每个人的相貌都会有所改变，浅层特征难以处理如此复杂的任务。
深度神经网络提高了模型的辨识度，但网络的训练耗费大量的运算资源和时间，变更模型时需要将在离线服务器上训练好的模型重新导入人脸识别设备；另一方面，神经网络结构固定，增加/删除成员时同样需要再次训练，为实际应用带来不便。为使人脸识别技术的使用更灵活且使用范围更广，需要一种简便精确的在线学习方法。
发明内容
本发明的发明目的是针对上述背景技术的不足，提供了一种在线学习的人脸识别方法，以有限的计算资源和简便的操作流程在终端设备实现了模型的训练和更新，解决了现有人脸识别技术在数据集变化时需重新训练模型的技术问题。
本发明为实现上述发明目的采用如下技术方案：
一种在线学习的人脸识别方法，
建立外部数据集：根据研究机构的公开人脸数据库或自行搜集的数据建立外部数据集，示例性地，人脸数据库可以选择CASIA-WebFace、VGG-FACE等公开数据库；也可以自行在 网络上抓取公众人物的图片。每张图片都应含有身份标注，指明该图片属于哪个个体。应当收集尽可能多的个体，每个个体包含尽可能多的样本，同时减少数据集中错误标注样本的数量。样本数量和类别数量的增加会提高训练精度，且不会改变人脸特征提取器的结构或增加训练难度；
建立本地数据集：假设由m个人组成本地成员集合U＝{u 1,u 2,...,u m}，给U中的每个成员u i拍摄n张对应的人脸样本{x i1,x i2,...,x in}，优选地，人脸样本应该是光照正常、表情自然的照片，当条件允许拍摄多张图片时，可以关注表情和姿态的多样性；
训练模型：使用卷积神经网络作为特征提取器，神经网络的输入为彩色图片，神经网络的输出为图片所属类别，分类层的长度等于外部数据集的类别数，损失函数可以采用softmaxloss，需要说明的是，神经网络采用外部数据集训练，因为外部数据集的样本数量和种类远超本地数据集，有利于神经网络学习到更好的特征，损失函数随着误差的反向传播不断下降，训练准确率不断上升，当损失函数收敛并不再继续下降时，保存卷积神经网络模型，把与分类层相连的l维向量作为输入图片的特征向量，特征向量的维度一般远小于类别数量，可以取几十到几百之间，记输入图片x到特征向量的映射为h(x)，用训练好的特征提取器提取本地数据集的样本特征，计算得到每个个体对应的参考特征
Figure PCTCN2019078474-appb-000001
其中，n代表人脸库中第i个人的人脸样本个数，建立参考特征空间S＝{y 1,y 2,...,y m}；
预测待测图片所属个体的身份：在视频帧中截取待测者的人脸区域图片，处理所截图片得到待测图片x，使用特征提取器提取待测图片x的特征向量y，y＝h(x)，对所有y i∈S计算y与y i的距离d：d i＝||y-y i|| 2，d表征了两个特征之间的相似度。d越大表征特征差距就越大，更进一步地，当d足够大时，可以认为两个特征属于不同的个体，找出S中与y距离最近的参考向量
Figure PCTCN2019078474-appb-000002
以及距离
Figure PCTCN2019078474-appb-000003
i *＝argmin i∈{1,2,...,m}d i，设定相似度阈值δ，如果
Figure PCTCN2019078474-appb-000004
输出
Figure PCTCN2019078474-appb-000005
否则输出
Figure PCTCN2019078474-appb-000006
u代表模型预测的待测者身份；
在线纠错：当待测者识别失败并希望更新自身特征时，暂停视频流，将待测者输入的身份标签u T归入本地成员集合U，u T∈U，系统根据如下公式更新特征空间：
Figure PCTCN2019078474-appb-000007
Figure PCTCN2019078474-appb-000008
Figure PCTCN2019078474-appb-000009
其中：u T代表待测者的真实身份，由待测者自行提供，u代表待测者的预测身份，y T代表待测者在参考特征空间中对应的真实特征向量，y代表特征提取器从图片x提取的特征向量，
Figure PCTCN2019078474-appb-000010
代表S中与y距离最近的参考向量，η代表表征模型纠错幅度的学习率，η∈(0,1)，需要说明的是，较小的η值表示模型更加信任本地数据集预先采集的图片，较大的η值表示模型更信任新采集的图片，特征空间更新完毕后，恢复视频流，继续执行识别功能。
为实现更高效卷积神经网络，在网络中增加了至少一个稠密连接块，每个稠密连接块至少包含两个依次连接的卷积层，当前卷积层输出的特征图和该卷积层之前所有卷积层输出的特征图拼接后作为至下一卷积层的输入特征图，每一个稠密连接块输出的特征图都经降采样后传输至下一稠密连接块的输入端，优选地，输入卷积神经网络的彩色人脸图片经多个等步长的卷积层和降采样层的处理后得到输入第一个稠密连接块的特征图，对最后一个稠密连接块输出的特征图再进行卷积操作和均值池化操作得到输入至分类层的特征向量。
进一步的，本申请还提供了一种添加/删除成员后无需重新训练模型的人脸识别方法，添加成员时，新成员完成一次人脸识别过程后提供自己的真实身份标签u k，
Figure PCTCN2019078474-appb-000011
暂停视频流传输，保存当前输入图片x及特征提取器从当前图片提取的特征向量y，更新本地成员集合为U′，U′＝U∪u k，更新参考特征空间为S′，S′＝S∪y，更新完毕后恢复视频流；删除成员时，暂停视频流传输，在本地成员集合U和参考特征空间S中移除待删除成员的信息，恢复视频流。
本申请还提供了一种实现上述人脸识别方法的终端设备，该设备包括：存储器、处理器及存储在存储器上并在处理器上运行的计算机程序，处理器执行所述程序时实现以下步骤：利用外部数据集训练人脸特征提取器，提取本地数据集中各成员对应的参考特征以构成参考特征空间，对比待测试样本的特征向量和参考特征以确定与待测试样本的特征向量最相似的参考特征，在与待测试样本的特征向量最相似的参考特征满足阈值要求时，以与待测试样本的特征向量最相似的参考特征所属成员的身份为待测试样本的身份，否则，返回待测试样本身份识别失败的消息，根据待测试样本的预测特征向量与其在参考特征空间中对应的真实特征向量的差异更新参考特征空间。
本发明采用上述技术方案，具有以下有益效果：
(1)本发明提出了一种在终端动态更新人脸识别模型、添加或删除成员的方法，该方法通过灵活调整从本地数据集提取的参考特征空间以适应数据集的变化，实现了人脸识别模型的离线更新，相比于重新收集样本再次训练的传统方法，操作简单，计算量小，可以更好地 适应人脸特征随时间推移而发生的变化，尤其适合需要频繁变更成员的场合；
(2)本发明通过稠密连接的卷积神经网络实现特征提取，通过连接若干同步长的卷积层构成稠密连接层，每个卷积层的输出特征图和之前卷积层的所有输出特征图拼接后作出下一卷积层的输入特征图，强化了特征复用，提升了网络性能，减少了参数数量和运算量，鲁棒性更强，适用范围更广。
附图说明
图1是本方法的人脸识别流程图。
图2是数据集的人脸截取样例。
图3是本发明在线学习的流程图。
图4是稠密连接块的结构示意图。
具体实施方式
为了更清楚地说明本发明的特征，下面结合附图和具体实施方式进行进一步的详细描述。需要说明的是，以下的阐述提到了许多具体细节以便于充分理解本发明，本发明包括但不限于以下实施方例。
图1给出了根据本发明人脸识别方法的流程图，该人脸识别方法包括以下五个步骤。
步骤一、建立外部数据集：采用CASIA-WebFace数据库作为外部数据集，图2给出了经过处理的CASIA-WebFace数据库的样本实例，如图2所示，人脸框应该比较紧密地贴合人脸边缘，所有图片缩放成卷积神经网络的输入尺寸。如从其它数据集获取外部数据集，也需要遵循人脸框紧密地贴合人脸边缘以及图片满足神经网络输入图片尺寸要求的处理方式。
步骤二、建立本地数据集：拍摄十个人的脸部照片，拍摄每一个人表情和姿态不同的多张人脸样本图片。
步骤三、建立卷积神经网络：以外部数据集合为样本集训练人脸特征提取器：本申请涉及了一种更高效的卷积神经网络，如图4所示，神经网络的输入是160*160像素的彩色人脸图片，彩色人脸图片首先依次经过三个步长为1的卷积层和一个降采样层得到80*80的特征图，80*80的特征图随后输入至第一个稠密连接块做为第一个稠密连接块的输入特征图。稠密连接块包含三个卷积层，输入特征图首先输入卷积层1，输入特征图与卷积层1的输出特征图拼接后输入卷积层2；卷积层1和卷积层2的输出特征图拼接后输入卷积层3。将卷积层3的输出特征图降采样到40*40后输入下一个稠密连接块，重复相同的操作。经过三个稠密连接块后，特征图大小变为20*20，20*20的特征图随后经过两次步长为2的卷积层得到64个3*3的特征图，64个3*3特征图输入均值池化层得到64维特征向量。训练时，在分类层输出训练图片所属类别，计算误差并反向传播；测试时，在特征层输出待测图片的特征，训 练神经网络直到损失函数收敛，记此时神经网络输入到输出的映射为h(x)。
步骤四、构建参考特征空间：由训练后的人脸特征提取器提取本地样本集的特征，计算得到每个个体对应的参考特征y i，
Figure PCTCN2019078474-appb-000012
本地样本集中各个体对应的参考特征构成参考特征空间S，S＝{y 1,y 2,...,y m}。
步骤五、对比待测样本的预测特征向量和参考特征空间中的各参考特征向量确定待测样本所属个体：使用训练好的特征提取器预测待测图片x的特征向量y，y＝h(x)，对所有y i∈S，计算y与y i的距离：d i＝||y-y i|| 2，找出S中与y距离最近的参考特征向量
Figure PCTCN2019078474-appb-000013
以及距离
Figure PCTCN2019078474-appb-000014
i *＝argmin i∈{1,2,...,m}d i，设定相似度阈值δ，如果
Figure PCTCN2019078474-appb-000015
输出
Figure PCTCN2019078474-appb-000016
否则，输出
Figure PCTCN2019078474-appb-000017
较大的δ代表更宽松的判断标准，宽松的判断标准更倾向于把待测者看作本地数据集的某个成员；较小的δ反之。
在待测试样本识别失败并希望更新自身特征时，如图3所示，暂停视频流，将待测者输入的身份标签u T归入本地成员集合U，u T∈U，根据如下三种方式更新特征空间：
第一种：
Figure PCTCN2019078474-appb-000018
第二种：
Figure PCTCN2019078474-appb-000019
第三种：
Figure PCTCN2019078474-appb-000020
更新完毕后恢复视频流。
第一种纠错方式针对将身份为本地成员的待测试者错误识别为本地成员集合中另一成员的情形，通过学习待测试样本的预测特征向量y和待测试样本在参考特征空间中对应的真实特征向量y T的误差，增强待测试样本的预测特征向量y和待测试样本在参考特征空间中对应的真实特征向量y T的相似度，降低错误身份对应的参考特征向量
Figure PCTCN2019078474-appb-000021
与待测试样本的预测特征向量y的相似度。
第二种纠错方式针对将身份为本地成员的待测试者错误识别为非本地成员的情形，通过学习待测试样本的预测特征向量y和待测试样本在参考特征空间中对应的真实特征向量y T的误差，增强待测试样本的预测特征向量y和待测试样本在参考特征空间中对应的真实特征向量y T的相似度。
第三种纠错方式针对将非本地成员的待测试者误识别为本地成员的情形，通过学习待测试样本的预测特征向量y和待测试样本在参考特征空间中对应的真实特征向量y T的误差，降低错误身份对应的参考特征向量
Figure PCTCN2019078474-appb-000022
与待测试样本的预测特征向量y的相似度。
本申请提供的人脸识别方法可在终端设备上实现，该设备包括至少一个包含更新成员按键、一个删除成员按键、一个输入模块、存储有上述人脸识别方法的计算机软件程序的存储器及处理器。示例性的，输入模块可以是供待测者输入自己的身份标签的刷卡装置或键盘。系统暂停视频流传输，保存当次的输入图片x和预测结果。可选地，设备还可以包括获取权限模块。
本发明还提供了一种简便的添加/删除成员方式。添加成员时，新成员完成一次人脸识别过程，通过设备的输入模块提供自己的真实身份标签，发出添加成员指令(待测者按下更新成员按键)后，系统暂停视频流传输，保存当次的输入图片x和特征向量y，更新本地个体集合U′＝U∪u k，更新参考特征空间S′＝S∪y k，y k＝y；删除成员时，待测试者通过输入模块提供待删除的成员标签，发出删除成员(待测试者按下删除成员按键)指令后，系统暂停视频流传输，在本地个体集合U和参考特征空间S中移除待删除成员的信息。通过设备的获取权限模块授予管理员添加/删除成员的权限。

Claims (10)

  1. 一种在线学习的人脸识别方法，其特征在于，利用外部数据集训练人脸特征提取器，提取本地数据集中各成员对应的参考特征以构成参考特征空间，对比待测试样本的特征向量和参考特征以确定与待测试样本的特征向量最相似的参考特征，在与待测试样本的特征向量最相似的参考特征满足阈值要求时，以与待测试样本的特征向量最相似的参考特征所属成员的身份为待测试样本的身份，否则，返回待测试样本身份识别失败的消息，根据待测试样本的预测特征向量与其在参考特征空间中对应的真实特征向量的差异更新参考特征空间。
  2. 根据权利要求1所述一种在线学习的人脸识别方法，其特征在于，在待测试样本身份识别失败为将身份为本地成员的待测试样本错误识别为另一本地成员的情形时，通过学习待测试样本的预测特征向量与其在参考特征空间中对应的真实特征向量的误差，增强待测试样本的预测特征向量与其在参考特征空间中对应的真实特征向量的相似度，降低错误身份对应的参考特征向量与待测试样本的预测特征向量的相似度。
  3. 根据权利要求1所述一种在线学习的人脸识别方法，其特征在于，在待测试样本身份识别失败为将身份为本地成员的待测试者样本识别为非本地成员的情形时，通过学习待测试样本的预测特征向量与其在参考特征空间中对应的真实特征向量的误差，增强待测试样本的预测特征向量与其在参考特征空间中对应的真实特征向量的相似度。
  4. 根据权利要求1所述一种在线学习的人脸识别方法，其特征在于，在待测试样本身份识别失败为将非本地成员的待测试样本误识别为本地成员的情形时，通过学习待测试样本的预测特征向量与其在参考特征空间中对应的真实特征向量的误差，降低错误身份对应的参考特征向量与待测试样本的预测特征向量的相似度。
  5. 根据权利要求1所述一种在线学习的人脸识别方法，其特征在于，对比待测试样本的特征向量和参考特征以确定与待测试样本的特征向量最相似的参考特征的具体方法为：计算待测试样本的特征向量和所有参考特征的距离，以与 待测试样本的特征向量距离最短的参考特征为最相似的参考特征。
  6. 根据权利要求1所述一种在线学习的人脸识别方法，其特征在于，添加本地成员时，将新添加成员的身份信息添加至本地数据集，提取新添加成员图片的特征并将所提取的特征添加至参考特征空间。
  7. 根据权利要求1所述一种在线学习的人脸识别方法，其特征在于，删除成员时，从本地数据集及参考特征空间移除待删除成员的数据。
  8. 根据权利要求1至7中任意一项所述在线学习的人脸识别方法，其特征在于，人脸特征提取器通过包含至少一个稠密连接块的卷积神经网络实现，每个稠密连接块包含至少两个依次连接的同步长卷积层，当前卷积层输出的特征图和该卷积层之前所有卷积层输出的特征图拼接后作为至下一卷积层的输入特征图，每一个稠密连接块输出的特征图都经降采样后传输至下一稠密连接块的输入端。
  9. 根据权利要求8所述在线学习的人脸识别方法，其特征在于，对最后一个稠密连接块输出的特征图再进行卷积操作和均值池化操作得到输入至分类层的特征向量。
  10. 一种人脸识别终端设备，包括：存储器、处理器及存储在存储器上并在处理器上运行的计算机程序，其特征在于，所述处理器执行所述程序时实现以下步骤：利用外部数据集训练人脸特征提取器，提取本地数据集中各成员对应的参考特征以构成参考特征空间，对比待测试样本的特征向量和参考特征以确定与待测试样本的特征向量最相似的参考特征，在与待测试样本的特征向量最相似的参考特征满足阈值要求时，以与待测试样本的特征向量最相似的参考特征所属成员的身份为待测试样本的身份，否则，返回待测试样本身份识别失败的消息，根据待测试样本的预测特征向量与其在参考特征空间中对应的真实特征向量的差异更新参考特征空间。
PCT/CN2019/078474 2018-06-30 2019-03-18 一种在线学习的人脸识别方法 WO2020001084A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810719313.0 2018-06-30
CN201810719313.0A CN109145717B (zh) 2018-06-30 2018-06-30 一种在线学习的人脸识别方法

Publications (1)

Publication Number Publication Date
WO2020001084A1 true WO2020001084A1 (zh) 2020-01-02

Family

ID=64799766

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/078474 WO2020001084A1 (zh) 2018-06-30 2019-03-18 一种在线学习的人脸识别方法

Country Status (2)

Country Link
CN (1) CN109145717B (zh)
WO (1) WO2020001084A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111274886A (zh) * 2020-01-13 2020-06-12 天地伟业技术有限公司 一种基于深度学习的行人闯红灯违法行为分析方法及系统
CN111339990A (zh) * 2020-03-13 2020-06-26 乐鑫信息科技（上海）股份有限公司 一种基于人脸特征动态更新的人脸识别系统和方法
CN113221683A (zh) * 2021-04-27 2021-08-06 北京科技大学 教学场景下基于cnn模型的表情识别方法
CN113392678A (zh) * 2020-03-12 2021-09-14 杭州海康威视数字技术股份有限公司 行人检测方法、设备和存储介质

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109145717B (zh) * 2018-06-30 2021-05-11 东南大学 一种在线学习的人脸识别方法
CN110147845B (zh) * 2019-05-23 2021-08-06 北京百度网讯科技有限公司 基于特征空间的样本采集方法和样本采集系统
CN110363150A (zh) * 2019-07-16 2019-10-22 深圳市商汤科技有限公司 数据更新方法及装置、电子设备和存储介质
CN110378092B (zh) * 2019-07-26 2020-12-04 北京积加科技有限公司 身份识别系统及客户端、服务器和方法
CN110532956B (zh) * 2019-08-30 2022-06-24 深圳市商汤科技有限公司 图像处理方法及装置、电子设备和存储介质
CN112418067A (zh) * 2020-11-20 2021-02-26 湖北芯楚光电科技有限公司 一种基于深度学习模型的简便人脸识别在线学习方法
CN112967062B (zh) * 2021-03-02 2022-07-05 东华大学 基于谨慎度的用户身份识别方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6345109B1 (en) * 1996-12-05 2002-02-05 Matsushita Electric Industrial Co., Ltd. Face recognition-matching system effective to images obtained in different imaging conditions
CN102982321A (zh) * 2012-12-05 2013-03-20 深圳Tcl新技术有限公司 人脸库采集方法及装置
CN106778653A (zh) * 2016-12-27 2017-05-31 北京光年无限科技有限公司 面向智能机器人的基于人脸识别样本库的交互方法和装置
CN106815560A (zh) * 2016-12-22 2017-06-09 广州大学 一种应用于自适应驾座的人脸识别方法
CN107609493A (zh) * 2017-08-25 2018-01-19 广州视源电子科技股份有限公司 优化人脸图片质量评价模型的方法及装置
CN109145717A (zh) * 2018-06-30 2019-01-04 东南大学 一种在线学习的人脸识别方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104036323B (zh) * 2014-06-26 2016-11-09 叶茂 一种基于卷积神经网络的车辆检测方法
CN106022317A (zh) * 2016-06-27 2016-10-12 北京小米移动软件有限公司 人脸识别方法及装置
CN106778842A (zh) * 2016-11-30 2017-05-31 电子科技大学 一种基于knn分类的优化方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6345109B1 (en) * 1996-12-05 2002-02-05 Matsushita Electric Industrial Co., Ltd. Face recognition-matching system effective to images obtained in different imaging conditions
CN102982321A (zh) * 2012-12-05 2013-03-20 深圳Tcl新技术有限公司 人脸库采集方法及装置
CN106815560A (zh) * 2016-12-22 2017-06-09 广州大学 一种应用于自适应驾座的人脸识别方法
CN106778653A (zh) * 2016-12-27 2017-05-31 北京光年无限科技有限公司 面向智能机器人的基于人脸识别样本库的交互方法和装置
CN107609493A (zh) * 2017-08-25 2018-01-19 广州视源电子科技股份有限公司 优化人脸图片质量评价模型的方法及装置
CN109145717A (zh) * 2018-06-30 2019-01-04 东南大学 一种在线学习的人脸识别方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HU HUAN: "Vehicle Feature Learning and Vehicle Identification", CHINA MASTER'S THESES FULL-TEXT DATABASE, no. 2, 15 February 2018 (2018-02-15), pages 29-31 - 42-45, ISSN: 1674-0246 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111274886A (zh) * 2020-01-13 2020-06-12 天地伟业技术有限公司 一种基于深度学习的行人闯红灯违法行为分析方法及系统
CN111274886B (zh) * 2020-01-13 2023-09-19 天地伟业技术有限公司 一种基于深度学习的行人闯红灯违法行为分析方法及系统
CN113392678A (zh) * 2020-03-12 2021-09-14 杭州海康威视数字技术股份有限公司 行人检测方法、设备和存储介质
CN111339990A (zh) * 2020-03-13 2020-06-26 乐鑫信息科技（上海）股份有限公司 一种基于人脸特征动态更新的人脸识别系统和方法
CN111339990B (zh) * 2020-03-13 2023-03-24 乐鑫信息科技（上海）股份有限公司 一种基于人脸特征动态更新的人脸识别系统和方法
CN113221683A (zh) * 2021-04-27 2021-08-06 北京科技大学 教学场景下基于cnn模型的表情识别方法

Also Published As

Publication number Publication date
CN109145717B (zh) 2021-05-11
CN109145717A (zh) 2019-01-04

Similar Documents

Publication Publication Date Title
WO2020001084A1 (zh) 一种在线学习的人脸识别方法
WO2020001083A1 (zh) 一种基于特征复用的人脸识别方法
CN110135249B (zh) 基于时间注意力机制和lstm的人体行为识别方法
CN110555481B (zh) 一种人像风格识别方法、装置和计算机可读存储介质
WO2020038136A1 (zh) 面部识别方法、装置、电子设备及计算机可读介质
WO2019228317A1 (zh) 人脸识别方法、装置及计算机可读介质
WO2017088432A1 (zh) 图像识别方法和装置
WO2022100337A1 (zh) 人脸图像质量评估方法、装置、计算机设备及存储介质
CN112364827B (zh) 人脸识别方法、装置、计算机设备和存储介质
KR20160101973A (ko) 비제약형 매체에 있어서 얼굴을 식별하는 시스템 및 방법
WO2021218238A1 (zh) 图像处理方法和图像处理装置
CN110175515B (zh) 一种基于大数据的人脸识别算法
CN113205002B (zh) 非受限视频监控的低清人脸识别方法、装置、设备及介质
WO2023231753A1 (zh) 一种神经网络的训练方法、数据的处理方法以及设备
WO2023123923A1 (zh) 人体重识别方法、人体重识别装置、计算机设备及介质
Xia et al. Face occlusion detection using deep convolutional neural networks
CN111898561A (zh) 一种人脸认证方法、装置、设备及介质
CN112488003A (zh) 一种人脸检测方法、模型创建方法、装置、设备及介质
CN113298158A (zh) 数据检测方法、装置、设备及存储介质
CN113128526B (zh) 图像识别方法、装置、电子设备和计算机可读存储介质
Echoukairi et al. Improved Methods for Automatic Facial Expression Recognition.
Fegade et al. Residential security system based on facial recognition
WO2020232697A1 (zh) 一种在线人脸聚类的方法及系统
Satpute et al. Online Classroom Attendance Marking System Using Face Recognition, Python, Computer Vision, and Digital Image Processing
CN113824989A (zh) 一种视频处理方法、装置和计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19824725

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19824725

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19824725

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS (EPO FORM 1205A DATED 10.08.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19824725

Country of ref document: EP

Kind code of ref document: A1