WO2023019699A1 - 一种基于3d人脸模型的俯角人脸识别方法及系统 - Google Patents

一种基于3d人脸模型的俯角人脸识别方法及系统 Download PDF

Info

Publication number
WO2023019699A1
WO2023019699A1 PCT/CN2021/122347 CN2021122347W WO2023019699A1 WO 2023019699 A1 WO2023019699 A1 WO 2023019699A1 CN 2021122347 W CN2021122347 W CN 2021122347W WO 2023019699 A1 WO2023019699 A1 WO 2023019699A1
Authority
WO
WIPO (PCT)
Prior art keywords
face
angle
depression angle
picture
model
Prior art date
Application number
PCT/CN2021/122347
Other languages
English (en)
French (fr)
Inventor
王中元
吴浩
黄宝金
王光成
曾康利
Original Assignee
武汉大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 武汉大学 filed Critical 武汉大学
Publication of WO2023019699A1 publication Critical patent/WO2023019699A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the invention belongs to the technical field of computer vision, and relates to a method and system for face recognition at a depression angle of a surveillance video, in particular to a method and a system for recognizing a face at a depression angle based on a 3D face model.
  • Public security organs use surveillance cameras to shoot and record video images, and use face recognition technology to track target suspects and identify criminals.
  • public surveillance cameras are usually placed in relatively high positions such as utility poles and eaves, and the shooting angle is usually in a bird's-eye view posture, and the bird's-eye view cameras often collect side faces, depression angles, and low-resolution face pictures.
  • There are problems such as lack of chin information and serious deformation in the face of the depressed angle, which makes the performance of general face recognition technologies such as Arcface and Facenet drop sharply.
  • the current multi-pose face recognition scheme is mainly based on the principles of face conversion and multi-frame information complementarity.
  • the goal of multi-pose face correction is to use an algorithm to synthesize a frontal face given a variable-angle face image, and Generative Adversarial Networks (GAN) has become the mainstream solution for face correction.
  • GAN Generative Adversarial Networks
  • the GAN-based method uses the principle of left-right symmetry of the face to correct the collected side face pictures to the front face pose to improve the recognition accuracy.
  • Due to the lack of left-right symmetry available in the profile face it is difficult to estimate the self-occlusion of the chin.
  • Directly applying the GAN multi-pose face recognition scheme to the face recognition of the down-angle face cannot achieve the expected results.
  • Face recognition based on video sequences uses multiple frames of face pictures with complementary information to synthesize a single recognition feature. Since multiple frames of pictures with different angles are required for feature fusion, it cannot be used for the recognition of a single face with a depression angle.
  • the present invention combines the 3D face model and face pose estimation to convert the front face picture in the face sample library into a face with the same angle as the face to be recognized at the depression angle, and then recognize it. Since the rotation and rendering of the 3D face model can be applied to any angle without losing details, firstly, the clear front face image in the sample library is established as a 3D face model; then the face pose estimation algorithm is used to estimate the face to be recognized at the depression angle The angle of the picture and turn the 3D face model into the same angle as the face picture of the depression angle; finally, input the generated face of the depression angle and the face picture of the depression angle to be recognized into the face recognition network for recognition. Since the frontal face is turned from a picture with more information to a picture with less information, the face distortion phenomenon can be effectively avoided.
  • the technical solution adopted in the method of the present invention is: a method for identifying people's faces at a depression angle based on a 3D face model, comprising the following steps:
  • Step 1 Construction of the depression angle face sample library
  • Collect a front face picture input the front face picture to the 3D face reconstruction network, generate a 3D face model, rotate the 3D face model according to the preset angle interval, and remap it back to a 2D face picture , saved to the depression angle face sample library;
  • Step 2 When a new image of a face with a depression angle to be recognized is input, the information of the depression angle of the face is estimated, and all face images with the closest angles to it are selected from the sample database of depression angle faces for face recognition.
  • a depression angle face recognition system based on a 3D face model comprising the following modules:
  • Collect a front face picture input the front face picture to the 3D face reconstruction network, generate a 3D face model, rotate the 3D face model according to the preset angle interval, and remap it back to a 2D face picture , saved to the depression angle face sample library;
  • Module 2 is used for estimating the depression angle information of a face when a new depression angle face picture to be recognized is input, and selecting all face pictures with the closest angles to it from the depression angle face sample database for face recognition.
  • the face database stores frontal face pictures, and during recognition, the input face with variable poses is corrected to a frontal face and compared with the frontal face in the face database.
  • the process of correcting a face with variable poses to a positive face is a process of converting an information-deficient object into an information-complete object, so distortion is inevitable, which affects the subsequent face recognition accuracy.
  • the present invention adopts the opposite strategy, directly saves the faces of different poses in the face database (the present invention adopts the mode of 3D modeling to generate the multi-pose version of the front face), and directly recognizes the input variable pose faces, and There is no need to correct in advance, thereby avoiding the distortion effect caused by attitude correction. Therefore, compared with the existing multi-pose face recognition method, the present invention has the following advantages and positive effects:
  • the present invention proposes a depression angle face recognition scheme based on a 3D face model from a frontal face to a depression angle face.
  • This scheme can not only overcome the problem of lack of symmetry information on faces with depression angles, but also effectively avoid face distortion due to the transformation from images with more information to images with less information when the face is turned from the front to the angle of depression.
  • the method of the present invention does not need the depression angle face data set for training. Reconstructing a face using a general-purpose 3D face model only requires reconstruction of the frontal image of the face, and the detection of key points of the face required to calculate the angle of rotation of the face with a depression angle only requires a general-purpose face dataset.
  • the method of the present invention adopts common human face data sets in the training process, thereby avoiding the problem of lack of depression angle human face data sets.
  • Fig. 1 a schematic block diagram of the method of the embodiment of the present invention.
  • Fig. 2 A flow chart of the construction of the depression angle face sample database according to the embodiment of the present invention.
  • Fig. 3 The flow chart of the face recognition of the embodiment of the present invention.
  • Fig. 4 Examples of human face pictures collected from different overlooking angles according to the embodiment of the present invention.
  • the depression angle face recognition system is different from the normal face recognition system in the construction of the sample library.
  • the sample library of the ordinary face recognition only needs the front face, records the ID and name information corresponding to the face, and then in the face recognition process , whenever a new face image to be recognized is input, all faces in the library can be directly retrieved.
  • the depression angle face recognition proposed by the present invention adds the steps of 3D face reconstruction and synthesis of depression angle face pictures in the construction of the sample library.
  • the depression angle face recognition system is different from the normal face recognition system in face recognition. Ordinary face recognition directly compares the face to be recognized with all face pictures in the sample library.
  • the depression angle face recognition system proposed by the present invention estimates the angle of the face picture to be recognized when inputting a face picture to be recognized, and selects the face with the closest angle from each user ID in the sample library. pictures for comparison.
  • a kind of depression angle face recognition method based on 3D face model provided by the present invention, comprises the following steps:
  • Step 1 Construction of the depression angle face sample library
  • the 3D face reconstruction network adopts existing networks, such as Prnet and 3DDFA-V2.
  • step 1 includes the following sub-steps:
  • Step 1.1 First use the camera to take a high-definition frontal picture of the new user ID.
  • Step 1.2 Use the face detection algorithm RetinaFace to detect the face border in the front face picture, and cut the face picture according to the face border. After cutting the face, use the 3D face reconstruction network to generate a 3D face model .
  • the face alignment algorithm Face_Alignment to return the key point coordinates of the face (68 key points, 96 key points, 106 key points or more dense key points); then map the 2D face to the 3D face according to the key point coordinates of the face
  • Step 1.3 Transform the 3D model created by the front face image at 15° angle intervals, and the angle transformation formula is:
  • V transform s*o*R*V+h
  • s represents the scaling factor of the 3D face model
  • o is an orthogonal matrix
  • R is a rotation matrix
  • h is an offset matrix
  • the pictures stored in each user ID in the depression angle face sample library are: the original real front face picture, the generated face pictures of different depression angles obtained by 3D face model transformation, denoted as:
  • I set ⁇ I 1 ,I 2 ,...,I M ⁇
  • I k indicates that all face angle pictures with user ID k are stored in the database.
  • I set represents all the pictures stored in the database, which are classified according to the user ID.
  • ⁇ represents the angle of the face images saved in the database. The specific number of angles and angle values are not fixed, and need to be determined through comprehensive experimental evaluation and storage consumption of the sample library.
  • the high-definition frontal face picture and the multiple face pictures obtained through step 1.3 are jointly numbered as the same ID, and the files are named according to the angle information. Create a new user ID in the database table, which stores the information of the picture.
  • the present invention uses a database to store the face sample library.
  • the table of the database designed by the present invention adopts MySQL database.
  • the primary key of the table is the ID number of the face picture, and each ID represents a group of pictures of each person.
  • Each ID number contains pictures from different angles of the same person, which can be distinguished and retrieved according to the key ⁇ in the table.
  • only the image path where the smallest angle ⁇ 1 is located is the real face image collected, and the other image paths represented by ⁇ 2 ,..., ⁇ n ⁇ are all created synthetic face images.
  • Step 2 When a new picture of a face with a depression angle to be recognized is input, use the face pose estimation algorithm to estimate the information of the depression angle of the face, and select all the sample library pictures with similar angles from the depression angle face sample library for face recognition .
  • an existing face pose estimation algorithm such as PFLD and FSA-Net, is selected.
  • step 2 includes the following sub-steps:
  • Step 2.1 Detect the position of the face in the input image, and crop the face area.
  • Step 2.2 Estimating the pose of the face to be recognized, using the face pose estimation algorithm to estimate the depression angle information of the face to be recognized
  • the specific process is as follows:
  • A y 1 -y 31
  • B x 31 -x 1
  • C x 1 y 2 -x 2 y 1 .
  • (x 1 ,y 1 ), (x 2 ,y 2 ), (x 31 ,y 31 ), (x 51 ,y 51 ) represent the coordinates of the 1st, 2nd, 31st, and 51st key points of the face, respectively.
  • Step 2.3 Input the face picture to be recognized and all the face pictures with angle ⁇ i in the id in the sample library to the existing face recognition network ArcFace to obtain the face feature vector. And compare each other to find the two faces with the greatest similarity.
  • Step 2.4 If the similarity is greater than the set threshold ⁇ , it indicates that the two pictures with the highest similarity are the same person, otherwise it indicates that the face picture to be recognized is not in the sample library.
  • the present invention sets the threshold ⁇ , that is, when the depression angle posture of the face picture to be recognized is estimated using the face posture estimation algorithm, when When the depression angle of the face picture to be recognized is ⁇ , the similarity between the face picture to be recognized and the frontal face picture in the face sample database is directly compared.
  • the present invention collects real depression angle face samples for experiments, and some face samples are shown in FIG. 4 .
  • Table 1 shows the results of face recognition accuracy at different depression angles. It can be seen that at low depression angles such as 15° and 30°, the accuracy of ArcFace direct recognition is higher, but at heights such as 45°, 60°, and 75° When the depression angle is used, the accuracy of the method of the present invention is obviously improved, especially when the angle is 75°, the accuracy is increased by more than 10%.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Collating Specific Patterns (AREA)

Abstract

一种基于3D人脸模型的俯角人脸识别方法及系统，通过将人脸样本库中清晰正脸转成与待识别的俯角人脸相同的角度进行识别。首先，采集清晰正脸图片构建人脸样本库，并将样本库中正脸图片生成3D人脸模型；然后，使用人脸姿态估计算法估计待识别俯角人脸图片的角度，并将3D人脸模型转成与俯角人脸图片相同的角度；最后，将生成的俯角人脸和待识别俯角人脸一起输入到人脸识别网络中进行识别。上述方法针对实际的俯视监控场景下的人脸识别难题，显著改善了俯角人脸的识别精度。

Description

一种基于3D人脸模型的俯角人脸识别方法及系统 技术领域
本发明属于计算机视觉技术领域，涉及一种监控视频的俯角人脸识别方法及系统，具体涉及一种基于3D人脸模型的俯角人脸识别方法及系统。
技术背景
公安机关通过监控摄像头拍摄和记录视频图像，利用人脸识别技术追踪目标嫌疑人，锁定犯罪分子的身份。然而，公共监控摄像头通常安放在电线杆、房檐等比较高的位置，拍摄角度通常呈俯视姿态，俯视摄像采集的往往是侧脸、俯角、低清的人脸图片。俯角人脸存在下巴信息缺失、形变严重等问题，使得Arcface、Facenet等通用人脸识别技术的性能急剧下降。
目前的多姿态人脸识别方案主要基于人脸转正和基于多帧信息互补的原理。多姿态人脸转正的目标是在给定变角度人脸图像的情况下利用算法合成正脸，生成对抗网络(Generative Adversarial Networks，GAN)成为人脸转正的主流方案。基于GAN的方法利用人脸左右对称性的原理，将采集到的侧脸图片校正到正脸姿态来提高识别精度。然而，由于俯视人脸缺少侧脸那样可利用的左右对称性，导致下巴自遮挡部分难以估计，直接将GAN多姿态人脸识别方案应用到俯角人脸识别上，并不能取得预期效果。此外，训练GAN网络需要海量数据，训练数据集过小会导致生成的正脸图片质量低下。基于视频序列的人脸识别使用多帧信息互补的人脸图片合成单个识别特征，由于需要多帧不同角度的图片进行特征融合，无法用于单张俯角人脸的识别。
总之，社会治安场合的监控摄像头通常高位安装，摄录的俯角人脸图像难以被现有的人脸识别系统准确识别，亟待为俯视监控场景下的人脸识别任务提出有效方案。
发明内容
为了解决上述技术问题，本发明结合3D人脸模型和人脸姿态估计，将人脸样本库中正脸图片转成与待识别俯角人脸相同角度的人脸，再进行识别。由于3D人脸模型旋转和渲染可以应用于任意角度，而不会丢失细节，所以首先将样本库中清晰正脸图片建立成3D人脸模型；然后使用人脸姿态估计算法估计待识别俯角人脸图片的角度并将3D人脸模型转成与俯角人脸图片相同的角度；最后， 将生成的俯角人脸和待识别俯角人脸图片一起输入到人脸识别网络中进行识别。由于正脸转俯角人脸是由信息多图片转为信息少的图片，因而有效避免人脸失真现象。
本发明的方法所采用的技术方案是：一种基于3D人脸模型的俯角人脸识别方法，包括以下步骤：
步骤1：俯角人脸样本库的构建；
采集一张人脸正脸图片，将人脸正脸图片输入到3D人脸重建网络，生成3D人脸模型，并按照预设角度间隔旋转3D人脸模型，将其重新映射回2D人脸图片，保存到俯角人脸样本库中；
步骤2：当新输入一张待识别俯角人脸图片时，估计人脸俯角信息，从俯角人脸样本库中选取所有与其最相近的角度的人脸图片，进行人脸识别。
本发明的系统所采用的技术方案是：一种基于3D人脸模型的俯角人脸识别系统，包括以下模块：
模块1，用于俯角人脸样本库的构建；
采集一张人脸正脸图片，将人脸正脸图片输入到3D人脸重建网络，生成3D人脸模型，并按照预设角度间隔旋转3D人脸模型，将其重新映射回2D人脸图片，保存到俯角人脸样本库中；
模块2，用于当新输入一张待识别俯角人脸图片时，估计人脸俯角信息，从俯角人脸样本库中选取所有与其最相近的角度的人脸图片，进行人脸识别。
现有的基于姿态校正的多姿态人脸识别方案中，人脸库存储正面脸图片，识别时，将输入的变姿态人脸校正为正脸后与人脸库中的正脸进行比对。将变姿态人脸校正为正脸的过程系由信息亏损对象转换为信息完整对象的处理，因而难免存在失真，从而影响了后续的人脸识别精度。本发明采取相反的策略，将不同姿态的人脸直接保存到人脸库中(本发明采取3D建模的方式生成正脸的多姿态版本)，直接对输入的变姿态人脸进行识别，而不用事先校正，从而避免了姿态校正带来的失真效应。因此，与现有的多姿态人脸识别方法相比，本发明具有以下的优点与积极效果：
(1)本发明提出了基于3D人脸模型的正脸转俯角人脸的俯角人脸识别方案。该方案不仅可以克服俯角人脸缺少对称性信息的问题，而且由于正脸转俯角人脸是由信息多图片转为信息少的图片，有效避免人脸失真现象。
(2)本发明方法不需要俯角人脸数据集进行训练。采用通用3D人脸模型重建人脸只需要重建人脸正脸图片，计算俯角人脸旋转角度所需要的人脸关键点检测只需要通用人脸数据集。本发明方法在训练过程中均采用通用的人脸数据集，从而避免了俯角人脸数据集缺少的问题。
附图说明
图1：本发明实施例的方法原理框图。
图2：本发明实施例的俯角人脸样本库构建的流程图。
图3：本发明实施例的人脸识别的流程图。
图4：本发明实施例采集的不同俯视角度的人脸图片样例。
具体实施方式
为了便于本领域普通技术人员理解和实施本发明，下面结合附图及实施案例对本发明做进一步的详细描述，应当理解，此处所描述的实施示例仅用于说明和解释本发明，并不用于限定本发明。
俯角人脸识别系统与正常人脸识别系统在样本库构建上有所不同，普通人脸识别的样本库只需要正脸，记录对应该人脸的ID和姓名信息，然后在人脸识别过程中，每当输入一张新的待识别人脸图片，就可以直接对库中所有人脸进行检索。但是本发明提出的俯角人脸识别在样本库的构建中增加了3D人脸重建及合成俯角人脸图片的步骤。
俯角人脸识别系统与正常人脸识别系统在人脸识别上有所不同。普通人脸识别直接将待识别人脸与样本库中所有人脸图片进行比较。本发明所提出的俯角人脸识别系统在输入一张待识别人脸图片的时候，对待识别人脸图片的角度进行估计，并从样本库中每个用户ID下角度与之最相近的人脸图片进行比较。
请见图1，本发明提供的一种基于3D人脸模型的俯角人脸识别方法，包括以下步骤：
步骤1：俯角人脸样本库的构建；
首先使用摄像头拍摄人脸正脸图片，将正脸图片输入到3D人脸重建网络，产生3D人脸模型，并按照一定角度间隔旋转3D人脸模型，将其重新映射回2D人脸图片，保存到俯角人脸样本库中。
本实施例中，3D人脸重建网络采用现有的网络，如Prnet，3DDFA-V2。
请见图2，本实施例中，步骤1的具体实现包括以下子步骤：
步骤1.1：首先使用摄像头拍摄一张新用户ID的高清人脸正脸图片。
步骤1.2：使用人脸检测算法RetinaFace检测出人脸正脸图片中的人脸边框，根据人脸边框对人脸图片进行裁剪，裁剪完人脸后，使用3D人脸重建网络生成3D人脸模型。
先利用人脸对齐算法Face_Alignment回归人脸的关键点坐标(68关键点、96关键点、106关键点或者更加稠密的关键点)；再根据人脸关键点坐标将2D人脸映射到3D人脸模型上，得到人脸3D形状信息V＝[v 1,v 2,…,v n]，n表示3D人脸模型的顶点个数，v i＝[x i,y i,z i] T表示顶点的空间位置；然后使用纹理坐标映射获得人脸纹理信息T＝[t 1,t 2,…,t n]，t i＝[r i,g i,b i] T表示顶点的纹理颜色信息；最后将人脸3D形状信息V和人脸纹理信息T进行融合，形成最终的3D人脸模型M＝{V,T}。
步骤1.3：将人脸正脸图片建立出的3D模型按照15°角度间隔进行转换，角度变换公式为：
V transform＝s*o*R*V+h
其中，s代表3D人脸模型的缩放因子，o是正交矩阵，R是旋转矩阵，h是偏移矩阵。
这样，俯角人脸样本库中每一个用户ID中存储的图片为：原始的真实正脸图片、利用3D人脸模型变换所获得的不同俯角的生成人脸图片，记作：
Figure PCTCN2021122347-appb-000001
I set＝{I 1,I 2,…，I M}
Θ＝{θ 1,θ 2,…，θ n}
其中，I k表示数据库中存储着用户ID为k的所有人脸角度图片。I set表示数据库中存储的所有图片，根据用户ID进行分类。Θ表示数据库中保存的人脸图像的角度。具体的角度的个数和角度值不固定，需要综合实验评价和样本库的存储消耗确定。
本实施例将高清正脸人脸图片和经过步骤1.3获得的多张人脸图片共同编号为同一ID，并且根据角度信息命名文件。在数据库的表中新建一个用户ID，里面存储图片的信息。
本实施例为了增加人脸样本库中人脸图片的存储效率和提高人脸识别的速度，本发明采用数据库存储人脸样本库。本发明设计的数据库的表采用MySQL数据库。其中，表的主键为人脸图片的ID号，每一个ID表示每个人的一组图片。每个ID号中包含同一个人的不同角度图片，根据表中的键θ进行区分和检索。其中只有最小的角度θ 1所在的图片路径为真实收集的人脸图片，其余{θ 2,…,θ n}所表示的图片路径均是建立出的合成人脸图片。
步骤2：当新输入一张待识别俯角人脸图片时，利用人脸姿态估计算法估计人脸俯角信息，从俯角人脸样本库中选取所有与其相近的角度的样本库图片，进行人脸识别。
本实施例中，选择现有的人脸姿态估计算法，如PFLD，FSA-Net。
请见图3，本实施例中，步骤2的具体实现包括以下子步骤：
步骤2.1：检测输入图片中人脸所在位置，裁剪人脸区域。
步骤2.2：对待识别人脸进行姿态估计，使用人脸姿态估计算法估计待识别人脸的俯角信息
Figure PCTCN2021122347-appb-000002
具体过程如下：
以98个人脸关键点为例，令(x i,y i)表示检测到的人脸关键点i的坐标，用d表示人脸关键点之间连线的距离：
Figure PCTCN2021122347-appb-000003
其中，A＝y 1-y 31,B＝x 31-x 1,C＝x 1y 2-x 2y 1。(x 1,y 1)、(x 2,y 2)、(x 31,y 31)、(x 51,y 51)分别代表人脸的第1、2、31、51个关键点的坐标。
则俯角
Figure PCTCN2021122347-appb-000004
的计算公式为：
Figure PCTCN2021122347-appb-000005
Figure PCTCN2021122347-appb-000006
然后在数据库中保存的人脸图像角度Θ中查找一个与待识别人脸图像角度
Figure PCTCN2021122347-appb-000007
最接近的角度θ i，即：找到一个θ i∈Θ,s.t
Figure PCTCN2021122347-appb-000008
有
Figure PCTCN2021122347-appb-000009
步骤2.3：将待识别人脸图片和样本库中所有id中角度为θ i的人脸图片输入到现有的人脸识别网络ArcFace中获得人脸特征向量。并两两之间进行比较，找出相似度最大的两张人脸。
步骤2.4：如果相似度大于设定的阈值ξ则表明相似度最大的两张图片为同一 个人，否则表明待识别人脸图片不在样本库中。
在具体的人脸识别过程中，为了提高本算法的俯角人脸识别的精度和速度，本发明设定了阈值ξ，即使用人脸姿态估计算法估计待识别人脸图片的俯角姿态时，当待识别人脸图片俯角θ≤ξ时,直接将待识别人脸图片与人脸样本库中正脸图片进行相似度比较。
本发明采集真实的俯角人脸样本进行了实验，部分人脸样例如图4所示。表1显示了不同俯角时人脸识别精度的结果，可以看出，在15°、30°等低度俯角时，ArcFace直接识别的精度更高，但在45°、60°、75°等高度俯角时，本发明方法的精度明显提升，尤其在75°时，精度提高了10％以上。
表1
Figure PCTCN2021122347-appb-000010
应当理解的是，本说明书未详细阐述的部分均属于现有技术。
应当理解的是，上述针对较佳实施例的描述较为详细，并不能因此而认为是对本发明专利保护范围的限制，本领域的普通技术人员在本发明的启示下，在不脱离本发明权利要求所保护的范围情况下，还可以做出替换或变形，均落入本发明的保护范围之内，本发明的请求保护范围应以所附权利要求为准。

Claims (6)

  1. 一种基于3D人脸模型的俯角人脸识别方法，其特征在于，包括以下步骤：
    步骤1：俯角人脸样本库的构建；
    采集一张人脸正脸图片，将人脸正脸图片输入到3D人脸重建网络，生成3D人脸模型，并按照预设角度间隔旋转3D人脸模型，将其重新映射回2D人脸图片，保存到俯角人脸样本库中；
    步骤2：当新输入一张待识别俯角人脸图片时，估计人脸俯角信息，从俯角人脸样本库中选取所有与其最相近的角度的人脸图片，进行人脸识别。
  2. 根据权利要求1所述的基于3D人脸模型的俯角人脸识别方法，其特征在于：步骤1中，使用人脸检测算法检测出人脸正脸图片中的人脸边框，根据人脸边框对人脸图片进行裁剪，使用3D人脸重建网络按如下过程生成3D人脸模型：
    先利用人脸对齐算法回归人脸的关键点坐标；再根据人脸关键点坐标将2D人脸映射到3D人脸模型上，得到人脸3D形状信息V＝[v 1,v 2,…,v n]，n表示3D人脸模型的顶点个数，v i＝[x i,y i,z i] T表示顶点的空间位置；然后使用纹理坐标映射获得人脸纹理信息T＝[t 1,t 2,…,t n]，t i＝[r i,g i,b i] T表示顶点的纹理颜色信息；最后将人脸3D形状信息V和人脸纹理信息T进行融合，形成最终的3D人脸模型M＝{V,T}。
  3. 根据权利要求2所述的基于3D人脸模型的俯角人脸识别方法，其特征在于：步骤1中，将人脸正脸图片建立出的3D模型按照15°角度间隔进行转换，获得俯角人脸样本库，角度变换公式为：
    V transform＝s*o*R*V+h
    其中，s代表3D人脸模型的缩放因子，o是正交矩阵，R是旋转矩阵，h是偏移矩阵。
    所述俯角人脸样本库中每一个用户ID中存储的图片包括：原始的真实正脸图片、利用3D人脸模型变换所获得的不同俯角的生成人脸图片，记作：
    Figure PCTCN2021122347-appb-100001
    I set＝{I 1,I 2,…，I N}
    Θ＝{θ 1,θ 2,…，θ n}
    其中，I k表示数据库中存储着用户ID为k的所有人脸角度图片，其中只有最 小的角度θ 1所在的图片路径为真实收集的人脸图片，其余{θ 2,…,θ n}所表示的图片路径均是建立出的合成人脸图片；I set表示数据库中存储的所有图片，根据用户ID进行分类，N表示用户总数；Θ表示数据库中保存的人脸图像的角度。
  4. 根据权利要求1所述的基于3D人脸模型的俯角人脸识别方法，其特征在于，步骤2的具体实现包括以下子步骤：
    步骤2.1：检测输入图片中人脸所在位置，裁剪人脸区域；
    步骤2.2：对待识别人脸进行姿态估计，估计待识别人脸的俯角信息
    Figure PCTCN2021122347-appb-100002
    然后在俯角人脸样本库中保存的人脸图像角度Θ中查找一个与待识别人脸图像角度
    Figure PCTCN2021122347-appb-100003
    最接近的角度θ i；
    步骤2.3：将待识别人脸图片和俯角人脸样本库中所有用户ID中角度为θ i的人脸图片输入到人脸识别网络中获得人脸特征向量；并两两之间进行比较，找出相似度最大的两张人脸；
    步骤2.4：如果相似度大于设定的阈值ε，则表明相似度最大的两张图片为同一个人，否则表明待识别人脸图片不在俯角人脸样本库中。
  5. 根据权利要求4所述的基于3D人脸模型的俯角人脸识别方法，其特征在于：步骤2.2中所述俯角信息
    Figure PCTCN2021122347-appb-100004
    计算方法：
    Figure PCTCN2021122347-appb-100005
    Figure PCTCN2021122347-appb-100006
    其中，d表示人脸关键点之间连线的距离，(x i,y i)表示人脸关键点i的坐标，y 51、y 31表示第51、31个关键点的y坐标。
  6. 一种基于3D人脸模型的俯角人脸识别系统，其特征在于，包括以下模块：
    模块1，用于俯角人脸样本库的构建；
    采集一张人脸正脸图片，将人脸正脸图片输入到3D人脸重建网络，生成3D人脸模型，并按照预设角度间隔旋转3D人脸模型，将其重新映射回2D人脸图片，保存到俯角人脸样本库中；
    模块2，用于当新输入一张待识别俯角人脸图片时，估计人脸俯角信息，从俯角人脸样本库中选取所有与其最相近的角度的人脸图片，进行人脸识别。
PCT/CN2021/122347 2021-08-16 2021-09-30 一种基于3d人脸模型的俯角人脸识别方法及系统 WO2023019699A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110935424.7A CN113705393A (zh) 2021-08-16 2021-08-16 一种基于3d人脸模型的俯角人脸识别方法及系统
CN202110935424.7 2021-08-16

Publications (1)

Publication Number Publication Date
WO2023019699A1 true WO2023019699A1 (zh) 2023-02-23

Family

ID=78652754

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/122347 WO2023019699A1 (zh) 2021-08-16 2021-09-30 一种基于3d人脸模型的俯角人脸识别方法及系统

Country Status (2)

Country Link
CN (1) CN113705393A (zh)
WO (1) WO2023019699A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115914583A (zh) * 2023-02-28 2023-04-04 中国科学院长春光学精密机械与物理研究所 基于视觉识别的老人监控设备及监控方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105654048A (zh) * 2015-12-30 2016-06-08 四川川大智胜软件股份有限公司 一种多视角人脸比对方法
CN107316340A (zh) * 2017-06-28 2017-11-03 河海大学常州校区 一种基于单张照片的快速人脸建模方法
CN107729806A (zh) * 2017-09-05 2018-02-23 西安理工大学 基于三维人脸重建的单视图多姿态人脸识别方法
CN108257210A (zh) * 2018-02-28 2018-07-06 浙江神造科技有限公司 一种通过单张照片生成人脸三维模型的方法
WO2020211347A1 (zh) * 2019-04-16 2020-10-22 平安科技（深圳）有限公司 基于人脸识别的修改图片的方法、装置和计算机设备

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106203400A (zh) * 2016-07-29 2016-12-07 广州国信达计算机网络通讯有限公司 一种人脸识别方法及装置
CN109684951A (zh) * 2018-12-12 2019-04-26 北京旷视科技有限公司 人脸识别方法、底库录入方法、装置及电子设备
CN110287880A (zh) * 2019-06-26 2019-09-27 西安电子科技大学 一种基于深度学习的姿态鲁棒性人脸识别方法
CN110569768B (zh) * 2019-08-29 2022-09-02 四川大学 人脸模型的构建方法、人脸识别方法、装置及设备
CN110991281B (zh) * 2019-11-21 2022-11-04 电子科技大学 一种动态人脸识别方法
CN111597894B (zh) * 2020-04-15 2023-09-15 新讯数字科技(杭州)有限公司 一种基于人脸检测技术的人脸库更新方法
SG10202004550UA (en) * 2020-05-15 2021-12-30 Yitu Pte Ltd A method, a device and a computer-readable storage medium for face frontalization
CN112101127B (zh) * 2020-08-21 2024-04-30 深圳数联天下智能科技有限公司 人脸脸型的识别方法、装置、计算设备及计算机存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105654048A (zh) * 2015-12-30 2016-06-08 四川川大智胜软件股份有限公司 一种多视角人脸比对方法
CN107316340A (zh) * 2017-06-28 2017-11-03 河海大学常州校区 一种基于单张照片的快速人脸建模方法
CN107729806A (zh) * 2017-09-05 2018-02-23 西安理工大学 基于三维人脸重建的单视图多姿态人脸识别方法
CN108257210A (zh) * 2018-02-28 2018-07-06 浙江神造科技有限公司 一种通过单张照片生成人脸三维模型的方法
WO2020211347A1 (zh) * 2019-04-16 2020-10-22 平安科技（深圳）有限公司 基于人脸识别的修改图片的方法、装置和计算机设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HU XIAO, LIAO QI-XIN; PENG SHAO-HU: "Video surveillance face recognition by single training sample", JOURNAL OF GUANGZHOU UNIVERSITY(NATURAL SCIENCE EDITION, vol. 14, no. 5, 15 October 2015 (2015-10-15), pages 67 - 70, XP093037003, ISSN: 1671-4229 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115914583A (zh) * 2023-02-28 2023-04-04 中国科学院长春光学精密机械与物理研究所 基于视觉识别的老人监控设备及监控方法
CN115914583B (zh) * 2023-02-28 2023-06-02 中国科学院长春光学精密机械与物理研究所 基于视觉识别的老人监控设备及监控方法

Also Published As

Publication number Publication date
CN113705393A (zh) 2021-11-26

Similar Documents

Publication Publication Date Title
CN108038420B (zh) 一种基于深度视频的人体行为识别方法
JP4939968B2 (ja) 監視画像処理方法、監視システム及び監視画像処理プログラム
CN113011329B (zh) 一种基于多尺度特征金字塔网络及密集人群计数方法
JP4951498B2 (ja) 顔画像認識装置、顔画像認識方法、顔画像認識プログラムおよびそのプログラムを記録した記録媒体
CN106875431B (zh) 具有移动预测的图像追踪方法及扩增实境实现方法
CN108596193B (zh) 一种搭建针对人耳识别的深度学习网络结构的方法及系统
WO2021213158A1 (zh) 一种智能视频会议终端的实时人脸摘要服务的方法及系统
WO2023082882A1 (zh) 一种基于姿态估计的行人摔倒动作识别方法及设备
CN111160291B (zh) 基于深度信息与cnn的人眼检测方法
CN103607554A (zh) 一种基于全自动人脸无缝合成的视频合成方法
WO2018040982A1 (zh) 一种用于增强现实的实时图像叠加方法及装置
CN107248174A (zh) 一种基于tld算法的目标跟踪方法
CN103198488A (zh) Ptz监控摄像机实时姿态快速估算方法
CN101739690A (zh) 多相机协同运动目标检测方法
CN111325051A (zh) 一种基于人脸图像roi选取的人脸识别方法及装置
CN102402784A (zh) 一种基于最近特征线流形学习的人脸图像超分辨率方法
CN104883548A (zh) 监控视频人脸抓取处理方法及其系统
CN113850136A (zh) 基于yolov5与BCNN的车辆朝向识别方法及系统
CN111582036B (zh) 可穿戴设备下基于形状和姿态的跨视角人物识别方法
WO2023019699A1 (zh) 一种基于3d人脸模型的俯角人脸识别方法及系统
JP2014116716A (ja) 追尾装置
CN114612933B (zh) 单目社交距离检测追踪方法
CN111881841B (zh) 一种基于双目视觉的人脸检测识别方法
CN113435367A (zh) 社交距离评估方法、装置及存储介质
WO2021248564A1 (zh) 一种全景大数据应用监测管控系统

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE