CN109409274B - Face image transformation method based on face three-dimensional reconstruction and face alignment - Google Patents
Face image transformation method based on face three-dimensional reconstruction and face alignment Download PDFInfo
- Publication number
- CN109409274B CN109409274B CN201811214776.8A CN201811214776A CN109409274B CN 109409274 B CN109409274 B CN 109409274B CN 201811214776 A CN201811214776 A CN 201811214776A CN 109409274 B CN109409274 B CN 109409274B
- Authority
- CN
- China
- Prior art keywords
- face
- reference picture
- dimensional
- image
- alignment feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Processing Or Creating Images (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Image Generation (AREA)
Abstract
A face image transformation method based on face three-dimensional reconstruction and face alignment adopts the following steps, step 1: the three-dimensional image acquisition device acquires a three-dimensional image of a human face as an input image and transmits the input image to the processing module; step 2: extracting three-dimensional characteristics of the input image by a face recognition module; the three-dimensional features of the input image comprise coordinate values of a face feature point set; storing a standard face alignment feature set in a standard face database, wherein the face alignment feature set is composed of face alignment feature points, and each face alignment feature point is semantic information of a corresponding point on a face; and step 3: the human face recognition system is provided with a reference picture database, and a processing module extracts a reference picture from the reference picture database and transmits the reference picture to the human face recognition module. The human face image transformation method based on human face three-dimensional reconstruction and human face alignment can adapt to various postures and expressions, and effectively maps large posture and expression changes which cannot be processed by the traditional method. The method can also process all human faces by using a universal model, and overcomes the defects of the GAN method.
Description
Technical Field
The invention relates to the field of image processing, in particular to a human face image transformation method based on human face three-dimensional reconstruction and human face alignment.
Background
With the rise of artificial intelligence trend, especially the large-scale research and application of the deep learning technology in the image field, the reconstruction of the three-dimensional structure of the human face and the alignment of the human face have better and better effects, for example, the accuracy and the speed of pr _ net proposed recently all reach practical requirements.
In the field of entertainment and games, there are many demands for changing one face to another image or editing the face image, such as synthetic ancient style or national photos popular with WeChat friends, synthetic admission notice books, etc.; in the process of post-processing of art photos, the expression of the face needs to be finely adjusted; in a game or animation, a human face needs to be attached to a model or a human face corresponding to an actor.
The traditional face fitting is based on key points, triangulation is carried out according to the key points, and then each triangle is subjected to mapping transformation. This approach can only deal with frontal faces, and is not applicable to situations where a human face has a large pose angle.
With the rise of deep learning technology, some GAN-based face mapping technologies have appeared. For example, the technology based on GAN can map faces with various postures and expressions, and can achieve good effect in a specific scene. But what is learned based on GAN is not the way in which the map is attached, but rather the features of the particular face that is learned. This requires that the corresponding model be trained for each specific face, and a generic model cannot be used, which is difficult to implement in many scenarios.
Disclosure of Invention
The invention provides a face image transformation method based on face three-dimensional reconstruction and face alignment, aiming at the defects of the prior art, and the specific technical scheme is as follows:
a face image transformation method based on face three-dimensional reconstruction and face alignment is characterized in that:
the following steps are adopted for the preparation of the anti-cancer medicine,
step 1: the three-dimensional image acquisition device acquires a three-dimensional image of a human face as an input image and transmits the input image to the processing module;
step 2: extracting three-dimensional characteristics of the input image by a face recognition module;
the three-dimensional features of the input image comprise coordinate values of a face feature point set;
storing a standard face alignment feature set in a standard face database, wherein the face alignment feature set is composed of face alignment feature points, and each face alignment feature point is semantic information of a corresponding point on a face;
and step 3: a reference picture database is arranged, and a processing module extracts a reference picture from the reference picture database and transmits the reference picture to a face recognition module;
the face recognition module extracts the three-dimensional features and the face alignment feature set of the reference picture and stores the three-dimensional features and the face alignment feature set of the reference picture in a temporary database;
and 4, step 4: the processing module maps the input image and maps the texture of the input image to the corresponding position of a reference image;
specifically, a processing module extracts three-dimensional features of an input picture to obtain textures of corresponding positions;
acquiring a corresponding three-dimensional coordinate point set of a reference picture according to a face alignment feature set of an input picture;
and 5: and the processing module assigns the texture acquired from the input picture to the corresponding coordinate in the three-dimensional coordinate point set of the reference picture in an interpolation mode.
Step 6: the processing module fuses the face picture into the reference picture through Poisson and replaces the face in the reference picture.
Further: and step 6, the processing module fuses the face picture into the reference picture through an alpha channel.
The invention has the beneficial effects that: the human face image transformation method based on human face three-dimensional reconstruction and human face alignment can adapt to various postures and expressions, and effectively maps large posture and expression changes which cannot be processed by the traditional method. The method can also process all human faces by using a universal model, and overcomes the defects of the GAN method. In addition, the method can also achieve the effect of real-time operation while ensuring the precision, and can be effectively applied to actual scenes.
Drawings
FIG. 1 is a flow chart of the operation of the present invention.
Detailed Description
The following detailed description of the preferred embodiments of the present invention, taken in conjunction with the accompanying drawings, will make the advantages and features of the invention easier to understand by those skilled in the art, and thus will clearly and clearly define the scope of the invention.
As shown in fig. 1: a human face image transformation method based on human face three-dimensional reconstruction and human face alignment,
the following steps are adopted for the preparation of the anti-cancer medicine,
step 1: the three-dimensional image acquisition device acquires a three-dimensional image of a human face as an input image and transmits the input image to the processing module;
step 2: extracting three-dimensional characteristics of the input image by a face recognition module;
the three-dimensional features of the input image comprise coordinate values of a face feature point set;
storing a standard face alignment feature set in a standard face database, wherein the face alignment feature set is composed of face alignment feature points, and each face alignment feature point is semantic information of a corresponding point on a face;
and step 3: a reference picture database is arranged, and a processing module extracts a reference picture from the reference picture database and transmits the reference picture to a face recognition module;
the face recognition module extracts the three-dimensional features and the face alignment feature set of the reference picture and stores the three-dimensional features and the face alignment feature set of the reference picture in a temporary database;
and 4, step 4: the processing module maps the input image and maps the texture of the input image to the corresponding position of a reference image;
specifically, a processing module extracts three-dimensional features of an input picture to obtain textures of corresponding positions;
acquiring a corresponding three-dimensional coordinate point set of a reference picture according to a face alignment feature set of an input picture;
and 5: and the processing module assigns the texture acquired from the input picture to the corresponding coordinate in the three-dimensional coordinate point set of the reference picture in an interpolation mode.
Step 6: the processing module fuses the face picture into the reference picture through an alpha channel or Poisson to replace the face in the reference picture.
Claims (2)
1. A face image transformation method based on face three-dimensional reconstruction and face alignment is characterized in that:
the following steps are adopted for the preparation of the anti-cancer medicine,
step 1: the three-dimensional image acquisition device acquires a three-dimensional image of a human face as an input image and transmits the input image to the processing module;
step 2: extracting three-dimensional characteristics of the input image by a face recognition module;
the three-dimensional features of the input image comprise coordinate values of a face feature point set;
storing a standard face alignment feature set in a standard face database, wherein the face alignment feature set is composed of face alignment feature points, and each face alignment feature point is semantic information of a corresponding point on a face;
and step 3: a reference picture database is arranged, and a processing module extracts a reference picture from the reference picture database and transmits the reference picture to a face recognition module;
the face recognition module extracts the three-dimensional features and the face alignment feature set of the reference picture and stores the three-dimensional features and the face alignment feature set of the reference picture in a temporary database;
and 4, step 4: the processing module maps the input image and maps the texture of the input image to the corresponding position of a reference image;
specifically, a processing module extracts three-dimensional features of an input picture to obtain textures of corresponding positions;
acquiring a corresponding three-dimensional coordinate point set of a reference picture according to a face alignment feature set of an input picture;
and 5: the processing module assigns the texture acquired from the input picture to the corresponding coordinate in the three-dimensional coordinate point set of the reference picture in an interpolation mode;
step 6: the processing module fuses the face picture into the reference picture through Poisson and replaces the face in the reference picture.
2. The method for transforming the human face image based on the human face three-dimensional reconstruction and the human face alignment as claimed in claim 1, wherein: and step 6, the processing module fuses the face picture into the reference picture through an alpha channel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811214776.8A CN109409274B (en) | 2018-10-18 | 2018-10-18 | Face image transformation method based on face three-dimensional reconstruction and face alignment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811214776.8A CN109409274B (en) | 2018-10-18 | 2018-10-18 | Face image transformation method based on face three-dimensional reconstruction and face alignment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109409274A CN109409274A (en) | 2019-03-01 |
| CN109409274B true CN109409274B (en) | 2020-09-04 |
Family
ID=65468396
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811214776.8A Active CN109409274B (en) | 2018-10-18 | 2018-10-18 | Face image transformation method based on face three-dimensional reconstruction and face alignment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109409274B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111091151B (en) * | 2019-12-17 | 2021-11-05 | 大连理工大学 | Construction method of generation countermeasure network for target detection data enhancement |
| CN111275057B (en) * | 2020-02-13 | 2023-06-20 | 腾讯科技（深圳）有限公司 | Image processing method, device and equipment |
| CN111738087B (en) * | 2020-05-25 | 2023-07-25 | 完美世界(北京)软件科技发展有限公司 | Method and device for generating face model of game character |
| CN112330529A (en) * | 2020-11-03 | 2021-02-05 | 上海镱可思多媒体科技有限公司 | Dlid-based face aging method, system and terminal |
| CN112734890B (en) * | 2020-12-22 | 2023-11-10 | 上海影谱科技有限公司 | Face replacement method and device based on three-dimensional reconstruction |
| CN112954205A (en) * | 2021-02-04 | 2021-06-11 | 重庆第二师范学院 | Image acquisition device applied to pedestrian re-identification system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6081275A (en) * | 1997-01-10 | 2000-06-27 | Sharp Kabushiki Kaisha | Image compositing apparatus |
| CN104157010A (en) * | 2014-08-29 | 2014-11-19 | 厦门幻世网络科技有限公司 | 3D human face reconstruction method and device |
| CN106204750A (en) * | 2016-07-11 | 2016-12-07 | 厦门幻世网络科技有限公司 | A kind of method and device based on 3D source model editor's 3D object module |
| CN107274493A (en) * | 2017-06-28 | 2017-10-20 | 河海大学常州校区 | A kind of three-dimensional examination hair style facial reconstruction method based on mobile platform |
| CN107945285A (en) * | 2017-10-11 | 2018-04-20 | 浙江慧脑信息科技有限公司 | A kind of threedimensional model is exchanged cards containing all personal details and become sworn brothers figure and deformation method |
| CN108447017A (en) * | 2018-05-31 | 2018-08-24 | Oppo广东移动通信有限公司 | Face virtual face-lifting method and device |
| CN108648142A (en) * | 2018-05-21 | 2018-10-12 | 北京微播视界科技有限公司 | Image processing method and device |
Family Cites Families (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6002782A (en) * | 1997-11-12 | 1999-12-14 | Unisys Corporation | System and method for recognizing a 3-D object by generating a 2-D image of the object from a transformed 3-D model |
| US7663628B2 (en) * | 2002-01-22 | 2010-02-16 | Gizmoz Israel 2002 Ltd. | Apparatus and method for efficient animation of believable speaking 3D characters in real time |
| JP5463866B2 (en) * | 2009-11-16 | 2014-04-09 | ソニー株式会社 | Image processing apparatus, image processing method, and program |
| CN102103756B (en) * | 2009-12-18 | 2012-10-03 | 华为技术有限公司 | Comic exaggeration method, device and system for human face digital image supporting position deflection |
| US8818131B2 (en) * | 2010-08-20 | 2014-08-26 | Adobe Systems Incorporated | Methods and apparatus for facial feature replacement |
| KR101333836B1 (en) * | 2012-02-28 | 2013-11-29 | 가톨릭대학교 산학협력단 | 3d facial pose and expression estimating method using aam and estimated depth information |
| CN104637078B (en) * | 2013-11-14 | 2017-12-15 | 腾讯科技（深圳）有限公司 | A kind of image processing method and device |
| US20150172637A1 (en) * | 2013-12-12 | 2015-06-18 | Electronics And Telecommunications Research Institute | Apparatus and method for generating three-dimensional output data |
| US9898836B2 (en) * | 2015-02-06 | 2018-02-20 | Ming Chuan University | Method for automatic video face replacement by using a 2D face image to estimate a 3D vector angle of the face image |
| CN104778712B (en) * | 2015-04-27 | 2018-05-01 | 厦门美图之家科技有限公司 | A kind of face chart pasting method and system based on affine transformation |
| CN106067190B (en) * | 2016-05-27 | 2019-04-30 | 俞怡斐 | A kind of generation of fast face threedimensional model and transform method based on single image |
| US10275909B2 (en) * | 2016-11-23 | 2019-04-30 | 3Dsystems, Inc. | Systems and methods for an integrated system for visualizing, simulating, modifying and 3D printing 3D objects |
| CN107067429A (en) * | 2017-03-17 | 2017-08-18 | 徐迪 | Video editing system and method that face three-dimensional reconstruction and face based on deep learning are replaced |
| CN107316340B (en) * | 2017-06-28 | 2020-06-19 | 河海大学常州校区 | Rapid face modeling method based on single photo |
| CN108257084B (en) * | 2018-02-12 | 2021-08-24 | 北京中视广信科技有限公司 | Lightweight face automatic makeup method based on mobile terminal |
| CN108492344A (en) * | 2018-03-30 | 2018-09-04 | 中国科学院半导体研究所 | A kind of portrait-cartoon generation method |
| CN108665555A (en) * | 2018-05-15 | 2018-10-16 | 华中师范大学 | A kind of autism interfering system incorporating real person's image |
-
2018
- 2018-10-18 CN CN201811214776.8A patent/CN109409274B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6081275A (en) * | 1997-01-10 | 2000-06-27 | Sharp Kabushiki Kaisha | Image compositing apparatus |
| CN104157010A (en) * | 2014-08-29 | 2014-11-19 | 厦门幻世网络科技有限公司 | 3D human face reconstruction method and device |
| CN106204750A (en) * | 2016-07-11 | 2016-12-07 | 厦门幻世网络科技有限公司 | A kind of method and device based on 3D source model editor's 3D object module |
| CN107274493A (en) * | 2017-06-28 | 2017-10-20 | 河海大学常州校区 | A kind of three-dimensional examination hair style facial reconstruction method based on mobile platform |
| CN107945285A (en) * | 2017-10-11 | 2018-04-20 | 浙江慧脑信息科技有限公司 | A kind of threedimensional model is exchanged cards containing all personal details and become sworn brothers figure and deformation method |
| CN108648142A (en) * | 2018-05-21 | 2018-10-12 | 北京微播视界科技有限公司 | Image processing method and device |
| CN108447017A (en) * | 2018-05-31 | 2018-08-24 | Oppo广东移动通信有限公司 | Face virtual face-lifting method and device |
Non-Patent Citations (1)
| Title |
|---|
| 自动三维人脸迁移算法研究;陆炎;《中国优秀硕士学位论文全文数据库 信息科技辑》;20180615(第06期);第13,29-46页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109409274A (en) | 2019-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109409274B (en) | Face image transformation method based on face three-dimensional reconstruction and face alignment | |
| CN111028330B (en) | Three-dimensional expression base generation method, device, equipment and storage medium | |
| CN109191548A (en) | Animation method, device, equipment and storage medium | |
| CN104123545B (en) | A kind of real-time human facial feature extraction and expression recognition method | |
| CN102509333B (en) | Action-capture-data-driving-based two-dimensional cartoon expression animation production method | |
| CN111160264B (en) | Cartoon character identity recognition method based on generation countermeasure network | |
| CN103198508A (en) | Human face expression animation generation method | |
| CN102567716B (en) | Face synthetic system and implementation method | |
| CN102157007A (en) | Performance-driven method and device for producing face animation | |
| CN109035415B (en) | Virtual model processing method, device, equipment and computer readable storage medium | |
| CN105957129B (en) | A kind of video display animation method based on voice driven and image recognition | |
| CN104915978A (en) | Realistic animation generation method based on Kinect | |
| CN103942843A (en) | Fairway and ship three-dimensional model dynamic presenting method based on video | |
| CN111638784B (en) | Facial expression interaction method, interaction device and computer storage medium | |
| US11887232B2 (en) | Enhanced system for generation of facial models and animation | |
| US20220398795A1 (en) | Enhanced system for generation of facial models and animation | |
| CN104268921A (en) | 3D face expression control method and system | |
| CN110503707A (en) | A kind of true man's motion capture real-time animation system and method | |
| CN109064548B (en) | Video generation method, device, equipment and storage medium | |
| CN100487732C (en) | Method for generating cartoon portrait based on photo of human face | |
| CN114723888A (en) | Three-dimensional hair model generation method, device, equipment, storage medium and product | |
| JP6052533B2 (en) | Feature amount extraction apparatus and feature amount extraction method | |
| CN115797851A (en) | Animation video processing method and system | |
| Zhang et al. | Activity object detection based on improved faster R-CNN | |
| CN103093211A (en) | Human motion tracking method based on deep nuclear information image feature |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200814 Address after: No.99, Jingrong South 3rd Street, Jiancha street, Tianfu New District, Chengdu 610000 China (Sichuan) pilot Free Trade Zone, Chengdu Applicant after: Sichuan Yuncong Tianfu Artificial Intelligence Technology Co., Ltd Address before: 511457 X1301-G4169, No. 106 Fengze East Road, Nansha District, Guangzhou City, Guangdong Province Applicant before: GUANGZHOU CLOUDWALK ARTIFICIAL INTELLIGENCE TECHNOLOGY Co.,Ltd. |