CN107067429A - Video editing system and method that face three-dimensional reconstruction and face based on deep learning are replaced - Google Patents
Video editing system and method that face three-dimensional reconstruction and face based on deep learning are replaced Download PDFInfo
- Publication number
- CN107067429A CN107067429A CN201710163711.4A CN201710163711A CN107067429A CN 107067429 A CN107067429 A CN 107067429A CN 201710163711 A CN201710163711 A CN 201710163711A CN 107067429 A CN107067429 A CN 107067429A
- Authority
- CN
- China
- Prior art keywords
- face
- video
- module
- threedimensional model
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Abstract
The video editing system and method replaced the invention discloses a kind of face three-dimensional reconstruction based on deep learning and face, are made up of deep learning module, three-dimensional reconstruction module, video pre-filtering module, two-dimension picture generation module, video editing module；Deep learning module obtains generating the call parameter of threedimensional model by training convolutional neural networks；Three-dimensional reconstruction module obtains corresponding threedimensional model according to human face photo/video of input；Video pre-filtering module is handled target video and then obtains the human face characteristic point corresponding to each frame and expression parameter；The threedimensional model of target face is generated the corresponding two-dimension human face picture of each frame by two-dimension picture generation module；New face is substituted on target video by video editing module, and smoothing processing and illumination condition adjustment.
Description
Technical field：
The invention belongs to computer realm, it is related to a kind of three-dimensional reconstruction and video editing system, it is especially a kind of based on deep
Spend video editing system and method that the face three-dimensional reconstruction and face of study are replaced.
Background technology：
In recent years, the virtual role generation technique of " A Fanda " formula is applied in production of film and TV and electronics trip by increasing
In play.Wherein, face reconstruct is whether accurate particularly important to virtual role.It is even more that can allow virtual to show facial expression in real time
Role is more life-like.Meanwhile, the rise of deep learning make it that insoluble computer vision problem have found new dash forward in the past
Cut.It can be effectively applied on face three-dimensional reconstruction by the convolutional neural networks necessarily trained, so as to substantially reduce pair
The requirement of shooting instrument and input content.
Chinese patent (201510406365.9) discloses a kind of real-time face recognition monitoring system, including monitoring system and
Face identification system, the vision signal of monitoring system collection is compressed by the machine, passes through network inputs to face identification system；People
Face identifying system mainly includes face and catches engine, interference reduction engine, face Modeling engine, face alignment engine, database；
Computer passes through interchanger connection monitoring system；The original comparison photo of photo library storage；Face catches engine and passes through video input
Equipment gather and obtain video or image information in the 2D portraits containing face biological characteristic；Interference reduction engine is to capturing
2D portraits are modified reduction；Face Modeling engine carries out 3D expansion to the 2D portraits for meeting modeling conditions collected and 3D is built
Mould, generates 3D characteristic values；Face alignment engine is generated to the 3D characteristic values of the 2D portraits generation of acquisition with photo in photo library
3D characteristic values are compared, and draw comparison result.The purpose of the patent is recognition of face, although generate threedimensional model, still
The expression shape change of threedimensional model is not accounted for completely, it is impossible to is accomplished grafting of expressing one's feelings, and be cannot act as video editing.
Chinese patent (201310105220.6) discloses a kind of Real Time Face Detection Algorithm Based, is related to natural human-machine interaction skill
Art, it is desirable to provide one kind takes less system resource and quick method for detecting human face.Technical key point includes：Whole frame inspection
Survey step：Whole frame Face datection is carried out to input picture, if being not detected by face information, whole frame people is carried out to next two field picture
Face is detected, until detecting face information；Record the human face region step that whole frame is detected：Record face and be located at the two field picture
Positional information and face rectangle size, obtain human face region；The whole frame is detected with the 1st frame after face information image
Image, the 2nd two field picture, the 3rd two field picture and the 4th two field picture are predicted position detecting step successively；Repeat above steps
The follow-up image of reason.The patent is only simple two-dimension human face detection, generation threedimensional model is had no, more without same face
Different expressions, also can not just accomplish grafting of expressing one's feelings, cannot act as video editing.
As described above, existing technical requirements use additional hardware (such as depth of field camera), or face to replace excessively simple
It is single, do not account for face's threedimensional model and expression factor.And can reach the technology of real-time criteria just more above step
Plus it is deficient.It is adapted to public use and effect face three-dimensional reconstruction true to nature and the video editing techniques of face replacement is almost a piece of
Blank.
The content of the invention：
It is an object of the invention to the shortcoming for overcoming above-mentioned prior art, there is provided the face three-dimensional reconstruction based on deep learning
And the video editing system and method for face replacement.User only needs to shoot a human face photo, it is possible to pass through the present invention's
Deep learning system obtains corresponding threedimensional model, and general video (such as interview class video) can be compiled in real time
Volume, on the face that this threedimensional model is substituted into former video.
The present invention is as follows：1. generated with existing small-sized three-dimensional face database in the case where different virtual opticals is shone
A large amount of virtual photos.2. by these virtual photos come training convolutional neural networks, complete deep learning.3. by target face
Photo is inputted into the neutral net trained, obtains corresponding threedimensional model.4. target video is pre-processed, pass through nerve
Network obtains threedimensional model, and calculates the human face characteristic point and expression parameter of each frame.5. the three-dimensional people that step 3 is generated
Face model adjusts expression parameter frame by frame, and each frame generates corresponding face picture.6. the face picture that step 5 is generated by
Frame is substituted on target video, and be smoothed with illumination condition adjustment, finally give video of changing face true to nature.
The purpose of the present invention is achieved through the following technical solutions：
Video editing system and method that a kind of face three-dimensional reconstruction and face based on deep learning are replaced, by depth
Module, three-dimensional reconstruction module, video pre-filtering module, two-dimension picture generation module, video editing module is practised to constitute.Deep learning
Module obtains generating the call parameter of threedimensional model by training convolutional neural networks；Three-dimensional reconstruction module is according to the people of input
Face photo/video obtains corresponding threedimensional model；Video pre-filtering module is handled target video and then obtains each frame
Corresponding human face characteristic point and expression parameter；The threedimensional model of target face is generated each frame phase by two-dimension picture generation module
Corresponding two-dimension human face picture；New face is substituted on target video by video editing module, and smoothing processing and illumination bar
Part is adjusted.
Described three-dimensional reconstruction and video editing method, in accordance with the following steps：
(1) before user uses the system, deep learning module is by virtual photo come training convolutional neural networks；
(2) user inputs target human face photo and target video in the system of the present invention；
(3) convolutional neural networks that three-dimensional reconstruction module is trained using deep learning module are to human face photo and target
Face in video carries out three-dimensional reconstruction respectively；
(4) video pre-filtering module is handled target video；
(5) threedimensional model of target face is generated the corresponding two-dimension human face figure of each frame by two-dimension picture generation module
Piece；
(6) new face is substituted on target video by video editing module.
The step (1) is as follows：
, it is necessary to just be trained to neutral net before user is using the system, the three-dimensional reconstruction part for after.
(note：This step need to only be carried out once, without repetition training)
A. virtual photo generation：With existing small-sized three-dimensional face database, a large amount of void are generated by converting illumination, angle
Intend photo；
B. neural metwork training：A convolutional neural networks are built, and use three-dimensional face and its corresponding virtual photo
It is trained.
The step (4) is as follows：
For each frame in target video, it is necessary to calculate and generate the characteristic point and expression parameter of face.
A. human face characteristic point is generated：Just can in real time it be marked by 68 characteristic points of face of standard；
B. human face expression parameter is calculated：The threedimensional model generated according to step (3), utilizes Basel faceform
(Basel Face Model) is just calculated the expression parameter corresponding to each frame in video.
The step (5) is as follows：
The characteristic point that the threedimensional model that step (3) is generated is generated with step (4) is compared with parameter, generates frame by frame
New two-dimension picture.Per pictures by following processing：
A. direction and rescaling：Threedimensional model is adjusted with the two-dimension picture in former video by facial feature points
Whole, alignment, makes two faces overlap as far as possible；
B. express one's feelings grafting：Deformation adjustment is carried out to threedimensional model according to the expression parameter in former video；
C. tripleplane：Threedimensional model after alignment, expression adjustment is projected on two-dimension picture, new two are obtained
Dimension picture is used as face's textures.
The step (6) is as follows：
It is substituted into original face on target video frame by frame with new face.
A. it is preliminary to replace：First, original face in video is substituted for the face textures that step (5) is generated；
B. smoothing processing：Face edge is subjected to smooth and color Fuzzy Processing, it is to avoid the distortions such as color change occur and show
As；
C. illumination is adjusted：The new face and original face that direct montage is produced have the situation that illumination is not inconsistent, therefore we
Need to act on original illumination on new face, so as to reach true to nature change face.
Beneficial effects of the present invention：
A. a neutral net is trained by deep learning so that individual face picture can also quickly generate high-precision three
Tie up faceform.
B. expression grafting is realized by the expression parameter of Basel faceform (Basel Face Model), and then carried out
Video editing, reaches real-time and true to nature effect of changing face.
Brief description of the drawings：
Fig. 1 is system schematic of the invention.
Fig. 2 is iteration error process of feedback figure of the invention.
Embodiment：
The present invention is described in further detail below in conjunction with the accompanying drawings：
As shown in figure 1, the present invention is included as deep learning module, three-dimensional reconstruction module, video pre-filtering module, X-Y scheme
Piece generation module, video editing module.
1. before user uses the system, deep learning module is by virtual photo come training convolutional neural networks.This
Training step is carried out on backstage, and is only needed to once.Deep learning needs mass data to be trained, and large-scale face
Database is not present again, therefore we generate a large amount of conjecture face databases firstly the need of by existing toy data base.Tool
Body step includes：
A. virtual photo generation：By existing small-sized three-dimensional face database, by each threedimensional model at different angles
Under the conditions of degree virtual video camera and virtual optical are shone, by the three-dimensional mapping to two dimension, a large amount of virtual photos are generated.It is empty for every group
Intend photo, although they are generated by identical threedimensional model, but are due to the difference of angle, illumination, background, can be considered
Different face 2-dimentional photos, the deep learning for after provides data.
B. neural metwork training：With human face three-dimensional model is obtained it is Basel faceform (Basel in the present invention
Face Model).By adjusting 200 geometric vectors, 84 expression 200 textures of vector sum, all faceforms are equal
Can be by " average face " model evolution.Therefore deep learning is just deployed around the training of these parameters.Build first
One convolutional neural networks ResNet, input is a two-dimension human face photo, and input is a threedimensional model.Used here as
The method of iteration error feedback is trained, as shown in Fig. 2 untill optimization method is restrained.After being trained by mass data
Convolutional neural networks, can generate a high accuracy three-dimensional faceform when reading in a new human face photo.
2. user inputs target human face photo and target video in the system of the present invention.Here we only need a face
Photo, particular/special requirement is had no to background, angle etc..Target video is generally face characteristic more clearly video, and such as scene is acute
Or talk show,
3. the convolutional neural networks that three-dimensional reconstruction module is trained using deep learning module are to human face photo and target
Face in video carries out three-dimensional reconstruction respectively.Human face photo can directly generate three-dimensional mould by the neutral net in step 1
Type.And the face in target video needs first one pictures of automatic interception, threedimensional model is being generated by neutral net.
4. video pre-filtering module is handled target video.For each frame in target video, it is necessary to calculate simultaneously
Generate the characteristic point and expression parameter of face.
A. human face characteristic point is generated：Just can in real time it be marked by 68 characteristic points of face of standard.
B. human face expression parameter is calculated：The threedimensional model generated according to step 3, utilizes Basel faceform (Basel
Face Model) the expression vector parameter corresponding to each frame in video is just calculated.
5. the threedimensional model of target face is generated the corresponding two-dimension human face picture of each frame by two-dimension picture generation module.
The characteristic point that the threedimensional model that step 3 is generated is generated with step 4 is compared with parameter, and new X-Y scheme is generated frame by frame
Piece.Comprised the following steps that for the video of each frame：
A. direction and rescaling：Projection of the threedimensional model in two-dimension picture and the two-dimension picture in former video are carried out
Compare.The size of threedimensional model, crevice projection angle are adjusted by both facial feature points, two faces is weighed as far as possible
Close.
B. express one's feelings grafting：Deformation adjustment is carried out to threedimensional model according to the expression vector parameter in former video, makes both
Expression is consistent.
C. tripleplane：Threedimensional model after alignment, expression adjustment is projected on two-dimension picture, new two are obtained
Dimension picture is used as face's textures.
6. new face is substituted on target video by video editing module.Target video is substituted into frame by frame with new face
On original face.Simple replacement face textures have very strong distortion effect, thus need further processing face light
And texture.Comprise the following steps that：
A. it is preliminary to replace：First, original face in video is substituted for the face textures that step 5 is generated, at this moment
Result have stronger feeling of unreality.
B. smoothing processing：Face edge is subjected to smooth and color Fuzzy Processing, it is to avoid the distortions such as color change occur and show
As.The color and profile of face can be more natural after this step.
C. illumination is adjusted：The new face and original face that direct montage is produced have the situation that illumination is not inconsistent, such as bloom
The position of appearance is different.According to the human face three-dimensional model in former video, we can be calculated by relationship between light and dark (shading)
The illumination condition gone out in video.Same illumination is being placed on new face afterwards, so that both illumination is consistent, reached
True to nature changes face.
The above described is only a preferred embodiment of the present invention, any formal limitation not is made to the present invention, though
So the present invention is disclosed above with preferred embodiment, but is not limited to the present invention, any to be familiar with this professional technology people
Member, without departing from the scope of the present invention, when method and technology contents using the disclosure above make it is a little more
Equivalent embodiment that is dynamic or being modified to equivalent variations, as long as being the content without departing from technical solution of the present invention, according to the present invention's
Any simple modification, equivalent variations and modification that technical spirit is made to above example, still fall within technical solution of the present invention
In the range of.
Claims (6)
1. the video editing system that a kind of face three-dimensional reconstruction and face based on deep learning are replaced, it is characterised in that：By depth
Study module, three-dimensional reconstruction module, video pre-filtering module, two-dimension picture generation module, video editing module is spent to constitute；Depth
Study module obtains generating the call parameter of threedimensional model by training convolutional neural networks；Three-dimensional reconstruction module is according to input
Human face photo/video obtain corresponding threedimensional model；Video pre-filtering module is handled target video and then obtains every
Human face characteristic point and expression parameter corresponding to one frame；Two-dimension picture generation module is each by the threedimensional model generation of target face
The corresponding two-dimension human face picture of frame；New face is substituted on target video by video editing module, and smoothing processing and light
According to condition adjustment.
2. the method based on the video editing system described in claim 1, it is characterised in that in accordance with the following steps：
(1) before user uses the system, deep learning module is by virtual photo come training convolutional neural networks；
(2) user inputs target human face photo and target video in the system of the present invention；
(3) convolutional neural networks that three-dimensional reconstruction module is trained using deep learning module are to human face photo and target video
In face carry out three-dimensional reconstruction respectively；
(4) video pre-filtering module is handled target video；
(5) threedimensional model of target face is generated the corresponding two-dimension human face picture of each frame by two-dimension picture generation module；
(6) new face is substituted on target video by video editing module.
3. the method based on the video editing system described in claim 1, it is characterised in that the step (1) is as follows：
, it is necessary to just be trained to neutral net before user is using the system, the three-dimensional reconstruction part for after；
A. virtual photo generation：With existing small-sized three-dimensional face database, a large amount of virtual photographs are generated by converting illumination, angle
Piece；
B. neural metwork training：A convolutional neural networks are built, and are carried out using three-dimensional face and its corresponding virtual photo
Training.
4. the method based on the video editing system described in claim 1, it is characterised in that the step (4) is as follows：
For each frame in target video, it is necessary to calculate and generate the characteristic point and expression parameter of face；
A. human face characteristic point is generated：Just can in real time it be marked by 68 characteristic points of face of standard；
B. human face expression parameter is calculated：The threedimensional model generated according to step (3), using Basel faceform in video
Expression parameter corresponding to each frame is just calculated.
5. the method based on the video editing system described in claim 1, it is characterised in that the step (5) is as follows：
The characteristic point that the threedimensional model that step (3) is generated is generated with step (4) is compared with parameter, generates frame by frame newly
Two-dimension picture, per pictures by following processing：
A. direction and rescaling：Two-dimension picture in threedimensional model and former video is adjusted by facial feature points, it is right
Together, two faces are made to overlap as far as possible；
B. express one's feelings grafting：Deformation adjustment is carried out to threedimensional model according to the expression parameter in former video；
C. tripleplane：Threedimensional model after alignment, expression adjustment is projected on two-dimension picture, new X-Y scheme is obtained
Piece is used as face's textures.
6. the method based on the video editing system described in claim 1, it is characterised in that the step (6) is as follows：
It is substituted into original face on target video frame by frame with new face；
A. it is preliminary to replace：First, original face in video is substituted for the face textures that step (5) is generated；
B. smoothing processing：Face edge is subjected to smooth and color Fuzzy Processing, it is to avoid the distortion phenomenons such as color change occur；
C. illumination is adjusted：The new face and original face that direct montage is produced have the situation that illumination is not inconsistent, therefore we need
Original illumination is acted on new face, so as to reach true to nature change face.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710163711.4A CN107067429A (en) | 2017-03-17 | 2017-03-17 | Video editing system and method that face three-dimensional reconstruction and face based on deep learning are replaced |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710163711.4A CN107067429A (en) | 2017-03-17 | 2017-03-17 | Video editing system and method that face three-dimensional reconstruction and face based on deep learning are replaced |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107067429A true CN107067429A (en) | 2017-08-18 |
Family
ID=59620655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710163711.4A Pending CN107067429A (en) | 2017-03-17 | 2017-03-17 | Video editing system and method that face three-dimensional reconstruction and face based on deep learning are replaced |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107067429A (en) |
Cited By (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107977439A (en) * | 2017-12-07 | 2018-05-01 | 宁波亿拍客网络科技有限公司 | A kind of facial image base construction method |
| CN108010122A (en) * | 2017-11-14 | 2018-05-08 | 深圳市云之梦科技有限公司 | A kind of human 3d model rebuilds the method and system with measurement |
| CN108122271A (en) * | 2017-12-15 | 2018-06-05 | 南京变量信息科技有限公司 | A kind of description photo automatic generation method and device |
| CN108280803A (en) * | 2018-01-22 | 2018-07-13 | 盎锐（上海）信息科技有限公司 | Image generating method and device based on 3D imagings |
| CN108460812A (en) * | 2018-04-04 | 2018-08-28 | 北京红云智胜科技有限公司 | A kind of expression packet generation system and method based on deep learning |
| GB2560218A (en) * | 2017-03-02 | 2018-09-05 | Adobe Systems Inc | Editing digital images utilizing a neural network with an in-network rendering layer |
| CN108537191A (en) * | 2018-04-17 | 2018-09-14 | 广州云从信息科技有限公司 | A kind of three-dimensional face identification method based on structure light video camera head |
| CN108596062A (en) * | 2018-04-12 | 2018-09-28 | 清华大学 | The real-time high-intensity region method and device of face picture based on deep learning |
| CN108682030A (en) * | 2018-05-21 | 2018-10-19 | 北京微播视界科技有限公司 | Face replacement method, device and computer equipment |
| CN108765537A (en) * | 2018-06-04 | 2018-11-06 | 北京旷视科技有限公司 | A kind of processing method of image, device, electronic equipment and computer-readable medium |
| CN108776983A (en) * | 2018-05-31 | 2018-11-09 | 北京市商汤科技开发有限公司 | Based on the facial reconstruction method and device, equipment, medium, product for rebuilding network |
| CN109151340A (en) * | 2018-08-24 | 2019-01-04 | 太平洋未来科技（深圳）有限公司 | Method for processing video frequency, device and electronic equipment |
| CN109255831A (en) * | 2018-09-21 | 2019-01-22 | 南京大学 | The method that single-view face three-dimensional reconstruction and texture based on multi-task learning generate |
| CN109255843A (en) * | 2018-09-26 | 2019-01-22 | 联想(北京)有限公司 | Three-dimensional rebuilding method, device and augmented reality AR equipment |
| CN109377544A (en) * | 2018-11-30 | 2019-02-22 | 腾讯科技（深圳）有限公司 | A kind of face three-dimensional image generating method, device and readable medium |
| CN109409274A (en) * | 2018-10-18 | 2019-03-01 | 广州云从人工智能技术有限公司 | A kind of facial image transform method being aligned based on face three-dimensional reconstruction and face |
| CN109636907A (en) * | 2018-12-13 | 2019-04-16 | 谷东科技有限公司 | A kind of terrain reconstruction method and system based on AR glasses |
| CN109754364A (en) * | 2019-01-20 | 2019-05-14 | 杭州富阳优信科技有限公司 | A kind of video character face's replacement method based on deep learning |
| CN109934156A (en) * | 2019-03-11 | 2019-06-25 | 重庆科技学院 | A kind of user experience evaluation method and system based on ELMAN neural network |
| CN109992809A (en) * | 2017-12-29 | 2019-07-09 | 深圳市优必选科技有限公司 | A kind of construction method of buildings model, device and storage device |
| CN110084675A (en) * | 2019-04-24 | 2019-08-02 | 文允 | Commodity selling method, the network terminal and the device with store function on a kind of line |
| CN110163814A (en) * | 2019-04-16 | 2019-08-23 | 平安科技（深圳）有限公司 | The method, apparatus and computer equipment of modification picture based on recognition of face |
| CN110267095A (en) * | 2019-05-05 | 2019-09-20 | 平安科技（深圳）有限公司 | Video flowing intercept method, device and storage medium |
| CN110267079A (en) * | 2018-03-30 | 2019-09-20 | 腾讯科技（深圳）有限公司 | The replacement method and device of face in video to be played |
| CN110349232A (en) * | 2019-06-17 | 2019-10-18 | 达闼科技（北京）有限公司 | Generation method, device, storage medium and the electronic equipment of image |
| CN110399858A (en) * | 2019-08-01 | 2019-11-01 | 浙江开奇科技有限公司 | Image treatment method and device for panoramic video image |
| CN110580733A (en) * | 2018-06-08 | 2019-12-17 | 北京搜狗科技发展有限公司 | Data processing method and device and data processing device |
| WO2019237299A1 (en) * | 2018-06-14 | 2019-12-19 | Intel Corporation | 3d facial capture and modification using image and temporal tracking neural networks |
| CN110610456A (en) * | 2019-09-27 | 2019-12-24 | 上海依图网络科技有限公司 | Imaging system and video processing method |
| CN110827342A (en) * | 2019-10-21 | 2020-02-21 | 中国科学院自动化研究所 | Three-dimensional human body model reconstruction method, storage device and control device |
| CN110838042A (en) * | 2019-10-29 | 2020-02-25 | 深圳市掌众信息技术有限公司 | Commodity display method and system |
| CN111429338A (en) * | 2020-03-18 | 2020-07-17 | 百度在线网络技术（北京）有限公司 | Method, apparatus, device and computer-readable storage medium for processing video |
| CN111461959A (en) * | 2020-02-17 | 2020-07-28 | 浙江大学 | Face emotion synthesis method and device |
| CN111741345A (en) * | 2020-06-23 | 2020-10-02 | 南京硅基智能科技有限公司 | Product display method and system based on video face changing |
| CN111861948A (en) * | 2019-04-26 | 2020-10-30 | 北京陌陌信息技术有限公司 | Image processing method, device, equipment and computer storage medium |
| CN111862275A (en) * | 2020-07-24 | 2020-10-30 | 厦门真景科技有限公司 | Video editing method, device and equipment based on 3D reconstruction technology |
| CN112597973A (en) * | 2021-01-29 | 2021-04-02 | 秒影工场(北京)科技有限公司 | High-definition video face alignment method based on convolutional neural network |
| CN112734890A (en) * | 2020-12-22 | 2021-04-30 | 上海影谱科技有限公司 | Human face replacement method and device based on three-dimensional reconstruction |
| CN113269872A (en) * | 2021-06-01 | 2021-08-17 | 广东工业大学 | Synthetic video generation method based on three-dimensional face reconstruction and video key frame optimization |
| CN113689538A (en) * | 2020-05-18 | 2021-11-23 | 北京达佳互联信息技术有限公司 | Video generation method and device, electronic equipment and storage medium |
| CN114022620A (en) * | 2022-01-06 | 2022-02-08 | 武汉大势智慧科技有限公司 | Method and system for eliminating scattered texture in three-dimensional scene reconstruction |
| CN116433812A (en) * | 2023-06-08 | 2023-07-14 | 海马云（天津）信息技术有限公司 | Method and device for generating virtual character by using 2D face picture |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103093490A (en) * | 2013-02-02 | 2013-05-08 | 浙江大学 | Real-time facial animation method based on single video camera |
| CN103971394A (en) * | 2014-05-21 | 2014-08-06 | 中国科学院苏州纳米技术与纳米仿生研究所 | Facial animation synthesizing method |
| CN105069746A (en) * | 2015-08-23 | 2015-11-18 | 杭州欣禾圣世科技有限公司 | Video real-time human face substitution method and system based on partial affine and color transfer technology |
-
2017
- 2017-03-17 CN CN201710163711.4A patent/CN107067429A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103093490A (en) * | 2013-02-02 | 2013-05-08 | 浙江大学 | Real-time facial animation method based on single video camera |
| CN103971394A (en) * | 2014-05-21 | 2014-08-06 | 中国科学院苏州纳米技术与纳米仿生研究所 | Facial animation synthesizing method |
| CN105069746A (en) * | 2015-08-23 | 2015-11-18 | 杭州欣禾圣世科技有限公司 | Video real-time human face substitution method and system based on partial affine and color transfer technology |
Non-Patent Citations (3)
| Title |
|---|
| LI ANNAN等: ""Recovering 3D facial shape via coupled 2D/3D space learning"", 《IEEE INTERNATIONAL CONFERENCE ON AUTOMATIC FACE AND GESTURE RECOGNITION》 * |
| XIAOGUANG LU等: ""Face Recognition with 3D Model-Based Synthesis"", 《LECTURE NOTES IN COMPUTER SCIENCE》 * |
| 徐成华等: ""三维人脸建模与应用"", 《中国图象图形学报》 * |
Cited By (60)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2560218A (en) * | 2017-03-02 | 2018-09-05 | Adobe Systems Inc | Editing digital images utilizing a neural network with an in-network rendering layer |
| US10430978B2 (en) | 2017-03-02 | 2019-10-01 | Adobe Inc. | Editing digital images utilizing a neural network with an in-network rendering layer |
| GB2560218B (en) * | 2017-03-02 | 2020-01-01 | Adobe Inc | Editing digital images utilizing a neural network with an in-network rendering layer |
| CN108010122A (en) * | 2017-11-14 | 2018-05-08 | 深圳市云之梦科技有限公司 | A kind of human 3d model rebuilds the method and system with measurement |
| CN108010122B (en) * | 2017-11-14 | 2022-02-11 | 深圳市云之梦科技有限公司 | Method and system for reconstructing and measuring three-dimensional model of human body |
| CN107977439A (en) * | 2017-12-07 | 2018-05-01 | 宁波亿拍客网络科技有限公司 | A kind of facial image base construction method |
| CN108122271A (en) * | 2017-12-15 | 2018-06-05 | 南京变量信息科技有限公司 | A kind of description photo automatic generation method and device |
| CN109992809B (en) * | 2017-12-29 | 2023-03-10 | 深圳市优必选科技有限公司 | Building model construction method and device and storage device |
| CN109992809A (en) * | 2017-12-29 | 2019-07-09 | 深圳市优必选科技有限公司 | A kind of construction method of buildings model, device and storage device |
| CN108280803A (en) * | 2018-01-22 | 2018-07-13 | 盎锐（上海）信息科技有限公司 | Image generating method and device based on 3D imagings |
| CN110267079A (en) * | 2018-03-30 | 2019-09-20 | 腾讯科技（深圳）有限公司 | The replacement method and device of face in video to be played |
| CN108460812A (en) * | 2018-04-04 | 2018-08-28 | 北京红云智胜科技有限公司 | A kind of expression packet generation system and method based on deep learning |
| CN108596062A (en) * | 2018-04-12 | 2018-09-28 | 清华大学 | The real-time high-intensity region method and device of face picture based on deep learning |
| CN108537191B (en) * | 2018-04-17 | 2020-11-20 | 云从科技集团股份有限公司 | Three-dimensional face recognition method based on structured light camera |
| CN108537191A (en) * | 2018-04-17 | 2018-09-14 | 广州云从信息科技有限公司 | A kind of three-dimensional face identification method based on structure light video camera head |
| CN108682030A (en) * | 2018-05-21 | 2018-10-19 | 北京微播视界科技有限公司 | Face replacement method, device and computer equipment |
| CN108776983A (en) * | 2018-05-31 | 2018-11-09 | 北京市商汤科技开发有限公司 | Based on the facial reconstruction method and device, equipment, medium, product for rebuilding network |
| CN108765537A (en) * | 2018-06-04 | 2018-11-06 | 北京旷视科技有限公司 | A kind of processing method of image, device, electronic equipment and computer-readable medium |
| CN110580733A (en) * | 2018-06-08 | 2019-12-17 | 北京搜狗科技发展有限公司 | Data processing method and device and data processing device |
| US11308675B2 (en) | 2018-06-14 | 2022-04-19 | Intel Corporation | 3D facial capture and modification using image and temporal tracking neural networks |
| WO2019237299A1 (en) * | 2018-06-14 | 2019-12-19 | Intel Corporation | 3d facial capture and modification using image and temporal tracking neural networks |
| CN109151340A (en) * | 2018-08-24 | 2019-01-04 | 太平洋未来科技（深圳）有限公司 | Method for processing video frequency, device and electronic equipment |
| WO2020037679A1 (en) * | 2018-08-24 | 2020-02-27 | 太平洋未来科技（深圳）有限公司 | Video processing method and apparatus, and electronic device |
| CN109255831B (en) * | 2018-09-21 | 2020-06-12 | 南京大学 | Single-view face three-dimensional reconstruction and texture generation method based on multi-task learning |
| CN109255831A (en) * | 2018-09-21 | 2019-01-22 | 南京大学 | The method that single-view face three-dimensional reconstruction and texture based on multi-task learning generate |
| CN109255843A (en) * | 2018-09-26 | 2019-01-22 | 联想(北京)有限公司 | Three-dimensional rebuilding method, device and augmented reality AR equipment |
| CN109409274A (en) * | 2018-10-18 | 2019-03-01 | 广州云从人工智能技术有限公司 | A kind of facial image transform method being aligned based on face three-dimensional reconstruction and face |
| CN109377544A (en) * | 2018-11-30 | 2019-02-22 | 腾讯科技（深圳）有限公司 | A kind of face three-dimensional image generating method, device and readable medium |
| CN109377544B (en) * | 2018-11-30 | 2022-12-23 | 腾讯科技（深圳）有限公司 | Human face three-dimensional image generation method and device and readable medium |
| CN109636907A (en) * | 2018-12-13 | 2019-04-16 | 谷东科技有限公司 | A kind of terrain reconstruction method and system based on AR glasses |
| CN109754364A (en) * | 2019-01-20 | 2019-05-14 | 杭州富阳优信科技有限公司 | A kind of video character face's replacement method based on deep learning |
| CN109934156A (en) * | 2019-03-11 | 2019-06-25 | 重庆科技学院 | A kind of user experience evaluation method and system based on ELMAN neural network |
| CN110163814A (en) * | 2019-04-16 | 2019-08-23 | 平安科技（深圳）有限公司 | The method, apparatus and computer equipment of modification picture based on recognition of face |
| WO2020211347A1 (en) * | 2019-04-16 | 2020-10-22 | 平安科技（深圳）有限公司 | Facial recognition-based image modification method and apparatus, and computer device |
| CN110084675A (en) * | 2019-04-24 | 2019-08-02 | 文允 | Commodity selling method, the network terminal and the device with store function on a kind of line |
| CN111861948B (en) * | 2019-04-26 | 2024-04-09 | 北京陌陌信息技术有限公司 | Image processing method, device, equipment and computer storage medium |
| CN111861948A (en) * | 2019-04-26 | 2020-10-30 | 北京陌陌信息技术有限公司 | Image processing method, device, equipment and computer storage medium |
| CN110267095A (en) * | 2019-05-05 | 2019-09-20 | 平安科技（深圳）有限公司 | Video flowing intercept method, device and storage medium |
| CN110349232B (en) * | 2019-06-17 | 2023-04-07 | 达闼科技（北京）有限公司 | Image generation method and device, storage medium and electronic equipment |
| CN110349232A (en) * | 2019-06-17 | 2019-10-18 | 达闼科技（北京）有限公司 | Generation method, device, storage medium and the electronic equipment of image |
| CN110399858A (en) * | 2019-08-01 | 2019-11-01 | 浙江开奇科技有限公司 | Image treatment method and device for panoramic video image |
| CN110610456A (en) * | 2019-09-27 | 2019-12-24 | 上海依图网络科技有限公司 | Imaging system and video processing method |
| CN110827342A (en) * | 2019-10-21 | 2020-02-21 | 中国科学院自动化研究所 | Three-dimensional human body model reconstruction method, storage device and control device |
| CN110838042B (en) * | 2019-10-29 | 2022-08-30 | 深圳市掌众信息技术有限公司 | Commodity display method and system |
| CN110838042A (en) * | 2019-10-29 | 2020-02-25 | 深圳市掌众信息技术有限公司 | Commodity display method and system |
| CN111461959B (en) * | 2020-02-17 | 2023-04-25 | 浙江大学 | Face emotion synthesis method and device |
| CN111461959A (en) * | 2020-02-17 | 2020-07-28 | 浙江大学 | Face emotion synthesis method and device |
| CN111429338B (en) * | 2020-03-18 | 2023-08-01 | 百度在线网络技术（北京）有限公司 | Method, apparatus, device and computer readable storage medium for processing video |
| CN111429338A (en) * | 2020-03-18 | 2020-07-17 | 百度在线网络技术（北京）有限公司 | Method, apparatus, device and computer-readable storage medium for processing video |
| WO2021232690A1 (en) * | 2020-05-18 | 2021-11-25 | 北京达佳互联信息技术有限公司 | Video generating method and apparatus, electronic device, and storage medium |
| CN113689538A (en) * | 2020-05-18 | 2021-11-23 | 北京达佳互联信息技术有限公司 | Video generation method and device, electronic equipment and storage medium |
| CN111741345A (en) * | 2020-06-23 | 2020-10-02 | 南京硅基智能科技有限公司 | Product display method and system based on video face changing |
| CN111862275A (en) * | 2020-07-24 | 2020-10-30 | 厦门真景科技有限公司 | Video editing method, device and equipment based on 3D reconstruction technology |
| CN112734890B (en) * | 2020-12-22 | 2023-11-10 | 上海影谱科技有限公司 | Face replacement method and device based on three-dimensional reconstruction |
| CN112734890A (en) * | 2020-12-22 | 2021-04-30 | 上海影谱科技有限公司 | Human face replacement method and device based on three-dimensional reconstruction |
| CN112597973A (en) * | 2021-01-29 | 2021-04-02 | 秒影工场(北京)科技有限公司 | High-definition video face alignment method based on convolutional neural network |
| CN113269872A (en) * | 2021-06-01 | 2021-08-17 | 广东工业大学 | Synthetic video generation method based on three-dimensional face reconstruction and video key frame optimization |
| CN114022620A (en) * | 2022-01-06 | 2022-02-08 | 武汉大势智慧科技有限公司 | Method and system for eliminating scattered texture in three-dimensional scene reconstruction |
| CN116433812A (en) * | 2023-06-08 | 2023-07-14 | 海马云（天津）信息技术有限公司 | Method and device for generating virtual character by using 2D face picture |
| CN116433812B (en) * | 2023-06-08 | 2023-08-25 | 海马云（天津）信息技术有限公司 | Method and device for generating virtual character by using 2D face picture |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107067429A (en) | Video editing system and method that face three-dimensional reconstruction and face based on deep learning are replaced | |
| CN110363858B (en) | Three-dimensional face reconstruction method and system | |
| CN108596024B (en) | Portrait generation method based on face structure information | |
| Li et al. | Realtime facial animation with on-the-fly correctives. | |
| Suwajanakorn et al. | What makes tom hanks look like tom hanks | |
| Wang et al. | High resolution acquisition, learning and transfer of dynamic 3‐D facial expressions | |
| CN108734776B (en) | Speckle-based three-dimensional face reconstruction method and equipment | |
| CN107204010A (en) | A kind of monocular image depth estimation method and system | |
| JP5206366B2 (en) | 3D data creation device | |
| CN104539928B (en) | A kind of grating stereo printing image combining method | |
| CN108053437B (en) | Three-dimensional model obtaining method and device based on posture | |
| CN105427385A (en) | High-fidelity face three-dimensional reconstruction method based on multilevel deformation model | |
| CN109816784B (en) | Method and system for three-dimensional reconstruction of human body and medium | |
| CN103198508A (en) | Human face expression animation generation method | |
| CN110827312B (en) | Learning method based on cooperative visual attention neural network | |
| CN111127668B (en) | Character model generation method and device, electronic equipment and storage medium | |
| CN104599317A (en) | Mobile terminal and method for achieving 3D (three-dimensional) scanning modeling function | |
| CN107578469A (en) | A kind of 3D human body modeling methods and device based on single photo | |
| CN109325994B (en) | Method for enhancing data based on three-dimensional face | |
| CN108615256A (en) | A kind of face three-dimensional rebuilding method and device | |
| WO2021063271A1 (en) | Human body model reconstruction method and reconstruction system, and storage medium | |
| Ye et al. | 3d morphable face model for face animation | |
| CN111105451B (en) | Driving scene binocular depth estimation method for overcoming occlusion effect | |
| CN111127642A (en) | Human face three-dimensional reconstruction method | |
| Lee et al. | High-quality depth estimation using an exemplar 3d model for stereo conversion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170818 |
|
| RJ01 | Rejection of invention patent application after publication |