CN106611426A - 3D (Three-dimensional) facial animation real-time capture technology based on depth camera - Google Patents
3D (Three-dimensional) facial animation real-time capture technology based on depth camera Download PDFInfo
- Publication number
- CN106611426A CN106611426A CN201510665845.7A CN201510665845A CN106611426A CN 106611426 A CN106611426 A CN 106611426A CN 201510665845 A CN201510665845 A CN 201510665845A CN 106611426 A CN106611426 A CN 106611426A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- depth
- facial
- helmet
- control chip
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/20—3D [Three Dimensional] animation
- G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
- G06T2207/10021—Stereoscopic video; Stereoscopic image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Abstract
The invention provides a 3D (Three-dimensional) facial animation real-time capture technology based on a depth camera. The technology comprises a depth scanner, a shape maintenance map, monitoring equipment, a control chip and a helmet, wherein the depth scanner is arranged in the helmet and is a basic geometric model which carries out high-quality calculation on mapping between two surfaces, and continuous recording is carried out in a way of optical grating diffraction; the depth scanner is connected with the control chip, wherein the control chip is remotely connected with the monitoring equipment; and the monitoring equipment carries out synchronous conversion on a signal on the basis of video card output. A facial expression can be captured in real time, real-time facial capture and limb capture can be simultaneously carried out, the quality of the animation can be greatly improved, waste can be reduced on an aspect of manufacture processes, and a manufacture period is greatly shortened to generate a profound influence on the animation industry.
Description
Technical field：
A kind of FA Facial Animation capturing technology of present invention design, more particularly to it is a kind of based on depth camera 3D face
Animation real-time capture technology.
Background technology：
At present domestic FA Facial Animation is all individually caught.Due to reasons such as resolution ratio of camera head, lead
Having caused motion capture and face to catch can not be while carries out, this is generated very in the Making programme of whole animation
Many problems.First, the data of limb action and the data of face action are urged apart, and final animation is at both
Synthesis on can not reach real-time synchronization, this will to the later stage of animation teacher adjustment work bring difficulty.Secondly,
In the practical operation of motion capture, the burden of performer.The action of limbs of performer Yao Xianyan, so
After reuse another set of facial equipment facial expression, performer recalls again limb action at that time to perform facial table
Feelings, the serious deviation of this asynchronous animation appearance that may make we have seen that.Traditional facial capturing technology
It is the expression animation for first catching performer, is preserved into a motion capture file to supply other three-dimensional softwares
This motion capture data is called, last animation is regenerated.It is to can't see facial expression most during this
Effect is made eventually, it is necessary to just can see in three-dimensional software animation when the data of motion capture are imported to
Effect.
The content of the invention：
The problem that the present invention is solved is to allow face to catch to catch while carrying out, the technical scheme for being adopted with limbs
It is, including depth scan instrument, guarantor's type map, monitor, control chip and the helmet that inner helmet is provided with
Depth scan instrument, it is between two surfaces individual to be mapped into row high-quality to calculate that described depth scan instrument is
Basic geometric model, is continuously recorded with the mode of optical grating diffraction, described depth scan instrument connection control
Coremaking piece, described control chip, described control chip remotely connects monitor, and described supervision sets
Standby exported based on video card, conversion is synchronized to signal, it is ensured that complete by the mapping relations of four screens
Three-dimensional premonitoring, is respectively left eye monitor, and right eye monitor, the operation interface and solid of computer software is pre-
Prison screen；Described guarantor's type map is the map that angle is not distorted, within depth scan.
Described depth scan instrument is go pro scanneies.
Beneficial effects of the present invention：Not only can in real time catch facial expression, and can real-time face catch with
Limbs catch to be carried out simultaneously, and this belongs in the world leading technology, and the Technique Popularizing to each animation makes
In company, the quality that can not only make animation is greatly improved, and also waste can be reduced on Making programme, greatly
The big fabrication cycle that shortens has profound influence to animation industry.Our technology is followed the trail of using conformal geometry method
Expression is caught, expression is extracted, so as to device enters cartoon figure face, there is high description degree in conjunction with exporting into
Personalized model.
Specific embodiment：
One kind is based on depth camera 3D FA Facial Animation real-time capture technology, including depth scan instrument, guarantor's type ground
Figure, monitor, control chip and the helmet, inner helmet is provided with depth scan instrument, described depth scan
It is between two surfaces a basic geometric model to be mapped into row high-quality to calculate that instrument is, uses optical grating diffraction
Mode continuously recorded, described depth scan instrument connection control chip, described control chip, institute
The control chip stated remotely connects monitor, and described monitor is exported based on video card, and signal is entered
The synchronous conversion of row, it is ensured that complete three-dimensional premonitoring by the mapping relations of four screens, be respectively left eye monitor,
Right eye monitor, the operation interface of computer software and three-dimensional premonitoring screen；Described guarantor's type map is angle
Without the map distorted, within depth scan.Described depth scan instrument is go pro scanneies.
Embodiment 1：
1st, facial scan model is obtained.When facial scan is carried out, dark environment is needed, but due to setting
The restriction of standby working method, during being scanned, the result of facial shadow region is very undesirable.Solution
Certainly method：We are attempted while keeping integrated environment dark, are facial shadow region using photography skill and technique
Domain has carried out faint light filling process makes it completely dead black, but is unlikely to too bright and loses scanning details.
2nd, obtain real and shoot video sequence.The video camera for being fixed to head should be relative to head position
Maintain static, but performer is when doing compared with big-movement, and head is rocked, and causes video camera to produce position with head,
Produce the image shot to rock, have a strong impact on seizure effect.Solution：A) strengthen the helmet in
Contiguity between head.B) it is small to rock inevitable, so we attempt right in later stage software
Image has carried out stabilized treatment, so as to obtain preferable photographed data.
3rd, calculate face and catch the locus of points.When two catch point lean on it is too near when, follow the trail of data can produce
Mistake, two points are admitted one's mistake.So we catch in face studied on point, seizure point is improved
Pattern.Interactive will likely put has carried out labelling with different colors of following the trail of, and so exists
Would not be to producing trace error in MathchMovie.Substantially increase automaticity.
4th, catch data to import.MathchMovie softwares can only export as * .RZ2 lattice after finishing really according to calculation
The 2 D animation data of formula, and three-dimensional software can not support this form.For this, we have especially looked into the number
Define according to the form of file, and a software has been customized for it, RZ2 forms can be imported in three-dimensional software
Data and automatically generate key-frame animation.Portion of program code is as follows.
As a result：
Spatial digitizer can drive the original expression passage of face by way of single mesh.
All facial expression threedimensional models of performer are all caught with depth scan instrument saves as three-dimensional mould
Type sequence facilitates subsequent technology to call.
Helmet-type equipment is recycled to carry out facial seizure, this helmet can be that any helmet is installed additional and tells shooting
Head, the image that then analysis single camera is taken carries out pretreatment and process is tracked in Trancking Software,
Obtain one 2 dimension tracking data.
It is last that the 2 dimension data information for importing are deduced into out corresponding 3-dimensional data message in three-dimensional software, most
The true man's expression for driving a depth scan instrument to scan by the two dimensional image of single camera shooting is completed eventually
Animation, so as to the function of completing but webcam driver role expresses one's feelings.
Claims (3)
1. it is a kind of to be based on depth camera 3D FA Facial Animation real-time capture technology, including depth scan instrument, guarantor
Type map, monitor, control chip and the helmet, it is characterised in that：Inner helmet is provided with depth scan instrument,
It is between two surfaces a basic geometry mould to be mapped into row high-quality to calculate that described depth scan instrument is
Type, is continuously recorded with the mode of optical grating diffraction, described depth scan instrument connection control chip, described
Control chip, described control chip remotely connects monitor, and described monitor is based on video card
Output, conversion is synchronized to signal, it is ensured that complete three-dimensional premonitoring by the mapping relations of four screens, point
It is not left eye monitor, right eye monitor, the operation interface of computer software and three-dimensional premonitoring screen；It is described
Guarantor's type map be the map do not distorted of angle, within depth scan.
2. according to claim 1 based on depth camera 3D FA Facial Animation real-time capture technology, it is special
Levy and be：Described depth scan instrument is go pro scanneies.
3. a kind of imaging method based on depth camera 3D FA Facial Animation real-time capture technology, it is characterised in that：
Imaging method is as follows：
A, all facial expression threedimensional models of performer are all caught with depth scan instrument save as three-dimensional
Model sequence facilitates subsequent technology to call；
B, helmet-type equipment is recycled to carry out facial seizure, this helmet can be that any helmet installs additional and tells and take the photograph
As head, the image that then analysis single camera is taken carries out pretreatment and process is tracked in Trancking Software,
Obtain one 2 dimension tracking data；
C, it is last the 2 dimension data information for importing are deduced into out corresponding 3-dimensional data message in three-dimensional software,
The true man's table for being finally completed by the two dimensional image of single camera shooting to drive a depth scan instrument to scan
Feelings animation, so as to the function of completing but webcam driver role expresses one's feelings.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510665845.7A CN106611426A (en) | 2015-10-16 | 2015-10-16 | 3D (Three-dimensional) facial animation real-time capture technology based on depth camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510665845.7A CN106611426A (en) | 2015-10-16 | 2015-10-16 | 3D (Three-dimensional) facial animation real-time capture technology based on depth camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106611426A true CN106611426A (en) | 2017-05-03 |
Family
ID=58610250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510665845.7A Pending CN106611426A (en) | 2015-10-16 | 2015-10-16 | 3D (Three-dimensional) facial animation real-time capture technology based on depth camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106611426A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107886558A (en) * | 2017-11-13 | 2018-04-06 | 电子科技大学 | A kind of human face expression cartoon driving method based on RealSense |
| CN108108725A (en) * | 2018-01-20 | 2018-06-01 | 灵然创智（天津）动画科技发展有限公司 | A kind of personage head based on ultra high-definition photo and facial expression data acquisition platform |
| CN109859297A (en) * | 2019-03-07 | 2019-06-07 | 灵然创智（天津）动画科技发展有限公司 | One kind is unmarked to put facial capture device and method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101310289A (en) * | 2005-08-26 | 2008-11-19 | 索尼株式会社 | Capturing and processing facial motion data |
| CN103324905A (en) * | 2012-03-21 | 2013-09-25 | 天津生态城动漫园投资开发有限公司 | Next-generation virtual photostudio facial capture system |
| CN103634583A (en) * | 2012-08-28 | 2014-03-12 | 天津市灵感创然动画制作有限公司 | 3D animation film forced working creation platform |
-
2015
- 2015-10-16 CN CN201510665845.7A patent/CN106611426A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101310289A (en) * | 2005-08-26 | 2008-11-19 | 索尼株式会社 | Capturing and processing facial motion data |
| CN103324905A (en) * | 2012-03-21 | 2013-09-25 | 天津生态城动漫园投资开发有限公司 | Next-generation virtual photostudio facial capture system |
| CN103634583A (en) * | 2012-08-28 | 2014-03-12 | 天津市灵感创然动画制作有限公司 | 3D animation film forced working creation platform |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107886558A (en) * | 2017-11-13 | 2018-04-06 | 电子科技大学 | A kind of human face expression cartoon driving method based on RealSense |
| CN108108725A (en) * | 2018-01-20 | 2018-06-01 | 灵然创智（天津）动画科技发展有限公司 | A kind of personage head based on ultra high-definition photo and facial expression data acquisition platform |
| CN109859297A (en) * | 2019-03-07 | 2019-06-07 | 灵然创智（天津）动画科技发展有限公司 | One kind is unmarked to put facial capture device and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106403845B (en) | Three-dimension sensor system and three-dimensional data acquisition methods | |
| CN107067429A (en) | Video editing system and method that face three-dimensional reconstruction and face based on deep learning are replaced | |
| CN103608847B (en) | A kind of method and apparatus built for iconic model | |
| CN104658038A (en) | Method and system for producing three-dimensional digital contents based on motion capture | |
| CN103473801A (en) | Facial expression editing method based on single camera and motion capturing data | |
| CN105631859B (en) | Three-degree-of-freedom bionic stereo visual system | |
| CN106611426A (en) | 3D (Three-dimensional) facial animation real-time capture technology based on depth camera | |
| CN108305308A (en) | It performs under the line of virtual image system and method | |
| US20230230304A1 (en) | Volumetric capture and mesh-tracking based machine learning 4d face/body deformation training | |
| Bleyer et al. | Temporally consistent disparity maps from uncalibrated stereo videos | |
| Tang et al. | Structure-aware person image generation with pose decomposition and semantic correlation | |
| CN111598983A (en) | Animation system, animation method, storage medium, and program product | |
| CN104715468A (en) | Naked-eye 3D content creation improving method based on Unity 3D | |
| CN103248910B (en) | Three-dimensional imaging system and image reproducing method thereof | |
| KR101118978B1 (en) | Aparatus for generating learning images for face recognition system and method thereof | |
| Basak et al. | Methodology for building synthetic datasets with virtual humans | |
| CN111399662A (en) | Human-robot interaction simulation device and method based on high-reality virtual avatar | |
| CN102111565B (en) | Initial positioning method and device for camera in virtual studio system | |
| CN109903360A (en) | 3 D human face animation control system and its control method | |
| CN111161399B (en) | Data processing method and assembly for generating three-dimensional model based on two-dimensional image | |
| Gang et al. | Research on tujia brocade craft visualization based on unmarked motion capture technique | |
| Liang et al. | Reconstructing 3D virtual face with eye gaze from a single image | |
| Zhang et al. | Face animation making method based on facial motion capture | |
| CN115953520B (en) | Recording and playback method and device for virtual scene, electronic equipment and medium | |
| Yang et al. | Disentangled human action video generation via decoupled learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170503 |