CN108961367A - The method, system and device of role image deformation in the live streaming of three-dimensional idol - Google Patents
The method, system and device of role image deformation in the live streaming of three-dimensional idol Download PDFInfo
- Publication number
- CN108961367A CN108961367A CN201810641244.6A CN201810641244A CN108961367A CN 108961367 A CN108961367 A CN 108961367A CN 201810641244 A CN201810641244 A CN 201810641244A CN 108961367 A CN108961367 A CN 108961367A
- Authority
- CN
- China
- Prior art keywords
- model
- actor model
- dimensional
- idol
- animation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000008569 process Effects 0.000 claims abstract description 12
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 7
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 7
- 230000009471 action Effects 0.000 claims description 16
- 230000005540 biological transmission Effects 0.000 claims description 15
- 230000033001 locomotion Effects 0.000 claims description 11
- 210000000988 bone and bone Anatomy 0.000 claims description 10
- 230000000694 effects Effects 0.000 claims description 9
- 239000000463 material Substances 0.000 claims description 9
- 230000007704 transition Effects 0.000 claims description 5
- 230000008921 facial expression Effects 0.000 claims description 4
- 230000008676 import Effects 0.000 claims description 3
- 230000002452 interceptive effect Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000010422 painting Methods 0.000 claims description 3
- 230000003190 augmentative effect Effects 0.000 claims description 2
- 238000004891 communication Methods 0.000 claims description 2
- 238000005314 correlation function Methods 0.000 claims description 2
- 230000001815 facial effect Effects 0.000 claims description 2
- 238000002156 mixing Methods 0.000 claims description 2
- 239000011800 void material Substances 0.000 claims 1
- 230000008859 change Effects 0.000 abstract description 7
- 241000282461 Canis lupus Species 0.000 abstract description 2
- 230000008901 benefit Effects 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 6
- 238000003860 storage Methods 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 238000009877 rendering Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000000151 deposition Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 235000015067 sauces Nutrition 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/20—3D [Three Dimensional] animation
- G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention relates to the method, system and device of role image deformation in a kind of live streaming of three-dimensional idol, for solving the 3D actor model and textures file of creation three-dimensional idol；3D actor model is deformed, corresponding metamorphic animation and skeleton cartoon are generated；3D actor model is directed into 3D game engine；Then the true stage business information of real-time capture is associated with and controls 3D actor model, and executes metamorphic animation, the synthesis animation of the 3D actor model is superimposed to the video pictures of recording.The invention has the benefit that being realized in the live streaming of three-dimensional idol through the invention, the image of oneself that virtual idol leading role can change in real time, and different floor show styles can be extended according to different images, improve the entertainment and appreciative value of program；It solves in previous three-dimensional idol live streaming, the deformation process of leading role can not be showed, become the specific deformation process of wolf man from people, become reality.
Description
Technical field
The present invention relates to the method, system and device of role image deformation in a kind of live streaming of three-dimensional idol, belong to meter
Field is broadcast live in calculation machine artificial intelligence.
Background technique
Lift virtual idol, estimate that everybody expects at the first time is exactly first sound future.Really, first before being born in 10 years
Sound future is one of most successful virtual idol in current worldwide.And in recent years, with Chinese " Quadratic Finite Element economy "
The arrival of upsurge, more and more Chinese companies start to aim at the new cake of " virtual idol " this block, are proposed oneself one after another
Virtual idol, and the team in some more forward positions develops more and has promising real-time virtual idol live streaming, it is such as hot at present complete
Japan's " love sauce " of ball etc..
But on the live programming content of these virtual idols, often there is very big technical limitation, these limitations are all
Fetter the diversity and intention of program, such as our metamorphic animations for frequently referring to.We are developed empty based on three-dimensional
The method of the role that quasi- idol is broadcast live image deformation in real time, device and system can allow virtual idol to change in real time certainly in performance
Oneself image, and there are some deformation effects, rather than simple substitution model, appreciation effect is provided thus.
Summary of the invention
Several exemplary aspects of the disclosure are summarized as follows.This summary is provided to the convenience of reader, to provide to this
The basic comprehension of a little embodiments rather than fully limit the scope of the invention.This summarizes the extensive of not all contemplated embodiments
Summary, and it is neither intended to mark crucial or important element in all aspects, the range in terms of any or all is not described yet.Its
Sole purpose is that some concepts of one or more embodiments are presented in simplified form, retouches in more detail as what is presented later
The prelude stated.For convenience, term " some embodiments " herein can be used for referring to the single embodiment or multiple of the disclosure
Embodiment.
The present invention provides the method, system and device of role image deformation in a kind of live streaming of three-dimensional idol, for solving
The certainly deficiency of prior art problem.
Technical side of the invention includes the method for role image deformation in the live streaming of three-dimensional idol, which is characterized in that should
Method is the following steps are included: A. creates the 3D actor model of three-dimensional idol, and creates corresponding textures for 3D actor model
File；B. the 3D actor model that step A is obtained is deformed, the 3D model file after obtaining variant, further, is based on the angle 3D
3D actor model vertex after color model and variant generates corresponding metamorphic animation and skeleton cartoon；C. described 3D role will be based on
3D actor model after model and variant is directed into 3D game engine, wherein import 3D engine based on 3D actor model and variant
3D actor model afterwards can be called by engine blueprint；D. then the true stage business information of real-time capture is associated with and controls
3D actor model, and metamorphic animation is executed, the synthesis animation of the 3D actor model is superimposed to the video pictures of recording.
According to the method for role image deformation in the three-dimensional idol live streaming, wherein step A includes: according to original painting
Data configures people for virtual three-dimensional idol and sets performance data, people set performance data include occupational information, personality information or
Personage's background characteristics, and corresponding 3D actor model is created for the three-dimensional idol；3D actor model is subjected to normal
Smooth treatment simultaneously creates mapping UV coordinate, according to the corresponding model pinup picture file of the virtual idol of UV coordinate drawing three-dimensional, further
Generate the material ball file of three-dimensional idol；Model pinup picture and 3D role's module are associated, and by the right material ball
File assigns model pinup picture file；Bone and brush weight model are bound to assign the 3D actor model of model pinup picture file.
According to the method for role image deformation in the three-dimensional idol live streaming, wherein step B includes: according to
Step A further creates the deformed variant 3D model file of three-dimensional idol, further, creates multiple based on identical bone
Different shapes model file；Then model and textures material file based on variant 3D model are created, be tied to it is described
Step A three-dimensional idol same model file simultaneously brushes weight processed；Create the 3D based on step A creation three-dimensional idol
The vertex deformation animation of actor model and variant 3D model, and then create the textured bone animation of corresponding two models.
According to the method for role image deformation in the three-dimensional idol live streaming, wherein step C includes: that will be based on three
The 3D actor model and receptor model of idol are tieed up, and, corresponding texturing and animation file imported into 3D game engine
Middle filing；The outpost blueprint for changing animation and corresponding plug-in unit, last joint debugging correlation function are created using 3D game engine, and are passed through
Interface is connected to control program.
According to the method for role image deformation in the three-dimensional idol live streaming, wherein step D includes: to configure at least
One group of light shows effect of shadow for the near vicinity in augmented reality, and attached in real scene and virtual image boundary
Nearly compensation shadow transition；Position and the angle of camera image are calculated in real time, and are configured to the virtual 3D actor model
In, to interact and cope with shooting；Limb action, facial expression and the sound for capturing performer, are converted to and 3D actor model
People set the action data of association of characteristics, be then associated with the corresponding 3D actor model into graphics engine, and described in configuring
The action message and sound real-time synchronization of the action message and sound of performer and the 3D actor model animation；Action message is passed
It is defeated into the relational graph work station for being loaded onto 3D game engine, drive the three-dimensional real-time animation in final 3D engine；Control phase
It closes metamorphic animation and controls program, be finally completed the real-time deformation of three-dimensional idol, and in real time will by video acquisition device
Picture transmission to live streaming platform.
According to the method for role image deformation in the three-dimensional idol live streaming, wherein action data includes that limbs are dynamic
Make data, facial motion data and role's audio mixing data.
According to the method for role image deformation in the three-dimensional idol live streaming, wherein action message includes that limbs are dynamic
Work and facial expression.
Technical solution of the present invention further includes the device of role image deformation in the live streaming of three-dimensional idol, and feature exists
In: the first module creates corresponding textures text for creating the 3D actor model of three-dimensional idol, and for 3D actor model
Part；Second module is deformed for 3D actor model, the 3D model file after obtaining variant, further, is based on 3D role's mould
3D actor model vertex after type and variant generates corresponding metamorphic animation and skeleton cartoon；Third module will be based on 3D role
3D actor model after model and variant is directed into 3D game engine, wherein import 3D engine based on 3D actor model and variant
3D actor model afterwards can be called by engine blueprint；4th module, the true stage business information of real-time capture, is then associated with
With control 3D actor model, and execution metamorphic animation, the synthesis animation of 3D actor model is superimposed to the video pictures of recording.
Technical solution of the present invention further includes the system of role image deformation in the live streaming of three-dimensional idol, and feature exists
In, which includes: graphics workstation, for making 3D model and the modeling of textures of three-dimensional idol, and, to three-dimensional
Animation carries out dynamic capture；Capture device is imaged, the true stage business information of real-time capture is used for, is then associated with and controls 3D
Actor model；Graphics engine, for generating and handling 3D actor model, the physical kinetics movement of operation 3D actor model is right
3D actor model carries out deformation process；Instructor in broadcasting's console work station issues Controlling model real-time deformation for providing interactive interface
Instruction；Data transmission device, transition animation video are exported to video transmission software, and, carry out the nothing of real-time Data Transmission
Line information transmission modular.
The invention has the benefit that
1) solve previous three-dimensional idol live streaming in, virtual idol whole process can only an image, cannot be substituted for
Other images, show slightly dull problem；
2) solve in the live streaming of previous three-dimensional idol, virtual idol change the image can only substitution model by force, pole
For unnatural situation, the role image anamorphotic system of technical solution of the present invention can reach smooth natural deformation effect.
Detailed description of the invention
Fig. 1 show overview flow chart according to the method for the present invention；
Fig. 2 show the flow chart of the 3D model and textures file of embodiment according to the present invention；
Fig. 3 show the flow chart of the vertex deformation animation and skeleton cartoon of embodiment according to the present invention；
Fig. 4 show the engine blueprint call flow chart of embodiment according to the present invention；
Fig. 5 show the flow chart of the picture real-time rendering of embodiment according to the present invention
Fig. 6 show role image real-time deformation schematic diagram in the live streaming of embodiment according to the present invention；
Fig. 7 is the schematic diagram of the apparatus and system of embodiment according to the present invention；
Fig. 8 is the scene figure used in real time according to embodiment of the present invention.
Specific embodiment
Technical solution of the present invention includes method, system and the dress of role image deformation in a kind of live streaming of three-dimensional idol
It sets, it is clear suitable for being carried out below with reference to technical effect of the embodiment and attached drawing to design of the invention, specific structure and generation
Chu, complete description, to be completely understood by the purpose of the present invention, scheme and effect.
It should be noted that unless otherwise specified, when a certain feature referred to as " fixation ", " connection " are in another feature,
It can directly fix, be connected to another feature, and can also fix, be connected to another feature indirectly.In addition, this
The descriptions such as the upper and lower, left and right used in open are only the mutual alignment pass relative to each component part of the disclosure in attached drawing
For system.The "an" of used singular, " described " and "the" are also intended to including most forms in the disclosure, are removed
Non- context clearly expresses other meaning.In addition, unless otherwise defined, all technical and scientific terms used herein
It is identical as the normally understood meaning of those skilled in the art.Term used in the description is intended merely to describe herein
Specific embodiment is not intended to be limiting of the invention.Term as used herein "and/or" includes one or more relevant
The arbitrary combination of listed item.
It will be appreciated that though various elements, but this may be described using term first, second, third, etc. in the disclosure
A little elements should not necessarily be limited by these terms.These terms are only used to for same type of element being distinguished from each other out.For example, not departing from
In the case where disclosure range, first element can also be referred to as second element, and similarly, second element can also be referred to as
One element.The use of provided in this article any and all example or exemplary language (" such as ", " such as ") is intended merely to more
Illustrate the embodiment of the present invention well, and unless the context requires otherwise, otherwise the scope of the present invention will not be applied and be limited.
Fig. 1 show overview flow chart according to the method for the present invention.It is the following steps are included: A. creation three-dimensional is even
The 3D actor model of picture, and corresponding textures file is created for 3D actor model；B. the 3D actor model that step A is obtained is carried out
Deformation, the 3D model file after obtaining variant are further generated based on the 3D actor model vertex after 3D actor model and variant
Corresponding metamorphic animation and skeleton cartoon；C. the 3D actor model based on after 3D actor model and variant is directed into 3D trip
Play engine, wherein importing being called by engine blueprint based on the 3D actor model after 3D actor model and variant for 3D engine；
D. then the true stage business information of real-time capture is associated with and controls 3D actor model, and executes metamorphic animation, will be described
The synthesis animation of 3D actor model is superimposed to the video pictures of recording.
Technical solution of the present invention is additionally provided including the device for realizing the above method comprising: skill of the invention
Art scheme further includes the device of role image deformation in the live streaming of three-dimensional idol, it is characterised in that: the first module, for creating
The 3D actor model of three-dimensional idol, and corresponding textures file is created for 3D actor model；Second module is used for 3D role
Model is deformed, the 3D model file after obtaining variant, further, based on the 3D actor model after 3D actor model and variant
Vertex generates corresponding metamorphic animation and skeleton cartoon；Third module, will be based on 3D role's mould after 3D actor model and variant
Type is directed into 3D game engine, wherein importing being drawn based on the 3D actor model after 3D actor model and variant for 3D engine
Hold up blueprint calling；Then 4th module, the true stage business information of real-time capture are associated with and control 3D actor model, and
Metamorphic animation is executed, the synthesis animation of 3D actor model is superimposed to the video pictures of recording.
Technical solution of the present invention further includes the system for realizing the above method, is specifically included: graphics workstation is used for
The modeling of the 3D model and textures of three-dimensional idol is made, and, dynamic capture is carried out to three-dimensional animation；Camera shooting captures dress
It sets, is used for the true stage business information of real-time capture, be then associated with and control 3D actor model；Graphics engine, for generating
With processing 3D actor model, the physical kinetics movement of operation 3D actor model carries out deformation process to 3D actor model；Instructor in broadcasting
Console work station, the instruction for issuing Controlling model real-time deformation for providing interactive interface；Data transmission device, transition animation
Video is exported to video transmission software, and, carry out the wireless information transfer module of real-time Data Transmission.
Fig. 2 show the flow chart of the 3D model and textures file of embodiment according to the present invention.Following steps: according to three
The related proposal and original painting design original text for tieing up virtual idol live streaming, make the 3D model file based on three-dimensional idol, and will
3D model carries out normal smooth treatment, and production mapping UV coordinate is drawn the model based on three-dimensional idol according to UV coordinate and pasted
Then map file makes the material ball file of three-dimensional idol, then completes the production.Textures are associated with into this model file,
Then material is assigned to model.And model binds bone and brush weight model thus.
Fig. 3 show the flow chart of the vertex deformation animation and skeleton cartoon of embodiment according to the present invention.Production is based on
The deformed variant 3D model file of three-dimensional idol makes the model file of the different shapes based on identical bone, then
Model and textures material file based on this model are made, the model file being tied to as three-dimensional idol simultaneously brushes system power
Weight, and the vertex deformation animation based on this two models is made, and make the associated bone metamorphic animation based on the two models.
Fig. 4 show the engine blueprint call flow chart of embodiment according to the present invention, comprising: will be based on three-dimensional idol
3D actor model and receptor model and relevant texturing and animation file, which are imported into 3D game engine, to be filed, Ke Yi
Both of these documents are called in engine, then the outpost blueprint of production variation animation and related plug-in unit, last joint debugging phase in engine
Function is closed, and is connected to control program
Fig. 5 show the flow chart of the picture real-time rendering of embodiment according to the present invention, comprising: even based on three-dimensional
The motion capture performer of picture enters motion capture room, closes the three-dimensional bone of three-dimensional idol in dynamic catch in software of related 3D
Be linked to it is dynamic catch on the body of performer it is infrared it is dynamic catch mark point, accomplish movement exact matching, then move and catch performer and start to perform,
Action message is transferred in the relational graph work station for loading 3D game engine, drives the three-dimensional in final 3D engine dynamic in real time
It draws.Controllers control related variation animation control program on instructor in broadcasting's console at this time, are finally completed three-dimensional idol
Real-time deformation.And by video frequency collection card in real time by picture transmission to live streaming platform.
Fig. 6 show role image real-time deformation schematic diagram in the live streaming of embodiment according to the present invention, comprising: the first mould
Block, for making the 3D model of three-dimensional idol and the modeling graphics workstation of textures；Second module is caught for carrying out movement
The infrared motion caught captures camera shooting room；Third module, for loading, the dynamic three-dimensional for catching software is dynamic in real time catches graphics workstation；The
Four modules, the graphics workstation of the effect for rendering final picture；5th module, instructor in broadcasting's control of Controlling model real-time deformation
Platform work station；6th module carries out the wireless information transfer module of real-time Data Transmission.
Fig. 7 is the schematic diagram of the apparatus and system of embodiment according to the present invention, and wherein device and system include: to deposit
Reservoir, for storing a computer applied algorithm, processor, for running the computer applied algorithm to execute following step
It is rapid: the relevant software programs of production 3D model and animation；Control is related dynamic to catch the dynamic of data and catches software program；Control related variation
The Console program of information and instructor in broadcasting's control；Real-time game engine program for final picture real-time rendering.
Described in reference diagram 8, this is specifically used situation case of the invention.For indicating the real-time change of 3D actor model
Change schematic diagram.
Method, apparatus and system according to the present invention can be used for realizing following functions: 1) realizing through the invention
In the live streaming of three-dimensional idol, the image of oneself that virtual idol leading role can change in real time, and can according to different images
To extend different floor show styles, the entertainment and appreciative value of program are improved；2) solve through the invention with
Toward three-dimensional idol live streaming in leading role replace image can only replace its 3D model, the model of script, which suddenly disappears, replaces with it
His model shows lofty unnatural；3) it solves in previous three-dimensional idol live streaming, the deformation process of leading role can not be showed, by
People becomes the specific deformation process of wolf man, becomes reality；Those skilled in the art are considering specification and are practicing disclosed herein
Invention after, will readily occur to other embodiments of the disclosure.The disclosure be intended to cover any modification of the disclosure, purposes or
Person's adaptive change, these variations, uses, or adaptations follow the general principles of this disclosure and including the disclosure not
Disclosed common knowledge or conventional techniques in the art.The description and examples are only to be considered as illustrative,
The true scope and spirit of the disclosure are indicated by the following claims.
It should be appreciated that the embodiment of the present invention can be by computer hardware, the combination of hardware and software or by depositing
The computer instruction in non-transitory computer-readable memory is stored up to be effected or carried out.Standard volume can be used in the method
Journey technology-includes that the non-transitory computer-readable storage media configured with computer program is realized in computer program,
In configured in this way storage medium computer is operated in a manner of specific and is predefined --- according in a particular embodiment
The method and attached drawing of description.Each program can with the programming language of level process or object-oriented come realize with department of computer science
System communication.However, if desired, the program can be realized with compilation or machine language.Under any circumstance, which can be volume
The language translated or explained.In addition, the program can be run on the specific integrated circuit of programming for this purpose.
In addition, the operation of process described herein can be performed in any suitable order, unless herein in addition instruction or
Otherwise significantly with contradicted by context.Process described herein (or modification and/or combination thereof) can be held being configured with
It executes, and is can be used as jointly on the one or more processors under the control of one or more computer systems of row instruction
The code (for example, executable instruction, one or more computer program or one or more application) of execution, by hardware or its group
It closes to realize.The computer program includes the multiple instruction that can be performed by one or more processors.
Further, the method can be realized in being operably coupled to suitable any kind of computing platform, wrap
Include but be not limited to PC, mini-computer, main frame, work station, network or distributed computing environment, individual or integrated
Computer platform or communicated with charged particle tool or other imaging devices etc..Each aspect of the present invention can be to deposit
The machine readable code on non-transitory storage medium or equipment is stored up to realize no matter be moveable or be integrated to calculating
Platform, such as hard disk, optical reading and/or write-in storage medium, RAM, ROM, so that it can be read by programmable calculator, when
Storage medium or equipment can be used for configuration and operation computer to execute process described herein when being read by computer.This
Outside, machine readable code, or part thereof can be transmitted by wired or wireless network.When such media include combining microprocessor
Or other data processors realize steps described above instruction or program when, invention as described herein including these and other not
The non-transitory computer-readable storage media of same type.When methods and techniques according to the present invention programming, the present invention
It further include computer itself.
Computer program can be applied to input data to execute function as described herein, to convert input data with life
At storing to the output data of nonvolatile memory.Output information can also be applied to one or more output equipments as shown
Device.In the preferred embodiment of the invention, the data of conversion indicate physics and tangible object, including the object generated on display
Reason and the particular visual of physical objects are described.
The above, only presently preferred embodiments of the present invention, the invention is not limited to above embodiment, as long as
It reaches technical effect of the invention with identical means, all within the spirits and principles of the present invention, any modification for being made,
Equivalent replacement, improvement etc., should be included within the scope of the present invention.Its technical solution within the scope of the present invention
And/or embodiment can have a variety of different modifications and variations.
Claims (9)
1. the method for role image deformation in the live streaming of three-dimensional idol, which is characterized in that method includes the following steps:
A. the 3D actor model of three-dimensional idol is created, and creates corresponding textures file for 3D actor model；
B. the 3D actor model that step A is obtained is deformed, the 3D model file after obtaining variant, further, is based on the angle 3D
3D actor model vertex after color model and variant generates corresponding metamorphic animation and skeleton cartoon；
C. the 3D actor model based on after 3D actor model and variant is directed into 3D game engine, wherein importing 3D engine
Can be called by engine blueprint based on the 3D actor model after 3D actor model and variant；
D. then the true stage business information of real-time capture is associated with and controls 3D actor model, and executes metamorphic animation, will
The synthesis animation of the 3D actor model is superimposed to the video pictures of recording.
2. the method for role image deformation in three-dimensional idol live streaming according to claim 1, which is characterized in that described
Step A includes:
According to original painting data, people is configured for virtual three-dimensional idol and sets performance data, it includes occupation letter that people, which sets performance data,
Breath, personality information or personage's background characteristics, and corresponding 3D actor model is created for the three-dimensional idol；
3D actor model is subjected to normal smooth treatment and creates mapping UV coordinate, according to the virtual idol pair of UV coordinate drawing three-dimensional
The model pinup picture file answered further generates the material ball file of three-dimensional idol；
Model pinup picture and 3D role's module are associated, and assign the right material ball file to model pinup picture file；
Bone and brush weight model are bound to assign the 3D actor model of model pinup picture file.
3. the method for role image deformation in three-dimensional idol live streaming according to claim 1, which is characterized in that described
Step B includes:
The deformed variant 3D model file of three-dimensional idol is further created according to the step A, further, is created multiple
The model file of different shapes based on identical bone；
Then model and textures material file based on variant 3D model are created, is tied to and the step A three-dimensional idol
Same model file simultaneously brushes weight processed；
The vertex deformation animation of 3D actor model and variant 3D model based on step A creation three-dimensional idol is created,
And then create the textured bone animation of corresponding two models.
4. the method for role image deformation in three-dimensional idol live streaming according to claim 1, which is characterized in that described
Step C includes:
By 3D actor model and receptor model based on three-dimensional idol, and, corresponding texturing and animation file import
File into 3D game engine；
The outpost blueprint for changing animation and corresponding plug-in unit, last joint debugging correlation function are created using 3D game engine, and are passed through and connect
Mouth is connected to control program.
5. the method for role image deformation in three-dimensional idol live streaming according to claim 1 or 4, which is characterized in that
The step D includes:
At least one set of light is configured, shows effect of shadow for the near vicinity in augmented reality, and in real scene and void
Quasi- image, which has a common boundary, nearby compensates shadow transition；
Position and the angle of camera image are calculated in real time, and is configured in the virtual 3D actor model, to be handed over
Mutually and reply is shot；
Limb action, facial expression and the sound for capturing performer, are converted to the movement that association of characteristics is set with the people of 3D actor model
Then data are associated with the corresponding 3D actor model into graphics engine, and configure the action message and sound of the performer
With the action message and sound real-time synchronization of the 3D actor model animation；
Action message is transferred in the relational graph work station for being loaded onto 3D game engine, drives the three-dimensional in final 3D engine
Real-time animation；
It controls related variation animation and controls program, be finally completed the real-time deformation of three-dimensional idol, and pass through video acquisition
Device is in real time by picture transmission to live streaming platform.
6. the method for role image deformation in three-dimensional idol live streaming according to claim 5, which is characterized in that described
Action data includes limb action data, facial motion data and role's audio mixing data.
7. the method for role image deformation in three-dimensional idol live streaming according to claim 5, which is characterized in that described
Action message includes limb action and facial expression.
8. the device of role image deformation in the live streaming of three-dimensional idol characterized by comprising
First module creates corresponding textures text for creating the 3D actor model of three-dimensional idol, and for 3D actor model
Part；
Second module is deformed for 3D actor model, the 3D model file after obtaining variant, further, is based on 3D role
3D actor model vertex after model and variant generates corresponding metamorphic animation and skeleton cartoon；
Third module will be directed into 3D game engine based on the 3D actor model after 3D actor model and variant, wherein importing 3D
Engine can be called based on the 3D actor model after 3D actor model and variant by engine blueprint；
Then 4th module, the true stage business information of real-time capture are associated with and control 3D actor model, and executes deformation
The synthesis animation of 3D actor model is superimposed to the video pictures of recording by animation.
9. the system of role image deformation in the live streaming of three-dimensional idol, which is characterized in that the system includes:
Graphics workstation, for making 3D model and the modeling of textures of three-dimensional idol, and, three-dimensional animation is moved
State captures；
Capture device is imaged, the true stage business information of real-time capture is used for, is then associated with and controls 3D actor model；
Graphics engine, for generating and handling 3D actor model, the physical kinetics movement of operation 3D actor model, to 3D role
Model carries out deformation process；
Instructor in broadcasting's console work station, the instruction for issuing Controlling model real-time deformation for providing interactive interface；
Data transmission device, transition animation video are exported to video transmission software, and, carry out the wireless communication of real-time Data Transmission
Cease transmission module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810641244.6A CN108961367A (en) | 2018-06-21 | 2018-06-21 | The method, system and device of role image deformation in the live streaming of three-dimensional idol |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810641244.6A CN108961367A (en) | 2018-06-21 | 2018-06-21 | The method, system and device of role image deformation in the live streaming of three-dimensional idol |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108961367A true CN108961367A (en) | 2018-12-07 |
Family
ID=64491520
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810641244.6A Pending CN108961367A (en) | 2018-06-21 | 2018-06-21 | The method, system and device of role image deformation in the live streaming of three-dimensional idol |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108961367A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740476A (en) * | 2018-12-25 | 2019-05-10 | 北京琳云信息科技有限责任公司 | Instant communication method, device and server |
| CN109829958A (en) * | 2018-12-24 | 2019-05-31 | 武汉西山艺创文化有限公司 | A kind of virtual idol based on transparent liquid crystal display performs in a radio or TV programme method and apparatus |
| CN110992495A (en) * | 2019-12-26 | 2020-04-10 | 珠海金山网络游戏科技有限公司 | Virtual model deformation method and device |
| CN111179392A (en) * | 2019-12-19 | 2020-05-19 | 武汉西山艺创文化有限公司 | Virtual idol comprehensive live broadcast method and system based on 5G communication |
| CN111563945A (en) * | 2020-04-30 | 2020-08-21 | 完美世界(北京)软件科技发展有限公司 | Generation method, device and equipment of character morphing animation and readable storage medium |
| CN112164128A (en) * | 2020-09-07 | 2021-01-01 | 广州汽车集团股份有限公司 | Music visual interaction method and computer equipment for vehicle-mounted multimedia |
| JP2021053181A (en) * | 2019-09-30 | 2021-04-08 | 株式会社コロプラ | Program, method and viewing terminal |
| CN112634416A (en) * | 2020-12-23 | 2021-04-09 | 北京达佳互联信息技术有限公司 | Method and device for generating virtual image model, electronic equipment and storage medium |
| CN113362263A (en) * | 2021-05-27 | 2021-09-07 | 百度在线网络技术（北京）有限公司 | Method, apparatus, medium, and program product for changing the image of a virtual idol |
| CN113590334A (en) * | 2021-08-06 | 2021-11-02 | 广州博冠信息科技有限公司 | Role model processing method, role model processing device, role model processing medium and electronic equipment |
| WO2022001652A1 (en) * | 2020-07-02 | 2022-01-06 | 腾讯科技（深圳）有限公司 | Virtual character control method and apparatus, computer device, and storage medium |
| CN113590334B (en) * | 2021-08-06 | 2024-06-04 | 广州博冠信息科技有限公司 | Method, device, medium and electronic equipment for processing character model |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050137015A1 (en) * | 2003-08-19 | 2005-06-23 | Lawrence Rogers | Systems and methods for a role-playing game having a customizable avatar and differentiated instant messaging environment |
| CN107154069A (en) * | 2017-05-11 | 2017-09-12 | 上海微漫网络科技有限公司 | A kind of data processing method and system based on virtual role |
| CN107197385A (en) * | 2017-05-31 | 2017-09-22 | 珠海金山网络游戏科技有限公司 | A kind of real-time virtual idol live broadcasting method and system |
| CN107248195A (en) * | 2017-05-31 | 2017-10-13 | 珠海金山网络游戏科技有限公司 | A kind of main broadcaster methods, devices and systems of augmented reality |
| US20180122147A1 (en) * | 2016-10-31 | 2018-05-03 | Dg Holdings, Inc. | Transferrable between styles virtual identity systems and methods |
-
2018
- 2018-06-21 CN CN201810641244.6A patent/CN108961367A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050137015A1 (en) * | 2003-08-19 | 2005-06-23 | Lawrence Rogers | Systems and methods for a role-playing game having a customizable avatar and differentiated instant messaging environment |
| US20180122147A1 (en) * | 2016-10-31 | 2018-05-03 | Dg Holdings, Inc. | Transferrable between styles virtual identity systems and methods |
| CN107154069A (en) * | 2017-05-11 | 2017-09-12 | 上海微漫网络科技有限公司 | A kind of data processing method and system based on virtual role |
| CN107197385A (en) * | 2017-05-31 | 2017-09-22 | 珠海金山网络游戏科技有限公司 | A kind of real-time virtual idol live broadcasting method and system |
| CN107248195A (en) * | 2017-05-31 | 2017-10-13 | 珠海金山网络游戏科技有限公司 | A kind of main broadcaster methods, devices and systems of augmented reality |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109829958A (en) * | 2018-12-24 | 2019-05-31 | 武汉西山艺创文化有限公司 | A kind of virtual idol based on transparent liquid crystal display performs in a radio or TV programme method and apparatus |
| CN109740476B (en) * | 2018-12-25 | 2021-08-20 | 北京琳云信息科技有限责任公司 | Instant messaging method, device and server |
| CN109740476A (en) * | 2018-12-25 | 2019-05-10 | 北京琳云信息科技有限责任公司 | Instant communication method, device and server |
| JP2021053181A (en) * | 2019-09-30 | 2021-04-08 | 株式会社コロプラ | Program, method and viewing terminal |
| CN111179392A (en) * | 2019-12-19 | 2020-05-19 | 武汉西山艺创文化有限公司 | Virtual idol comprehensive live broadcast method and system based on 5G communication |
| CN110992495A (en) * | 2019-12-26 | 2020-04-10 | 珠海金山网络游戏科技有限公司 | Virtual model deformation method and device |
| CN110992495B (en) * | 2019-12-26 | 2023-09-08 | 珠海金山数字网络科技有限公司 | Method and device for deforming virtual model |
| CN111563945A (en) * | 2020-04-30 | 2020-08-21 | 完美世界(北京)软件科技发展有限公司 | Generation method, device and equipment of character morphing animation and readable storage medium |
| WO2022001652A1 (en) * | 2020-07-02 | 2022-01-06 | 腾讯科技（深圳）有限公司 | Virtual character control method and apparatus, computer device, and storage medium |
| CN112164128A (en) * | 2020-09-07 | 2021-01-01 | 广州汽车集团股份有限公司 | Music visual interaction method and computer equipment for vehicle-mounted multimedia |
| CN112164128B (en) * | 2020-09-07 | 2024-06-11 | 广州汽车集团股份有限公司 | Vehicle-mounted multimedia music visual interaction method and computer equipment |
| CN112634416A (en) * | 2020-12-23 | 2021-04-09 | 北京达佳互联信息技术有限公司 | Method and device for generating virtual image model, electronic equipment and storage medium |
| CN112634416B (en) * | 2020-12-23 | 2023-07-28 | 北京达佳互联信息技术有限公司 | Method and device for generating virtual image model, electronic equipment and storage medium |
| CN113362263A (en) * | 2021-05-27 | 2021-09-07 | 百度在线网络技术（北京）有限公司 | Method, apparatus, medium, and program product for changing the image of a virtual idol |
| CN113362263B (en) * | 2021-05-27 | 2023-09-15 | 百度在线网络技术（北京）有限公司 | Method, apparatus, medium and program product for transforming an image of a virtual idol |
| CN113590334A (en) * | 2021-08-06 | 2021-11-02 | 广州博冠信息科技有限公司 | Role model processing method, role model processing device, role model processing medium and electronic equipment |
| CN113590334B (en) * | 2021-08-06 | 2024-06-04 | 广州博冠信息科技有限公司 | Method, device, medium and electronic equipment for processing character model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108961367A (en) | The method, system and device of role image deformation in the live streaming of three-dimensional idol | |
| KR102069964B1 (en) | Virtual reality-based apparatus and method to generate a three dimensional(3d) human face model using image and depth data | |
| CN108986190A (en) | A kind of method and system of the virtual newscaster based on human-like persona non-in three-dimensional animation | |
| CN107274466A (en) | The methods, devices and systems that a kind of real-time double is caught | |
| CN107248195A (en) | A kind of main broadcaster methods, devices and systems of augmented reality | |
| US9196074B1 (en) | Refining facial animation models | |
| KR20130016318A (en) | A method of real-time cropping of a real entity recorded in a video sequence | |
| CN103916621A (en) | Method and device for video communication | |
| CN107248185A (en) | A kind of virtual emulation idol real-time live broadcast method and system | |
| CN109035415B (en) | Virtual model processing method, device, equipment and computer readable storage medium | |
| EP4176412A1 (en) | Generating an animation rig for use in animating a computer-generated character based on facial scans of an actor and a muscle model | |
| KR102353556B1 (en) | Apparatus for Generating Facial expressions and Poses Reappearance Avatar based in User Face | |
| US11640687B2 (en) | Volumetric capture and mesh-tracking based machine learning 4D face/body deformation training | |
| CN108961368A (en) | The method and system of real-time live broadcast variety show in three-dimensional animation environment | |
| US20160086365A1 (en) | Systems and methods for the conversion of images into personalized animations | |
| Hauswiesner et al. | Image-based clothes transfer | |
| CN110096144A (en) | A kind of interaction holographic projection methods and system based on three-dimensional reconstruction | |
| US11682156B2 (en) | Method for controlling digital feather growth between two manifolds in a computer simulated creature | |
| CN108848286A (en) | Three-dimensional idol live broadcasting method, apparatus and system based on wire ancillary equipment | |
| CN115914660A (en) | Method for controlling actions and facial expressions of digital people in meta universe and live broadcast | |
| US20220076409A1 (en) | Systems and Methods for Building a Skin-to-Muscle Transformation in Computer Animation | |
| CN107369195A (en) | It is a kind of to realize 3D game racquets according to the method, apparatus and system shared | |
| Huang et al. | A process for the semi-automated generation of life-sized, interactive 3D character models for holographic projection | |
| Hou et al. | Real-time markerless facial motion capture of personalized 3D real human research | |
| US11450054B2 (en) | Method for operating a character rig in an image-generation system using constraints on reference nodes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211222 Address after: 430000 Room 408, floor 4, building B24, phase 2.7, financial background service center base construction project, No. 77, Guanggu Avenue, Donghu New Technology Development Zone, Wuhan, Hubei Province Applicant after: Wuhan Jinshan Shiyou Technology Co.,Ltd. Address before: 519000 building 3, Jinshan Software Park, 325 Qiandao Ring Road, Xiangzhou District, Zhuhai City, Guangdong Province Applicant before: ZHUHAI KINGSOFT ONLINE GAME TECHNOLOGY Co.,Ltd. Applicant before: ZHUHAI XISHANJU INTERACTIVE ENTERTAINMENT TECHNOLOGY Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181207 |
|
| RJ01 | Rejection of invention patent application after publication |