CN107277599A - A kind of live broadcasting method of virtual reality, device and system - Google Patents
A kind of live broadcasting method of virtual reality, device and system Download PDFInfo
- Publication number
- CN107277599A CN107277599A CN201710400733.8A CN201710400733A CN107277599A CN 107277599 A CN107277599 A CN 107277599A CN 201710400733 A CN201710400733 A CN 201710400733A CN 107277599 A CN107277599 A CN 107277599A
- Authority
- CN
- China
- Prior art keywords
- model
- main broadcaster
- real
- live
- facial expression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/131—Protocols for games, networked simulations or virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a kind of live broadcasting method of virtual reality, device and system.Method comprises the following steps：Virtual 3D role is created, and is directed into graphics engine, VR main broadcaster's model is generated；The real stage business information of real-time capture, then associates and controls VR main broadcaster's model；VR main broadcaster's model is blended into VR scenes, then docked in real time with live platform, live picture is generated.Device includes corresponding to the module of each step of methods described.System includes：Dynamic device for catching, the limb action for the true performer of real-time capture；Facial expression catcher, the facial expression for catching true performer；Graphics engine, for generating and handling 3D actor models, the physical kinetics action of computing 3D actor models, transition animation video frequency output to video transmission software；Special efficacy generation module, for coordinating VR scenes, increases special efficacy；Application program for performing the above method.The solution of the present invention can effectively improve the telepresenc and audience interaction of VR main broadcaster.
Description
Technical field
The present invention relates to Video Composition technical field, more particularly to a kind of live broadcasting method of virtual reality, device and system.
Background technology
Video playback performance platform turns into entertainment medium popular at present.Main broadcasters can may be used in arbitrary legal place
To carry out net cast or perform in a radio or TV programme.But existing main broadcaster's content is more dull and not strong with spectators' sense of participation, telepresenc is not
By force.
Virtual reality (VR), is a kind of by real world information and the integrated new technology of virtual world information " seamless ", leads to
The cutting edge technologies such as computer are crossed, true are combined what script can not be experienced in real world with illusory.After analog simulation again
Superposition, illusory role or object are added in the real world, perceived by human vision sense organ, surmount existing so as to reach
Real experience.Thus true environment and illusory object can be in real time added in same space.
And do not occur VR virtual newscasters also in the current direct-seeding of industry.Existing VR broadcasting content is single, lacks
The communication of few and spectators and interaction.
The content of the invention
The present invention, to solve above-mentioned technical problem, is carried by providing a kind of live broadcasting method of virtual reality, device and system
The telepresenc and audience interaction of high VR main broadcaster.
On the one hand the technical solution adopted by the present invention is a kind of live broadcasting method of virtual reality, is comprised the following steps：A, wound
Virtual actor model is built, and is directed into graphics engine, VR main broadcaster's model is generated；The real stage business letter of B, real-time capture
Breath, then associates and controls VR main broadcaster's model, wherein, the action message includes limb action, facial expression and sound；
C, VR main broadcaster's model is blended into VR scenes, then docked in real time with live platform, generate live picture.
Further, wherein the step A includes：It is that the virtual role configuration people sets characteristic quantity according to original painting data
According to people sets performance data and includes occupational information, personality information or personage's background characteristics, and configures patch for VR main broadcaster's model
Figure, material and 3D animation bones.
Further, wherein the step A also includes：VR main broadcaster's model and its role animation bone of matching are imported
The computing skeleton cartoon into the graphics engine.
Further, wherein the step B includes：Limb action, facial expression and the sound of the performer is caught, is converted to
Limb action data, facial motion data and the role's audio mixing data of association of characteristics are set with the people of the role, then associate to
Corresponding VR main broadcaster's model in graphics engine, and configure limb action, facial expression and the sound of the performer with it is described
Limb action, facial expression and the sound of VR main broadcaster's model animation being capable of real-time synchronizations.
Further, wherein the step B also includes：The skeleton model of the performer is extracted, is then introduced into the figure
In engine, to match the animation bone of VR main broadcaster's model；According to the skeleton model come real-time capture and the limb of conversion performer
The limb action data of real-time capture are generated action directive by body action data, are generated by the graphics engine corresponding
VR main broadcaster's model movement posture；Computing generates limbs animation between the movement posture of VR main broadcaster's model.
Further, wherein the step B also includes：The facial skeleton of the performer is extracted, is then introduced into the figure
In engine, to match the facial skeleton covering of VR main broadcaster's model；According to the facial skeleton come real-time capture and conversion performer
Facial motion data, the facial motion data of real-time capture is generated into facial expression control instruction, passes through the graphics engine
Generate the facial expression shape of corresponding VR main broadcaster's model；Between the facial expression shape of the corresponding facial positions of VR main broadcaster's model
Computing generates facial expression animation transition.
Further, wherein the step C includes：Light and music are added, passes through the instructor in broadcasting face of VR background process clients
Plate is controlled；Position and the angle of camera image are calculated in real time, and is configured in the virtual role, to be handed over
Mutual and reply is shot；The picture composition that multiple video cameras are shot generates panorama display picture, is then forwarded to VR background process
Client is shown with generating panorama.
Technical scheme second aspect is a kind of live broadcast device of virtual reality, including：First module, for creating
Virtual 3D role is built, and is directed into graphics engine, VR main broadcaster's model is generated；Second module, is really drilled for real-time capture
Member's action message, then associates and controls VR main broadcaster's model；3rd module, VR are blended into by VR main broadcaster's model
Scape, is then docked with live platform in real time, generates live picture；Wherein, the action message includes limb action, facial expression
And sound.
Further, first module also includes capture module, is used for：Catch the limb action of the performer, facial table
Feelings and sound, the people be converted to the role set limb action data, facial motion data and the role's audio mixing of association of characteristics
Data, then associate corresponding VR main broadcaster's model into graphics engine, and configure the limb action of the performer, facial table
Limb action, facial expression and the sound of feelings and sound and VR main broadcaster's model animation being capable of real-time synchronizations.
The technical scheme third aspect is a kind of live broadcast system of virtual reality, including：Dynamic device for catching, for reality
When catch the limb action of true performer；Facial expression catcher, the facial expression for catching true performer；Graphics engine,
For generating and handling actor model, the physical kinetics action of computing actor model, transition animation video frequency output to video is passed
Broadcast software；Light and special efficacy generation module, for coordinating light, illumination and VR scenes, increase special efficacy；And application program.Should
Application program is used to perform following steps：Virtual role is created, and is directed into graphics engine, VR main broadcaster's model is generated；In real time
Real stage business information is caught, VR main broadcaster's model is then associated and control；VR main broadcaster's model is blended into VR
Scene, is then docked with live platform in real time, generates live picture.
Beneficial effects of the present invention are：1) live simply personage chat is sung before solving, and body as no VR is live faces
The sensation in its border；2) the problem of real main broadcaster's content is without telepresenc is solved；3) live content is dull before solving, and does not have
The problem of having 3D virtual roles.
Brief description of the drawings
Fig. 1 is the flow chart of the live broadcasting method of the virtual reality according to embodiments of the invention；
Fig. 2 is the schematic diagram of VR main broadcaster's modelling process in embodiments of the invention；
Fig. 3 is the schematic diagram of VR main broadcaster's bone manufacturing process in embodiments of the invention；
Fig. 4 is the schematic diagram of VR main broadcaster's kinetic model configuration process in embodiments of the invention；
Fig. 5 is the schematic diagram to carrying out the real-time playing process of VR in embodiments of the invention；
Fig. 6 is the block diagram of the live broadcast system of the virtual reality in embodiments of the invention；
Fig. 7 is the schematic diagram of VR main broadcaster's modelling process in specific embodiment of the invention；
Fig. 8 is the schematic diagram of VR main broadcaster's kinetic model configuration process in specific embodiment of the invention；
Fig. 9 is that then the real stage business information of real-time capture in the specific embodiment according to the present invention associate and control
Make the schematic diagram of VR main broadcaster's model；
Figure 10 is the schematic diagram that VR panoramas and special efficacy in specific embodiment of the invention are generated.
Embodiment
The term used in the disclosure is the purpose only merely for description specific embodiment, and is not intended to be limiting the disclosure.
" one kind ", " described " and "the" of singulative used in disclosure and the accompanying claims book are also intended to including majority
Form, unless context clearly shows that other implications.It is also understood that term "and/or" used herein refers to and wrapped
It may be combined containing one or more associated any or all of project listed.
It will be appreciated that though various information, but this may be described using term first, second, third, etc. in the disclosure
A little information should not necessarily be limited by these terms.These terms are only used for same type of information being distinguished from each other out.For example, not departing from
In the case of disclosure scope, the first information can also be referred to as the second information, similarly, and the second information can also be referred to as
One information.
Hereinafter, it reference will also be given to accompanying drawing and the present invention be explained in greater detail.In all of the figs, identical reference table
Show identical feature.
The flow chart of the live broadcasting method of virtual reality shown in reference picture 1.This method includes following key step：A, establishment
Virtual 3D role, and graphics engine is directed into, generate VR main broadcaster's model；The real stage business information of B, real-time capture, so
After associate and control VR main broadcaster's model, wherein, the action message include limb action, facial expression and sound；C, general
VR main broadcaster's model is blended into VR scenes, is then docked in real time with live platform, generates live picture.
As shown in Fig. 2 including VR main broadcaster's modelling process in main step A：
S201, is that the virtual actor model configuration people sets performance data according to original painting data make actor model.People
If performance data includes occupational information, personality information or personage's background characteristics.
S202, carries out the preliminary Role Modeling of VR main broadcaster.According to actual conditions, it can continue in step S204 further
Modification, sets requirement, or meet the requirement newly changed to meet actor model people.
S203, is VR main broadcaster's model configuration textures, material.
As shown in figure 3, including VR main broadcaster's bone manufacturing process in main step A：
S301, the 3D animation bones for making role；
S302, for the animation skeleton model bind and make weight.
As shown in figure 4, also including VR main broadcaster's kinetic model configuration process in main step A：
S401, analytic dynamics model object, such as body, hair and the clothes of actor model.These objects can make
With the technology in terms of physical kinetics, moved according to physics law.
S402, for the model configure kinetic parameter and dynamics application.
S403, VR main broadcaster's model and its role animation bone of matching be directed into graphics engine, set dynamic of guiding
Mechanical model object carries out preview action.According to actually required, motivation of adjustment it can also configure and change in step s 404
Kinetic model object.Preferably, the graphics engine is drawn using advanced generation game engine with other game graphs
Difference is held up, the game engine used here does not need the support of third party software with regard to that can handle physical effect, sound and animation.
Further, wherein the step B includes：Limb action, facial expression and the sound of the performer is caught, is converted to
Limb action data, facial motion data and the role's audio mixing data of association of characteristics are set with the people of the 3D role, are then associated
Corresponding VR main broadcaster's model into graphics engine, and configure limb action, facial expression and sound and the institute of the performer
Limb action, facial expression and the sound for stating VR main broadcaster's model animation being capable of real-time synchronizations.
As shown in figure 5, the process that preferably progress VR is played in real time is as follows：
S501, one famous actor of setting, which are that its adjustment is dynamic, catches equipment, to catch its limb action, facial expression and sound.Drill
Member can be carried out motion test and expression tested according to the lines and drama planned in advance.Here, it is necessary to enter to the sound of performer
Row processing, is converted to and meets the tone color that the role people sets.Such as, the acoustic tones for recording performer can be changed.Can be with
The excellent basic pronunciation for prerecording the role of sound is selected in advance, the voice recognition of performer is then gone out into word, then again with advance
The basic pronunciation for recording the role is combined into the pronunciation of role.
S502 and graphics engine carry out joint debugging.The limb action, facial expression and the sound that catch the performer are converted to
Limb action data, facial motion data and the role's audio mixing data of association of characteristics are set with the people of the 3D role, are then associated
Corresponding VR main broadcaster's model into graphics engine, and configure limb action, facial expression and sound and the institute of the performer
Limb action, facial expression and the sound for stating VR main broadcaster's model animation being capable of real-time synchronizations.If encountered problems, return to step
S501 adjustment is dynamic to catch equipment.
S503, importing scene and light file.Show effect of shadow for its near vicinity in virtual reality.Preferably,
Scenario parameters and lighting programmers can be pre-configured with, position and the angle of camera image are then calculated in real time, and are configured to
In the virtual 3D role, to interact.
S504, commence play out in playing platform (such as live platform).Allow service end subscriber see VR main broadcaster add to
The picture of VR scenes.
It should be appreciated that embodiments of the invention can be by computer hardware, the combination of hardware and software or by depositing
The computer instruction in non-transitory computer-readable memory is stored up to be effected or carried out.Methods described can use standard
Programming technique-realized including being configured with the non-transitory computer-readable storage media of computer program in computer program,
Wherein so configured storage medium cause computer operated in specific and predefined mode-according to retouching in a particular embodiment
The method and accompanying drawing stated.Each program can be realized with the programming language of level process or object-oriented with computer system
Communication.But, if desired, the program can be realized with compilation or machine language.Under any circumstance, the language can be compiling
Or the language explained.In addition, the program can be run on the application specific integrated circuit of programming for this purpose.
Further, this method can be operably coupled to any types of suitable tomographic data scanning means
Calculating platform in realize, including but not limited to PC, mini-computer, main frame, work station, network or distributed meter
Calculate environment, single or integrated computer platform or communicated with charged particle instrument or other imaging devices etc..This hair
Bright each side can be realized with being stored in the machine readable code in non-transitory storage medium or equipment, either removable
It is dynamic to be also integrated into calculating platform, such as hard disk, optically read and/or write-in storage medium, RAM, ROM so that it can be by
Programmable calculator is read, and can be used for configuring and operate computer to perform when storage medium or equipment are read by computer
Process described by this.In addition, machine readable code, or part thereof can pass through wired or wireless network transmission.As such matchmaker
When body realizes instruction or the program of steps described above including combination microprocessor or other data processors, hair as described herein
It is bright including these and other different types of non-transitory computer-readable storage medias.When according to method of the present invention and
When technology is programmed, present invention additionally comprises computer in itself.
Computer program can be applied to input data to perform function as described herein, so as to change input data with life
Into storing to the output data of nonvolatile memory.Output information can also be applied to one or more output equipments as shown
Device.In the preferred embodiment of the invention, the data of conversion represent physics and tangible object, including the thing produced on display
Reason and the particular visual of physical objects are described.
Particularly referring to Fig. 6-10, the present invention is by a kind of live broadcast system of virtual reality, to implement the above method.This is
System includes application program 60, graphics engine 61, model creation platform 62, dynamic device for catching 63, facial expression catcher 64, images and set
Standby 65, lighting system 66 and special efficacy generation module 67.Dynamic device for catching 63 is used for the limb action of the true performer of real-time capture.Face
Expression catcher 64 is used for the facial expression for catching true performer.Graphics engine 61 is used to generating and handling 3D actor models, transports
Calculate the physical kinetics action of 3D actor models, transition animation video frequency output to video transmission software.Special efficacy generation module 67, can
It is used to coordinate scene to integrate, increases special efficacy.Application program 60, is used for：Virtual 3D angles are created by model creation platform 62
Color, and graphics engine 61 is directed into, generate VR main broadcaster's model；Set by dynamic device for catching 63, facial expression catcher 64 and recording
The real stage business information of standby real-time capture, then associates and controls VR main broadcaster's model；And by VR main broadcaster's mould
Type is blended into VR scenes, is then docked in real time with live platform, generates live picture.
Based on technical scheme, user takes Head-mounted display system (such as VR glasses), passes through multisensor
(gyroscope, gravity sensor etc.) fusion, real-time head tracking and scene fusion, for example, spectators' viewing it is live when can be with
Yaw watches the live place (as shown in Figure 10) of all directions.So, it is greatly improved telepresenc of the spectators for picture.
The preferred embodiments of the present invention are further described below by Fig. 7-10.
The system according to the present invention provides model creation platform first, and user is set drafting according to original painting and people is used for
The virtual role model 1 of VR synthesis, as shown to the left in figure 7.Need to handle various limbs submodels, head during generation actor model
Hair, clothing, then color matching, textures and configuration material.Then the details to actor model is adjusted, and it is reached original as far as possible
If in romantic charm.Then the corresponding virtual role bone 2 of the actor model 1 is made, is tied on actor model, and adjust power
Weight, makes it have the muscular sensation of the mankind as real as possible.This is similar with the principle of skeleton cartoon, due to skeletal system and model
It is separate, in order to allow bone energy driving model to produce rational amoeboid movement.Model is associated with bone, occurred
Relation is binding.Bone comes each face of Controlling model using covering controller as intermediary.Coverage of each joint to model
Controlled and adjusted by weight.If it is intended to changing the scope of the influence model surface in each joint, then can be by many
It is secondary to change weight to realize.Covering is exactly briefly：The point on model, match on bone, then with the motion of bone
Drive model sport.Preferably, the facial expression 10 of actor model can also be equally made by the way of Skeletal Skinned, this
The method of kind is more flexible, can easily realize a variety of expressions.First for mask create bone, for example can to eyebrow, eyelid,
Bone has been respectively created in cheek, nose, lip, lower jaw.After side bone is created, mirror image is to opposite side, it should be noted that
There is a main bone, be so more convenient in brush weight.Second step, selects bone successively, then adds modeling type, carries out
Covering.Then animation is carried out to bone, to complete various expressions.
Further, it is also possible to set the physical motion parameter of the body and hair of actor model, clothes.As shown in figure 8, being angle
The hair 11 of color 1, necktie 12, the configuration of skirt 13 swing parameter, such as in the environment of wind, these objects can be transported according to physics
Dynamic rule is swung.
Then, the actor model of above-mentioned VR role, bone and corresponding configuration parameter are imported into graphics engine.
As shown in figure 9, making a real performer put on facial expression catcher 64, dynamic device for catching 63 (ratio is then dressed
In this way motion capture take) catch performer limb motion.Here technology (such as industry can be caught using existing facial expression is dynamic in real time
Interior Face Moca software products) implement.The data transfer that dynamic device for catching 63 and facial expression catcher 64 are gathered is to figure
Engine 61, so as to associate the model of control VR role.As in Fig. 9, performer's right hand shows " scissors hand " action and performance is sold and sprouts table
After feelings, graphics engine can calculate facial characteristics and limbs skeleton character, then match the model of VR role, control the mould
Type makes corresponding action.
In the present embodiment, backstage performer (Fig. 9) " variable body " turns into virtual VR main broadcaster (Figure 10).The solution of the present invention energy
Enough realize the immediate interactive of VR main broadcaster, it is not necessary to which the good VR animations of pre-production, spectators ask a question, backstage performer answers or done
Make, it is possible to control corresponding VR role and audience interaction, obtain and preferably experience when participating in the cintest.Moreover, it is also possible to according to action need
Want, the special efficacy of the various virtual realities of generation, such as selling in Figure 10, which sprouts expression, can trigger " flash of light " special efficacy.
Those skilled in the art will readily occur to its of the disclosure after considering specification and putting into practice invention disclosed herein
Its embodiment.The disclosure is intended to any modification, purposes or the adaptations of the disclosure, these modifications, purposes or
Person's adaptations follow the general principle of the disclosure and including the undocumented common knowledge in the art of the disclosure
Or conventional techniques.Description and embodiments are considered only as exemplary, and the true scope of the disclosure and spirit are by following
Claim point out.
The preferred embodiment of the disclosure is the foregoing is only, not to limit the disclosure, all essences in the disclosure
God is with principle, and any modification, equivalent substitution and improvements done etc. should be included within the scope of disclosure protection.
Claims (10)
1. a kind of live broadcasting method of virtual reality, it is characterised in that comprise the following steps：
The virtual 3D role of A, establishment, is directed into graphics engine, obtains VR main broadcaster's model；
The real stage business information of B, real-time capture, then associates and controls VR main broadcaster's model；
C, VR main broadcaster's model is blended into VR scenes, then docked in real time with live platform, generate live picture；
Wherein, the action message includes limb action, facial expression and sound.
2. the live broadcasting method of virtual reality according to claim 1, wherein the step A includes：
It is that the virtual role configuration people sets performance data, people sets performance data and includes occupational information, property according to original painting data
Lattice information or personage's background characteristics, and be VR main broadcaster's model configuration textures, material and role animation bone.
3. the live broadcasting method of virtual reality according to claim 2, wherein the step A also includes：
The role animation bone of VR main broadcaster's model and its matching is directed into computing skeleton cartoon in the graphics engine.
4. the live broadcasting method of virtual reality according to claim 1, wherein the step B includes：
Limb action, facial expression and the sound of the performer is caught, the people be converted to the role sets the limb of association of characteristics
Body action data, facial motion data and role's audio mixing data, then associate corresponding VR main broadcaster's model into graphics engine,
And configure limb action, facial expression and the sound of the performer and limb action, the face of VR main broadcaster's model animation
Expression and sound being capable of real-time synchronizations.
5. the live broadcasting method of virtual reality according to claim 4, wherein the step B also includes：
The skeleton model of the performer is extracted, is then introduced into the graphics engine, to match the angle of VR main broadcaster's model
Color animation bone；
According to the skeleton model come real-time capture and the limb action data of conversion performer, by the limb action data of real-time capture
Action directive is generated, the movement posture of corresponding VR main broadcaster's model is generated by the graphics engine；
Computing generates limbs animation between the movement posture of VR main broadcaster's model.
6. the live broadcasting method of virtual reality according to claim 4, wherein the step B also includes：
The facial skeleton of the performer is extracted, is then introduced into the graphics engine, to match the face of VR main broadcaster's model
Portion's Skeletal Skinned；
According to the facial skeleton come real-time capture and the facial motion data of conversion performer, by the facial motion data of real-time capture
Facial expression control instruction is generated, the facial expression shape of corresponding VR main broadcaster's model is generated by the graphics engine；
Computing generates facial expression animation transition between the facial expression shape of the corresponding facial positions of VR main broadcaster's model.
7. the live broadcasting method of virtual reality according to claim 1, wherein the step C includes：
Light and music are added, is controlled by instructor in broadcasting's panel of VR background process clients；
Position and the angle of camera image are calculated in real time, and is configured in the virtual role, to interact and answer
To shooting；
The picture composition that multiple video cameras are shot generates panorama display picture, is then forwarded to VR background process clients with life
Shown into panorama.
8. a kind of live broadcast device of virtual reality, it is characterised in that including：
First module, the virtual role for creating, and graphics engine is directed into, generate VR main broadcaster's model；
Second module, for the real stage business information of real-time capture, then associates and controls VR main broadcaster's model；
3rd module, is blended into VR scenes by VR main broadcaster's model, is then docked in real time with live platform, generates live draw
Face；
Wherein, the action message includes limb action, facial expression and sound.
9. the live broadcast device of virtual reality according to claim 8, first module also includes capture module, it is used for：
Limb action, facial expression and the sound of the performer is caught, the people be converted to the role sets the limb of association of characteristics
Body action data, facial motion data and role's audio mixing data, then associate corresponding VR main broadcaster's model into graphics engine,
And configure limb action, facial expression and the sound of the performer and limb action, the face of VR main broadcaster's model animation
Expression and sound being capable of real-time synchronizations.
10. a kind of live broadcast system of virtual reality, it is characterised in that including：
Dynamic device for catching, the limb action for the true performer of real-time capture；
Facial expression catcher, the facial expression for catching true performer；
Graphics engine, for generating and handling actor model, the physical kinetics action of computing actor model, transition animation video
Export to video transmission software；
Light and special efficacy generation module, for coordinating illumination and VR scenes, increase special efficacy；And
Application program, the application program is used to perform following steps：
Virtual role is created, and is directed into graphics engine, VR main broadcaster's model is generated；
The real stage business information of real-time capture, then associates and controls VR main broadcaster's model；
VR main broadcaster's model is blended into VR scenes, then docked in real time with live platform, live picture is generated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710400733.8A CN107277599A (en) | 2017-05-31 | 2017-05-31 | A kind of live broadcasting method of virtual reality, device and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710400733.8A CN107277599A (en) | 2017-05-31 | 2017-05-31 | A kind of live broadcasting method of virtual reality, device and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107277599A true CN107277599A (en) | 2017-10-20 |
Family
ID=60064831
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710400733.8A Pending CN107277599A (en) | 2017-05-31 | 2017-05-31 | A kind of live broadcasting method of virtual reality, device and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107277599A (en) |
Cited By (37)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108108026A (en) * | 2018-01-18 | 2018-06-01 | 珠海金山网络游戏科技有限公司 | A kind of VR virtual realities motion capture system and motion capture method |
| CN108305308A (en) * | 2018-01-12 | 2018-07-20 | 北京蜜枝科技有限公司 | It performs under the line of virtual image system and method |
| CN108564643A (en) * | 2018-03-16 | 2018-09-21 | 中国科学院自动化研究所 | Performance based on UE engines captures system |
| CN108564642A (en) * | 2018-03-16 | 2018-09-21 | 中国科学院自动化研究所 | Unmarked performance based on UE engines captures system |
| CN108769802A (en) * | 2018-06-21 | 2018-11-06 | 北京密境和风科技有限公司 | Implementation method, the device and system of network performance |
| CN108810561A (en) * | 2018-06-21 | 2018-11-13 | 珠海金山网络游戏科技有限公司 | A kind of three-dimensional idol live broadcasting method and device based on artificial intelligence |
| CN108833740A (en) * | 2018-06-21 | 2018-11-16 | 珠海金山网络游戏科技有限公司 | A kind of real-time word extractor method and apparatus based on three-dimensional animation live streaming |
| CN108848286A (en) * | 2018-06-21 | 2018-11-20 | 珠海金山网络游戏科技有限公司 | Three-dimensional idol live broadcasting method, apparatus and system based on wire ancillary equipment |
| CN108961368A (en) * | 2018-06-21 | 2018-12-07 | 珠海金山网络游戏科技有限公司 | The method and system of real-time live broadcast variety show in three-dimensional animation environment |
| CN108965746A (en) * | 2018-07-26 | 2018-12-07 | 北京竞业达数码科技股份有限公司 | Image synthesizing method and system |
| CN108986190A (en) * | 2018-06-21 | 2018-12-11 | 珠海金山网络游戏科技有限公司 | A kind of method and system of the virtual newscaster based on human-like persona non-in three-dimensional animation |
| CN108986189A (en) * | 2018-06-21 | 2018-12-11 | 珠海金山网络游戏科技有限公司 | Method and system based on real time multi-human motion capture in three-dimensional animation and live streaming |
| CN109035415A (en) * | 2018-07-03 | 2018-12-18 | 百度在线网络技术（北京）有限公司 | Processing method, device, equipment and the computer readable storage medium of dummy model |
| CN109104619A (en) * | 2018-09-28 | 2018-12-28 | 联想(北京)有限公司 | Image processing method and device for live streaming |
| CN109447020A (en) * | 2018-11-08 | 2019-03-08 | 郭娜 | Exchange method and system based on panorama limb action |
| CN109637518A (en) * | 2018-11-07 | 2019-04-16 | 北京搜狗科技发展有限公司 | Virtual newscaster's implementation method and device |
| CN109740476A (en) * | 2018-12-25 | 2019-05-10 | 北京琳云信息科技有限责任公司 | Instant communication method, device and server |
| CN109876450A (en) * | 2018-12-14 | 2019-06-14 | 深圳壹账通智能科技有限公司 | Implementation method, server, computer equipment and storage medium based on AR game |
| CN109922355A (en) * | 2019-03-29 | 2019-06-21 | 广州虎牙信息科技有限公司 | Virtual image live broadcasting method, virtual image live broadcast device and electronic equipment |
| WO2019127369A1 (en) * | 2017-12-29 | 2019-07-04 | 腾讯科技（深圳）有限公司 | Live broadcast sharing method, and related device and system |
| CN110472099A (en) * | 2018-05-10 | 2019-11-19 | 腾讯科技（深圳）有限公司 | Interdynamic video generation method and device, storage medium |
| CN110503707A (en) * | 2019-07-31 | 2019-11-26 | 北京毛毛虫森林文化科技有限公司 | A kind of true man's motion capture real-time animation system and method |
| CN110650354A (en) * | 2019-10-12 | 2020-01-03 | 苏州大禹网络科技有限公司 | Live broadcast method, system, equipment and storage medium for virtual cartoon character |
| CN110865809A (en) * | 2019-11-14 | 2020-03-06 | 北京西山居互动娱乐科技有限公司 | Method and device for importing data into illusion engine |
| CN111193957A (en) * | 2018-11-14 | 2020-05-22 | 技嘉科技股份有限公司 | Method for analyzing performer film and method for increasing performance effect |
| CN111200747A (en) * | 2018-10-31 | 2020-05-26 | 百度在线网络技术（北京）有限公司 | Live broadcasting method and device based on virtual image |
| CN111240482A (en) * | 2020-01-10 | 2020-06-05 | 北京字节跳动网络技术有限公司 | Special effect display method and device |
| CN111630573A (en) * | 2018-02-06 | 2020-09-04 | 雅马哈株式会社 | Information processing method |
| CN111970535A (en) * | 2020-09-25 | 2020-11-20 | 魔珐(上海)信息科技有限公司 | Virtual live broadcast method, device, system and storage medium |
| CN112543342A (en) * | 2020-11-26 | 2021-03-23 | 腾讯科技（深圳）有限公司 | Virtual video live broadcast processing method and device, storage medium and electronic equipment |
| WO2021103613A1 (en) * | 2019-11-28 | 2021-06-03 | 北京市商汤科技开发有限公司 | Method and apparatus for driving interactive object, device, and storage medium |
| CN113473159A (en) * | 2020-03-11 | 2021-10-01 | 广州虎牙科技有限公司 | Digital human live broadcast method and device, live broadcast management equipment and readable storage medium |
| CN113487709A (en) * | 2021-07-07 | 2021-10-08 | 上海商汤智能科技有限公司 | Special effect display method and device, computer equipment and storage medium |
| CN114401442A (en) * | 2022-01-14 | 2022-04-26 | 北京字跳网络技术有限公司 | Video live broadcast and special effect control method and device, electronic equipment and storage medium |
| CN114531601A (en) * | 2022-04-24 | 2022-05-24 | 海马云（天津）信息技术有限公司 | Live broadcast method and device |
| CN114638918A (en) * | 2022-01-26 | 2022-06-17 | 武汉艺画开天文化传播有限公司 | Real-time performance capturing virtual live broadcast and recording system |
| CN115619912A (en) * | 2022-10-27 | 2023-01-17 | 深圳市诸葛瓜科技有限公司 | Cartoon character display system and method based on virtual reality technology |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103179437A (en) * | 2013-03-15 | 2013-06-26 | 苏州跨界软件科技有限公司 | System and method for recording and playing virtual character videos |
| CN104658038A (en) * | 2015-03-12 | 2015-05-27 | 南京梦宇三维技术有限公司 | Method and system for producing three-dimensional digital contents based on motion capture |
| CN104867176A (en) * | 2015-05-05 | 2015-08-26 | 中国科学院自动化研究所 | Cryengine-based interactive virtual deduction system |
| CN106375347A (en) * | 2016-11-18 | 2017-02-01 | 上海悦野健康科技有限公司 | Tourism live broadcast platform based on virtual reality |
| CN106385576A (en) * | 2016-09-07 | 2017-02-08 | 深圳超多维科技有限公司 | Three-dimensional virtual reality live method and device, and electronic device |
| CN106648071A (en) * | 2016-11-21 | 2017-05-10 | 捷开通讯科技(上海)有限公司 | Social implementation system for virtual reality |
-
2017
- 2017-05-31 CN CN201710400733.8A patent/CN107277599A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103179437A (en) * | 2013-03-15 | 2013-06-26 | 苏州跨界软件科技有限公司 | System and method for recording and playing virtual character videos |
| CN104658038A (en) * | 2015-03-12 | 2015-05-27 | 南京梦宇三维技术有限公司 | Method and system for producing three-dimensional digital contents based on motion capture |
| CN104867176A (en) * | 2015-05-05 | 2015-08-26 | 中国科学院自动化研究所 | Cryengine-based interactive virtual deduction system |
| CN106385576A (en) * | 2016-09-07 | 2017-02-08 | 深圳超多维科技有限公司 | Three-dimensional virtual reality live method and device, and electronic device |
| CN106375347A (en) * | 2016-11-18 | 2017-02-01 | 上海悦野健康科技有限公司 | Tourism live broadcast platform based on virtual reality |
| CN106648071A (en) * | 2016-11-21 | 2017-05-10 | 捷开通讯科技(上海)有限公司 | Social implementation system for virtual reality |
Cited By (50)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019127369A1 (en) * | 2017-12-29 | 2019-07-04 | 腾讯科技（深圳）有限公司 | Live broadcast sharing method, and related device and system |
| CN108305308A (en) * | 2018-01-12 | 2018-07-20 | 北京蜜枝科技有限公司 | It performs under the line of virtual image system and method |
| CN108108026A (en) * | 2018-01-18 | 2018-06-01 | 珠海金山网络游戏科技有限公司 | A kind of VR virtual realities motion capture system and motion capture method |
| CN111630573A (en) * | 2018-02-06 | 2020-09-04 | 雅马哈株式会社 | Information processing method |
| CN111630573B (en) * | 2018-02-06 | 2023-11-03 | 雅马哈株式会社 | Information processing method |
| CN108564643A (en) * | 2018-03-16 | 2018-09-21 | 中国科学院自动化研究所 | Performance based on UE engines captures system |
| CN108564642A (en) * | 2018-03-16 | 2018-09-21 | 中国科学院自动化研究所 | Unmarked performance based on UE engines captures system |
| CN110472099A (en) * | 2018-05-10 | 2019-11-19 | 腾讯科技（深圳）有限公司 | Interdynamic video generation method and device, storage medium |
| CN108769802A (en) * | 2018-06-21 | 2018-11-06 | 北京密境和风科技有限公司 | Implementation method, the device and system of network performance |
| CN108961368A (en) * | 2018-06-21 | 2018-12-07 | 珠海金山网络游戏科技有限公司 | The method and system of real-time live broadcast variety show in three-dimensional animation environment |
| CN108986190A (en) * | 2018-06-21 | 2018-12-11 | 珠海金山网络游戏科技有限公司 | A kind of method and system of the virtual newscaster based on human-like persona non-in three-dimensional animation |
| CN108986189A (en) * | 2018-06-21 | 2018-12-11 | 珠海金山网络游戏科技有限公司 | Method and system based on real time multi-human motion capture in three-dimensional animation and live streaming |
| CN108848286A (en) * | 2018-06-21 | 2018-11-20 | 珠海金山网络游戏科技有限公司 | Three-dimensional idol live broadcasting method, apparatus and system based on wire ancillary equipment |
| CN108833740B (en) * | 2018-06-21 | 2021-03-30 | 珠海金山网络游戏科技有限公司 | Real-time prompter method and device based on three-dimensional animation live broadcast |
| CN108833740A (en) * | 2018-06-21 | 2018-11-16 | 珠海金山网络游戏科技有限公司 | A kind of real-time word extractor method and apparatus based on three-dimensional animation live streaming |
| CN108810561A (en) * | 2018-06-21 | 2018-11-13 | 珠海金山网络游戏科技有限公司 | A kind of three-dimensional idol live broadcasting method and device based on artificial intelligence |
| CN108986189B (en) * | 2018-06-21 | 2023-12-19 | 武汉金山世游科技有限公司 | Method and system for capturing and live broadcasting of real-time multi-person actions based on three-dimensional animation |
| CN109035415A (en) * | 2018-07-03 | 2018-12-18 | 百度在线网络技术（北京）有限公司 | Processing method, device, equipment and the computer readable storage medium of dummy model |
| CN108965746A (en) * | 2018-07-26 | 2018-12-07 | 北京竞业达数码科技股份有限公司 | Image synthesizing method and system |
| CN109104619A (en) * | 2018-09-28 | 2018-12-28 | 联想(北京)有限公司 | Image processing method and device for live streaming |
| CN111200747A (en) * | 2018-10-31 | 2020-05-26 | 百度在线网络技术（北京）有限公司 | Live broadcasting method and device based on virtual image |
| CN109637518B (en) * | 2018-11-07 | 2022-05-24 | 北京搜狗科技发展有限公司 | Virtual anchor implementation method and device |
| CN109637518A (en) * | 2018-11-07 | 2019-04-16 | 北京搜狗科技发展有限公司 | Virtual newscaster's implementation method and device |
| CN109447020A (en) * | 2018-11-08 | 2019-03-08 | 郭娜 | Exchange method and system based on panorama limb action |
| CN111193957A (en) * | 2018-11-14 | 2020-05-22 | 技嘉科技股份有限公司 | Method for analyzing performer film and method for increasing performance effect |
| CN109876450A (en) * | 2018-12-14 | 2019-06-14 | 深圳壹账通智能科技有限公司 | Implementation method, server, computer equipment and storage medium based on AR game |
| CN109740476A (en) * | 2018-12-25 | 2019-05-10 | 北京琳云信息科技有限责任公司 | Instant communication method, device and server |
| CN109922355A (en) * | 2019-03-29 | 2019-06-21 | 广州虎牙信息科技有限公司 | Virtual image live broadcasting method, virtual image live broadcast device and electronic equipment |
| CN110503707A (en) * | 2019-07-31 | 2019-11-26 | 北京毛毛虫森林文化科技有限公司 | A kind of true man's motion capture real-time animation system and method |
| CN110650354A (en) * | 2019-10-12 | 2020-01-03 | 苏州大禹网络科技有限公司 | Live broadcast method, system, equipment and storage medium for virtual cartoon character |
| CN110865809B (en) * | 2019-11-14 | 2023-05-09 | 珠海金山数字网络科技有限公司 | Method and device for importing data into illusion engine |
| CN110865809A (en) * | 2019-11-14 | 2020-03-06 | 北京西山居互动娱乐科技有限公司 | Method and device for importing data into illusion engine |
| WO2021103613A1 (en) * | 2019-11-28 | 2021-06-03 | 北京市商汤科技开发有限公司 | Method and apparatus for driving interactive object, device, and storage medium |
| CN111240482A (en) * | 2020-01-10 | 2020-06-05 | 北京字节跳动网络技术有限公司 | Special effect display method and device |
| CN113473159A (en) * | 2020-03-11 | 2021-10-01 | 广州虎牙科技有限公司 | Digital human live broadcast method and device, live broadcast management equipment and readable storage medium |
| CN113473159B (en) * | 2020-03-11 | 2023-08-18 | 广州虎牙科技有限公司 | Digital person live broadcast method and device, live broadcast management equipment and readable storage medium |
| CN111970535B (en) * | 2020-09-25 | 2021-08-31 | 魔珐(上海)信息科技有限公司 | Virtual live broadcast method, device, system and storage medium |
| US11785267B1 (en) | 2020-09-25 | 2023-10-10 | Mofa (Shanghai) Information Technology Co., Ltd. | Virtual livestreaming method, apparatus, system, and storage medium |
| CN111970535A (en) * | 2020-09-25 | 2020-11-20 | 魔珐(上海)信息科技有限公司 | Virtual live broadcast method, device, system and storage medium |
| CN112543342A (en) * | 2020-11-26 | 2021-03-23 | 腾讯科技（深圳）有限公司 | Virtual video live broadcast processing method and device, storage medium and electronic equipment |
| WO2022111110A1 (en) * | 2020-11-26 | 2022-06-02 | 腾讯科技（深圳）有限公司 | Virtual video livestreaming processing method and apparatus, storage medium, and electronic device |
| CN112543342B (en) * | 2020-11-26 | 2023-03-14 | 腾讯科技（深圳）有限公司 | Virtual video live broadcast processing method and device, storage medium and electronic equipment |
| US11991423B2 (en) | 2020-11-26 | 2024-05-21 | Tencent Technology (Shenzhen) Company Limited | Virtual video live streaming processing method and apparatus, storage medium and electronic device |
| CN113487709A (en) * | 2021-07-07 | 2021-10-08 | 上海商汤智能科技有限公司 | Special effect display method and device, computer equipment and storage medium |
| CN114401442A (en) * | 2022-01-14 | 2022-04-26 | 北京字跳网络技术有限公司 | Video live broadcast and special effect control method and device, electronic equipment and storage medium |
| CN114401442B (en) * | 2022-01-14 | 2023-10-24 | 北京字跳网络技术有限公司 | Video live broadcast and special effect control method and device, electronic equipment and storage medium |
| CN114638918B (en) * | 2022-01-26 | 2023-03-28 | 武汉艺画开天文化传播有限公司 | Real-time performance capturing virtual live broadcast and recording system |
| CN114638918A (en) * | 2022-01-26 | 2022-06-17 | 武汉艺画开天文化传播有限公司 | Real-time performance capturing virtual live broadcast and recording system |
| CN114531601A (en) * | 2022-04-24 | 2022-05-24 | 海马云（天津）信息技术有限公司 | Live broadcast method and device |
| CN115619912A (en) * | 2022-10-27 | 2023-01-17 | 深圳市诸葛瓜科技有限公司 | Cartoon character display system and method based on virtual reality technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107277599A (en) | A kind of live broadcasting method of virtual reality, device and system | |
| CN107248195A (en) | A kind of main broadcaster methods, devices and systems of augmented reality | |
| CN107274465A (en) | A kind of main broadcaster methods, devices and systems of virtual reality | |
| CN107274466A (en) | The methods, devices and systems that a kind of real-time double is caught | |
| CN107274464A (en) | A kind of methods, devices and systems of real-time, interactive 3D animations | |
| CN111970535B (en) | Virtual live broadcast method, device, system and storage medium | |
| CN107154069B (en) | Data processing method and system based on virtual roles | |
| US9667574B2 (en) | Animated delivery of electronic messages | |
| CN107197385A (en) | A kind of real-time virtual idol live broadcasting method and system | |
| CN107170030A (en) | A kind of virtual newscaster's live broadcasting method and system | |
| CN102822869B (en) | Capture view and the motion of the performer performed in the scene for generating | |
| KR101713772B1 (en) | Apparatus and method for pre-visualization image | |
| CN106993195A (en) | Virtual portrait role live broadcasting method and system | |
| CN107194979A (en) | The Scene Composition methods and system of a kind of virtual role | |
| JP2012181704A (en) | Information processor and information processing method | |
| CN107248185A (en) | A kind of virtual emulation idol real-time live broadcast method and system | |
| US11005796B2 (en) | Animated delivery of electronic messages | |
| US11783523B2 (en) | Animation control method and apparatus, storage medium, and electronic device | |
| CN109032339A (en) | A kind of method and system that real-time intelligent body-sensing is synchronous | |
| Nitsche | Experiments in the use of game technology for pre-visualization | |
| JP2004355567A (en) | Image output device, image output method, image output processing program, image distribution server and image distribution processing program | |
| JP2006217183A (en) | Data processor and program for generating multimedia data | |
| US11527032B1 (en) | Systems and methods to generate and utilize content styles for animation | |
| Pejsa et al. | Architecture of an animation system for human characters | |
| Parker | Theater as virtual reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20171020 |