CN112367534B - Virtual-real mixed digital live broadcast platform and implementation method - Google Patents
Virtual-real mixed digital live broadcast platform and implementation method Download PDFInfo
- Publication number
- CN112367534B CN112367534B CN202011251613.4A CN202011251613A CN112367534B CN 112367534 B CN112367534 B CN 112367534B CN 202011251613 A CN202011251613 A CN 202011251613A CN 112367534 B CN112367534 B CN 112367534B
- Authority
- CN
- China
- Prior art keywords
- virtual
- image
- real
- scene
- initial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
- H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
Abstract
The invention discloses a virtual-real mixed digital live broadcast platform which comprises a virtual scene setting unit, a real scene image shooting unit, a virtual scene projection unit, a virtual-real mixed image shooting unit, a virtual-real mixed image display unit and a cloud service center, wherein the virtual scene setting unit, the real scene image shooting unit, the virtual scene projection unit and the virtual-real mixed image shooting unit are arranged at a live broadcast user end, the virtual-real mixed image display unit is arranged at a watching user end, and the cloud service center is arranged at a platform end. The invention provides a function of automatically selecting virtual scenes for live broadcast users, virtual and real scenes selected by the live broadcast users are mixed according to real scene objects currently shot by the live broadcast users, so that the aim of setting up and replacing the live broadcast scenes for the live broadcast users in the live broadcast process is fulfilled, finally, live broadcast images mixed by the virtual and real scenes are transmitted to a watching user terminal for watching by the watching user, the scenes in the live broadcast process are subjected to virtual image projection processing, the scenes can be replaced easily, and time and labor are saved without setting up real objects.
Description
Technical Field
The invention relates to the technical field of live broadcast platforms, in particular to a virtual-real mixed digital live broadcast platform and an implementation method.
Background
The network live broadcast is characterized in that independent signal acquisition equipment (audio and video) is erected on site and led into a broadcast guide terminal (broadcast guide equipment or platform), then the signal acquisition equipment is uploaded to a server through a network and is released to a website for people to watch, the network live broadcast absorbs and continues the advantages of the internet, the network live broadcast is carried out in a video mode, content such as product display, related conferences, background introduction, scheme evaluation, online investigation, conversation interview, online training and the like can be released on the internet on site, and the popularization effect of an activity site is enhanced by the aid of the characteristics of intuition and quickness of the internet, good expression form, rich content, strong interactivity, unlimited region, divisible audience and the like.
Present live network needs the preselection before the shooting to shoot the scene and arrange the scene to make the shooting effect reach the best, but live network's length of time generally is three hours to five hours, because shoot the time consuming and labouring of setting up of scene in such long live broadcast duration, consequently be difficult to change, live broadcast user can only select to change the dress and break through the aesthetic fatigue who watches the user, lead to live broadcast scene comparatively single boring, be difficult to obtain constantly and snatch the effect of watching the user's field of vision.
Disclosure of Invention
The invention aims to provide a virtual-real mixed digital live broadcast platform to solve the technical problems that in the prior art, live broadcast scenes are single and dull, and the effect of capturing and watching the visual field of a user at any time is difficult to obtain.
In order to solve the technical problems, the invention specifically provides the following technical scheme:
a virtual-real mixed digital live broadcast platform comprises a virtual scene setting unit, a real scene image shooting unit, a virtual scene projection unit and a virtual-real mixed image shooting unit which are arranged at a live broadcast user end, a virtual-real mixed image display unit arranged at a watching user end and a cloud service center arranged at a platform end;
the virtual scene setting unit is used for directly broadcasting a virtual scene application submitted to the cloud service center by a user;
the live-action image shooting and recording unit is used for shooting an initial live-action image of a live-action object of a live-action user and uploading the initial live-action image to the cloud service center;
the virtual scene projection unit is used for receiving the final virtual scene image output by the cloud service center and projecting the final virtual scene image to a real scene object of a live broadcast user;
the virtual-real mixed image shooting unit is used for shooting an initial virtual-real mixed image formed by a real scene object of a live broadcast user and a final virtual scene image together, and uploading the initial virtual-real mixed image to the cloud service center;
the virtual-real mixed image display unit is used for receiving the final virtual-real mixed image output from the cloud service center and displaying the final virtual-real mixed image to a watching user;
the cloud service center is used for receiving a virtual scene application of the virtual scene setting unit, calibrating a corresponding initial virtual scene image according to the virtual scene application and coordinates of an initial real scene image from the real scene image shooting unit, and outputting a calibrated final virtual scene image to the virtual scene projection unit; and the virtual-real mixed image display unit is used for receiving the initial virtual-real mixed image from the virtual-real mixed image shooting unit, performing image fusion processing on the initial virtual-real mixed image, and outputting the final virtual-real mixed image after fusion processing to the virtual-real mixed image display unit.
As a preferred scheme of the present invention, the virtual scene setting unit includes a voice receiver for a live user to apply for a virtual scene using a voice and a keyboard receiver for a live user to apply for a virtual scene using a key, the voice receiver receives a virtual scene application voice of the live user, and converts the virtual scene application voice into an application text to upload to the cloud service center; and the keyboard receiver receives the virtual scene keying application text of the live broadcast user and uploads the virtual scene keying application text to the cloud service center.
As a preferred scheme of the present invention, the specific steps of the cloud service center receiving the virtual view application and providing the virtual view image corresponding to the virtual view application are as follows:
a1, a cloud service center receives an application text of a virtual scene application from a virtual scene setting unit;
a2, the cloud service center extracts effective keywords in the application text;
and A3, the cloud service center searches an initial virtual image corresponding to the effective keyword from a plurality of virtual images pre-stored in the cloud service center according to the effective keyword.
As a preferred scheme of the present invention, the specific steps of calibrating the initial virtual image by the cloud service center according to the coordinates of the initial real image from the real image recording unit are as follows:
b1, the cloud service center receives an initial live-action image from the live-action image shooting unit;
b2, the cloud service center cuts and scales the initial virtual scene image into an image with the same specification as the initial real scene image;
b3, the cloud service center respectively performs coordinate processing on the initial real-scene image and the initial virtual-scene image;
and B4, the cloud service center selects a coordinate point which is coincident with the original coordinate point of the initial virtual image from the coordinate points of the initial virtual image to replace the original coordinate point of the initial virtual image to serve as a new coordinate point, and then all coordinates of the initial virtual image are calibrated according to the new coordinate point.
As a preferred embodiment of the present invention, in B3, the cloud service center performs coordinate processing on the initial live-action image and the initial virtual-action image respectively, and the specific steps include:
b301, taking L as a spacing distance on a coordinate horizontal axis and a coordinate vertical axis;
b302, taking the left boundary line of the initial real scene image and the initial virtual scene image as a coordinate longitudinal axis, taking the lower boundary line as a coordinate horizontal axis, and taking the intersection point of the coordinate horizontal axis and the coordinate longitudinal axis as a coordinate origin;
b303, performing horizontal grid drawing according to L on the horizontal axis of coordinates, performing horizontal grid drawing according to L on the vertical axis of coordinates, and marking the horizontal axis on the horizontal axis of coordinates as (x) 1 ,x 2 ,x 3 …,x n ) N is the number of the horizontal grid points, and the vertical coordinate marking on the vertical axis of the coordinate is (y) 1 ,y 2 ,y 3 …,y m ) M is the number of longitudinal grid points;
b304, the coordinate of the initial live-action image is expressed as (x) True n ,y Real m ) The real n is the number of horizontal grid points occupied by the initial real image, the real m is the number of vertical grid points occupied by the initial real image, and the coordinate of the initial virtual image is expressed as (x) Virtual n ,y Virtual m ) The virtual n is the number of horizontal grid points occupied by the initial virtual scene image, and the virtual m is occupied by the initial virtual scene imageThe number of the vertical grid points.
In a preferred embodiment of the present invention, in B4, a coordinate point where the origin of coordinates (0, 0) of the initial real-view image and the initial virtual-view image coincide is (x) t ,y p ) The initial virtual scene image is represented by a coordinate point (x) t ,y p ) The result is calibrated for the new origin of coordinates as (x) Virtual n -x t ,y Virtual m -y p ) The final virtual scene image coordinate is expressed as (x) Virtual n -x t ,y Virtual m -y p )。
As a preferred aspect of the present invention, the projection angle of the virtual image projection unit is adjusted such that the origin of coordinates of the projected image coincides with the origin of coordinates of the initial real image, and the virtual-real mixed image obtained by projecting the final virtual image onto the real object through the virtual image projection unit has a uniform coordinate system.
As a preferred scheme of the present invention, the live-action image recording unit and the virtual-real mixed image recording unit are both cameras, the virtual-action projection unit is a projector, the virtual-real mixed image display unit is an intelligent display terminal, the intelligent display terminal is a television, a computer or an intelligent mobile phone, and the cloud service center is a distributed data processing system with operation processing and data storage functions, which is constructed by a plurality of servers and a computing host.
As a preferred scheme of the present invention, the present invention provides an implementation method of a digital live broadcast platform according to the virtual-real mixture, which includes the following steps:
s1, a live broadcast user submits a virtual scene application to a cloud service center through a virtual scene setting unit;
s2, the live-action image shooting and recording unit shoots an initial live-action image of a live-action object of the live-action user and uploads the initial live-action image to the cloud service center;
s3, the cloud service center receives a virtual scene application of the virtual scene setting unit, calibrates a corresponding initial virtual scene image according to the virtual scene application and the coordinates of an initial real scene image from the real scene image shooting unit, and outputs a calibrated final virtual scene image to the virtual scene projection unit;
s4, receiving a final virtual image output by the cloud service center by a virtual image projection unit, and projecting the final virtual image to a live-action object of a live-action user;
s5, shooting an initial virtual-real mixed image formed by a real scene object of a live broadcast user and a final virtual scene image by a virtual-real mixed image shooting unit, and uploading the initial virtual-real mixed image to a cloud service center;
s6, the cloud service center receives the initial virtual-real mixed image from the virtual-real mixed image shooting unit, performs image fusion processing on the initial virtual-real mixed image, and outputs the final virtual-real mixed image after fusion processing to a virtual-real mixed image display unit;
and S7, the virtual and real mixed image display unit receives the final virtual and real mixed image output from the cloud service center and displays the final virtual and real mixed image to a watching user.
Compared with the prior art, the invention has the following beneficial effects:
the invention provides a function of automatically selecting a virtual scene for a live broadcast user, virtual and real mixing is carried out on a virtual live scene selected by the live broadcast user according to a live scene object currently shot by the live broadcast user, so that the aim of building and replacing the live broadcast scene for the live broadcast user in the live broadcast process is fulfilled, finally, a live broadcast image mixed with the virtual and real is transmitted to a viewing user terminal for viewing by the viewing user, the scene in the live broadcast process is subjected to virtual image projection processing, the scene can be replaced easily, time and labor are saved without building a real object, in addition, in the process of projecting the virtual scene image to the live scene object for mixing, the coordinate calibration of the virtual scene image and the live scene object is carried out, the virtual scene image is matched with the current live scene object, the virtual scene image needs to be re-calibrated after the live scene object is replaced, and the private formulation of the virtual scene is carried out for the live broadcast user.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary and that other implementation drawings may be derived from the provided drawings by those of ordinary skill in the art without inventive effort.
Fig. 1 is a flowchart of an implementation method of a virtual-real hybrid digital live broadcast platform according to an embodiment of the present invention;
fig. 2 is a schematic diagram illustrating coordinate representation of an initial live-action image according to an embodiment of the present invention;
fig. 3 is a schematic diagram illustrating coordinate representation of an initial virtual scene image according to an embodiment of the present invention;
fig. 4 is a schematic diagram illustrating a coordinate representation of a final virtual scene image according to an embodiment of the present invention;
fig. 5 is a schematic diagram illustrating a coordinate representation of an initial virtual-real hybrid image according to an embodiment of the present invention.
The reference numerals in the drawings denote the following, respectively:
1-false scene setting unit; 2-live-action image shooting and recording unit; 3-a false scene projection unit; 4-virtual-real mixed image shooting unit; 5-a virtual and real mixed image display unit; 6-a cloud service center; 7-real object.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the present invention provides a virtual-real mixed digital live broadcasting platform, which includes a virtual scene setting unit 1, a real scene image recording unit 2, a virtual scene projection unit 3, a virtual-real mixed image recording unit 4, a virtual-real mixed image display unit 5, and a cloud service center 6, which are disposed at the live broadcasting client, and are disposed at the platform;
the virtual scene setting unit 1 is used for live broadcasting virtual scene applications submitted by users to the cloud service center 6;
the live-action image shooting and recording unit 2 is used for shooting an initial live-action image of a live-action object of a live-action user and uploading the initial live-action image to the cloud service center 6;
the live broadcast method is used for obtaining real scene objects in real scenes of live broadcast users and carrying out virtual scene construction aiming at the current real scene objects, so that the virtual scenes are matched with the real scene objects of the current users, and each live broadcast user can have customized service of virtual-real mixing matched with the real scene objects in the live broadcast user.
The virtual scene projection unit 3 is used for receiving the final virtual scene image output by the cloud service center 6 and projecting the final virtual scene image onto a real scene object of a live broadcast user;
the virtual-real mixed image shooting unit 4 is used for shooting an initial virtual-real mixed image formed by a real scene object of a live broadcast user and a final virtual scene image together, and uploading the initial virtual-real mixed image to the cloud service center 6;
a virtual-real mixed image display unit 5, configured to receive a final virtual-real mixed image output from the cloud service center 6, and display the final virtual-real mixed image to a viewing user;
the cloud service center 6 is used for receiving the virtual scene application of the virtual scene setting unit 1, calibrating the corresponding initial virtual scene image according to the virtual scene application and the coordinates of the initial real scene image from the real scene image shooting and recording unit 2, and outputting the calibrated final virtual scene image to the virtual scene projection unit 3; and the virtual-real mixed image display unit is used for receiving the initial virtual-real mixed image from the virtual-real mixed image recording unit 4, performing image fusion processing on the initial virtual-real mixed image, and outputting the final virtual-real mixed image after fusion processing to the virtual-real mixed image display unit 5.
The virtual scene setting unit 1 comprises a voice receiver for the live broadcast user to use the voice to apply for the virtual scene and a keyboard receiver for the live broadcast user to use the key control to apply for the virtual scene, wherein the voice receiver receives the virtual scene application voice of the live broadcast user, converts the virtual scene application voice into an application text and uploads the application text to the cloud service center 6; the keyboard receiver receives the virtual scene keying application text of the live broadcast user and uploads the virtual scene keying application text to the cloud service center 6.
The cloud service center 6 receives the virtual image application and provides a virtual image corresponding to the virtual image application, which includes the following steps:
a1, a cloud service center 6 receives an application text of a virtual application from a virtual setting unit 1;
a2, the cloud service center 6 extracts effective keywords in the application text;
and A3, the cloud service center 6 searches out an initial virtual image corresponding to the effective keyword from a plurality of virtual images pre-stored in the cloud service center 6 according to the effective keyword.
The cloud service center 6 provides storage and reading functions, and regularly updates a plurality of virtual scene images pre-stored in the cloud service center 6, so that the live broadcast industry with rapid updating is more suitable, and the aesthetic fatigue is reduced.
The specific steps of calibrating the initial virtual image by the cloud service center 6 according to the coordinates of the initial real image from the real image capturing unit 2 are as follows:
b1, the cloud service center 6 receives an initial live-action image from the live-action image shooting unit 2;
b2, the cloud service center 6 cuts and scales the initial virtual image into an image with the same specification as the initial real image;
b3, the cloud service center 6 respectively coordinates the initial real-scene image and the initial virtual-scene image;
and B4, the cloud service center 6 selects a coordinate point which is coincident with the original coordinate point of the initial virtual image from the coordinate points of the initial virtual image to replace the original coordinate point of the initial virtual image to serve as a new coordinate point, and then all coordinates of the initial virtual image are calibrated according to the new coordinate point.
As shown in fig. 2 and 3, in B3, the cloud service center 6 performs the coordinate processing on the initial real-scene image and the initial virtual-scene image respectively, and the specific steps are as follows:
b301, taking L as a spacing distance on a coordinate horizontal axis and a coordinate vertical axis;
b302, taking the left boundary line of the initial real scene image and the initial virtual scene image as a coordinate longitudinal axis, taking the lower boundary line as a coordinate horizontal axis, and taking the intersection point of the coordinate horizontal axis and the coordinate longitudinal axis as a coordinate origin;
b303, performing horizontal grid drawing according to L on the horizontal axis of coordinates, performing horizontal grid drawing according to L on the vertical axis of coordinates, and marking the horizontal axis on the horizontal axis of coordinates as (x) 1 ,x 2 ,x 3 …,x n ) N is the number of the horizontal grid points, and the marking of the vertical coordinate on the vertical axis of the coordinate is (y) 1 ,y 2 ,y 3 …,y m ) M is the number of longitudinal grid points;
b304, the coordinate of the initial live-action image is expressed as (x) True n ,y Real m ) The real n is the number of horizontal grid points occupied by the initial real image, the real m is the number of vertical grid points occupied by the initial real image, and the coordinate of the initial virtual image is expressed as (x) Virtual n ,y Virtual m ) The virtual n is the number of horizontal grid points occupied by the initial virtual scene image, and the virtual m is the number of vertical grid points occupied by the initial virtual scene image.
As shown in fig. 4, in B4, the coordinate point where the origin of coordinates (0, 0) of the initial real-view image and the initial virtual-view image coincide is (x) t ,y p ) The initial virtual scene image is represented by a coordinate point (x) t ,y p ) The result is calibrated for the new origin of coordinates as (x) Virtual n -x t ,y Virtual m -y p ) The final virtual scene image coordinate is expressed as (x) Virtual n -x t ,y Virtual m -y p )。
The projection angle of the virtual image projection unit 3 is adjusted to make the coordinate origin of the projection image coincide with the coordinate origin of the initial real image, and the final virtual image is projected onto the real object through the virtual image projection unit 3 to obtain an initial virtual-real mixed image with a uniform coordinate system, and the initial virtual-real mixed image is as shown in fig. 5, so that the virtual image in the virtual-real mixed image can be matched with the real object to better conform to the real state, and the phenomenon that only image superposition is adopted to shield the real object or reduce the sense of reality is avoided.
The live-action image shooting unit 2 and the virtual-real mixed image shooting unit 4 are cameras, the virtual-action projection unit 3 is a projector, the virtual-real mixed image display unit 5 is an intelligent display terminal, the intelligent display terminal is a television, a computer or an intelligent mobile phone, the intelligent display terminal is provided with a live broadcast platform login portal, such as a software APP and a small program, the cloud service center 6 is a distributed data processing system which is formed by a plurality of servers and a computing host and has the functions of arithmetic processing and data storage, when live broadcast golden time is carried out, live broadcast flow load is greater than that of an original distributed data processing system, only the operation host needs to be accessed into the distributed data processing system to expand the arithmetic quantity and the storage quantity so as to bear large data live broadcast flow, and hardware support is provided for ensuring that each user can obtain live broadcast resources.
The system comprises a virtual scene setting unit 1, a real scene image shooting unit 2, a virtual scene projection unit 3, a virtual and real mixed image shooting unit 4, a virtual and real mixed image display unit 5 and a cloud service center 6 of a platform end of a live broadcast user end, wherein data exchange and service interaction are carried out among the live broadcast user end, the watching user end and the platform end through network communication.
Based on the structure of the virtual-real mixed digital live broadcast platform, the invention provides an implementation method, which comprises the following steps:
s1, a live broadcast user submits a virtual scene application to a cloud service center through a virtual scene setting unit;
s2, the live-action image shooting and recording unit shoots an initial live-action image of a live-action object of the live-action user and uploads the initial live-action image to the cloud service center;
s3, the cloud service center receives a virtual scene application of the virtual scene setting unit, calibrates a corresponding initial virtual scene image according to the virtual scene application and the coordinates of an initial real scene image from the real scene image shooting unit, and outputs a calibrated final virtual scene image to the virtual scene projection unit;
s4, receiving a final virtual image output by the cloud service center by a virtual image projection unit, and projecting the final virtual image to a live-action object of a live-action user;
s5, shooting an initial virtual-real mixed image formed by a real scene object of a live broadcast user and a final virtual scene image by a virtual-real mixed image shooting unit, and uploading the initial virtual-real mixed image to a cloud service center;
s6, the cloud service center receives the initial virtual-real mixed image from the virtual-real mixed image shooting unit, performs image fusion processing on the initial virtual-real mixed image, and outputs the final virtual-real mixed image after fusion processing to a virtual-real mixed image display unit;
and S7, the virtual and real mixed image display unit receives the final virtual and real mixed image output from the cloud service center and displays the final virtual and real mixed image to a watching user.
The invention provides a function of automatically selecting a virtual scene for a live broadcast user, virtual and real mixing is carried out on the virtual real scene selected by the live broadcast user according to a real scene object currently shot by the live broadcast user, so that the aim of building and replacing the live broadcast scene for the live broadcast user in the live broadcast process is achieved, finally, the live broadcast image mixed with the virtual and real is transmitted to a watching user terminal for watching by the watching user, virtual image projection processing is carried out on the scene in the live broadcast process, the scene can be easily replaced, time and labor are saved without building a real object, in addition, in the process of projecting the virtual scene image to the real scene object for mixing, the coordinate calibration of the virtual scene image and the real scene object is carried out, the virtual scene image is matched with the current real scene object, the virtual scene image needs to be re-calibrated after the real scene object is replaced, and the private formulation of the virtual scene is carried out for the live broadcast user.
The above embodiments are only exemplary embodiments of the present application, and are not intended to limit the present application, and the protection scope of the present application is defined by the claims. Various modifications and equivalents may be made by those skilled in the art within the spirit and scope of the present application and such modifications and equivalents should also be considered to be within the scope of the present application.
Claims (8)
1. The utility model provides a live platform of mixed digit of virtuality and reality which characterized in that: the live broadcast video shooting system comprises a virtual scene setting unit (1), a real scene video shooting and recording unit (2), a virtual scene projection unit (3) and a virtual-real mixed video shooting and recording unit (4) which are arranged at a live broadcast user end, a virtual-real mixed video display unit (5) which is arranged at a watching user end, and a cloud service center (6) which is arranged at a platform end;
the virtual scene setting unit (1) is used for directly broadcasting a virtual scene application submitted to the cloud service center (6) by a user;
the live-action image shooting and recording unit (2) is used for shooting an initial live-action image of a live-action object of a live-action user and uploading the initial live-action image to the cloud service center (6);
the virtual scene projection unit (3) is used for receiving the final virtual scene image output by the cloud service center (6) and projecting the final virtual scene image onto a real scene object of a live broadcast user;
a virtual-real mixed image shooting unit (4) for shooting an initial virtual-real mixed image formed by a live-action user real-action object and a final virtual-action image together, and uploading the initial virtual-real mixed image to a cloud service center (6);
the virtual-real mixed image display unit (5) is used for receiving the final virtual-real mixed image output by the cloud service center (6) and displaying the final virtual-real mixed image to a watching user;
the cloud service center (6) is used for receiving the virtual scene application of the virtual scene setting unit (1), calibrating the corresponding initial virtual scene image according to the virtual scene application and the coordinates of the initial real scene image from the real scene image shooting and recording unit (2), and outputting the calibrated final virtual scene image to the virtual scene projection unit (3); the virtual-real mixed image display unit is used for receiving the initial virtual-real mixed image from the virtual-real mixed image shooting unit (4), carrying out image fusion processing on the initial virtual-real mixed image and outputting the final virtual-real mixed image after fusion processing to the virtual-real mixed image display unit (5);
the specific steps of calibrating the initial virtual image by the cloud service center (6) according to the coordinates of the initial real image from the real image shooting unit (2) are as follows:
b1, the cloud service center (6) receives an initial live-action image from the live-action image shooting unit (2);
b2, the cloud service center (6) fuses and scales the initial virtual image into an image with the same specification as the initial real image;
b3, the cloud service center (6) respectively coordinates the initial real-scene image and the initial virtual-scene image;
and B4, the cloud service center (6) selects a coordinate point which is coincident with the original coordinate point of the initial virtual image from the coordinate points of the initial virtual image to replace the original coordinate point of the initial virtual image to serve as a new coordinate point, and then all coordinates of the initial virtual image are calibrated according to the new coordinate point.
2. The virtual-real mixed digital live broadcast platform according to claim 1, wherein: the virtual scene setting unit (1) comprises a voice receiver and a keyboard receiver, wherein the voice receiver is used for a live user to apply for a virtual scene by using voice, the keyboard receiver is used for the live user to apply for the virtual scene by using a key, the voice receiver receives the virtual scene application voice of the live user, converts the virtual scene application voice into an application text and uploads the application text to the cloud service center (6); and the keyboard receiver receives the virtual scene keying application text of the live broadcast user and uploads the virtual scene keying application text to the cloud service center (6).
3. The virtual-real mixed digital live broadcast platform according to claim 2, wherein: the cloud service center (6) receives the virtual application and provides a virtual image corresponding to the virtual application, which comprises the following specific steps:
a1, a cloud service center (6) receives an application text of a virtual scene application from a virtual scene setting unit (1);
a2, the cloud service center (6) extracts effective keywords in the application text;
a3, the cloud service center (6) searches an initial virtual image corresponding to the effective keyword from a plurality of virtual images pre-stored in the cloud service center (6) according to the effective keyword.
4. A virtual-real mixed digital live broadcast platform according to claim 3, characterized in that: in B3, the cloud service center (6) respectively coordinates the initial real-scene image and the initial virtual-scene image, and the method specifically comprises the following steps:
b301, taking L as a spacing distance on a coordinate horizontal axis and a coordinate vertical axis;
b302, taking the left boundary line of the initial real scene image and the initial virtual scene image as a coordinate longitudinal axis, taking the lower boundary line as a coordinate horizontal axis, and taking the intersection point of the coordinate horizontal axis and the coordinate longitudinal axis as a coordinate origin;
b303, performing horizontal grid drawing according to L on the horizontal axis of coordinates, performing horizontal grid drawing according to L on the vertical axis of coordinates, and marking the horizontal axis on the horizontal axis of coordinates as (x) 1 ,x 2 ,x 3 …,x n ) N is the number of the horizontal grid points, and the marking of the vertical coordinate on the vertical axis of the coordinate is (y) 1 ,y 2 ,y 3 …,y m ) M is the number of longitudinal grid points;
b304, the coordinate of the initial live-action image is expressed as (x) True n ,y Real m ) The real n is the number of horizontal grid points occupied by the initial real image, the real m is the number of vertical grid points occupied by the initial real image, and the coordinate of the initial virtual image is represented as (x) Virtual n ,y Virtual m ) The virtual n is the number of horizontal grid points occupied by the initial virtual scene image, and the virtual m is the number of vertical grid points occupied by the initial virtual scene image.
5. The virtual-real mixed digital live broadcast platform of claim 4, wherein: in B4, the coordinate point where the origin of coordinates (0, 0) of the initial real-scene image and the initial virtual-scene image coincide is (x) t ,y p ) The initial virtual scene image is represented by a coordinate point (x) t ,y p ) The result is calibrated for the new origin of coordinates as (x) Virtual n -x t ,y Virtual m -y p ) The final virtual scene image coordinate is expressed as (x) Virtual n -x t ,y Virtual m -y p )。
6. The virtual-real mixed digital live broadcast platform of claim 5, wherein: the projection angle of the virtual image projection unit (3) is adjusted to make the origin of coordinates of the projection image coincide with the origin of coordinates of the initial real image, and the virtual and real mixed image obtained by projecting the final virtual image onto the real object through the virtual image projection unit (3) has a uniform coordinate system.
7. The virtual-real mixed digital live broadcast platform according to claim 1, wherein: the real-scene image shooting and recording unit (2) and the virtual-real mixed image shooting and recording unit (4) are cameras, the virtual-scene projection unit (3) is a projector, the virtual-real mixed image display unit (5) is an intelligent display terminal, the intelligent display terminal is a television, a computer or an intelligent mobile phone, and the cloud service center (6) is a distributed data processing system which is formed by a plurality of servers and a computing host and has the functions of operation processing and data storage.
8. A method for implementing the virtual-real mixed digital live platform according to any one of claims 1 to 7, comprising the following steps:
s1, a live broadcast user submits a virtual scene application to a cloud service center through a virtual scene setting unit;
s2, a live-action image shooting unit shoots an initial live-action image of a live-action object of a live-action user and uploads the initial live-action image to a cloud service center;
s3, the cloud service center receives a virtual scene application of the virtual scene setting unit, calibrates a corresponding initial virtual scene image according to the virtual scene application and the coordinates of an initial real scene image from the real scene image shooting unit, and outputs a calibrated final virtual scene image to the virtual scene projection unit;
s4, receiving a final virtual image output by the cloud service center by a virtual image projection unit, and projecting the final virtual image to a live-action object of a live-action user;
s5, shooting an initial virtual-real mixed image formed by a real scene object of a live broadcast user and a final virtual scene image by a virtual-real mixed image shooting unit, and uploading the initial virtual-real mixed image to a cloud service center;
s6, the cloud service center receives the initial virtual-real mixed image from the virtual-real mixed image shooting unit, performs image fusion processing on the initial virtual-real mixed image, and outputs the final virtual-real mixed image after fusion processing to the virtual-real mixed image display unit;
and S7, the virtual and real mixed image display unit receives the final virtual and real mixed image output from the cloud service center and displays the final virtual and real mixed image to a watching user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011251613.4A CN112367534B (en) | 2020-11-11 | 2020-11-11 | Virtual-real mixed digital live broadcast platform and implementation method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011251613.4A CN112367534B (en) | 2020-11-11 | 2020-11-11 | Virtual-real mixed digital live broadcast platform and implementation method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112367534A CN112367534A (en) | 2021-02-12 |
| CN112367534B true CN112367534B (en) | 2023-04-11 |
Family
ID=74514506
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011251613.4A Active CN112367534B (en) | 2020-11-11 | 2020-11-11 | Virtual-real mixed digital live broadcast platform and implementation method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112367534B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10327350A (en) * | 1997-05-23 | 1998-12-08 | Sony Corp | Image forming device and image forming method |
| CN106296789A (en) * | 2016-08-05 | 2017-01-04 | 深圳迪乐普数码科技有限公司 | A kind of it is virtually implanted method and the terminal that object shuttles back and forth in outdoor scene |
| CN108124194A (en) * | 2017-12-28 | 2018-06-05 | 北京奇艺世纪科技有限公司 | A kind of net cast method, apparatus and electronic equipment |
| CN109660818A (en) * | 2018-12-30 | 2019-04-19 | 广东彼雍德云教育科技有限公司 | A kind of virtual interactive live broadcast system |
| CN208969397U (en) * | 2018-11-21 | 2019-06-11 | 深圳市旅行家科技有限公司 | A kind of virtual background film studio |
| CN109905754A (en) * | 2017-12-11 | 2019-06-18 | 腾讯科技（深圳）有限公司 | Virtual present collection methods, device and storage equipment |
| CN110798634A (en) * | 2019-11-28 | 2020-02-14 | 东北大学 | Image self-adaptive synthesis method and device and computer readable storage medium |
| CN111432235A (en) * | 2020-04-01 | 2020-07-17 | 网易（杭州）网络有限公司 | Live video generation method and device, computer readable medium and electronic equipment |
| CN111880649A (en) * | 2020-06-24 | 2020-11-03 | 合肥安达创展科技股份有限公司 | Demonstration method and system of AR viewing instrument and computer readable storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1862969A1 (en) * | 2006-06-02 | 2007-12-05 | Eidgenössische Technische Hochschule Zürich | Method and system for generating a representation of a dynamically changing 3D scene |
| US20130249947A1 (en) * | 2011-08-26 | 2013-09-26 | Reincloud Corporation | Communication using augmented reality |
-
2020
- 2020-11-11 CN CN202011251613.4A patent/CN112367534B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10327350A (en) * | 1997-05-23 | 1998-12-08 | Sony Corp | Image forming device and image forming method |
| CN106296789A (en) * | 2016-08-05 | 2017-01-04 | 深圳迪乐普数码科技有限公司 | A kind of it is virtually implanted method and the terminal that object shuttles back and forth in outdoor scene |
| CN109905754A (en) * | 2017-12-11 | 2019-06-18 | 腾讯科技（深圳）有限公司 | Virtual present collection methods, device and storage equipment |
| CN108124194A (en) * | 2017-12-28 | 2018-06-05 | 北京奇艺世纪科技有限公司 | A kind of net cast method, apparatus and electronic equipment |
| CN208969397U (en) * | 2018-11-21 | 2019-06-11 | 深圳市旅行家科技有限公司 | A kind of virtual background film studio |
| CN109660818A (en) * | 2018-12-30 | 2019-04-19 | 广东彼雍德云教育科技有限公司 | A kind of virtual interactive live broadcast system |
| CN110798634A (en) * | 2019-11-28 | 2020-02-14 | 东北大学 | Image self-adaptive synthesis method and device and computer readable storage medium |
| CN111432235A (en) * | 2020-04-01 | 2020-07-17 | 网易（杭州）网络有限公司 | Live video generation method and device, computer readable medium and electronic equipment |
| CN111880649A (en) * | 2020-06-24 | 2020-11-03 | 合肥安达创展科技股份有限公司 | Demonstration method and system of AR viewing instrument and computer readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 新闻演播室背景的发展;史晓辉;《卫星电视与宽带多媒体》;20120620(第12期);全文 * |
| 电竞到电视的跨界之作――访上海日向信息技术有限公司技术总监李坤;孙晓超;《现代电视技术》;20181115(第11期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112367534A (en) | 2021-02-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108737882B (en) | Image display method, image display device, storage medium and electronic device | |
| CN110708589B (en) | Information sharing method and device, storage medium and electronic device | |
| US11722639B2 (en) | Method and apparatus for capturing a group photograph during a video conference session | |
| CN105608715A (en) | Online group shot method and system | |
| CN106792230B (en) | Advertisement interaction method and system based on live video | |
| CN103338405A (en) | Screen capture application method, equipment and system | |
| CN104010206B (en) | Based on the method and system of the virtual reality video playback in geographical position | |
| CN113411656B (en) | Information processing method, information processing device, computer equipment and storage medium | |
| CN103002349A (en) | Adaptive adjustment method and device for video playing | |
| CN106303634A (en) | A kind of TV equipment barrage sends system and method | |
| CN108874471B (en) | Method and system for adding additional elements between functions of house resources | |
| CN111569436A (en) | Processing method, device and equipment based on interaction in live broadcast fighting | |
| CN106791915A (en) | A kind of method and apparatus for showing video image | |
| CN105306872B (en) | Control the methods, devices and systems of multipoint videoconference | |
| CN111225287A (en) | Bullet screen processing method and device, electronic equipment and storage medium | |
| CN111464788A (en) | Remote sacrifice method and system | |
| CN106792237B (en) | Message display method and system | |
| CN112367534B (en) | Virtual-real mixed digital live broadcast platform and implementation method | |
| US20170171634A1 (en) | Method and electronic device for pushing reservation message | |
| CN111225223A (en) | Method, system, server and storage medium for optimizing list information of live broadcast platform | |
| CN116208733A (en) | Video conference interaction method and device | |
| CN116962748A (en) | Live video image rendering method and device and live video system | |
| CN107172478A (en) | The display systems of video and audio file | |
| CN107015980A (en) | It is a kind of enter row information displaying method and apparatus | |
| KR20170096341A (en) | System and Method for Broadcasting Bi-directional Software Coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |