CN115278364A - Video stream synthesis method and device - Google Patents
Video stream synthesis method and device Download PDFInfo
- Publication number
- CN115278364A CN115278364A CN202210910457.0A CN202210910457A CN115278364A CN 115278364 A CN115278364 A CN 115278364A CN 202210910457 A CN202210910457 A CN 202210910457A CN 115278364 A CN115278364 A CN 115278364A
- Authority
- CN
- China
- Prior art keywords
- video stream
- parameters
- user
- receiving
- server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001308 synthesis method Methods 0.000 title description 8
- 238000000034 method Methods 0.000 claims abstract description 75
- 230000002194 synthesizing effect Effects 0.000 claims abstract description 40
- 238000012545 processing Methods 0.000 claims description 22
- 238000012937 correction Methods 0.000 claims description 14
- 230000015572 biosynthetic process Effects 0.000 claims description 7
- 238000003786 synthesis reaction Methods 0.000 claims description 7
- 238000012805 post-processing Methods 0.000 claims description 5
- 239000002131 composite material Substances 0.000 abstract description 7
- 230000015654 memory Effects 0.000 description 17
- 238000010586 diagram Methods 0.000 description 12
- 230000006870 function Effects 0.000 description 10
- 239000000203 mixture Substances 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000005540 biological transmission Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 238000011156 evaluation Methods 0.000 description 4
- 230000003993 interaction Effects 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 235000011888 snacks Nutrition 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/265—Mixing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
This specification provides a video stream synthesizing method and apparatus, including: the server receives the input video stream and the initial parameters and generates a virtual scene; synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream; receiving and adjusting the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream; and sending the result video stream to a client; and the client receives and plays the result video stream sent by the server. Based on the method, the problem that the composite video stream in the existing method can not meet the diversified requirements of the user can be solved, the composite video stream with higher interactivity is obtained, and the watching experience of the user is improved.
Description
Technical Field
The present disclosure relates to video processing, and in particular, to a method and an apparatus for synthesizing a video stream.
Background
The video stream synthesis technology is widely applied to the fields of virtual live broadcast, video post-processing and the like. In the prior art, video stream synthesis is usually performed based on a video workstation, but the diversified demands of users cannot be met, and the user experience is poor.
In view of the above technical problems, no effective solution has been proposed at present.
Disclosure of Invention
The present specification provides a video stream synthesis method and apparatus, which can solve the problem that the synthesized video stream in the existing method cannot meet the diversified requirements of the user, and implement the interactive operation between the synthesized video stream and the user.
An object of an embodiment of the present specification is to provide a video stream composition method, which is applied to a server and includes:
receiving an input video stream, initial parameters and generating a virtual scene;
synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream;
receiving and adjusting the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream; and sending the resulting video stream to a client.
Further, in another embodiment of the method, before the receiving and adjusting the first video stream with a virtual camera according to the customized parameter to obtain a result video stream, the method further includes: and sending the first video stream to a client.
Further, in another embodiment of the method, the customizing parameters include: camera parameters, foreground parameters, virtual scene parameters.
Further, in another embodiment of the method, the synthesizing the input video stream and the virtual scene according to the initial parameter to obtain a first video stream includes:
performing background removal processing on the input video stream to obtain a target object;
and synthesizing the target object and the virtual scene according to the initial parameters to obtain a first video stream.
Further, in another embodiment of the method, the receiving and adjusting the first video stream with a virtual camera according to the customized parameter to obtain a result video stream includes:
receiving the user-defined parameters, and configuring the operation parameters of the virtual camera according to the user-defined parameters;
and processing the first video stream by using the configured virtual camera to obtain a result video stream.
Further, in another embodiment of the method, the input video stream comprises a video stream to be presented to a viewer in a virtual live broadcast.
An embodiment of the present specification provides a video stream synthesizing method, which is applied to a client and includes:
sending the input video stream and the initial parameters to a server; the server is used for synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream;
receiving a custom parameter input by a user, and sending the custom parameter to a server; the server is used for receiving and adjusting the first video stream by adopting a virtual camera according to a user-defined parameter to obtain a result video stream;
and receiving and playing the result video stream sent by the server.
Further, in another embodiment of the method, before the receiving a user-defined parameter input by a user and sending the user-defined parameter to a server, the method further includes: and receiving and playing the first video stream.
Further, in another embodiment of the method, after the receiving and playing the resulting video stream sent by the server, the method further includes:
detecting whether a correction parameter is received;
and under the condition that the correction parameters are determined not to be received, inputting the result video stream into a virtual camera to perform post-processing and live streaming.
In another aspect, the present application provides a video stream synthesizing apparatus, including:
the generating module is used for receiving the input video stream and the initial parameters and generating a virtual scene;
the synthesis module is used for synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream;
the adjusting module is used for receiving and adjusting the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream; and sending the resulting video stream to a client.
The video stream synthesis method and device provided by the specification can be applied to a client and a server, wherein the client receives an input video stream and initial parameters input by a user; sending the input video stream and the initial parameters to a server; the server receives the input video stream and the initial parameters and generates a virtual scene; synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream; the client receives a user-defined parameter input by a user and sends the user-defined parameter to the server; the server receives and adjusts the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream; and sending the result video stream to a client; and the client receives and plays the result video stream sent by the server.
When the input video stream and the virtual scene are synthesized according to the initial parameters to obtain a first video stream, performing background removal processing on the input video stream to obtain a target object; and synthesizing the target object and the virtual scene according to the initial parameters to obtain a first video stream.
Further, when receiving and adjusting the first video stream by using a virtual camera according to the user-defined parameters to obtain a result video stream, receiving the user-defined parameters and configuring the operation parameters of the virtual camera according to the user-defined parameters; and processing the first video stream by using the configured virtual camera to obtain a result video stream.
Drawings
In order to more clearly illustrate the embodiments of the present description, the drawings needed for the embodiments will be briefly described below, the drawings in the following description are only some of the embodiments described in the present description, and other drawings may be obtained by those skilled in the art without inventive efforts.
Fig. 1 is a schematic flowchart of an embodiment of a video stream synthesizing method applied to a server according to the present disclosure;
fig. 2 is a schematic flow chart of an embodiment of a video stream synthesizing method applied to a client according to the present disclosure;
fig. 3 is a schematic diagram of a first video stream generation process provided in this specification;
fig. 4 is a schematic diagram of a video stream composition method based on a server according to the present disclosure;
FIG. 5 is a schematic diagram of a client-based video stream composition method provided by the present disclosure;
fig. 6 is a schematic block diagram of an embodiment of a video stream synthesizing apparatus provided in this specification;
fig. 7 is a schematic structural component diagram of a server provided in this specification.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step should fall within the scope of protection of the present specification.
Considering that the existing video stream synthesizing method is usually performed based on a video workstation, the method is high in manufacturing cost.
Further, it is still considered that the synthesized video stream obtained based on the prior art is usually played only based on a fixed angle and a fixed position, and a user cannot perform operations such as angle adjustment and position adjustment on the video stream according to own needs, so that interactive operation with the user cannot be realized, diversified needs of the user cannot be met, and the user experience is poor.
Aiming at the problems existing in the existing method and the specific reasons for the problems, the method for synthesizing the video stream considers that the video stream which can be operated by the user is introduced to realize the adjustment of the video stream and meet the diversified requirements of the user.
Based on the above thought, the present specification proposes a video stream synthesis method. The method is applied to a server side, and comprises the steps of firstly, receiving an input video stream and initial parameters, and generating a virtual scene; then, synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream; finally, receiving and adjusting the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream; and sending the resulting video stream to a client. The server may be a server with a video stream processing function. Specifically, the server may be a cloud server deployed in a cloud.
The method is applied to a client, and firstly, an input video stream and initial parameters input by a user are received; sending the input video stream and the initial parameters to a server; then, receiving a user-defined parameter input by a user, and sending the user-defined parameter to a server; and finally, receiving and playing the result video stream sent by the server. The client can be a terminal device with functions of data acquisition, data transmission and the like. Specifically, the terminal device may be a desktop computer, a tablet computer, a notebook computer, a smart phone, and the like. Alternatively, the terminal device may be a software application that can be run on the electronic device. For example, it may be some APP running on a cell phone, etc.
Referring to fig. 1, an embodiment of the present disclosure provides a video stream synthesizing method, which is applied to a server, and when implemented, the method may include the following.
S101: an input video stream, initial parameters, and a virtual scene are received.
In some embodiments, the video stream is a processed, transmitted video; when the conventional video is processed, the conventional video is completely processed as a whole and then is sent out as a whole, and the mode has low efficiency and low speed; when the video stream is subjected to video processing, the video stream is firstly decomposed into a plurality of small segments according to the playing sequence, the small segments are processed one by one and are sent to a user, and finally a continuous video stream is generated.
In some embodiments, the virtual scene may be specifically understood as a video scene generated based on a virtual engine and a related algorithm, and matched with a target object.
Specifically, the virtual scene may include video scenes of a plurality of different scene types. The scene type may specifically include: shopping scenes, work scenes, entertainment scenes, weather scenes, and the like. It is further possible to include a corresponding plurality of sub-scene types for each of the scene types listed above. For example, the weather scenario described above may further include: a rainy scene, a sunny scene, a snowy scene, etc. The shopping scenario may further include: clothing item scenes, snack item scenes, toy item scenes, and the like.
In some embodiments, before specific implementation, the server may further perform statistical sorting on video scenes in a large number of sample videos, and screen out a plurality of scene types with high use frequency as preset scene types; and generating a scene video related to the preset scene type as a preset virtual scene by using a virtual engine and a related algorithm so as to construct a preset virtual scene library. A plurality of preset virtual scenes can be stored in the preset virtual scene library, and each preset virtual scene corresponds to one virtual scene identifier.
In specific implementation, the server can receive the selection operation data of the user for the scene type of the virtual scene, which is collected by the user through the client; determining a virtual scene identifier corresponding to the virtual scene type selected by the user according to the selection operation data; and then, according to the virtual scene identification, acquiring a matched preset virtual scene from a preset virtual scene library as the virtual scene.
The determined virtual scene identifier may include a plurality of different virtual scene identifiers. Correspondingly, the server side can acquire a plurality of matched preset virtual scenes from a preset virtual scene library to combine to obtain a combined virtual scene; and then taking the combined virtual scene as the virtual scene.
Specifically, for example, a user wants to make a composite video for promoting a raincoat, and selects a rainy day scene and a clothing commodity scene in the client at the same time. Correspondingly, the server side can firstly acquire a preset rainy day virtual scene and a preset clothing commodity virtual scene from a preset virtual scene library; and overlapping the two virtual scenes to obtain a virtual scene meeting the requirements of the user.
In some embodiments, the initial parameter may specifically include: initial angle parameters, initial position parameters and initial camera parameters; specifically, the camera parameters may include: sensitivity parameter, color temperature parameter, focal length parameter, aperture parameter.
Specifically, the initial parameter may be a parameter input by a user through a client, or may be a default parameter provided by a server. The default parameters may be angle parameters, position parameters and camera parameters which are sorted by the server after analyzing and counting a large amount of video data and accord with the preference of most audiences.
S102: and synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream.
In some embodiments, referring to fig. 3, the synthesizing the input video stream and the virtual scene according to the initial parameter to obtain the first video stream may include:
s1: performing background removal processing on the input video stream to obtain a target object;
s2: and synthesizing the target object and the virtual scene according to the initial parameters to obtain a first video stream.
In some embodiments, the target object may specifically include one or more of the following: a target merchandise object, a target character object, a target animal object, etc.
Specifically, for example, the server may determine a target object that needs to be retained by the user in the input video stream through interaction with the client; then, respectively carrying out image recognition on each frame of image in the input video stream to determine a target object in each frame of image; then determining a target area where the target object is located in each frame of image and a background area except the target object; and then segmenting the background area in each frame of image to obtain the required target object.
S103: receiving and adjusting the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream; and sending the resulting video stream to a client.
In some embodiments, the customized parameters may specifically include: camera parameters, foreground parameters, virtual scene parameters; the camera parameters may specifically include: a push-pull rocking distance angle parameter, a lens parameter and a sound parameter; the foreground parameters may specifically include: keying parameters, image transformation parameters and input source sound parameters; the virtual scene parameters may specifically include: ambient lighting parameters, scene item interaction parameters.
The self-defined parameters are used for setting the shooting mode of the virtual camera to obtain a result video stream.
In some embodiments, before receiving and adjusting the first video stream with the virtual camera according to the customized parameter to obtain the resultant video stream, the method further includes: and sending the first video stream to a client.
Specifically, the client may present a first video stream to a user, and receive a user-defined parameter input by the user for the first video stream; and then sending the self-defined parameters to a server.
In addition, the client can also receive feedback evaluation input by the user for the first video stream, and send the feedback evaluation to the server. After receiving the feedback evaluation, the server side can modify the initial parameters according to the feedback evaluation to obtain parameters which can meet the user preference as custom parameters.
In some embodiments, the receiving and adjusting the first video stream with a virtual camera according to the customized parameter to obtain a result video stream may include:
s1: receiving the user-defined parameters, and configuring the operation parameters of the virtual camera according to the user-defined parameters;
s2: and processing the first video stream by using the configured virtual camera to obtain a result video stream.
In some embodiments, the virtual camera is a virtual scene shooting device located in a server and constructed based on three-dimensional engine software, and can realize multi-angle and multi-position video stream shooting in a virtual scene according to user-defined parameters configured by a user. Compared with a real camera, the virtual camera is simpler and more convenient to operate and lower in use cost.
Based on the above embodiment, the user can change the shooting mode of the virtual camera by inputting the custom parameters, and adjust the first video stream (for example, angle adjustment, position adjustment, brightness adjustment) in the virtual scene to generate the resultant video stream meeting the requirements of the client.
Specifically, for example, the input video stream includes a video stream to be presented to a viewer in a virtual live broadcast. That is, the video stream synthesizing method provided in this specification may be applied to process a video stream to be shown to a viewer in a virtual live broadcast, so that the viewer may obtain a better viewing experience.
The method comprises the steps that a live broadcast user uses a client to perform format conversion on a coded video to be displayed to audiences to obtain a video after the format conversion; then according to the corresponding coding rule, the video after format conversion is coded to obtain the coded video meeting the requirement; the encoding rule may specifically be bridge encoding. The bridge coding is a simple packet transfer protocol (RUDP) coding mode based on a reliable data protocol.
Then, the live broadcast user can determine the type of the virtual scene by using the client side and simultaneously send a synthesis instruction to the server side; wherein the composition instruction carries the encoded video and the initial parameters.
Then, the server side can generate a corresponding virtual scene according to the virtual scene type selected by the live broadcast user; meanwhile, decoding processing is carried out on the coded video to obtain an input video stream; and responding to the synthesis instruction, and synthesizing the virtual scene and the input video stream according to the initial parameters to obtain a first synthesized video (or called a first video stream). Further, the server side can also obtain an adjustment instruction of the live broadcast user for the first composite video. The adjusting instruction at least carries a custom parameter of a live user. Then, in response to the adjustment instruction, the first composite video may be adjusted accordingly by using the custom parameter, so as to obtain a second composite video (or called a resultant video stream). The second composite video may then be played to the viewer.
Referring to fig. 2, an embodiment of the present disclosure provides a video stream synthesizing method, which is applied to a client and may include the following contents when implemented.
S201: sending the input video stream and the initial parameters to a server; and the server is used for synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream.
In some embodiments, the input video stream may be a real-time video captured by a video capture device, or may be a video file that has already been captured.
S202: receiving a user-defined parameter input by a user, and sending the user-defined parameter to a server; and the server is used for receiving and adjusting the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream.
In some embodiments, before the receiving the user-input custom parameter and sending the custom parameter to the server, the method further includes: and receiving and playing the first video stream.
In a specific scenario example, the client may generate an input page while playing the first video stream, and the user may input various customized parameters in a corresponding text box of the input page according to the need of the user while watching the first video stream, so as to adjust the first video stream. Or, the user may also perform a mouse or finger dragging action on the first video stream played by the client, and the client captures a dragging action path of the user, calculates a corresponding push-pull rocking distance angle parameter according to the dragging action path, and sends the corresponding push-pull rocking distance angle parameter to the server to adjust the first video stream.
Based on the embodiment, the user can watch the first video stream at the client, and then inputs the custom parameters to adjust the first video stream according to the own needs.
S203: and receiving and playing the result video stream sent by the server.
In some embodiments, after receiving and playing the result video stream sent by the server, the method further includes:
s1: detecting whether correction parameters are received;
s2: and under the condition that the correction parameters are determined not to be received, inputting the result video stream into a virtual camera so as to perform post-processing and live streaming.
In some embodiments, the correction parameters may specifically include: camera parameters, foreground parameters, virtual scene parameters; the camera parameters may specifically include: a push-pull rocking distance angle parameter, a lens parameter and a sound parameter; the foreground parameters may specifically include: keying parameters, image transformation parameters and input source sound parameters; the virtual scene parameters may specifically include: ambient lighting parameters, scene item interaction parameters.
In some embodiments, based on the above correction parameters, correction adjustment may be performed on the resulting video stream received by the user, so as to meet diversified needs of the user. For example: the user inputs the correction parameters, and the client receives the correction parameters and sends the correction parameters to the server; and the server receives the correction parameters, reconfigures the virtual camera according to the correction parameters, generates a new result video stream by shooting with the reconfigured virtual camera, and sends the new result video stream to the client.
In some embodiments, the virtual camera-based video stream can be imported into a third-party live platform for live streaming. In other embodiments, the virtual camera-based camera may also perform video post-processing on the resulting video stream, such as cut-merge, cut-crop on the resulting video stream. In other embodiments, the virtual camera-based video stream may also be recorded and saved.
Specifically, for example, the client may be an APP running on a mobile phone, and the user may view the resulting video stream in the APP of the mobile phone; further, the result video stream can be guided into a virtual camera and a third-party live broadcast platform is opened, the virtual camera can complete interaction with the third-party live broadcast platform, and the third-party live broadcast platform can acquire the result video stream in the virtual camera and carry out live broadcast in the live broadcast platform so as to display the result video stream to audiences; the third-party live broadcast platform specifically may include: e-commerce live broadcast platform and social live broadcast platform. For another example, the result video stream may be simultaneously imported into a plurality of virtual cameras, and accordingly, a plurality of third-party live broadcast platforms are opened, and each third-party live broadcast platform acquires the result video stream from a different virtual camera and performs live broadcast in the live broadcast platform, thereby realizing multi-channel output of the result video stream.
In a specific scene example, referring to fig. 4, fig. 4 is a schematic diagram of a video stream synthesis method based on a server provided in this specification, in which a virtual scene is started by the server, and then it is detected whether a user inputs a video stream through a client, in the case that the user inputs the video stream, a target object (for example, a target character object, a target commodity object, a target animal object, etc.) in the input video stream is extracted by removing a background frame by frame, and then the input video stream and the virtual scene are synthesized according to initial parameters to obtain a first video stream; correspondingly, the first video stream and the virtual scene can be synthesized and adjusted according to the user-defined parameters to obtain a result video stream. Referring to fig. 5, fig. 5 is a schematic diagram illustrating a video stream synthesis method based on a client according to the present disclosure, where a cloud in the diagram may be a server device deployed in the cloud, and first the client receives a first video stream returned by the cloud and determines whether adjustment is needed; under the condition of needing to be adjusted, a user can input a self-defining parameter through the client and send a corresponding control instruction (such as the self-defining parameter) to the cloud; the cloud returns the adjusted result video stream; the resulting video stream may be used when the user believes that the resulting video stream no longer requires adjustment (e.g., the resulting video stream is input into a virtual camera for subsequent live streaming).
Although the present specification provides method steps as described in the examples or flowcharts, additional or fewer steps may be included based on conventional or non-inventive means. The order of steps recited in the embodiments is merely one manner of performing the steps in a multitude of orders and does not represent the only order of execution. When an apparatus or client product in practice executes, it may execute sequentially or in parallel (e.g., in a parallel processor or multithreaded processing environment, or even in a distributed data processing environment) according to the embodiments or methods shown in the figures. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, article, or apparatus that comprises the recited elements is not excluded. The terms first, second, etc. are used to denote names, but not any particular order.
Based on the video stream synthesis method, one or more embodiments of the present specification further provide a video stream synthesis apparatus. The apparatus may include systems (including distributed systems), software (applications), modules, components, servers, clients, etc. that use the methods described in the embodiments of the present specification in conjunction with any necessary apparatus to implement the hardware. Based on the same innovative conception, embodiments of the present specification provide an apparatus as described in the following embodiments. Since the implementation scheme of the apparatus for solving the problem is similar to that of the method, the specific implementation of the apparatus in the embodiment of the present specification may refer to the implementation of the foregoing method, and repeated details are not repeated. As used hereinafter, the term "unit" or "module" may be a combination of software and/or hardware that implements a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Specifically, fig. 6 is a schematic block diagram of an embodiment of a video stream composition apparatus provided in this specification, and as shown in fig. 6, the video stream composition apparatus provided in this specification may include: a generating module 601, a synthesizing module 602 and an adjusting module 603.
A generating module 601, configured to receive an input video stream and initial parameters, and generate a virtual scene;
a synthesizing module 602, configured to synthesize the input video stream and the virtual scene according to the initial parameter to obtain a first video stream;
an adjusting module 603, configured to receive and adjust the first video stream by using a virtual camera according to the user-defined parameter, so as to obtain a result video stream; and sending the resulting video stream to a client.
It should be noted that the above-mentioned description of the apparatus according to the method embodiment may also include other embodiments, and specific implementation manners may refer to the description of the related method embodiment, which is not described herein again.
The present specification also provides a server comprising a processor and a memory for storing processor-executable instructions, which when executed by the processor, implement any of the above method embodiments. For example, the instructions when executed by the processor implement steps comprising: receiving an input video stream and initial parameters, and generating a virtual scene; synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream; receiving and adjusting the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream; and sending the resulting video stream to a client.
It should be noted that the above-mentioned server may also include other embodiments according to the description of the method or apparatus embodiments. The specific implementation manner may refer to the description of the related method embodiment, and is not described in detail herein.
Specifically, fig. 7 is a block diagram of a hardware structure of an embodiment of a server provided in this specification. As shown in fig. 7, the server 10 may include one or more (only one shown) processors 100 (the processors 100 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 200 for storing data, and a transmission module 300 for communication functions. It will be understood by those skilled in the art that the structure shown in fig. 7 is only an illustration and is not intended to limit the structure of the electronic device. For example, the server 10 may also include more or fewer components than shown in FIG. 7, and may also include other processing hardware, such as a database or multi-level cache, a GPU, or have a different configuration than shown in FIG. 7, for example.
The memory 200 may be used to store software programs and modules of application software, such as program instructions/modules corresponding to the video stream synthesizing method in the embodiment of the present specification, and the processor 100 executes various functional applications and data processing by executing the software programs and modules stored in the memory 200. Memory 200 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 200 may further include memory located remotely from processor 100, which may be connected to a computer terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission module 300 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal. In one embodiment, the transmission module 300 includes a Network adapter (NIC) that can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission module 300 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
The present specification also provides a computer readable storage medium storing computer program instructions that, when executed, implement: receiving an input video stream, initial parameters and generating a virtual scene; synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream; receiving and adjusting the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream; and sending the resulting video stream to a client.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The method or apparatus provided by the present specification and described in the foregoing embodiments may implement service logic through a computer program and record the service logic on a storage medium, where the storage medium may be read and executed by a computer, so as to implement the effect of the solution described in the embodiments of the present specification. The storage medium may include a physical device for storing information, and typically, the information is digitized and then stored using an electrical, magnetic, or optical media. The storage medium may include: devices that store information using electrical energy, such as various types of memory, e.g., RAM, ROM, etc.; devices that store information using magnetic energy such as hard disks, floppy disks, tapes, core memories, bubble memories, and usb disks; devices that store information optically, such as CDs or DVDs. Of course, there are other ways of storing media that can be read, such as quantum memory, graphene memory, and so forth.
The above video stream synthesizing method or apparatus embodiments provided in this specification may be implemented in a computer by a processor executing corresponding program instructions, such as a multimedia instruction implementation based on an x86/x64 processor and a mobile ARM processor, or a general-purpose parallel computing implementation based on a GPU, a processing logic implementation based on a quantum computer, and so on.
It should be noted that descriptions of the apparatuses and devices described above according to the related method embodiments in the specification may also include other embodiments, and specific implementation manners may refer to descriptions of corresponding method embodiments, which are not described in detail herein.
The embodiments in the present application are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the hardware + program class embodiment, since it is substantially similar to the method embodiment, the description is simple, and the relevant points can be referred to the partial description of the method embodiment.
For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, when implementing one or more of the present description, the functions of some modules may be implemented in one or more software and/or hardware, or the modules implementing the same functions may be implemented by a plurality of sub-modules or sub-units, etc.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus, and devices according to embodiments of the invention. It will be understood that the implementation can be by computer program instructions which can be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
One skilled in the art will appreciate that one or more embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, one or more embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects.
While the specification has been described with examples, those skilled in the art will appreciate that there are numerous variations and permutations of the specification without departing from the spirit of the specification, and it is intended that the appended claims encompass such variations and modifications without departing from the spirit of the specification.
Claims (10)
1. A video stream synthesizing method is applied to a server and comprises the following steps:
receiving an input video stream, initial parameters and generating a virtual scene;
synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream;
receiving and adjusting the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream; and sending the resulting video stream to a client.
2. The method of claim 1, wherein prior to receiving and adjusting the first video stream with a virtual camera according to custom parameters to obtain a resultant video stream, the method further comprises: and sending the first video stream to a client.
3. The method of claim 1, wherein the customized parameters comprise: camera parameters, foreground parameters, virtual scene parameters.
4. The method of claim 1, wherein synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream comprises:
performing background removal processing on the input video stream to obtain a target object;
and synthesizing the target object and the virtual scene according to the initial parameters to obtain a first video stream.
5. The method of claim 1, wherein receiving and adjusting the first video stream with a virtual camera according to custom parameters to obtain a resulting video stream comprises:
receiving the user-defined parameters, and configuring the operation parameters of the virtual camera according to the user-defined parameters;
and processing the first video stream by using the configured virtual camera to obtain a result video stream.
6. The method of claim 1, wherein the input video stream comprises a video stream to be presented to a viewer in a virtual live broadcast.
7. A video stream synthesizing method is applied to a client and comprises the following steps:
sending the input video stream and the initial parameters to a server; the server is used for synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream;
receiving a user-defined parameter input by a user, and sending the user-defined parameter to a server; the server is used for receiving and adjusting the first video stream by adopting a virtual camera according to a user-defined parameter to obtain a result video stream;
and receiving and playing the result video stream sent by the server.
8. The method of claim 7, wherein before receiving the user-entered customization parameters and sending the customization parameters to the server, the method further comprises: and receiving and playing the first video stream.
9. The method of claim 7, wherein after receiving and playing the resulting video stream sent by the server, the method further comprises:
detecting whether a correction parameter is received;
and under the condition that the correction parameters are determined not to be received, inputting the result video stream into a virtual camera so as to perform post-processing and live streaming.
10. A video stream synthesizing apparatus, comprising:
the generating module is used for receiving an input video stream and initial parameters and generating a virtual scene;
the synthesis module is used for synthesizing the input video stream and the virtual scene according to the initial parameters to obtain a first video stream;
the adjusting module is used for receiving and adjusting the first video stream by adopting a virtual camera according to the user-defined parameters to obtain a result video stream; and sending the resulting video stream to a client.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210910457.0A CN115278364B (en) | 2022-07-29 | 2022-07-29 | Video stream synthesis method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210910457.0A CN115278364B (en) | 2022-07-29 | 2022-07-29 | Video stream synthesis method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115278364A true CN115278364A (en) | 2022-11-01 |
| CN115278364B CN115278364B (en) | 2024-05-17 |
Family
ID=83747225
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210910457.0A Active CN115278364B (en) | 2022-07-29 | 2022-07-29 | Video stream synthesis method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115278364B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030052965A1 (en) * | 2001-09-18 | 2003-03-20 | Stephen Junkins | Portable virtual reality |
| CN110460794A (en) * | 2019-09-09 | 2019-11-15 | 北京西山居互动娱乐科技有限公司 | A kind of method and device of video record |
| CN112135158A (en) * | 2020-09-17 | 2020-12-25 | 重庆虚拟实境科技有限公司 | Live broadcasting method based on mixed reality and related equipment |
| CN113395540A (en) * | 2021-06-09 | 2021-09-14 | 广州博冠信息科技有限公司 | Virtual broadcasting system, virtual broadcasting implementation method, device and equipment, and medium |
| WO2021238804A1 (en) * | 2020-05-29 | 2021-12-02 | 深圳市瑞立视多媒体科技有限公司 | Mixed reality virtual preview photographing system |
| CN114339405A (en) * | 2022-01-04 | 2022-04-12 | 广州博冠信息科技有限公司 | AR video data stream remote manufacturing method and device, equipment and storage medium |
-
2022
- 2022-07-29 CN CN202210910457.0A patent/CN115278364B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030052965A1 (en) * | 2001-09-18 | 2003-03-20 | Stephen Junkins | Portable virtual reality |
| CN110460794A (en) * | 2019-09-09 | 2019-11-15 | 北京西山居互动娱乐科技有限公司 | A kind of method and device of video record |
| WO2021238804A1 (en) * | 2020-05-29 | 2021-12-02 | 深圳市瑞立视多媒体科技有限公司 | Mixed reality virtual preview photographing system |
| CN112135158A (en) * | 2020-09-17 | 2020-12-25 | 重庆虚拟实境科技有限公司 | Live broadcasting method based on mixed reality and related equipment |
| CN113395540A (en) * | 2021-06-09 | 2021-09-14 | 广州博冠信息科技有限公司 | Virtual broadcasting system, virtual broadcasting implementation method, device and equipment, and medium |
| CN114339405A (en) * | 2022-01-04 | 2022-04-12 | 广州博冠信息科技有限公司 | AR video data stream remote manufacturing method and device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115278364B (en) | 2024-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11694358B2 (en) | Computer vision on broadcast video | |
| CN110166827B (en) | Video clip determination method and device, storage medium and electronic device | |
| CN102326356B (en) | Video sharing | |
| US9047376B2 (en) | Augmenting video with facial recognition | |
| CN103581705A (en) | Method and system for recognizing video program | |
| CN110119700B (en) | Avatar control method, avatar control device and electronic equipment | |
| CN111988658B (en) | Video generation method and device | |
| US10580143B2 (en) | High-fidelity 3D reconstruction using facial features lookup and skeletal poses in voxel models | |
| CN202998337U (en) | Video program identification system | |
| CN105893412A (en) | Image sharing method and apparatus | |
| US20220172476A1 (en) | Video similarity detection method, apparatus, and device | |
| CN105701762B (en) | Picture processing method and electronic equipment | |
| CN108124170A (en) | A kind of video broadcasting method, device and terminal device | |
| CN110266955B (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
| CN114139491A (en) | Data processing method, device and storage medium | |
| CN110300118B (en) | Streaming media processing method, device and storage medium | |
| CN108320331B (en) | Method and equipment for generating augmented reality video information of user scene | |
| CN111510746B (en) | Media resource delivery method and device, storage medium and electronic device | |
| CN108713313B (en) | Multimedia data processing method and device, and equipment/terminal/server | |
| CN113542909A (en) | Video processing method and device, electronic equipment and computer storage medium | |
| CN115278364B (en) | Video stream synthesis method and device | |
| US20220070501A1 (en) | Social video platform for generating and experiencing content | |
| CN114501060A (en) | Live broadcast background switching method and device, storage medium and electronic equipment | |
| CN109905766A (en) | A kind of dynamic video poster generation method, system, device and storage medium | |
| US11902603B2 (en) | Methods and systems for utilizing live embedded tracking data within a live sports video stream |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |