WO2024251215A1 - 多人视频直播方法、装置、电子设备及存储介质 - Google Patents
多人视频直播方法、装置、电子设备及存储介质 Download PDFInfo
- Publication number
- WO2024251215A1 WO2024251215A1 PCT/CN2024/097855 CN2024097855W WO2024251215A1 WO 2024251215 A1 WO2024251215 A1 WO 2024251215A1 CN 2024097855 W CN2024097855 W CN 2024097855W WO 2024251215 A1 WO2024251215 A1 WO 2024251215A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- live broadcast
- terminal device
- target
- broadcast room
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/434—Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/81—Monomedia components thereof
- H04N21/816—Monomedia components thereof involving special video data, e.g 3D video
Definitions
- the embodiments of the present disclosure relate to the field of Internet technology, and in particular to a method, device, electronic device, and storage medium for multi-person video live broadcast.
- multi-person live broadcasting refers to multiple anchor users jointly broadcasting through video links, so that audience users can watch the interaction between multiple anchors and improve the live video broadcasting effect.
- the existing technology for implementing multi-person live broadcasting usually uses split-screen display for each anchor user, which causes a strong sense of spatial fragmentation and poor visual effects, affecting the overall live broadcast effect in the multi-person live broadcast scenario.
- the embodiments of the present disclosure provide a method, device, electronic device and storage medium for multi-person video live broadcasting, so as to overcome the problem of strong spatial separation between the screens of anchor users and poor visual effects during multi-person live broadcasting.
- an embodiment of the present disclosure provides a multi-person video live broadcast method, which is applied to a host terminal device, including: creating a three-dimensional live broadcast room; creating a first target image in the three-dimensional live broadcast room based on a first video live stream sent by a first terminal device, wherein the first terminal device corresponds to an anchor user, and the first target image represents a live broadcast room screen of a two-dimensional live broadcast room corresponding to the first video live stream; sending a second video live stream corresponding to the three-dimensional live broadcast room to a target terminal device, wherein the target terminal device includes the first terminal device and/or the second terminal device, and the second terminal device corresponds to an audience user.
- an embodiment of the present disclosure provides a multi-person video live broadcast method, which is applied to a first terminal device, including: entering a three-dimensional live broadcast room; sending a first video live stream to a host terminal device that creates the three-dimensional live broadcast room to create a first target image in the three-dimensional live broadcast room, wherein the first terminal device corresponds to an anchor user, and the first target image represents a live broadcast room screen of a two-dimensional live broadcast room corresponding to the first video live stream; receiving a second video live stream corresponding to the three-dimensional live broadcast room, and playing the second video live stream.
- an embodiment of the present disclosure provides a multi-person video live broadcast device, including: a creation module, used to create a three-dimensional live broadcast room; a generation module, used to create a first target image in the three-dimensional live broadcast room based on a first video live stream sent by a first terminal device, wherein the first terminal device corresponds to an anchor user, and the first target image represents a live broadcast room screen of a two-dimensional live broadcast room corresponding to the first video live stream; a transceiver module, used to send a second video live stream corresponding to the three-dimensional live broadcast room to a target terminal device, wherein the target terminal device includes the first terminal device and/or the second terminal device, and the second terminal device corresponds to an audience user.
- an embodiment of the present disclosure provides a multi-person video live broadcast device, comprising: a processing module for entering a three-dimensional live broadcast room; a sending module for sending a first video live stream to a host terminal device that creates the three-dimensional live broadcast room, so as to create a first target image in the three-dimensional live broadcast room, wherein the first terminal device corresponds to an anchor user, and the first target image represents a live broadcast room screen of a two-dimensional live broadcast room corresponding to the first video live stream; a receiving module for receiving a second video live stream corresponding to the three-dimensional live broadcast room, and playing the second video live stream.
- an embodiment of the present disclosure provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the multi-person video live broadcast method as described in the first aspect and various possible designs of the first aspect, or to implement the multi-person video live broadcast method as described in the second aspect and various possible designs of the second aspect.
- an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions.
- the multi-person video live broadcast method as described in the first aspect and various possible designs of the first aspect is implemented, or the multi-person video live broadcast method as described in the second aspect and various possible designs of the second aspect is implemented.
- an embodiment of the present disclosure provides a computer program product, including a computer program, When the computer program is executed by a processor, it implements the multi-person video live broadcast method as described in the first aspect and various possible designs of the first aspect, or it implements the multi-person video live broadcast method as described in the second aspect and various possible designs of the second aspect.
- FIG1 is a diagram of an application scenario of a multi-person video live broadcast method provided by an embodiment of the present disclosure
- FIG2 is a flow chart of a multi-person video live broadcast method according to an embodiment of the present disclosure
- FIG3 is a schematic diagram of creating a first target image in a three-dimensional live broadcast room provided by an embodiment of the present disclosure
- FIG4 is a flow chart of a specific implementation of step S102 in the embodiment shown in FIG2 ;
- FIG5 is a flow chart of a specific implementation of step S104 in the embodiment shown in FIG2 ;
- FIG6 is a flow chart of a specific implementation of step S105 in the embodiment shown in FIG2 ;
- FIG7 is a schematic diagram of a target virtual object moving in a three-dimensional live broadcast room provided by an embodiment of the present disclosure
- FIG8 is a second flow chart of a multi-person video live broadcast method provided by an embodiment of the present disclosure.
- FIG9 is a flow chart of a specific implementation of step S208 in the embodiment shown in FIG8 ;
- FIG10 is a schematic diagram of determining a target display direction provided by an embodiment of the present disclosure.
- FIG11 is a third flow chart of a multi-person video live broadcast method provided by an embodiment of the present disclosure.
- FIG12 is a structural block diagram of a multi-person video live broadcast device provided in an embodiment of the present disclosure.
- FIG13 is a structural block diagram of another multi-person video live broadcast device provided by an embodiment of the present disclosure.
- FIG14 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure.
- FIG. 15 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure.
- request information including but not limited to user device information, user personal information, etc.
- data including but not limited to data used for analysis, stored data, displayed data, etc.
- request information including but not limited to user device information, user personal information, etc.
- data including but not limited to data used for analysis, stored data, displayed data, etc.
- the request information are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
- the multi-person video live broadcast method provided by the embodiment of the present disclosure can be applied to the application scenario of multi-person online live broadcast.
- Figure 1 is an application scenario diagram of the multi-person video live broadcast method provided by the embodiment of the present disclosure.
- the method provided by the embodiment of the present disclosure can be applied to the terminal device used by the anchor user. For example, taking the terminal device A used by the anchor user User_1 as an example, after the terminal device A runs the live broadcast application client (Application, APP), it responds to the user's instructions and creates a video live broadcast room for video live broadcast. Afterwards, the live broadcast user User_1 can connect the video with the terminal device B used by the anchor user User_2 through the terminal device A, also known as "connection PK".
- connection PK also known as "connection PK"
- the video screen of the anchor user User_1 and the video screen of the anchor user User_2 are displayed simultaneously in the live broadcast application client on the side of the terminal device A (based on needs, it can also be displayed on the terminal device B at the same time), thereby realizing multi-person interactive live broadcast between multiple anchor users.
- the video live broadcast stream of the above-mentioned live broadcast room is distributed to the terminal device C used by the audience user through the server, so that the audience user can watch the live broadcast content of the multi-person live broadcast in the above-mentioned live broadcast room.
- a method for implementing multi-person live broadcast is usually to perform split-screen display for each anchor user, which causes a strong sense of spatial fragmentation and poor visual effects, affecting the overall live broadcast effect in a multi-person live broadcast scenario.
- the disclosed embodiment provides a multi-person video live broadcast method to solve the above problems.
- FIG. 2 is a flow chart of a multi-person video live broadcast method provided by an embodiment of the present disclosure.
- the method of this embodiment can be applied to a video live broadcast server or a host terminal device.
- a host terminal device which can be a terminal corresponding to a host/room owner user
- the multi-person video live broadcast method provided by this embodiment includes:
- Step S101 Create a three-dimensional live broadcast room.
- a three-dimensional live broadcast room can be created, and the three-dimensional live broadcast room is a live broadcast room based on the three-dimensional live broadcast room form.
- the live broadcast room is a logical unit provided by the server of the live broadcast application for outputting the video live stream.
- the audience user who enters the live broadcast room can receive the video live stream corresponding to the live broadcast room, so as to watch the live content in the live broadcast room.
- the live broadcast room in the prior art is a two-dimensional live broadcast room, that is, on a fixed plane background, the video content corresponding to the anchor user is displayed, and the data constituting such a two-dimensional live broadcast room includes a live broadcast data stream and a plane image. Therefore, in the scenario of multi-person live broadcast, the two-dimensional live broadcast room receives the live broadcast data stream of different anchor users, and displays the live broadcast screen of different anchor users on the plane image by displaying in different areas.
- the three-dimensional live broadcast room in this embodiment can be a live broadcast room with a three-dimensional stereoscopic effect constructed by a three-dimensional image engine.
- the data constituting the three-dimensional live broadcast room includes image model data and video live stream corresponding to the three-dimensional live broadcast room.
- the three-dimensional live broadcast room provided in this embodiment refers to the three-dimensional live broadcast room, which needs to load three-dimensional image data, render three-dimensional live broadcast room, and other steps to achieve the creation of the three-dimensional live broadcast room. Furthermore, the three-dimensional live broadcast room corresponding to the created three-dimensional live broadcast room contains virtual objects. Different virtual environment scenes can be displayed through virtual objects, thereby realizing the simulation of real environment scenes, such as concert scenes, variety show scenes, competition scenes, etc. The specific content in the above-mentioned three-dimensional live broadcast room and the realization of the overall virtual scene can be selected through user instructions.
- the three-dimensional live broadcast room can be initialized and created through an existing three-dimensional game engine, and the specific implementation method will not be repeated here.
- Step S102 Based on the first video live stream sent by the first terminal device, create a first target image in the three-dimensional live broadcast room, wherein the first terminal device corresponds to the anchor user, and the first target image represents the live broadcast room screen of the two-dimensional live broadcast room corresponding to the first video live stream.
- the user operating the host terminal device can be regarded as the host/room owner user.
- the host/room owner user can send an invitation message to the first terminal devices corresponding to one or more anchor users through the host terminal device, so that the anchor users join the 3D live broadcast room.
- the anchor user directly joins the 3D live broadcast room by receiving link push, autonomous search, etc.
- the first terminal device sends the first video live stream to the host terminal directly or indirectly.
- the host terminal device renders the video content in the first video live stream into the three-dimensional live broadcast room, thereby generating a first target image representing the image of the anchor user in the first video live stream in the three-dimensional live broadcast room.
- FIG3 is a schematic diagram of creating a first target image in a three-dimensional live broadcast room provided by an embodiment of the present disclosure.
- the host terminal device renders a corresponding three-dimensional live broadcast room.
- the three-dimensional live broadcast room includes virtual objects rendered based on a three-dimensional image engine, such as a stage, an auditorium, and spectators in the auditorium.
- the host terminal device will receive the first video live stream steam_1 (shown as steam_1 in the figure) corresponding to the anchor user U1 sent by the terminal device D2_1, and the first video live stream steam_2 (shown as steam_2 in the figure) corresponding to the anchor user U2 sent by the terminal device D2_2.
- the host terminal device sets the live screen P1 generated based on the first video live stream steam_1 and the live screen P2 generated based on the first video live stream steam_2 (the live screen P1 and the live screen P2 are the first target images) to play at the pos_1 and pos_2 positions in the three-dimensional live broadcast room. Therefore, visually, the image of the anchor user U1 and the image of the anchor user U2 are simultaneously located in the three-dimensional live broadcast room, realizing the integration of the image of the anchor user and the virtual environment.
- the host terminal device after receiving the first video live stream, the host terminal device first processes the first video live stream, separates the anchor image part from the background part, and then generates the first target image in the three-dimensional live studio. This makes the first target image generated in the three-dimensional live studio more detailed and realistic.
- the specific implementation of step S102 includes:
- Step S1021 determining a target area of a live image frame in the first live video stream, where the target area is an area where the host image is located in the live image frame.
- Step S1022 Segment the live image frame based on the target area to generate a first target image.
- Step S1023 setting the first target image in the three-dimensional live broadcast room.
- the host terminal device can obtain the corresponding live image frame after necessary decoding processing.
- the specific implementation process is prior art and will not be described here.
- the host terminal device then performs image recognition and segmentation on each live image frame, segments the image in the target area where the host image is located in the live image frame, and discards the image in the area outside the host image in the live image frame, thereby obtaining the host image with the background removed, that is, the first target image.
- the target area in the live image frame can be directly The target area is determined by the image color in the live image frame.
- the first video live stream is generated by the first terminal device after shooting the anchor user in combination with a green screen.
- the host terminal device can divide the target area by color.
- the host terminal device can also determine the current area therein by determining the outline of the image elements in the live image frame and performing feature recognition on the image elements.
- the specific implementation method for determining the target area above can be set according to the specific situation and is not specifically limited here.
- the method further includes:
- Step S104 When the audience user enters the 3D live broadcast room, a target virtual object corresponding to the audience user is generated in the 3D live broadcast room.
- the three-dimensional live broadcast room includes at least one first spatial position and/or at least one second spatial position, wherein the first spatial position is used to display the first target image, and the second spatial position is used to display the virtual object.
- the first spatial position is determined based on the first request information of the anchor user entering the three-dimensional live broadcast room
- the second spatial position is determined based on the second request information of the audience user entering the three-dimensional live broadcast room.
- the first request information is the identifier of the anchor user entering the target anchor room
- the second request information is the identifier of the audience user entering the target anchor room.
- the host terminal device can obtain the above-mentioned first request information and second request information through the server of the live broadcast application.
- the first spatial position where the first target image is located and the second spatial position where the virtual object is located are set.
- the corresponding first spatial position is determined according to the association degree between the anchor user and the host/room owner user.
- the corresponding second spatial position is determined according to the association degree between the audience user and the anchor user.
- step S104 includes:
- Step S1041 when the target audience user enters the 3D live broadcast room, the second request information of the target audience user entering the 3D live broadcast room is obtained.
- Step S1042 Based on the second request information, obtain the target appearance of the corresponding target virtual object.
- Step S1043 Generate a target virtual object in the three-dimensional live broadcast room based on the target appearance.
- mapping relationship between the second request information and the appearance of the virtual object.
- the level of the live broadcast room represented by the second request information is mapped to the corresponding appearance color. Therefore, based on the second request information and the above mapping relationship, the target appearance of the corresponding target virtual object can be obtained. Afterwards, the target appearance is used as a rendering parameter to render the target virtual object into the three-dimensional live broadcast room, thereby achieving the purpose of generating the target virtual object corresponding to the second request information in the three-dimensional live broadcast room.
- the above mapping relationship is only exemplary, and the specific mapping implementation between the second request information representation and the target appearance can be set as needed, and no specific limitation is made here.
- step S104 the method further includes:
- Step S105 In response to the action request sent by the second terminal device, control the target virtual object to move in the three-dimensional live broadcast room.
- the audience user can directly or indirectly send an action request to the host terminal device by operating the second terminal device.
- the second terminal device sends an action request to the host terminal device through the server.
- the host terminal device determines the target audience user corresponding to the action request, and then determines the target virtual object corresponding to the target audience user in the three-dimensional live broadcast room.
- the target virtual object is produced by a three-dimensional image engine, and then, based on the specific content represented by the action request, the target virtual object is controlled to move in the three-dimensional live broadcast room.
- interaction and communication between the audience user and the anchor user are realized.
- step S105 includes:
- Step S1051 in response to the action request sent by the second terminal device, obtaining action information corresponding to the target audience user.
- Step S1052 Based on the action information, a target action of a target virtual object corresponding to the target audience user is generated.
- Step S1053 Control the target virtual object to move in the three-dimensional live broadcast room based on the target action.
- FIG7 is a schematic diagram of a target virtual object moving in a three-dimensional live broadcast room provided by an embodiment of the present disclosure.
- the host terminal device receives an entry message Info_1 (shown as Info_1 in the figure), which can be indirectly sent by the second terminal device to the host terminal device through the server.
- Info_1 shown as Info_1 in the figure
- the host terminal device creates a corresponding target virtual object for the target audience user in the virtual three-dimensional space corresponding to the three-dimensional live broadcast room.
- P1 In the initial state, the virtual object P1 is in a standing posture with arms hanging down.
- the host terminal device receives the action information Info_2 (shown as Info_2 in the figure) indirectly sent by the first terminal device D2 through the server.
- the host terminal device determines the corresponding target action based on the action information Info_2, such as a cheering action of raising both hands, and controls the movement of the virtual object P1 based on the target action to make the cheering action of raising both hands, thereby enabling the audience user to control the corresponding virtual object in the three-dimensional live broadcast room.
- Enable interaction between audience users and anchor users, and between audience users improve the interactive effect of the three-dimensional live broadcast room, and improve the overall live broadcast effect in the multi-person live broadcast scene.
- Step S103 Sending a second video live stream corresponding to the three-dimensional live broadcast room to a target terminal device, wherein the target terminal device includes a first terminal device and/or a second terminal device, and the second terminal device corresponds to an audience user.
- the host terminal device uses the data corresponding to the three-dimensional live broadcast room to render the corresponding live data stream, that is, the second video live stream, and uploads it to the server of the live broadcast application.
- the server of the live broadcast application distributes the second video live stream to the first terminal device and/or the second terminal device, that is, the target terminal device.
- the first terminal device and/or the second terminal device can watch the actual image in the three-dimensional live broadcast room on the side of the running live broadcast client (or anchor client).
- multiple anchor users are located in the same three-dimensional live broadcast room, and the image of the anchor user is integrated with the virtual environment, so it has better integrity and authenticity. At the same time, it can realize the interaction and communication between multiple anchor users in the three-dimensional live broadcast room, and provide the interaction efficiency of the three-dimensional live broadcast room.
- a three-dimensional live broadcast room is created; based on the first video live stream sent by the first terminal device, a first target image is created in the three-dimensional live broadcast room, wherein the first terminal device corresponds to the anchor user, and the first target image represents the live broadcast room screen of the two-dimensional live broadcast room corresponding to the first video live stream; and a second video live stream corresponding to the three-dimensional live broadcast room is sent to the target terminal device, wherein the target terminal device includes the first terminal device and/or the second terminal device, and the second terminal device corresponds to the audience user.
- the image (anchor image) of the anchor user is projected into the three-dimensional live broadcast room for display, so that the images of different anchor users are integrated with the environment, reducing the sense of spatial fragmentation, improving the difference in visual effects, and improving the overall live broadcast effect in a multi-person live broadcast scene.
- FIG8 is a flow chart of a multi-person video live broadcast method provided by an embodiment of the present disclosure.
- the method for multi-person video live broadcast comprises:
- Step S201 Create a three-dimensional live broadcast room.
- Step S202 Acquire the target screen of the room owner/host user corresponding to the host terminal device.
- Step S203 Based on the target screen of the host user, a second target image is generated in the three-dimensional live broadcast room, where the second target image represents the virtual object corresponding to the host user.
- the host terminal device after creating a three-dimensional live broadcast room, the host terminal device further creates a virtual object based on its own image in the three-dimensional live broadcast room as the host role of the three-dimensional live broadcast room. Specifically, first, the host terminal device collects the target screen of the host user located on one side of the host terminal device through the image acquisition unit. Then, based on feature extraction, the host image features are obtained. After that, the obtained host image features are input as parameters into the three-dimensional image engine, so as to render the corresponding virtual object in the three-dimensional live broadcast room, such as a virtual character with the appearance of the host, that is, the second target image.
- Step S204 Send invitation information to the first terminal device, where the invitation information is used to invite the first terminal device to enter the three-dimensional live broadcast room.
- Step S205 After receiving the response information sent by the first terminal device, receive the first video live stream sent by the first terminal device.
- the host terminal device sends an invitation message to the first terminal device based on the user's instructions to invite the anchor user on the first terminal device side to join the three-dimensional live broadcast room.
- the first terminal device displays the corresponding prompt logo and the corresponding selection component in the client of the live broadcast application.
- the prompt logo is, for example, "XXX user invites you to join the XR live broadcast room", and the appearance of the corresponding displayed selection component is "yes" and "no".
- the anchor user clicks the selection component corresponding to "yes” the first terminal device sends a response message to the host terminal device.
- the host terminal device After receiving the response message, the host terminal device establishes a video connection with the first terminal device, and the first terminal device sends the live video stream corresponding to the live broadcast room where the anchor user is currently located, that is, the first video live stream, to the host terminal device. Thereby, the host terminal device receives the first video live stream and performs subsequent processing steps.
- Step S206 creating a first target image in the three-dimensional live broadcast room based on the first video live stream sent by the first terminal device.
- Step S207 receiving control information sent by the target terminal device, the control information represents the observation position and/or observation angle of the observation point in the three-dimensional live broadcast room, and the observation point is used to obtain the observation position and/or observation angle of the observation point in the three-dimensional live broadcast room. image.
- Step S208 Generate a second video live stream corresponding to the target terminal device according to the control information.
- the target terminal device may be a first terminal device or a second terminal device, that is, a terminal device used by an anchor user or an audience user. Both the anchor user and the audience user can adjust the position and viewing angle of the live screen in the three-dimensional live studio through the corresponding terminal device (the client running inside). The following is a detailed description of the audience user.
- the target terminal device is a second terminal device, and after the second terminal device receives the operation instruction of the audience user, it sends control information to the host terminal device.
- the operation instruction is such as a rotation lens instruction and a mobile lens position instruction.
- the position parameters and angle parameters of the picture lens in the three-dimensional live studio that is, the control information
- the picture lens is the observation point in the three-dimensional live studio
- the position parameter is the information representing the observation position
- the angle parameter is the information representing the observation angle.
- control information may include information corresponding to both the observation position and the observation angle, or may only include information corresponding to the observation position or the observation angle.
- the other parameter may be automatically generated or fixed.
- the observation angle may be automatically determined based on the direction of the vector formed by the observation position and the first spatial position where the first target image is located, so that the observation angle is always directed toward the first target image.
- the observation position may be a preset value, which is determined, for example, based on a previous instruction to move the lens position. The specific implementation method may be determined as needed and will not be described in detail here.
- step S208 includes three specific implementation steps: step S2081, S2082, and S2083:
- Step S2081 According to the control information sent by the target terminal device, the spatial observation range of the observation point in the three-dimensional live broadcast room is obtained.
- Step S2082 According to the control information, a target display direction of the first target image is obtained, wherein an angle between the target display direction and the direction of the position vector is less than an angle threshold, and the position vector is composed of a first spatial position corresponding to the first target image and an observation position corresponding to the control information.
- Step S2083 Based on the spatial observation range and the target display direction, image acquisition is performed on the three-dimensional live broadcast room to generate a second video live stream.
- the control information includes an observation position and an observation angle.
- the observation position the position of the observation point in the three-dimensional live broadcast room can be determined.
- the observation angle and the preset field of view angle for example, an angle range of 60 degrees up and down and 120 degrees left and right with the observation angle as the center
- a spatial observation range can be determined.
- the first target image the first spatial position where it is located
- the three-dimensional live broadcast room is imaged and encoded based on the spatial observation range to generate a second video live stream consisting of two-dimensional image frames.
- control information is generated in the target terminal device.
- the host terminal device After being sent by the target terminal device, the host terminal device generates at least one two-dimensional image frame according to the control information, and sends it back to the target terminal device for display through the second video live stream, thereby producing a three-dimensional view effect.
- the target display direction of the first target image is further obtained according to the control information.
- the display direction of the first target image is the orientation of the first target image
- the target display direction of the first target image is determined by the direction of the position vector formed by the first spatial position corresponding to the first target image and the observation position corresponding to the control information.
- the target display direction of the first target image is parallel to the direction of the position vector formed by the first spatial position and the observation position, or the angle is less than the angle threshold.
- the first target image displayed based on the target display direction and the three-dimensional live broadcast room corresponding to the spatial observation range are imaged and encoded to generate a second video live stream consisting of two-dimensional image frames.
- the display direction of the first target image is always toward the observation point, avoiding the problem of the first target image being deflected in the live screen due to the movement of the observation point, thereby improving the picture effect.
- FIG10 is a schematic diagram of determining a target display direction provided by an embodiment of the present disclosure. The above steps are further described below in conjunction with FIG10.
- the control information vers_1 shown as vers_1 in the figure
- the target display direction of the first target image at position B is seta_1
- seta_1 is the direction of the line connecting position A_1 and position B
- the host terminal device collects the image of the three-dimensional live broadcast room within the spatial observation range, and the first target image displayed with seta_1 as the display direction.
- the two are then fused, rendered and encoded to generate a second video live stream steam_1.
- control information vers_2 (shown as vers_2 in the figure) sent by the second terminal device, when the observation point is set at position A_2 in the three-dimensional live broadcast room, the target display direction of the first target image at position B is seta_2, and seta_2 is the direction of the line connecting position A_2 and position B.
- the host terminal device collects the image of the three-dimensional live broadcast room within the spatial observation range, and the first target image displayed with seta_2 as the display direction. Then, the two are merged, rendered and encoded to generate the second video live stream steam_2.
- the second video live stream steam_1 and the second video live stream steam_2 correspond to observation positions of different observation points, since the target display direction is dynamically adjusted with the observation position, it is ensured that in the second video live stream steam_1 and the second video live stream steam_2, the first target image always faces the live camera (observation point), ensuring that the first target image can be normally displayed at any observation position.
- Step S209 Send the corresponding second video live stream to each target terminal device.
- the second video live stream is generated, it is sent to the first terminal device and/or the second terminal device, so that each anchor user and each audience user entering the live broadcast room can see the live broadcast picture generated based on the control information set by each of them.
- steps S201, S206, and S209 are implemented in the same manner as steps S101-S103 in the above embodiment.
- steps S101-S103 for detailed discussion, please refer to the relevant discussion in the embodiment shown in FIG. 2, which will not be repeated here.
- the multi-person video live broadcast method, device, electronic device and storage medium create a three-dimensional live broadcast room; based on the first video live broadcast stream sent by the first terminal device, create a first target image in the three-dimensional live broadcast room, wherein the first terminal device corresponds to the anchor user, and the first target image represents the live broadcast room screen of the two-dimensional live broadcast room corresponding to the first video live broadcast stream; send the second video live broadcast stream corresponding to the three-dimensional live broadcast room to the target terminal device, wherein the target terminal device includes the first terminal device and/or the second terminal device, and the second terminal device corresponds to the audience user.
- the anchor image screen is projected into the three-dimensional live broadcast room for display, so that the anchor images of different users are integrated with the environment, reducing the sense of spatial fragmentation, improving the visual effect difference, and improving the overall live broadcast effect in the multi-person live broadcast scene.
- FIG. 11 is a flow chart of a multi-person video live broadcast method provided in an embodiment of the present disclosure.
- the method of this embodiment can be applied to a first terminal device used by a host user.
- the multi-person video live broadcast method provided in this embodiment includes:
- Step S301 Enter the 3D live broadcast room.
- Step S302 Send a first video live stream to a host terminal device that creates a three-dimensional live room to create a first target image in the three-dimensional live room, wherein the first target image represents a live room screen of a two-dimensional live room corresponding to the first video live stream.
- Step S303 receiving a second video live stream corresponding to the three-dimensional live broadcast room, and playing the second video live stream.
- the method provided in this embodiment uses the first terminal device on the side of the anchor user as the execution subject.
- the first terminal device After entering the three-dimensional live broadcast room, the first terminal device sends the first video live stream to the host terminal device, so that the live broadcast screen of the anchor user is generated in the three-dimensional live broadcast room.
- the second video live stream generated after the host terminal device renders the three-dimensional live broadcast room is received, and based on the second video live stream, it is played, so that the anchor connection of different anchor users in a three-dimensional virtual environment can be realized. Since each anchor user can see the anchor image of himself and other anchor users through the above steps, it is possible to realize live interaction in a three-dimensional environment and improve the live broadcast effect of multi-person live broadcast.
- the specific implementation process of the above steps has been introduced in the previous embodiment.
- the specific implementation method can refer to the interaction steps between the host terminal device and the first terminal device in the previous embodiment, which will not be repeated here.
- the method provided in this embodiment also includes:
- Step S304 sending control information, where the control information represents an observation position and/or observation angle of an observation point in the three-dimensional live broadcast room, and the observation point is used to obtain an image in the three-dimensional live broadcast room.
- step S303 After sending the control information, the specific implementation method of step S303 is: receiving the second video live stream generated based on the control information corresponding to the three-dimensional live room, and playing the second video live stream.
- the anchor user after the anchor user joins the 3D live broadcast room, he can further operate the first terminal device to send control information to the host terminal device, thereby controlling the observation position and angle of the observation point in the 3D live broadcast room, thereby achieving a better interaction effect with other anchor users.
- the specific implementation method can refer to the relevant steps of the second terminal device sending control information to the host terminal device in the previous embodiment, and the implementation method is similar, which will not be repeated here.
- FIG12 is a structural block diagram of a multi-person video live broadcast device provided by the embodiment of the present disclosure, which is applied to a host terminal device. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown.
- the multi-person video live broadcast device 4 includes:
- the creation module 41 is used to create a three-dimensional live broadcast room.
- the generating module 42 is configured to generate a first video live stream in a three-dimensional video format based on the first video live stream sent by the first terminal device.
- a first target image is created in the live broadcast room, wherein the first terminal device corresponds to the anchor user, and the first target image represents the live broadcast room screen of the two-dimensional live broadcast room corresponding to the first video live stream.
- the transceiver module 43 is used to send the second video live stream corresponding to the three-dimensional live room to the target terminal device, wherein the target terminal device includes the first terminal device and/or the second terminal device, and the second terminal device corresponds to the audience user.
- the generation module 42 is specifically used to: determine a target area of a live image frame in a first video live stream, where the target area is the area where the host image is located in the live image frame; segment the live image frame based on the target area to generate a first target image; and set the first target image in a three-dimensional live broadcast room.
- the generation module 42 is also used to: receive control information sent by the target terminal device, the control information represents the observation position and/or observation angle of the observation point in the three-dimensional live broadcast room, and the observation point is used to obtain the image in the three-dimensional live broadcast room; and generate a second video live stream corresponding to the target terminal device according to the control information.
- the generation module 42 when the generation module 42 generates a second video live stream corresponding to the target terminal device according to the control information, it is specifically used to: obtain the spatial observation range of the observation point in the three-dimensional live broadcast room according to the control information sent by the target terminal device; perform image acquisition on the three-dimensional live broadcast room based on the spatial observation range to generate the second video live stream.
- the generation module 42 when the generation module 42 generates a second video live stream corresponding to a target terminal device according to control information, it is specifically used to: obtain a target display direction of the first target image according to the control information, wherein an angle between the target display direction and the direction of a position vector is less than an angle threshold, and the position vector is composed of a first spatial position corresponding to the first target image and an observation position corresponding to the control information; generate a second video live stream according to the target display direction.
- a three-dimensional live broadcast room includes at least one first spatial position and/or at least one second spatial position, wherein the first spatial position is used to display a first target image, and the second spatial position is used to display a virtual object, and the first spatial position is determined based on first request information of a host user entering the three-dimensional live broadcast room, and the second spatial position is determined based on second request information of an audience user entering the three-dimensional live broadcast room.
- the generation module 42 is further used to: obtain first request information of a target audience user entering the three-dimensional live broadcast room; obtain the target appearance of the corresponding target virtual object based on the first request information; and generate the target virtual object based on the target appearance.
- the generation module 42 is also used to: obtain action information corresponding to the target audience user in response to the action request sent by the second terminal device; generate a target action of the target virtual object corresponding to the target audience user based on the action information; and control the movement of the target virtual object in the three-dimensional live broadcast room based on the target action.
- the generation module 42 is further used to: obtain a target screen of a host user corresponding to the host terminal device; and generate a second target image in the three-dimensional live broadcast room based on the target screen of the host user, wherein the second target image represents a virtual object corresponding to the host user.
- the transceiver module 42 is also used to: send an invitation message to the first terminal device, the invitation message is used to invite the first terminal device to enter the three-dimensional live broadcast room; after receiving the response message sent by the first terminal device, receive the first video live stream sent by the first terminal device.
- the creation module 41, the generation module 42, and the transceiver module 43 are connected in sequence.
- the multi-person video live broadcast device 4 provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated in this embodiment.
- FIG13 is a structural block diagram of another multi-person video live broadcast device provided by an embodiment of the present disclosure, which is applied to a first terminal device. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown.
- the multi-person video live broadcast device 5 includes:
- the processing module 51 is used to enter the three-dimensional live broadcast room.
- the transceiver module 52 is used to send a first video live stream to a host terminal device that creates a three-dimensional live room to create a first target image in the three-dimensional live room.
- the first target image represents the live room screen of the two-dimensional live room corresponding to the first video live stream.
- the playing module 53 is used to receive the second video live stream corresponding to the three-dimensional live room and play the second video live stream.
- the transceiver module 52 is also used to: send control information, the control information represents the observation position and/or observation angle of the observation point in the three-dimensional live broadcast room, and the observation point is used to obtain the image in the three-dimensional live broadcast room;
- the playback module 53 is specifically used to: receive the second video live stream generated based on the control information corresponding to the three-dimensional live broadcast room, and play the second video live stream generated based on the control information.
- FIG14 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure.
- the electronic device 6 includes: a processor 61, and a memory 62 connected to the processor 61 for communication;
- the memory 62 stores computer-executable instructions;
- the processor 61 executes the computer-executable instructions stored in the memory 62 to implement the multi-person video live broadcast method in the embodiments shown in Figures 2 to 11.
- the processor 61 and the memory 62 are connected via a bus 63 .
- An embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored.
- the computer execution instructions are executed by a processor, they are used to implement the multi-person video live broadcast method provided by any one of the embodiments corresponding to Figures 2 to 11 of the present disclosure.
- the embodiments of the present disclosure provide a computer program product, including a computer program.
- the computer program is executed by a processor, the multi-person video live broadcast method in the embodiments shown in FIG. 2 to FIG. 11 is implemented.
- FIG. 15 it shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure
- the electronic device 900 may be a terminal device or a server.
- the terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
- PDAs personal digital assistants
- PADs Portable Android Devices
- PMPs portable multimedia players
- vehicle terminals such as vehicle navigation terminals
- fixed terminals such as digital TVs, desktop computers, etc.
- the electronic device shown in FIG. 15 is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present disclosure.
- the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903.
- a processing device e.g., a central processing unit, a graphics processing unit, etc.
- RAM random access memory
- Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903.
- the processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904.
- An input/output (I/O) interface 905 is also connected to the bus 904.
- the following devices may be connected to the I/O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909.
- the communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange information.
- FIG. 15 shows an electronic device 900 having various devices, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.
- an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
- the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902.
- the processing device 901 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
- the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
- the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
- Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
- a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.
- a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried.
- This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above.
- the computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device.
- the program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
- the computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
- the computer-readable medium carries one or more programs.
- the electronic device executes the method shown in the above embodiment.
- Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages.
- the program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
- the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
- LAN Local Area Network
- WAN Wide Area Network
- each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function.
- the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
- each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
- the units involved in the embodiments described in the present disclosure may be implemented by software or hardware.
- the name of a unit does not limit the unit itself in some cases.
- the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".
- exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
- FPGAs field programmable gate arrays
- ASICs application specific integrated circuits
- ASSPs application specific standard products
- SOCs systems on chips
- CPLDs complex programmable logic devices
- a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.
- a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
- a machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing.
- a more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
- RAM random access memory
- ROM read-only memory
- EPROM or flash memory erasable programmable read-only memory
- CD-ROM portable compact disk read-only memory
- CD-ROM compact disk read-only memory
- magnetic storage device or any suitable combination of the foregoing.
- a multi-person video live broadcast method including: creating a three-dimensional live broadcast room; creating a first target image in the three-dimensional live broadcast room based on a first video live stream sent by a first terminal device, wherein the first terminal device corresponds to an anchor user, and the first target image represents a live broadcast room screen of a two-dimensional live broadcast room corresponding to the first video live stream; sending a second video live stream corresponding to the three-dimensional live broadcast room to a target terminal device, wherein the target terminal device includes the first terminal device and/or the second terminal device, and the second terminal device corresponds to an audience user.
- the first target image is created in the three-dimensional live broadcast room based on the first video live stream sent by the first terminal device, including: determining a target area of a live image frame in the first video live stream, the target area being the area where the anchor image is located in the live image frame; segmenting the live image frame based on the target area to generate the first target image; and setting the first target image in the three-dimensional live broadcast room.
- the method further includes: receiving control information sent by the target terminal device, the control information representing an observation position and/or observation angle of an observation point in the three-dimensional live broadcast room, the observation point being used to obtain an image in the three-dimensional live broadcast room; and generating the second video live stream according to the control information.
- generating the second video live stream according to the control information includes: obtaining the spatial observation range of the observation point in the three-dimensional live broadcast room according to the control information sent by the target terminal device; and performing image acquisition on the three-dimensional live broadcast room based on the spatial observation range to generate the second video live stream.
- the first The second video live stream includes: obtaining a target display direction of the first target image according to the control information, wherein an angle between the target display direction and a direction of a position vector is less than an angle threshold, and the position vector is composed of a first spatial position corresponding to the first target image and an observation position corresponding to the control information; generating the second video live stream according to the target display direction.
- the three-dimensional live broadcast room includes at least one first spatial position and/or at least one second spatial position, wherein the first spatial position is used to display the first target image, the second spatial position is used to display a virtual object, the first spatial position is determined based on first request information of a host user entering the three-dimensional live broadcast room, and the second spatial position is determined based on second request information of an audience user entering the three-dimensional live broadcast room.
- the method further includes: obtaining first request information of a target audience user entering the three-dimensional live broadcast room; based on the first request information, obtaining a target appearance of a corresponding target virtual object; and based on the target appearance, generating the target virtual object.
- the method also includes: obtaining action information corresponding to the target audience user in response to the action request sent by the second terminal device; generating a target action of the target virtual object corresponding to the target audience user based on the action information; and controlling the target virtual object to move in the three-dimensional live broadcast room based on the target action.
- the method further includes: obtaining a target screen of a host user corresponding to a host terminal device; and generating a second target image in the three-dimensional live broadcast room based on the target screen of the host user, wherein the second target image represents a virtual object corresponding to the host user.
- after creating a three-dimensional live broadcast room also includes: sending an invitation message to the first terminal device, wherein the invitation message is used to invite the first terminal device to enter the three-dimensional live broadcast room; after receiving the response message sent by the first terminal device, receiving the first video live stream sent by the first terminal device.
- a method for live streaming video comprising: entering a three-dimensional live broadcast room; sending a first video live stream to a host terminal device that creates the three-dimensional live broadcast room to create a first target image in the three-dimensional live broadcast room, wherein the first terminal device corresponds to an anchor user, and the first target image represents a live broadcast room screen of a two-dimensional live broadcast room corresponding to the first video live stream; receiving a second video live stream corresponding to the three-dimensional live broadcast room, and playing the second video live stream.
- the method also includes: sending control information, wherein the control information represents an observation position and/or an observation angle of an observation point in the three-dimensional live broadcast room, and the observation point is used to obtain an image in the three-dimensional live broadcast room; receiving a second video live stream corresponding to the three-dimensional live broadcast room, and playing the second video live stream, including: receiving a second video live stream generated based on the control information corresponding to the three-dimensional live broadcast room, and playing the second video live stream generated based on the control information.
- a multi-person video live broadcast device including: a creation module for creating a three-dimensional live broadcast room; a generation module for creating a first target image in the three-dimensional live broadcast room based on a first video live stream sent by a first terminal device, wherein the first terminal device corresponds to an anchor user, and the first target image represents a live broadcast room screen of a two-dimensional live broadcast room corresponding to the first video live stream; a transceiver module for sending a second video live stream corresponding to the three-dimensional live broadcast room to a target terminal device, wherein the target terminal device includes the first terminal device and/or the second terminal device, and the second terminal device corresponds to an audience user.
- the generation module is specifically used to: determine a target area of a live image frame in the first video live stream, the target area being the area where the anchor image is located in the live image frame; segment the live image frame based on the target area to generate the first target image; and set the first target image in the three-dimensional live broadcast room.
- the generation module is further used to: receive control information sent by the target terminal device, the control information represents the observation position and/or observation angle of the observation point in the three-dimensional live broadcast room, and the observation point is used to obtain the image in the three-dimensional live broadcast room; generate the second video live stream according to the control information.
- the generation module when the generation module generates the second video live stream according to the control information, it is specifically used to: obtain the spatial observation range of the observation point in the three-dimensional live broadcast room according to the control information sent by the target terminal device; and perform image acquisition on the three-dimensional live broadcast room based on the spatial observation range to generate the second video live stream.
- the generation module when the generation module generates the second video live stream according to the control information, it is specifically used to: obtain the target display direction of the first target image according to the control information, wherein the angle between the target display direction and the direction of the position vector is less than an angle threshold, and the position vector is composed of the first spatial position corresponding to the first target image and the observation position corresponding to the control information; generate the second video live stream according to the target display direction Live streaming.
- the three-dimensional live broadcast room includes at least one first spatial position and/or at least one second spatial position, wherein the first spatial position is used to display the first target image, the second spatial position is used to display a virtual object, the first spatial position is determined based on first request information of a host user entering the three-dimensional live broadcast room, and the second spatial position is determined based on second request information of an audience user entering the three-dimensional live broadcast room.
- the generation module is further used to: obtain first request information of a target audience user entering the three-dimensional live broadcast room; based on the first request information, obtain the target appearance of the corresponding target virtual object; and based on the target appearance, generate the target virtual object.
- the generation module is also used to: obtain action information corresponding to the target audience user in response to the action request sent by the second terminal device; generate a target action of the target virtual object corresponding to the target audience user based on the action information; and control the movement of the target virtual object in the three-dimensional live broadcast room based on the target action.
- the generation module is also used to: obtain a target screen of a host user corresponding to a host terminal device; and generate a second target image in the three-dimensional live broadcast room based on the target screen of the host user, wherein the second target image represents a virtual object corresponding to the host user.
- the transceiver module is also used to: send an invitation message to the first terminal device, wherein the invitation message is used to invite the first terminal device to enter the three-dimensional live broadcast room; after receiving the response message sent by the first terminal device, receive the first video live stream sent by the first terminal device.
- a multi-person video live broadcast device including: a processing module for entering a three-dimensional live broadcast room; a transceiver module for sending a first video live stream to a host terminal device that creates the three-dimensional live broadcast room, so as to create a first target image in the three-dimensional live broadcast room, wherein the first target image represents an image of a host in the first video live stream; and a playback module for receiving a second video live stream corresponding to the three-dimensional live broadcast room, and playing the second video live stream.
- the transceiver module is further used to: send control information, wherein the control information represents the observation position and/or observation angle of the observation point in the three-dimensional live broadcast room, The observation point is used to obtain the image in the three-dimensional live broadcast room; the playback module is specifically used to: receive the second video live stream generated based on the control information corresponding to the three-dimensional live broadcast room, and play the second video live stream generated based on the control information.
- an electronic device comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the multi-person video live broadcast method as described in the first aspect and various possible designs of the first aspect, or to implement the multi-person video live broadcast method as described in the second aspect and various possible designs of the second aspect.
- a computer-readable storage medium stores computer execution instructions.
- the multi-person video live broadcast method as described in the first aspect and various possible designs of the first aspect is implemented, or the multi-person video live broadcast method as described in the second aspect and various possible designs of the second aspect is implemented.
- an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the multi-person video live broadcast method as described in the first aspect and various possible designs of the first aspect, or implements the multi-person video live broadcast method as described in the second aspect and various possible designs of the second aspect.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Business, Economics & Management (AREA)
- Marketing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
本公开实施例提供一种多人视频直播方法、装置、电子设备及存储介质,通过创建三维直播间;基于第一终端设备发送的第一视频直播流,在三维直播间内创建第一目标图像,其中,第一终端设备对应主播用户,第一目标图像表征第一视频直播流对应的二维直播间的直播间画面;向目标终端设备发送三维直播间对应的第二视频直播流,其中,目标终端设备包括第一终端设备和/或第二终端设备,第二终端设备对应观众用户。由此提升了三维直播间的展示效果。
Description
本申请要求2023年6月8日递交的、标题为“多人视频直播方法、装置、电子设备及存储介质”、申请号为202310679137.3的中国发明专利申请的优先权,该申请的全部内容通过引用结合在本申请中。
本公开实施例涉及互联网技术领域,尤其涉及一种多人视频直播方法、装置、电子设备及存储介质。
当前,在视频直播的应用场景中,多人直播是指多个主播用户之间,通过视频连线的方式进行共同直播,使观众用户可以观看到多个主播之间的互动,提高视频直播效果。
然而,现有技术实现多人直播的方式,通常是针对各主播用户进行分屏展示,造成了空间割裂感强、视觉效果差的问题,影响多人直播场景下的整体直播效果。
发明内容
本公开实施例提供一种多人视频直播方法、装置、电子设备及存储介质,以克服多人直播过程中主播用户画面间空间割裂感强、视觉效果差的问题。
第一方面,本公开实施例提供一种多人视频直播方法,应用于主机终端设备,包括:创建三维直播间;基于第一终端设备发送的第一视频直播流,在所述三维直播间内创建第一目标图像,其中,所述第一终端设备对应主播用户,所述第一目标图像表征所述第一视频直播流对应的二维直播间的直播间画面;向目标终端设备发送所述三维直播间对应的第二视频直播流,其中,所述目标终端设备包括所述第一终端设备和/或第二终端设备,所述第二终端设备对应观众用户。
第二方面,本公开实施例提供一种多人视频直播方法,应用于第一终端设备,包括:进入三维直播间;向创建所述三维直播间的主机终端设备发送第一视频直播流,以在所述三维直播间内创建第一目标图像,其中,所述第一终端设备对应主播用户,所述第一目标图像表征所述第一视频直播流对应的二维直播间的直播间画面;接收所述三维直播间对应的第二视频直播流,并对所述第二视频直播流进行播放。
第三方面,本公开实施例提供一种多人视频直播装置,包括:创建模块,用于创建三维直播间;生成模块,用于基于第一终端设备发送的第一视频直播流,在所述三维直播间内创建第一目标图像,其中,所述第一终端设备对应主播用户,所述第一目标图像表征所述第一视频直播流对应的二维直播间的直播间画面;收发模块,用于向目标终端设备发送所述三维直播间对应的第二视频直播流,其中,所述目标终端设备包括所述第一终端设备和/或第二终端设备,所述第二终端设备对应观众用户。
第四方面,本公开实施例提供一种多人视频直播装置,包括:处理模块,用于进入三维直播间;发送模块,用于向创建所述三维直播间的主机终端设备发送第一视频直播流,以在所述三维直播间内创建第一目标图像,其中,所述第一终端设备对应主播用户,所述第一目标图像表征所述第一视频直播流对应的二维直播间的直播间画面;接收模块,用于接收所述三维直播间对应的第二视频直播流,并对所述第二视频直播流进行播放。
第五方面,本公开实施例提供一种电子设备,包括:处理器,以及与所述处理器通信连接的存储器;所述存储器存储计算机执行指令;所述处理器执行所述存储器存储的计算机执行指令,以实现如上第一方面以及第一方面各种可能的设计所述的多人视频直播方法,或者,以实现如上第二方面以及第二方面各种可能的设计所述的多人视频直播方法。
第六方面,本公开实施例提供一种计算机可读存储介质,所述计算机可读存储介质中存储有计算机执行指令,当处理器执行所述计算机执行指令时,实现如上第一方面以及第一方面各种可能的设计所述的多人视频直播方法,或者,实现如上第二方面以及第二方面各种可能的设计所述的多人视频直播方法。
第六方面,本公开实施例提供一种计算机程序产品,包括计算机程序,
该计算机程序被处理器执行时实现如上第一方面以及第一方面各种可能的设计所述的多人视频直播方法,或者,实现如上第二方面以及第二方面各种可能的设计所述的多人视频直播方法。
为了更清楚地说明本公开实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍,显而易见地,下面描述中的附图是本公开的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的多人视频直播方法的一种应用场景图;
图2为本公开实施例提供的多人视频直播方法的流程示意图一;
图3为本公开实施例提供的一种在三维直播间内创建第一目标图像的示意图;
图4为图2所示实施例中步骤S102的具体实现方式的流程图;
图5为图2所示实施例中步骤S104的具体实现方式的流程图;
图6为图2所示实施例中步骤S105的具体实现方式的流程图;
图7为本公开实施例提供的一种在目标虚拟对象在三维直播间内运动的示意图;
图8为本公开实施例提供的多人视频直播方法的流程示意图二;
图9为图8所示实施例中步骤S208的具体实现方式的流程图;
图10为本公开实施例提供的一种确定目标展示方向的示意图;
图11为本公开实施例提供的多人视频直播方法的流程示意图三;
图12为本公开实施例提供的一种多人视频直播装置的结构框图;
图13为本公开实施例提供的另一种多人视频直播装置的结构框图;
图14为本公开实施例提供的一种电子设备的结构示意图;
图15为本公开实施例提供的电子设备的硬件结构示意图。
为使本公开实施例的目的、技术方案和优点更加清楚,下面将结合本公开实施例中的附图,对本公开实施例中的技术方案进行清楚、完整地描述,
显然,所描述的实施例是本公开一部分实施例,而不是全部的实施例。基于本公开中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都属于本公开保护的范围。
需要说明的是,本公开所涉及的请求信息(包括但不限于用户设备信息、用户个人信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等),均为经用户授权或者经过各方充分授权的信息和数据,并且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准,并提供有相应的操作入口,供用户选择授权或者拒绝。
下面对本公开实施例的应用场景进行解释:
本公开实施例提供的多人视频直播方法,可以应用于多人连线直播的应用场景下。图1为本公开实施例提供的多人视频直播方法的一种应用场景图,具体地,本公开实施例提供的方法,可以应用于主播用户所使用的终端设备。例如,以主播用户User_1所使用的终端设备A为例,终端设备A在运行直播应用客户端(Application,APP)后,响应用户指令,创建视频直播间进行视频直播。之后,直播用户User_1可以通过终端设备A与主播用户User_2所使用的终端设备B进行视频连线,也称为“连线PK”。在视频连线建立后,在终端设备A(基于需要,还可以同时在终端设备B)一侧的直播应用客户端内,同时显示主播用户User_1的视频画面和主播用户User_2的视频画面,从而实现多个主播用户之间的多人互动直播。与此同时,上述直播间的视频直播流,通过服务器向观众用户所使用的终端设备C进行分发,从而使观众用户可以观看到上述直播间内多人直播的直播内容。
现有技术中,如图1中所示,实现多人直播的方式,通常是针对各主播用户进行分屏展示,造成了空间割裂感强、视觉效果差的问题,影响多人直播场景下的整体直播效果。本公开实施例提供一种多人视频直播方法以解决上述问题。
参考图2,图2为本公开实施例提供的多人视频直播方法的流程示意图一。本实施例的方法可以应用于视频直播服务器,或者主机终端设备中。本实施例中以应用于主机终端设备(可以是主持/房主用户对应的终端)的情况为例进行介绍。示例性地,本实施例提供的进行多人视频直播方法,包括:
步骤S101:创建三维直播间。
示例性地,参考图1所示的应用场景示意图,主机终端设备在运行直播应用后,响应于用户的房间创建指令,可以创建一个三维直播间,该三维直播间时基于三维直播间形式的直播间。其中,在视频直播的应用场景下,直播间是由直播应用的服务端提供的一个用于输出视频直播流的逻辑单元,进入直播间的观众用户,即可接收该直播间对应的视频直播流,从而观看该直播间内的直播内容。现有技术中的直播间为二维直播间,即在一个固定的平面背景之上,展示主播用户对应的视频内容,构成此类二维直播间的数据包括直播数据流和平面图像。因此在多人直播的场景下,二维直播间接收不同主播用户的直播数据流,并通过分区域显示的方式,在平面图像上显示不同的主播用户的直播画面。
而本实施例中的三维直播间可以是通过三维图像引擎构建的具有三维立体效果的直播间,构成三维直播间的数据包括三维直播间对应的图像模型数据和视频直播流。
而本实施例提供的三维直播间,是指在三维直播间时,需要进行三维图像数据加载、渲染三维直播间渲染等步骤,从而实现三维直播间的创建。进一步地,创建后的三维直播间对应的三维直播间内,包含有虚拟物体。通过虚拟物体可以表现出不同的虚拟环境场景,从而实现对现实环境场景的模拟,例如演唱会场景、综艺节目场景、比赛场景等。上述三维直播间内的具体内容以及整体虚拟场景的实现,可以通过用户指令进行选择,三维直播间可以通过现有的三维游戏引擎来进行初始化和创建,具体实现方式此处不再赘述。
步骤S102:基于第一终端设备发送的第一视频直播流,在三维直播间内创建第一目标图像,其中,第一终端设备对应主播用户,第一目标图像表征第一视频直播流对应的二维直播间的直播间画面。
示例性地,在主机终端设备创建三维直播间后,操作主机终端设备的用户可以视为主持/房主用户。之后,主持/房主用户可以通过主机终端设备向一个或多个主播用户对应的第一终端设备发送邀请信息,从而使主播用户加入该三维直播间。或者,主播用户通过接收链接推送、自主搜索等方式,直接加入该三维直播间。在主播用户加入该三维直播间后,相当于主播用户所对应的第一终端设备与创建该三维直播间的主机终端设备之间建立连接状态。之后,第一终端设备将第一视频直播流以直接或间接的方式发送至主机终端
设备,主机终端设备将第一视频直播流中的视频内容渲染至三维直播间中,从而在三维直播间内生成一个表征第一视频直播流中主播用户图像画面的第一目标图像。
图3为本公开实施例提供的一种在三维直播间内创建第一目标图像的示意图。如图3所示,示例性地,主机终端设备在创建三维直播间后,渲染一个对应的三维直播间。参考图3中所示,三维直播间中包括基于三维图像引擎渲染的虚拟对象,例如舞台、观众席,以及观众席上的观众等。之后,主机终端设备将接收到由终端设备D2_1发送的主播用户U1对应的第一视频直播流steam_1(图中示为steam_1),以及由终端设备D2_2发送的主播用户U2对应的第一视频直播流steam_2(图中示为steam_2)。主机终端设备分别将基于第一视频直播流steam_1生成的直播画面P1和基于第一视频直播流steam_2生成的直播画面P2(直播画面P1和直播画面P2即为第一目标图像)设置于三维直播间中的pos_1和pos_2位置处播放。从而在视觉上使主播用户U1的图像画面和主播用户U2的图像画面同时位于三维直播间中,实现主播用户的图像画面与虚拟环境的融合。
一种可能的实现方式中,主机终端设备在接收到第一视频直播流后,首先对第一视频直播流进行处理,将主播图像部分与背景部分进行分割,之后再在三维直播间内生成的第一目标图像。使在三维直播间内生成的第一目标图像更加细致和真实。示例性地,如图4所示,步骤S102的具体实现方式包括:
步骤S1021:确定第一视频直播流中的直播图像帧的目标区域,目标区域为直播图像帧中主播图像所在的区域。
步骤S1022:基于目标区域分割直播图像帧,生成第一目标图像。
步骤S1023:将第一目标图像设置于三维直播间内。
示例性地,主机终端设备在得到第一视频直播流后,通过必要的解码处理后,可以得到对应的直播图像帧,具体实现过程为现有技术,此处不再赘述。之后主机终端设备针对每一直播图像帧进行图像识别和分割,将直播图像帧中主播图像所在的目标区域内的图像分割出来,而将直播图像帧中主播图像之外的区域的图像丢弃,从而得到去除背景的主播图像,即第一目标图像。在一种可能的实现方式中,上述直播图像帧中的目标区域,可以通过直
播图像帧中的图像颜色确定。例如,第一视频直播流是第一终端设备结合绿幕对主播用户进行拍摄后生成的,在该第一视频直播流的直播图像帧中,人物背景为绿色,因此主机终端设备可通过颜色对目标区域进行划分。另一种可能的实现方式中,则主机终端设备通过确定直播图像帧中的图像元素的轮廓,并对图像元素进行特征识别,也可确定其中的目前区域。上述确定目标区域的具体实现方式,可以根据具体情况设置,此处不做具体限制。
示例性地,可选地,在步骤S101之后,还包括:
步骤S104:当观众用户进入三维直播间后,在三维直播间内生成观众用户对应的目标虚拟对象。
示例性地,三维直播间中包括至少一个第一空间位置和/或至少一个第二空间位置,其中,第一空间位置用于显示第一目标图像,第二空间位置用于显示虚拟对象。第一空间位置基于进入三维直播间的主播用户的第一请求信息确定,第二空间位置基于进入三维直播间的观众用户的第二请求信息确定。
示例性地,第一请求信息即进入目标主播间的主播用户的标识,而第二请求信息即进入目标主播间的观众用户的标识。主机终端设备可以通过直播应用的服务端获得上述第一请求信息和第二请求信息。进而,基于第一请求信息和第二请求信息,设置第一目标图像所在的第一空间位置,以及虚拟对象所在的第二空间位置。例如,根据主播用户与主持/房主用户的关联度,确定对应的第一空间位置。再例如,根据观众用户和主播用户的关联度,确定对应的第二空间位置。
本实施例中,通过获取主播用户和观众用户的标识,来确定对应的空间位置,能够在三维直播间实现更加精确的图像元素的位置控制,使三维直播间所表现的模拟场景更加真实。
进一步地,基于之前实施例步骤中的介绍,对于进行三维直播间的主播用户,通过在三维直播间内展示包含主播图像的视频画面,可以实现主播用户之间的互动。然而,对于进入三维直播间的观众用户,则无法在三维直播间中进行差异化展示,以及无法与主播用户进行互动。为解决上述问题,如图5所示,步骤S104的具体实现方式包括:
步骤S1041:当目标观众用户进入三维直播间后,获取进入三维直播间的目标观众用户的第二请求信息。
步骤S1042:基于第二请求信息,得到对应的目标虚拟对象的目标外观。
步骤S1043:基于目标外观,在三维直播间内生成目标虚拟对象。
示例性地,第二请求信息与虚拟对象的外观之间存在预设的映射关系。例如,第二请求信息表征的直播间级别,映射对应的外观颜色。因此基于第二请求信息和上述映射关系,即可得到对应的目标虚拟对象的目标外观。之后,以该目标外观作为渲染参数,将目标虚拟对象渲染至三维直播间中,实现三维直播间内生成第二请求信息对应的目标虚拟对象的目的。其中,上述映射关系仅为示例性地,第二请求信息表征与目标外观之间的具体映射实现可以根据需要设置,此处不做具体限制。
进一步地,可选地,在步骤S104之后,还包括:
步骤S105:响应于第二终端设备发送的动作请求,控制目标虚拟对象在三维直播间内运动。
示例性地,观众用户通过操作第二终端设备,可以向主机终端设备直接或间接的发送动作请求。例如,通过在第二终端设备一侧触发指定的组件、道具后,第二终端设备通过服务器向主机终端设备发送动作请求,主机终端设备接收动作请求后,确定该动作请求对应的目标观众用户,进而确定目标观众用户在三维直播间中对应的目标虚拟对象。例如通过三维图像引擎生产的目标虚拟对象,之后,基于该动作请求所表征的具体内容,控制目标虚拟对象在三维直播间内运动。从而实现观众用户与主播用户之间的互动和交流。
如图6所示,示例性地,步骤S105的具体实现方式包括:
步骤S1051:响应于第二终端设备发送的动作请求,获得目标观众用户对应的动作信息。
步骤S1052:基于动作信息,生成目标观众用户对应的目标虚拟对象的目标动作。
步骤S1053:控制目标虚拟对象在三维直播间内基于目标动作运动。
图7为本公开实施例提供的一种在目标虚拟对象在三维直播间内运动的示意图。如图7所示,在目标观众用户进入三维直播间后,主机终端设备接收到进入消息Info_1(图中示为Info_1),该消息Info_1可以是第二终端设备通过服务器间接发送给主机终端设备的。之后主机终端设备在三维直播间所对应的虚拟三维空间中,为该目标观众用户创建一个对应的目的虚拟对象
P1。在初始状态下,该虚拟对象P1处于双臂垂下的站立姿态。之后,在主机终端设备接收到第一终端设备D2通过服务器间接发送的动作信息Info_2(图中示为Info_2),主机终端设备基于该动作信息Info_2,确定对应的目标动作,例如为高举双手的欢呼动作,并基于该目标动作,控制该虚拟对象P1运动,以做出该高举双手的欢呼动作,从而实现观众用户对位于三维直播间中对应的虚拟对象的控制。使观众用户与主播用户、观众用户与观众用户之间产生互动,提高三维直播间的互动效果,以及提高多人直播场景下的整体直播效果。
步骤S103:向目标终端设备发送三维直播间对应的第二视频直播流,其中,目标终端设备包括第一终端设备和/或第二终端设备,第二终端设备对应观众用户。
示例性地,主机终端设备在创建的上述三维直播间中生成第一目标图像以及虚拟对象后,利用三维直播间对应的数据渲染为对应的直播数据流,即第二视频直播流,并上传给直播应用的服务器。之后由直播应用的服务器将该第二视频直播流进行分发至第一终端设备和/或第二终端设备,即目标终端设备。从而使第一终端设备和/或第二终端设备能够在所运行的直播客户端(或者主播客户端)一侧,观看到实际的三维直播间内的图像。由于经过上述步骤后,多个主播用户位于同一个三维直播间中,主播用户的图像与虚拟环境相融合,因此具有更好的整体性和真实性。同时可以实现多个主播用户之间在三维直播间中的互动与交流,提供三维直播间的互动效率。
在本实施例中,通过创建三维直播间;基于第一终端设备发送的第一视频直播流,在三维直播间内创建第一目标图像,其中,第一终端设备对应主播用户,第一目标图像表征第一视频直播流对应的二维直播间的直播间画面;向目标终端设备发送三维直播间对应的第二视频直播流,其中,目标终端设备包括第一终端设备和/或第二终端设备,第二终端设备对应观众用户。通过创建三维直播间,将主播用户的图像(主播图像)画面透射到该三维直播间中进行展示,从而使不同主播用户的图像画面与环境融为一体,减少空间割裂感、提高视觉效果差,提升多人直播场景下的整体直播效果。
参考图8,图8为本公开实施例提供的多人视频直播方法的流程示意图二。本实施例中详细描述在图2所示实施例的基础上,进一步增加了基于控
制信息进行处理的步骤,该多人视频直播方法包括:
步骤S201:创建三维直播间。
步骤S202:获取主机终端设备对应的房主/主持用户的目标画面。
步骤S203:基于主持用户的目标画面,在三维直播间内生成第二目标图像,第二目标图像表征主持用户对应的虚拟对象。
示例性地,在本实施例中,主机终端设备在创建三维直播间后,进一步在三维直播间内创建基于自身形象形成的虚拟对象,作为该三维直播间的主持人角色。具体的,首先,主机终端设备通过图像采集单元,采集位于主机终端设备一侧的主持用户的目标画面。之后,基于特征提取,得到主播图像特征。再之后,以得到的主播图像特征作为参数输入三维图像引擎,从而在三维直播间内渲染出对应的虚拟对象,例如为主持人外观的虚拟人物,也即第二目标图像。
步骤S204:向第一终端设备发送邀请信息,邀请信息用于邀请第一终端设备进入三维直播间。
步骤S205:在接收到第一终端设备发送的响应信息后,接收第一终端设备发送的第一视频直播流。
进一步地,在创建三维直播间后,一种可能的实现方式中,主机终端设备基于用户指令,向第一终端设备发送邀请信息,以邀请第一终端设备一侧的主播用户加入该三维直播间。第一终端设备在接收到该邀请信息后,在直播应用的客户端内显示对应的提示标识以及对应的选择组件。提示标识例如为“XXX用户邀请您加入XR直播间”,对应显示的选择组件的外观为“是”和“否”。当主播用户点击“是”对应的选择组件后,第一终端设备向主机终端设备发送响应信息。主机终端设备在接收到该响应信息后,与第一终端设备建立视频连接,第一终端设备将对应主播用户当前所在直播间的直播视频流,即第一视频直播流,发送至主机终端设备。从而使主机终端设备接收到第一视频直播流并进行后续的处理步骤。
步骤S206:基于第一终端设备发送的第一视频直播流,在三维直播间内创建第一目标图像。
步骤S207:接收目标终端设备发送的控制信息,控制信息表征三维直播间内的观察点的观察位置和/或观察角度,观察点用于获取三维直播间中内的
图像。
步骤S208:根据控制信息,生成目标终端设备对应的第二视频直播流。
示例性地,目标终端设备可以为第一终端设备或第二终端设备,即主播用户或观众用户所使用的终端设备。主播用户和观众用户均可以通过对应的终端设备(内运行的客户端)来调节直播画面在三维直播间中的位置和视角。下面以观众用户为例,进行详细介绍。示例性地,目标终端设备为第二终端设备,第二终端设备接收到观众用户的操作指令后,向主机终端设备发送控制信息。在一些实施例中,操作指令例如旋转镜头指令、移动镜头位置指令。通过上述旋转镜头指令、移动镜头位置指令,可以得到画面镜头在三维直播间内的位置参数和角度参数,即控制信息。在一些实施例中,该画面镜头即三维直播间内的观察点,位置参数即表征观察位置的信息,角度参数即表征观察角度的信息。主机终端设备在接收到上述控制信息后,基于该控制信息所表征的观察位置和/或观察角度,渲染得到对应的二维图像,从而生成第二视频直播流,并返回给第二终端设备。从而使第二终端设备一侧的观众用户能够看到与其所施加的旋转镜头指令、移动镜头位置指令相匹配的直播画面。
其中,需要说明的是,控制信息可以同时包括观察位置和观察角度对应的信息,也可以仅包括观察位置或观察角度对应的信息。当控制信息仅包括观察位置或观察角度对应的信息时,另一个参数为可自动生成或固定设置。例如,控制信息中包括观察位置的参数时,则观察角度可以根据观察位置和第一目标图像所在的第一空间位置构成的向量的方向自动确定,从而使观察角度始终朝向第一目标图像。当控制信息中包括观察角度的参数时,则观察位置可以为预设值,该预设值例如是基于之前的移动镜头位置指令确定的。具体实现方式可以根据需要确定,此处不再赘述。
一种可能的实现方式中,如图9所示,步骤S208包括步骤S2081、S2082和S2083三个具体的实现步骤:
步骤S2081:根据目标终端设备发送的控制信息,得到观察点在三维直播间内的空间观察范围。
步骤S2082:根据控制信息,获得第一目标图像的目标展示方向,其中,目标展示方向与位置向量的方向的夹角小于角度阈值,位置向量由第一目标图像对应的第一空间位置和控制信息对应的观察位置构成。
步骤S2083:基于空间观察范围和目标展示方向,对三维直播间进行图像采集,生成第二视频直播流。
示例性地,控制信息包括观察位置和观察角度,根据观察位置,可以确定观察点在三维直播间中的定位。之后根据观察角度和预设的视野角度(例如为以观察角度为中心上下60度、左右120度的角度范围),可以确定一个空间观察范围。之后,一种可能的实现方式中,若第一目标图像(所在的第一空间位置)位于该空间观察范围之外,则基于空间观察范围对三维直播间进行图像采集、编码,生成由二维图像帧构成的第二视频直播流。在具体实现过程中,用户每次通过输入操作指令(例如旋转视角),在目标终端设备生成控制信息。并由目标终端设备发送后,主机终端设备根据控制信息,生成至少一帧二维图像帧,并通过第二视频直播流发送回目标终端设备进行显示,从而产生三维视图的效果。
另一种可能的实现方式中,若第一目标图像(所在的第一空间位置)位于该空间观察范围之内,则进一步根据控制信息,获得第一目标图像的目标展示方向。在一些实施例中,第一目标图像的展示方向即第一目标图像的朝向,第一目标图像的目标展示方向由第一目标图像对应的第一空间位置和控制信息对应的观察位置构成的位置向量的方向确定。例如,第一目标图像的目标展示方向与该第一空间位置和观察位置构成的位置向量的方向平行,或夹角小于角度阈值。之后将基于目标展示方向展示的第一目标图像和空间观察范围对应的三维直播间进行图像采集、编码,生成由二维图像帧构成的第二视频直播流。在该第二视频直播流的二维图像帧中,第一目标图像的展示方向始终朝向观察点,避免由于观察点移动而导致第一目标图像在直播画面中发生偏转的问题,提高画面效果。
图10为本公开实施例提供的一种确定目标展示方向的示意图。下面结合体图10对上述步骤进行进一步说明。如图10所示,根据第二终端设备发送的控制信息vers_1(图中示为vers_1),当观察点被设置在三维直播间中的A_1位置时,位于B位置的第一目标图像的目标展示方向为seta_1,seta_1即A_1位置和B位置的连线方向,主机终端设备采集三维直播间在空间观察范围内的图像,以及以seta_1为展示方向进行展示的第一目标图像。之后对二者进行融合渲染和编码,生成第二视频直播流steam_1。
根据第二终端设备发送的控制信息vers_2(图中示为vers_2),当观察点被设置在三维直播间中的A_2位置时,位于B位置的第一目标图像的目标展示方向为seta_2,seta_2即A_2位置和B位置的连线方向。主机终端设备采集三维直播间在空间观察范围内的图像,以及以seta_2为展示方向进行展示的第一目标图像。之后对二者进行融合渲染和编码,生成第二视频直播流steam_2。
基于上图所示实施例,第二视频直播流steam_1和第二视频直播流steam_2虽然对应不同的观察点的观察位置,但由于目标展示方向随观察位置进行动态调整,因此保证了在第二视频直播流steam_1和第二视频直播流steam_2中,第一目标图像始终面向直播镜头(观察点),保证第一目标图像在任一观察位置均能够被正常展示。
步骤S209:向各目标终端设备发送对应的第二视频直播流。
示例性地,在生成第二视频直播流后,向第一终端设备和/或第二终端设备发送,从而使进入直播间的各主播用户和各观众用户能够看到基于其各自所设置的控制信息而生成的直播画面。
在本实施例中,步骤S201、S206、S209与上述实施例中步骤S101-S103的实现方式一致,详细论述请参考图2所示实施例中的相关的论述,这里不再赘述。
本实施例提供的多人视频直播方法、装置、电子设备及存储介质,通过创建三维直播间;基于第一终端设备发送的第一视频直播流,在所述三维直播间内创建第一目标图像,其中,所述第一终端设备对应主播用户,所述第一目标图像表征所述第一视频直播流对应的二维直播间的直播间画面;向目标终端设备发送所述三维直播间对应的第二视频直播流,其中,所述目标终端设备包括所述第一终端设备和/或第二终端设备,所述第二终端设备对应观众用户。通过创建三维直播间,将主播图像画面透射到该三维直播间中进行展示,从而使不同用户主播图像画面与环境融为一体,减少空间割裂感、提高视觉效果差,提升多人直播场景下的整体直播效果。
参考图11,图11为本公开实施例提供的多人视频直播方法的流程示意图三。本实施例的方法可以应用于主播用户所使用的第一终端设备中,示例性地,本实施例提供的进行多人视频直播方法,包括:
步骤S301:进入三维直播间。
步骤S302:向创建三维直播间的主机终端设备发送第一视频直播流,以在三维直播间内创建第一目标图像,其中,第一目标图像表征第一视频直播流对应的二维直播间的直播间画面。
步骤S303:接收三维直播间对应的第二视频直播流,并对第二视频直播流进行播放。
示例性地,本实施例提供的方法,以主播用户一侧的第一终端设备为执行主体,第一终端设备在进入三维直播间后,向主机终端设备发送第一视频直播流,以使主播用户的直播画面在三维直播间中生成。之后,接收主机终端设备对三维直播间进行渲染后生成的第二视频直播流,并基于第二视频直播流进行播放,即可实现不同主播用户在三维虚拟环境下的主播连线。由于各主播用户通过上述步骤,可以看到自己和其他主播用户的主播图像,因此可以实现三维环境下的直播互动,提高多人直播的直播效果。上述步骤的具体实现过程,在之前实施例中已进行介绍,其具体实现方式可参见之前实施例中主机终端设备而第一终端设备之间的交互步骤,此处不再赘述。
进一步地,在此基础上,本实施例提供的方法还包括:
步骤S304,发送控制信息,控制信息表征三维直播间内的观察点的观察位置和/或观察角度,观察点用于获取三维直播间中内的图像。
在发送控制信息后,步骤S303的具体实现方式为:接收三维直播间对应的基于控制信息生成的第二视频直播流,并对第二视频直播流进行播放。
示例性地,在主播用户加入三维直播间后,还可以进一步的通过操作第一终端设备,向主机终端设备发送控制信息,从而控制三维直播间内的观察点的观察位置和角度,从而实现与其他主播用户之间更好的互动效果。其具体实现方式可参见之前实施例中第二终端设备向主机终端设备发送控制信息的相关步骤,其实现方式类似,此处不再赘述。
对应于上文实施例的多人视频直播方法,图12为本公开实施例提供的一种多人视频直播装置的结构框图,应用于主机终端设备。为了便于说明,仅示出了与本公开实施例相关的部分。参照图12,多人视频直播装置4包括:
创建模块41,用于创建三维直播间。
生成模块42,用于基于第一终端设备发送的第一视频直播流,在三维直
播间内创建第一目标图像,其中,第一终端设备对应主播用户,第一目标图像表征第一视频直播流对应的二维直播间的直播间画面。
收发模块43,用于向目标终端设备发送三维直播间对应的第二视频直播流,其中,目标终端设备包括第一终端设备和/或第二终端设备,第二终端设备对应观众用户。
在本公开的一个实施例中,生成模块42,具体用于:确定第一视频直播流中的直播图像帧的目标区域,目标区域为直播图像帧中主播图像所在的区域;基于目标区域分割直播图像帧,生成第一目标图像;将第一目标图像设置于三维直播间内。
在本公开的一个实施例中,生成模块42,还用于:接收目标终端设备发送的控制信息,控制信息表征三维直播间内的观察点的观察位置和/或观察角度,观察点用于获取三维直播间中内的图像;根据控制信息,生成目标终端设备对应的第二视频直播流。
在本公开的一个实施例中,生成模块42在根据控制信息,生成目标终端设备对应的第二视频直播流时,具体用于:根据目标终端设备发送的控制信息,得到观察点在三维直播间内的空间观察范围;基于空间观察范围对三维直播间进行图像采集,生成第二视频直播流。
在本公开的一个实施例中,生成模块42在根据控制信息,生成目标终端设备对应的第二视频直播流时,具体用于:根据控制信息,获得第一目标图像的目标展示方向,其中,目标展示方向与位置向量的方向的夹角小于角度阈值,位置向量由第一目标图像对应的第一空间位置和控制信息对应的观察位置构成;根据目标展示方向,生成第二视频直播流。
在本公开的一个实施例中,三维直播间中包括至少一个第一空间位置和/或至少一个第二空间位置,其中,第一空间位置用于显示第一目标图像,第二空间位置用于显示虚拟对象,第一空间位置基于进入三维直播间的主播用户的第一请求信息确定,第二空间位置基于进入三维直播间的观众用户的第二请求信息确定。
在本公开的一个实施例中,生成模块42,还用于:获取进入三维直播间的目标观众用户的第一请求信息;基于第一请求信息,得到对应的目标虚拟对象的目标外观;基于目标外观,生成目标虚拟对象。
在本公开的一个实施例中,生成模块42,还用于:响应于第二终端设备发送的动作请求,获得目标观众用户对应的动作信息;基于动作信息,生成目标观众用户对应的目标虚拟对象的目标动作;控制目标虚拟对象在三维直播间内基于目标动作运动。
在本公开的一个实施例中,生成模块42,还用于:获取主机终端设备对应的主持用户的目标画面;基于主持用户的目标画面,在三维直播间内生成第二目标图像,第二目标图像表征主持用户对应的虚拟对象。
在本公开的一个实施例中,在创建三维直播间之后,收发模块42,还用于:向第一终端设备发送邀请信息,邀请信息用于邀请第一终端设备进入三维直播间;在接收到第一终端设备发送的响应信息后,接收第一终端设备发送的第一视频直播流。
其中,创建模块41、生成模块42、收发模块43依次连接。本实施例提供的多人视频直播装置4可以执行上述方法实施例的技术方案,其实现原理和技术效果类似,本实施例此处不再赘述。
图13为本公开实施例提供的另一种多人视频直播装置的结构框图,应用于第一终端设备。为了便于说明,仅示出了与本公开实施例相关的部分。参照图13,多人视频直播装置5包括:
处理模块51,用于进入三维直播间。
收发模块52,用于向创建三维直播间的主机终端设备发送第一视频直播流,以在三维直播间内创建第一目标图像,第一目标图像表征第一视频直播流对应的二维直播间的直播间画面。
播放模块53,用于接收三维直播间对应的第二视频直播流,并对第二视频直播流进行播放。
在本公开的一个实施例中,收发模块52,还用于:发送控制信息,控制信息表征三维直播间内的观察点的观察位置和/或观察角度,观察点用于获取三维直播间中内的图像;播放模块53,具体用于:接收三维直播间对应的基于控制信息生成的第二视频直播流,并对基于控制信息生成的第二视频直播流进行播放。
图14为本公开实施例提供的一种电子设备的结构示意图,如图14所示,该电子设备6包括:处理器61,以及与处理器61通信连接的存储器62;存
储器62存储计算机执行指令;处理器61执行存储器62存储的计算机执行指令,以实现如图2-图11所示实施例中的多人视频直播方法。
在一些实施例中,可选地,处理器61和存储器62通过总线63连接。
相关说明可以对应参见图2-图11所对应的实施例中的步骤所对应的相关描述和效果进行理解,此处不做过多赘述。
本公开实施例提供一种计算机可读存储介质,计算机可读存储介质中存储有计算机执行指令,计算机执行指令被处理器执行时用于实现本公开图2-图11所对应的实施例中任一实施例提供的多人视频直播方法。
本公开实施例提供一种计算机程序产品,包括计算机程序,该计算机程序被处理器执行时实现如图2-图11所示实施例中的多人视频直播方法。
参考图15,其示出了适于用来实现本公开实施例的电子设备900的结构示意图,该电子设备900可以为终端设备或服务器。其中,终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant,简称PDA)、平板电脑(Portable Android Device,简称PAD)、便携式多媒体播放器(Portable Media Player,简称PMP)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图15示出的电子设备仅仅是一个示例,不应对本公开实施例的功能和使用范围带来任何限制。
如图15所示,电子设备900可以包括处理装置(例如中央处理器、图形处理器等)901,其可以根据存储在只读存储器(ReadOnly Memory,简称ROM)902中的程序或者从存储装置908加载到随机访问存储器(Random Access Memory,简称RAM)903中的程序而执行各种适当的动作和处理。在RAM 903中,还存储有电子设备900操作所需的各种程序和数据。处理装置901、ROM 902以及RAM 903通过总线904彼此相连。输入/输出(I/O)接口905也连接至总线904。
通常,以下装置可以连接至I/O接口905:包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置906;包括例如液晶显示器(Liquid Crystal Display,简称LCD)、扬声器、振动器等的输出装置907;包括例如磁带、硬盘等的存储装置908;以及通信装置909。通信装置909可以允许电子设备900与其他设备进行无线或有线通信以交换
数据。虽然图15示出了具有各种装置的电子设备900,但是应理解的是,并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地,根据本公开的实施例,上文参考流程图描述的过程可以被实现为计算机软件程序。例如,本公开的实施例包括一种计算机程序产品,其包括承载在计算机可读介质上的计算机程序,该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中,该计算机程序可以通过通信装置909从网络上被下载和安装,或者从存储装置908被安装,或者从ROM 902被安装。在该计算机程序被处理装置901执行时,执行本公开实施例的方法中限定的上述功能。
需要说明的是,本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件,或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于:具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中,计算机可读存储介质可以是任何包含或存储程序的有形介质,该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中,计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号,其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式,包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质,该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输,包括但不限于:电线、光缆、RF(射频)等等,或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的;也可以是单独存在,而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序,当上述一个或者多个程序被该电子设备执行时,使得该电子设备执行上述实施例所示的方法。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码,上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++,还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中,远程计算机可以通过任意种类的网络——包括局域网(Local Area Network,简称LAN)或广域网(Wide Area Network,简称WAN)—连接到用户计算机,或者,可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图,图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上,流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分,该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意,在有些作为替换的实现中,方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如,两个接连地表示的方框实际上可以基本并行地执行,它们有时也可以按相反的顺序执行,这依所涉及的功能而定。也要注意的是,框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合,可以用执行规定的功能或操作的专用的基于硬件的系统来实现,或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现,也可以通过硬件的方式来实现。其中,单元的名称在某种情况下并不构成对该单元本身的限定,例如,第一获取单元还可以被描述为“获取至少两个网际协议地址的单元”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如,非限制性地,可以使用的示范类型的硬件逻辑部件包括:现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中,机器可读介质可以是有形的介质,其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备,或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
第一方面,根据本公开的一个或多个实施例,提供了一种多人视频直播方法,包括:创建三维直播间;基于第一终端设备发送的第一视频直播流,在所述三维直播间内创建第一目标图像,其中,所述第一终端设备对应主播用户,所述第一目标图像表征所述第一视频直播流对应的二维直播间的直播间画面;向目标终端设备发送所述三维直播间对应的第二视频直播流,其中,所述目标终端设备包括所述第一终端设备和/或第二终端设备,所述第二终端设备对应观众用户。
根据本公开的一个或多个实施例,所述基于第一终端设备发送的第一视频直播流,在所述三维直播间内创建第一目标图像,包括:确定所述第一视频直播流中的直播图像帧的目标区域,所述目标区域为所述直播图像帧中主播图像所在的区域;基于所述目标区域分割所述直播图像帧,生成所述第一目标图像;将所述第一目标图像设置于所述三维直播间内。
根据本公开的一个或多个实施例,所述方法还包括:接收所述目标终端设备发送的控制信息,所述控制信息表征所述三维直播间内的观察点的观察位置和/或观察角度,所述观察点用于获取所述三维直播间中内的图像;根据所述控制信息,生成所述第二视频直播流。
根据本公开的一个或多个实施例,所述根据所述控制信息,生成所述第二视频直播流,包括:根据所述目标终端设备发送的控制信息,得到所述观察点在所述三维直播间内的空间观察范围;基于所述空间观察范围对所述三维直播间进行图像采集,生成所述第二视频直播流。
根据本公开的一个或多个实施例,所述根据所述控制信息,生成所述第
二视频直播流,包括:根据所述控制信息,获得所述第一目标图像的目标展示方向,其中,所述目标展示方向与位置向量的方向的夹角小于角度阈值,所述位置向量由所述第一目标图像对应的第一空间位置和所述控制信息对应的观察位置构成;根据所述目标展示方向,生成所述第二视频直播流。
根据本公开的一个或多个实施例,所述三维直播间中包括至少一个第一空间位置和/或至少一个第二空间位置,其中,所述第一空间位置用于显示所述第一目标图像,所述第二空间位置用于显示虚拟对象,所述第一空间位置基于进入所述三维直播间的主播用户的第一请求信息确定,所述第二空间位置基于进入所述三维直播间的观众用户的第二请求信息确定。
根据本公开的一个或多个实施例,所述方法还包括:获取进入所述三维直播间的目标观众用户的第一请求信息;基于所述第一请求信息,得到对应的目标虚拟对象的目标外观;基于所述目标外观,生成所述目标虚拟对象。
根据本公开的一个或多个实施例,所述方法还包括:响应于所述第二终端设备发送的动作请求,获得目标观众用户对应的动作信息;基于所述动作信息,生成所述目标观众用户对应的目标虚拟对象的目标动作;控制所述目标虚拟对象在所述三维直播间内基于所述目标动作运动。
根据本公开的一个或多个实施例,所述方法还包括:获取主机终端设备对应的主持用户的目标画面;基于所述主持用户的目标画面,在所述三维直播间内生成第二目标图像,所述第二目标图像表征所述主持用户对应的虚拟对象。
根据本公开的一个或多个实施例,在创建三维直播间之后,还包括:向所述第一终端设备发送邀请信息,所述邀请信息用于邀请所述第一终端设备进入所述三维直播间;在接收到所述第一终端设备发送的响应信息后,接收所述第一终端设备发送的第一视频直播流。
第二方面,根据本公开的一个或多个实施例,提供了一种人视频直播方法,包括:进入三维直播间;向创建所述三维直播间的主机终端设备发送第一视频直播流,以在所述三维直播间内创建第一目标图像,其中,所述第一终端设备对应主播用户,所述第一目标图像表征所述第一视频直播流对应的二维直播间的直播间画面;接收所述三维直播间对应的第二视频直播流,并对所述第二视频直播流进行播放。
根据本公开的一个或多个实施例,所述方法还包括:发送控制信息,所述控制信息表征所述三维直播间内的观察点的观察位置和/或观察角度,所述观察点用于获取所述三维直播间中内的图像;所述接收所述三维直播间对应的第二视频直播流,并对所述第二视频直播流进行播放,包括:接收所述三维直播间对应的基于所述控制信息生成的第二视频直播流,并对基于所述控制信息生成的第二视频直播流进行播放。
第三方面,根据本公开的一个或多个实施例,提供了一种多人视频直播装置,包括:创建模块,用于创建三维直播间;生成模块,用于基于第一终端设备发送的第一视频直播流,在所述三维直播间内创建第一目标图像,其中,所述第一终端设备对应主播用户,所述第一目标图像表征所述第一视频直播流对应的二维直播间的直播间画面;收发模块,用于向目标终端设备发送所述三维直播间对应的第二视频直播流,其中,所述目标终端设备包括所述第一终端设备和/或第二终端设备,所述第二终端设备对应观众用户。
根据本公开的一个或多个实施例,所述生成模块,具体用于:确定所述第一视频直播流中的直播图像帧的目标区域,所述目标区域为所述直播图像帧中主播图像所在的区域;基于所述目标区域分割所述直播图像帧,生成所述第一目标图像;将所述第一目标图像设置于所述三维直播间内。
根据本公开的一个或多个实施例,所述生成模块,还用于:接收所述目标终端设备发送的控制信息,所述控制信息表征所述三维直播间内的观察点的观察位置和/或观察角度,所述观察点用于获取所述三维直播间中内的图像;根据所述控制信息,生成所述第二视频直播流。
根据本公开的一个或多个实施例,所述生成模块在根据所述控制信息,生成所述第二视频直播流时,具体用于:根据所述目标终端设备发送的控制信息,得到所述观察点在所述三维直播间内的空间观察范围;基于所述空间观察范围对所述三维直播间进行图像采集,生成所述第二视频直播流。
据本公开的一个或多个实施例,所述生成模块在根据所述控制信息,生成所述第二视频直播流时,具体用于:根据所述控制信息,获得所述第一目标图像的目标展示方向,其中,所述目标展示方向与位置向量的方向的夹角小于角度阈值,所述位置向量由所述第一目标图像对应的第一空间位置和所述控制信息对应的观察位置构成;根据所述目标展示方向,生成所述第二视
频直播流。
根据本公开的一个或多个实施例,所述三维直播间中包括至少一个第一空间位置和/或至少一个第二空间位置,其中,所述第一空间位置用于显示所述第一目标图像,所述第二空间位置用于显示虚拟对象,所述第一空间位置基于进入所述三维直播间的主播用户的第一请求信息确定,所述第二空间位置基于进入所述三维直播间的观众用户的第二请求信息确定。
根据本公开的一个或多个实施例,所述生成模块,还用于:获取进入所述三维直播间的目标观众用户的第一请求信息;基于所述第一请求信息,得到对应的目标虚拟对象的目标外观;基于所述目标外观,生成所述目标虚拟对象。
根据本公开的一个或多个实施例,所述生成模块,还用于:响应于所述第二终端设备发送的动作请求,获得目标观众用户对应的动作信息;基于所述动作信息,生成所述目标观众用户对应的目标虚拟对象的目标动作;控制所述目标虚拟对象在所述三维直播间内基于所述目标动作运动。
根据本公开的一个或多个实施例,所述生成模块,还用于:获取主机终端设备对应的主持用户的目标画面;基于所述主持用户的目标画面,在所述三维直播间内生成第二目标图像,所述第二目标图像表征所述主持用户对应的虚拟对象。
根据本公开的一个或多个实施例,在创建三维直播间之后,所述收发模块,还用于:向所述第一终端设备发送邀请信息,所述邀请信息用于邀请所述第一终端设备进入所述三维直播间;在接收到所述第一终端设备发送的响应信息后,接收所述第一终端设备发送的第一视频直播流。
第四方面,根据本公开的一个或多个实施例,提供了一种多人视频直播装置,包括:处理模块,用于进入三维直播间;收发模块,用于向创建所述三维直播间的主机终端设备发送第一视频直播流,以在所述三维直播间内创建第一目标图像,所述第一目标图像表征所述第一视频直播流中主播图像画面;播放模块,用于接收所述三维直播间对应的第二视频直播流,并对所述第二视频直播流进行播放。
根据本公开的一个或多个实施例,所述收发模块,还用于:发送控制信息,所述控制信息表征所述三维直播间内的观察点的观察位置和/或观察角度,
所述观察点用于获取所述三维直播间中内的图像;所述播放模块,具体用于:接收所述三维直播间对应的基于所述控制信息生成的第二视频直播流,并对基于所述控制信息生成的第二视频直播流进行播放。
第五方面,根据本公开的一个或多个实施例,提供了一种电子设备,包括:处理器,以及与所述处理器通信连接的存储器;所述存储器存储计算机执行指令;所述处理器执行所述存储器存储的计算机执行指令,以实现如上第一方面以及第一方面各种可能的设计所述的多人视频直播方法,或者,以实现如上第二方面以及第二方面各种可能的设计所述的多人视频直播方法。
第六方面,根据本公开的一个或多个实施例,提供了一种计算机可读存储介质,所述计算机可读存储介质中存储有计算机执行指令,当处理器执行所述计算机执行指令时,实现如上第一方面以及第一方面各种可能的设计所述的多人视频直播方法,或者,实现如上第二方面以及第二方面各种可能的设计所述的多人视频直播方法。
第七方面,本公开实施例提供一种计算机程序产品,包括计算机程序,该计算机程序被处理器执行时实现如上第一方面以及第一方面各种可能的设计所述的多人视频直播方法,或者,实现如上第二方面以及第二方面各种可能的设计所述的多人视频直播方法。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解,本公开中所涉及的公开范围,并不限于上述技术特征的特定组合而成的技术方案,同时也应涵盖在不脱离上述公开构思的情况下,由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外,虽然采用特定次序描绘了各操作,但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下,多任务和并行处理可能是有利的。同样地,虽然在上面论述中包含了若干具体实现细节,但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地,在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题,但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反,上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。
Claims (17)
- 一种多人视频直播方法,包括:创建三维直播间;基于第一终端设备发送的第一视频直播流,在所述三维直播间内创建第一目标图像,其中,所述第一终端设备对应主播用户,所述第一目标图像表征所述第一视频直播流对应的二维直播间的直播间画面;向目标终端设备发送所述三维直播间对应的第二视频直播流,其中,所述目标终端设备包括所述第一终端设备和/或第二终端设备,所述第二终端设备对应观众用户。
- 根据权利要求1所述的方法,其中,所述基于第一终端设备发送的第一视频直播流,在所述三维直播间内创建第一目标图像,包括:确定所述第一视频直播流中的直播图像帧的目标区域,所述目标区域为所述直播图像帧中主播图像所在的区域;基于所述目标区域分割所述直播图像帧,生成所述第一目标图像;将所述第一目标图像设置于所述三维直播间内。
- 根据权利要求1所述的方法,其中,所述方法还包括:接收所述目标终端设备发送的控制信息;根据所述控制信息,生成所述第二视频直播流。
- 根据权利要求3所述的方法,其中,所述根据所述控制信息,生成所述第二视频直播流,包括:根据所述目标终端设备发送的控制信息对所述三维直播间进行图像采集,生成所述第二视频直播流。
- 根据权利要求3所述的方法,其中,所述根据所述控制信息,生成所述第二视频直播流,包括:根据所述控制信息,获得所述第一目标图像的目标展示方向;根据所述目标展示方向,生成所述第二视频直播流。
- 根据权利要求1所述的方法,其中,所述三维直播间中包括至少一个第一空间位置和/或至少一个第二空间位置,其中,所述第一空间位置用于显示所述第一目标图像,所述第二空间位置用于显示虚拟对象,所述第一空间位置基于进入所述三维直播间的主播用户的第一请求信息确定,所述第二空 间位置基于进入所述三维直播间的观众用户的第二请求信息确定。
- 根据权利要求6所述的方法,其中,所述方法还包括:获取进入所述三维直播间的目标观众用户的第一请求信息;基于所述第一请求信息,得到对应的目标虚拟对象的目标外观;基于所述目标外观,生成所述目标虚拟对象。
- 根据权利要求6所述的方法,其中,所述方法还包括:响应于所述第二终端设备发送的动作请求,获得目标观众用户对应的动作信息;基于所述动作信息,生成所述目标观众用户对应的目标虚拟对象的目标动作;控制所述目标虚拟对象在所述三维直播间内基于所述目标动作运动。
- 根据权利要求1所述的方法,其中,所述方法还包括:获取主机终端设备对应的主持用户的目标画面;基于所述主持用户的目标画面,在所述三维直播间内生成第二目标图像,所述第二目标图像表征所述主持用户对应的虚拟对象。
- 根据权利要求1所述的方法,其中,在创建三维直播间之后,还包括:向所述第一终端设备发送邀请信息,所述邀请信息用于邀请所述第一终端设备进入所述三维直播间;在接收到所述第一终端设备发送的响应信息后,接收所述第一终端设备发送的第一视频直播流。
- 一种多人视频直播方法,包括:进入三维直播间;向创建所述三维直播间的主机终端设备发送第一视频直播流,以在所述三维直播间内创建第一目标图像,其中,所述第一目标图像表征所述第一视频直播流中主播图像画面;接收所述三维直播间对应的第二视频直播流,并对所述第二视频直播流进行播放。
- 根据权利要求11所述的方法,其中,所述方法还包括:发送控制信息;所述接收所述三维直播间对应的第二视频直播流,并对所述第二视频直播流进行播放,包括:接收所述三维直播间对应的基于所述控制信息生成的第二视频直播流,并对基于所述控制信息生成的第二视频直播流进行播放。
- 一种多人视频直播装置,包括:创建模块,用于创建三维直播间;生成模块,用于基于第一终端设备发送的第一视频直播流,在所述三维直播间内创建第一目标图像,其中,所述第一终端设备对应主播用户,所述第一目标图像表征所述第一视频直播流对应的二维直播间的直播间画面;收发模块,用于向目标终端设备发送所述三维直播间对应的第二视频直播流,其中,所述目标终端设备包括所述第一终端设备和/或第二终端设备,所述第二终端设备对应观众用户。
- 一种多人视频直播装置,包括:处理模块,用于进入三维直播间;收发模块,用于向创建所述三维直播间的主机终端设备发送第一视频直播流,以在所述三维直播间内创建第一目标图像,所述第一目标图像表征所述第一视频直播流中主播图像画面;播放模块,用于接收所述三维直播间对应的第二视频直播流,并对所述第二视频直播流进行播放。
- 一种电子设备,包括:处理器,以及与所述处理器通信连接的存储器;所述存储器存储计算机执行指令;所述处理器执行所述存储器存储的计算机执行指令,以实现如权利要求1至12中任一项所述的多人视频直播方法。
- 一种计算机可读存储介质,所述计算机可读存储介质中存储有计算机执行指令,当处理器执行所述计算机执行指令时,实现如权利要求1至12任一项所述的多人视频直播方法。
- 一种计算机程序产品,包括计算机程序,该计算机程序被处理器执行时实现权利要求1至12中任一项所述的多人视频直播方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310679137.3 | 2023-06-08 | ||
| CN202310679137.3A CN119110115A (zh) | 2023-06-08 | 2023-06-08 | 多人视频直播方法、装置、电子设备及存储介质 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2024251215A1 true WO2024251215A1 (zh) | 2024-12-12 |
Family
ID=93710319
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2024/097855 Ceased WO2024251215A1 (zh) | 2023-06-08 | 2024-06-06 | 多人视频直播方法、装置、电子设备及存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN119110115A (zh) |
| WO (1) | WO2024251215A1 (zh) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018095174A1 (zh) * | 2016-11-22 | 2018-05-31 | 广州华多网络科技有限公司 | 直播间视频流合成控制方法、装置及终端设备 |
| CN113965812A (zh) * | 2021-12-21 | 2022-01-21 | 广州虎牙信息科技有限公司 | 直播方法、系统及直播设备 |
| CN115345975A (zh) * | 2022-08-23 | 2022-11-15 | 广州博冠信息科技有限公司 | 直播间的展示方法及装置、存储介质、电子设备 |
| CN115442658A (zh) * | 2022-08-04 | 2022-12-06 | 珠海普罗米修斯视觉技术有限公司 | 直播方法、装置、存储介质、电子设备及产品 |
| US11601618B1 (en) * | 2021-09-08 | 2023-03-07 | Benjamin Slotznick | Method of using online, real-time, interactive, multiplane camera view to enhance videoconferencing platforms |
| CN115842936A (zh) * | 2022-12-02 | 2023-03-24 | 上海哔哩哔哩科技有限公司 | 多主播直播方法及装置 |
| CN116074545A (zh) * | 2023-01-17 | 2023-05-05 | 广州方硅信息技术有限公司 | 基于三维虚拟形象的直播比拼展示方法、装置、电子设备 |
-
2023
- 2023-06-08 CN CN202310679137.3A patent/CN119110115A/zh active Pending
-
2024
- 2024-06-06 WO PCT/CN2024/097855 patent/WO2024251215A1/zh not_active Ceased
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018095174A1 (zh) * | 2016-11-22 | 2018-05-31 | 广州华多网络科技有限公司 | 直播间视频流合成控制方法、装置及终端设备 |
| US11601618B1 (en) * | 2021-09-08 | 2023-03-07 | Benjamin Slotznick | Method of using online, real-time, interactive, multiplane camera view to enhance videoconferencing platforms |
| CN113965812A (zh) * | 2021-12-21 | 2022-01-21 | 广州虎牙信息科技有限公司 | 直播方法、系统及直播设备 |
| CN115442658A (zh) * | 2022-08-04 | 2022-12-06 | 珠海普罗米修斯视觉技术有限公司 | 直播方法、装置、存储介质、电子设备及产品 |
| CN115345975A (zh) * | 2022-08-23 | 2022-11-15 | 广州博冠信息科技有限公司 | 直播间的展示方法及装置、存储介质、电子设备 |
| CN115842936A (zh) * | 2022-12-02 | 2023-03-24 | 上海哔哩哔哩科技有限公司 | 多主播直播方法及装置 |
| CN116074545A (zh) * | 2023-01-17 | 2023-05-05 | 广州方硅信息技术有限公司 | 基于三维虚拟形象的直播比拼展示方法、装置、电子设备 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119110115A (zh) | 2024-12-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108616731B (zh) | 一种360度vr全景图形图像及视频实时生成方法 | |
| CN100442313C (zh) | 三维虚拟空间模拟器及其系统 | |
| US20240291938A1 (en) | Video fusion method, apparatus, electronic device and storage medium | |
| WO2022088918A1 (zh) | 虚拟图像的显示方法、装置、电子设备及存储介质 | |
| CN112135158B (zh) | 基于混合现实的直播方法及相关设备 | |
| JP7583165B2 (ja) | ビデオ処理方法、装置、電子機器及び記憶媒体 | |
| US20230182028A1 (en) | Game live broadcast interaction method and apparatus | |
| CN114745598A (zh) | 视频数据展示方法、装置、电子设备及存储介质 | |
| CN110710203B (zh) | 用于生成和渲染沉浸式视频内容的方法、系统和介质 | |
| WO2023143217A1 (zh) | 特效道具的展示方法、装置、设备及存储介质 | |
| CN115002442B (zh) | 一种图像展示方法、装置、电子设备及存储介质 | |
| EP4529177A1 (en) | Video live streaming method and apparatus, electronic device and storage medium | |
| CN116614543A (zh) | 虚拟互动方法、装置、设备和介质 | |
| CN115039410A (zh) | 信息处理系统、信息处理方法和程序 | |
| CN116801037A (zh) | 将真人主播影像投射于远程真实环境的增强现实直播方法 | |
| CN111097169A (zh) | 一种游戏图像的处理方法、装置、设备及存储介质 | |
| CN110225384A (zh) | 状态消息的推送方法、交互内容的切换方法、装置及设备 | |
| WO2024251215A1 (zh) | 多人视频直播方法、装置、电子设备及存储介质 | |
| CN109636917B (zh) | 三维模型的生成方法、装置、硬件装置 | |
| CN110070617A (zh) | 数据同步方法、装置、硬件装置 | |
| CN107635131A (zh) | 一种虚拟现实的实现方法及系统 | |
| WO2021088973A1 (zh) | 直播流显示方法、装置、电子设备及可读存储介质 | |
| KR20210056414A (ko) | 혼합 현실 환경들에서 오디오-가능 접속된 디바이스들을 제어하기 위한 시스템 | |
| CN118537455A (zh) | 动画编辑方法、播放方法、介质、电子设备及程序产品 | |
| CN118710779A (zh) | 动画播放方法、装置、介质、电子设备及程序产品 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 24818746 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |