WO2025256645A1 - 直播画面的处理方法、装置、电子设备、介质和程序产品 - Google Patents

直播画面的处理方法、装置、电子设备、介质和程序产品

Info

Publication number
WO2025256645A1
WO2025256645A1 PCT/CN2025/100985 CN2025100985W WO2025256645A1 WO 2025256645 A1 WO2025256645 A1 WO 2025256645A1 CN 2025100985 W CN2025100985 W CN 2025100985W WO 2025256645 A1 WO2025256645 A1 WO 2025256645A1
Authority
WO
WIPO (PCT)
Prior art keywords
effect
target
live stream
user
interactive
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2025/100985
Other languages
English (en)
French (fr)
Inventor
王浩安
周恩杰
郭璇
廖雪莹
寇枫易
王冰
张一晗
莫智强
冯苏柳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025256645A1 publication Critical patent/WO2025256645A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting

Definitions

  • This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device, medium, and program product for processing live broadcast images.
  • Screen-to-screen PK is a type of interactive live streaming where multiple users can connect via live stream to compare popularity (viewership), etc.
  • viewers can send gifts to live streamers (or hosts) they follow (or are in their own live stream room), and the corresponding effects of the gifts will be displayed in the live streamer's display area. For example, if a viewer sends a sports car as a gift to a live streamer they follow, the sports car animation will be displayed in that live streamer's display area.
  • a method for processing live stream footage comprising: in response to a viewing user's triggering operation on a target effect in an interactive live stream, acquiring resources for the target effect, wherein the interactive live stream is jointly participated in by multiple live stream users, the target effect acts on at least two target live stream users among the multiple live stream users, the target effect includes an effect corresponding to each of the at least two target live stream users, and the effects corresponding to the at least two target live stream users are associated; rendering the effect corresponding to each target live stream user according to the resources of the target effect and the layout information of the merged screen of the interactive live stream; and presenting the target effect in the merged screen of the interactive live stream.
  • a live streaming processing apparatus comprising: an acquisition module configured to acquire resources of the target effect in response to a viewing user's triggering operation on a target effect in an interactive live stream, wherein the interactive live stream is participated in by multiple live stream users, the target effect acts on at least two target live stream users among the multiple live stream users, the target effect includes an effect corresponding to each of the at least two target live stream users, and the effects corresponding to the at least two target live stream users are associated; a rendering module configured to render the effect corresponding to each target live stream user based on the resources of the target effect and layout information of the merged screen of the interactive live stream; and a display module configured to present the target effect in the merged screen of the interactive live stream.
  • an electronic device including: a memory; and a processor coupled to the memory, the processor being configured to execute a live screen processing method of any embodiment of the present disclosure based on instructions stored in the memory.
  • a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs a method for processing live video feeds according to any embodiment of the present disclosure.
  • a computer program product comprising: instructions, wherein, when executed by a processor, the instructions implement a method for processing live video feeds according to any embodiment of this disclosure.
  • Figure 1 shows a flowchart illustrating a method for processing live stream images according to some embodiments of this disclosure
  • Figure 2A illustrates a schematic diagram of the target effects of some embodiments of this disclosure
  • Figure 2B illustrates a schematic diagram of the target effects of some other embodiments of this disclosure.
  • Figure 2C illustrates a schematic diagram of the target effects of some embodiments of the present disclosure
  • Figure 2D illustrates a schematic diagram of the target effects of some embodiments of the present disclosure
  • Figure 3 shows a flowchart illustrating a method for processing live stream images according to other embodiments of this disclosure
  • Figure 4 shows a flowchart illustrating a method for processing live video footage according to some embodiments of this disclosure
  • Figure 5A is a schematic diagram illustrating a method for processing live stream images of live stream users and viewers according to some embodiments of this disclosure
  • Figure 5B illustrates a schematic diagram of the playback process of some embodiments of this disclosure
  • Figure 6 shows a flowchart illustrating a method for processing live video footage according to some embodiments of the present disclosure
  • Figure 7 shows a schematic diagram of the structure of a live broadcast processing apparatus according to some embodiments of the present disclosure
  • Figure 8 shows a schematic diagram of the structure of an electronic device according to some embodiments of the present disclosure.
  • Figure 9 shows a schematic diagram of the structure of a computer system according to some embodiments of the present disclosure.
  • the term “comprising” and its variations are open-ended terms that include at least the following elements/features but do not exclude other elements/features, i.e., “including but not limited to”. Furthermore, as used in this disclosure, the term “including” and its variations are open-ended terms that include at least the following elements/features but do not exclude other elements/features, i.e., “including but not limited to”. Therefore, “comprising” and “including” are synonymous.
  • the term “based on” means “at least partially based on”.
  • the terms “one embodiment,” “some embodiments,” or “embodiment” mean that a specific feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the invention.
  • the term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; and the term “some embodiments” means “at least some embodiments.”
  • the appearance of the phrases “in one embodiment,” “in some embodiments,” or “in an embodiment” in various places throughout the specification does not necessarily refer to the same embodiment, but may refer to the same embodiment.
  • first,” “second,” etc. used in this disclosure are used only to distinguish different devices, modules, or units, and are not intended to define the order of functions performed by these devices, modules, or units or their interdependencies. Unless otherwise specified, the concepts of "first,” “second,” etc., are not intended to imply that the objects described herein must be in a given temporal, spatial, rank, or any other given order.
  • This disclosure proposes a method for processing live streaming images, which can enrich the effects in interactive live streaming, improve the interactivity of the effects, thereby enhancing the fun and display effects of interactive live streaming, enriching the interaction forms between viewers and live streaming users, and improving the user experience.
  • Figure 1 is a flowchart of some embodiments of the live broadcast processing method of this disclosure.
  • the method of this embodiment includes steps S102 to S106.
  • the method of this embodiment can be executed by a client.
  • the client can be the client of multiple live broadcast users participating in interactive live broadcast or the client of a viewing user.
  • the client can be application software or hardware; for example, the client can be a live broadcast application, a hardware device in a user device, etc.
  • step S102 in response to a display request for a target effect in the interactive live stream, the resources for the target effect are obtained.
  • Interactive live streaming involves multiple live streaming users, and the target effect (also known as the linkage effect) applies to at least two of the target live streaming users.
  • the playback interface can simultaneously display the live stream feeds of all four users, and the target effect can be applied to two, three, or all four target live streaming users.
  • the target live stream users affected by the target effect can be selected by the viewers, or the target live stream users can be automatically selected by the server or client based on the type of target effect and the number of target live stream users affected by the target effect, and is not limited to the examples given.
  • the target effects include the effects corresponding to each of at least two target live stream users.
  • the effects corresponding to at least two target live stream users are linked, interactive, and interconnected. For example, when the animation of the effect corresponding to one or more target live stream users is displayed, the animation of the effects corresponding to other target live stream users will follow.
  • the target effects include a guiding effect, a transition effect, and a feedback effect.
  • the playback order of the guiding effect, transition effect, and feedback effect is from first to last.
  • the transition effect connects the guiding effect and the feedback effect, and the feedback effect provides feedback on the guiding effect.
  • one target live stream user may have a guiding effect, while the other target live stream users may have transition effects and feedback effects.
  • the target live stream users corresponding to the guiding effect, transition effects, and feedback effects can be selected by the viewing user or automatically selected by the server or client. There can also be multiple target live stream users corresponding to the guiding effect.
  • Viewers can trigger the target effect by sending gifts to live stream users (hosts or guests, etc.), or by using the target effect trigger control, but are not limited to the examples given.
  • the display request in response to a triggering operation of a viewing user's request to display a target effect in an interactive live stream, the display request is received, wherein the display request includes an identifier of a resource for the target effect; and the resource for the target effect is obtained based on the identifier of the resource for the target effect.
  • Viewers can trigger a display request for a target effect. For example, if a viewer triggers a display request for a target effect by giving a gift, the display request can include the gift's identifier as the identifier of the target effect's resource, allowing the user to retrieve the corresponding target effect's resource.
  • the resources for a target effect include the assets of that target effect, and the target effect may include assets of multiple effects with interrelationships.
  • the resources for a target effect may also include the control logic, dynamic scripts, and other information used to render the target effect, and are not limited to the examples given.
  • the resources for a target effect can be stored as a resource package on the client or server. If stored on the server, the client can download the resource package of the target effect from the server based on the resource's identifier.
  • step S104 the effect corresponding to each target live stream user is rendered based on the layout information of the resources of the target effect and the merged screen of the interactive live stream.
  • the merged screen of an interactive live stream can be divided into multiple areas based on the number of live stream users. For example, when two live stream users participate in an interactive live stream, the merged screen is divided into left and right areas to display the screens of the two live stream users respectively.
  • the layout information of the merged screen for interactive live streams includes information about the display areas for multiple live stream users. Each display area can be represented using area identifiers, coordinate information, etc., without limitation.
  • the layout information of the merged screen for interactive live streams can use the information in SEI (Supplemental Enhancement Information) representing the display areas for multiple live stream users.
  • SEI Supplemental Enhancement Information
  • the display area for each target live stream user can be determined by the layout information of the merged screen of the interactive live stream. Based on the display area of each target live stream user, the corresponding effect for each target live stream user can be rendered in the merged screen of the interactive live stream.
  • the guiding effect, transition effect, and feedback effect can correspond to the same or different formats.
  • the guiding effect may use MP4 format
  • the transition effect may use a sequence frame format
  • the feedback effect may use JSON format (e.g., Lottie).
  • JSON format e.g., Lottie
  • the guiding effect is rendered in the display area of the target live user corresponding to the guiding effect
  • the connecting effect is rendered in the connecting area from the display area of the target live user corresponding to the guiding effect to the display area of the target live user corresponding to the feedback effect
  • the feedback effect is rendered in the display area of the target live user corresponding to the feedback effect.
  • step 106 the target effect is presented in the merged screen of the interactive live stream.
  • the target effect is rendered into the combined screen of the interactive live stream, and the effects corresponding to multiple target live stream users can be displayed in a linked manner on the clients of the live stream users and the viewers.
  • a merge screen of interactive live streams is displayed.
  • the guidance effect is presented in the display area and interaction area of the target live stream user corresponding to the guidance effect.
  • a transition effect is presented in the transition area from the display area of the target live stream user corresponding to the guidance effect to the display area of the target live stream user corresponding to the feedback effect.
  • a feedback effect is presented in the display area of the target live stream user corresponding to the feedback effect. The transition effect can span the display areas of two target live stream users.
  • the display area of the left live streamer shows a car throwing snowballs moving from left to right (or from a distance) and throwing snowballs, serving as a guiding effect.
  • the transition area between the display areas of the left and right live streamers shows the trajectory of the snowballs, serving as a transition effect.
  • the display area of the right live streamer shows a snowball hitting the right streamer's head, breaking the snowball, and a snow scene with falling snowflakes in the background, serving as a feedback effect.
  • the method described in the above embodiments in response to a display request for a target effect acting on at least two target live streaming users in an interactive live stream, acquires resources for the target effect, and then renders the effect corresponding to each target live streaming user based on the resources of the target effect and the layout information of the merged screen of the interactive live stream.
  • This allows the target effect to be presented when the interactive live stream screen is displayed, resulting in the related display of effects for at least two target live streaming users.
  • the method described in the above embodiments can enrich the effects in interactive live streams, improve the interactivity of the effects, thereby enhancing the fun and display effects of interactive live streams, enriching the forms of interaction between viewers and live streaming users, and between multiple live streaming users, and improving the user experience.
  • rendering the effect corresponding to each target live streaming user in conjunction with the layout information of the merged screen of the interactive live stream can improve the accuracy of the target effect rendering.
  • the layout information of the merged screen of an interactive live stream includes information about the display area of each of the multiple live stream users and information about the interaction area.
  • the interaction area is used to display interaction information between the viewing user and the live stream user. For example, the interaction area may display messages sent by the viewing user to the live stream user.
  • the guiding effect is rendered in the display area of the target live user corresponding to the guiding effect
  • the feedback effect is rendered in the display area of the target live user corresponding to the feedback effect
  • the guiding effect is rendered in the interaction area
  • the transition effect is rendered in the transition area from the interaction area to the display area of the target live user corresponding to the feedback effect.
  • the merging screen of interactive live streams is displayed.
  • the guiding effect is presented in the display area and the interaction area of the target live stream user corresponding to the guiding effect.
  • the connecting area from the interaction area to the display area of the target live stream user corresponding to the feedback effect is presented.
  • the feedback effect is presented in the display area of the target live stream user corresponding to the feedback effect.
  • the resources for the target effect include resources for the guiding effect, the transition effect, and the feedback effect, which are used to render the guiding effect, the transition effect, and the feedback effect, respectively.
  • the guiding effect can be rendered and displayed in the corresponding target live stream user's display area and interactive area, but the rendered materials and display format can be the same or different.
  • the guiding effect can include a background and a foreground. In the target live stream user's display area, the background may not be rendered and displayed, only the foreground. In the interactive area, both the background and foreground may be rendered and displayed simultaneously.
  • the transition effect can be rendered and displayed in the interactive area
  • the transition effect can be rendered and displayed in the transition area from the interactive area to the target live stream user's display area corresponding to the feedback effect.
  • the guiding effect displayed in the target live stream user's display area and the interactive area corresponding to the guiding effect is synchronized. In consecutive frames corresponding to the guiding effect, the guiding effect is rendered synchronously in the target live stream user's display area and the interactive area corresponding to the guiding effect.
  • the lower area is the interactive area, where a guiding effect can be displayed, such as a car throwing snowballs entering and throwing snowballs.
  • a guiding effect can be displayed, such as a car throwing snowballs entering and throwing snowballs.
  • the transition area from the interactive area to the live stream user's display area on the right displays the trajectory of the movement and the snowballs as a transition effect. This transition effect can span across the display areas of both the interactive area and the target live stream user corresponding to the feedback effect.
  • the following describes some examples of how to render the effect corresponding to each target live stream user based on the layout information of the resources for the target effect and the combined screen of the interactive live stream.
  • the playback order information of the effects corresponding to each target live user is determined; based on the resources of the target effects, the layout information of the merged screen of the interactive live stream, and the playback order information of the effects corresponding to each target live user, the effects corresponding to each target live user are rendered.
  • the playback order can be determined based on the type of effect corresponding to each target live stream user. For example, if the effect corresponding to a target live stream user is a guide effect, it will be played first. If the effect corresponding to a target live stream user includes transition effects and feedback effects, the transition effects will be played after the guide effects, followed by the feedback effects.
  • the resources for the target effects can include various effect types and their corresponding playback order information.
  • the frame corresponding to the effect in the merging of interactive live streams is determined based on the playback order information of the effects; and the effect's material is added to the frame corresponding to the effect in the merging of interactive live streams based on the layout information of the merging screen of interactive live streams, so as to realize the rendering of the effect.
  • the resources for the target effect can include the duration of various effects.
  • the corresponding frames for each effect in the merged interactive live stream can be determined, and the effect's assets can then be added to the corresponding frames in the merged interactive live stream.
  • each frame in the merged interactive live stream can pass through the effect rendering chain, and the effect's assets can be added to the corresponding frames according to the effect's processing logic (e.g., duration, playback order, etc.).
  • the method information for adding effects corresponding to each target live stream user is determined; based on the resources of the target effect, the layout information of the merged screen of the interactive live stream, and the method information for adding effects corresponding to each target live stream user, the effect corresponding to each target live stream user is rendered.
  • Different target effects can correspond to different addition methods, also known as gameplay information. For example, some effects need to be added to specific parts of the target live stream user, some effects need to be added to the background screen, and some effects have specific motion trajectories, etc., not limited to the examples given.
  • the effect addition method information includes the target position of the effect in the display area of the target live stream user corresponding to it, and/or, the trajectory information of the effect.
  • the resources for the target effect include the material of the effect corresponding to each target live user.
  • the target image corresponding to the target position is extracted from the merged frame of the interactive live stream.
  • the material of the effect is then combined with the target image to generate a rendered frame.
  • Compositing can involve directly adding source material to a target image, or it can involve using source material to deform, enhance, or otherwise process the target image, and is not limited to the examples given.
  • the target area can also be the entire display area, allowing the effect's source material to be composited with the entire display area.
  • the movement direction of the effect's material is adjusted according to the trajectory information corresponding to that effect; based on the layout information of the merged interactive live stream frame, the adjusted effect's material is added to the merged frame of the interactive live stream.
  • the trajectory information may be information describing the direction from one display area to another, or trajectory direction information, etc., and is not limited to the examples given.
  • the trajectory information for that effect can be determined based on the effect type and the display area of the target live stream user.
  • the effect is a guiding effect, i.e., a car throwing snowballs.
  • the display area of the target live stream user is the left-hand display area. Therefore, the trajectory information for this effect is from left to right.
  • the movement direction of the effect material can be adjusted to be from the left to the right of the target live stream user's display area.
  • the effect is a transition effect, i.e., the trajectory of the movement and the snowballs. Therefore, the trajectory and direction of the snowballs need to be adjusted based on the trajectory information of this effect.
  • the above embodiments can be combined arbitrarily to achieve the rendering of the target effect. For example, based on the resources of the target effect, the layout information of the merged screen of the interactive live stream, the playback order information of the effects corresponding to each target live stream user, and the addition method information of the effects corresponding to each target live stream user, the effect corresponding to each target live stream user is rendered.
  • the foregoing embodiments please refer to the foregoing embodiments, which will not be repeated here.
  • the above embodiments can render the target effect.
  • the rendering of the target effect can be achieved more accurately, so that the guiding effect, the connecting effect and the feedback effect can be better connected and present better linkage.
  • the identifiers of at least two target live stream users are obtained. Based on the identifier of each target live stream user and the layout information of the merged screen of the interactive live stream, the display area of each target live stream user in the merged screen of the interactive live stream is determined. Based on the resources of the target effect and the display area of each target live stream user in the merged screen of the interactive live stream, the effect corresponding to each target live stream user is rendered.
  • the layout information of the merged screen in interactive live streaming can include the correspondence between the identifier of each live streamer and the display area.
  • the display area can be determined based on the identifier of each live streamer.
  • an effect selection area is displayed, wherein the effect selection area includes one or more effects, and the one or more effects include a target effect; in response to a viewing user's selection of a target effect, a selection area of the live stream user to which the target effect applies is displayed; in response to a viewing user's selection of at least two target live stream users, a display request for the target effect is triggered, and the identifiers of at least two target live stream users are obtained.
  • the effect selection area for example, is a gift selection area. Viewers can trigger the display of this area using an effect trigger control. Viewers can select the target effect through clicking, voice commands, or other methods. Viewers can choose the livestreamer for which the target effect applies, or a default livestreamer can be provided for direct confirmation. For example, the streamer in the viewer's current livestream or a livestreamer the viewer follows can be used as the default livestreamer. Alternatively, if there are only two livestreamers, the viewer doesn't need to make a selection; both can be directly set as the default livestreamers.
  • FIG 3 is a flowchart of some other embodiments of the live broadcast processing method of this disclosure. As shown in Figure 3, the method of this embodiment includes: steps S302 to S312, which are executed by the live broadcast user's client.
  • step S302 the first live stream of the current live user is rendered with local effects and sent to the server.
  • Local effects refer to the effects selected by the current live stream user, such as beauty filters and props.
  • the current live stream user's client will push the first live stream it captures to the server.
  • step S304 a second live stream from a user other than the current live stream user is obtained.
  • the client of the current live stream user can pull a second live stream from the server (or cloud). If there are multiple other live stream users, multiple second live streams can be pulled.
  • step S306 the first live stream and the second live stream are merged to obtain the merged interactive live stream.
  • step S308 in response to a user's request to display a target effect in the interactive live stream, the resource for the target effect is obtained.
  • step S310 the effect corresponding to each target live stream user is rendered based on the layout information of the resources of the target effect and the combined screen of the interactive live stream.
  • step S312 the target effect is presented in the merged screen of the interactive live broadcast.
  • the first and second live streams are continuously generated, and the merging and rendering processes are also continuously executed. For each frame, the merging process can be performed first, and then the rendering can be performed.
  • the method described in the above embodiments enables the rendering and display of the target effect on the live streaming user's client, enhancing the interactivity between the live streaming user and the viewing user, as well as among the live streaming users themselves, thereby improving the user experience. Furthermore, rendering the target effect within the merged frame of an interactive live stream can improve the continuity of the target effect and enhance the display quality.
  • FIG 4 is a flowchart of some further embodiments of the live broadcast processing method of this disclosure. As shown in Figure 4, the method of this embodiment includes: steps S402 to S408, executed by the client of the viewing user.
  • step S402 the merged interactive live stream is obtained from the server.
  • Viewing users' clients can pull the merged stream of interactive live streams from the server (or cloud).
  • This merged stream can be obtained by merging the video streams of multiple live stream users on the server.
  • step S404 in response to the user's triggering operation on the target effect in the interactive live stream, the resource of the target effect is obtained.
  • step S406 the effect corresponding to each target live stream user is rendered based on the layout information of the resources for the target effect and the merged screen of the interactive live stream.
  • step S408 the target effect is presented in the merged screen of the interactive live broadcast.
  • the method described in the above embodiments enables the rendering and display of the target effect on the viewing user's client, enhancing the interactivity between live stream users and viewing users, as well as among live stream users themselves, thereby improving the user experience. Furthermore, rendering the target effect within the merged frame of an interactive live stream can improve the continuity of the target effect and enhance the display quality.
  • broadcaster A as a live user in the interactive live stream, has their live stream captured by the camera.
  • the client's Live Core module renders Effect 1 within broadcaster A's live stream.
  • the rendered live stream of broadcaster A is then pushed to the server or cloud.
  • Broadcaster A's client further pulls broadcaster B's live stream and merges them.
  • Effect 2 is rendered in the merged stream and displayed on broadcaster A's screen.
  • the client can pull the merged interactive live stream from the server or cloud, render Effect 2 in the player, and then display the rendered interactive live stream on the viewer's screen.
  • Effect 2 can be the target effect or a portion of the target effect (e.g., a feedback effect).
  • the guiding effect can be rendered in the live stream of the corresponding live user's client, while the transition effect and feedback effect can be rendered in the merge screen of the interactive live stream, rendering the target effect in two stages.
  • the playback flow on the viewer's end is not described in detail.
  • the broadcaster's end can obtain the layout information of the merging screen of the interactive live stream and then transmit it to the viewer's end through merging.
  • the MP4 format effect can be played directly by adding a layer to the merging screen, while the Lottie format feedback effect needs to be merged with the merging screen.
  • the target position can be identified and the target image can be extracted to achieve the merging of the feedback effect and the merging screen.
  • FIG. 6 is a flowchart of some further embodiments of the live broadcast image processing method of this disclosure. As shown in Figure 6, the method of this embodiment includes steps S601 to S619.
  • step S601 the push-pull flow module responds to the triggering of preset conditions and creates an effect instance in the effect module.
  • step S602 the push-pull stream module transmits the layout information of the merged screen of the interactive live broadcast.
  • step S603 the effect module stores the layout information.
  • step S604 the business module responds to the change in layout information by transmitting the layout information to the effect module through the push-pull flow module.
  • step S605 the effect module stores the layout information.
  • step S606 the business module responds to the received trigger message (display request) for the target effect and starts rendering the effect.
  • step S607 the push-pull flow module switches to the effect rendering link.
  • the effect rendering chain includes a first effect rendering chain and a second effect rendering chain.
  • the guiding effect and the transition effect use the first effect rendering chain, and the feedback effect uses the second effect rendering chain.
  • Different rendering methods can be used.
  • the effect rendering chain includes a first effect rendering chain, a second effect rendering chain, and a third effect rendering chain.
  • the guiding effect uses the first effect rendering chain
  • the transition effect uses the second effect rendering chain
  • the feedback effect uses the third effect rendering chain. Different rendering methods can be used.
  • step S608 the business module sets the resources for the target effect.
  • the business module can also set resources for the algorithms involved in the target effect. Different resources can be set for different effect rendering chains. For example, for the first effect rendering chain, resources for guiding and transition effects can be set, and for the second effect rendering chain, resources for feedback effects can be set.
  • step S609 the push-pull flow module passes the resources of the target effect to the effect module.
  • step S610 the effects module loads the effects resources.
  • Different effects can be loaded for different effect rendering chains. For example, for the first effect rendering chain, resources for guiding effects and transition effects are loaded, while for the second effect rendering chain, resources for feedback effects are loaded.
  • step S611 the business module sends the gameplay information (addition method information) to the effect module via a message.
  • step S612 the effect module processes the effects according to the processing logic in the gameplay information.
  • the rendering chain for different effects can be processed according to the processing logic of different effects.
  • step S613 the effects module renders each frame in the merged interactive live stream.
  • the effects module can render effects in each frame based on the effects' resources, gameplay information, and stored layout information. Different effects can be rendered for different effect rendering chains. For example, for the first effect rendering chain, guide effects and transition effects are rendered, while for the second effect rendering chain, feedback effects are rendered.
  • step S614 the effect module sends an end message to the business module in response to the end of the target effect playback.
  • step S615 the service module receives an end message.
  • step S616 the business module removes the resources for the target effect and sends a resource removal command to the effect module.
  • step S617 the effects module removes the resources for the target effect.
  • step S618 the business module disables effect rendering.
  • step S619 the push-pull flow module skips effect rendering.
  • the method described in the above embodiments through the cooperation of various modules and the setting of the effect chain, transmits layout information, target effect resources, and gameplay information, thereby realizing the rendering of the target effect in the client and improving the accuracy of the target effect rendering.
  • This disclosure also provides a processing device for live broadcast images, which will be described below with reference to Figure 7.
  • FIG. 7 is a structural diagram of some embodiments of the live broadcast processing apparatus of this disclosure.
  • the live broadcast processing apparatus 70 of this embodiment includes: an acquisition module 710, a rendering module 720, and a display module 730.
  • the acquisition module 710 is configured to acquire the resources of the target effect in response to the triggering operation of the viewing user on the target effect in the interactive live broadcast.
  • the interactive live broadcast is jointly participated in by multiple live broadcast users.
  • the target effect applies to at least two target live broadcast users among the multiple live broadcast users.
  • the target effect includes the effect corresponding to each of the at least two target live broadcast users.
  • the effects corresponding to the at least two target live broadcast users are related.
  • the acquisition module 710 is configured to receive a display request in response to a triggering operation by a viewing user requesting the display of a target effect in an interactive live stream, wherein the display request includes an identifier of a resource for the target effect; and acquire the resource for the target effect based on the identifier of the resource for the target effect.
  • the rendering module 720 is configured to render the effect corresponding to each target live stream user based on the layout information of the resources of the target effect and the merged screen of the interactive live stream.
  • the rendering module 720 is configured to determine the playback order information of the effects corresponding to each target live user; and to render the effects corresponding to each target live user based on the resources of the target effects, the layout information of the merged screen of the interactive live stream, and the playback order information of the effects corresponding to each target live user.
  • the resources for the target effect include the material of the effect corresponding to each target live stream user.
  • the rendering module 720 is configured to, for each target live stream user, determine the frame corresponding to the effect in the merging of the interactive live stream based on the playback order information of the effect; and add the material of the effect to the frame corresponding to the effect in the merging of the interactive live stream based on the layout information of the merging screen of the interactive live stream, so as to realize the rendering of the effect.
  • the rendering module 720 is configured to determine the effect addition method information corresponding to each target live user; and render the effect corresponding to each target live user based on the resources of the target effect, the layout information of the merged screen of the interactive live stream, and the effect addition method information corresponding to each target live user.
  • the resources for the target effect include the material of the effect corresponding to each target live stream user
  • the layout information of the merging screen of the interactive live stream includes the display area information of each target live stream user
  • the effect addition method information includes the target position of the effect in the display area of the target live stream user corresponding to it.
  • the rendering module 720 is configured to extract the target image corresponding to the target position from the merging frame of the interactive live stream according to the display area of the target live stream user corresponding to the effect in the merging screen of the interactive live stream and the target position of the effect in the display area of the target live stream user corresponding to it; and to synthesize the material of the effect with the target image to generate the rendered frame.
  • the resources for the target effect include the material of the effect corresponding to each target live stream user.
  • the effect addition method information includes the trajectory information of the effect.
  • the rendering module 720 is configured to adjust the movement direction of the material of the effect according to the trajectory information corresponding to the effect for each target live stream user; and add the adjusted material of the effect to the frame of the merging of the interactive live stream according to the layout information of the merging screen of the interactive live stream.
  • the effects corresponding to at least two target live streaming users are linked.
  • the effect corresponding to one of the at least two target live streaming users is a guiding effect
  • the effects corresponding to the other target live streaming users include transition effects and feedback effects.
  • the playback order of the guiding effect, transition effect and feedback effect is from first to last.
  • the layout information of the merged screen of the interactive live stream includes information about the display area of each of the multiple live stream users and information about the interaction area.
  • the interaction area is used to display the interaction information between the viewing user and the live stream user.
  • the rendering module 720 is configured to render the guiding effect in the display area of the target live stream user corresponding to the guiding effect, render the feedback effect in the display area of the target live stream user corresponding to the feedback effect, render the guiding effect in the interaction area, and render the transition effect in the transition area from the interaction area to the display area of the target live stream user corresponding to the feedback effect.
  • the acquisition module 710 is further configured to acquire the identifiers of at least two target live streaming users in response to a display request for a target effect in the interactive live streaming
  • the rendering module 720 is configured to determine the display area of each target live streaming user in the combined screen of the interactive live streaming based on the identifier of each target live streaming user and the layout information of the combined screen of the interactive live streaming; and to render the effect corresponding to each target live streaming user based on the resources of the target effect and the display area of each target live streaming user in the combined screen of the interactive live streaming.
  • Display module 730 is configured to present the target effect in the merged screen of interactive live streaming.
  • the display module 730 is configured to display the merged screen of the interactive live stream, presenting the guidance effect in the display area and the interaction area of the target live stream user corresponding to the guidance effect, presenting the connection effect in the connection area from the interaction area to the display area of the target live stream user corresponding to the feedback effect, and presenting the feedback effect in the display area of the target live stream user corresponding to the feedback effect.
  • the display module 730 is further configured to display an effect selection area, wherein the effect selection area includes one or more effects, and the one or more effects include a target effect; in response to the viewing user's selection of the target effect, the selection area of the live streaming user to which the target effect applies is displayed; the acquisition module 710 is configured to, in response to the viewing user's selection of at least two target live streaming users, trigger a display request for the target effect and acquire the identifiers of at least two target live streaming users.
  • the live streaming processing device 70 when the live streaming processing device 70 is configured on the client side of any live streaming user among multiple live streaming users, the live streaming processing device 70 further includes: a merging module, configured to render the first live streaming stream of any live streaming user with local effects and send it to the server; obtain the second live streaming stream of other live streaming users besides the live streaming user; and merge the first live streaming stream and the second live streaming stream to obtain the merging of interactive live streaming.
  • a merging module configured to render the first live streaming stream of any live streaming user with local effects and send it to the server; obtain the second live streaming stream of other live streaming users besides the live streaming user; and merge the first live streaming stream and the second live streaming stream to obtain the merging of interactive live streaming.
  • modules are logical modules divided according to their specific functions, and are not intended to limit the specific implementation method. For example, they can be implemented in software, hardware, or a combination of both. In actual implementation, the above-described units can be implemented as independent physical entities, or they can be implemented by a single entity (e.g., a processor (CPU or DSP, etc.), integrated circuit, etc.). Furthermore, the units shown in the accompanying drawings with dashed lines indicate that these units may not actually exist, and the operations/functions they perform can be implemented by the processing circuitry itself.
  • the device may also include a memory that can store various information generated by the device and its constituent units during operation, programs and data used for operation, data to be transmitted by the communication unit, etc.
  • the memory can be volatile memory and/or non-volatile memory.
  • the memory may include, but is not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), and flash memory.
  • RAM random access memory
  • DRAM dynamic random access memory
  • SRAM static random access memory
  • ROM read-only memory
  • flash memory flash memory
  • the device may also include a communication unit that can be used to communicate with other devices.
  • the communication unit can be implemented in a manner known in the art, such as including communication components such as antenna arrays and/or radio frequency links, various types of interfaces, communication units, etc. These will not be described in detail here.
  • the device may also include other components not shown, such as radio frequency links, baseband processing units, network interfaces, processors, controllers, etc. These will not be described in detail here.
  • FIG. 8 shows a block diagram of some embodiments of the electronic device of this disclosure.
  • the electronic device 80 can be various types of devices, such as mobile terminals including, but not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device 80 may include a display panel for displaying data and/or execution results utilized in the scheme according to this disclosure.
  • the display panel can be of various shapes, such as a rectangular panel, an elliptical panel, or a polygonal panel.
  • the display panel can be not only a planar panel, but also a curved panel, or even a spherical panel.
  • the electronic device 80 of this embodiment includes a memory 81 and a processor 82 coupled to the memory 81. It should be noted that the components of the electronic device 80 shown in FIG8 are merely exemplary and not limiting; the electronic device 80 may also have other components depending on the actual application requirements.
  • the processor 82 can control other components in the electronic device 80 to perform desired functions.
  • memory 81 is used to store one or more computer-readable instructions.
  • processor 82 executes the computer-readable instructions, the computer-readable instructions are executed by processor 82 to implement the method according to any of the above embodiments.
  • processor 82 executes the computer-readable instructions, the computer-readable instructions are executed by processor 82 to implement the method according to any of the above embodiments.
  • processor 82 and memory 81 can communicate with each other directly or indirectly.
  • processor 82 and memory 81 can communicate via a network.
  • the network can include wireless networks, wired networks, and/or any combination of wireless and wired networks.
  • Processor 82 and memory 81 can also communicate with each other via a system bus, which is not limited in this disclosure.
  • processor 82 can be embodied in various suitable processors, processing devices, such as central processing unit (CPU), graphics processing unit (GPU), network processor (NP), etc.; it can also be digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • the central processing unit (CPU) can be an x86 or ARM architecture, etc.
  • memory 81 can include any combination of various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • Memory 81 can include, for example, system memory, which stores, for example, the operating system, application programs, boot loader, database, and other programs. Various application programs and various data can also be stored in the storage medium.
  • FIG. 9 is a block diagram illustrating an example structure of a computer system that may be employed in embodiments of this disclosure.
  • the Central Processing Unit (CPU) 901 performs various processes according to a program stored in the Read-Only Memory (ROM) 902 or a program loaded from the storage section 908 into the Random Access Memory (RAM) 903.
  • the RAM 903 also stores data required as needed when the CPU 901 performs various processes.
  • the CPU is merely exemplary and can be other types of processors, such as the various processors described above.
  • the ROM 902, RAM 903, and storage section 908 can be various forms of computer-readable storage media, as described below. It should be noted that although the ROM 902, RAM 903, and storage device 908 are shown separately in Figure 9, one or more of them can be combined or located in the same or different memories or storage modules.
  • CPU 901, ROM 902 and RAM 903 are interconnected via bus 904.
  • Input/output interface 905 is also connected to bus 904.
  • input section 906 such as a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.
  • output section 907 including displays such as cathode ray tube (CRT), liquid crystal display (LCD), speakers, vibrators, etc.
  • storage section 908 including hard disk, magnetic tape, etc.
  • communication section 909 including network interface cards such as LAN cards, modems, etc.
  • the communication section 909 allows communication processing to be performed via a network such as the Internet. It is readily understood that although the various devices or modules in the computer system 90 shown in Figure 9 communicate via bus 904, they can also communicate via a network or other means, wherein the network can include wireless networks, wired networks, and/or any combination of wireless and wired networks.
  • drive 910 is also connected to input/output interface 905.
  • Removable media 911 such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 910 as needed, so that computer programs read from them can be installed into storage section 908 as needed.
  • the program constituting the software can be installed from a network such as the Internet or from a storage medium such as a removable medium 911.
  • embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts.
  • the computer program can be downloaded and installed from a network via communication section 909, or installed from storage section 908, or installed from ROM 902.
  • the computer program When the computer program is executed by CPU 901, it performs the functions defined in the methods of embodiments of this disclosure.
  • a computer-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof.
  • a computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.
  • a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.
  • a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code.
  • Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof.
  • the computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.
  • the program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
  • the aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.
  • a computer program comprising: instructions that, when executed by a processor, cause the processor to perform the processing method for the live broadcast image of any of the above embodiments.
  • the instructions may be embodied in computer program code.
  • computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof.
  • programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages.
  • the program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).
  • LAN local area network
  • WAN wide area network
  • each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function.
  • the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.
  • each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
  • modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware.
  • the names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.
  • exemplary hardware logic components include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.
  • FPGAs field-programmable gate arrays
  • ASICs application-specific integrated circuits
  • ASSPs application-specific standard products
  • SoCs system-on-a-chip
  • CPLDs complex programmable logic devices
  • a live streaming processing apparatus comprising: an acquisition module configured to acquire resources of a target effect in response to a viewing user's triggering operation on a target effect in an interactive live stream, wherein the interactive live stream is participated in by multiple live stream users, the target effect acts on at least two target live stream users, and the target effect includes an effect corresponding to each of the at least two target live stream users; a rendering module configured to render the effect corresponding to each target live stream user based on the resources of the target effect and the layout information of the combined screen of the interactive live stream; and a display module configured to present the target effect in the combined screen of the interactive live stream.
  • an electronic device comprising: a memory; and a processor coupled to the memory, the processor being configured to perform a live video processing method as described in any embodiment of the present disclosure based on instructions stored in the memory.
  • a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a method for processing live video feeds according to any embodiment of the present disclosure.
  • a computer program comprising: instructions that, when executed by a processor, cause the processor to perform a method for processing live video feeds according to any embodiment of the present disclosure.
  • a computer program product including instructions that, when executed by a processor, implement a method for processing live video feeds according to any embodiment of this disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Information Transfer Between Computers (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本公开涉及一种直播画面的处理方法、装置、电子设备、介质和程序产品，涉及计算机技术领域。本公开的方法包括：响应于针对互动直播中目标效果的显示请求，获取目标效果的资源，其中，互动直播由多个直播用户共同参与，目标效果作用于多个直播用户中至少两个目标直播用户，目标效果包括至少两个目标直播用户中每个目标直播用户对应的效果，至少两个目标直播用户对应的效果是关联的；根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果；在互动直播的合流画面中呈现目标效果。

Description

直播画面的处理方法、装置、电子设备、介质和程序产品
相关申请的交叉引用
本申请是以中国申请号为202410764675.7，申请日为2024年6月13日的申请为基础，并主张其优先权，该中国申请的公开内容在此作为整体引入本申请中。
技术领域
本公开涉及计算机技术领域，特别涉及一种直播画面的处理方法、装置、电子设备、介质和程序产品。
背景技术
多个直播用户(主播、嘉宾等)参与互动直播时，多个直播用户的多路直播流可以被合流进行播放，在播放端可以同时显示多个直播用户的直播画面，并且多个直播用户可以进行实时互动。连屏PK是互动直播的一种，多个直播用户可以通过直播连线，比较受欢迎程度(人气)等。
目前，在互动直播中，观看用户可以向自己关注(或所在直播间)的直播用户(或主播)赠送礼物，在该直播用户的显示区域中则可以显示礼物对应的效果。例如，观看用户赠送自己关注的直播用户一辆礼物跑车，在该直播用户的显示区域中则可以显示跑车的动画。
发明内容
提供该发明内容部分以便以简要的形式介绍构思，这些构思将在后面的具体实施方式部分被详细描述。该发明内容部分并不旨在标识要求保护的技术方案的关键特征或必要特征，也不旨在用于限制所要求的保护的技术方案的范围。
根据本公开的一些实施例，提供了一种直播画面的处理方法，包括：响应于观看用户对互动直播中目标效果的触发操作，获取所述目标效果的资源，其中，所述互动直播由多个直播用户共同参与，所述目标效果作用于所述多个直播用户中至少两个目标直播用户，所述目标效果包括所述至少两个目标直播用户中每个目标直播用户对应的效果，所述至少两个目标直播用户对应的效果是关联的；根据所述目标效果的资源和所述互动直播的合流画面的布局信息，渲染所述每个目标直播用户对应的效果；在所述互动直播的合流画面中呈现所述目标效果。
根据本公开的另一些实施例，提供一种直播画面的处理装置，包括：获取模块，被配置为响应于观看用户对互动直播中目标效果的触发操作，获取所述目标效果的资源，其中，所述互动直播由多个直播用户共同参与，所述目标效果作用于所述多个直播用户中至少两个目标直播用户，所述目标效果包括所述至少两个目标直播用户中每个目标直播用户对应的效果，所述至少两个目标直播用户对应的效果是关联的；渲染模块，被配置为根据所述目标效果的资源和所述互动直播的合流画面的布局信息，渲染所述每个目标直播用户对应的效果；显示模块，被配置为在所述互动直播的合流画面中呈现所述目标效果。
根据本公开的又一些实施例，提供一种电子设备，包括：存储器；和耦接至存储器的处理器，处理器被配置为基于存储在存储器中的指令，执行本公开中的任一实施例的直播画面的处理方法。
根据本公开的再一些实施例，提供一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时执行本公开中的任一实施例的直播画面的处理方法。
根据本公开的又一些实施例，提供一种计算机程序产品，包括：指令，其中，该指令被处理器执行时实现本公开中的任一实施例的直播画面的处理方法
通过以下参照附图对本公开的示例性实施例的详细描述，本公开的其它特征、方面及其优点将会变得清楚。
附图说明
下面参照附图说明本公开的优选实施例。此处所说明的附图用来提供对本公开的进一步理解，各附图连同下面的具体描述一起包含在本说明书中并形成说明书的一部分，用于解释本公开。应当理解的是，下面描述中的附图仅仅涉及本公开的一些实施例，而非对本公开构成限制。在附图中：
图1示出本公开一些实施例的直播画面的处理方法的流程示意图；
图2A示出本公开一些实施例的目标效果的示意图；
图2B示出本公开另一些实施例的目标效果的示意图；
图2C示出本公开又一些实施例的目标效果的示意图；
图2D示出本公开再一些实施例的目标效果的示意图；
图3示出本公开另一些实施例的直播画面的处理方法的流程示意图；
图4示出本公开又一些实施例的直播画面的处理方法的流程示意图；
图5A示出本公开一些实施例的直播用户和观看用户的直播画面的处理方法的示意图；
图5B示出本公开一些实施例的播放流程的示意图；
图6示出本公开再一些实施例的直播画面的处理方法的流程示意图；
图7示出本公开一些实施例的直播画面的处理装置的结构示意图；
图8示出本公开一些实施例的电子设备的结构示意图；
图9示出本公开一些实施例的计算机系统的结构示意图。
应当明白，为了便于描述，附图中所示出的各个部分的尺寸并不一定是按照实际的比例关系绘制的。在各附图中使用了相同或相似的附图标记来表示相同或者相似的部件。因此，一旦某一项在一个附图中被定义，则在随后的附图中可能不再对其进行进一步讨论。
具体实施方式
下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，但是显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。以下对实施例的描述实际上也仅仅是说明性的，决不作为对本公开及其应用或使用的任何限制。应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例。
应当理解，本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值应被解释为仅仅是示例性的，不限制本公开的范围。
本公开中使用的术语“包括”及其变型意指至少包括后面的元件/特征、但不排除其他元件/特征的开放性术语，即“包括但不限于”。此外，本公开使用的术语“包含”及其变型意指至少包含后面的元件/特征、但不排除其他元件/特征的开放性术语，即“包含但不限于”。因此，包括与包含是同义的。术语“基于”意指“至少部分地基于”。
整个说明书中所称“一个实施例”、“一些实施例”或“实施例”意味着与实施例结合描述的特定的特征、结构或特性被包括在本发明的至少一个实施例中。例如，术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。而且，短语“在一个实施例中”、“在一些实施例中”或“在实施例中”在整个说明书中各个地方的出现不一定全都指的是同一个实施例，但是也可以指同一个实施例。
需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。除非另有指定，否则“第一”、“第二”等概念并非意图暗示如此描述的对象必须按时间上、空间上、排名上的给定顺序或任何其他方式的给定顺序。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
下面结合附图对本公开的实施例进行详细说明，但是本公开并不限于这些具体的实施例。下面这些具体实施例可以相互结合，对于相同或者相似的概念或过程可能在某些实施例不再赘述。此外，在一个或多个实施例中，特定的特征、结构或特性可以由本领域的普通技术人员从本公开将清楚的任何合适的方式组合。
目前，在互动直播中只支持在单个直播用户的显示区域中显示独立的效果，效果不能作用于其他直播用户，形式比较单一，互动性不强。
本公开提出一种直播画面的处理方法，可以丰富互动直播中的效果，提高效果的互动性，从而增强互动直播的趣味性和显示效果，丰富了观看用户、直播用户之间的互动形式，提升用户体验。
图1为本公开直播画面的处理方法的一些实施例的流程图。如图1所示，该实施例的方法包括：步骤S102～S106。该实施例的方法可以由客户端执行。客户端为参与互动直播的多个直播用户的客户端或观看用户的客户端。客户端可以是应用软件或硬件，例如，客户端可以是直播应用、用户设备中的硬件装置等。
在步骤S102中，响应于针对互动直播中目标效果的显示请求，获取目标效果的资源。
互动直播由多个直播用户共同参与，目标效果(也可以称为联动效果)作用于多个直播用户中至少两个目标直播用户。
例如，4个直播用户参与互动直播，则播放界面中可以同时显示4个直播用户的直播画面，目标效果可以作用于4个直播用户中的2个、3个或4个目标直播用户。
目标效果作用的目标直播用户可以由观看用户选择，也可以根据目标效果的类型和目标效果所作用的目标直播用户的数量，由服务端或客户端自动选择目标直播用户，不限于所举示例。
目标效果包括至少两个目标直播用户中每个目标直播用户对应的效果。至少两个目标直播用户对应的效果是联动的、交互的、相互关联的。例如，当某一个或多个目标直播用户对应的效果的动画显示后，其他目标直播用户对应的效果的动画跟着显示。
在一些实施例中，目标效果包括引导效果，衔接效果、反馈效果。引导效果、衔接效果和反馈效果的播放顺序由先到后。衔接效果用于衔接引导效果和反馈效果，反馈效果是对引导效果的反馈。例如，至少两个目标直播用户中可以有一个目标直播用户对应的效果为引导效果，其他目标直播用户对应的效果包括衔接效果和反馈效果。引导效果对应的目标直播用户，衔接效果和反馈效果对应的目标直播用户，可以由观看用户选择，也可以由服务端或客户端自动选择。引导效果对应的目标直播用户也可以为多个。
观看用户(即观众)可以通过向直播用户(主播或嘉宾等)赠送礼物触发礼物对应的目标效果，也可以通过目标效果的触发控件，触发目标效果，但不限于所举示例。
在一些实施例中，响应于观看用户针对互动直播中目标效果的显示请求的触发操作，接收该显示请求，其中，该显示请求中包括目标效果的资源的标识；根据目标效果的资源的标识，获取目标效果的资源。
观看用户可以触发针对目标效果的显示请求。例如，如果观看用户是通过赠送礼物触发礼物对应的目标效果的显示请求，则显示请求中可以包括礼物的标识作为目标效果的资源的标识，可以获取到对应的目标效果的资源。
例如，目标效果的资源包括目标效果的素材，目标效果可以包括多个具有联动关系的效果的素材。目标效果的资源还可以包括目标效果的控制逻辑、动态脚本等用于渲染目标效果的信息，不限于所举示例。目标效果的资源可以以资源包的形式存储在客户端或服务端，如果存储在服务端，客户端可以根据目标效果的资源的标识从服务端下载目标效果的资源包。
在步骤S104中，根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果。
互动直播的合流画面可以根据直播用户的数量划分为多个区域。例如，当两个直播用户参与互动直播的情况下，合流画面被划分为左右两个区域分别显示两个直播用户的画面。例如，互动直播的合流画面的布局信息包括多个直播用户的显示区域的信息。每个显示区域可以采用区域标识、坐标信息等表示，在此不作限制。例如，互动直播的合流画面的布局信息可以采用SEI(Supplemental Enhancement Information，媒体补充增强信息)中表示多个直播用户的显示区域的信息。
通过互动直播的合流画面的布局信息可以确定每个目标直播用户的显示区域，根据每个目标直播用户的显示区域可以在互动直播的合流画面中渲染每个目标直播用户对应的效果。
在一些实施例中，引导效果，衔接效果、反馈效果可以对应相同或不同的格式。例如，引导效果采用MP4格式，衔接效果采用序列帧格式，反馈效果采用JSON格式(例如，Lottie)。不同的格式可以对应不同的渲染方式。
在一些实施例中，在引导效果对应的目标直播用户的显示区域，渲染引导效果，在由引导效果对应的目标直播用户的显示区域到反馈效果对应的目标直播用户的显示区域的衔接区域中，渲染衔接效果，在反馈效果对应的目标直播用户的显示区域，渲染反馈效果。
不同的效果在不同的区域中显示，可以呈现出引导效果与反馈效果之间的联动的效果、互动的效果。
在步骤106中，在互动直播的合流画面中呈现目标效果。
将目标效果渲染至互动直播的合流画面中，在直播用户和观看用户的客户端，可以将多个目标直播用户对应的效果联动显示。
在一些实施例中，显示互动直播的合流画面，在引导效果对应的目标直播用户的显示区域和互动区域中，呈现引导效果，在由引导效果对应的目标直播用户的显示区域到反馈效果对应的目标直播用户的显示区域的衔接区域中，呈现衔接效果，在反馈效果对应的目标直播用户的显示区域中，呈现反馈效果。衔接效果可以跨越两个目标直播用户的显示区域进行显示。
如图2A所示，两个直播用户在进行互动直播，目标效果触发后，左侧的直播用户的显示区域中显示投掷雪球的小车从左侧驶向右侧(或者从远处驶来的效果)，并投掷雪球，作为引导效果。如图2B所示，由左侧的直播用户的显示区域到右侧的直播用户的显示区域的衔接区域中显示运动的轨迹和雪球，作为衔接效果。如图2C所示，在右侧的直播用户的显示区域中显示雪球砸在右侧主播的头上，雪球破裂，并且背景中飘落雪花的雪景画面，作为反馈效果。
上述实施例的方法，响应于针对互动直播中作用于至少两个目标直播用户的目标效果的显示请求，获取目标效果的资源，进而根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果，从而在显示互动直播的画面时，呈现出目标效果，使得至少两个目标直播用户对应的效果关联显示。上述实施例的方法可以丰富互动直播中的效果，提高效果的互动性，从而增强互动直播的趣味性和显示效果，丰富了观看用户和直播用户之间、以及多个直播用户之间的互动形式，提升用户体验。此外，结合互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果，可以提高目标效果渲染的准确性。
在一些实施例中，互动直播的合流画面的布局信息包括多个直播用户中每个直播用户的显示区域的信息和互动区域的信息，互动区域用于显示观看用户与直播用户的互动信息。例如，互动区域中可以显示观看用户向直播用户发送的消息。
在一些实施例中，根据目标效果的资源，在引导效果对应的目标直播用户的显示区域中，渲染引导效果，在反馈效果对应的目标直播用户的显示区域中，渲染反馈效果，在互动区域中，渲染引导效果，在由互动区域到反馈效果对应的目标直播用户的显示区域的衔接区域中，渲染衔接效果。
在一些实施例中，显示互动直播的合流画面，在引导效果对应的目标直播用户的显示区域和互动区域中，呈现引导效果，在由互动区域到反馈效果对应的目标直播用户的显示区域的衔接区域中，呈现衔接效果，在反馈效果对应的目标直播用户的显示区域中，呈现反馈效果。
目标效果的资源包括引导效果的资源、衔接效果的资源和反馈效果的资源分别用于渲染引导效果、衔接效果和反馈效果。引导效果可以在对应的目标直播用户的显示区域和互动区域中渲染和显示，但是渲染的素材和显示的形式可以相同或不同。例如，引导效果可以包括背景部分和前景部分，引导效果在其对应的目标直播用户的显示区域中可以不渲染和显示背景部分，仅渲染和显示前景部分，在互动区域中同时渲染和显示背景部分和前景部分。由于互动区域中可以渲染和显示引导效果，因此可以在由互动区域到反馈效果对应的目标直播用户的显示区域的衔接区域中，渲染和显示衔接效果。引导效果对应的目标直播用户的显示区域和互动区域中显示的引导效果是同步的。在引导效果对应的连续的帧中，在引导效果对应的目标直播用户的显示区域和互动区域中同步渲染引导效果。
如图2A～2C所示，下方区域为互动区域，互动区域中可以显示引导效果即投掷雪球的小车驶入并投掷雪球。另外，如图2D所示，由互动区域到右侧的直播用户的显示区域的衔接区域中显示运动的轨迹和雪球，作为衔接效果。衔接效果可以跨越互动区域和反馈效果对应的目标直播用户的显示区域进行显示。
下面描述如何根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果的一些实施例。
在一些实施例中，确定每个目标直播用户对应的效果的播放顺序信息；根据目标效果的资源、互动直播的合流画面的布局信息和每个目标直播用户对应的效果的播放顺序信息，渲染每个目标直播用户对应的效果。
可以根据每个目标直播用户对应的效果的类型确定播放顺序，例如，对于每个目标直播用户，如果该目标直播用户对应的效果为引导效果，则最先播放，如果该目标直播用户对应的效果包括衔接效果和反馈效果，则在引导效果播放完后播放衔接效果再播放反馈效果。目标效果的资源中可以包括各种效果的类型以及对应的播放顺序信息。
在一些实施例中，针对每个目标直播用户对应的效果，根据效果的播放顺序信息，确定效果在互动直播的合流中对应的帧；根据互动直播的合流画面的布局信息，将效果的素材添加到效果在互动直播的合流中对应的帧上，以实现对效果的渲染。
目标效果的资源中可以包括各种效果的时长，根据播放顺序和时长可以确定各种效果在互动直播的合流中对应的帧，从而将效果的素材添加到效果在互动直播的合流中对应的帧上。在互动直播过程中，互动直播的合流中每一帧可以经过效果渲染链路，根据效果的处理逻辑(例如，时长、播放顺序等)将效果的素材添加到对应的帧上。
在一些实施例中，确定每个目标直播用户对应的效果的添加方式信息；根据目标效果的资源、互动直播的合流画面的布局信息和每个目标直播用户对应的效果的添加方式信息，渲染每个目标直播用户对应的效果。
不同的目标效果(或效果)可以对应不同的添加方式信息，也可以称为玩法信息，例如，有一些效果需要在目标直播用户的特定部位进行添加，有一些效果需要在背景画面中添加，有一些效果有特定的运动轨迹等，不限于所举示例。
在一些实施例中，针对每个目标直播用户对应的效果，该效果的添加方式信息包括该效果在与其对应的目标直播用户的显示区域中的目标位置，和/或，该效果的轨迹信息。
在一些实施例中，目标效果的资源包括每个目标直播用户对应的效果的素材，针对每个目标直播用户对应的效果，根据该效果对应的目标直播用户在互动直播的合流画面中的显示区域和效果在与其对应的目标直播用户的显示区域中的目标位置，从互动直播的合流的帧中提取目标位置对应的目标图像；将该效果的素材与目标图像合成，生成渲染后的帧。
合成可以是直接将素材添加到目标图像上，合成也可以是利用素材对目标图像进行变形、美化等处理，不限于所举示例。目标位置也可以是整个显示区域，可以将效果的素材与整个显示区域合成。
在一些实施例中，针对每个目标直播用户对应的效果，根据该效果对应的轨迹信息调整该效果的素材的运动方向；根据互动直播的合流画面的布局信息，将调整后的该效果的素材添加到互动直播的合流的帧中。轨迹信息可以是描述由一个显示区域指向另一个显示区域的信息，或者轨迹的方向信息等，不限于所举示例。
针对每个目标直播用户对应的效果，该效果对应的轨迹信息可以根据该效果的类型、该效果对应的目标直播用户的显示区域确定。例如，如图2A所示，该效果为引导效果即投掷雪球的小车，该效果对应的目标直播用户的显示区域为左侧的目标直播用户的显示区域，则该效果对应的轨迹信息为从左到右，进一步可以调整效果素材的运动方向为从目标直播用户的显示区域的左侧到右侧。如图2B或2D所示，该效果为衔接效果即运动的轨迹和雪球，则需要根据该效果的轨迹信息，调整轨迹和雪球的方向。
上述各个实施例可以任意进行结合，来实现对目标效果的渲染。例如，根据目标效果的资源、互动直播的合流画面的布局信息、每个目标直播用户对应的效果的播放顺序信息和每个目标直播用户对应的效果的添加方式信息，渲染每个目标直播用户对应的效果。具体可以参考前述各个实施例，在此不再赘述。
上述实施例可以实现对目标效果的渲染，并且结合互动直播的合流画面的布局信息、每个目标直播用户对应的效果的播放顺序信息、每个目标直播用户对应的效果的添加方式信息等信息，可以更加准确的实现对目标效果的渲染，使引导效果、衔接效果和反馈效果之间更好的衔接，呈现出更好的联动性。
在一些实施例中，响应于观看用户对互动直播中目标效果的触发操作，获取至少两个目标直播用户的标识，根据每个目标直播用户的标识和互动直播的合流画面的布局信息，确定每个目标直播用户在互动直播的合流画面中的显示区域；根据目标效果的资源和每个目标直播用户在互动直播的合流画面中的显示区域，渲染每个目标直播用户对应的效果。
互动直播的合流画面的布局信息可以包括每个直播用户的标识与显示区域的对应关系，基于每个直播用户的标识可以确定其显示区域。
在一些实施例中，显示效果选择区域，其中，效果选择区域包括一个或多个效果，一个或多个效果包括目标效果；响应于观看用户对目标效果的选择，显示目标效果作用的直播用户的选择区域；响应于观看用户对至少两个目标直播用户的选择，触发针对目标效果的显示请求，获取至少两个目标直播用户的标识。
效果选择区域例如为礼物选择区域，观看用户可以通过效果触发控件，触发效果选择区域的显示。观看用户可以通过点击、语音等方式对目标效果进行选择。观看用户可以选择目标效果作用的直播用户，也可以提供目标效果作用的默认的直播用户，观看用户可以直接确认。例如，观看用户所在直播间的主播或者观看用户关注的直播用户可以作为默认的直播用户。例如，只有两个直播用户的情况下，可以不需要观看用户进行选择，直接将两个直播用户作为默认的直播用户。
下面结合图3、4分别描述本公开的直播画面的处理方法在直播用户的客户端和观看用户的客户端执行的情况下的一些实施例。
图3为本公开直播画面的处理方法的另一些实施例的流程图。如图3所示，该实施例的方法包括：步骤S302～S312，由直播用户的客户端执行。
在步骤S302中，将当前直播用户的第一直播流采用本地效果进行渲染，并发送至服务端。
本地效果即当前直播用户自己选取的效果，例如，美颜、道具等效果。当前直播用户的客户端将采集的第一直播流推流到服务端。
在步骤S304中，获取当前直播用户以外的其他直播用户的第二直播流。
当前直播用户的客户端可以从服务端(或云端)拉取第二直播流，如果其他直播用户为多个，则可以拉取多个第二直播流。
在步骤S306中，将第一直播流和第二直播流进行合流，得到互动直播的合流。
在步骤S308中，响应于观看用户针对互动直播中目标效果的显示请求的触发操作，获取目标效果的资源。
在步骤S310中，根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果。
在步骤S312中，在互动直播的合流画面中呈现目标效果。
第一直播流和第二直播流是不断生成的，合流过程和渲染过程也是不断执行的，针对每一帧画面，可以是先进行合流的处理，再进行渲染的。
上述实施例的方法，实现了在直播用户的客户端进行目标效果的渲染和显示，增强了直播用户和观看用户之间的互动性，以及直播用户之间的互动性，提升用户体验。此外，在互动直播的合流画面中渲染目标效果可以提高目标效果的连贯性，提升显示效果。
图4为本公开直播画面的处理方法的又一些实施例的流程图。如图4所示，该实施例的方法包括：步骤S402～S408，由观看用户的客户端执行。
在步骤S402中，从服务端获取互动直播的合流。
观看用户的客户端可以从服务端(或云端)拉取互动直播的合流，该合流可以是在服务端根据多个直播用户的视频流进行合流得到的。
在步骤S404中，响应于观看用户对互动直播中目标效果的触发操作，获取目标效果的资源。
在步骤S406中，根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果。
在步骤S408中，在互动直播的合流画面中呈现目标效果。
上述实施例的方法，实现了在观看用户的客户端进行目标效果的渲染和显示，增强了直播用户和观看用户之间的互动性，以及直播用户之间的互动性，提升用户体验。此外，在互动直播的合流画面中渲染目标效果可以提高目标效果的连贯性，提升显示效果。
如图5A所示，主播A作为互动直播中的一个直播用户，相机(camera)采集主播A的直播流，客户端的Live Core(实时核心)模块可以将效果1(effect 1)在主播A的直播流中进行渲染。将渲染后的主播A的直播流推流到服务端或云端。主播A的客户端会进一步拉取主播B的直播流，进行合流。在合流画面中渲染效果2(effect2)，在主播A的屏幕上显示渲染后的互动直播画面。对于观众(观看用户)，客户端可以从服务端或云端拉取互动直播的合流，在播放器中渲染效果2(effect 2)。进而在观众的屏幕上显示渲染后的互动直播画面。效果2可以是目标效果或者目标效果中的部分效果(例如，反馈效果)。
例如，引导效果可以在其对应的直播用户的客户端的直播流中进行渲染，衔接效果和反馈效果可以在互动直播的合流画面中进行渲染，分两个阶段进行目标效果的渲染。
在一些实施例中，针对每个目标直播用户对应的效果，在该效果为第一格式的情况下，在该效果对应的帧上新增图层渲染该效果，在该效果为第二格式的情况下，在该效果对应的帧中渲染该效果。第一格式例如为MP4、序列帧等，第二格式例如为JSON格式(例如Lottie)等，根据效果的格式可以确定渲染的方式。
以引导效果、衔接效果为MP4，反馈效果为Lottie为例，如图5B所示为目标效果的播放流程，主播端(直播用户端)和观众端(观看用户端)分别进行播放处理，处理过程类似。例如，主播端接收到目标效果的触发消息(显示请求)，例如为礼物消息，会将对应的目标效果的资源放入效果队列，按照队列中各个效果的顺序进行播放，首先会播放MP4格式的效果，先播放引导效果，再播放衔接效果，进而加载反馈效果添加至互动直播的合流中进行播放，播放结束后再移除效果。观众端的播放流程不再赘述。主播端可以获取互动直播的合流画面的布局信息之后，通过合流传递到观众端。MP4格式的效果可以合流画面上增加图层直接播放，Lottie格式的反馈效果需要与合流画面进行融合，例如，识别目标位置，提取目标图像等处理来实现反馈效果与合流画面的融合。
下面结合图6描述客户端内部的目标效果的处理逻辑的一些实施例。
图6为本公开直播画面的处理方法的再一些实施例的流程图。如图6所示，该实施例的方法包括：步骤S601～S619。
在步骤S601中，推拉流模块响应于预设条件的触发，向效果(effect)模块创建效果实例。
对于观看用户推拉流模块可以用于从服务端(或云端)拉取互动直播的合流，对于直播用户推拉流模块主要用于将本地的直播流推流到服务端，以及从服务端(或云端)拉取其他直播用户的直播流。对于观看用户触发条件例如为在一次互动直播中首次播放目标效果，对于直播用户触发条件例如为开启互动直播。
在步骤S602中，推拉流模块透传互动直播的合流画面的布局信息。
在步骤S603中，效果模块将布局信息进行存储。
效果模块存储的布局信息可以用于后续目标效果的渲染。
在步骤S604中，业务模块响应于布局信息的变化，通过推拉流模块向效果模块透传布局信息。
在步骤S605中，效果模块将布局信息进行存储。
在步骤S606中，业务模块响应于接收到目标效果的触发消息(显示请求)，开启效果的渲染。
在步骤S607中，推拉流模块切换到效果渲染链路。
在一些实施例中，效果渲染链路包括第一效果渲染链路和第二效果渲染链路，引导效果和衔接效果采用第一效果渲染链路，反馈效果采用第二效果渲染链路，可以采用不同的渲染方式。或者，效果渲染链路包括第一效果渲染链路、第二效果渲染链路和第三效果渲染链路，引导效果采用第一效果渲染链路，衔接效果采用第二效果渲染链路，反馈效果采用第三效果渲染链路，可以采用不同的渲染方式。
在步骤S608中，业务模块设置目标效果的资源。
业务模块同时可以设置目标效果涉及的算法的资源。针对不同的效果渲染链路可以设置不同的资源，例如，针对第一效果渲染链路，设置引导效果和衔接效果的资源，针对第二效果渲染链路，设置反馈效果的资源。
在步骤S609中，推拉流模块将目标效果的资源透传至效果模块。
在步骤S610中，效果模块加载效果的资源。
针对不同的效果渲染链路可以加载不同的效果，例如，针对第一效果渲染链路，加载引导效果和衔接效果的资源，针对第二效果渲染链路，加载反馈效果的资源。
在步骤S611中，业务模块通过消息将玩法信息(添加方式信息)发送至效果模块。
在步骤S612中，效果模块根据玩法信息中效果的处理逻辑进行处理。
针对不同的效果渲染链路可以根据不同效果的处理逻辑进行处理。
在步骤S613中，效果模块对互动直播的合流中每帧进行渲染。
效果模块可以根据效果的资源，玩法信息以及存储的布局信息，对每帧中的效果进行渲染。针对不同的效果渲染链路可以渲染不同的效果，例如，针对第一效果渲染链路，渲染引导效果和衔接效果，针对第二效果渲染链路，渲染反馈效果。
在步骤S614中，效果模块响应于目标效果播放结束，向业务模块发送结束消息。
在步骤S615中，业务模块接收结束消息。
在步骤S616中，业务模块移除目标效果的资源，并向效果模块发起资源移除指令。
在步骤S617中，效果模块移除目标效果的资源。
在步骤S618中，业务模块关闭效果渲染。
在步骤S619中，推拉流模块跳过效果渲染。
上述实施例的方法，通过各个模块的配合，通过效果链路的设置，布局信息、目标效果的资源以及玩法信息的传递，在客户端中实现了目标效果的渲染，并且提高了目标效果渲染的准确性。
本公开还提供一种直播画面的处理装置，下面结合图7进行描述。
图7为本公开直播画面的处理装置的一些实施例的结构图。如图7所示，该实施例的直播画面的处理装置70包括：获取模块710，渲染模块720，显示模块730。
获取模块710被配置为响应于观看用户对互动直播中目标效果的触发操作，获取目标效果的资源，其中，互动直播由多个直播用户共同参与，目标效果作用于多个直播用户中至少两个目标直播用户，目标效果包括至少两个目标直播用户中每个目标直播用户对应的效果，至少两个目标直播用户对应的效果是关联的。
在一些实施例中，获取模块710被配置为响应于观看用户针对互动直播中目标效果的显示请求的触发操作，接收显示请求，其中，显示请求中包括目标效果的资源的标识；根据目标效果的资源的标识，获取目标效果的资源。
渲染模块720被配置为根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果。
在一些实施例中，渲染模块720被配置为确定每个目标直播用户对应的效果的播放顺序信息；根据目标效果的资源、互动直播的合流画面的布局信息和每个目标直播用户对应的效果的播放顺序信息，渲染每个目标直播用户对应的效果。
在一些实施例中，目标效果的资源包括每个目标直播用户对应的效果的素材，渲染模块720被配置为针对每个目标直播用户对应的效果，根据效果的播放顺序信息，确定效果在互动直播的合流中对应的帧；根据互动直播的合流画面的布局信息，将效果的素材添加到效果在互动直播的合流中对应的帧上，以实现对效果的渲染。
在一些实施例中，渲染模块720被配置为确定每个目标直播用户对应的效果的添加方式信息；根据目标效果的资源、互动直播的合流画面的布局信息和每个目标直播用户对应的效果的添加方式信息，渲染每个目标直播用户对应的效果。
在一些实施例中，目标效果的资源包括每个目标直播用户对应的效果的素材，互动直播的合流画面的布局信息包括每个目标直播用户的显示区域的信息，针对每个目标直播用户对应的效果，效果的添加方式信息包括效果在与其对应的目标直播用户的显示区域中的目标位置，渲染模块720被配置为针对每个目标直播用户对应的效果，根据效果对应的目标直播用户在互动直播的合流画面中的显示区域和效果在与其对应的目标直播用户的显示区域中的目标位置，从互动直播的合流的帧中提取目标位置对应的目标图像；将效果的素材与目标图像合成，生成渲染后的帧。
在一些实施例中，目标效果的资源包括每个目标直播用户对应的效果的素材，针对每个目标直播用户对应的效果，效果的添加方式信息包括效果的轨迹信息，渲染模块720被配置为针对每个目标直播用户对应的效果，根据效果对应的轨迹信息调整效果的素材的运动方向；根据互动直播的合流画面的布局信息，将调整后的效果的素材添加到互动直播的合流的帧中。
在一些实施例中，至少两个目标直播用户对应的效果是联动的，至少两个目标直播用户中一个目标直播用户对应的效果为引导效果，其他目标直播用户对应的效果包括衔接效果和反馈效果，引导效果、衔接效果和反馈效果的播放顺序由先到后。
在一些实施例中，互动直播的合流画面的布局信息包括多个直播用户中每个直播用户的显示区域的信息和互动区域的信息，互动区域用于显示观看用户与直播用户的互动信息，渲染模块720被配置为根据目标效果的资源，在引导效果对应的目标直播用户的显示区域中，渲染引导效果，在反馈效果对应的目标直播用户的显示区域中，渲染反馈效果，在互动区域中，渲染引导效果，在由互动区域到反馈效果对应的目标直播用户的显示区域的衔接区域中，渲染衔接效果。
在一些实施例中，获取模块710还被配置为响应于针对互动直播中目标效果的显示请求，获取至少两个目标直播用户的标识，渲染模块720被配置为根据每个目标直播用户的标识和互动直播的合流画面的布局信息，确定每个目标直播用户在互动直播的合流画面中的显示区域；根据目标效果的资源和每个目标直播用户在互动直播的合流画面中的显示区域，渲染每个目标直播用户对应的效果。
显示模块730被配置为在互动直播的合流画面中呈现目标效果。
在一些实施例中，显示模块730被配置为显示互动直播的合流画面，在引导效果对应的目标直播用户的显示区域和互动区域中，呈现引导效果，在由互动区域到反馈效果对应的目标直播用户的显示区域的衔接区域中，呈现衔接效果，在反馈效果对应的目标直播用户的显示区域中，呈现反馈效果。
在一些实施例中，显示模块730还被配置为显示效果选择区域，其中，效果选择区域包括一个或多个效果，一个或多个效果包括目标效果；响应于观看用户对目标效果的选择，显示目标效果作用的直播用户的选择区域；获取模块710被配置为响应于观看用户对至少两个目标直播用户的选择，触发针对目标效果的显示请求，获取至少两个目标直播用户的标识。
在一些实施例中，在直播画面的处理装置70被配置在多个直播用户中任意直播用户的客户端侧的情况下，直播画面的处理装置70还包括：合流模块，被配置为针对任意直播用户，将该直播用户的第一直播流采用本地效果进行渲染，并发送至服务端；获取该直播用户以外的其他直播用户的第二直播流；将第一直播流和第二直播流进行合流，得到互动直播的合流。
应注意，上述各个单元(模块)仅是根据其所实现的具体功能划分的逻辑模块，而不是用于限制具体的实现方式，例如可以以软件、硬件或者软硬件结合的方式来实现。在实际实现时，上述各个单元可被实现为独立的物理实体，或者也可由单个实体(例如，处理器(CPU或DSP等)、集成电路等)来实现。此外，上述各个单元在附图中用虚线示出指示这些单元可以并不实际存在，而它们所实现的操作/功能可由处理电路本身来实现。
此外，尽管未示出，该设备也可以包括存储器，其可以存储由设备、设备所包含的各个单元在操作中产生的各种信息、用于操作的程序和数据、将由通信单元发送的数据等。存储器可以是易失性存储器和/或非易失性存储器。例如，存储器可以包括但不限于随机存储存储器(RAM)、动态随机存储存储器(DRAM)、静态随机存取存储器(SRAM)、只读存储器(ROM)、闪存存储器。当然，存储器可也位于该设备之外。可选地，尽管未示出，但是该设备也可以包括通信单元，其可用于与其它装置进行通信。在一个示例中，通信单元可以被按照本领域已知的适当方式来实现，例如包括天线阵列和/或射频链路等通信部件，各种类型的接口、通信单元等等。这里将不再详细描述。此外，设备还可以包括未示出的其它部件，诸如射频链路、基带处理单元、网络接口、处理器、控制器等。这里将不再详细描述。
本公开的一些实施例还提供一种电子设备。图8示出本公开的电子设备的一些实施例的框图。例如，在一些实施例中，电子设备80可以为各种类型的设备，例如可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。例如，电子设备80可以包括显示面板，以用于显示根据本公开的方案中所利用的数据和/或执行结果。例如，显示面板可以为各种形状，例如矩形面板、椭圆形面板或多边形面板等。另外，显示面板不仅可以为平面面板，也可以为曲面面板，甚至球面面板。
如图8所示，该实施例的电子设备80包括：存储器81以及耦接至该存储器81的处理器82。应当注意，图8所示的电子设备80的组件只是示例性的，而非限制性的，根据实际应用需要，该电子设备80还可以具有其他组件。处理器82可以控制电子设备80中的其它组件以执行期望的功能。
在一些实施例中，存储器81用于存储一个或多个计算机可读指令。处理器82用于运行计算机可读指令时，计算机可读指令被处理器82运行时实现根据上述任一实施例的方法。关于该方法的各个步骤的具体实现以及相关解释内容可以参见上述的实施例，重复之处在此不作赘述。
例如，处理器82和存储器81之间可以直接或间接地互相通信。例如，处理器82和存储器81可以通过网络进行通信。网络可以包括无线网络、有线网络、和/或无线网络和有线网络的任意组合。处理器82和存储器81之间也可以通过系统总线实现相互通信，本公开对此不作限制。
例如，处理器82可以体现为各种适当的处理器、处理装置等，诸如中央处理器(CPU)、图形处理器(Graphics Processing Unit，GPU)、网络处理器(NP)等；还可以是数字信号处理器(DSP)、专用集成电路(ASIC)、现场可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。中央处理元(CPU)可以为X86或ARM架构等。例如，存储器81可以包括各种形式的计算机可读存储介质的任意组合，例如易失性存储器和/或非易失性存储器。存储器81例如可以包括系统存储器，系统存储器例如存储有操作系统、应用程序、引导装载程序(Boot Loader)、数据库以及其他程序等。在存储介质中还可以存储各种应用程序和各种数据等。
另外，根据本公开的一些实施例，根据本公开的各种操作/处理在通过软件和/或固件实现的情况下，可从存储介质或网络向具有专用硬件结构的计算机系统，例如图9所示的计算机系统(或电子设备)90安装构成该软件的程序，该计算机系统在安装有各种程序时，能够执行各种功能，包括诸如前文的功能等等。图9是示出根据本公开的实施例的中可采用的计算机系统的示例结构的框图。
在图9中，中央处理单元(CPU)901根据只读存储器(ROM)902中存储的程序或从存储部分908加载到随机存取存储器(RAM)903的程序执行各种处理。在RAM 903中，也根据需要存储当CPU 901执行各种处理等时所需的数据。中央处理单元仅仅是示例性的，其也可以是其它类型的处理器，诸如前文的各种处理器。ROM 902、RAM 903和存储部分908可以是各种形式的计算机可读存储介质，如下文。需要注意的是，虽然图9中分别示出了ROM 902、RAM 903和存储装置908，但是它们中的一个或多个可以合并或者位于相同或不同的存储器或存储模块中。
CPU 901、ROM 902和RAM 903经由总线904彼此连接。输入/输出接口905也连接到总线904。
下述部件连接到输入/输出接口905：输入部分906，诸如触摸屏、触摸板、键盘、鼠标、图像传感器、麦克风、加速度计、陀螺仪等；输出部分907，包括显示器，比如阴极射线管(CRT)、液晶显示器(LCD)，扬声器，振动器等；存储部分908，包括硬盘，磁带等；和通信部分909，包括网络接口卡比如LAN卡、调制解调器等。通信部分909允许经由网络比如因特网执行通信处理。容易理解的是，虽然图9中示出计算机系统90中的各个装置或模块是通过总线904来通信的，但它们也可以通过网络或其它方式进行通信，其中，网络可以包括无线网络、有线网络、和/或无线网络和有线网络的任意组合。
根据需要，驱动器910也连接到输入/输出接口905。可拆卸介质911比如磁盘、光盘、磁光盘、半导体存储器等等根据需要被安装在驱动器910上，使得从中读出的计算机程序根据需要被安装到存储部分908中。
在通过软件实现上述系列处理的情况下，可以从网络比如因特网或存储介质比如可拆卸介质911安装构成软件的程序。
根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分909从网络上被下载和安装，或者从存储部分908被安装，或者从ROM 902被安装。在该计算机程序被CPU 901执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，在本公开的上下文中，计算机可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是，但不限于：电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
在一些实施例中，还提供了一种计算机程序，包括：指令，指令当由处理器执行时使处理器执行上述任一个实施例的直播画面的处理方法。例如，指令可以体现为计算机程序代码。
在本公开的实施例中，可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言，诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络(，包括局域网(LAN)或广域网(WAN))连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的模块、部件或单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，模块、部件或单元的名称在某种情况下并不构成对该模块、部件或单元本身的限定。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示例性的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
根据本公开的一些实施例，提供的一种直播画面的处理方法，包括：响应于针对互动直播中目标效果的显示请求，获取目标效果的资源，其中，互动直播由多个直播用户共同参与，目标效果作用于多个直播用户中至少两个目标直播用户，目标效果包括至少两个目标直播用户中每个目标直播用户对应的效果，至少两个目标直播用户对应的效果是关联的；根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果；在互动直播的合流画面中呈现目标效果。
在一些实施例中，根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果包括：确定每个目标直播用户对应的效果的播放顺序信息；根据目标效果的资源、互动直播的合流画面的布局信息和每个目标直播用户对应的效果的播放顺序信息，渲染每个目标直播用户对应的效果。
在一些实施例中，目标效果的资源包括每个目标直播用户对应的效果的素材，根据目标效果的资源、互动直播的合流画面的布局信息和每个目标直播用户对应的效果的播放顺序信息，渲染每个目标直播用户对应的效果包括：针对每个目标直播用户对应的效果，根据该效果的播放顺序信息，确定该效果在互动直播的合流中对应的帧；根据互动直播的合流画面的布局信息，将该效果的素材添加到该效果在互动直播的合流中对应的帧上，以实现对效果的渲染。
在一些实施例中，根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果包括：确定每个目标直播用户对应的效果的添加方式信息；根据目标效果的资源、互动直播的合流画面的布局信息和每个目标直播用户对应的效果的添加方式信息，渲染每个目标直播用户对应的效果。
在一些实施例中，目标效果的资源包括每个目标直播用户对应的效果的素材，互动直播的合流画面的布局信息包括每个目标直播用户的显示区域的信息，针对每个目标直播用户对应的效果，该效果的添加方式信息包括该效果在与其对应的目标直播用户的显示区域中的目标位置，根据目标效果的资源、互动直播的合流画面的布局信息和每个目标直播用户对应的效果的添加方式信息，渲染每个目标直播用户对应的效果包括：针对每个目标直播用户对应的效果，根据该效果对应的目标直播用户在互动直播的合流画面中的显示区域和该效果在与其对应的目标直播用户的显示区域中的目标位置，从互动直播的合流的帧中提取目标位置对应的目标图像；将该效果的素材与目标图像合成，生成渲染后的帧。
在一些实施例中，目标效果的资源包括每个目标直播用户对应的效果的素材，针对每个目标直播用户对应的效果，该效果的添加方式信息包括该效果的轨迹信息，根据目标效果的资源、互动直播的合流画面的布局信息和每个目标直播用户对应的效果的添加方式信息，渲染每个目标直播用户对应的效果包括：针对每个目标直播用户对应的效果，根据该效果对应的轨迹信息调整该效果的素材的运动方向；根据互动直播的合流画面的布局信息，将调整后的该效果的素材添加到互动直播的合流的帧中。
在一些实施例中，至少两个目标直播用户对应的效果是联动的，至少两个目标直播用户中一个目标直播用户对应的效果为引导效果，其他目标直播用户对应的效果包括衔接效果和反馈效果，引导效果、衔接效果和反馈效果的播放顺序由先到后。
在一些实施例中，述互动直播的合流画面的布局信息包括多个直播用户中每个直播用户的显示区域的信息和互动区域的信息，互动区域用于显示观看用户与直播用户的互动信息，根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果包括：根据目标效果的资源，在引导效果对应的目标直播用户的显示区域中，渲染引导效果，在反馈效果对应的目标直播用户的显示区域中，渲染反馈效果，在互动区域中，渲染引导效果，在由互动区域到反馈效果对应的目标直播用户的显示区域的衔接区域中，渲染衔接效果。
在一些实施例中，在互动直播的合流画面中呈现目标效果包括：显示互动直播的合流画面，在引导效果对应的目标直播用户的显示区域和互动区域中，呈现引导效果，在由互动区域到反馈效果对应的目标直播用户的显示区域的衔接区域中，呈现衔接效果，在反馈效果对应的目标直播用户的显示区域中，呈现反馈效果。
在一些实施例中，在处理方法由多个直播用户中任意直播用户的客户端执行的情况下，处理方法还包括：针对任意直播用户，将该直播用户的第一直播流采用本地效果进行渲染，并发送至服务端；获取该直播用户以外的其他直播用户的第二直播流；将第一直播流和第二直播流进行合流，得到互动直播的合流。
在一些实施例中，响应于针对互动直播中目标效果的显示请求，获取目标效果的资源包括：响应于观看用户针对互动直播中目标效果的显示请求的触发操作，接收显示请求，其中，显示请求中包括目标效果的资源的标识；根据目标效果的资源的标识，获取目标效果的资源。
在一些实施例中，处理方法还包括：响应于针对互动直播中目标效果的显示请求，获取至少两个目标直播用户的标识，其中，根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果包括：根据每个目标直播用户的标识和互动直播的合流画面的布局信息，确定每个目标直播用户在互动直播的合流画面中的显示区域；根据目标效果的资源和每个目标直播用户在互动直播的合流画面中的显示区域，渲染每个目标直播用户对应的效果。
在一些实施例中，响应于针对互动直播中目标效果的显示请求，获取至少两个目标直播用户的标识包括：显示效果选择区域，其中，效果选择区域包括一个或多个效果，一个或多个效果包括目标效果；响应于观看用户对目标效果的选择，显示目标效果作用的直播用户的选择区域；响应于观看用户对至少两个目标直播用户的选择，触发针对目标效果的显示请求，获取至少两个目标直播用户的标识。
根据本公开的另一些实施例，提供的一种直播画面的处理装置，包括：获取模块，被配置为响应于观看用户对互动直播中目标效果的触发操作，获取目标效果的资源，其中，互动直播由多个直播用户共同参与，目标效果作用于多个直播用户中至少两个目标直播用户，目标效果包括作用于至少两个目标直播用户中每个目标直播用户对应的效果；渲染模块，被配置为根据目标效果的资源和互动直播的合流画面的布局信息，渲染每个目标直播用户对应的效果；显示模块，被配置为在互动直播的合流画面中呈现目标效果。
根据本公开的又一些实施例，提供的一种电子设备，包括：存储器；以及耦接至存储器的处理器，处理器被配置为基于存储在存储器中的指令，执行如本公开中的任一实施例的直播画面的处理方法。
根据本公开的再一些实施例，提供的一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现本公开中的任一实施例的直播画面的处理方法。
根据本公开的又一些实施例，提供的一种计算机程序，包括：指令，指令当由处理器执行时使处理器执行本公开中的任一实施例的直播画面的处理方法。
根据本公开的一些实施例，提供的一种计算机程序产品，包括指令，指令当由处理器执行时实现本公开中的任一实施例的直播画面的处理方法。
以上描述仅为本公开的一些实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
在本文提供的描述中，阐述了许多特定细节。然而，理解的是，可以在没有这些特定细节的情况下实施本发明的实施例。在其他情况下，为了不模糊该描述的理解，没有对众所周知的方法、结构和技术进行详细展示。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
虽然已经通过示例对本公开的一些特定实施例进行了详细说明，但是本领域的技术人员应该理解，以上示例仅是为了进行说明，而不是为了限制本公开的范围。本领域的技术人员应该理解，可在不脱离本公开的范围和精神的情况下，对以上实施例进行修改。本公开的范围由所附权利要求来限定。

Claims (18)

  1. 一种直播画面的处理方法，包括：
    响应于针对互动直播中目标效果的显示请求，获取所述目标效果的资源，其中，所述互动直播由多个直播用户共同参与，所述目标效果作用于所述多个直播用户中至少两个目标直播用户，所述目标效果包括所述至少两个目标直播用户中每个目标直播用户对应的效果，所述至少两个目标直播用户对应的效果是关联的；
    根据所述目标效果的资源和所述互动直播的合流画面的布局信息，渲染所述每个目标直播用户对应的效果；
    在所述互动直播的合流画面中呈现所述目标效果。
  2. 根据权利要求1所述的处理方法，其中，所述根据所述目标效果的资源和所述互动直播的合流画面的布局信息，渲染所述每个目标直播用户对应的效果包括：
    确定所述每个目标直播用户对应的效果的播放顺序信息；
    根据所述目标效果的资源、所述互动直播的合流画面的布局信息和所述每个目标直播用户对应的效果的播放顺序信息，渲染所述每个目标直播用户对应的效果。
  3. 根据权利要求2所述的处理方法，其中，所述目标效果的资源包括所述每个目标直播用户对应的效果的素材，所述根据所述目标效果的资源、所述互动直播的合流画面的布局信息和所述每个目标直播用户对应的效果的播放顺序信息，渲染所述每个目标直播用户对应的效果包括：
    针对所述每个目标直播用户对应的效果，根据所述效果的播放顺序信息，确定所述效果在所述互动直播的合流中对应的帧；
    根据所述互动直播的合流画面的布局信息，将所述效果的素材添加到所述效果在所述互动直播的合流中对应的帧上，以实现对所述效果的渲染。
  4. 根据权利要求1-3任一项所述的处理方法，其中，所述根据所述目标效果的资源和所述互动直播的合流画面的布局信息，渲染所述每个目标直播用户对应的效果包括：
    确定所述每个目标直播用户对应的效果的添加方式信息；
    根据所述目标效果的资源、所述互动直播的合流画面的布局信息和所述每个目标直播用户对应的效果的添加方式信息，渲染所述每个目标直播用户对应的效果。
  5. 根据权利要求4所述的处理方法，其中，所述目标效果的资源包括所述每个目标直播用户对应的效果的素材，所述互动直播的合流画面的布局信息包括所述每个目标直播用户的显示区域的信息，针对所述每个目标直播用户对应的效果，所述效果的添加方式信息包括所述效果在与其对应的目标直播用户的显示区域中的目标位置，
    所述根据所述目标效果的资源、所述互动直播的合流画面的布局信息和所述每个目标直播用户对应的效果的添加方式信息，渲染所述每个目标直播用户对应的效果包括：
    针对所述每个目标直播用户对应的效果，根据所述效果对应的目标直播用户在所述互动直播的合流画面中的显示区域和所述效果在与其对应的目标直播用户的显示区域中的目标位置，从所述互动直播的合流的帧中提取所述目标位置对应的目标图像；
    将所述效果的素材与所述目标图像合成，生成渲染后的帧。
  6. 根据权利要求4或5所述的处理方法，其中，所述目标效果的资源包括所述每个目标直播用户对应的效果的素材，针对所述每个目标直播用户对应的效果，所述效果的添加方式信息包括所述效果的轨迹信息，
    所述根据所述目标效果的资源、所述互动直播的合流画面的布局信息和所述每个目标直播用户对应的效果的添加方式信息，渲染所述每个目标直播用户对应的效果包括：
    针对所述每个目标直播用户对应的效果，根据所述效果对应的轨迹信息调整所述效果的素材的运动方向；
    根据所述互动直播的合流画面的布局信息，将调整后的所述效果的素材添加到所述互动直播的合流的帧中。
  7. 根据权利要求1-6任一项所述的处理方法，其中，所述至少两个目标直播用户对应的效果是联动的，所述至少两个目标直播用户中一个目标直播用户对应的效果为引导效果，其他目标直播用户对应的效果包括衔接效果和反馈效果，所述引导效果、衔接效果和反馈效果的播放顺序由先到后。
  8. 根据权利要求7所述的处理方法，其中，所述互动直播的合流画面的布局信息包括所述多个直播用户中每个直播用户的显示区域的信息和互动区域的信息，所述互动区域用于显示观看用户与直播用户的互动信息，所述根据所述目标效果的资源和所述互动直播的合流画面的布局信息，渲染所述每个目标直播用户对应的效果包括：
    根据所述目标效果的资源，在所述引导效果对应的目标直播用户的显示区域中，渲染所述引导效果，在所述反馈效果对应的目标直播用户的显示区域中，渲染所述反馈效果，在所述互动区域中，渲染所述引导效果，在由所述互动区域到所述反馈效果对应的目标直播用户的显示区域的衔接区域中，渲染所述衔接效果。
  9. 根据权利要求8所述的处理方法，其中，所述在所述互动直播的合流画面中呈现所述目标效果包括：
    显示所述互动直播的合流画面，在所述引导效果对应的目标直播用户的显示区域和所述互动区域中，呈现所述引导效果，在由所述互动区域到所述反馈效果对应的目标直播用户的显示区域的衔接区域中，呈现所述衔接效果，在所述反馈效果对应的目标直播用户的显示区域中，呈现所述反馈效果。
  10. 根据权利要求1-9任一项所述的处理方法，其中，在所述处理方法由所述多个直播用户中任意直播用户的客户端执行的情况下，所述处理方法还包括：
    针对所述任意直播用户，将所述直播用户的第一直播流采用本地效果进行渲染，并发送至服务端；
    获取所述直播用户以外的其他直播用户的第二直播流；
    将所述第一直播流和所述第二直播流进行合流，得到所述互动直播的合流。
  11. 根据权利要求1-10任一项所述的处理方法，其中，所述响应于针对互动直播中目标效果的显示请求，获取所述目标效果的资源包括：
    响应于观看用户针对所述互动直播中所述目标效果的显示请求的触发操作，接收所述显示请求，其中，所述显示请求中包括所述目标效果的资源的标识；
    根据所述目标效果的资源的标识，获取所述目标效果的资源。
  12. 根据权利要求1-10任一项所述的处理方法，还包括：
    响应于针对所述互动直播中所述目标效果的显示请求，获取所述至少两个目标直播用户的标识，
    其中，所述根据所述目标效果的资源和所述互动直播的合流画面的布局信息，渲染所述每个目标直播用户对应的效果包括：
    根据所述每个目标直播用户的标识和所述互动直播的合流画面的布局信息，确定所述每个目标直播用户在所述互动直播的合流画面中的显示区域；
    根据所述目标效果的资源和所述每个目标直播用户在所述互动直播的合流画面中的显示区域，渲染所述每个目标直播用户对应的效果。
  13. 根据权利要求12所述的处理方法，其中，所述响应于针对所述互动直播中所述目标效果的显示请求，获取所述至少两个目标直播用户的标识包括：
    显示效果选择区域，其中，所述效果选择区域包括一个或多个效果，其中，所述一个或多个效果包括所述目标效果；
    响应于观看用户对所述目标效果的选择，显示所述目标效果作用的直播用户的选择区域；
    响应于所述观看用户对所述至少两个目标直播用户的选择，触发针对所述目标效果的显示请求，获取所述至少两个目标直播用户的标识。
  14. 一种直播画面的处理装置，包括：
    获取模块，被配置为响应于观看用户对互动直播中目标效果的触发操作，获取所述目标效果的资源，其中，所述互动直播由多个直播用户共同参与，所述目标效果作用于所述多个直播用户中至少两个目标直播用户，所述目标效果包括所述至少两个目标直播用户中每个目标直播用户对应的效果，所述至少两个目标直播用户对应的效果是关联的；
    渲染模块，被配置为根据所述目标效果的资源和所述互动直播的合流画面的布局信息，渲染所述每个目标直播用户对应的效果；
    显示模块，被配置为在所述互动直播的合流画面中呈现所述目标效果。
  15. 一种电子设备，包括：
    处理器；以及
    耦接至所述处理器的存储器，用于存储指令，所述指令被所述处理器执行时，使所述处理器执行如权利要求1-13任一项所述的直播画面的处理方法。
  16. 一种计算机可读存储介质，其上存储有计算机程序，其中，该程序被处理器执行时实现权利要求1-13任一项所述的直播画面的处理方法的步骤。
  17. 一种计算机程序产品，包括：指令，其中，该指令被处理器执行时实现权利要求1-13任一项所述的直播画面的处理方法的步骤。
  18. 一种计算机程序，包括：指令，其中，该指令被处理器执行时实现权利要求1-13任一项所述的直播画面的处理方法。
PCT/CN2025/100985 2024-06-13 2025-06-13 直播画面的处理方法、装置、电子设备、介质和程序产品 Pending WO2025256645A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410764675.7A CN121151573A (zh) 2024-06-13 2024-06-13 直播画面的处理方法、装置、电子设备、介质和程序产品
CN202410764675.7 2024-06-13

Publications (1)

Publication Number Publication Date
WO2025256645A1 true WO2025256645A1 (zh) 2025-12-18

Family

ID=97994928

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2025/100985 Pending WO2025256645A1 (zh) 2024-06-13 2025-06-13 直播画面的处理方法、装置、电子设备、介质和程序产品

Country Status (2)

Country Link
CN (1) CN121151573A (zh)
WO (1) WO2025256645A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022062896A1 (zh) * 2020-09-22 2022-03-31 北京达佳互联信息技术有限公司 直播互动方法及装置
CN116896649A (zh) * 2023-09-11 2023-10-17 北京达佳互联信息技术有限公司 直播互动方法、装置、电子设备及存储介质
CN117641031A (zh) * 2023-11-24 2024-03-01 北京字跳网络技术有限公司 直播互动方法、装置、设备和存储介质
CN118118723A (zh) * 2024-02-29 2024-05-31 北京字跳网络技术有限公司 特效处理方法、装置、电子设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022062896A1 (zh) * 2020-09-22 2022-03-31 北京达佳互联信息技术有限公司 直播互动方法及装置
CN116896649A (zh) * 2023-09-11 2023-10-17 北京达佳互联信息技术有限公司 直播互动方法、装置、电子设备及存储介质
CN117641031A (zh) * 2023-11-24 2024-03-01 北京字跳网络技术有限公司 直播互动方法、装置、设备和存储介质
CN118118723A (zh) * 2024-02-29 2024-05-31 北京字跳网络技术有限公司 特效处理方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN121151573A (zh) 2025-12-16

Similar Documents

Publication Publication Date Title
US10229651B2 (en) Variable refresh rate video capture and playback
US20250056068A1 (en) Live broadcasting comment presentation method and apparatus, and device, program product and medium
US20220398786A1 (en) Interaction method and apparatus, and electronic device and computer-readable storage medium
WO2017148410A1 (zh) 一种信息交互的方法、设备及系统
CN117519544A (zh) 媒体内容的生成方法、显示方法和相关设备
US20250200857A1 (en) Animation effect generation method and apparatus, device, and medium
WO2022033494A1 (zh) 交互类直播连麦的控制方法、装置、系统、设备及介质
CN113806306B (zh) 媒体文件处理方法、装置、设备、可读存储介质及产品
WO2023202360A1 (zh) 视频预览方法、装置、可读介质及电子设备
CN115830224A (zh) 多媒体数据的编辑方法、装置、电子设备及存储介质
CN114419213A (zh) 图像处理方法、装置、设备和存储介质
CN115883857A (zh) 一种直播礼物云渲染方法、装置、电子设备及存储介质
WO2024179508A1 (zh) 一种虚拟礼物生成方法、装置、设备及介质
WO2023143217A1 (zh) 特效道具的展示方法、装置、设备及存储介质
WO2025118949A1 (zh) 特效处理方法、装置、电子设备及存储介质
US20250294209A1 (en) Method, apparatus, electronic device and storage medium for video live streaming
CN117729347A (zh) 直播方法、直播交互方法、装置、电子设备和介质
WO2022213979A1 (zh) 特效显示方法、装置、设备、存储介质及产品
US12555328B2 (en) Video playing method and apparatus, electronic device, and storage medium
WO2025256645A1 (zh) 直播画面的处理方法、装置、电子设备、介质和程序产品
WO2023284414A1 (zh) 音频合流方法、音频上传方法、设备及程序产品
US20240160341A1 (en) Video processing methods, device, storage medium, and program product
CN118890530A (zh) 视频生成方法及装置、计算机可读存储介质、程序产品
CN116112617B (zh) 演播画面的处理方法、装置、电子设备及存储介质
US12457466B2 (en) Audio control method, control device, driving circuit and readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25821345

Country of ref document: EP

Kind code of ref document: A1