CN112423013B

CN112423013B - Online interaction method, client, server, computing device and storage medium

Info

Publication number: CN112423013B
Application number: CN202011304869.7A
Authority: CN
Inventors: 陈裕龙; 宋晨光
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-11-19
Filing date: 2020-11-19
Publication date: 2021-11-16
Anticipated expiration: 2040-11-19
Also published as: CN112423013A

Abstract

The invention provides an online interaction method for a client, which comprises the following steps: in a first interaction turn of the online interaction, sending task execution data for a first interaction task based on input from a first category of users, the first interaction task being executed by a first category of user groups including the first category of users and at least one other first category of users; receiving and presenting a notification of completion of a first interactive task by the first category of users, the notification generated in response to a cumulative value of contribution data obtained by the first category of users being greater than or equal to a first contribution data threshold, wherein the contribution data was provided by a second category of users based on task execution data; receiving and presenting a notification of a second turn of interaction into the online interaction, the notification generated in response to at least two first category users of the first category user group completing the first interaction task. The method is beneficial to improving the probability of the second category of users participating in the interaction, so that the utilization efficiency of online interaction resources is improved.

Description

Online interaction method, client, server, computing device and storage medium

Technical Field

The present invention relates to the field of network communication technologies, and in particular, to an online interaction method, a client, a server, a computing device, and a storage medium.

Background

The development of network communication technology has led to a plurality of emerging online interaction modes, wherein the live network is popular among a plurality of users, and becomes a popular online interaction mode. Generally, webcasting is conducted via a live platform and involves a main broadcast (i.e., a presenter) and a viewer (i.e., a viewer). The audio and/or video data of the anchor may be captured, processed, and transmitted via the client of the anchor and the live platform such that viewers may use their clients to enter the live room of the respective anchor to obtain the audio and/or video content of the respective anchor. In addition, in order to enrich the interaction mode and improve the interaction experience, some live broadcast platforms also provide a live broadcast mode of connecting multiple people with the wheat, namely, multiple anchor broadcasters can interact through connecting the wheat to jointly complete at least one section of live broadcast content, and audiences can enter a live broadcast room of any anchor broadcaster participating in connecting the wheat and simultaneously acquire the audio and/or video content of the multiple anchor broadcasters.

However, in the current multi-people live broadcast mode, the interaction mode between the audience and the anchor is single, and is generally limited to simple leaving a message and presenting a gift, and the participation of the audience cannot be well improved, so that the audience is easy to lose the interaction interest, further the interactivity of the live broadcast network is poor, the utilization efficiency of resources such as network bandwidth, storage space and processing unit occupied by the live broadcast network is low, and the waste of the resources is caused.

Disclosure of Invention

In view of the above, the present invention provides an online interaction method, a client, a server, a computing device and a storage medium, which aim to alleviate, reduce or even eliminate the above mentioned problems and other problems that may exist.

According to an aspect of the present invention, an online interaction method for a client is provided, including: in a first interaction turn of the online interaction, sending task execution data for a first interaction task based on input from a first category user, wherein the first interaction task is executed by a first category user group, and the first category user group comprises the first category user and at least one other first category user; receiving and presenting a notification of completion of a first interactive task by the first category of users, the notification generated in response to a cumulative value of contribution data obtained by the first category of users being greater than or equal to a first contribution data threshold, wherein the contribution data is provided by the second category of users; receiving and presenting a notification of entry into a second turn of the online interaction, the notification generated in response to at least two first category users of the first category user group completing a first interaction task.

In some embodiments, the method further comprises: receiving and presenting a notification that the first category user did not complete the first interactive task, the notification generated in response to the cumulative value of contribution data made by the first category user being less than a first contribution data threshold.

In some embodiments, the method further comprises: a notification of the first category user's win is received and presented, the notification generated in response to only the first category user completing the first interactive task.

In some embodiments, said sending task execution data for the first interactive task based on input from a first category of users comprises: acquiring video data of the first category of users; receiving video data from other first category users in the first category user group; mixed-flow video data is generated as at least part of the task execution data based on the video data of the first category user and the video data of the other first category users.

In some embodiments, the mixed stream video data comprises video encoded data and flow control data, and wherein said generating mixed stream video data comprises: laying out the video data of the first class user and the video data of other first class users in different areas; encoding the video data after layout to generate the video encoding data; control information is written in the flow control data based on a progress of the online interaction, the control information indicating that different image information is presented in different areas.

In some embodiments, the encoding the laid out video data comprises: the mixed flow video data is encoded based on a region of interest in the video data.

In some embodiments, the process of online-interaction-based writing control information in the flow control data comprises: and in response to that the first class user does not complete the first interaction task, identifying a face region corresponding to the first class user, and writing control information indicating that punishment information is presented in the face region into the flow control data.

In some embodiments, said receiving video data from other first category users of said first category user group comprises: video data from other first category users is received in real time over the end-to-end connection.

In some embodiments, said sending task execution data for the first interactive task based on input from a first category of users comprises: acquiring audio data of the first category of users; receiving audio data from other first category users in the first category user group; mixed-flow audio data is generated as at least a part of the task execution data based on the audio data of the first category user and the audio data of the other first category users.

In some embodiments, the generating mixed flow audio data comprises: simulating at least two audio channels; adding the audio data of each first category user to a corresponding audio channel; and combining the audio data in the plurality of audio channels to obtain mixed flow audio data.

In some embodiments, the first interaction turn includes at least two interaction periods, and the first interaction task is sequentially executed by the first category user and at least one first category user within the corresponding interaction period, wherein the notification that the first category user completes the first interaction task is generated in response to the cumulative value of the contribution data obtained by the first category user within the corresponding interaction period being greater than or equal to a first contribution data threshold.

In some embodiments, the method further comprises: receiving an interaction initiation input from the first category of users, the interaction initiation input being for initiating the online interaction; and responding to the interaction initiation input, and sending a request for starting the online interaction.

In some embodiments, the method further comprises: receiving a request for starting the online interaction; an interaction consent input is received from the first category user, the interaction consent input indicating that the first category user consents to open the online interaction.

In some embodiments, the method further comprises: in a second interaction turn, task execution data for a second interaction task is sent based on input from the first category of users, wherein a second contribution data threshold corresponding to the second interaction task is greater than the first contribution data threshold.

According to another aspect of the present invention, there is provided an online interaction method for a client, including: receiving and presenting task execution data of a first category user aiming at a first interaction task in a first interaction turn of the online interaction, wherein the first interaction task is executed by a first category user group, and the first category user group comprises the first category user and at least one other first category user; transmitting contribution data based on input from a second category of users; receiving and presenting a notification of completion of a first interactive task by a first category of users, the notification generated in response to an accumulated value of contribution data obtained by the first category of users being greater than or equal to a first contribution data threshold; receiving and presenting a notification of entry into a second turn of the online interaction, the notification generated in response to at least two first category users of the first category user group completing a first interaction task.

In some embodiments, the method further comprises: presenting a resource package, the resource package generated in response to entering a second interaction turn; sending a request for acquiring the resource package based on the input of the second category user request for acquiring the resource package; and presenting the resources in the resource package in response to receiving the notification of successful acquisition of the resource package.

In some embodiments, the receiving and presenting task execution data for a first interactive task for a first category of users comprises: receiving mixed flow video data; presenting video data from at least two first category users based on the mixed stream video data.

In some embodiments, the mixed stream video data comprises video encoded data and stream control data, and the presenting video data from at least two first category users comprises: decoding the video coding data; presenting video data of each first category user in different regions based on the decoded video data; presenting different image information in different areas based on control information in the flow control data, wherein the control information is written based on a progress of an online interaction.

In some embodiments, the first interaction turn includes at least two interaction periods, and the first interaction task is sequentially executed by the first category user and at least one other first category user within the respective interaction periods, wherein the notification that the first category user completed the first interaction task is generated in response to the cumulative value of the contribution data obtained by the first category user within the respective interaction periods being greater than or equal to a first contribution data threshold.

In some embodiments, the input of the second category of users comprises input indicating that a special effect is achieved, the method further comprising: in response to receiving an input indicating a special effect, presenting a notification related to at least one of the following special effects: increasing the time length of the current interaction time period; decreasing a first contribution data threshold associated with a current interaction period; the order in which the first interactive tasks are opened for the at least two first category users is updated.

According to another aspect of the present invention, there is provided an online interaction method for a server, including: sending a notification for starting a first interactive task in a first interactive turn of the online interaction, wherein the first interactive task is executed by a first category user group, and the first category user group comprises at least two first category users; receiving task execution data for the first interactive task from a client of each first category user in a first category user group; sending task execution data of the first category of users to the second category of users; receiving contribution data from clients of users of a second category; accumulating the resulting contribution data for each first category of users; generating and sending a notification that at least one first category user in the first category user group completes a first interaction task in response to an accumulated value of contribution data obtained by the at least one first category user being greater than or equal to a first contribution data threshold; and generating and sending a notification of a second interaction turn entering the online interaction in response to at least two first category users in the first category user groups completing the first interaction task.

In some embodiments, the method further comprises: changing a code rate of the received task execution data to obtain at least two task execution data with different code rates, wherein the task execution data includes at least one of mixed stream video data and mixed stream audio data, and wherein the sending the task execution data of the first category of users to the second category of users includes: and selectively sending task execution data with corresponding code rate to the second category of users based on network conditions or code rate selection requests from the clients of the second category of users.

According to another aspect of the present invention, there is provided a client, including: a sending module configured to send task execution data for a first interactive task based on input from a first category of users in a first interaction turn of the online interaction, the first interactive task being executed by a first category of user groups, the first category of user groups including the first category of users and at least one other first category of users; a first presentation module configured to receive and present a notification of completion of a first interactive task by the first category of users, the notification generated in response to a cumulative value of contribution data made by the first category of users being greater than or equal to a first contribution data threshold, wherein the contribution data was provided by a second category of users; a second presentation module configured to receive and present a notification of a second turn of interaction into the online interaction, the notification generated in response to at least two first category users of the first category user group completing a first interaction task.

According to another aspect of the present invention, there is provided a client, including: a first presentation module configured to receive and present task execution data of a first category user for a first interactive task in a first interaction turn of the online interaction, the first interactive task being executed by a first category user group, the first category user group including the first category user and at least one other first category user; a sending module configured to send contribution data based on input from a second category of users; a second presentation module configured to receive and present a notification of completion of the first interactive task by a first category of users, the notification generated in response to a cumulative value of contribution data made by the first category of users being greater than or equal to a first contribution data threshold; a third presentation module configured to receive and present a notification of entry into a second interaction turn of the online interaction, the notification generated in response to at least two first category users of the first category user group completing a first interaction task.

According to another aspect of the present invention, there is provided a server comprising: a first sending module configured to send a notification of opening a first interactive task in a first interactive turn of the online interaction, the first interactive task being executed by a first category user group, the first category user group including at least two first category users; a first receiving module configured to receive task execution data for the first interactive task from a client of each first category user in a first category user group; a second sending module configured to send the task execution data of the first category user to the second category user; a second receiving module configured to receive contribution data from clients of users of a second category; an accumulation module configured to accumulate the resulting contribution data for each first category of users; a third sending module, configured to generate and send a notification that at least one first category user in the first category user group completes a first interactive task in response to an accumulated value of contribution data obtained by the at least one first category user being greater than or equal to a first contribution data threshold; a fourth sending module configured to generate and send a notification of a second interaction turn into the online interaction in response to at least two first category users in the first category user group completing the first interaction task.

According to another aspect of the present invention, there is provided a computing device comprising a memory and a processor, the memory being configured to store thereon computer-executable instructions that, when executed on the processor, perform the method of the preceding aspect.

According to another aspect of the present invention, there is provided a computer-readable storage medium having stored thereon computer-executable instructions which, when executed on a processor, perform the method of the preceding aspect.

In the online interaction method provided by the invention, in one interaction turn, whether a first category user (such as a main broadcasting) completes a current interaction task can be determined based on the accumulated value of contribution data provided by a second category user (such as a audience), and when at least two first category users (such as main broadcasting) complete the interaction task, the next interaction turn can be entered. By this scheme, it is allowed to determine whether a first category user will continue to occupy the live broadcast resource (e.g. associated network bandwidth, storage space, processing unit, etc.) by the interactive behavior of the second category user providing contribution data, thereby facilitating the tilting of the live broadcast resource to the first category user that can bring more interactive behavior, and improving the utilization efficiency of the live broadcast resource. Meanwhile, the scheme is beneficial to increasing the degree of association between the second category users and the first category users, improving the intention of the second category users to participate in interaction, and increasing the probability of the occurrence of interaction behaviors, thereby being beneficial to further improving the utilization efficiency of live broadcast resources.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.

Drawings

Further details, features and advantages of the invention are disclosed in the following description of exemplary embodiments, taken in conjunction with the accompanying drawings, in which:

FIGS. 1A and 1B schematically illustrate examples of online interactive interfaces in the related art;

FIG. 2 schematically illustrates an example scenario in which aspects of the present technique may be applied;

FIG. 3 schematically illustrates an example flow diagram of an online interaction method in accordance with some embodiments of the invention;

FIG. 4 schematically illustrates an example process of video blending at a client of a first category of users, according to some embodiments of the invention;

FIG. 5 schematically illustrates an architecture diagram of the WebRTC protocol;

FIG. 6 schematically illustrates a flow control scheme according to some embodiments of the inventions;

FIG. 7 schematically illustrates an example process of audio-video blending at a client of a first category of users, according to some embodiments of the invention;

FIG. 8 schematically illustrates an example flow diagram of a method of online interaction according to further embodiments of the invention;

9A-G schematically illustrate example interface diagrams according to some embodiments of the invention;

FIG. 10 schematically illustrates an example flow diagram of a method of online interaction in accordance with further embodiments of the invention;

FIG. 11 schematically illustrates an example flow diagram of a method of online interaction in accordance with further embodiments of the invention;

FIG. 12 schematically illustrates an example block diagram of a client in accordance with some embodiments of the invention;

FIG. 13 schematically illustrates an example block diagram of a client in accordance with further embodiments of the invention;

FIG. 14 schematically illustrates an example block diagram of a server in accordance with some embodiments of the invention;

FIG. 15 schematically illustrates an example block diagram of a computing device in accordance with some embodiments of this invention.

Detailed Description

Before describing embodiments of the present invention in detail, some relevant concepts are explained first:

1. and (4) live broadcast: refers to a technology for acquiring data (at least one of audio, video, etc.) of a broadcasting party through a certain device, performing a series of processing (e.g., video coding compression, etc.) on the data to form a media stream which can be viewed and transmitted, and outputting the media stream to a watching client, where live broadcasting may include live audio and live video;

2. a plurality of people connecting with wheat: refers to a technology for collecting media streams (e.g., at least one of video streams, audio streams, etc.) of multiple broadcasters into one media stream and outputting the media stream to a viewing user terminal.

Fig. 1A and 1B schematically show example interfaces 100A and 100B for online interaction during live multi-person coupled live broadcasting in the related art. The interface 100A and the interface 100B may be presented by a client of a user watching a live broadcast.

As shown in fig. 1A, assuming five anchor casts are connecting to the wheat, live video of each anchor can be presented in the video presentation area 110. In the related art, multiple anchor broadcasters who are connecting to the wheat can complete unified interactive tasks within a specified time period. For example, as indicated by reference numeral 120, five anchor casts are performing talent spellings within a specified time, and viewers can give gifts to the heart's anchor by clicking on the gift button 140 to help the anchor complete the interactive task based on the video content of the anchor performing the talent spellings. In addition, the audience may leave a message for the anchor, and the message content 130, the gift-giving information, and the like may be presented in the interactive information presentation area. The value may be accumulated for the anchor based on the gifts given by the viewers, such as the sum of the forces of the battle in interface 100B of fig. 1B. When the total of the battle force reaches 10000, as indicated by reference numeral 150, the interactive task is considered to be completed, wherein the live broadcast room of the anchor with the most battle force can be considered as winning, and the audience in the live broadcast room can obtain the opportunity of drawing a prize from a treasure box.

The interactive process is simple, the interestingness is not strong, and audiences are easy to fatigue and lose interactive interest. In addition, in the interaction process, no obvious reward and punishment difference exists between the victory anchor and the failure anchor, the interaction atmosphere cannot be well driven, and the interaction is not favorable for being effectively carried out. In addition, in the mechanism, in most cases, the audience only pays unilaterally, but rarely receives reward feedback, which also easily makes the audience lose the interactive willingness. Therefore, under such an interaction mechanism, the amount of interaction generated by the live broadcast may not be satisfactory, and may be greatly reduced over time, so that communication, storage, processing resources and the like occupied by the live broadcast cannot be efficiently utilized, and thus, resources are wasted.

In view of the above analysis, the present invention provides a new online interaction scheme that helps solve or avoid the above problems as much as possible.

Fig. 2 schematically shows an example scenario 200 in which the solution provided by the present invention may be applied. The scenario 200 may comprise at least two users of a first category (e.g. anchor) 210 and their terminal devices (i.e. first terminal devices) 220, a server 230 and at least one user of a second category (e.g. viewer) 240 and their terminal devices (i.e. second terminal devices) 250. The first terminal device 220, the server 230, and the second terminal device 250 may have respective programs deployed thereon for performing the various methods provided by the present disclosure.

According to an embodiment of the present invention, at least two first category users 210 may engage in microphone to participate in online interaction, and at least one second category user 240 may enter the interactive room of any one of the first category users 210 to participate in online interaction. Illustratively, the first category of users 210 may be anchor, the second category of users may be viewers, the interactive room may be a live room, and the viewers may enter any live room of the anchor that is online and participating in the online interaction to participate in the online interaction.

For clarity, the first terminal device 220 and the second terminal device 250 are depicted in fig. 2 as two different devices. However, in practice, the first terminal device 220 and the second terminal device 250 may be the same terminal device, and the plurality of first terminal devices 220 may be different terminal devices, and likewise, the plurality of second terminal devices 250 may also be different terminal devices. Examples of the first terminal device 220 and the second terminal device 250 include, but are not limited to, desktop computers, laptop computers, tablet computers, smart phones, smart watches, and the like. Further, the program deployed on the first terminal device 220 or the second terminal device 250 may be a separate application program, may be some function embedded in another application program, or may relate to an applet or a web program that can be accessed via another application program, or the like.

Server 230 may be a single server or a cluster of servers, or may be other computing devices with communication, storage, and communication capabilities. Further, although the server 230 is shown separately in fig. 2, it is also possible that it is integrated with the first terminal device 220 or the second terminal device 250, or that one part is integrated with the first terminal device 220 and the other part is integrated with the second terminal device 250. It should be understood that when in the integrated state, communication via the network may be replaced by internal communication of the device.

The specific structures of the first terminal device 220, the server 230, and the second terminal device 250 will be described in detail below. Further, it should be understood that "network" as described herein may include the internet, a Local Area Network (LAN), a telephone network, an intranet, other public and/or proprietary networks, and combinations thereof, and may involve wireless networks (such as in the form of cellular networks, WiFi, bluetooth, LiFi, ZigBee, etc.) as well as wired networks (such as via cable, fiber optics, etc.). Accordingly, the first terminal device 220, the server 230, and the second terminal device 250 may have interfaces that support communication via respective networks.

FIG. 3 schematically illustrates an example flow diagram of an online interaction method 300 in accordance with some embodiments of the invention. Illustratively, the online interaction method 300 may be implemented by means of the first terminal device 220 of any one of the first category users 210 shown in fig. 2, and the client in the method 300 may be deployed on the corresponding first terminal device 220. As shown, the online interaction method 300 includes the following steps: in a first interaction turn of the online interaction, sending task execution data for a first interaction task based on input from a first category of users, the first interaction task being executed by a first category of user groups, the first category of user groups including the first category of users and at least one other first category of users (step 310); receiving and presenting a notification of completion of a first interactive task by the first category of users, the notification generated in response to a cumulative value of contribution data available to the first category of users being greater than or equal to a first contribution data threshold, wherein the contribution data was provided by the second category of users (step 320); a notification of entry into a second turn of the online interaction is received and presented, the notification generated in response to at least two first category users of the first category user group completing the first interaction task (step 330).

Specifically, in step 310, in a first interaction turn of the online interaction, the client of the first category user may receive an input from the first category user through the input interface of the first terminal device, and send task execution data for the first interaction task to the server based on the input, and the task execution data may be further sent to the client of the second category user by the server. Alternatively, the server may perform additional processing on the received task execution data and transmit the processed task execution data to the client of the second category user. This additional processing will be described in detail below. The first interactive task may be executed by the first category user group, and may be a preset task, a task determined according to a preset rule, a task set by the first category user or the second category user, or a task set at random. The first category group may include the first category user and at least one other first category user, i.e., include at least two first category users participating in the online interaction.

In some embodiments, the first category of users may be anchor broadcasters conducting live broadcasting, and the first category of user groups may be user groups consisting of at least two anchor broadcasters conducting live broadcasting. In some embodiments, the first category of user input may include one or both of video input, audio input, and accordingly, the transmitted task performance data may include a media stream generated based on these inputs. Alternatively, the first category of user input may also include other types of input, such as text input, input from other applications, and so forth.

In some embodiments, the input of the first category of users may include video input and accordingly, the task execution data may include video stream data. Specifically, sending task execution data for the first interactive task based on input from the first category of users may include: acquiring video data of the first category of users; receiving video data from other first category users in the first category user group; mixed flow video data is generated as at least part of the task execution data based on the video data of the first category user and the video data of the other first category users.

For example, a client of a first category user may acquire video data of the first category user via a camera or the like and receive video data of other first category users from a server via a network. The video data of the other first category users may be acquired by the clients of the other first category users via a camera or the like and then transmitted to the server via the network. Subsequently, the client of the first category user may mix the acquired and received video data, generating mixed flow video data. The generated mixed-flow video data may be sent to a server and then be pushed to clients of users of the second category. Alternatively, mixed-stream video data generated at a client of a certain first class user may be pushed to a client of a second class user entering the interactive room of the first class user. Or, optionally, the mixed-stream video data generated at a client of a certain first-class user may be pushed to a client of a second-class user entering an interactive room of any one of the first-class users in the first-class user group. The method comprises the steps that video data of each first category user in a first category user group are mixed at a client of the first category user, so that video frames of a plurality of first category users can be aligned conveniently, and compared with the method that the video data of each first category user is pushed to the client of a second category user independently, the phenomenon that the video frames of the first category users are not synchronous when the mixed-flow video data is pushed to the client of the second category user does not occur. In addition, compared with the mixed flow at the server side, the mixed flow at the client side of the first class of users can reduce the pressure of the server, fully utilize the processing resources of the client side and further reduce the cost of the processing resources.

Fig. 4 schematically illustrates an example process 400 of video mixing at a client of a first category of users. As shown, the first category user a client may receive video data from the first category user B, C, D client and mix the video data with the first category user a client at the local client. The mixed-flow video data may be sent to the server based on an RTMP (real time messaging protocol), and the server may push the mixed-flow video data to the clients of the corresponding second category users through an HTTP-FLV protocol or an HLS protocol. The above protocols are merely exemplary and other protocols may be selected as desired. Furthermore, although only four first category users are shown here, the video data of fewer or more first category users may be mixed, as desired.

In some embodiments, receiving video data from other first category users in the first category user group may include: video data from other first category users is received in real time over the end-to-end connection. Alternatively, as shown in fig. 4, such an end-to-end connection may be implemented based on the WebRTC protocol. Fig. 5 schematically illustrates an architecture 500 of the WebRTC protocol. As shown in the figure, the WebRTC protocol is built on the basis of UDP protocol, wherein ICE, STUN and TURN protocols are mainly used for penetrating the intranet end to end, thereby solving the problem of end-to-end connection of users under NAT protocol. While the DTLS protocol is encryption of the transmitted data. The SRTP and SCTP protocols are encapsulation and transmission control protocols for media data, wherein the SCTP protocol is a stream control transmission protocol providing TCP-like features, and SCTP can be built over UDP, and in WebRTC over DTLS protocol. The RTCPeerConnection protocol is used to establish and maintain an end-to-end connection and provide efficient audio and video stream transmission, and the RTCDataChannel protocol is used to support end-to-end arbitrary binary data transmission. Through the WebRTC real-time communication protocol, the cache-free real-time performance of transmission can be ensured, and the low-delay interaction among the first class users is facilitated. It should be understood that other protocols may be selected as desired to achieve such end-to-end connectivity.

In some embodiments, the mixed stream video data may include video encoded data and flow control data, and generating the mixed stream video data may include: laying out the video data of the first class user and the video data of other first class users in different areas; encoding the video data after layout to generate video encoded data; the process based on the online interaction writes control information in the stream control data, the control information indicating that different image information is presented in different areas.

For example, the clients of the first category users may draw the video data of each first category user on a preset canvas, and draw the video data of each first category user in different areas. Optionally, the video frames of the respective first category users may be cropped to accommodate different rendering regions. Optionally, the video data of the first category of users using the client may be drawn in the largest area, so that the second category of users entering the interactive room of the first category of users may better acquire the video picture of the first category of users. Alternatively, the video data of the respective first category users may be drawn in equally sized areas. Or, optionally, the layout area of the video data of each first category user may be adjusted based on the progress of the online interaction. The video data of the first category users are distributed differently, so that the visual center of gravity of the second category users can be guided, and the watching requirements of the different second category users can be better met.

For example, the client of the first class of users may encode mixed-stream video data to compress the video data, thereby reducing the transmission bandwidth requirement. Alternatively, the encoding may be based on codec standards such as H261, H263, H264, and the like. In some embodiments, encoding the laid out video data may include: the mixed flow video data is encoded based on a region of interest in the video data. Illustratively, the region of interest and the region of non-interest may be encoded with different degrees of compression, e.g. the region of interest is compressed to a lower degree to ensure that it is not distorted; and compressing the non-interested region to a higher degree so as to save transmission bandwidth and storage space. Region-of-interest-based encoding can be performed by an ROI (region of interest) encoding method in the related art. Furthermore, the region-of-interest based encoding may be performed on the basis of encoding performed on the basis of the above-mentioned codec standard to further compress the video data.

In some embodiments, during the online interaction process, specific image information may be presented at the clients of the second category of users based on the progress of the interaction, such as animations, prompt messages, etc. indicating that the first category of users did not complete the interaction task, that the first category of users completed the interaction task, or entered a second interaction order, etc. If the image information participates in the mixed flow at the client of the first category of users, when the client of the second category of users plays the video data with low resolution, the image information may have a blur problem, so that the viewing experience of the second category of users is affected, and the acquisition of related information is not facilitated. If the image information is rendered separately at the clients of the second category of users, the problem that the image information is not synchronized with the mixed-flow video may be caused, which may seriously affect the viewing experience of the second category of users. To solve this problem, the client of the first class user may write control information in the streaming control data during the process of generating the mixed-flow video data to instruct to present different image information in different areas based on the progress of the online interaction, that is, to present different image information in the video data presentation areas of different first class users. Alternatively, the client of a certain first-class user may write only the control information related to the first-class user in the stream control data, and the control information related to other first-class users may be written by the server in a subsequent process. Or, alternatively, the client of a certain first-class user may write control information related to each first-class user in the stream control data. Illustratively, this process may be implemented by SEI information in the H264 coding standard, i.e., control information may be written in SEI information of frame data of mixed-flow video, and sent to the server in synchronization with the frame data and streamed to the clients of the second category of users. Fig. 6 schematically shows such a scheme for implementing flow control based on SEI. H264 coding consists of two parts, VCL (video coding layer) and NAL (network abstraction layer), the NAL partitioning the data into different NALU (NAL unit) blocks, where each NALU includes a part of parameter data describing the video, these parameter data being of different Type (Type), where the parameter data of Type =6 is the SEI information carried by the NAL.

In some embodiments, writing control information in the flow control data based on the process of the online interaction may include: and in response to that the first class user does not complete the first interaction task, identifying a face area corresponding to the first class user, and writing control information indicating that punishment information is presented in the face area in the stream control data. For example, the client of the first class user may identify a face region in the video data through a face recognition technology, and write control information in the stream control data (for example, the SEI information mentioned above) based on an interaction progress of the first interaction task that is not completed by the first class user, indicating that punitive information, such as a character or an expression of "declining", a pig head image, and the like, is presented in the identified face region. Alternatively, the client of the first class user may write penalty information related to the first class user only in the flow control data, and then the server may write penalty information related to other first class users in the flow control data. Or, optionally, the clients of the first class users may write punishability information associated with the respective first class users in the flow control data. The punishment information is helpful for driving the interactive atmosphere and adding the interest of online interaction.

In some embodiments, the input of the first category of users may include audio input and accordingly, the task execution data may include audio stream data. In some embodiments, sending task execution data for the first interactive task based on input from the first category of users may include: acquiring audio data of the first category of users; receiving audio data from other first category users in the first category user group; mixed-flow audio data is generated as at least a part of the task execution data based on the audio data of the first category user and the audio data of the other first category users.

For example, similar to the above-described video mixing process, the client of the first category user may acquire the audio data of the first category user via a microphone or the like and receive the audio data of other first category users from the server via the network. The audio data of the other first category users may be acquired by the clients of the other first category users via a microphone or the like and then transmitted to the server via the network. Subsequently, the client of the first category user may mix the acquired and received audio data, generating mixed-flow audio data. The generated mixed flow audio data may be sent to a server and subsequently streamed to clients of users of the second category. Alternatively, mixed-stream audio data generated at a client of a certain first class user may be streamed to a client of a second class user entering the interactive room of the first class user. Or, optionally, the mixed-stream audio data generated at a client of a certain first-class user may be streamed to a client of a second-class user entering an interactive room of any one of the first-class users in the first-class user group. Optionally, when performing audio mixing, the volume of different users in the first category may be adjusted, for example, to maximize the audio volume of the user in the first category who is performing a task or maximize the audio volume of the user in the first category who uses the client, so that the user in the second category can better obtain audio information. And, optionally, when audio mixing is performed, system sound effects may be added, for example, sound effects that prompt the first class user to complete an interactive task, the first class user to not complete an interactive task, enter the next interactive round, and the like. The mixing of the audio data of the individual first category users in the first category user group at the client of the first category users is also beneficial for the alignment of the audio data of the individual first category users and for reducing server stress.

Illustratively, the video mix and the audio mix at the clients of the first category of users may be concurrent and separate. This process is schematically illustrated in fig. 7. As shown in fig. 7, the clients of the first category users may extract each input video stream and each input audio stream from the input media data of each first category user, and mix them to obtain an output video stream and an output audio stream. For example, the input media streams may be obtained by clients of respective first category users via a camera and a microphone. Thereby, the input media data of the respective first category users are converted into a single output video stream and a single output audio stream, which may reduce the need for transmission bandwidth and increase the transmission speed.

In some embodiments, generating mixed flow audio data may include: simulating at least two audio channels; adding the audio data of each first category user to a corresponding audio channel; and combining the audio data in the plurality of audio channels to obtain mixed flow audio data. Alternatively, the clients of the first class users may sample-encode the audio data of the respective first class users, for example by PCM (pulse code modulation), and then mix the PCM data. This mixing can be done by simulating at least two audio channels to achieve a low noise, clear sound effect. Alternatively, the mixing may be performed by other schemes, such as averaging after superimposing the audio data or PCM data, performing adaptive weighted averaging, and so on.

At step 320, the client of the first category user may receive a notification from the server that the first category user completed the first interactive task and present the notification. The notification may be generated by the server in response to the cumulative value of the contribution data made by the first category of users being greater than or equal to a first contribution data threshold. The contribution data may be provided by users of the second category. Illustratively, the second category of users may receive task execution data for the first category of users from the server through their clients and provide contribution data for the task execution data. In the first interaction turn, the server may accumulate the obtained contribution data for each first category user, and when the accumulated contribution data is greater than or equal to a first contribution data threshold, may generate a notification that the corresponding first category user completes the first interaction task, and send the notification to at least the client of the first category user. Alternatively, the server may accumulate, for each first category user, the contribution data that it obtained in the time period corresponding to the current interaction turn, or may accumulate the contribution data that it obtained in the time period in which the interaction task was performed, or may accumulate the contribution data that it obtained in some preset time period. Additionally, the server may send the notification to clients of the second category of users, and optionally to clients of other first category of users in the first category of users. Thus, optionally, the client of the first category of users may also receive notifications of the completion of the first interactive task by other first category of users and present these notifications. Illustratively, presentation herein may include presentation in various ways, such as by visual elements such as text, pictures, animations, or by non-visual elements such as audio.

In some embodiments, the second category of users may be viewers watching a live broadcast, who may enter an interactive room of any one of the anchor broadcasts to watch the live broadcast, i.e. to obtain a media stream from the corresponding anchor broadcast. The first interactive task may include obtaining contribution data that meets a first contribution data threshold. Optionally, the first interactive task may also include completing other set tasks, such as singing a song, talent show, game play, and the like. The contribution data may include data resulting from the interactive behavior of the second category of users, such as at least one of an amount of approval, an amount of sharing, a number of gifts gifted, a gift value corresponding to the gifted gift, a number of gifted specific gifts, and the like. The specific type of the contribution data and the first contribution data threshold may be preset, may be set by the first category user or the second category user, or may be randomly set.

In some embodiments, the online interaction method 300 further comprises: receiving and presenting a notification that the first category user did not complete the first interactive task, the notification generated in response to the cumulative value of contribution data made by the first category user being less than a first contribution data threshold. This step may be implemented similarly to step 320 and will not be described further herein.

In some embodiments, the first interaction turn may include at least two interaction periods, and the first interaction task is sequentially executed by the first category user and at least one other first category user within the respective interaction periods, wherein the notification that the first category user completed the first interaction task is generated in response to the cumulative value of the contribution data obtained by the first category user within the respective interaction periods being greater than or equal to a first contribution data threshold.

For example, at the beginning of the online interaction, numbers may be assigned to each first-category user in the first-category user groups participating in the interaction, for example, numbers 1 to 5 may be assigned to 5 first-category users participating in the interaction, and optionally, video data thereof may be arranged in order from left to right and from top to bottom according to the numbering order. Subsequently, the first category of users may perform the first interactive task in different time periods in sequence based on the numbering order. The time length of the interaction period can be preset, or set by the first category user or the second category user, or set randomly. In this case, when the cumulative value of the contribution data obtained by the first category user in the corresponding interaction period is greater than or equal to the first contribution data threshold, it may be considered that the first interaction task is completed, and when the cumulative value of the contribution data obtained by the first category user in the corresponding interaction period is less than the first contribution data threshold, it may be considered that the first interaction task is not completed, that is, eliminated in the current round of interaction. Optionally, when a first interactive task is not completed by a certain first-class user, another interactive task can be opened for the certain first-class user. For example, the first category of users may enter a to-be-revived state and open a revival task for them. The interactive task may be the same or different from the first interactive task. If the first category user completes the other interactive task within a specified time period, the first category user may be changed from the deselected state back to the normal state (revival was successful). This also contributes to further increasing the interest of the interaction.

In step 330, the clients of the first category of users may receive a notification from the server to enter a second interaction turn for the online interaction. The notification may be generated by the server in response to at least two first category users of the first category user group completing the first interactive task.

In some embodiments, a second interaction round may be entered when there are at least two anchor completing a first interaction task in a first interaction round. Optionally, similar to the first interactive turn, the second interactive turn may have a corresponding second interactive task, which may be the same as the first interactive task, or may be similar to the first interactive task, i.e. the task type is the same but the corresponding second contribution data threshold is different, or may also be completely different from the first interactive task. Similarly, the first interactive task may be executed by a first category user group, and in particular, by a first category user entering a second interactive turn in the first category user group, which may be a preset task, or a task determined according to a preset rule, or a task set by the first category user or the second category user, or a task set at random.

In some embodiments, in a second interaction turn, a client of a first category user entering the second interaction turn may send task execution data for a second interaction task based on input from the first category user, wherein a second contribution data threshold corresponding to the second interaction task is greater than the first contribution data threshold. For example, the second contribution data threshold may correspond to a higher amount of like, share, gift value, or number of particular gifts, etc. than the first contribution data threshold corresponds to. The second interaction round may be performed similarly to the first interaction round and will not be described in detail herein. Optionally, the online interaction may include several levels of interaction tasks, such as level 10. The level of the interactive tasks may be increased from turn to turn, and when increased to the highest level, may remain at that level until the online interaction is completed, i.e., a certain first-category user wins or all first-category users are eliminated in a certain turn. For example, in the description of the present invention, the first interaction turn may be understood as a current ongoing interaction turn, and the second interaction turn may be understood as a next interaction turn, each of which may have a corresponding interaction task.

In some embodiments, the method 300 further comprises: a notification of the first category user's win is received and presented, the notification generated in response to only the first category user completing the first interactive task. For example, if only one first-class user completes the interaction task in a certain interaction turn, that is, all other first-class users are eliminated, the first-class user wins the online interaction. Alternatively, the corresponding image information indicating the winning may be displayed in the video area of the winning first class of users based on the above-described flow control scheme. In addition, the notification generation, receiving and presenting in this step can be implemented in a similar manner to step 320, and are not described herein again.

In some embodiments, the online interaction may be initiated by a particular first category of users, by any first category of users, or by a second category of users. Illustratively, the client of the first category of users may further perform the following operations: receiving an interaction initiation input from the first category of users, the interaction initiation input being for initiating the online interaction; and responding to the interaction initiation input, and sending a request for starting the online interaction. Or, for example, the client of the first category user may further perform the following operations: receiving a request for starting online interaction; an interaction consent input is received from the first category of users, the interaction consent input indicating that the first category of users consents to open an online interaction.

Illustratively, any first category user in the first category user group (e.g., any anchor who is connecting to wheat) may choose to open an online interaction by entering an interaction initiation input through its client. For example, a first category of users may enter an interaction initiation input via some option button presented by the client, or may enter an interaction initiation input via a text input, or may enter an interaction initiation input via a voice input, or may enter an interaction initiation input via a gesture input, and so on. And, optionally, the first category users may choose to initiate an interaction between specified first category users, or to choose first category users randomly determined by the server to participate in the interaction. After the client of the first class user obtains the interaction initiation input of the first class user, a request for starting online interaction can be sent to the server so as to request for starting online interaction. The server may process the received request and send a request to open an online interaction to at least one first class user specified or randomly determined by the first class user initiating the interaction. The at least one first category user may enter an interaction consent input through their client to indicate consent to open an online interaction. The manner of entry may be the same or different than the manner of entry of the interactive initiation input. After the client obtains the interaction agreement input, a feedback message can be sent to the server to indicate that the first class user agrees to open the online interaction. Optionally, after receiving interaction consent from all specified or randomly determined first class users, the server may determine to open the online interaction, generate a notification of opening the online interaction, and send the notification to the clients of the first class users participating in the online interaction. Alternatively, the server may determine to open the online interaction after receiving the interaction agreement from a predetermined number (e.g., 4, 5, etc.) of the first category users from the specified or randomly determined first category users. In this process, the server may update the number of people who agree to participate in the interaction in real time, and send information including the number of people to the client of the first category user who initiated the online interaction and optionally the client of the first category user who agree to participate in the online interaction and/or the client of the second category user.

FIG. 8 schematically illustrates an example flow diagram of an online interaction method 800 according to further embodiments of the invention. Illustratively, the online interaction method 800 may be implemented by means of the second terminal device 250 of any one of the second class users 240 shown in fig. 2, and the client in the method 800 may be deployed on the corresponding second terminal device 250. As shown, the online interaction method 800 includes the following steps: receiving and presenting task execution data of a first category user for a first interaction task in a first interaction turn of the online interaction, wherein the first interaction task is executed by a first category user group, and the first category user group comprises the first category user and at least one other first category user (step 810); sending contribution data based on input from the second category of users (step 820); receiving and presenting a notification of completion of the first interactive task by a first category of users, the notification generated in response to a cumulative value of contribution data made by the first category of users being greater than or equal to a first contribution data threshold (step 830); a notification of entry into a second turn of the online interaction is received and presented, the notification generated in response to completion of the first interaction task by at least two first category users of the first category user group (step 840).

At step 810, the client of the second category of users may receive task execution data for the first interactive task of the first category of users from the server. For example, a second category of users may use their clients to enter an interaction room of any first category of users participating in an online interaction and receive task execution data from the clients of the first category of users from a server. The task performance data may be generated at the clients of the first category of users as described with respect to step 310 of fig. 3. Alternatively, the server may additionally process the task execution data sent by the client of the first category user to serve as new task execution data and push the new task execution data to the client of the second category user. This additional processing will be described in detail below.

In some embodiments, the task performance data of the first category of users may include mixed-flow video data, which may be generated as described with respect to step 310 of fig. 3. In this case, receiving and presenting task execution data for the first interactive task for the first category of users includes: receiving mixed flow video data; video data from at least two first category users is presented based on the mixed-flow video data. For example, a client of a second category of users may receive mixed-stream video data from a server from a first category of users. Alternatively, the server may perform appropriate additional processing on the mixed-stream video data sent by the clients of the first class of users, which will be described in detail below. Subsequently, the clients of the second category of users may present video data of at least two first category of users based on the received mixed-flow video data.

In some embodiments, the mixed stream video data may include video encoded data and stream control data, and presenting video data from at least two first category users includes: decoding the video encoded data; presenting video data of each first category user in different regions based on the decoded video data; different image information is presented in different areas based on control information in the flow control data, wherein the control information is written based on the progress of the online interaction. For example, the clients of the second category of users may decode the received video encoding data based on a protocol or scheme corresponding to a protocol or scheme of the clients of the first category of users encoding the mixed stream video data. Subsequently, the clients of the second category of users may present the video data of the respective first category of users in different areas of the video presentation area based on the decoded video data. Meanwhile, the client of the second category of users can read the control information in the flow control data, and present corresponding image information in different areas (namely areas where the video data of different first category of users are located) based on the control information to show the process of online interaction. Such control information may be generated as described with respect to step 310 of fig. 3.

Further, the task execution data may also include mixed flow audio data, which may be generated as described according to step 310 of fig. 3, and the clients of the second category of users may play audio based on the received mixed flow audio data.

At step 820, the client of the second category of users may obtain input from the second category of users and send contribution data to the server based on the input. For example, the second category of users may provide such input through option buttons provided by the client, or may also provide such input through speech, text, gestures, and so forth. For example, in a live scene, the second category of users may provide such contribution data by selecting a gifted gift (either any gift or a particular gift) or by agreeing, sharing, leaving a message, etc., while watching the live. The manner in which the contribution data is provided may be pre-specified, such as being set by the first category of users or the second category of users, randomly set, or pre-set by the system, etc.

In some embodiments, the first interaction turn comprises at least two interaction periods, and the first interaction task is sequentially executed by the first category user and at least one other first category user within the respective interaction periods, wherein the notification that the first category user completed the first interaction task is generated in response to the cumulative value of the contribution data obtained by the first category user within the respective interaction periods being greater than or equal to a first contribution data threshold. This is described in detail with respect to method 300 of fig. 3 and is not described in detail herein.

In some embodiments, the input of the second category of users comprises input indicating that a special effect is achieved, the method further comprising: in response to receiving an input indicating a special effect, presenting a notification related to at least one of the following special effects: increasing the time length of the current interaction time period; decreasing a first contribution data threshold associated with a current interaction period; the order in which the first interactive tasks are opened for the at least two first category users is updated. Illustratively, users of the second category may provide input via their clients indicating that a special effect is achieved. For example, in a live scene, the viewer may select, via their client, to gift a special effect that may achieve at least one of the effects of exchanging anchor locations, increasing interactive task execution times, and reducing interactive task difficulty. By setting the input with special effect, the interest of online interaction can be further increased, the participation sense of the users of the second category on the online interaction is increased, and the interaction willingness is improved.

9A-C schematically illustrate example interface diagrams for gifting a gift having a special effect in a live scene, according to some embodiments of the invention. These interfaces may be presented at the clients of the second category of users. Clients of the first category of users may also have similarly presented interfaces.

For gifts implementing exchanging anchor locations, as shown in interface 900A in fig. 9A, a notification may be displayed indicating that the anchor gifted the gift exchanges locations (i.e., exchange numbers) with any anchor that is not eliminated, e.g., the notification shown indicates that anchor numbers 4 and 3 exchange locations. For example, in a certain interaction turn, if the anchor No. 1-4 has already executed an interaction task, and now No. 5 is executing the task, and the gifted gift exchanges location with No. 4, after the anchor executes the task, the new anchor No. 5 (original No. 4) needs to accept the task again. Or, for example, in a certain interaction turn, if no interactive task is executed by any of the anchor masters No. 2-5, the anchor master No. 1 is executing an interactive task, and a gift is given to exchange a position with the anchor master No. 3, the original anchor master No. 3 (new anchor master No. 1) may not execute the interactive task in the turn, and the original anchor master No. 1 as the new anchor master No. 3 needs to execute the interactive task again after the anchor master No. 2 finishes executing the interactive task. The above-described exchange mechanism is merely exemplary, and different order update mechanisms may be set as necessary. Alternatively, the use condition of the gift may be set to be used only for the anchor that is not eliminated, and optionally, the gift may be used only once for one anchor in each interaction turn. When the viewer attempts to gift the corresponding gift without complying with the use condition, a notification may be displayed indicating that the gift is not currently giftable, and optionally, a condition under which the gift can be gifted.

For gifts that increase the execution time of the interactive task, as shown in interface 900B of fig. 9B, the period for which the anchor gifted the gift performs the interactive task may be extended, for example, 30 seconds as shown in the figure. Alternatively, the use condition of the gift may be set to be used only for the anchor who is performing the interactive task. When the viewer attempts to gift the corresponding gift without complying with the use condition, a notification may be displayed indicating that the gift is not currently giftable, and optionally, a condition under which the gift can be gifted.

For gifts with reduced interactive task difficulty, as shown in interface 900C of fig. 9C, the interactive task difficulty performed by the anchor gifted the gift is reduced, for example, by one step as shown in the figure, or the contribution data threshold may be reduced by a predetermined value, etc. Alternatively, the use condition of the gift may be set to be used only for the anchor who is performing the interactive task. When the viewer attempts to gift the corresponding gift without complying with the use condition, a notification may be displayed indicating that the gift is not currently giftable, and optionally, a condition under which the gift can be gifted.

In step 830, the client of the second category of users may receive a notification from the server that the first category of users completed the first interaction task, and present the notification in the form of text, pictures, animations, sounds, etc. The notification may be generated as described with respect to step 320 of fig. 3 and will not be repeated here. In addition, the client of the second category of users may also receive a notification from the server that the first category of users did not complete the first interaction task, and present the notification in the form of text, pictures, animations, sounds, etc., and the notification may also be generated as described with reference to fig. 3, and will not be described again here. Illustratively, FIG. 9D schematically illustrates an example interface diagram 900D for performing interactive tasks in a live scene, in accordance with some embodiments of the present invention. As shown in fig. 9D, currently, anchor No. 3 is executing an interactive task, anchor No. 2 is in a state to be revived, and anchor No. 5 is obsolete. In the example shown in fig. 9D, the current interactive task is 5 gifts named "to embrace". The inactive state and the deselect state are described in the description with respect to fig. 3 and will not be described in detail here.

In step 840, the clients of the second category of users may receive a notification from the server of a second interaction turn into the online interaction and present the notification in text, pictures, animations, sounds, etc. The notification may be generated as described with respect to step 330 of fig. 3 and will not be repeated here.

In some embodiments, the online interaction method 800 further comprises: presenting a resource package, the resource package generated in response to entering the second interaction turn; sending a request for acquiring the resource package based on the input of the second category user request for acquiring the resource package; in response to receiving the notification of successful acquisition of the resource package, resources in the resource package are presented. Illustratively, upon entering a second interactive round (or next interactive round), the server may generate a rewarding resource package and send a notification to the clients of the second category of users indicating that participation in contending for the resource package is possible. Alternatively, the server may send such a notification to the clients of the second category of users in the interaction room of the first category of users who successfully entered the second interaction turn. The clients of the second category of users may present the notification in any manner and provide the second category of users with a physical or virtual interface that participates in contending for the resource package. For example, as shown in interface diagram 900E in FIG. 9E, clients of users of the second category (e.g., viewers) may present a treasure box image. The second category of users may participate in contending for the resource package by clicking on the image. The clients of the second category of users may send respective requests to the server based on the second category of users participating in the input contending for the resource package. The server may determine, based on the time at which the request is received, randomly or according to other preset rules, second category users who successfully acquire resources in the resource package, and send corresponding notification messages to the clients of the second category users, and add corresponding resources to accounts of the second category users at the same time. Clients of respective second category users may, in response to receiving such notification messages, present the user with a notification of successful acquisition of the resource in any manner. By allowing the second category users to participate in contending for the resource packet, interactive feedback can be provided for the second category users, the interactive activity of the second category users is improved, and the conflict caused by the fact that feedback cannot be obtained all the time is avoided.

Further, similar to that described with respect to fig. 3, the method 800 may further include: a notification of a first category user winning is received and presented, the notification generated in response to only the first category user completing the first interactive task. FIG. 9F schematically illustrates an example interface 900F that presents results of an online interaction. As shown in fig. 9F, when the online interaction proceeds to level 9, anchor No. 1 wins. Optionally, a tally may be added to the winning anchor based on the results of the online interaction, such as "30 points" as shown in interface 900F, which may be used to rank the anchors on the live platform. This is beneficial for motivating the initiative of the anchor to participate in online interactions.

Optionally, an interactive warm-up period may also be included, as described with respect to fig. 3 or 8, with fig. 9G schematically illustrating an example interface 900G for the warm-up period. The time period may be set after the online interaction is turned on. During this time period, users of the first category participating in the interaction may communicate to be familiar with each other, which facilitates smooth progress of the online interaction.

FIG. 10 schematically illustrates an example flow diagram of an online interaction method 1000 in accordance with further embodiments of the invention. Illustratively, the online interaction method 1000 may be implemented by means of the server 230 shown in fig. 2, and the client in the method 1000 may be deployed on the server 230. As shown, the online interaction method 1000 includes the following steps: in a first interaction turn of the online interaction, sending a notification for starting a first interaction task, wherein the first interaction task is executed by a first category user group, and the first category user group comprises at least two first category users (step 1010); receiving task execution data for the first interactive task from a client of each first category user in a first category user group (step 1020); sending task execution data for the first category of users to the second category of users (step 1030); receiving contribution data from clients of users of the second category (step 1040); accumulating the resulting contribution data for each first category of users (step 1050); in response to the cumulative value of the contribution data obtained by at least one first category user in the first category user group being greater than or equal to the first contribution data threshold, generating and sending a notification that the at least one first category user completed the first interactive task (step 1060); in response to at least two first category users of the first category user group completing the first interaction task, a notification of a second interaction turn to enter the online interaction is generated and sent (step 1070). These steps are already introduced in the description of fig. 3 and 8 and will not be described again here.

In some embodiments, the server may additionally process the received task execution data from the first category of user clients and send the processed task execution data to the second category of user clients. Optionally, the method 1000 further comprises: changing a code rate of the received task execution data to obtain at least two task execution data with different code rates, wherein the task execution data comprises at least one of mixed stream video data and mixed stream audio data, and wherein sending the task execution data of the first category of users to the second category of users comprises: and selectively sending task execution data with corresponding code rate to the second category of users based on network conditions or code rate selection requests from the clients of the second category of users. For example, upon receiving at least one of mixed stream video data and mixed stream audio data from a client of a first category of users, the server may generate data of different bitrate based on the received data. Optionally, when the network status is good, the server may send data with a higher code rate to the clients of the second category of users, so as to achieve a better presentation effect at the clients of the second category of users; when the network state is poor, the server can send data with a lower code rate to the client of the second class user, so that a higher transmission speed is realized, and the phenomenon that the client of the second class user is blocked is avoided. Alternatively, the second category user may select the bit rate through its client, and its client may send a bit rate selection request to the server based on the selection of the second category user, and the server may send data of the corresponding bit rate to the client based on the request. This allows the second category of users to autonomously select the code rate of the received data to meet the diversified user requirements. Furthermore, optionally, the server may also perform other additional processing on the received task execution data, such as writing control information related to other users of the first category and/or control information related to the overall online interaction process (e.g., a cut scene about entering a next interaction turn, etc.) in the flow control data of the mixed-flow video data, and so on.

FIG. 11 schematically illustrates an example flow diagram of an online interaction method 1100 in accordance with further embodiments of the invention. As shown in fig. 11, method 1100 includes step 1030 and

optional steps

1010, 1020, 1040. At step 1010, at least two first category users may be matched and online interaction initiated. The matching process may be implemented as described with respect to fig. 3 with respect to how online interactions are initiated by a first category of users. At step 1020, a warm-up time (e.g., 90 seconds) may be entered, which is described in the context of FIG. 8. In step 1030, formally entering the competition time among a plurality of first-class users, wherein in each round, each first-class user can execute an interactive task within a preset time period (for example, 3 minutes), and if a certain first-class user finishes the interactive task, entering the next round of interaction; if the first class of users do not complete the interaction task, the first class of users are eliminated. Optionally, there may be a revival mechanism as described previously. Alternatively, the difficulty of the interactive task may increase from turn to turn until the highest difficulty is reached. When all the users of the first category are eliminated in a certain turn or only one user of the first category completes the interaction task (the user of the first category wins), the competition stage of the online interaction is finished. At step 1040, a post-match communication time may be entered and the first category of users participating in the current online interaction may communicate, which may have a predetermined time, such as 90 seconds as shown in fig. 11. And then the online interaction is finished. Various operations related to the above steps may be performed according to the steps described with respect to fig. 3, 8, and 10, and are not described in detail herein.

It should be understood that various features in the online interaction methods described above with respect to fig. 3, 8, 10 may be corresponding, and that various features in the methods implemented at one of the clients of the first category of users, the clients of the second category of users, the server may also have corresponding implementations at the other two thereof.

It should also be understood that the various steps described above and shown in the flowcharts need not be performed in the steps described or shown. Some of the steps may be performed in parallel, or in reverse order of that described or illustrated, as the case may be.

Fig. 12 schematically illustrates an example block diagram of a client 1200 in accordance with some embodiments of the invention. As shown, the client 1200 includes a sending module 1210, a first rendering module 1220, and a second rendering module 1230.

In particular, the sending module 1210 may be configured to send task execution data for a first interactive task based on input from a first category of users in a first interaction turn of the online interaction, the first interactive task being executed by a first category of user groups, the first category of user groups including the first category of users and at least one other first category of users; the first presentation module 1220 may be configured to receive and present a notification of completion of a first interactive task by the first category of users, the notification generated in response to a cumulative value of contribution data made by the first category of users being greater than or equal to a first contribution data threshold, wherein the contribution data was provided by the second category of users; the second presentation module 1230 can be configured to receive and present a notification of a second interaction turn into the online interaction, the notification generated in response to at least two first category users of the first category user group completing the first interaction task.

Fig. 13 schematically illustrates an example block diagram of a client 1300 in accordance with further embodiments of the invention. As shown, the client 1300 includes a first rendering module 1310, a sending module 1320, a second rendering module 1330, and a third rendering module 1340.

In particular, the first presentation module 1310 may be configured to receive and present task execution data for a first interactive task for a first category user in a first interaction turn of the online interaction, the first interactive task being executed by a first category user group, the first category user group including the first category user and at least one other first category user; the sending module 1320 may be configured to send the contribution data based on input from the second category of users; the second presentation module 1330 may be configured to receive and present a notification of completion of the first interactive task by a first category of users generated in response to the first category of users having a cumulative value of contribution data greater than or equal to a first contribution data threshold; the third presentation module 1340 may be configured to receive and present a notification of entry into a second interaction turn of the online interaction, the notification being generated in response to at least two first category users of the first category user group completing the first interaction task.

Fig. 14 schematically illustrates an example block diagram of a server 1400 in accordance with some embodiments of the invention. As shown, the server 1400 includes a first transmitting module 1410, a first receiving module 1420, a second transmitting module 1430, a second receiving module 1440, an accumulating module 1450, a third transmitting module 1460, and a fourth transmitting module 1470.

Specifically, the first sending module 1410 may be configured to send a notification to start a first interactive task in a first interactive turn of the online interaction, the first interactive task being executed by a first category user group, the first category user group including at least two first category users; the first receiving module 1420 may be configured to receive task execution data for the first interactive task from the client of each first category user in the first category user group; the second transmitting module 1430 may be configured to transmit the task execution data of the first category user to the second category user; the second receiving module 1440 may be configured to receive contribution data from clients of users of the second category; the accumulation module 1450 may be configured to accumulate the resulting contribution data for each first category of user; the third sending module 1460 may be configured to generate and send a notification that at least one first category user completes the first interactive task in response to an accumulated value of contribution data obtained by the at least one first category user in the first category user group being greater than or equal to a first contribution data threshold; the fourth sending module 1470 may be configured to generate and send a notification to enter a second interaction turn for the online interaction in response to at least two first category users of the first category user group completing the first interaction task.

Client 1200 may be deployed on first terminal device 220 shown in fig. 2, client 1300 may be deployed on second terminal device 250 shown in fig. 2, and server 1400 may be deployed on server 230 shown in fig. 2. It is to be understood that client 1200, client 1300, and server 1400 can be implemented in software, hardware, or a combination of software and hardware. Several different modules may be implemented in the same software or hardware configuration, or one module may be implemented by several different software or hardware configurations.

In addition, the client 1200, the client 1300 and the server 1400 can be used for implementing the online interaction method described above, and the relevant details thereof have been described in detail above and will not be repeated here for the sake of brevity. Client 1200, client 1300, and server 1400 may have the same features and advantages as described with respect to the aforementioned online interaction method.

FIG. 15 schematically illustrates an example block diagram of a computing device in accordance with some embodiments of this invention. Which may represent, for example, the first terminal device 220, the second terminal device 250, or the server 230 in fig. 2.

Computing device 1500 may be, for example, a server, a desktop computer, a laptop computer, a tablet, a smartphone, a smartwatch, a wearable device, or any other suitable computing device or computing system, which may include various levels of devices ranging from full resource devices with substantial storage and processing resources to low-resource devices with limited storage and/or processing resources.

As shown, the example computing device 1500 includes a processing system 1501, one or more computer-readable media 1502, and one or more I/O interfaces 1503 communicatively coupled to each other. Although not shown, the computing device 1500 may also include a system bus or other data and command transfer system that couples the various components to one another. A system bus can include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or local bus that utilizes any of a variety of bus architectures, or that also includes data lines, such as control and data lines.

Processing system 1501 represents functionality that uses hardware to perform one or more operations. Thus, the processing system 1501 is illustrated as including hardware elements 1504 that may be configured as processors, functional blocks, and the like. This may include implementing an application specific integrated circuit or other logic device formed using one or more semiconductors in hardware. The hardware elements 1504 are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, a processor may be comprised of semiconductor(s) and/or transistors (e.g., electronic Integrated Circuits (ICs)). In such a context, processor-executable instructions may be electronically-executable instructions.

The computer-readable medium 1502 is illustrated as including a memory/storage device 1505. Memory/storage 1505 represents memory/storage associated with one or more computer-readable media. Memory/storage 1505 may include volatile storage media (such as Random Access Memory (RAM)) and/or nonvolatile storage media (such as Read Only Memory (ROM), flash memory, optical disks, magnetic disks, and so forth). Memory/storage 1505 may include fixed media (e.g., RAM, ROM, a fixed hard drive, etc.) as well as removable media (e.g., flash memory, a removable hard drive, an optical disk, and so forth). For example, the memory/storage device 1505 may be used for storing various audio-video data, online interactive process data, and other data generated during the execution of the method, etc. as mentioned above. The computer-readable medium 1502 may be configured in various other ways as further described below.

One or more input/output interfaces 1503 are representative of functionality that allows a user to enter commands and information to computing device 1500, and also allows information to be presented to the user and/or sent to other components or devices using various input/output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone (e.g., for voice input), a scanner, touch functionality (e.g., capacitive or other sensors configured to detect physical touch), a camera (e.g., motion that does not involve touch may be detected as gestures using visible or invisible wavelengths such as infrared frequencies), a network card, a receiver, and so forth. Examples of output devices include a display device (e.g., a monitor or projector), speakers, a printer, a haptic response device, a network card, a transmitter, and so forth. Exemplarily, in the above-described embodiment, the first category user 210 may enter audio and/or video input and perform operations such as initiating an online interaction, receiving an online interaction invitation, and the like through the input interface of the first terminal device 220, and may view various notifications and the like sent by the server through the output interface of the first terminal device 220, and the second category user 240 may perform operations of providing contribution data and the like through the input interface of the second terminal device 250, and may view task execution data and the like of the first category user through the output interface of the second terminal device 250.

Computing device 1500 also includes online interaction policy 1506. Online interaction policy 1506 may be stored as computer program instructions in memory/storage 1505. Online interaction policy 1506 may implement, along with processing system 1501, etc., all of the functionality of the various modules of client 1200, client 1300, or server 1400 described with respect to fig. 12, 13, or 14.

Various techniques may be described herein in the general context of software, hardware, elements, or program modules. Generally, these modules include routines, programs, objects, elements, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The terms "module," "functionality," and the like, as used herein generally represent software, firmware, hardware, or a combination thereof. The features of the techniques described herein are platform-independent, meaning that the techniques may be implemented on a variety of computing platforms having a variety of processors.

An implementation of the described modules and techniques may be stored on or transmitted across some form of computer readable media. Computer readable media can include a variety of media that can be accessed by computing device 1500. By way of example, and not limitation, computer-readable media may comprise "computer-readable storage media" and "computer-readable signal media".

"computer-readable storage medium" refers to a medium and/or device, and/or a tangible storage apparatus, capable of persistently storing information, as opposed to mere signal transmission, carrier wave, or signal per se. Accordingly, computer-readable storage media refers to non-signal bearing media. Computer-readable storage media include hardware such as volatile and nonvolatile, removable and non-removable media and/or storage devices implemented in a method or technology suitable for storage of information such as computer-readable instructions, data structures, program modules, logic elements/circuits or other data. Examples of computer readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical storage, hard disks, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other storage devices, tangible media, or an article of manufacture suitable for storing the desired information and accessible by a computer.

"computer-readable signal medium" refers to a signal-bearing medium configured to transmit instructions to the hardware of computing device 1500, such as via a network. Signal media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave, data signal or other transport mechanism. Signal media also includes any information delivery media. By way of example, and not limitation, signal media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media.

As previously described, the hardware element 1501 and the computer-readable medium 1502 represent instructions, modules, programmable device logic, and/or fixed device logic implemented in hardware, which in some embodiments may be used to implement at least some aspects of the techniques described herein. The hardware elements may include integrated circuits or systems-on-chips, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), Complex Programmable Logic Devices (CPLDs), and other implementations in silicon or components of other hardware devices. In this context, a hardware element may serve as a processing device that performs program tasks defined by instructions, modules, and/or logic embodied by the hardware element, as well as a hardware device for storing instructions for execution, such as the computer-readable storage medium described previously.

Combinations of the foregoing may also be used to implement the various techniques and modules described herein. Thus, software, hardware, or program modules and other program modules may be implemented as one or more instructions and/or logic embodied on some form of computer-readable storage medium and/or by one or more hardware components 1501. Computing device 1500 may be configured to implement particular instructions and/or functions corresponding to software and/or hardware modules. Thus, for example, by using a computer-readable storage medium of a processing system and/or hardware components 1501, modules may be implemented at least partially in hardware as modules executable by computing device 1500 as software. The instructions and/or functions may be executed/operable by, for example, one or more computing devices 1500 and/or processing systems 1501 to implement the techniques, modules, and examples described herein.

The techniques described herein may be supported by these various configurations of computing device 1500 and are not limited to specific examples of the techniques described herein.

It will be appreciated that embodiments of the disclosure have been described with reference to different functional units for clarity. However, it will be apparent that the functionality of each functional unit may be implemented in a single unit, in a plurality of units or as part of other functional units without departing from the disclosure. For example, functionality illustrated to be performed by a single unit may be performed by a plurality of different units. Thus, references to specific functional units are only to be seen as references to suitable units for providing the described functionality rather than indicative of a strict logical or physical structure or organization. Thus, the present disclosure may be implemented in a single unit or may be physically and functionally distributed between different units and circuits.

It will be further understood that, although the terms first, second, etc. may be used herein to describe various devices, elements, components or sections, these devices, elements, components or sections should not be limited by these terms. These terms are only used to distinguish one device, element, component or section from another device, element, component or section.

Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed subject matter, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. An online interaction method for a client, comprising:

in a first interaction turn of the online interaction, sending task execution data for a first interaction task based on input from a first category user, wherein the first interaction task is executed by a first category user group, and the first category user group comprises the first category user and at least one other first category user;

receiving and presenting a notification that the first category user completed a first interactive task, the notification being generated in response to an accumulated value of contribution data obtained by the first category user being greater than or equal to a first contribution data threshold, wherein the contribution data is provided by a second category user, and the first interactive task is eliminated in the first interactive turn without completing the first interactive task when the accumulated value of the contribution data obtained by the first category user is less than the first contribution data threshold;

receiving and presenting a notification of a second interaction turn entering the online interaction, the notification being generated in response to at least two first category users of the first category user groups completing a first interaction task;

wherein the first interaction turn comprises at least two interaction periods, the first interaction task being performed in turn by the first category user and at least one other first category user within the respective interaction periods, and wherein the input of the second category user comprises an input indicating that a special effect is achieved, and in response to receiving the input indicating a special effect, presenting a notification relating to at least one of the following special effects: increasing the time length of the current interaction time period; decreasing a first contribution data threshold associated with a current interaction period; the order in which the first interactive tasks are opened for the at least two first category users is updated.

2. The method of claim 1, wherein the sending task execution data for the first interactive task based on input from a first category of users comprises:

acquiring video data of the first category of users;

receiving video data from other first category users in the first category user group;

mixed-flow video data is generated as at least part of the task execution data based on the video data of the first category user and the video data of the other first category users.

3. The method of claim 2, wherein the mixed flow video data comprises video encoded data and flow control data, and wherein the generating mixed flow video data comprises:

laying out the video data of the first class user and the video data of other first class users in different areas;

encoding the video data after layout to generate the video encoding data;

control information is written in the flow control data based on a progress of the online interaction, the control information indicating that different image information is presented in different areas.

4. The method of claim 3, wherein the process based on online interaction writing control information in the flow control data comprises:

and in response to that the first class user does not complete the first interaction task, identifying a face region corresponding to the first class user, and writing control information indicating that punishment information is presented in the face region into the flow control data.

5. An online interaction method for a client, comprising:

receiving and presenting task execution data of a first category user aiming at a first interaction task in a first interaction turn of the online interaction, wherein the first interaction task is executed by a first category user group, and the first category user group comprises the first category user and at least one other first category user;

transmitting contribution data based on input from a second category of users;

receiving and presenting a notification that a first category user completes a first interactive task, the notification being generated in response to an accumulated value of contribution data obtained by the first category user being greater than or equal to a first contribution data threshold, the notification being eliminated in the first interactive turn without completing the first interactive task when the accumulated value of the contribution data obtained by the first category user is less than the first contribution data threshold;

6. The method of claim 5, further comprising:

presenting a resource package, the resource package generated in response to entering a second interaction turn;

sending a request for acquiring the resource package based on the input of the second category user request for acquiring the resource package;

and presenting the resources in the resource package in response to receiving the notification of successful acquisition of the resource package.

7. The method of claim 5, wherein the receiving and presenting task execution data for a first interactive task for a first category of users comprises:

receiving mixed flow video data, wherein the mixed flow video data comprises video coding data and flow control data;

decoding the video coding data;

presenting video data of each first category user in different regions based on the decoded video data;

presenting different image information in different areas based on control information in the flow control data, wherein the control information is written based on a progress of an online interaction.

8. An online interaction method for a server, comprising:

sending a notification for starting a first interactive task in a first interactive turn of the online interaction, wherein the first interactive task is executed by a first category user group, and the first category user group comprises at least two first category users;

receiving task execution data for the first interactive task from a client of each first category user in a first category user group;

sending task execution data of the first category of users to the second category of users;

receiving contribution data from clients of users of a second category;

accumulating the resulting contribution data for each first category of users;

generating and sending a notification that at least one first category user in the first category user group completes a first interaction task in response to an accumulated value of contribution data obtained by the at least one first category user being greater than or equal to a first contribution data threshold; when the accumulated value of the contribution data obtained by the first category user is smaller than the first contribution data threshold value, the first interaction task is not completed and is eliminated in the first interaction turn;

responding to at least two first-class users in the first-class user groups to finish a first interaction task, and generating and sending a notification of a second interaction turn entering the online interaction;

9. The method of claim 8, further comprising:

changing the code rate of the received task execution data to obtain at least two task execution data with different code rates, wherein the task execution data comprises at least one of mixed flow video data and mixed flow audio data,

and wherein said sending task execution data for the first category of users to the second category of users comprises:

and selectively sending task execution data with corresponding code rate to the second category of users based on network conditions or code rate selection requests from the clients of the second category of users.

10. A client, comprising:

a sending module configured to send task execution data for a first interactive task based on input from a first category of users in a first interactive round of online interaction, the first interactive task being executed by a first category of user groups, the first category of user groups including the first category of users and at least one other first category of users;

a first presentation module configured to receive and present a notification that the first category user completed a first interactive task, the notification generated in response to a cumulative value of contribution data obtained by the first category user being greater than or equal to a first contribution data threshold, wherein contribution data is provided by a second category user, and the first interactive task is eliminated in the first interactive turn without completing the first interactive task when the cumulative value of contribution data obtained by the first category user is less than the first contribution data threshold;

a second presentation module configured to receive and present notifications of entry into a second interaction turn of the online interaction, the notifications generated in response to at least two first category users of the first category user groups completing a first interaction task,

11. A client, comprising:

the first presentation module is configured to receive and present task execution data of a first category user for a first interaction task in a first interaction turn of online interaction, wherein the first interaction task is executed by a first category user group, and the first category user group comprises the first category user and at least one other first category user;

a sending module configured to send contribution data based on input from a second category of users;

a second presentation module configured to receive and present a notification that a first category user completed a first interactive task, the notification being generated in response to a cumulative value of contribution data obtained by the first category user being greater than or equal to a first contribution data threshold, the first category user being eliminated in the first interactive turn without completing the first interactive task when the cumulative value of contribution data obtained by the first category user is less than the first contribution data threshold;

a third presentation module configured to receive and present notifications of entry into a second interaction turn of the online interaction, the notifications generated in response to at least two first category users of the first category user groups completing a first interaction task,

12. A server, comprising:

the first sending module is configured to send a notification for starting a first interactive task in a first interactive turn of online interaction, wherein the first interactive task is executed by a first category user group, and the first category user group comprises at least two first category users;

a first receiving module configured to receive task execution data for the first interactive task from a client of each first category user in a first category user group;

a second sending module configured to send the task execution data of the first category user to the second category user;

a second receiving module configured to receive contribution data from clients of users of a second category;

an accumulation module configured to accumulate the resulting contribution data for each first category of users;

a third sending module, configured to generate and send a notification that at least one first category user in the first category user group completes a first interactive task in response to an accumulated value of contribution data obtained by the at least one first category user being greater than or equal to a first contribution data threshold; when the accumulated value of the contribution data obtained by the first category user is smaller than the first contribution data threshold value, the first interaction task is not completed and is eliminated in the first interaction turn;

a fourth sending module configured to generate and send a notification of a second turn of interaction into the online interaction in response to at least two first category users of the first category user groups completing the first interaction task,

13. A computing device comprising a memory and a processor, the memory configured to store thereon computer-executable instructions that, when executed on the processor, perform the method of any of claims 1-9.

14. A computer-readable storage medium having stored thereon computer-executable instructions that, when executed on a processor, perform the method of any of claims 1-9.