CN114222190B

CN114222190B - Remote control processing and response method and device, equipment, medium and product thereof

Info

Publication number: CN114222190B
Application number: CN202111531921.7A
Authority: CN
Inventors: 巫金生
Original assignee: Guangzhou Huaduo Network Technology Co Ltd
Current assignee: Guangzhou Huaduo Network Technology Co Ltd
Priority date: 2021-12-14
Filing date: 2021-12-14
Publication date: 2023-10-17
Anticipated expiration: 2041-12-14
Also published as: CN114222190A

Abstract

The application discloses a remote control processing and responding method for a living room, a device, equipment, a medium and a product thereof, wherein the method comprises the following steps: pushing a control application instruction to a target user terminal in a live broadcasting room; receiving a video recording stream pushed by a target user side in response to a control application instruction, outputting the video recording stream to a control area of a live broadcasting room for playing, and enabling the video recording stream to be responsible for forwarding and pushing by a media server; monitoring an operation event acting on a control area of the live broadcasting room in real time, and generating a corresponding operation message according to the operation event; pushing the operation message to the target user terminal, so that the target user terminal analyzes the operation message, executes the operation instruction in the operation message in the position corresponding to the operation position information in the live broadcasting room interface, controls the currently-performed live broadcasting room online service, and the operation message is responsible for forwarding pushing by the communication server. The application provides remote assistance service for users of the live broadcast platform, and video streams and instructions are distributed, forwarded and pushed by the corresponding servers to realize data synchronization.

Description

Remote control processing and response method and device, equipment, medium and product thereof

Technical Field

The application relates to the field of network live broadcasting, in particular to a method for processing and responding to remote control of a live broadcasting room, and further relates to a device, equipment, a nonvolatile storage medium and a computer program product corresponding to the method.

Background

With the development of network live broadcast, the network live broadcast is more and more popular and is favored by users, a live broadcast platform provides live broadcast room service for platform users, a live broadcast room is created in the network live broadcast platform by a host user in the platform, and audience users of the platform enter the live broadcast room to participate in the live broadcast business started by the host user, so that the audience users watch content such as talent performance, game live broadcast explanation and the like of the host broadcast in the live broadcast room, communication among users is greatly facilitated, and social progress is promoted.

However, when the user in the live network platform uses the application program of the live network platform in daily use, a few problems are often encountered, for example, a new user cannot use the corresponding live network function by hands due to insufficient guidance provided by the platform, or interaction behaviors of audience users in the live network platform cannot be maintained when live broadcast service is busy by a host user in the platform, or some errors which cannot be repaired by the user can occasionally occur in the live network application program, so that the equipment where the user is located cannot normally operate the application program, the user of the platform is helpless when facing the problems, and only the problems encountered by the user of the platform can be fed back to the platform to prompt the platform to provide corresponding help, the waiting time is long, and the user often abandons the application program using the live network platform, so that the user of the platform runs away.

In view of the above-described problems, the present inventors have made a corresponding search for consideration of the problems.

Disclosure of Invention

The present application aims to meet the needs of the prior art or overcome at least part of the deficiencies of the prior art to provide a method for processing remote control of a living room and corresponding apparatus, electronic device, non-volatile storage medium and computer program product.

In order to achieve the purpose of the application, the following technical scheme is adopted:

the application provides a remote control processing method of a live broadcasting room, which is suitable for the purpose of a home page, and comprises the following steps:

pushing a control application instruction to a target user terminal in a live broadcasting room;

receiving a video recording stream formed by recording a live broadcasting room interface of the target user side after the target user side responds to the control application instruction, outputting the video recording stream to a live broadcasting room control area for playing, and forwarding and pushing the video recording stream to a current control side by a media server;

monitoring an operation event acting on the live broadcasting room control area in real time, and generating a corresponding operation message according to the operation event, wherein the operation message comprises an operation instruction which can be analyzed to act on a live broadcasting room interface of the target user terminal and operation position information carried by the operation instruction according to requirements;

And pushing the operation message to the target user side so that after the target user side analyzes the operation message, executing an operation instruction in the operation message on a live broadcasting room interface of the target user side, acting on a corresponding position pointed by operation position information according to the requirement to realize control of the currently-performed live broadcasting room online service, and forwarding and pushing the operation message to the client side by a communication server.

In a further embodiment, the method comprises the following pre-steps:

receiving a help-seeking notification pushed by a user side in a live broadcasting room, and displaying a notification window in a current graphical user interface to display user characteristic information and live broadcasting room characteristic information contained in the help-seeking notification in the notification window;

and responding to the help-seeking acceptance instruction acted on the notification window, and generating a control application instruction pointing to the user located in the live broadcasting room.

In a further embodiment, the step of receiving a video recording stream formed by the target user side on the live broadcasting room interface of the target user side after the target user side responds to the control application instruction, and outputting the video recording stream to the live broadcasting room control area for playing includes the following steps:

receiving a video recording stream recorded by a target user terminal forwarded and pushed by a media server;

Outputting the video stream of the screen recording to a live broadcasting room control area of a current graphical user interface for playing;

and displaying an operation instruction set control in the current graphical user interface, wherein the operation instruction set control comprises a pause control instruction, a continuous control instruction and an end control instruction which cannot be generated through the live broadcasting room control area.

In a further embodiment, the step of monitoring, in real time, an operation event acting on the live broadcasting room control area, and generating a corresponding operation message according to the operation event, where the operation message includes an operation instruction that can be parsed into an interface acting on the target user side of the live broadcasting room and operation position information carried by the operation instruction as required, includes the following steps:

monitoring an operation event triggered by a current user through the live broadcasting room control area in real time;

responding to the operation event, and acquiring an operation instruction currently executed by a current user in the live broadcasting room control area and the operation position information corresponding to the operation instruction in the live broadcasting room control area;

and packaging the operation instruction and the corresponding operation position information thereof to generate the operation message.

In a further embodiment, the operation event triggered by the current user through the live room control area includes any one of the following: click operation event, long press operation event, drag operation event, slide operation event, and touch operation event.

In a further embodiment, the live room online service is any one of the following: live house audience management service, live house play service, live house error reporting service.

A remote control processing device for a living room according to the first object of the present application comprises:

the application instruction pushing module is used for pushing a control application instruction to a target user end in the live broadcasting room;

the video recording and playing module is used for receiving a video recording stream formed by a live broadcasting room interface of the target user side after the target user side responds to the control application instruction, outputting the video recording stream to a live broadcasting room control area for playing, and forwarding and pushing the video recording stream to the current control side by the media server;

the operation message generation module is used for monitoring operation events acting on the live broadcasting room control area in real time, and generating corresponding operation messages according to the operation events, wherein the operation messages comprise operation instructions which can be analyzed to act on a live broadcasting room interface of the target user side and operation position information carried by the operation instructions according to requirements;

and the operation message pushing module is used for pushing the operation message to the target user terminal so that after the target user terminal analyzes the operation message, an operation instruction in the operation message is executed on a live broadcasting room interface of the target user terminal, the operation instruction acts on a corresponding position pointed by the operation position information according to the requirement to realize the control of the currently-performed live broadcasting room online service, and the operation message is forwarded and pushed to the client terminal by the communication server.

In a further embodiment, the recording video stream playing module includes:

the recording video stream receiving sub-module is used for receiving the recording video stream recorded by the target user terminal forwarded and pushed by the media server;

the video recording video stream playing sub-module is used for outputting the video recording video stream to a live broadcasting room control area of the current graphic user interface for playing;

and the operation instruction set control part display sub-module is used for displaying an operation instruction set control in the current graphical user interface, wherein the operation instruction set control comprises a pause control instruction, a continue control instruction and an end control instruction which cannot be generated through the live broadcasting room control area.

In a further embodiment, the operation message generation module includes:

the operation event monitoring sub-module is used for monitoring the operation event triggered by the current user in the live broadcasting room control area in real time;

an operation event response sub-module, configured to respond to the operation event, and obtain an operation instruction currently executed by a current user in the live broadcast room control area and the operation position information corresponding to the operation instruction in the live broadcast room control area;

and the operation message generation sub-module is used for packaging the operation instruction and the corresponding operation position information thereof to generate the operation message.

The application provides a remote control response method for a live broadcasting room, which comprises the following steps:

responding to a control application instruction pushed by a control end, and displaying a control application control in a current live broadcasting room interface;

responding to an acceptance control instruction acting on the control application control, pushing a screen recording video stream generated by recording a graphical user interface of a current live broadcasting room interface to the control end, and forwarding and pushing the screen recording video stream to the control end by a media server;

responding to the operation message pushed by the control end, executing the operation instruction in the position corresponding to the operation position information in the current live broadcasting room interface according to the operation instruction and the operation position information corresponding to the operation instruction contained in the operation message, controlling the live broadcasting room online service in the current live broadcasting room, and forwarding and pushing the operation message to the current client by the communication server.

A remote control processing device for a living room according to another object of the present application includes:

the application instruction response module is used for responding to the control application instruction pushed by the control end and displaying a control application control in the current live broadcasting room interface;

The control application control module is used for receiving a control application control, and receiving a control application control, wherein the control application control is used for receiving a control application control, and sending a video recording stream generated by recording a graphical user interface of a current live broadcasting room interface to the control end;

the operation message response module is used for responding to the operation message pushed by the control end, executing the operation instruction in the position corresponding to the operation position information in the current live broadcasting room interface according to the operation instruction and the corresponding operation position information contained in the operation message, and controlling the online service of the live broadcasting room in the current live broadcasting room, wherein the operation message is forwarded and pushed to the current client by the communication server.

An electronic device adapted to the object of the application comprises a central processor and a memory, the central processor being adapted to invoke the steps of running a computer program stored in the memory for executing the live room remote control processing method or the live room remote control processing method.

A non-volatile storage medium adapted to the object of the application stores a computer program implemented according to the live room remote control processing method or the live room remote control processing method, which when being called by a computer to run, performs the steps comprised by the corresponding method.

In order to solve the above technical problems, an embodiment of the present application further provides a computer program product, which includes a computer program and computer instructions, where the computer program and the computer instructions, when executed by a processor, cause the processor to execute the steps of the live broadcast data hierarchical prediction method or the live broadcast room remote control processing method.

Compared with the prior art, the application has the following advantages:

the application can provide remote assistance service for users in the live broadcasting platform to provide the users with the assistance required by the users when the users routinely use live broadcasting application programs of the platform or conduct live broadcasting business in the application, is similar to providing real-time manual assistance for the users, for example, helping the live broadcasting users to provide the service for maintaining audience order, improving the live broadcasting experience of the live broadcasting users, or helping the users with errors in the live broadcasting room to provide error repair service, or providing guidance for the users unfamiliar with live broadcasting playing activities started in the live broadcasting room, and the like, thereby improving the experience of the live broadcasting platform users using the live broadcasting application programs in a remote control assistance mode.

Secondly, the video stream recorded on the screen and the operation message are forwarded and pushed through different server shunting, so that when the control end executes remote control through the video stream recorded on the screen, the synchronization between the video stream played by the control end and the operation in the user end remotely controlled by the control end is ensured, and the data transmission efficiency in the process of bidirectional transmission of different data is improved.

Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

Drawings

The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

FIG. 1 is a schematic diagram of a typical network deployment architecture according to an embodiment of the present application;

FIG. 2 is a flow chart of an exemplary embodiment of a remote control processing method for a living room according to the present application;

FIG. 3 is a diagram of a graphical user interface for a client visual control application instruction according to the present application;

FIG. 4 is a diagram of a graphical user interface of a live room interface of a client according to the present application;

FIG. 5 is a diagram of a graphical user interface for a control end to play a video stream recorded on a screen according to the present application;

FIG. 6 is a diagram of a graphical user interface of a user at a control end for remote control via a live room area in accordance with the present application;

FIG. 7 is a diagram of a graphical user interface for executing a corresponding operation after receiving an operation message by a user terminal according to the present application;

FIG. 8 is a schematic diagram of a graphical user interface for synchronously playing the latest video stream recorded in a live broadcasting room area after a control end pushes an operation message;

Fig. 9 is a schematic flow chart formed by an embodiment of receiving a help notification pushed by a user terminal in a live broadcast room by a control terminal in the present application;

FIG. 10 is a diagram of a graphical user interface for control-side visual help notifications in accordance with the present application;

fig. 11 is a schematic flow chart formed by an embodiment of playing a video stream of a receiving and recording screen at a control end in the present application;

FIG. 12 is a flowchart illustrating an embodiment of generating an operation message by a control terminal according to the present application;

fig. 13 is a functional block diagram of an exemplary embodiment of a live room remote control processing device of the present application;

fig. 14 is a flow chart of an exemplary embodiment of a live room remote control response method of the present application;

fig. 15 is a functional block diagram of an exemplary embodiment of a live room remote control response device of the present application;

fig. 16 is a basic structural block diagram of a computer device according to an embodiment of the present application.

Detailed Description

Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the application.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.

It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

As used herein, "client," "terminal device," and "terminal device" are understood by those skilled in the art to include both devices that include only wireless signal receivers without transmitting capabilities and devices that include receiving and transmitting hardware capable of two-way communication over a two-way communication link. Such a device may include: a cellular or other communication device such as a personal computer, tablet, or the like, having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display; a PCS (Personal Communications Service, personal communication system) that may combine voice, data processing, facsimile and/or data communication capabilities; a PDA (Personal Digital Assistant ) that can include a radio frequency receiver, pager, internet/intranet access, web browser, notepad, calendar and/or GPS (Global Positioning System ) receiver; a conventional laptop and/or palmtop computer or other appliance that has and/or includes a radio frequency receiver. As used herein, "client," "terminal device" may be portable, transportable, installed in a vehicle (aeronautical, maritime, and/or land-based), or adapted and/or configured to operate locally and/or in a distributed fashion, at any other location(s) on earth and/or in space. As used herein, a "client," "terminal device," or "terminal device" may also be a communication terminal, an internet terminal, or a music/video playing terminal, for example, a PDA, a MID (Mobile Internet Device ), and/or a mobile phone with music/video playing function, or may also be a device such as a smart tv, a set top box, or the like.

The application refers to hardware such as a server, a client, a service node, and the like, which essentially is an electronic device with personal computer and other functions, and is a hardware device with necessary components disclosed by von neumann principles such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, an output device, and the like, wherein a computer program is stored in the memory, and the central processing unit calls the program stored in the memory to run, executes instructions in the program, and interacts with the input and output devices, thereby completing specific functions.

It should be noted that the concept of the present application, called "server", is equally applicable to the case of server clusters. The servers should be logically partitioned, physically separate from each other but interface-callable, or integrated into a physical computer or group of computers, according to network deployment principles understood by those skilled in the art. Those skilled in the art will appreciate this variation and should not be construed as limiting the implementation of the network deployment approach of the present application.

Referring to fig. 1, the hardware base required for implementing the related technical solution of the present application may be deployed according to the architecture shown in the figure. The server 80 of the present application is deployed at the cloud as a service server, and may be responsible for further connecting to related data servers and other servers providing related support, so as to form a logically related service cluster, to provide services for related terminal devices, such as a smart phone 81 and a personal computer 82 shown in the figure, or a third party server (not shown). The smart phone and the personal computer can access the internet through a well-known network access mode, and establish a data communication link with the cloud server 80 so as to run a terminal application program related to the service provided by the server.

For the server, the application program is usually constructed as a service process, and a corresponding program interface is opened for remote call of the application program running on various terminal devices.

The application program refers to an application program running on a server or terminal equipment, the application program adopts a programming mode to realize the related technical scheme of the application, the program codes of the application program can be stored in a nonvolatile storage medium which can be identified by a computer in the form of computer executable instructions, and the program codes are called by a central processing unit to run in a memory, and the related device of the application is constructed by the running of the application program on the computer.

The technical solution of the application, which is suitable for implementation in a terminal device, may also be programmed to be built into an application providing live webcasting as part of which the functionality is extended. The network live broadcast refers to a live broadcast room network service realized based on the network deployment architecture.

The application relates to a live broadcast room, which is a video chat room realized by means of an Internet technology and generally has an audio and video broadcast control function, wherein the video chat room comprises a host user and a spectator user, and the spectator user can comprise registered users registered in a platform or unregistered guest users; the registered user who pays attention to the anchor user may be a registered user who pays attention to the anchor user or an unregistered user. The interaction between the anchor user and the audience user can be realized through the well-known online interaction modes such as voice, video, text and the like, generally, the anchor user performs programs for the audience user in the form of audio and video streams, and economic transaction behaviors can be generated in the interaction process. Of course, the application form of the live broadcast room is not limited to online entertainment, and can be popularized to other related scenes, such as educational training scenes, video conference scenes, product recommendation sales scenes and any other scenes needing similar interaction.

Those skilled in the art will appreciate that: although the various methods of the present application are described based on the same concepts so as to be common to each other, the methods may be performed independently of each other unless specifically indicated otherwise. Similarly, for the various embodiments disclosed herein, all concepts described herein are presented based on the same general inventive concept, and thus, concepts described herein with respect to the same general inventive concept, and concepts that are merely convenient and appropriately modified, although different, should be interpreted as equivalents.

The various embodiments of the present application to be disclosed herein, unless the plain text indicates a mutually exclusive relationship with each other, the technical features related to the various embodiments may be cross-combined to flexibly construct a new embodiment as long as such combination does not depart from the inventive spirit of the present application and can satisfy the needs in the art or solve the deficiencies in the prior art. This variant will be known to the person skilled in the art.

Referring to fig. 2, in an exemplary embodiment of the present application, a remote control processing method for a live broadcast room includes the following steps performed by a control terminal:

step S11, pushing a control application instruction to a target user terminal in a live broadcasting room:

The current control end pushes the control application instruction to the target user end in the live broadcasting room so as to control the online service of the target user end in the live broadcasting room.

The control end acquires a video recording stream of the target user end recording the interface of the live broadcasting room by pushing the application instruction to the target user end positioned in the live broadcasting room, and synchronously controls the online service of the target user end in the live broadcasting room through the video recording stream.

The control application instruction is generally forwarded and pushed by a communication server constructed based on a TPC network communication protocol, and the current control end pushes the control application instruction to the communication server, so that the communication server responds to the control application instruction, determines a target user end receiving the control application instruction, and further forwards and pushes the control application instruction to the target user end.

After the target user receives the control application instruction, a control application control is displayed in a graphical user interface of a live broadcasting room interface of the target user, so that the target user triggers an acceptance control event for accepting the control application instruction or a rejection control event for rejecting the control application instruction through the control application control, when the target user triggers the acceptance control event, the target user records the graphical user interface of the live broadcasting room interface of the target user to generate a recorded screen video stream and pushes the recorded screen video stream to the current control end, and the related specific embodiment refers to the subsequent steps, which are not repeated.

The control application instruction is generally pushed actively by a current control end, the current control end selects a target user end in any live broadcasting room, the target user end is generally a main broadcasting user end, then the control application instruction is pushed to the target user end so as to apply for remote control to the target user end, in addition, after the control application instruction responds to a help seeking request pushed by any user end in any live broadcasting room for the current control end, the control application instruction is pushed to the user end so as to help the user end to perform online service of the live broadcasting room in the live broadcasting room.

Step S12, receiving a video recording stream formed by recording a live broadcasting room interface of the target user terminal after the target user terminal responds to the control application instruction, outputting the video recording stream to a live broadcasting room control area for playing, and forwarding and pushing the video recording stream to a current control terminal by a media server:

after the current control end pushes the control application instruction to the target user end, the target user end waits for receiving the control application instruction, and after the target user end receives the control application instruction, the target user end pushes the video recording stream formed by recording the live broadcasting room interface to the current control end, so that the video recording stream is received by the current control end and output to a live broadcasting room control area for playing, and further, live broadcasting room online service carried out on a live broadcasting room where the target user end is located is carried out through the video recording stream played in the live broadcasting room control area.

Referring to fig. 3 and fig. 4, after the target ue receives the control application command pushed by the current control end, the control application control 301 shown in fig. 3 is displayed in the live broadcast room interface, and the user at the target ue may generate the control command through the control command 302 shown in the control application control, so that the target ue responds to the command to record the video recording stream generated by the gui of the live broadcast room interface shown in fig. 4, and push the video recording stream to the media server, so as to drive the media server to forward the video recording stream to the current control end.

Referring to fig. 5, after the current control end receives the video recording stream forwarded and pushed by the media server, the video recording stream is output to the live broadcasting room control area 501 shown in fig. 5 for playing, so that a user at the current control end controls the target user end through the live broadcasting room control area 501, in addition, after the current control end outputs and plays the video recording stream, the operation instruction set control 502 shown in fig. 5 is displayed in the graphical user interface, and the operation instruction set control 502 includes a pause control instruction, a continue control instruction and an end control instruction which cannot be generated through the live broadcasting room control area 501.

The media server is responsible for forwarding and pushing the video stream recorded and generated by the target user side, and the target user side firstly pushes the video stream recorded and generated and the corresponding forwarding instruction to the media server together, and after the media server receives the video stream recorded and generated, the media server determines a control side for receiving the video stream recorded and generated according to the forwarding instruction, so that the video stream recorded and generated is pushed to the control side for output and playing.

And the current control end establishes a data communication link for continuously transmitting the video stream recorded by the target user end, and the data communication link is responsible for maintenance by the media server, so that the current control end continuously receives the video stream recorded and generated by the target control end through the data communication link to generate and play, and the video stream recorded and played in the live broadcasting room control area of the current control end is ensured to be synchronous with the live broadcasting room interface in the target user end.

Step S13, monitoring an operation event acting on the live broadcasting room control area in real time, and generating a corresponding operation message according to the operation event, wherein the operation message comprises an operation instruction which can be analyzed to act on a live broadcasting room interface of the target user side and operation position information carried by the operation instruction as required:

And after the current control end outputs the screen recording video stream to the live broadcasting room control area for playing, monitoring the operation event triggered by the user at the current control end in the live broadcasting room control area in real time so as to output a corresponding operation message according to the monitored operation event.

The current control end monitors operation events carried out in the live broadcasting room control area by a current user at the current control end in real time, for example, the current user clicks or presses for a long time in the live broadcasting room control area or touches a clicking operation event or a long time pressing operation event or a touch operation event which are triggered by any control corresponding to any control in the live broadcasting room interface displayed in the screen recording video stream displayed in the current playing state, or the live broadcasting room interface displayed in the live broadcasting room control area drags to drag the dragging operation event triggered by any control in the live broadcasting room interface, or the live broadcasting room interface displayed in the screen recording video stream displayed in the live broadcasting room control area by the current user slides to switch the live broadcasting room interface or activates and displays the sliding operation event triggered by the corresponding control.

After the current control end monitors that the current user triggers a corresponding operation event in a real-time live broadcast room control area, the current control end responds to the operation event, determines an operation instruction corresponding to the operation event, acquires operation position information of a position operated by the operation instruction in a live broadcast room interface displayed in the video recording video stream, packages the operation instruction and the corresponding operation position information into the operation message, so that the target user end can execute corresponding operation in the live broadcast room interface according to the operation instruction and the operation position information contained in the operation message after analyzing the operation message, and controls live broadcast room online service in the live broadcast room where the operation instruction is located.

Referring to fig. 6, fig. 6 is a schematic diagram of a current user at a current control end when performing an operation event in the live broadcast room control area, where clicking a viewer utterance 602 in the live broadcast room control area 601 shown in fig. 6 triggers a click operation event, an operation corresponding to the click operation event is pointed to be a click operation instruction, and operation position information of the click operation corresponding to the click operation instruction is an operation position 603 in the live broadcast room control area 601 shown in fig. 6, where the operation position 603 is a position of the click operation in the live broadcast room control area 601 by the current user.

Step S14, pushing the operation message to the target user terminal so that after the target user terminal analyzes the operation message, executing an operation instruction in the operation message on a live broadcast room interface thereof, acting on a corresponding position pointed by operation position information according to requirements to realize control of the currently-performed live broadcast room online service, and forwarding and pushing the operation message to the client terminal by a communication server:

after the current control end generates the operation message, the operation message is pushed to the target user end so as to control the online service of the live broadcasting room in the live broadcasting room where the target user end is currently located, specifically, after the target user end receives the operation message, the operation instruction and the corresponding operation position information contained in the operation message are obtained through analysis so as to execute the operation instruction in the position corresponding to the operation position information in the current live broadcasting room interface.

The communication server receives the operation information, then verifies the current control end to determine the target user end establishing a data communication link with the current control end, and then pushes the operation information to the target user end.

The live broadcasting room online service generally comprises live broadcasting room audience management service, live broadcasting room play service and live broadcasting room error reporting service, wherein the live broadcasting room audience management service is generally oriented to a live broadcasting user side in a live broadcasting room, a current control side is responsible for the management online service of audience users in the live broadcasting room where the live broadcasting user side is located, the live broadcasting room play service is generally online service with play interaction properties such as a live-wheat online service or a live-broadcasting lottery online service which is started in the live broadcasting room, the current control side can control the online service with play interaction properties such as the live broadcasting room play service or the live broadcasting user side so as to provide corresponding play service for the live broadcasting room error reporting service, the live broadcasting room error reporting service is generally online service with errors or faults in the live broadcasting room, and when errors or faults occur in the live broadcasting room where any user side is located, the live broadcasting room error reporting service can push a help request to the current control side so as to request the current control side to repair the errors or faults in the live broadcasting room.

Referring to fig. 6, 7 and 8, fig. 7 is a schematic diagram of a live broadcasting interface of a live broadcasting room in which a live broadcasting room online service is the live broadcasting room audience management service, a live broadcasting room user terminal responds to the operation message pushed by a current control terminal, when the current control terminal triggers an operation message generated by clicking an operation event at an operation position 603 of an audience speaking 602 in a live broadcasting room control area 601 shown in fig. 6 to push the operation message to the target user terminal, the target user terminal pairs to the operation position 701 in fig. 7 according to the operation position information contained in the operation message, and executes clicking operation represented in the operation instruction, so that an audience management control 702 for speaking to a user UVW is displayed in fig. 7, and a report speaking event, a prohibit speaking event and a kick-out room event can be executed for the user UVW in the audience management control 702; correspondingly, after the target user side executes the event of controlling the live broadcasting room online service according to the operation instruction and the corresponding operation position information contained in the operation message, the video stream recorded in the live broadcasting room control area of the graphical user interface of the current control side is converted from fig. 6 to the video stream recorded in the live broadcasting room control area 801 shown in fig. 8.

According to the method, a control end pushes a control application instruction to a user end in a live broadcasting room to apply for controlling live broadcasting room online service of the user end in the live broadcasting room, after the user end receives the control application instruction, a screen recording video stream of a live broadcasting room interface of the user end is pushed to the control end, so that the control end can conveniently control a user to know the current live broadcasting room interface of the user end by playing the screen recording video stream, and the control end can provide corresponding assistant service for a user in the live broadcasting room by operating the live broadcasting room online service in the live broadcasting room where the user end is located in a live broadcasting room control area where the screen recording video stream is displayed, for example, the live broadcasting room online service is provided for a user in the live broadcasting room, the user experience of managing audience is improved by helping a host user in live broadcasting, or error repairing service is provided for a user in error in the live broadcasting room.

The above exemplary embodiments and variations thereof fully disclose embodiments of the live room remote control processing method of the present application, but various variations of the method may still be deduced by transforming and augmenting some technical means, and the following summary illustrates other embodiments:

In one embodiment, referring to fig. 9 and 10, the method includes the following steps:

step S09, receiving help-seeking notification pushed by a user terminal in a live broadcasting room, and displaying a notification window in a current graphical user interface so as to display user characteristic information and live broadcasting room characteristic information contained in the help-seeking notification in the notification window:

the current control end receives the help seeking notification pushed by the user end in the live broadcasting room, displays the notification window in the current graphical user interface, and outputs the user characteristic information and the live broadcasting room characteristic information contained in the help seeking notification to the notification window for display.

Referring to fig. 10, fig. 10 is a schematic diagram of a graphical user interface after the help-seeking notification is received by the current control unit, and after the help-seeking notification is received by the current control unit, a notification window 1001 as shown in fig. 10 is displayed in the current graphical user interface, and the user feature information and the live-broadcasting room feature information included in the help-seeking notification are output to the notification window 1001 for display.

Step S10, responding to a help-seeking acceptance instruction acting on the notification window, and generating a control application instruction pointing to a user terminal located in the live broadcasting room:

Referring to fig. 10, after a user at the current control end touches the accept control 1002 in the notification window 1001 shown in fig. 10, the job-seeking accept instruction is triggered and output, so that the current control end responds to the job-seeking accept instruction and generates the control application instruction pointing to the user end corresponding to the user feature information displayed in the notification window 1001.

In this embodiment, the control end may receive a help-seeking notification pushed by a user end that needs a helper service, so as to receive a help-seeking request of the user end, provide the helper service for the user end, and further improve the use experience of a user in a live broadcast room.

In an embodiment, please refer to fig. 5 and 11, the step of receiving a video recording stream formed by the target user side recording the live broadcasting room interface of the target user side after the target user side responds to the control application command, and outputting the video recording stream to the live broadcasting room control area for playing includes the following steps:

step S121, receiving a video stream recorded by the target client forwarded and pushed by the media server:

after the current control end pushes the control application instruction to the target user end and the target user end receives the control application instruction, the current control end receives the video stream recorded on the live broadcasting room interface of the target user end pushed and recorded by the target user end forwarded by the media server.

Step S122, outputting the video stream to the live broadcasting room control area of the current graphical user interface for playing:

referring to fig. 5, after the current control end receives the video recording stream, the video recording stream is output to the live broadcasting room control area 501 shown in fig. 5 for playing, so that the user at the current control end controls the live broadcasting room online service performed by the target user end in the live broadcasting room through the live broadcasting room control area 501.

Step S123, displaying an operation instruction set control in the current graphical user interface, where the operation instruction set control includes a pause control instruction, a continue control instruction, and an end control instruction that cannot be generated through the live room control area:

referring to fig. 5, the current control end outputs the video stream to the live broadcasting room control area 501 shown in fig. 5 for playing, and simultaneously displays an operation instruction set control 502 shown in fig. 5 in the current graphical user interface, where the operation instruction set control is used for generating a pause control instruction, a continue control instruction and an end control instruction which cannot be generated through the live broadcasting room control area 501.

In this embodiment, the control end receives the video stream recorded on the screen forwarded by the media server, plays and displays the video stream, so that the user knows the interface of the live broadcasting room of the target user end through the video stream recorded on the screen, further performs corresponding operation event to control the online service of the live broadcasting room performed in the live broadcasting room, and displays an operation instruction set control in the interface, so that the user can operate the operation which cannot be realized through the control area of the live broadcasting room.

In an embodiment, please refer to fig. 6 and fig. 12, the step of monitoring the operation event acting on the live broadcasting room control area in real time, and generating a corresponding operation message according to the operation event, where the operation message includes an operation instruction which can be parsed into a live broadcasting room interface acting on the target user side and operation position information carried by the operation instruction as required, includes the following steps:

step S131, monitoring, in real time, an operation event triggered by the current user through the live broadcast room control area:

Step S132, in response to the operation event, acquiring an operation instruction currently executed by a current user in the live broadcast room control area and the operation position information corresponding to the operation instruction in the live broadcast room control area:

Step S133, encapsulating the operation instruction and the corresponding operation position information to generate the operation message:

the current control end encapsulates the operation instruction and the corresponding operation position information thereof into the operation message so as to push the operation message to the target user end later, and after the target user end receives the operation message, the operation instruction and the corresponding operation position information thereof contained in the operation message are analyzed and acquired so as to execute the operation instruction in the position corresponding to the operation position information in the current live broadcasting room interface.

In this embodiment, the control end monitors the operation event of the user in the control area of the live broadcasting room in real time, so as to generate a corresponding operation message for pushing, and further synchronize the operation behavior of the user in the control end to the target user end, so as to realize the online service of the live broadcasting room in the remote control user end.

Further, by performing the functionalization of each step in the method disclosed in the foregoing embodiments, a remote control processing device for a living room according to the present application may be constructed, and in this way, referring to fig. 13, in one exemplary embodiment, the device includes: the application instruction pushing module 11 is used for pushing a control application instruction to a target user terminal in a live broadcasting room; the video recording and video stream playing module 12 is configured to receive a video recording stream formed by the target user side on the live broadcasting room interface of the target user side after the target user side responds to the control application instruction, output the video recording stream to the live broadcasting room control area for playing, and forward and push the video recording stream to the current control side by the media server; an operation message generating module 13, configured to monitor an operation event acting on the live broadcast room control area in real time, and generate a corresponding operation message according to the operation event, where the operation message includes an operation instruction that can be parsed into a live broadcast room interface acting on the target user side and operation position information carried by the operation instruction as required; and the operation message pushing module 14 is configured to push the operation message to the target user terminal, so that after the target user terminal parses the operation message, an operation instruction in the operation message is executed on a live broadcast room interface of the target user terminal, the operation instruction acts on a corresponding position pointed by operation position information according to requirements, and control the currently-performed live broadcast room online service is realized, and the operation message is forwarded and pushed to the client terminal by the communication server.

In one embodiment, the recording video stream playing module 12 includes: the recording video stream receiving sub-module is used for receiving the recording video stream recorded by the target user terminal forwarded and pushed by the media server; the video recording video stream playing sub-module is used for outputting the video recording video stream to a live broadcasting room control area of the current graphic user interface for playing; and the operation instruction set control part display sub-module is used for displaying an operation instruction set control in the current graphical user interface, wherein the operation instruction set control comprises a pause control instruction, a continue control instruction and an end control instruction which cannot be generated through the live broadcasting room control area.

In one embodiment, the operation message generation module 13 includes: the operation event monitoring sub-module is used for monitoring the operation event triggered by the current user in the live broadcasting room control area in real time; an operation event response sub-module, configured to respond to the operation event, and obtain an operation instruction currently executed by a current user in the live broadcast room control area and the operation position information corresponding to the operation instruction in the live broadcast room control area; and the operation message generation sub-module is used for packaging the operation instruction and the corresponding operation position information thereof to generate the operation message.

Referring to fig. 14, in an exemplary embodiment of a remote control response method in a living room of the present application, the method includes the following steps performed by a user terminal:

step S21, in response to a control application instruction pushed by a control end, displaying a control application control in a current live broadcast room interface:

the current user side responds to the control application instruction pushed by the control side to display the control application control in the current live broadcast room interface, specifically, please refer to fig. 3, fig. 3 is a schematic diagram of a graphical user interface of the current user side displaying the control application control in the current live broadcast room interface, the control application control 301 shown in fig. 3 is provided with an accept control 302 and a reject control 303, and when the user touches the accept control 302 in the control application control 301, the subsequent steps are executed.

The control application instruction is generally forwarded and pushed by a communication server constructed based on TCP, and the control end pushes the control application instruction to the communication server so as to drive the communication server to forward and push the control application instruction to the current user end.

Step S22, in response to the control instruction applied to the control application control, pushing the video recording stream generated by recording the graphical user interface of the current live broadcasting room interface to the control end, wherein the video recording stream is forwarded and pushed to the control end by a media server:

After a user at the current user terminal touches an acceptance control in the control application control, triggering to generate the acceptance control instruction, so that the current user terminal responds to the acceptance control instruction, recording a graphical user interface of a current live broadcasting room interface to generate the screen recording video stream, and pushing the screen recording video stream to the control terminal for output playing.

Referring to fig. 4 and fig. 5, after the current ue responds to the control command, the gui shown in fig. 4 is recorded, and the recorded video stream including the gui is generated and pushed to the control terminal, so that the control terminal plays the gui shown in fig. 4 in the live broadcasting room control area 501 in the gui shown in fig. 4.

The media server is responsible for forwarding and pushing the video stream recorded and generated by the current user side, and the current user side firstly pushes the video stream recorded and generated and the corresponding forwarding instruction thereof to the media server together, and after the media server receives the video stream recorded and generated, the media server determines a control side for receiving the video stream recorded and generated according to the forwarding instruction, so that the video stream recorded and generated is pushed to the control side for output and playing.

Step S23, responding to the operation message pushed by the control end, executing the operation instruction in the position corresponding to the operation position information in the current live broadcasting room interface according to the operation instruction and the corresponding operation position information contained in the operation message, and controlling the live broadcasting room online service in the current live broadcasting room, wherein the operation message is forwarded and pushed to the current client by the communication server:

the current user side responds to the operation message pushed by the control side which receives the video stream of the recording screen and forwards the operation message, so as to execute the operation represented by the operation instruction in the position corresponding to the operation position message in the current live broadcasting room interface according to the operation instruction contained in the operation message and the operation position message corresponding to the operation instruction, and further the control side controls the live broadcasting room online service in the current live broadcasting room.

The communication server receives the operation information, and then verifies the current control end to determine a target user end establishing a data communication link with the current control end, and then pushes the operation information to the target user end.

Referring to fig. 6, fig. 7, and fig. 8, the current user side is a hosting user side in a living room in which a living room online service is the living room audience management service, when an operation message generated by triggering a click operation event at an operation position 603 of clicking an audience speech 602 in a living room control area 601 shown in fig. 6 is pushed to the current user side, the current user side performs a click operation represented in the operation instruction at the operation position 701 in fig. 7 according to the operation position information contained in the operation message, and further displays an audience management control 702 pointing to "user UVW" in fig. 7, and in the audience management control 702, can perform a report speech event, prohibit a speech event, and kick out an inter-living event for "user UVW"; correspondingly, after the current user side executes the event of controlling the live broadcasting room online service according to the operation instruction and the corresponding operation position information contained in the operation message, the video stream recorded in the live broadcasting room control area of the graphical user interface of the control side is converted from fig. 6 to the video stream recorded in the live broadcasting room control area 801 shown in fig. 8.

According to the exemplary embodiment, the user terminal in the live broadcasting room can respond to the control application instruction pushed by the control terminal, and the live broadcasting room online service performed in the live broadcasting room is operated by the control terminal, and the live broadcasting room online service is played and displayed by pushing the video stream of the video recording self graphical user interface to the control terminal, so that the user of the live broadcasting platform can be provided with remote assistance service according to the operation information pushed by the control terminal, the live broadcasting service of the host broadcasting user of the assistance or auxiliary platform can be provided for the user unfamiliar with the live broadcasting room online service of the platform, and the use experience of the user in the platform can be improved.

Further, by performing the functionalization of the steps in the method disclosed in the foregoing embodiments, a remote control response device for a living room according to the present application may be constructed, and in this way, referring to fig. 15, in one exemplary embodiment, the device includes: the application instruction response module 21 is used for responding to the control application instruction pushed by the control end and displaying a control application control in the current live broadcasting room interface; the control application control module 22 is configured to respond to a control application control, and send a video recording stream generated by recording a graphical user interface of a current live broadcasting room interface to the control end, where the video recording stream is forwarded and sent to the control end by a media server; the operation message response module 23 is configured to respond to an operation message pushed by the control end, execute the operation instruction in a position corresponding to the operation position information in the current live broadcast room interface according to the operation instruction and the operation position information corresponding to the operation instruction included in the operation message, and control online service of the live broadcast room in the current live broadcast room, where the operation message is forwarded and pushed to the current client by the communication server.

In order to solve the technical problem, the embodiment of the application also provides a computer device for running a computer program realized according to the live broadcasting room remote control processing method or the live broadcasting room remote control processing method. Referring specifically to fig. 16, fig. 16 is a basic structural block diagram of a computer device according to the present embodiment.

As shown in fig. 16, the internal structure of the computer device is schematically shown. The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected by a system bus. The nonvolatile storage medium of the computer device stores an operating system, a database and computer readable instructions, the database can store a control information sequence, and when the computer readable instructions are executed by the processor, the processor can realize a remote control processing method of a live broadcasting room or a remote control processing method of the live broadcasting room. The processor of the computer device is used to provide computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may store computer readable instructions that, when executed by the processor, cause the processor to perform a live room remote control processing method or a live room remote control processing method. The network interface of the computer device is for communicating with a terminal connection. It will be appreciated by those skilled in the art that the structure shown in fig. 15 is merely a block diagram of a portion of the structure associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements are applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

In this embodiment, the processor is configured to execute specific functions of each module/sub-module in the live broadcast room remote control processing device or the live broadcast room remote control processing device according to the present application, and the memory stores program codes and various data required for executing the modules. The network interface is used for data transmission between the user terminal or the server. The memory in this embodiment stores program codes and data required for executing all modules/sub-modules in the live broadcast room remote control processing apparatus or the live broadcast room remote control processing apparatus, and the server can call the program codes and data of the server to execute the functions of all sub-modules.

The present application also provides a non-volatile storage medium, in which the live-room remote control processing method or the live-room remote control processing method is written as a computer program, and the computer program is stored in the storage medium in the form of computer readable instructions, where the computer readable instructions when executed by one or more processors mean that the program runs in a computer, thereby causing the one or more processors to execute the steps of the live-room remote control processing method or the live-room remote control processing method according to any one of the embodiments described above.

Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored in a computer-readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. The storage medium may be a nonvolatile storage medium such as a magnetic disk, an optical disk, a Read-Only Memory (ROM), or a random access Memory (Random Access Memory, RAM).

In summary, the application provides remote assistance service for users of the live platform, and video streams and instructions are distributed, forwarded and pushed by the corresponding servers to realize data synchronization.

It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.

Those of skill in the art will appreciate that the various operations, methods, steps in the flow, acts, schemes, and alternatives discussed in the present application may be alternated, altered, combined, or eliminated. Further, other steps, means, or steps in a process having various operations, methods, or procedures discussed herein may be alternated, altered, rearranged, disassembled, combined, or eliminated. Further, steps, measures, schemes in the prior art with various operations, methods, flows disclosed in the present application may also be alternated, altered, rearranged, decomposed, combined, or deleted.

The foregoing is only a partial embodiment of the present application, and it should be noted that it will be apparent to those skilled in the art that modifications and adaptations can be made without departing from the principles of the present application, and such modifications and adaptations are intended to be comprehended within the scope of the present application.

Claims

1. The remote control processing method for the live broadcasting room is applied to a control end and is characterized by comprising the following steps:

pushing a control application instruction to a target user end in a live broadcasting room, wherein the target user end is a main broadcasting user end in the live broadcasting room;

Receiving a video recording stream formed by the target user side on the live broadcasting room interface of the target user side after responding to the control application instruction, outputting the video recording stream to a live broadcasting room control area of the live broadcasting room interface of the current control side for playing, and forwarding and pushing the video recording stream to the current control side by a media server;

monitoring an operation event acting on the live broadcasting room control area in real time, and generating a corresponding operation message according to the operation event, wherein the operation message comprises an operation instruction which can be analyzed to act on a live broadcasting room interface of the target user side and operation position information carried by the operation instruction;

and pushing the operation message to the target user side so that after the target user side analyzes the operation message, an operation instruction in the operation message is executed on a live broadcasting room interface of the target user side, the operation instruction acts on a corresponding position pointed by operation position information, and the current live broadcasting room online service is controlled to provide assistant service for managing audiences for a host broadcasting user of the target user side, the operation message is forwarded and pushed to the target user side by a communication server, and the live broadcasting room online service is live broadcasting room audience management service.

2. The method according to claim 1, characterized by the following pre-steps:

receiving a help seeking notification pushed by a user terminal in a live broadcasting room, and displaying a notification window in a current graphical user interface so as to display user characteristic information and live broadcasting room characteristic information contained in the help seeking notification in the notification window;

and responding to the help-seeking acceptance instruction acted on the notification window, and generating a control application instruction pointing to the target user terminal positioned in the live broadcasting room.

3. The method of claim 1, wherein the step of receiving a video recording stream formed by the target client in response to the control application command and recording the live broadcasting room interface of the target client, and outputting the video recording stream to the live broadcasting room control area for broadcasting includes the steps of:

4. The method according to claim 1, wherein the step of monitoring the operation event acting on the live broadcasting room control area in real time, and generating a corresponding operation message according to the operation event, wherein the operation message includes an operation instruction which can be resolved into a live broadcasting room interface acting on the target user side and operation position information carried by the operation instruction as required includes the following steps:

5. The method of claim 4, wherein the operation event triggered by the current user through the live room control area comprises any one of: click operation event, long press operation event, slide operation event.

6. The remote control response method for the live broadcasting room is applied to a target user side where a host broadcasting user is located, and is characterized by comprising the following steps:

responding to the operation message pushed by the control end, executing the operation instruction in a position corresponding to the operation position information in the current live broadcasting room interface according to the operation instruction and the operation position information corresponding to the operation instruction, controlling live broadcasting room online service in the current live broadcasting room to provide assistant service for managing audiences for the anchor users of the target user end, forwarding and pushing the operation message to the target user end by a communication server, and enabling the live broadcasting room online service to be the live broadcasting room audience management service.

7. An electronic device comprising a central processor and a memory, characterized in that the central processor is adapted to invoke a computer program stored in the memory for performing the steps of the method according to any of claims 1 to 5.

8. A non-volatile storage medium, characterized in that it stores in form of computer readable instructions a computer program implemented according to the method of any one of claims 1 to 5, which when invoked by a computer, performs the steps comprised by the method.