WO2022117028A1

WO2022117028A1 - Game livestreaming-based anchor input operation sharing and displaying methods and device

Info

Publication number: WO2022117028A1
Application number: PCT/CN2021/134995
Authority: WO
Inventors: 万安庆
Original assignee: 上海达龙信息科技有限公司
Priority date: 2020-12-03
Filing date: 2021-12-02
Publication date: 2022-06-09
Also published as: CN113730923A

Abstract

A game livestreaming-based anchor input operation sharing and displaying methods and a device. The game livestreaming-based anchor input operation sharing method comprises the steps: capturing anchor input operation data generated when an anchor operates at least one input device; for local computer livestreaming, respectively adding a timestamp when capturing for each anchor input operation data, and sending the data to a stream pushing server for stream pushing; and for cloud game livestreaming, sending each anchor input operation data to a remote cloud desktop server so that the remote desktop server firstly separately adds timestamps when the operation data is received and then sends the data to the stream pushing server for stream pushing. According to the method, the anchor can share the input operation thereof during livestreaming of a client game, and audiences can visually see the specific operation of the anchor and study the operation skill and the hand speed of the anchor.

Description

An anchor input operation sharing and display method and device based on game live broadcast

technical field

The present invention relates to the field of live game broadcasting, and in particular, to a method and device for sharing and displaying an anchor input operation of live game broadcasting (including local computer games and cloud games).

Background technique

At present, the game live broadcast industry is very hot, and the anchor interacts with the audience through screen sharing and voice. However, for the operation of the anchor, for mobile games, due to the particularity of mobile games, the anchor's screen will be directly recorded during the live broadcast. The operation of the virtual buttons in the game will also be recorded, and the audience can see the anchor's operation method and operation skills for the mobile game, so as to learn; for the terminal game, the operation of the anchor's keyboard and mouse cannot be directly reflected in the main screen of the live broadcast. , the anchor will use another camera to shoot the anchor's hand and open a small window for operations such as keyboard and mouse. The purpose of this behavior is only to indicate that the anchor is operating and has not opened it. However, because the display window of this part is small, it is impossible to The operation speed and operation skills of the anchor are completely displayed on the live broadcast, so it is not friendly to some viewers who want to learn operation skills and abilities from watching the live broadcast of the anchor.

SUMMARY OF THE INVENTION

In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a method and device for sharing and displaying an anchor input operation based on game live broadcast, so as to solve the above deficiencies in the prior art.

In order to achieve the above purpose and other related purposes, the present invention provides a method for sharing an input operation of an anchor based on a live game broadcast, which is applied to an anchor terminal; the method includes: capturing the anchor input operation data generated by the anchor operating at least one input device; For local computer game live broadcast, add the time stamp when capturing the input operation data of each anchor, and send it to the streaming server for streaming; for cloud game live broadcast, send the input operation data of each anchor to the remote cloud desktop A server, so that the remote desktop server first adds a timestamp when the operation data is received, and then sends it to the streaming server for streaming.

In an embodiment of the present invention, the method further includes: before capturing the anchor input operation data, first detecting whether there is an instruction to share the anchor input operation; when detecting the instruction to share the anchor input operation, identifying the device information of the at least one input device, and send the identification information of the at least one input device to the central control server; wherein, the address of the streaming server is provided by the central control server to the host terminal.

In an embodiment of the present invention, the at least one input device includes: an external physical input device connected to the host terminal in the local computer live broadcast, an external physical input device connected to the host terminal in the cloud game live broadcast, and the host. The virtual device started by the terminal; for the virtual device started by the host terminal, the method further includes: judging whether the virtual device is a virtual device downloaded from the central control server and uploaded by other terminal devices; if not, then Save the customized virtual device and upload it to the central control server.

In order to achieve the above purpose and other related purposes, the present invention provides a method for displaying an input operation of an anchor based on a live game broadcast, which is applied to a viewer terminal; the method includes: submitting a viewing request to a central control server to obtain information from the central control server. The address of the provided streaming server; the anchor input operation data of the game live broadcast is pulled from the streaming server; when decoding and rendering a live broadcast frame, the same timestamp is performed according to the timestamp of the live broadcast frame. The anchor inputs operation data to reproduce the anchor input operation event, so as to restore the anchor input operation on the display interface.

In an embodiment of the present invention, a virtual device is displayed on the display interface, and the input operation of the host is restored on the virtual device and displayed with special effects.

In an embodiment of the present invention, the method further includes: displaying, on the display interface, startup entries of different virtual devices provided by the central control server and corresponding to the identification information of the input device used by the anchor terminal, so as to start After downloading, display the opened virtual device; or, display the download entry of the virtual device provided by the central control server and corresponding to the identification information of the input device used by the anchor terminal on the display interface, so as to display the downloaded virtual device after downloading. equipment.

In order to achieve the above purpose and other related purposes, the present invention provides a method for sharing an input operation of an anchor based on a live game broadcast, which is applied to a central control server; the method includes: obtaining a live broadcast request from the anchor terminal; The live broadcast request is provided, and the stream push address is provided to the anchor terminal, so that the anchor terminal sends the anchor input operation data and the stream push address to the remote cloud desktop server, and the remote desktop server pushes the anchor input operation data to the corresponding obtain the viewing request of the viewer terminal; in response to the viewing request, provide the viewer terminal with the address of the streaming server, so that the viewer terminal can pull the anchor input operation from the streaming server data and reproduce the host input operation event.

In an embodiment of the present invention, for cloud game live broadcast, the method further includes: acquiring a custom virtual device uploaded by the host terminal and saving it; providing the viewer terminal with the same information as the input device used by the host terminal. The startup entry of the corresponding virtual device or the download entry of the virtual device.

In order to achieve the above purpose and other related purposes, the present invention provides a computer-readable storage medium, wherein a computer program is stored, and when the computer program is loaded and executed by a processor, the described method for sharing an anchor input operation based on a live game broadcast is realized. , or the method for displaying an input operation of an anchor based on a live game broadcast.

In order to achieve the above object and other related objects, the present invention provides an electronic device, comprising: a processor and a memory; wherein, the memory is used to store a computer program; the processor is used to load and execute the computer program, so that all The electronic device executes the method for sharing the host input operation based on the game live broadcast, or the method for displaying the host input operation based on the game live broadcast.

As mentioned above, in the present invention's method and device for sharing and displaying the host's input operation based on live game broadcast, the host only needs to install the cloud game application (the cloud game here refers to computer games, not cloud mobile games), and then Sharing of input operations during live streaming of client games through smartphones, tablets, laptops, desktops, etc. (Note: The input operations here do not include voice input and video input, but refer to peripheral mouse, peripheral keyboard, Input operations such as peripheral controllers and virtual keyboard and mouse controllers). Operating live is an auxiliary function of live video. The viewer client can restore the real situation of the live broadcast by executing the input event with the same timestamp when decoding and rendering the video stream. Through this method, the audience can intuitively watch which buttons the anchor has operated, and study the anchor's operating skills and operating speed, which is highly practical for learning audiences. In addition, the present invention is not limited to the live broadcast of cloud games, but can also be applied to the sharing of the anchor's input operation in the live broadcast of ordinary computer games.

Description of drawings

FIG. 1A is a schematic diagram of an application scenario of a local computer game live broadcast according to an embodiment of the present invention.

FIG. 1B is a schematic diagram of an application scenario of cloud game live broadcasting in another embodiment of the present invention.

FIG. 2 shows a flowchart of a method for sharing an input operation of an anchor based on a live game broadcast performed by an anchor terminal according to an embodiment of the present invention.

FIG. 3 is a flowchart showing a method for displaying an input operation of a host based on a live game broadcast performed by a viewer terminal in an embodiment of the present invention.

FIG. 4 is a flowchart of a method for sharing an input operation of an anchor based on a live game broadcast performed by a central control server according to an embodiment of the present invention.

FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed ways

The embodiments of the present invention are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other under the condition of no conflict.

It should be noted that the diagrams provided in the following embodiments are only used to illustrate the basic concept of the present invention in a schematic way, so the diagrams only show the components related to the present invention rather than the number, shape and For dimension drawing, the type, quantity and proportion of each component can be changed at will in actual implementation, and the component layout may also be more complicated.

In view of the deficiencies in the prior art, the present application provides an anchor input operation sharing and display method and device based on game live broadcast. When the anchor plays a computer game, there is no need to install a camera to live broadcast the hand operation, and the anchor uses a peripheral device (such as a mouse, Keyboard, handle), virtual devices (such as virtual keyboard and mouse, etc.), the anchor operation data is captured, processed, and transmitted through the anchor terminal. The audience downloads and displays the virtual keyboard through the audience terminal when watching the live broadcast, and the anchor input operation will be It is reproduced in the virtual keyboard on the screen of the audience terminal and displayed with special effects. Through the technical solution of the present application, the audience can intuitively watch which buttons the anchor has operated, and study the anchor's operating skills, operating hand speed, etc., which is highly practical for learning audiences.

As shown in FIG. 1A , it is a schematic diagram of an application scenario of live broadcasting a local computer game in an embodiment of the present invention, and the CDN server is not shown. The anchor terminal (such as smart phone, tablet computer, laptop, desktop, etc.) communicates with the central control server, and the input devices used by the anchor terminal include peripheral keyboard and mouse, peripheral handle, etc. The central control server communicates with several audience terminals (such as smart phones, tablet computers, notebook computers, desktop computers, etc.). Push and pull streaming servers provide live broadcast services for broadcaster terminals and viewer terminals.

As shown in Figure 2, the host terminal in Figure 1 implements the following method for sharing the host input operation of the computer game live broadcast:

S21: Capture the host input operation data generated by the host operating at least one input device;

Among them, the game runs locally, the input device is generally a physical input device, and the anchor input operation data only includes the operation data input by the physical input device, including: mouse pointer type, event, coordinates, as well as keyboard operation events, handle operation events Wait.

Preferably, before capturing the input operation data of the anchor, first detect whether there is an instruction to share the anchor input operation; when detecting the instruction to share the anchor input operation, identify the device information of the at least one input device, and The identification information of the at least one input device is sent to the central control server.

For example, the host selects the sharing input operation in the host terminal, the host terminal determines the type of the input device used, and sends the input device ID to the central control server. When receiving the live broadcast request from the host, the central control verifies the validity of the host information after Return the push stream address to it.

S22: Add a time stamp at the time of capturing to each of the anchor input operation data, and send it to the streaming server for streaming.

Specifically, the anchor terminal captures the anchor input operation data, including: 01 mouse operation data (press and lift event, moving relative coordinates, absolute coordinate data, etc.), 03 peripheral keyboard, peripheral handle operation data (keyboard input event (virtual key code value, press and lift event), handle input event (press virtual key code value, press and lift event, joystick operation offset value)), and timestamp each operation data when it is captured. Subsequently, the anchor terminal pushes the captured anchor input operation data according to the streaming address returned by the central control server.

As shown in FIG. 1B , it is a schematic diagram of an application scenario of live streaming cloud games in another embodiment of the present invention. Different from FIG. 1A , in this embodiment, a remote desktop server is also communicatively connected between the host terminal and the streaming server, the game runs on the remote server, and the remote server provides game running data to the host terminal for display. In this way, the host input operation data may be peripheral device operation data or virtual keyboard operation data.

In this embodiment, different from the above method embodiments, step S21 captures the data generated by the host operating the physical input device and the virtual device (which is activated by the host terminal, such as virtual keyboard, virtual mouse, virtual handle, etc.). The anchor inputs operation data, such as: cursor push data, 02 virtual device operation events (virtual key code value, press-up event, etc.). In addition, step S22 is changed to: the anchor terminal sends the operation data input by each anchor to the remote cloud desktop server, and the remote desktop server first adds a timestamp when the operation data is received, and then sends it to the remote desktop server. The streaming server described above is used for streaming.

In one embodiment, if the input device is a virtual device started by the host terminal, the host terminal needs to determine whether the virtual device is a virtual device downloaded from the central control server and uploaded by other terminal devices; If not, save the customized virtual device and upload it to the central control server.

For example, before the host uses the virtual keyboard of the host terminal for live broadcast, if the virtual keyboard is downloaded from the central control server and uploaded by others, there is no need to save the virtual keyboard and upload it to the central control server. If the virtual keyboard is customized in the host terminal, the virtual keyboard needs to be saved and uploaded to the central control server.

As shown in FIG. 3 , the viewer terminal in FIGS. 1A and 1B executes the following method for displaying the anchor input operation based on the game live broadcast:

S31: Submit a viewing request to the central control server to obtain the address of the streaming server provided by the central control server;

Specifically, the viewer terminal submits a viewing request to the central control server (including whether it is necessary to open the virtual keyboard to watch the host using 02 and 03 devices to input and operate the live broadcast), and the central control server verifies the legitimacy of the viewer information, and then sends a request to the viewer terminal. Returns the address of the pull stream server.

S32: Pulling the anchor input operation data from the streaming server;

Specifically, after receiving the streaming address returned by the central control server, the viewer terminal pulls the streamer input operation data from the streaming server during the live broadcast of the game.

S33: When decoding and rendering a live image frame, execute the anchor input operation data with the same timestamp according to the timestamp of the live image frame to reproduce the anchor input operation event, so as to restore the anchor input operation on the display interface.

For example, when receiving the mouse input operation data from the host, the viewer terminal draws the corresponding mouse and executes corresponding mouse operation events, such as moving, clicking, etc., when decoding and rendering live video frames of the same time stamp. Preferably, the viewer terminal adds a click special effect to the drawn mouse click, so that the viewer can clearly see that the anchor is performing a click operation.

For another example, when the viewer terminal receives the host's peripheral keyboard, peripheral handle operation, and input operation data of the virtual keyboard, it executes the execution on the virtual keyboard in its own display interface when decoding and rendering the live video frame of the same time stamp. Corresponding operation events, such as virtual button click, joystick movement, etc. Preferably, the viewer terminal applies special operation effects when performing the corresponding operation of the drawing, so that the viewer can clearly see what operation the anchor is performing.

In one embodiment, before restoring the host input operation, the viewer terminal displays, on the display interface, start-up portals of different virtual devices provided by the viewer terminal and corresponding to the identification information of the input device used by the host terminal, To display the opened virtual device after startup; or, display on the display interface the download entry of the virtual device provided by the central control server and corresponding to the identification information of the input device used by the anchor terminal, to display after downloading Downloaded virtual device.

For example, the central control server will pop up the download entry/open window corresponding to the virtual keyboard according to the received input device ID of the host terminal after receiving the viewer terminal to watch the host using 02 and 03 equipment to input the operation requirements for live broadcast (if the host uses a virtual keyboard) Operation, the viewer terminal will pop up the download portal of the virtual keyboard for the viewer to download. If the host uses a peripheral keyboard or peripheral controller, the viewer terminal will pop up the corresponding virtual keyboard as required, such as a virtual controller and a 61-key virtual keyboard. , 87-key virtual keyboard, 104-key virtual keyboard and simplified keyboard for specific games, etc. are open for audience to use).

In one embodiment, when receiving the operation data input by the anchor, the viewer terminal synchronizes the timestamp of the operation data input by the anchor with the timestamp of the live game video stream pulled from another streaming server by the CDN server. Handles restores that perform streamer input operations. It should be noted that the remote desktop server running the current game pushes the live video stream of the game to the other streaming server, and the viewer terminal accesses the CDN node through the streaming address provided by the central control server, and then sends the video stream to the other streaming server. A pull-stream server pulls the stream.

As shown in FIG. 4 , the central control server in FIGS. 1A and 1B executes the following method for sharing the host input operation based on the game live broadcast:

S41: Obtain the live broadcast request of the host terminal;

S42: For the local computer game live broadcast, in response to the live broadcast request, provide the stream push address to the host terminal, so that the host terminal can push the host input operation data to the corresponding stream push server; for the cloud game live broadcast, respond to all the The live broadcast request is provided, and the stream push address is provided to the anchor terminal, so that the anchor terminal sends the anchor input operation data and the stream push address to the remote cloud desktop server, and the remote desktop server pushes the anchor input operation data to the corresponding the push server;

S43: Obtain the viewing request of the viewer terminal;

S44: In response to the viewing request, provide the viewer terminal with the address of the streaming server, so that the viewer terminal can pull the anchor input operation data from the streaming server and reproduce the anchor input operation event.

In the embodiment of the cloud game live broadcast shown in FIG. 1B , the central control server is further configured to acquire and save the custom virtual device uploaded by the host terminal; and provide the viewer terminal with the input used by the host terminal The startup entry of the virtual device corresponding to the device or the download entry of the virtual device.

Since the specific implementation of this embodiment is in principle the same as the foregoing method embodiment, the implementation details will not be repeated here.

All or part of the steps for implementing the above method embodiments may be completed by hardware related to computer programs. Based on this understanding, the present invention also provides a computer program product comprising one or more computer instructions. The computer instructions may be stored in a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a server, data center, etc., which includes one or more available media integrated. The usable medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a Solid State Disk (SSD)), and the like.

Referring to FIG. 5 , this embodiment provides an electronic device, and the electronic device may be a desktop computer, a portable computer, a tablet computer, a smart phone, or the like. In detail, the electronic device includes at least a memory and a processor connected through a bus, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so as to execute all or part of the steps performed by the aforementioned host terminal, or Perform all or part of the steps performed by the aforementioned viewer terminal, or perform all or part of the steps performed by the aforementioned central control server.

The system bus mentioned above may be a Peripheral Pomponent Interconnect (PCI for short) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA for short) bus or the like. The system bus can be divided into address bus, data bus, control bus and so on. For ease of presentation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize the communication between the database access device and other devices (eg client, read-write library and read-only library). The memory may include random access memory (Random Access Memory, RAM for short), and may also include non-volatile memory (non-volatile memory), such as at least one disk storage.

The above-mentioned processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; may also be a digital signal processor (Digital Signal Processing, referred to as DSP) , Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

To sum up, the method and device for sharing and displaying the host's input operation based on the game live broadcast of the present invention enable the host to share his input operation during the live broadcast of the terminal game, so that the audience can intuitively see the specific operation of the host and study the operation of the host. Skill and hand speed effectively overcome various shortcomings in the existing technology and have high industrial application value.

The above-mentioned embodiments merely illustrate the principles and effects of the present invention, but are not intended to limit the present invention. Anyone skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed in the present invention should still be covered by the claims of the present invention.

Claims

A method for sharing an anchor input operation based on a live game broadcast, characterized in that it is applied to an anchor terminal; the method includes:

Capture the host input operation data generated by the host operating at least one input device;

For the local computer game live broadcast, add the time stamp when capturing the input operation data of each anchor, and send it to the streaming server for streaming; for the cloud game live broadcast, send the input operation data of each anchor to the remote cloud A desktop server, so that the remote desktop server first adds a timestamp when the operation data is received and then sends it to the streaming server for streaming.
The method of claim 1, further comprising:

Before capturing the anchor input operation data, firstly detect whether there is an instruction to share the anchor input operation;

When detecting the instruction of the input operation of the sharing host, identify the device information of the at least one input device, and send the identification information of the at least one input device to the central control server;

The address of the streaming server is provided by the central control server to the host terminal.
The method according to claim 2, wherein the at least one input device comprises: an external physical input device connected to the host terminal in the local computer live broadcast, and for the external physical input device connected to the host terminal in the cloud game live broadcast device and a virtual device started by the host terminal; for the virtual device started by the host terminal, the method further includes:

Determine whether the virtual device is a virtual device downloaded from the central control server and uploaded by other terminal devices;

If not, save the customized virtual device and upload it to the central control server.
A method for displaying an input operation of an anchor based on a live game broadcast, characterized in that it is applied to a viewer terminal; the method comprises:

Submit a viewing request to the central control server to obtain the address of the streaming server provided by the central control server;

Pulling the host input operation data of the game live broadcast from the streaming server;

When decoding and rendering a live image frame, the anchor input operation data with the same timestamp is executed according to the timestamp of the live image frame to reproduce the anchor input operation event, so as to restore the anchor input operation on the display interface.
The method according to claim 4, wherein a virtual device is displayed on the display interface, the host input operation is restored on the virtual device, and special effects are displayed.
The method of claim 5, further comprising:

displaying, on the display interface, startup entries of different virtual devices provided by the viewer terminal and corresponding to the identification information of the input device used by the host terminal, so as to display the opened virtual devices after startup; or,

The download entry of the virtual device provided by the central control server and corresponding to the identification information of the input device used by the host terminal is displayed on the display interface, so as to display the downloaded virtual device after downloading.
A method for sharing an input operation of an anchor based on a live game broadcast, characterized in that it is applied to a central control server; the method comprises:

Get the live broadcast request of the host terminal;

For local computer game live broadcast, in response to the live broadcast request, provide the host terminal with a streaming address, so that the host terminal can push the anchor input operation data to the corresponding streaming server; for cloud game live broadcast, respond to the live broadcast Request, provide the stream push address to the anchor terminal, so that the anchor terminal can send the anchor input operation data and the stream push address to the remote cloud desktop server, and the remote desktop server will push the anchor input operation data to the corresponding pusher. streaming server;

Obtain the viewing request of the viewer terminal;

In response to the viewing request, the viewer terminal is provided with the address of the streaming server, so that the viewer terminal can pull the host input operation data from the streaming server and reproduce the host input operation event.
The method according to claim 7, wherein, for cloud game live broadcast, the method further comprises:

Obtain the custom virtual device uploaded by the host terminal, and save it;

The viewer terminal is provided with a startup entry of a virtual device or a download entry of the virtual device corresponding to the input device used by the host terminal.
A computer-readable storage medium, wherein a computer program is stored, wherein, when the computer program is loaded and executed by a processor, it realizes the sharing of the anchor input operation based on the game live broadcast according to any one of claims 1 to 3 method, or the method for displaying an input operation of an anchor based on a live game as claimed in any one of claims 4 to 6, or the method for sharing an input operation of an anchor based on a live game of the game as claimed in claim 7 or 8.
An electronic device, comprising: a processor and a memory; wherein,

the memory is used to store computer programs;

The processor is configured to load and execute the computer program, so that the electronic device executes the method for sharing an anchor input operation based on a live game broadcast according to any one of claims 1 to 3, or any one of claims 4 to 6. 1. The method for displaying an input operation of an anchor based on a live game of a game, or the method for sharing an input operation of an anchor based on a live game of the game as claimed in claim 7 or 8.