CN117939175A

CN117939175A - Live broadcast method and device based on terminal equipment, electronic equipment and storage medium

Info

Publication number: CN117939175A
Application number: CN202311760051.XA
Authority: CN
Inventors: 葛家澍
Original assignee: Netease Hangzhou Network Co Ltd
Current assignee: Netease Hangzhou Network Co Ltd
Priority date: 2023-12-19
Filing date: 2023-12-19
Publication date: 2024-04-26

Abstract

The present application relates to the field of terminal applications, and in particular, to a live broadcast method and apparatus based on a terminal device, an electronic device, and a storage medium. The method and the system establish the virtual screen by responding to the live broadcast request of the host user, further, render the region content to be live broadcast to the virtual screen, further, render the image corresponding to each frame of the region content to be live broadcast acquired from the virtual screen to the image view component for display to the host user, and push the region content to be live broadcast acquired from the virtual screen to the server for the server to live broadcast the region content to the audience terminal, wherein the region content to be live broadcast can be any website content loaded by the target application program through the webpage view. In this way, the application can realize live broadcast of any content to be live broadcast through the virtual screen technology, and the live broadcast image is displayed to the anchor user through the image view component, so that the efficiency of live broadcast display can be improved, and the requirement of the anchor user can be met.

Description

Live broadcast method and device based on terminal equipment, electronic equipment and storage medium

Technical Field

The present application relates to the field of terminal applications, and in particular, to a live broadcast method and apparatus based on a terminal device, an electronic device, and a storage medium.

Background

With the development of hardware and communication network technologies of mobile devices such as smart phones and Ipad, more users live through the mobile devices. The live broadcast platform is used as a medium between the anchor and the audience, so that a plurality of audiences can watch live broadcast contents of the anchor through a network at the same time, and one network live broadcast can realize the interaction between the anchor and tens of thousands and hundreds of thousands of audiences.

Currently, some users directly live the content shot by the camera of the mobile device, live the video watched by the users to other users, or live the process of playing games to other users. In the related technical scheme, the adopted live broadcasting technology is mainly live broadcasting realized by means of screenshot, screen recording and the like, wherein the live broadcasting is realized by means of live broadcasting of content which is recorded or intercepted and is irrelevant to live broadcasting content, or the live broadcasting is not recorded for some special content such as web games, videos and the like, and the efficiency of recording or capturing images is generally low, so that the requirements of a host user are difficult to meet.

Disclosure of Invention

In view of this, the embodiments of the present application at least provide a live broadcast method, apparatus, electronic device, and storage medium based on terminal devices, which can implement live broadcast of any content to be live broadcast, and improve the efficiency of live broadcast display, so as to meet the needs of hosting users.

The application mainly comprises the following aspects:

in a first aspect, an embodiment of the present application provides a live broadcast method based on a terminal device, where the method includes: responding to a live broadcast request of a host user for the content of the region to be live broadcast, and creating a virtual screen; the region content to be live broadcast is any region content of a target application program or any website content loaded by the target application program through a webpage view; rendering the content of the region to be live broadcast to a virtual screen; and acquiring an image corresponding to the region content to be live broadcast of each frame from the virtual screen, rendering the image corresponding to the region content to be live broadcast of each frame to an image view component to be displayed to the anchor user, and pushing the region content to be live broadcast acquired from the virtual screen to a server so that the server can live broadcast the region content to be live broadcast to a viewer terminal.

In a second aspect, an embodiment of the present application further provides a live broadcast apparatus based on a terminal device, where the apparatus includes: the creation module is used for responding to a live broadcast request of a host user for the content of the region to be live broadcast and creating a virtual screen; the region content to be live broadcast is any region content of a target application program or any website content loaded by the target application program through a webpage view; the rendering module is used for rendering the content of the region to be live broadcast to a virtual screen; the live broadcast module is used for acquiring images corresponding to the region content to be live broadcast of each frame from the virtual screen, rendering the images corresponding to the region content to be live broadcast of each frame to an image view component to be displayed to the anchor user, and pushing the region content to be live broadcast acquired from the virtual screen to a server so that the server can live broadcast the region content to be live broadcast to a spectator terminal.

In a third aspect, an embodiment of the present application further provides an electronic device, including: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine readable instructions being executable by the processor to perform the steps of the terminal device based live method described in the first aspect or any of the possible implementation manners of the first aspect.

In a fourth aspect, the embodiment of the present application further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor performs the steps of the terminal device based live broadcast method described in the first aspect or any possible implementation manner of the first aspect.

According to the live broadcasting method, the device, the electronic equipment and the storage medium based on the terminal equipment, the virtual screen is created by responding to the live broadcasting request of the host user, the content of the area to be live broadcast is rendered to the virtual screen, further, the image corresponding to each frame of the content of the area to be live broadcast obtained from the virtual screen is rendered to the image view component and displayed to the host user, and the content of the area to be live broadcast obtained from the virtual screen is pushed to the server so that the server can live broadcast the content of the area to be live broadcast to the audience terminal.

In order to make the above objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 shows a schematic diagram of interaction between ends in a live broadcast system based on terminal equipment according to an embodiment of the present application;

Fig. 2 shows a flowchart of a live broadcast method based on a terminal device according to an embodiment of the present application;

fig. 3 shows a flowchart of another live broadcast method based on a terminal device according to an embodiment of the present application;

Fig. 4 shows one of functional block diagrams of a live broadcast device based on a terminal device according to an embodiment of the present application;

Fig. 5 shows a second functional block diagram of a live broadcast device based on a terminal device according to an embodiment of the present application;

Fig. 6 shows a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for the purpose of illustration and description only and are not intended to limit the scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. A flowchart, as used in this disclosure, illustrates operations implemented according to some embodiments of the present application. It should be appreciated that the operations of the flow diagrams may be implemented out of order and that steps without logical context may be performed in reverse order or concurrently. Moreover, one or more other operations may be added to or removed from the flow diagrams by those skilled in the art under the direction of the present disclosure.

In addition, the described embodiments are only some, but not all, embodiments of the application. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art based on embodiments of the application without making any inventive effort, fall within the scope of the application.

In order to enable those skilled in the art to make and use the present disclosure, the following embodiments are provided in connection with a particular application scenario "webcast", and the generic principles defined herein may be applied to other embodiments and applications scenarios to one skilled in the art without departing from the spirit or scope of the present disclosure.

The method, the device, the electronic equipment or the computer readable storage medium can be applied to any scene requiring network live broadcast, the embodiment of the application does not limit specific application scenes, and any scheme using the live broadcast method and the device based on the terminal equipment provided by the embodiment of the application is within the protection scope of the application.

It should be noted that, before the present application proposes a related technical solution, the adopted live broadcasting technology is mainly live broadcasting implemented by means of screenshot, screen recording, etc., where the live broadcasting is performed by recording or intercepting content unrelated to the live broadcasting content, or where some special content, such as web games, video, etc., cannot be recorded, and the efficiency of recording or capturing images is generally low, so that it is difficult to meet the needs of the hosting user.

In the related technical scheme, for example, in a live broadcast mode, a screen recording function provided by an Android system is used for realizing screen recording of the whole mobile phone, and then push live broadcast is realized on recorded content, but if a user switches to an application program (APPlication, APP), content irrelevant to an APP is also recorded, and in the live broadcast mode, because the recorded content is the recording of the screen, the recorded content is the whole content of the APP, and the live broadcast of the content of a partial area of the APP cannot be realized. In another live broadcast mode, the content to be recorded in the APP is drawn onto a Canvas (Canvas) through an application programming interface (APPlication Programming Interface, API) provided by a system to obtain the content drawn on the Canvas, and then push live broadcast is realized on the obtained content, but in the live broadcast mode, the API can only record part of the content, and for some special content such as web games, video and the like, the API can not record the content; in a live broadcast mode, screenshot of content which needs to be broadcast by the APP is realized by using a screenshot (PixelCopy) provided by an Android system, and push live broadcast is realized on the intercepted content.

In view of the above problem, the embodiment of the present application creates a virtual screen by responding to a live broadcast request of a host user, further renders the region content to be live broadcast to the virtual screen, further renders an image corresponding to each frame of the region content to be live broadcast acquired from the virtual screen to an image view component for display to the host user, and pushes the region content to be live broadcast acquired from the virtual screen to a server for the server to live broadcast the region content to a viewer terminal, where the region content to be live broadcast may be any website content loaded by a target application program through a web page view. In this way, the application can realize live broadcast of any content to be live broadcast through the virtual screen technology, and the live broadcast image is displayed to the anchor user through the image view component, so that the efficiency of live broadcast display can be improved, and the requirement of the anchor user can be met.

It should be noted that, in the following embodiments of the present application, the content in the region to be live of the mobile terminal in the Android system is mainly live, and the anchor user may live the content played on the mobile terminal, so that the audience user of other mobile terminals may watch the content played on the screen of the mobile terminal of the anchor user. For example, a video being watched by a host user or a game being played may be live such that other users may watch the video being played by the host user or the course of playing the game. Of course, the method is not limited to video or game, and other content may be live broadcast by the anchor user, for example, an operation procedure of the mobile terminal by the anchor user, which is not limited in detail in the embodiment of the present application.

The electronic device in the embodiment of the invention can be a terminal device. The terminal device may be a mobile terminal, and the mobile terminal may be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, an ultra mobile personal computer, a netbook, a personal digital assistant, or the like.

In order to facilitate understanding of the present application, the following detailed description of the technical solution provided by the present application is provided in connection with specific embodiments.

Fig. 1 shows a schematic diagram of interaction between each end in a live broadcast system based on terminal devices, where the live broadcast system based on terminal devices includes terminal devices corresponding to anchor users, audience terminals corresponding to a plurality of audience users, and a server, and the terminal devices may be mobile terminals. At the mobile terminal of the anchor user, the target application program renders the region content to be live broadcast (such as any website content loaded by the webpage view) to a virtual screen by utilizing the virtual screen technology of the Android system, and an image of each frame of the region content to be live broadcast (namely, an image corresponding to the region content to be live broadcast displayed by the image view component) is acquired through the virtual screen and is displayed to the anchor user. And then, the acquired content of the region to be live broadcast is pushed to a server in a push mode, and audience terminals in the application program realize live broadcast watching through server push. Wherein the audience terminals are mobile terminals of other users (audiences) than the anchor user, the three audience terminals shown in fig. 1 are only exemplary, and the number of the audience terminals is not limited in the embodiment of the present application.

Fig. 2 is a flowchart of a live broadcast method based on a terminal device according to an embodiment of the present application. As shown in fig. 2, the live broadcast method based on terminal equipment provided by the embodiment of the application can be applied to a mobile terminal, and the method comprises the following steps:

S201: and responding to a live broadcast request of a host user for the content of the live broadcast area, and creating a virtual screen.

The region content to be live broadcast is any region content of a target application program or any website content loaded by the target application program through a webpage view.

In a specific implementation, the live user may initiate a live broadcast request through a target application program in the mobile terminal, for example, when the live user plays a video or a game with the target application program, the target application program may provide a live broadcast button for live broadcasting the video or the game that is being played by the target application program, specifically, when an operation that the live user clicks the live broadcast button is detected, the detected operation that the live user clicks the live broadcast button is used as a live broadcast request for the content of the area to be live, which is input by the live user. Here, other manners than the above manner of receiving the live broadcast request input by the anchor user may be adopted, and this embodiment is not particularly limited thereto.

Here, the content of any area of the target application itself may be the content played by the target application itself; any web site content that the target application loads through the web page view may be, for example, a web game, video, etc., such as an H5 game. The web page View (WebView) can be embedded in the mobile terminal, the WebView is a special View on the Android system, and can be used for displaying the content of a website, playing videos and playing web games, and the WebView class can be used for displaying only an online web page in the APP or can be used as a browser, so that the APP can load any website content through the WebView.

It should be noted that, the live broadcast manner provided in the embodiment of the present application is different from the traditional live broadcast manner, and is mainly reflected in different collection manners of live broadcast content, specifically, the traditional live broadcast manner is mainly that a host user collects live broadcast content by using a camera and live broadcast content by using content shot by the camera, while the embodiment of the present application mainly collects content in a live broadcast area by using a virtual screen technology, so that live broadcast of content in any area of a target application program itself or content in any website loaded by the target application program through a webpage view can be realized.

It should be further noted that, the Virtual screen (Virtual Display) is created according to a live broadcast request, and in implementation of the specific scheme, after receiving a live broadcast request input by a host user, the Virtual screen is created according to the live broadcast request, so as to render the content of the region to be live broadcast of the target application program onto the Virtual screen, so as to realize collection of the content of the region to be live broadcast. The virtual screen can be understood as a virtual display screen, is an API built in the Android system, allows the APP to render any content onto an independent invisible drawing surface, namely, any content desired to be displayed can be displayed, but the virtual screen is not a screen in the true sense, and a user cannot feel touch, namely, the content rendered onto the virtual screen is invisible to the user, and the user needs to see the content of the region to be live broadcast on the physical screen.

Here, the mobile terminal has a dual-screen display function including a physical screen, which is a physical screen of the mobile terminal itself, and a virtual screen, which is a window where a user interacts with the mobile terminal, all operations of the user are presented through the physical screen, for example, playing a video, a game, and other operations of the user. Specifically, in practical application, each application is an independent single layer, after the layers of the application are synthesized on the physical screen of the mobile terminal, the user can watch the corresponding content, such as video or game, for example, after the screen content is drawn on the physical screen, the user can perform the corresponding operation on the game interface.

S202: and rendering the content of the region to be live broadcast to a virtual screen.

In specific implementation, after a virtual screen is created in response to a live broadcast request of a live broadcast user for live broadcast area content, the live broadcast area content can be rendered on the virtual screen to acquire the live broadcast area content.

Here, only the content of the region to be live broadcast can be acquired through the virtual screen technology, and other acquired content covered on the region to be live broadcast can be avoided; in addition, special contents such as video, web games and the like loaded on the web page view can be acquired through a virtual screen technology; meanwhile, the scheme provided by the embodiment of the application can be compatible with the Android systems of all versions of the main stream on the market.

S203: and acquiring an image corresponding to the region content to be live broadcast of each frame from the virtual screen, rendering the image corresponding to the region content to be live broadcast of each frame to an image view component to be displayed to the anchor user, and pushing the region content to be live broadcast acquired from the virtual screen to a server so that the server can live broadcast the region content to be live broadcast to a viewer terminal.

In specific implementation, the virtual screen technology of the Android system can be utilized to firstly render the region content to be live broadcast onto the virtual screen, then, the image of each frame of the region content to be live broadcast acquired from the virtual screen is rendered onto the image view component and displayed to the anchor user through the physical screen, and meanwhile, the region content to be live broadcast acquired from the virtual screen is pushed to the server in a pushing way, so that the audience terminal corresponding to the target application program can realize watching live broadcast through server pulling.

It should be noted that, since the content rendered on the virtual screen of the mobile terminal is invisible to the anchor user, the anchor user needs to see the content on the virtual screen, specifically, the images corresponding to the content in the to-be-live area of each frame may be sequentially acquired from the virtual screen, and the images corresponding to the content in the to-be-live area of each frame may be sequentially rendered on the image view component to be displayed to the anchor user, so that the anchor user views the content live by himself. The embodiment of the application uses the virtual screen technology of the Android system to display the content of the region to be live broadcast on the virtual screen to realize the acquisition of the content to be live broadcast, the real content of the region to be live broadcast displayed on the virtual screen is invisible to the host user, and the content of the region to be live broadcast seen by the host user is essentially the screenshot of each frame corresponding to the image of the region to be live broadcast after the image of each frame on the virtual screen is acquired and then dynamically displayed in the target application program.

Among them, the image View component (imageView) inherits from the View class, whose main function is to display pictures. In this way, the image view component is used for directly displaying the live image to the anchor user, so that the efficiency of live display can be improved, and the requirement of the anchor user can be met.

In addition, the network live broadcast also needs to enable the audience user of the audience terminal to watch the content of the region to be live broadcast by the anchor user, and the network live broadcast can be realized by pushing the target application program of the mobile terminal of the anchor user through the server, namely, the server acquires the content of the region to be live broadcast which is newly rendered by the virtual screen, and then the content of the region to be live broadcast is live broadcast through the server in a pushing mode.

Specifically, the content of the region to be live acquired by the virtual screen may be pushed to the server, and then the server pushes the content of the region to be live to the audience terminal corresponding to the target application program, so that the audience user of the audience terminal can see the content of the region to be live of the mobile terminal of the anchor user. For example, the video data may be generated according to the content of the region to be live provided by the virtual screen, and transmitted to the server, so that the server can live the video data to other audience terminals.

In the embodiment of the application, the virtual screen is created by responding to the live broadcast request of the host user, and then, the content of the region to be live broadcast is rendered to the virtual screen, further, the image corresponding to each frame of the content of the region to be live broadcast obtained from the virtual screen is rendered to the image view component to be displayed to the host user, and the content of the region to be live broadcast obtained from the virtual screen is pushed to the server so as to be live broadcast to the audience terminal by the server, wherein the content of the region to be live broadcast can be any website content loaded by the target application program through the webpage view. In this way, the application can realize live broadcast of any content to be live broadcast through the virtual screen technology, and the live broadcast image is displayed to the anchor user through the image view component, so that the efficiency of live broadcast display can be improved, and the requirement of the anchor user can be met.

Fig. 3 is a flowchart of another live broadcast method based on terminal equipment according to an embodiment of the present application. As shown in fig. 3, the live broadcast method based on terminal equipment provided by the embodiment of the application can be applied to a mobile terminal, and the method comprises the following steps:

s301: and responding to a live broadcast request of a host user for the content of the live broadcast area, and creating an image reader according to the display size of the live broadcast area.

S302: a drawing surface is obtained with the image reader.

S303: creating the virtual screen associated with the drawing surface.

In a specific implementation, the virtual screen is created in response to a live broadcast request of a host user, specifically, an image reader can be created first, then the drawing surface is acquired by using the image reader, and further, the virtual screen associated with the drawing surface is created, so that the creation of the virtual screen is realized.

Here, a specific procedure of creating an image reader will be described below, and a display service (DISPLAY SERVICE) provided by the Android system is acquired first to create an image reader (IMAGEREADER) with a width and height of a live region (i.e., a live region display size). The image reader is an API provided by the Android system, and can be created by using an API image reader.newInstance instantiation provided by the Android system according to the display size of the live broadcast area.

Describing the process of generating a drawing Surface (Surface for drawing) and a virtual screen, a separate drawing Surface may be obtained by an image reader, and then a virtual screen associated with the drawing Surface may be created by a display service, specifically, the virtual screen may be created by the drawing Surface provided by the image reader using an Android system API createVirtualDisplay, and in addition, the image reader allows an application to directly acquire graphic data rendered to the drawing Surface.

S304: rendering the region content to be live on the drawing surface associated with the virtual screen.

In the implementation, the content of the region to be live, such as any website content loaded by the target application program through the webpage view, which needs to be live, can be rendered on the virtual screen through the virtual screen technology, and the acquisition of the content of the region to be live can be realized. Specifically, a pop-up window which is displayed by using a virtual screen technology can be created through a dual-screen display service (Presentation) provided by an Android system, and finally, the content of the region to be live broadcast is set to a drawing surface associated with the created pop-up window for rendering through a setContentView (set content view) method of the dual-screen display service, wherein the setting of the pop-up window in the Android is not different from the setting of a common pop-up window in the Android, so that any content can be set as the content of the region to be live broadcast, such as a video and a web game, and the difference is that the embodiment of the application renders the content of the region to be live broadcast to the drawing surface associated with the virtual screen.

S305: and acquiring an image corresponding to the region content to be live broadcast of each frame from the virtual screen, rendering the image corresponding to the region content to be live broadcast of each frame to an image view component to be displayed to the anchor user, and pushing the region content to be live broadcast acquired from the virtual screen to a server so that the server can live broadcast the region content to be live broadcast to a viewer terminal.

The description of S305 may refer to the description of S203, and the same technical effects can be achieved, which will not be described in detail.

In a possible implementation manner, the step S203 or S305 of acquiring, from the virtual screen, an image corresponding to the region content to be live, and rendering, to an image view component, the image corresponding to the region content to be live, and displaying the image to the anchor user, includes the following steps:

Step 3051: and monitoring the update of the region content to be live broadcast on the virtual screen, and acquiring an image corresponding to the region content to be live broadcast of the current latest frame when the update is monitored.

Step 3052: and converting the image corresponding to the content of the region to be live broadcast of the current latest frame into a first bitmap image.

Step 3053: and displaying the first bitmap image to the anchor user by utilizing the image view component.

It should be noted that, in the embodiment of the present application, the content of the region to be live that is rendered onto the virtual screen is first obtained, and then the obtained content of the region to be live is rendered into the target application program, so that the anchor user sees the content of the region to be live that is to be live. The method comprises the steps of monitoring the content of the region to be live broadcast on the virtual screen continuously, acquiring an image corresponding to the content of the region to be live broadcast once when the update of the content of the region to be live broadcast is monitored each time, namely, intercepting the content of the region to be live broadcast of each frame from the virtual screen, and converting the image corresponding to the content of the region to be live broadcast of the latest frame acquired each time into a first bitmap image, so that the first bitmap image can be displayed to a host user through a physical screen by utilizing an image view component.

Specifically, through the previously created image-available listener method (screen recording, setOnImageAvailableListener) of the image reader, the update of the rendered content on the virtual screen can be monitored in the available image method (onImageAvailable), the content of the latest frame rendered on the virtual screen can be acquired through the next image method (acquireNextImage) of the image reader, and then each acquired frame of image is converted into a first bitmap image, so that the image view component of the display picture provided by the Android system can be utilized to display the rendered content on the virtual screen.

Among them, bitmap images (bitmaps), also called dot matrix images or raster images, are composed of individual dots called pixels (picture elements), which can be variously arranged and colored to constitute. The pattern bitmap is characterized in that the change of color and the fine transition of color can be represented, thereby generating realistic effects.

In a possible implementation manner, in a live broadcast process, the method further comprises the following steps:

Step a: and responding to the interaction event of the anchor user aiming at the image view component, and triggering the updating of the content of the region to be live broadcast on the virtual screen.

Step b: and triggering the updating of the image corresponding to the region content to be live broadcast of the image view component based on the updating of the region content to be live broadcast on the virtual screen.

In the live broadcast process, the anchor user may want to perform some operations on the interface of the content of the region to be live broadcast to realize interaction with the mobile terminal. However, the content displayed to the anchor user in the embodiment of the present application is essentially an image corresponding to the content of the region to be live displayed through the image view component, that is, the image seen by the anchor user is the content rendered onto the image view component, which is essentially a frame-by-frame image displayed, and the image cannot respond to the touch, input method input, and other operations of the anchor user, wherein the touch is a click operation, a long press operation, a scroll operation, and the like. Aiming at the problems, the embodiment of the application provides a solution to the problem that the operation of the anchor user cannot be responded due to the adoption of the virtual screen technical scheme, namely the problem that the operation of the anchor user cannot be responded aiming at the content to be live broadcast in the image view component can be realized, and further, when the anchor user interacts with the image view component, the update of the content of the area to be live broadcast on the virtual screen can be triggered, and the update of the corresponding image of the content of the area to be live broadcast in the image view component can be triggered based on the update of the content of the area to be live broadcast on the virtual screen. Therefore, the anchor user can not sense the picture which is displayed by the image view component and is operated by the user himself, and the anchor user can sense the direct operation of the content of the region to be live broadcast like a real interaction event.

Here, in the embodiment of the present application, when the anchor user operates and sees each frame image (non-real region content to be live) corresponding to the region content to be live displayed by the image view component, the corresponding interaction event is transferred to the real region content to be live on the virtual screen. In addition, any website content loaded by the webpage view on the virtual screen or any area content of the target application program can update the content, and finally the content is synchronized to the image view assembly seen by the anchor user, so that the anchor user cannot perceive that the anchor user can not operate the interface of the non-real area content to be live broadcast.

Wherein the interaction event includes, but is not limited to, a touch event and a text input event. The touch event includes any one of the following events: click events, long press events, scroll events.

In a possible implementation manner, after the responding to the interaction event of the anchor user for the image view component in the step a triggers the updating of the content of the region to be live on the virtual screen, the method further includes the following steps:

Step c: and acquiring updated regional content to be live broadcast from the virtual screen, and pushing the updated regional content to be live broadcast to the server so that the server can live broadcast the updated regional content to be live broadcast to the audience terminal.

In specific implementation, the server acquires updated region content to be live broadcast from the virtual screen in real time, and live-broadcasts the updated region content to be live broadcast to the audience terminal in real time. The updated content of the region to be live can be the update of the content (such as the update of the video with playing) of the region to be live, or can be the update in response to the operation of the host user.

In a possible implementation manner, if the interaction event is a touch event, in the step a, the responding to the interaction event of the anchor user for the image view component triggers the update of the content of the region to be live on the virtual screen, including the following steps:

step a1: a first location coordinate for a touch event of the image view assembly is acquired and converted to a second location coordinate.

Step a2: and distributing the touch event with the converted position coordinates to the virtual screen, and triggering the updating of the content of the region to be live broadcast on the virtual screen.

In the implementation, the touch event is distributed to the popup window of the virtual screen created before through the distribution event method (dispatchTouchEvent) of View after the touch event is subjected to coordinate conversion by the onTouchEvent method of the rewriting image View component, after the distribution is finished, the content in the popup window can respond to the touch event of a user, meanwhile, the touch events trigger refreshing of the content in the popup window because of the drawing mechanism of the Android system, and updated content is rendered onto the drawing surface obtained by the image reader created before, namely, the updating of the content in the region to be live broadcast on the virtual screen is realized.

In a possible implementation manner, the updating of the image corresponding to the region content to be live on the image view component based on the updating of the region content to be live on the virtual screen in the step b includes the following steps:

step b1: and when the update of the region content to be live broadcast on the virtual screen in response to the touch event is monitored, acquiring an image corresponding to the updated region content to be live broadcast.

Step b2: and converting the updated image corresponding to the region content to be live broadcast into a second bitmap image, and displaying the second bitmap image to the anchor user by utilizing the image view component so as to update the image corresponding to the region content to be live broadcast of the image view component.

In a specific implementation, each time the update of the region content to be live on the virtual screen is monitored, an image corresponding to the acquired updated region content to be live is displayed to a host user, wherein when the update of the region content to be live of the virtual screen in response to a touch event is monitored, the acquired image corresponding to the updated region content to be live is directly converted into a second bitmap image and displayed to the host user through an image view component, specifically, each newly rendered frame image is monitored, callback is performed at this time, the conversion of the acquired image of the latest frame into the second bitmap image is continuously executed, and the second bitmap image is rendered to the image view component, so that the host user can visually see that the interaction event is responsive.

In one possible implementation manner, if the interaction event is a text input event and the region content to be live broadcast is any website content loaded by the target application program through a webpage view; in the step a, the response to the interaction event of the anchor user for the image view component triggers the update of the content of the region to be live on the virtual screen, and the method comprises the following steps:

Creating an input connection for the image view component and the web page view; after input connection exists between the image view component and the webpage view, monitoring finger lifting actions of the text input event, and if so, determining that the anchor user is input for an input box of the webpage view; and acquiring a focus, and pulling up an input method through an input method manager to trigger updating of the content of the region to be live broadcast on the virtual screen.

In a specific implementation, when a text input event sent by a host user is received, an input connection is firstly established between an image view component and a webpage view, and then after the input connection is established, whether a finger of the text input event has a lifting action or not is monitored, if the lifting action is monitored, the host user is determined to be an input operation aiming at an input box of the webpage view, at the moment, a focus is acquired, namely, content input by the host user is acquired, and updating of the content in a region to be live broadcast on a virtual screen is triggered.

Specifically, firstly, the input method inputs a text editor method (onCheckIsTextEditor) which needs to rewrite the image view component and returns true, so that the image view component can accept the input of the input method, and meanwhile, the input connection method (onCreateInputConnection) is rewritten on the image view component, and onCreateInputConnnection of the region to be live, which is displayed by a rebound window, is returned in the method, so that the content input to the image view component can be input to the webpage view (WebView) through the input connection. Then, after the above steps are completed, the keyboard cannot be pulled up on the image view component to input, the OnTouchEvent method of rewriting the web page view is needed, and when the touch event of lifting the finger is monitored, whether the input box of the web page view is input by the anchor user is judged according to the result (HitTestResult) of the hit test of the web page view. Further, if so, the image view component can obtain focus through an obtained focus method (requestFocus) of the image view component, and the input method is pulled up through a display soft input method (showSoftInput) of the input method manager (InputMethodManager); if not, it may be determined whether there is currently a pop-up input method, if so, a soft input method (hideSoftInputFromWindow) from the window is required to be invoked to hide the input method by the input method manager.

Here, some compatibility problems of different Android versions are also related to the processing, and some special processing needs to be performed for the systems of different versions. Also here text input events trigger an update of the content on the virtual screen and thus the rendered image on the image view component.

Based on the same application conception, the embodiment of the application also provides a live broadcast device based on terminal equipment, which corresponds to the live broadcast method based on terminal equipment provided by the embodiment of the application.

Fig. 4 and fig. 5 show one of the functional block diagrams of a live broadcast device 400 based on a terminal device according to an embodiment of the present application, and fig. 5 shows the second functional block diagram of the live broadcast device 400 based on a terminal device according to an embodiment of the present application. As shown in fig. 4, the live broadcast apparatus based on the terminal device includes: a creation module 410, configured to create a virtual screen in response to a live broadcast request of a host user for content in a region to be live broadcast; the region content to be live broadcast is any region content of a target application program or any website content loaded by the target application program through a webpage view; a rendering module 420, configured to render the region content to be live to a virtual screen; and the live broadcast module 430 is configured to acquire an image corresponding to the region content to be live broadcast of each frame from the virtual screen, render the image corresponding to the region content to be live broadcast of each frame onto an image view component for displaying to the anchor user, and push the region content to be live broadcast acquired from the virtual screen to a server for the server to live broadcast the region content to be live broadcast to a viewer terminal.

In one possible implementation, as shown in fig. 4, the creation module 410 is specifically configured to create a virtual screen according to the following steps: responding to a live broadcast request of a live broadcast user for the content of the region to be live broadcast, and creating an image reader according to the display size of the live broadcast region; obtaining a drawing surface with the image reader; creating the virtual screen associated with the drawing surface.

In a possible implementation manner, as shown in fig. 4, the rendering module 420 is specifically configured to render the content of the region to be live to a virtual screen according to the following steps: rendering the region content to be live on the drawing surface associated with the virtual screen.

In one possible implementation, as shown in fig. 4, the live broadcast module 430 is specifically configured to: monitoring the update of the region content to be live broadcast on the virtual screen, and acquiring an image corresponding to the region content to be live broadcast of the current latest frame when the update is monitored; converting an image corresponding to the content of the region to be live broadcast of the current latest frame into a first bitmap image; and displaying the first bitmap image to the anchor user by utilizing the image view component.

In a possible implementation manner, as shown in fig. 5, the terminal device-based live broadcast apparatus 400 further includes an interaction module 440; the interaction module 440 is configured to trigger, in a live broadcast process, updating of the content of the region to be live on the virtual screen in response to an interaction event of the anchor user with respect to the image view component; and triggering the updating of the image corresponding to the region content to be live broadcast of the image view component based on the updating of the region content to be live broadcast on the virtual screen.

In one possible implementation, as shown in fig. 5, after the interaction module 440 triggers the update of the content of the region to be live on the virtual screen in response to the interaction event of the hosting user for the image view component, the live broadcast module 430 is further configured to: and acquiring updated regional content to be live broadcast from the virtual screen, and pushing the updated regional content to be live broadcast to the server so that the server can live broadcast the updated regional content to be live broadcast to the audience terminal.

In a possible implementation, as shown in fig. 5, if the interaction event is a touch event, the interaction module 440 is specifically configured to obtain a first position coordinate of the touch event for the image view component, and convert the first position coordinate into a second position coordinate; and distributing the touch event with the converted position coordinates to the virtual screen, and triggering the updating of the content of the region to be live broadcast on the virtual screen.

In a possible implementation manner, as shown in fig. 5, the interaction module 440 is specifically configured to, when monitoring the update of the content of the region to be live on the virtual screen in response to the touch event, acquire an image corresponding to the updated content of the region to be live; and converting the updated image corresponding to the region content to be live broadcast into a second bitmap image, and displaying the second bitmap image to the anchor user by utilizing the image view component so as to update the image corresponding to the region content to be live broadcast of the image view component.

In a possible implementation manner, as shown in fig. 5, if the interaction event is a text input event, and the region content to be live is any website content loaded by the target application program through a web page view; the interaction module 440 is specifically configured to create an input connection for the image view component and the web page view; after input connection exists between the image view component and the webpage view, monitoring finger lifting actions of the text input event, and if so, determining that the anchor user is input for an input box of the webpage view; and acquiring a focus, and pulling up an input method through an input method manager to trigger updating of the content of the region to be live broadcast on the virtual screen.

Based on the same application concept, referring to fig. 6, a schematic structural diagram of an electronic device 600 according to an embodiment of the present application includes: a processor 610, a memory 620 and a bus 630, said memory 620 storing machine readable instructions executable by said processor 610, said processor 610 and said memory 620 communicating through said bus 630 when said electronic device 600 is running, said machine readable instructions being executed by said processor 610 to perform the steps of a terminal device based live method as described in any of the above embodiments.

In particular, the machine-readable instructions, when executed by the processor 610, may perform the following: responding to a live broadcast request of a host user for the content of the region to be live broadcast, and creating a virtual screen; the region content to be live broadcast is any region content of a target application program or any website content loaded by the target application program through a webpage view; rendering the content of the region to be live broadcast to a virtual screen; and acquiring an image corresponding to the region content to be live broadcast of each frame from the virtual screen, rendering the image corresponding to the region content to be live broadcast of each frame to an image view component to be displayed to the anchor user, and pushing the region content to be live broadcast acquired from the virtual screen to a server so that the server can live broadcast the region content to be live broadcast to a viewer terminal.

In one possible implementation, the machine-readable instructions, when executed by the processor 610, may perform the following: responding to a live broadcast request of a live broadcast user for the content of the region to be live broadcast, and creating an image reader according to the display size of the live broadcast region; obtaining a drawing surface with the image reader; creating the virtual screen associated with the drawing surface.

In one possible implementation, the machine-readable instructions, when executed by the processor 610, may perform the following: rendering the region content to be live on the drawing surface associated with the virtual screen.

In one possible implementation, the machine-readable instructions, when executed by the processor 610, may perform the following: monitoring the update of the region content to be live broadcast on the virtual screen, and acquiring an image corresponding to the region content to be live broadcast of the current latest frame when the update is monitored; converting an image corresponding to the content of the region to be live broadcast of the current latest frame into a first bitmap image; and displaying the first bitmap image to the anchor user by utilizing the image view component.

In one possible implementation, the machine-readable instructions, when executed by the processor 610, may perform the following: in the live broadcast process, responding to an interaction event of the anchor user aiming at the image view component, and triggering the update of the content of the region to be live broadcast on the virtual screen; and triggering the updating of the image corresponding to the region content to be live broadcast of the image view component based on the updating of the region content to be live broadcast on the virtual screen.

In one possible implementation, the machine-readable instructions, when executed by the processor 610, may perform the following: and after the response of the interaction event of the anchor user aiming at the image view component triggers the update of the region content to be live broadcast on the virtual screen, acquiring updated region content to be live broadcast from the virtual screen, and pushing the updated region content to be live broadcast to the server so that the server can live broadcast the updated region content to the audience terminal.

In one possible implementation, the machine-readable instructions, when executed by the processor 610, may perform the following: if the interaction event is a touch event, acquiring a first position coordinate of the touch event aiming at the image view assembly, and converting the first position coordinate into a second position coordinate; and distributing the touch event with the converted position coordinates to the virtual screen, and triggering the updating of the content of the region to be live broadcast on the virtual screen.

In one possible implementation, the machine-readable instructions, when executed by the processor 610, may perform the following: when the update of the region content to be live broadcast on the virtual screen responding to the touch event is monitored, acquiring an image corresponding to the updated region content to be live broadcast; and converting the updated image corresponding to the region content to be live broadcast into a second bitmap image, and displaying the second bitmap image to the anchor user by utilizing the image view component so as to update the image corresponding to the region content to be live broadcast of the image view component.

In one possible implementation, the machine-readable instructions, when executed by the processor 610, may perform the following: if the interaction event is a text input event and the region content to be live broadcast is any website content loaded by the target application program through a webpage view; creating an input connection for the image view component and the web page view; after input connection exists between the image view component and the webpage view, monitoring finger lifting actions of the text input event, and if so, determining that the anchor user is input for an input box of the webpage view; and acquiring a focus, and pulling up an input method through an input method manager to trigger updating of the content of the region to be live broadcast on the virtual screen.

Based on the same application concept, the embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the steps of the live broadcast method based on the terminal device provided in the above embodiment are executed.

In a possible implementation, the computer program, when executed by a processor, performs the following processing: responding to a live broadcast request of a live broadcast user for the content of the region to be live broadcast, and creating an image reader according to the display size of the live broadcast region; obtaining a drawing surface with the image reader; creating the virtual screen associated with the drawing surface.

In a possible implementation, the computer program, when executed by a processor, performs the following processing: rendering the region content to be live on the drawing surface associated with the virtual screen.

In a possible implementation, the computer program, when executed by a processor, performs the following processing: monitoring the update of the region content to be live broadcast on the virtual screen, and acquiring an image corresponding to the region content to be live broadcast of the current latest frame when the update is monitored; converting an image corresponding to the content of the region to be live broadcast of the current latest frame into a first bitmap image; and displaying the first bitmap image to the anchor user by utilizing the image view component.

In a possible implementation, the computer program, when executed by a processor, performs the following processing: in the live broadcast process, responding to an interaction event of the anchor user aiming at the image view component, and triggering the update of the content of the region to be live broadcast on the virtual screen; and triggering the updating of the image corresponding to the region content to be live broadcast of the image view component based on the updating of the region content to be live broadcast on the virtual screen.

In a possible implementation, the computer program, when executed by a processor, performs the following processing: and after the response of the interaction event of the anchor user aiming at the image view component triggers the update of the region content to be live broadcast on the virtual screen, acquiring updated region content to be live broadcast from the virtual screen, and pushing the updated region content to be live broadcast to the server so that the server can live broadcast the updated region content to the audience terminal.

In a possible implementation, the computer program, when executed by a processor, performs the following processing: if the interaction event is a touch event, acquiring a first position coordinate of the touch event aiming at the image view assembly, and converting the first position coordinate into a second position coordinate; and distributing the touch event with the converted position coordinates to the virtual screen, and triggering the updating of the content of the region to be live broadcast on the virtual screen.

In a possible implementation, the computer program, when executed by a processor, performs the following processing: when the update of the region content to be live broadcast on the virtual screen responding to the touch event is monitored, acquiring an image corresponding to the updated region content to be live broadcast; and converting the updated image corresponding to the region content to be live broadcast into a second bitmap image, and displaying the second bitmap image to the anchor user by utilizing the image view component so as to update the image corresponding to the region content to be live broadcast of the image view component.

In a possible implementation, the computer program, when executed by a processor, performs the following processing: if the interaction event is a text input event and the region content to be live broadcast is any website content loaded by the target application program through a webpage view; creating an input connection for the image view component and the web page view; after input connection exists between the image view component and the webpage view, monitoring finger lifting actions of the text input event, and if so, determining that the anchor user is input for an input box of the webpage view; and acquiring a focus, and pulling up an input method through an input method manager to trigger updating of the content of the region to be live broadcast on the virtual screen.

Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, and the like, and when the computer program on the storage medium is run, the live broadcast method based on the terminal device can be executed, and the live broadcast image is displayed to the host user through the image view component, so that the efficiency of live broadcast display can be improved, and the requirements of the host user are met.

In the embodiments of the present application, the computer program may also execute other machine readable instructions when executed by a processor to perform the methods as described in other embodiments, and the specific implementation of the method steps and principles are referred to in the description of the embodiments and are not described in detail herein.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments provided in the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

It should be noted that: like reference numerals and letters in the following figures denote like items, and thus once an item is defined in one figure, no further definition or explanation of it is required in the following figures, and furthermore, the terms "first," "second," "third," etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.

Finally, it should be noted that: the above examples are only specific embodiments of the present application, and are not intended to limit the scope of the present application, but it should be understood by those skilled in the art that the present application is not limited thereto, and that the present application is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the corresponding technical solutions. Are intended to be encompassed within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A live broadcast method based on terminal equipment, the method comprising:

Responding to a live broadcast request of a host user for the content of the region to be live broadcast, and creating a virtual screen; the region content to be live broadcast is any region content of a target application program or any website content loaded by the target application program through a webpage view;

rendering the content of the region to be live broadcast to a virtual screen;

And acquiring an image corresponding to the region content to be live broadcast of each frame from the virtual screen, rendering the image corresponding to the region content to be live broadcast of each frame to an image view component to be displayed to the anchor user, and pushing the region content to be live broadcast acquired from the virtual screen to a server so that the server can live broadcast the region content to be live broadcast to a viewer terminal.

2. The method of claim 1, wherein creating a virtual screen in response to a live request from a host user for content in a region to be live, comprises:

Responding to a live broadcast request of a live broadcast user for the content of the region to be live broadcast, and creating an image reader according to the display size of the live broadcast region;

obtaining a drawing surface with the image reader;

creating the virtual screen associated with the drawing surface.

3. The method of claim 2, wherein the rendering the region content to be live to a virtual screen comprises:

Rendering the region content to be live on the drawing surface associated with the virtual screen.

4. A method according to any one of claims 1 to 3, wherein the capturing, from the virtual screen, an image corresponding to the content of the region to be live for each frame, and rendering the image corresponding to the content of the region to be live for each frame onto an image view component for display to the anchor user, includes:

monitoring the update of the region content to be live broadcast on the virtual screen, and acquiring an image corresponding to the region content to be live broadcast of the current latest frame when the update is monitored;

converting an image corresponding to the content of the region to be live broadcast of the current latest frame into a first bitmap image;

and displaying the first bitmap image to the anchor user by utilizing the image view component.

5. The method of claim 4, wherein during the live broadcast, the method further comprises:

responding to an interaction event of the anchor user aiming at the image view component, and triggering the update of the content of the region to be live broadcast on the virtual screen;

And triggering the updating of the image corresponding to the region content to be live broadcast of the image view component based on the updating of the region content to be live broadcast on the virtual screen.

6. The method of claim 5, wherein after triggering the update of the region content to be live on the virtual screen in response to the interaction event of the anchor user with respect to the image view component, the method further comprises:

and acquiring updated regional content to be live broadcast from the virtual screen, and pushing the updated regional content to be live broadcast to the server so that the server can live broadcast the updated regional content to be live broadcast to the audience terminal.

7. The method of claim 5, wherein, if the interaction event is a touch event, the responding to the interaction event of the anchor user for the image view component, triggering the update of the content of the region to be live on the virtual screen, comprises:

acquiring a first position coordinate of a touch event aiming at the image view component, and converting the first position coordinate into a second position coordinate;

And distributing the touch event with the converted position coordinates to the virtual screen, and triggering the updating of the content of the region to be live broadcast on the virtual screen.

8. The method of claim 7, wherein the triggering of the update of the image corresponding to the region content to be live of the image view component based on the update of the region content to be live on the virtual screen comprises:

when the update of the region content to be live broadcast on the virtual screen responding to the touch event is monitored, acquiring an image corresponding to the updated region content to be live broadcast;

And converting the updated image corresponding to the region content to be live broadcast into a second bitmap image, and displaying the second bitmap image to the anchor user by utilizing the image view component so as to update the image corresponding to the region content to be live broadcast of the image view component.

9. The method of claim 7, wherein the touch event comprises any one of the following events:

click events, long press events, scroll events.

10. The method of claim 5, wherein if the interactive event is a text input event and the region content to be live is any website content loaded by the target application program through a web page view; the response to the interaction event of the anchor user aiming at the image view component triggers the update of the content of the region to be live on the virtual screen, and the method comprises the following steps:

Creating an input connection for the image view component and the web page view;

After input connection exists between the image view component and the webpage view, monitoring finger lifting actions of the text input event, and if so, determining that the anchor user is input for an input box of the webpage view;

And acquiring a focus, and pulling up an input method through an input method manager to trigger updating of the content of the region to be live broadcast on the virtual screen.

11. A terminal device-based live broadcast apparatus, the apparatus comprising:

The creation module is used for responding to a live broadcast request of a host user for the content of the region to be live broadcast and creating a virtual screen; the region content to be live broadcast is any region content of a target application program or any website content loaded by the target application program through a webpage view;

The rendering module is used for rendering the content of the region to be live broadcast to a virtual screen;

The live broadcast module is used for acquiring images corresponding to the region content to be live broadcast of each frame from the virtual screen, rendering the images corresponding to the region content to be live broadcast of each frame to an image view component to be displayed to the anchor user, and pushing the region content to be live broadcast acquired from the virtual screen to a server so that the server can live broadcast the region content to be live broadcast to a spectator terminal.

12. An electronic device, comprising: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory in communication via the bus when the electronic device is running, the machine readable instructions when executed by the processor performing the steps of the terminal device based live method according to any of claims 1 to 10.

13. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of the terminal device based live broadcast method according to any of claims 1 to 10.