CN114416330A

CN114416330A - Video redirection method, virtual machine, terminal, device and storage medium

Info

Publication number: CN114416330A
Application number: CN202111453138.3A
Authority: CN
Inventors: 万齐齐
Original assignee: Sangfor Technologies Co Ltd
Current assignee: Sangfor Technologies Co Ltd
Priority date: 2021-12-01
Filing date: 2021-12-01
Publication date: 2022-04-29

Abstract

The embodiment of the application provides a video redirection method, a virtual machine, a terminal, equipment and a storage medium, wherein the method comprises the following steps: responding to a webpage loading instruction in a first browser, and redirecting page information of the first browser to a second browser; analyzing the redirected page information in the second browser to obtain multimedia information in the page information, and redirecting the multimedia information to a terminal through the second browser so that the terminal decodes the multimedia information; rendering and displaying other page data which at least does not comprise the multimedia information in the redirected page information in the second browser to obtain a first picture; and transmitting a target picture to the terminal, wherein the target picture comprises the first picture and a second picture, and the second picture is a rendered picture except webpage information in the first browser.

Description

Video redirection method, virtual machine, terminal, device and storage medium

Technical Field

The present application relates to the field of computer application technologies, and in particular, to a video redirection method, a virtual machine, a terminal, a device, and a storage medium.

Background

In a Virtual Desktop Infrastructure (VDI) scenario, the cost of graphics card virtualization is very high, and no graphics card exists in a common virtualization server. Video decoding is required for playing web page (H5) video, a large amount of CPU (Central Processing Unit) resources of a virtualization platform are consumed, so that the concurrency of VDIs is reduced, and the cost of an enterprise server is increased. In addition, the frame rate and the image quality of H5 video playing cannot reach the playing effect of a physical machine, which results in poor experience.

Disclosure of Invention

In order to solve the foregoing technical problem, embodiments of the present application provide a video redirection method, a virtual machine, a terminal, a device, and a storage medium.

The embodiment of the application provides a video redirection method, which is applied to a virtual machine and comprises the following steps:

responding to a webpage loading instruction in a first browser, and redirecting page information of the first browser to a second browser;

analyzing the redirected page information in the second browser to obtain multimedia information in the page information, and redirecting the multimedia information to a terminal through the second browser so that the terminal decodes the multimedia information;

rendering and displaying other page data which at least does not comprise the multimedia information in the redirected page information in the second browser to obtain a first picture;

and transmitting a target picture to the terminal, wherein the target picture comprises the first picture and a second picture, and the second picture is a rendered picture except webpage information in the first browser.

The embodiment of the application provides a video redirection method, which is applied to a terminal and comprises the following steps:

receiving multimedia information of redirected page information in a second browser sent by a virtual machine and a target picture of the virtual machine; the target picture comprises a first picture and a second picture; the first picture is obtained by rendering other page data which do not include the multimedia information in the page information by the second browser; the second picture is a rendered picture except webpage information in the first browser;

and decoding the multimedia information, and displaying the multimedia information and the target picture after superposition on a first browser.

An embodiment of the present application provides a virtual machine, where the virtual machine includes:

the first redirection module is used for responding to a webpage loading instruction in a first browser and redirecting the page information of the first browser to a second browser;

the second redirection module is used for analyzing the redirected page information in the second browser to obtain multimedia information in the page information; redirecting the multimedia information to a terminal through the second browser so that the terminal decodes and displays the multimedia information;

a determining module, configured to render and display, in the second browser, other page data that does not include the multimedia information in the redirected page information, to obtain a first picture;

and the transmission module is used for transmitting a target picture to the terminal, wherein the target picture comprises the first picture and a second picture, and the second picture is a rendering picture except webpage information in the first browser.

An embodiment of the present application provides a terminal, including:

the receiving module is used for receiving multimedia information of the redirected page information in the second browser and a target picture of the virtual machine, wherein the multimedia information is sent by the virtual machine; the target picture comprises a first picture and a second picture; the first picture is obtained by rendering other page data which do not include the multimedia information in the page information by the second browser; the second picture is a rendered picture except webpage information in the first browser;

and the processing module is used for decoding the multimedia information, overlapping the multimedia information and the target picture and displaying the overlapped multimedia information and the target picture on the first browser.

An embodiment of the present application provides a video redirection device, where the video redirection device includes a memory and a processor, where the memory stores a computer program that can be run on the processor, and the processor implements the steps in the above-mentioned video redirection method when executing the program.

Embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps in the above-mentioned video redirection method.

The embodiment of the application provides a video redirection method, a virtual machine, a terminal, a device and a storage medium, by responding to the instruction of video loading in the first browser, redirecting the page information of the first browser to the second browser, parsing the redirected page information in the second browser, determining multimedia information in the redirected page information, redirecting the multimedia information to the terminal through the second browser, rendering and displaying other page data which do not comprise the multimedia information in the redirected page information in the second browser, sending rendered pictures which comprise the first picture and the second picture, namely the rendered pictures except the webpage information in the first browser to the terminal, the terminal can decode the multimedia information and superpose and display the decoded multimedia information and the target picture; therefore, under the condition that a user uses a standard browser, namely H5 video loading is carried out in a first browser, the habit of using the browser by the user can not be changed, the page in the first browser is automatically redirected on the virtual machine side, the redirected page information is analyzed by adopting a customized browser, namely a second browser, the multimedia information in the redirected page information is obtained, the multimedia information is redirected to the terminal again, and the part consuming a graphic processor in the H5 video page loading process in the virtual machine can be processed at the terminal.

Drawings

Fig. 1 is a schematic flowchart of playing a web video according to an embodiment of the present application;

fig. 2 is a schematic flowchart illustrating an implementation process of a video redirection method according to an embodiment of the present application;

fig. 3 is a schematic flowchart of another implementation of a video redirection method according to an embodiment of the present application;

fig. 4 is a schematic flowchart of another implementation flow of the web page video redirection method according to the embodiment of the present application;

fig. 5 is a schematic block diagram of a video redirection procedure according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a virtual machine according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a terminal according to an embodiment of the present application;

fig. 8 is a hardware entity diagram of a video redirection device according to an embodiment of the present disclosure.

Detailed Description

In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the attached drawings, the described embodiments should not be considered as limiting the present application, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

In the following description, references to the terms "first \ second \ third" are only to distinguish similar objects and do not denote a particular order, but rather the terms "first \ second \ third" are used to interchange specific orders or sequences, where appropriate, so as to enable the embodiments of the application described herein to be practiced in other than the order shown or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.

Before further detailed description of the embodiments of the present application, terms and expressions referred to in the embodiments of the present application will be described, and the terms and expressions referred to in the embodiments of the present application will be used for the following explanation.

H5 video: video elements on a Hyper Text Markup Language (HTML 5) page are labeled as < video >, and the video format supports formats such as MP4(MPEG-4), media files (WebM), and audio compression (OGGVObis, Ogg). H5 video obtains good support at mainstream browser and mobile terminal, can be used to online audio-visual, video live broadcast scene.

HLS protocol: the Streaming media transmission Protocol is called Http Live Streaming, and is proposed by apple inc. The protocol splits a video file into a plurality of small fragments, and the fragment information is described by an M3u8 format file (an M3U file in UTF-8 encoding format). The client continuously receives the video clips to finish playing.

MSE criterion: the Media Source extension is called Media sources Extensions API in English, and the browser is a standard for realizing the enhancement of H5 video playing capability. Based on MSE, the H5 video playing can be flexibly controlled by the scripting language (JavaScript), and video playing protocols such as HLS can be realized.

Thin terminal (Thin Client): a computing dummy terminal in a client-server network system basically does not need an application program, and the cost is lower than that of a common terminal. It communicates with the server via some protocols and then accesses the local area network. The thin client transmits the input of a mouse, a keyboard and the like to the server for processing, and the server transmits the processing result back to the thin client, namely the terminal for displaying.

In order to facilitate understanding of the technical solutions of the embodiments of the present application, the following description is made of related art of the embodiments of the present application.

An embodiment of the present application provides a process for playing a web video, and fig. 1 is a schematic view of a process for playing a web video provided in an embodiment of the present application, and as shown in fig. 1, when playing an H5 video, an H5 video playing process 10 needs to pass through the following processes: 11 page load, 12 video tag parsing, 13HLS format parsing, 14H5 video stuffing, and 15H5 video playback. That is, when H5 video playing is performed, the H5 video needs to be redirected to the client for playing, and in the related art, the H5 redirection scheme 100 scheme adopted includes:

1. page redirection 101: the browser page is redirected to the terminal according to a Uniform Resource Locator (URL) rule, so that the implementation is simple, but the terminal is required to run a complete browser, and the pressure on a Central Processing Unit (CPU) of the terminal is high.

2. Tag redirection 102: and identifying a < video > tag in the H5 page, acquiring a video source, and sending the video source to the client H5 player. The technology is easy to implement, but the HLS format video is not supported, and the conventional mainstream video sites are in the HLS format, so that the compatibility of the scheme is poor.

3. MSE hook 103: video data are captured through a JavaScript hook MSE API interface, the implementation process is complex, and HLS format video can be supported. However, the subsequent processing of the video depends on a browser, so that the terminal also needs to run the browser, and the pressure on a terminal CPU is large.

4. Video redirection 104: and acquiring the data after audio and video shunting through a customized browser, and transmitting the data to the client for playing. The client side decodes the video by using a Graphics Processing Unit (GPU), has low pressure on the CPU, and is suitable for thin terminal scenes. However, the use habit is changed by using the customized browser, and the user acceptance degree is not high.

Therefore, the following technical solutions are proposed in the embodiments of the present application, and in order to more thoroughly understand the features and technical contents of the embodiments of the present application, the embodiments of the present application will be described in detail below with reference to the accompanying drawings, which are provided for reference and are not intended to limit the embodiments of the present application.

The embodiment of the application provides a video redirection method, which is applied to a virtual machine. Fig. 2 is a schematic flow chart of an implementation of a video redirection method provided in an embodiment of the present application, and as shown in fig. 2, the video redirection method includes the following steps:

step S201: and responding to a webpage loading instruction in the first browser, and redirecting the page information of the first browser to the second browser.

Here, the first browser is a standard browser, which generally refers to a browser that provides excellent support for the CSS2.1 specification defined by W3C and is capable of perfect rendering. Current standard browsers include: opera10.10, Safari 4.0.4, Intelnet Explorer 8, Firefox 3, and Chrome, among other browsers. The page information of the first browser comprises contents such as a navigation bar, a webpage picture, a bookmark bar, an address bar, a label page and the like, and the first browser contains data of a user. The webpage loading instruction refers to a webpage loading instruction generated by a user in response to the operation of the user when a first browser in the virtual machine is opened to include the webpage. The second browser is a customized browser in the virtual machine, the page information of the second browser is different from that of the first browser, only the webpage picture in the page information of the first browser is displayed in the second browser, and the content of a bookmark bar, a theme and the like in the first browser is not displayed.

In some embodiments, the technical solution of the embodiments of the present application may be applied in a VDI scenario. The VDI is a method for virtualizing a desktop of a user by running an operating system on a server of a data center; a user interfaces with the virtual desktop via a client computing protocol from a client device (client or home computing device) to access the desktop as if it were a conventional locally installed desktop.

In some implementation manners, the played video is loaded in the first browser of the cloud desktop virtual machine, and can be redirected to the terminal computer to be played, so that the computing resources of the virtual machine are saved.

In some implementations, the user loads the H5 video in the first browser in a VDI scenario. A browser plug-in is arranged in a first browser of the virtual machine, and the loading of pages in the first browser can be monitored. When the page loading in the first browser is an instruction for performing the page loading, the page information in the first browser can be detected according to the uniform resource positioning, and under the condition that the network address in the page information of the first browser is matched with the address in the preset address library, namely when the network address in the page information of the first browser is a video playing address, the page information of the first browser is automatically redirected to the second browser.

In some realizable embodiments, in the case that the first browser plug-in monitors the page loading condition in the first browser, and determines to redirect the webpage information loaded by the first browser, the loading process of the original webpage is stopped. And simultaneously, calling a webpage redirection service, starting a customized browser in the virtual machine, namely a second browser, redirecting the page loaded by the first browser to the second browser, and displaying redirected page information in the second browser.

Step S202: analyzing the redirected page information in the second browser to obtain multimedia information in the page information, and redirecting the multimedia information to a terminal through the second browser so that the terminal decodes the multimedia information.

Here, the multimedia information may be video data and audio data included in the page information of the first browser. And the second browser receives the redirected page information and analyzes the hypertext markup language corresponding to the redirected page information. In one example, in the second browser, an application interface, that is, an audio or video operation interface, of a hypertext markup language control series provided by the second browser may be called, a hypertext transfer protocol request plug-in for the current redirected page is created, a hypertext transfer protocol request on the current redirected page is detected, in the hypertext transfer protocol request plug-in, all hypertext transfer protocol data requests issued on the current redirected page are filtered, and a video data loading request is separated therefrom to obtain multimedia information in the page information.

In some realizable embodiments, a customized browser, i.e., a second browser, may be implemented based on an open source browser (chrome), i.e., an audio or video operation interface is set in the chrome browser, and a web page parsing engine in the chrome browser may also be adjusted to obtain the second browser. After the second browser is started, the second browser filters all hypertext transfer protocol data requests sent by the current redirection page by adopting an audio or video operation interface, and separates out video data loading requests from the hypertext transfer protocol data requests to obtain audio data and video data in page information, namely multimedia information.

In some implementations, after the multimedia information is obtained, the multimedia information may be redirected to the terminal through the second browser, so that the multimedia information is decoded at the terminal and the decoded multimedia information is displayed. The terminal is a physical computer terminal used by a user, namely a computer device connected with the virtual desktop. In some embodiments, customizing a browser, namely a second browser, can be realized based on the chrome browser, and modifying an audio/video analysis engine of the chrome browser, so that a processing flow of audio/video data of an H5 page can be modified, and after the second browser analyzes the page information and obtains multimedia information, the analyzed multimedia information is redirected to the terminal again, and the multimedia information is decoded and displayed at the terminal.

In some realizable embodiments, the virtual machine sends the acquired multimedia information to a VDI client program of the terminal, the VDI client program decodes the multimedia information by using terminal hardware to obtain video pictures and audio data, and sends the video pictures to a VDI client main module, and the video pictures are synthesized into a complete screen graph by the main module and displayed on a screen of the terminal.

Step S203: and rendering and displaying other page data which at least does not comprise the multimedia information in the redirected page information in the second browser to obtain a first picture.

Here, only the multimedia data in the page information is redirected to the terminal again after the page information of the first browser is redirected to the second browser, and the other page data excluding the video picture data and the audio data and excluding the bookmark and the address bar of the user in the redirected page information may be acquired after the multimedia data is redirected to the terminal again.

After the second browser analyzes the redirected page information, multimedia information in the redirected page information is obtained, and after the multimedia information in the redirected page information is sent to the terminal, other page data which do not include the multimedia information in the redirected page information can be obtained, and the other page data are rendered and displayed in the second browser. That is, the web page picture which does not include multimedia information in the redirected page and does not include the bookmark and the address bar of the user is finally displayed in the second browser. Thus, the part of the H5 loaded video page consuming the CPU can be processed in the virtual machine, and the part consuming the GPU can be processed in the terminal.

In the second browser, the hypertext transfer protocol documents corresponding to other page data are analyzed into a node tree, then a style sheet is processed to form a cascading style sheet model, then the node tree and the cascading style sheet model are combined into a rendering tree, and finally each node of the rendering tree is drawn on a screen of the second browser to realize rendering and displaying of other page data to obtain a first picture.

And after the second browser analyzes the redirected page data, redirecting the multimedia information again, and rendering and displaying other page data at least not including the multimedia information in the second browser of the virtual machine server. Therefore, the rendering work can be performed in the second browser of the virtual machine, and the load of the CPU of the virtualization server is reduced.

Step S204: and transmitting a target picture to the terminal, wherein the target picture comprises the first picture and a second picture, and the second picture is a rendered picture except webpage information in the first browser.

Here, after the page information of the first browser is redirected for the first time, a second screen may be obtained by rendering a screen formed by the bookmark, the address bar, and the navigation bar from which the page information is removed by the first browser.

After obtaining the target pictures, namely the first picture and the second picture, the virtual machine may transmit the target pictures to the terminal in the following two ways:

the first method is as follows: synthesizing the first picture and the second picture to obtain the target picture; and transmitting the target picture to the terminal.

After the target pictures, namely the first picture and the second picture, are obtained, the first picture and the second picture can be synthesized in the virtual machine to obtain a first complete picture, namely the target picture, and then the target picture is sent to the terminal, and then the terminal superimposes the decoded multimedia information and the second complete picture to be drawn into a final picture displayed in a first browser of the terminal.

The second method comprises the following steps: and respectively transmitting the first picture and the second picture to the terminal.

After obtaining the target pictures, i.e., the first picture and the second picture, the virtual machine may transmit the first picture and the second picture to the terminal, respectively, so that the terminal synthesizes the first picture and the second picture to obtain a second complete picture, and then the terminal superimposes the decoded multimedia information and the second complete picture to draw a final picture displayed in a first browser of the terminal.

In some realizable embodiments, after the virtual machine redirects the multimedia information to the terminal by using the second browser, the contents of the first browser and the second browser are merged in the virtual machine, the merged picture of the first browser and the second browser is displayed in the virtual machine, and the picture of the second browser is displayed on the upper layer of the first browser.

In the embodiment of the application, when a user uses a standard browser, that is, performs H5 video loading in a first browser, the user can automatically redirect the page in the first browser on the virtual machine side without changing the habit of using the browser, the customized browser, that is, a second browser, is adopted to analyze the redirected page information, obtain the multimedia information in the redirected page information, redirect the multimedia information to a terminal, and process the part consuming the GPU in the loading process of the H5 video page in the virtual machine at the terminal, so that the consumption of CPU resources of a virtualization platform can be reduced, the number of concurrent users is increased, and the overall experience is consistent with that of a physical machine.

In some embodiments, when the video is loaded in the first browser, the redirection of the page information of the first browser to the second browser may be implemented by the following steps, that is, the step S101 may be implemented by the steps S1011 and S1012:

step S1011: and responding to the webpage loading instruction, and acquiring the page information in the first browser.

Here, a browser plug-in is provided in the first browser of the virtual machine, and the loading of the page in the first browser can be monitored. When the page loading in the first browser is an instruction for loading the web page, the browser plug-in can intercept and acquire the page information of the web page.

Step S1012: and redirecting the page information to the second browser under the condition that the page information meets a preset condition.

After the page information of the first browser is acquired, the page information in the first browser is detected by adopting uniform resource positioning, and the page information is redirected to the second browser under the condition that the page information meets preset conditions.

In some implementation embodiments, when the page information meets the preset condition, the web page redirection service is invoked, the second browser is started, the page information is redirected to the second browser, and the redirected page information is displayed in the second browser. And simultaneously stopping the loading flow of the original page in the first browser.

In the embodiment of the application, when the instruction for loading the webpage in the first browser is received and the content of the webpage information meets the preset condition, the webpage information can be redirected to the second browser, so that the redirected webpage information can be analyzed and redirected again in the second browser, and the energy consumption of a CPU (central processing unit) of the virtual machine is reduced.

In some embodiments, after the page information of the first browser is obtained, it may be determined whether to redirect the page information of the first browser to the second browser according to a uniform resource locator rule, that is, step S1012 may be implemented by:

the method comprises the following steps: and detecting a uniform resource locator in the page information.

Here, the uniform resource locator is a representation method for specifying a location of information on a web service program of the internet. A uniform resource locator is a compact representation of the location and access method of a resource available from the internet, and is the address of a standard resource on the internet. Each file on the internet has a unique uniform resource locator that contains information indicating the location of the file and how the browser should handle it.

In some implementations, the first browser is a standard browser, the page information includes content such as a navigation bar, a web page frame, a bookmark bar, an address bar, a tab page, and the like, and the uniform resource locator in the address bar in the page information can be detected.

Step two: and redirecting the page information to the second browser under the condition that the uniform resource locator is matched with an address in a preset address database.

Here, the predetermined address database includes a plurality of url addresses that can play the video. Such as: http:// player.you.com/player.php/sid/xxxxxxxxx/v.swf, www.iqiyi.com, http:// cache.tv.qq.com/qqplayrout.swfvaid ═ o013075230b, and the like. After the uniform resource locator in the redirected page information is obtained, matching the domain name system of the uniform resource locator with the domain name system of each address stored in a preset address library, indicating that the redirected page information comprises video elements under the condition of matching the domain name systems, and redirecting the page information to a second browser.

In the embodiment of the application, the uniform resource locator in the redirection page information can be detected through the uniform resource locator rule, so that the page information corresponding to the uniform resource locator matched with the address in the preset address database can be redirected.

In some embodiments, after the page information in the first browser is redirected to the second browser, the redirected page information may be parsed in the second browser to obtain the multimedia information in the redirected page information. That is, the above step S102 can be realized by the steps S1021 and S1022:

step S1021: and adopting a video analysis engine to analyze the redirected page information in the second browser to acquire audio and video data to be decoded.

Here, the second browser receives the redirected page information, invokes an application interface of a hypertext markup language control series provided by the second browser, analyzes the hypertext markup language corresponding to the page information, redirects the webpage of the first browser inside the virtual machine, and displays the webpage on the second browser. The web page picture of the first browser does not include the bookmark and address bar other page information of the user in the first browser page information. The audio-video data includes video picture data and audio data. The video analysis engine is a webpage analysis engine obtained by adjusting a chrome browser. In some embodiments, a customized browser, i.e., a second browser, may be implemented based on an open source browser (chrome), i.e., an audio or video operation interface is set in the chrome browser, and a web page parsing engine in the chrome browser is adjusted to obtain the second browser. In the second browser, an audio or video operation interface is adopted to filter all hypertext transfer protocol data requests sent from the current redirection page, and an adjusted webpage analysis engine is adopted to separate out video data loading requests from the page information, so that audio data and video picture data in the page information are obtained.

In some implementation manners, the second browser obtains audio data and video picture data after analyzing the page information by using the video analysis engine, intercepts subsequent processes, and does not decode and play the audio data and the video data.

Step S1022: and determining the audio and video data to be decoded as multimedia information in the page information.

In the embodiment of the application, the redirection page can be analyzed in the second browser through the video analysis engine, and the multimedia information in the page information of the first browser is obtained.

The embodiment of the application provides a video redirection method, which is applied to a terminal. Fig. 3 is a schematic flowchart of another implementation flow of a video redirection method according to an embodiment of the present application, and as shown in fig. 3, the video redirection method includes the following steps:

step S301: and receiving multimedia information of the redirected page information in the second browser sent by the virtual machine and a target picture of the virtual machine.

Here, the target screen includes a first screen and a second screen; the first picture is obtained by rendering other page data which do not include the multimedia information in the page information by the second browser; the second picture is a rendered picture except the webpage information in the first browser.

The terminal is computer equipment connected with the virtual desktop. And the VDI client program of the terminal receives multimedia information of the redirected page information in the second browser, which is sent by the virtual machine, namely audio and video data in the redirected page information.

After the terminal receives the target picture, if the target picture is a composite picture comprising a first picture and a second picture, the terminal can directly superimpose the target picture and the received redirected multimedia information to form a final picture displayed on a first browser of the terminal.

After the terminal receives the target picture, if the target picture is the first picture and the second picture, the terminal may synthesize the first picture and the second picture to obtain a second complete picture, and then superimpose the second complete picture with the received redirected multimedia information to form a final picture displayed on a first browser of the terminal.

Step S302: and decoding the multimedia information, and displaying the multimedia information and the target picture after superposition on the first browser.

Here, the VDI client program of the terminal decodes the multimedia information by using terminal hardware, for example, the terminal decodes the H5 video hardware by using a player module and a graphics card to obtain video pictures and audio data, and sends the video pictures and audio data to the VDI client main module, which is collectively called as a complete screen graph and displayed on the screen of the terminal. In some embodiments, a first browser is opened on a display screen of a terminal, a first picture of the first browser and a second picture of a second browser are merged after decoding multimedia information in a redirected page sent by the second browser to obtain a desktop cloud picture, and a decoded video picture is displayed on an upper layer of the desktop cloud picture.

In the embodiment of the application, the multimedia information of the redirected page information in the second browser and the target picture of the virtual machine are received through the terminal, the multimedia information is decoded, and the decoded multimedia information and the target picture are synthesized and displayed, so that a video decoding part for page loading in the first browser of the virtual machine can be processed on the terminal side, the CPU energy consumption of the virtual machine side is reduced, the compatibility of video formats can be guaranteed, the access of a low-end terminal is supported, and the terminal cost is saved.

In some embodiments, after the terminal receives the multimedia information and the target picture sent by the virtual machine through the second browser and used for redirecting the page information in the first browser, the multimedia information and the target picture can be decoded and displayed through steps S3021 and S3022:

step S3021: and decoding the multimedia information by adopting terminal hardware to obtain audio and video data.

Here, the terminal hardware is a player module and a display card of the terminal. Because the audio data and the video data are stored and transmitted by compression, otherwise, the data volume is too large, and the compression needs to be carried out by certain encoding so as to store and transmit the audio and video data with the highest quality by the minimum capacity. Therefore, when data needs to be played, the data needs to be decoded by a decoder. The audio and video data are video pictures and audio data.

In some implementations, the player module and the graphics card are used to decode video and audio to determine video pictures and audio data.

In some realizable embodiments, after the decoded video pictures and audio data are obtained, the video pictures are sent to a VDI client main module of the terminal, and the video pictures are synthesized by the main module into a complete screen graph and displayed.

Step S3022: and overlapping the audio and video data and the target picture by adopting a preset graphic transmission protocol and displaying the audio and video data and the target picture on the first browser.

Here, the preset graphic transfer protocol is at least a VDI-based graphic transfer protocol. The terminal can draw a display desktop by adopting a preset graphic transmission protocol, such as drawing pictures, drawing rectangles and the like. After multimedia information, namely video, in the page information of the first browser is redirected, a desktop of the VDI is divided into two parts, namely a video picture and a non-video picture, namely a target picture, when seen from a terminal, the desktop needs to be drawn to overlap audio and video data with the target picture, and the complete display desktop can be displayed on the terminal side.

And synthesizing the audio and video data and the target picture, redirecting the audio and video data in the page information of the first browser to the terminal, and displaying the audio and video data on a display desktop of the terminal. In some embodiments, the VDI client main module of the terminal may synthesize the audio/video data and a display desktop obtained by drawing the target picture to obtain a complete screen graph, and the complete screen graph is displayed on the display interface of the terminal.

In some implementations, the process of drawing the desktop and compositing the desktop is not perceived by the user.

In the embodiment of the application, the redirected multimedia information in the first browser can be decoded in a hard decoding mode at the terminal, and the desktop formed by multimedia and the target picture is drawn to be displayed completely, so that the energy consumption of the virtual machine server can be reduced, the decoding and displaying of multimedia information in various formats are supported, the compatibility of video redirection is ensured, and the terminal access cost is reduced.

An embodiment of the present application provides a workflow of video redirection in HLS format H5 in a VDI scene, and fig. 4 is a schematic diagram of another implementation flow of the method for redirecting web page video provided in the embodiment of the present application, as shown in fig. 4, a specific workflow is as follows:

1. default screen 41

The default screen of the first browser, i.e. the page information of the first browser, can be divided into at least three parts, i.e. a navigation bar 401, a page 402 and a video element 403.

2. After web page redirection 42

And calling a webpage redirection service inside the virtual machine, redirecting the H5 webpage to a second browser for displaying, and obtaining a webpage 404 after the webpage is redirected. The second browser may parse the page data, strip H5 the audio-visual data, and then send these data to the terminal.

3. Terminal final picture 43

After the terminal draws the desktop based on the VDI display protocol, the H5 video is displayed based on the H5 redirection protocol. From the user perspective, the H5 page is displayed in its entirety.

The operation of the default screen 41 and the web page redirection are implemented in the virtual machine 44, and the operation of the terminal final screen is implemented in the terminal 45.

The embodiment of the application provides a module schematic diagram of a video redirection process. Fig. 5 is a schematic block diagram of a video redirection procedure according to an embodiment of the present application. As shown in fig. 5, the H5 video redirection process in the present application involves the following key modules:

1. browser plug-in 501

The browser plug-in 501 is used for loading a monitoring page and determining whether the page information of the first browser needs to be redirected based on a uniform resource positioning rule. And when redirection is needed, sending the page information of the first browser to a webpage redirection service, and then stopping the original page loading process.

2. Web page redirection service 502

And after the webpage redirection service receives the page information, starting a second browser in the virtual machine and displaying the redirection page. The second browser can separate resources needing CPU rendering and multimedia information needing GPU decoding, namely audio and video data, in the page information, display the separated pages in the virtual machine, send the separated audio and video data to the terminal, and decode and display the data by the terminal.

3. H5 redirect client 503

Here, the H5 redirection client 503 processes the received redirected audio/video data, which may be implemented based on a general-purpose player, independent of the format of the browser. The redirection client 503 of H5 mainly consumes GPU resources, and it can be satisfied by checking display at the low end of the thin terminal scene.

The browser plug-in 501 and the web page redirection service module are located in the virtual machine 51, and the web page video redirection client 503 is located in the terminal 52.

In the embodiment of the application, the page information of the first browser, namely the part of the H5 page consuming the CPU, is processed in the virtual machine, and the part of the GPU is processed in the terminal, so that the number of concurrent users can be increased, the operation habit does not need to be adjusted by a user, and the overall experience is consistent with that of a physical machine.

An embodiment of the present application provides a virtual machine, and fig. 6 is a schematic structural diagram of the virtual machine provided in the embodiment of the present application, and as shown in fig. 6, the virtual machine 600 includes:

a first redirection module 601, configured to redirect page information of a first browser to a second browser in response to a web page loading instruction in the first browser;

a second redirection module 602, configured to parse the redirected page information in the second browser to obtain multimedia information in the page information, and redirect the multimedia information to a terminal through the second browser, so that the terminal decodes the multimedia information;

a determining module 603, configured to render and display, in the second browser, other page data that at least does not include the multimedia information in the redirected page information, so as to obtain a first picture;

a transmission module 604, configured to transmit a target picture to the terminal, where the target picture includes the first picture and a second picture, and the second picture is a rendered picture of the first browser except for web page information.

In the above virtual machine 600, the first redirection module 601 includes:

and the first obtaining sub-module is used for responding to the webpage loading instruction and obtaining the page information in the first browser.

And the redirection submodule is used for redirecting the page information to the second browser under the condition that the page information meets the preset condition.

In the above virtual machine 600, the redirection sub-module includes:

and the detection unit is used for detecting the uniform resource locator in the page information.

And the redirection unit is used for redirecting the page information to the second browser under the condition that the uniform resource locator is matched with an address in a preset address database.

In the virtual machine 600, the second redirection module 602 includes:

and the analysis submodule is used for analyzing the redirected page information by adopting a video analysis engine in the second browser to acquire audio and video data to be decoded.

And the first determining submodule is used for determining the audio and video data to be decoded as the multimedia information in the page information.

In the virtual machine 600, the transmission module 604 includes:

and the second determining submodule is used for synthesizing the first picture and the second picture to obtain the target picture.

The first transmission submodule is used for transmitting the target picture to the terminal; or the like, or, alternatively,

and the second transmission submodule is used for respectively transmitting the first picture and the second picture to the terminal.

An embodiment of the present application provides a terminal, and fig. 7 is a schematic structural diagram of the terminal provided in the embodiment of the present application, and as shown in fig. 7, the terminal 700 includes:

a receiving module 701, configured to receive multimedia information of redirected page information in a second browser sent by a virtual machine, and a target picture of the virtual machine; the target picture comprises a first picture and a second picture; the first picture is obtained by rendering other page data which do not include the multimedia information in the page information by the second browser; the second picture is a rendered picture except the webpage information in the first browser.

A processing module 702, configured to decode the multimedia information, and display the multimedia information and the target picture in the first browser after superimposing them.

In the terminal 700, the processing module 702 includes:

and the decoding submodule is used for decoding the multimedia information by adopting terminal hardware to obtain audio and video data.

And the processing submodule is used for overlapping the audio and video data and the target picture by adopting a preset graphic transmission protocol and displaying the audio and video data and the target picture on the first browser.

Correspondingly, an embodiment of the present application provides a video redirection device 800, for example, a computer device, fig. 8 is a schematic diagram of hardware entities of the video redirection device provided in the embodiment of the present application, and as shown in fig. 8, the hardware entities of the device 800 include: a processor 801, a communication interface 802, and a memory 803, wherein:

the processor 801 generally controls the overall operation of the device 800.

The communication interface 802 may enable the device 800 to communicate with other terminals or servers via a network.

The Memory 803 is configured to store instructions and applications executable by the processor 801, and may also buffer data (e.g., image data, audio data, voice communication data, and video communication data) to be processed or already processed by the processor 801 and modules in the device 800, and may be implemented by a FLASH Memory (FLASH) or a Random Access Memory (RAM).

The embodiment of the present application further provides a video redirection device, which includes modules, sub-modules and units included in the modules, and can be implemented by a processor in a terminal; of course, the implementation can also be realized through a specific logic circuit; in the implementation process, the processor may be a CPU, a Microprocessor Unit (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like.

It should be noted that the above description of the terminal embodiment is similar to the description of the method embodiment, and has similar beneficial effects to the method embodiment. For technical details not disclosed in the embodiments of the terminal of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

Correspondingly, the present application provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps in the video redirection method provided in the above embodiments.

Here, it should be noted that: the above description of the storage medium and device embodiments is similar to the description of the method embodiments above, with similar advantageous effects as the method embodiments. For technical details not disclosed in the embodiments of the storage medium and apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.

Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.

Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the related art may be embodied in the form of a software product stored in a storage medium, and including several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.

The above description is only for some embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of changes or substitutions within the technical scope of the present application, and all such changes or substitutions are included in the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A video redirection method is applied to a virtual machine, and comprises the following steps:

2. The method of claim 1, wherein redirecting page information of a first browser to a second browser in response to a web load instruction in the first browser comprises:

responding to the webpage loading instruction, and acquiring page information in the first browser;

and redirecting the page information to the second browser under the condition that the page information meets a preset condition.

3. The method according to claim 2, wherein the redirecting the page information to the second browser in the case that the page information satisfies a preset condition comprises:

detecting a uniform resource locator in the page information;

and redirecting the page information to the second browser under the condition that the uniform resource locator is matched with an address in a preset address database.

4. The method of claim 1, wherein parsing the redirected page information in the second browser to obtain multimedia information in the page information comprises:

adopting a video analysis engine to analyze the redirected page information in the second browser to acquire audio and video data to be decoded;

and determining the audio and video data to be decoded as multimedia information in the page information.

5. The method of claim 1, wherein transmitting a target picture to the terminal, the target picture comprising the first picture and a second picture, comprises:

synthesizing the first picture and the second picture to obtain the target picture;

transmitting the target picture to the terminal; or the like, or, alternatively,

and respectively transmitting the first picture and the second picture to the terminal.

6. A video redirection method is applied to a terminal, and comprises the following steps:

and decoding the multimedia information, and displaying the multimedia information and the target picture after superposition on the first browser.

7. The method of claim 6, wherein the decoding the multimedia information and displaying the multimedia information and the target picture after superimposing the multimedia information and the target picture on the first browser comprises:

decoding the multimedia information by adopting terminal hardware to obtain audio and video data;

and overlapping the audio and video data and the target picture by adopting a preset graphic transmission protocol and displaying the audio and video data and the target picture on the first browser.

8. A virtual machine, comprising:

9. A terminal, characterized in that the terminal comprises:

10. A video redirection device comprising a memory and a processor, the memory storing a computer program operable on the processor, wherein the processor implements the steps of the video redirection method of any of claims 1 to 5 when executing the program, or the processor implements the steps of the video redirection method of claims 6 or 7 when executing the program.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the video redirection method as claimed in any one of the claims 1 to 5, or which, when being executed by a processor, carries out the steps of the video redirection method as claimed in claim 6 or 7.