CN116112476A

CN116112476A - Multimedia playing method, system, equipment and storage medium of cloud desktop

Info

Publication number: CN116112476A
Application number: CN202310041638.9A
Authority: CN
Inventors: 孟令超; 张贵强; 张献涛; 任晋奎
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2023-01-11
Filing date: 2023-01-11
Publication date: 2023-05-12

Abstract

The embodiment of the application provides a multimedia playing method, a system, equipment and a storage medium of a cloud desktop. The method comprises the steps that a redirection server is deployed in a cloud desktop protocol server in a cloud desktop server, redirection clients are deployed in various multimedia playing terminals which are possibly used in the cloud desktop, one-to-many communication can be conducted between the redirection server and each redirection client according to a designated protocol, and the designated protocol can be compatible with various players and various browsers. Based on the system architecture, the redirection server can be compatible with various multimedia playing terminals in various playing scenes in the cloud desktop, and redirect the un-decoded media stream to the cloud desktop terminal, so that decoding and playing are performed in the cloud desktop terminal. In this way, in the embodiment of the application, multiple play scenes in the cloud desktop can be covered for multimedia redirection, decoding and other operations in the cloud desktop server are not needed, and the occupation of resources of the cloud desktop server can be effectively reduced.

Description

Multimedia playing method, system, equipment and storage medium of cloud desktop

Technical Field

The present disclosure relates to the field of communications technologies, and in particular, to a method, a system, an apparatus, and a storage medium for playing multimedia on a cloud desktop.

Background

With the popularization of cloud desktop products and the continuous enrichment of multimedia resources, the demand for multimedia playing in the cloud desktop is also increasing.

At present, the multimedia playing scheme of the cloud desktop is basically as follows: the multimedia playing end in the cloud desktop server plays multimedia, the cloud desktop is collected into images in the multimedia playing process of the cloud desktop, and the collected images are transmitted to the cloud desktop terminal for display after compression encoding. In the multimedia playing process of the cloud desktop, the operations related to playing such as multimedia decoding and the like are all carried out in the cloud desktop server, so that a large amount of CPU, GPU and network resources in the cloud desktop server are occupied, and a heavy burden is caused to the cloud desktop server.

Disclosure of Invention

Aspects of the present application provide a method, a system, a device, and a storage medium for playing multimedia on a cloud desktop, so as to reduce resource occupation of the cloud desktop server caused by multimedia playing.

The embodiment of the application provides a multimedia playing system of a cloud desktop, which comprises a cloud desktop terminal and a cloud desktop server, wherein the cloud desktop server comprises a cloud desktop protocol server and a multimedia playing end, the cloud desktop terminal comprises a cloud desktop protocol client, and the cloud desktop protocol server comprises a redirection server;

The redirection server is used for receiving an undecoded media stream sent by a redirection client running in the multimedia playing end according to a specified protocol, wherein the undecoded media stream is obtained from the multimedia playing end by the redirection client;

the cloud desktop protocol server is used for sending the undecoded media stream received by the redirection server to the cloud desktop protocol client;

the cloud desktop protocol client is used for decoding the undecoded media stream; calling a local player to play the decoded multimedia;

the multimedia playing end comprises a plurality of types of players and a plurality of types of browsers, and the specified protocol is compatible with the plurality of types of players and the plurality of types of browsers and supports a one-to-many communication mechanism.

The embodiment of the application also provides a multimedia playing method of the cloud desktop, which is applicable to a cloud desktop server, wherein the cloud desktop server comprises a cloud desktop protocol server and a multimedia playing end, the cloud desktop protocol server comprises a redirection server, and the method comprises the following steps:

receiving an undecoded media stream sent by a redirection client running in a multimedia playing end by using a redirection server contained in a cloud desktop protocol server according to a specified protocol, wherein the undecoded media stream is acquired from the multimedia playing end by the redirection client;

The undecoded media stream received by the redirection server is sent to a cloud desktop protocol client in a cloud desktop terminal through a cloud desktop protocol server, so that the cloud desktop protocol client decodes the undecoded media stream and calls a local player of the cloud desktop terminal to play the decoded multimedia;

The embodiment of the application also provides a multimedia playing method of the cloud desktop, which is suitable for the cloud desktop terminal, wherein the cloud desktop terminal comprises a cloud desktop protocol client, and the method comprises the following steps:

receiving, by the cloud desktop protocol client, an undecoded media stream sent by a cloud desktop protocol server in a cloud desktop server, where the undecoded media stream is received by a redirection server in the cloud desktop protocol server from a redirection client running in a multimedia playing end according to a specified protocol, and the redirection client obtains the undecoded media stream from the multimedia playing end;

Decoding the un-decoded media stream;

calling a local player to play the decoded multimedia;

the multimedia playing end operates in the cloud desktop server and comprises a plurality of types of players and a plurality of types of browsers, and the specified protocol is compatible with the plurality of types of players and the plurality of types of browsers and supports a one-to-many communication mechanism.

The embodiment of the application also provides a cloud desktop server, which comprises a memory and a processor;

the memory is used for storing one or more computer instructions;

the processor is coupled to the memory for executing the one or more computer instructions for:

The embodiment of the application also provides a cloud desktop terminal, which comprises a memory and a processor;

the memory is used for storing one or more computer instructions;

receiving an undecoded media stream sent by a cloud desktop protocol server in a cloud desktop server through a cloud desktop protocol client, wherein the undecoded media stream is received by a redirection server in the cloud desktop protocol server from a redirection client running in a multimedia playing end according to a specified protocol, and the redirection client acquires the undecoded media stream from the multimedia playing end;

decoding the un-decoded media stream;

calling a local player to play the decoded multimedia;

Embodiments of the present application also provide a computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the foregoing method for playing multimedia on a cloud desktop.

In the embodiment of the application, the cloud desktop protocol server side in the cloud desktop server and the cloud desktop protocol client side in the cloud desktop terminal are improved, the redirection server side is deployed in the cloud desktop protocol server side, redirection client sides are respectively deployed in various multimedia playing sides possibly used in the cloud desktop, one-to-many communication can be carried out between the redirection server side and each redirection client side according to a specified protocol, and the specified protocol can be compatible with various players and various browsers. Based on the system architecture, the redirection server can be compatible with various multimedia playing terminals in various playing scenes in the cloud desktop, and redirect the un-decoded media stream to the cloud desktop terminal, so that decoding and playing are performed in the cloud desktop terminal. In this way, in the embodiment of the application, multiple play scenes in the cloud desktop can be covered for multimedia redirection, decoding and other operations in the cloud desktop server are not needed, and the occupation of resources of the cloud desktop server can be effectively reduced.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:

fig. 1 is a schematic structural diagram of a multimedia playing system of a cloud desktop according to an exemplary embodiment of the present application;

fig. 2 is a schematic diagram of an internal exemplary structure of three exemplary multimedia playing ends according to an exemplary embodiment of the present application;

FIG. 3 is a logic diagram of determining whether to perform multimedia redirection according to an exemplary embodiment of the present application;

FIG. 4 is a logic diagram of another embodiment of determining whether to redirect multimedia according to the present application;

fig. 5 is an internal exemplary structural schematic diagram of a cloud desktop protocol server according to an exemplary embodiment of the present application;

fig. 6 is a logic schematic diagram of a data interaction manner between a cloud desktop server and a cloud desktop terminal according to an exemplary embodiment of the present application;

fig. 7 is a flowchart of a method for playing multimedia on a cloud desktop according to another exemplary embodiment of the present application;

Fig. 8 is a flowchart of another method for playing multimedia on a cloud desktop according to another exemplary embodiment of the present application;

fig. 9 is a schematic structural diagram of a cloud desktop server according to another exemplary embodiment of the present application;

fig. 10 is a schematic structural diagram of a cloud desktop terminal according to another exemplary embodiment of the present application.

Detailed Description

For the purposes, technical solutions and advantages of the present application, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

At present, the multimedia playing scheme of the cloud desktop needs to rely on the cloud desktop server to perform decoding and other processes, so that a large amount of resources of the cloud desktop server are occupied, and a heavy burden is caused to the cloud desktop server. To this end, in some embodiments of the present application: the cloud desktop protocol server in the cloud desktop server and the cloud desktop protocol client in the cloud desktop terminal are improved, a redirection server is deployed in the cloud desktop protocol server, redirection clients are respectively deployed in various multimedia playing terminals which are possibly used in the cloud desktop, one-to-many communication can be carried out between the redirection server and each redirection client according to a specified protocol, and the specified protocol can be compatible with various players and various browsers. Based on the system architecture, the redirection server can be compatible with various multimedia playing terminals in various playing scenes in the cloud desktop, and redirect the un-decoded media stream to the cloud desktop terminal, so that decoding and playing are performed in the cloud desktop terminal. In this way, in the embodiment of the application, multiple play scenes in the cloud desktop can be covered for multimedia redirection, decoding and other operations in the cloud desktop server are not needed, and the occupation of resources of the cloud desktop server can be effectively reduced.

The following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.

Fig. 1 is a schematic structural diagram of a multimedia playing system of a cloud desktop according to an exemplary embodiment of the present application. As shown in fig. 1, the system includes: cloud desktop server and cloud desktop terminal.

In this embodiment, the cloud desktop server may refer to a cloud server for supporting operation of a cloud desktop. It should be understood that the cloud desktop server may include components such as a CPU, a memory, a mirror image, and a cloud hard disk, and is a cloud server that is available at any time and elastically expandable. The cloud desktop terminal may refer to a terminal device of a cloud desktop technology, which may connect to a cloud desktop server and render a cloud desktop to a front end, and redirect output and input data such as a mouse operation of the front end to the cloud desktop server.

Referring to fig. 1, in this embodiment, a cloud desktop server and a cloud desktop terminal may communicate through a specific communication protocol, where the cloud desktop server may include a cloud desktop protocol server, and the cloud desktop terminal may include a cloud desktop protocol client. The cloud desktop protocol server side is used for realizing the specific communication protocol on the cloud desktop server side, and the cloud desktop protocol client side is used for realizing the specific communication protocol on the cloud desktop terminal side. It should be understood that the cloud desktop protocol server may be a process in the cloud desktop server, and likewise, the cloud desktop protocol client may be a process in the cloud desktop terminal.

With continued reference to fig. 1, in this embodiment, a redirection server is further deployed in the cloud desktop protocol server, where the redirection server may be a thread included in a process corresponding to the cloud desktop protocol server.

In addition, in this embodiment, the redirection client may be deployed in various multimedia playing terminals that may be used in the cloud desktop, where the multimedia playing terminals operate in the cloud desktop server, and similarly, the multimedia playing terminal may be a process in the cloud desktop server, and the redirection client deployed therein may be a thread included in the process corresponding to the multimedia playing terminal. The multimedia playing end can be an application program used for playing multimedia in the cloud desktop server. In this embodiment, the multimedia playing end may be various players or various browsers. The multimedia playing end may be, for example, a VLC player, a WMP player, a Chrome browser, an Edge browser, or a firefox browser, which are also merely exemplary, and the present embodiment is not limited thereto, and may support various multimedia playing ends that may be used in the cloud desktop at present and new types of multimedia playing ends that may be amplified in the future.

On this basis, referring to fig. 1, it is proposed in this embodiment that one-to-many communication between the redirect server and each redirect client may be performed according to a specified protocol, and the specified protocol may be compatible with various types of players and various types of browsers. Preferably, in this embodiment, the specified protocol may be a websocket protocol, and of course, in this embodiment, the specified protocol may also be a pipe protocol, and other protocols that may occur in the present or future and can meet the requirements of the foregoing protocol in this embodiment, which is not limited thereto. Thus, based on the specified protocol, the redirection service end can communicate with the redirection clients deployed by the various multimedia playing ends so as to compatibly support the redirection of the multimedia by the various multimedia playing ends.

Referring to fig. 1, a redirection client deployed in a multimedia playing end may obtain an undecoded media stream from the multimedia playing end and may send the undecoded media stream to a redirection service end according to the aforementioned specified protocol. In this way, the redirect server may receive the undecoded media streams sent by the redirect client running in various multimedia players according to a specified protocol. That is, the redirection client deployed in various multimedia playing ends possibly used in the cloud desktop may send the undecoded media stream to the redirection server in the embodiment according to the specified protocol, so that the redirection server may perform subsequent processing.

The inventor finds that in the research process, playing scenes in the cloud desktop are very various, for example, for local multimedia, a player is usually expected to be used for playing, and for multimedia on a webpage, a browser is usually expected to be used for playing, so that the types of multimedia playing ends which can be used in the cloud desktop are very many. In this embodiment, a communication mechanism is provided in which a redirection server interfaces with a plurality of redirection clients according to a specified protocol, and various play scenes in a cloud desktop can be covered by using the deployment architecture. The user can select the multimedia playing end which is expected to be used according to the requirement, and the redirection of the multimedia can be realized by multiplexing the redirection server; in addition, new types of multimedia playing ends can be continuously expanded in the cloud desktop, and the new types of multimedia playing ends can be covered only by arranging the redirection client provided in the embodiment in the new types of multimedia playing ends, so that the expandability of the multimedia playing ends in the cloud desktop can be effectively improved.

In this embodiment, the multimedia playing end can be categorized into at least three types: non-customized players, browsers, and customized redirect-specific players. Fig. 2 is a schematic diagram of an internal exemplary structure of three exemplary multimedia playback terminals according to an exemplary embodiment of the present application. With reference to fig. 2, the interaction logic between the redirect client and the redirect server will be described below for these three exemplary types, respectively.

1. Referring to fig. 2, if the multimedia playing end is any non-customized player, a player plug-in may be installed in the non-customized player. In this case, the multimedia playing end will not actually play the multimedia source file to be played, but will still read the multimedia source file to be played, for which, the player plug-in may acquire the multimedia source file to be played read by the multimedia playing end, and perform the demultiplexing process on the multimedia source file to be played to generate the audio stream and the video stream. Based on this, the player plug-in may provide the demultiplexed audio and video streams as undecoded media streams to a redirect client in the multimedia player; the redirection client can send the received audio stream and video stream to the redirection server. The demultiplexing means that an audio stream and a video stream are split from a multimedia source file to be played. An off-the-shelf player refers to a player developed by a third party.

Fig. 3 is a logic diagram for determining whether to redirect a multimedia according to an exemplary embodiment of the present application, and referring to fig. 3, in this case, an off-the-shelf player may preferentially invoke the player plug-in provided in this embodiment, and the player plug-in may be further used to: if the cloud desktop terminal is determined to be incapable of supporting the coding format corresponding to the multimedia source file to be played, de-multiplexing and subsequent processes are not executed; and feeding back a back instruction to the non-customized player to trigger the non-customized player to play the multimedia. The non-customized player can provide the coding format corresponding to the multimedia source file to be played to the player plug-in the embodiment, the player plug-in can also obtain the coding format supported in the cloud desktop terminal, and the player plug-in can determine whether the cloud desktop terminal can support the coding format corresponding to the multimedia source file to be played or not through comparison. For example, the coding format corresponding to the multimedia source file to be played is H264, but the cloud desktop terminal does not support decoding such coding format, and the foregoing rollback mechanism may be executed. That is, the non-customized server may call the player plug-in provided in this embodiment preferentially, and in the case that it is determined that the multimedia redirection cannot be performed, the non-customized player may switch to call the decoding plug-in to decode and play the multimedia source file, so as to return to the conventional play mode, so as to ensure that the multimedia play is completed normally.

2. Referring to fig. 2, if the multimedia playing end is any browser, a browser plug-in may be installed in the browser. In this case, the browser plug-in can capture the video address in the current webpage, download the video address to the multimedia source file to be played from the video address in a mode of http request and the like, and provide the downloaded multimedia source file to be played to the redirection client in the browser; the redirection client may send the multimedia source file to be played to the redirection server in this embodiment. Based on this, the redirection server is further configured to: and de-multiplexing the multimedia source file to be played, which is sent by the redirection client in the browser, so as to generate a corresponding audio stream and video stream.

In this case, since the browser plug-in is usually in JS format, the processing efficiency is not high, and therefore, the demultiplexing processing work is submitted to the redirection server in the present embodiment.

In addition, in this case, the base station,

fig. 4 is a logic diagram of another embodiment of determining whether to perform multimedia redirection. Referring to fig. 4, in this case, the browser may preferentially call the browser plug-in provided in the present embodiment, and the browser plug-in may be further used to: if the video address captured from the current webpage does not support file downloading, the operation of sending the multimedia source file to be played to the redirection server is not executed any more, and a rollback instruction is fed back to the browser to trigger the browser to play the multimedia. Wherein, preferably, in this case, the browser plug-in is further operable to: checking whether video labels exist in each webpage opened by the browser, if so, judging whether the specification of the video window reaches the specified specification according to the video labels, and if not, executing the rollback mechanism to trigger the browser to play multimedia; if so, executing the judgment of judging whether the video address captured in the current webpage supports file downloading. The video tag is the existing identification information in the browser and is used for identifying whether the video to be played exists in the webpage or not, and playing parameters such as video window specification and the like.

In some scenarios, the video address does not support file downloading, possibly due to rights issues, and therefore multimedia redirection is no longer performed and the playback is rolled back to the browser for multimedia playback. If the video address supports downloading, the multimedia source file to be played can be sent to a redirection server through a redirection client, and after the redirection server receives the multimedia source file, if the cloud desktop terminal is determined to be incapable of supporting the coding format corresponding to the multimedia source file to be played, the subsequent processing of the multimedia source file to be played is stopped; and feeding back a rollback instruction to the browser to trigger the browser to play the multimedia. In this way, the redirection server can judge whether the cloud desktop terminal supports the coding format corresponding to the multimedia source file to be played, thereby determining whether to implement the rollback mechanism.

3. Referring to fig. 2, if the multimedia playing end is a customized redirection dedicated player, the redirection dedicated player may read a multimedia source file to be played, perform demultiplexing processing on the multimedia file to be played to generate an audio stream and a video stream, and send the audio stream and the video stream as undecoded media streams to the redirection server through the redirection client. The special redirection player is preset with program logic for acquiring the multimedia file to be played and performing demultiplexing processing on the multimedia file to be played.

In this case, the program logic related to the player plug-in the non-customized player is preset in the redirection-specific player, so that the player plug-in is not required to be built in any more, and the generated audio stream and video stream can be directly provided to the redirection client as the undecoded media stream according to the preset program logic.

In addition, in this case, the redirection-specific player may be further configured to: if the cloud desktop terminal is determined to be incapable of supporting the coding and decoding type corresponding to the multimedia source file to be played, playing the multimedia to be played without performing de-multiplexing and subsequent processing. That is, the redirect-specific player also supports the aforementioned rollback mechanism.

By taking the above three types of multimedia playing terminals as examples, it can be known that, in this embodiment, for different multimedia playing terminals, transmission of an undecoded media stream can be achieved through cooperation of the redirection client and the redirection server, and when it is determined in the link that multimedia redirection cannot be performed, multimedia playing can be performed by returning to the conventional playing mode through the aforementioned returning mechanism, so as to ensure that multimedia playing is normally completed.

For a cloud desktop protocol server, the undecoded media stream received by the redirection server may be sent to the cloud desktop protocol client. As mentioned above, the redirection server may be essentially a thread in the cloud desktop protocol server, so the undecoded media stream acquired in the thread may be acquired by a process and may be sent to another process in the cloud desktop terminal (i.e. the cloud desktop protocol client) by the process. So far, the un-decoded media stream is successfully sent to the cloud desktop terminal.

On the cloud desktop terminal side, the cloud desktop protocol client can decode the received un-decoded media stream; and calling the local player to play the decoded multimedia. Therefore, the decoding work and the playing work are carried out by the cloud desktop terminal and are not carried out by the cloud desktop server, and the resource occupation of the cloud desktop server by multimedia playing can be effectively reduced.

In summary, in this embodiment, a cloud desktop protocol server in a cloud desktop server and a cloud desktop protocol client in a cloud desktop terminal are improved, a redirection server is deployed in the cloud desktop protocol server, and redirection clients are respectively deployed in various multimedia playing ends possibly used in the cloud desktop, and one-to-many communication can be performed between the redirection server and each redirection client according to a specified protocol, and the specified protocol can be compatible with various players and various browsers. Based on the system architecture, the redirection server can be compatible with various multimedia playing terminals in various playing scenes in the cloud desktop, and redirect the un-decoded media stream to the cloud desktop terminal, so that decoding and playing are performed in the cloud desktop terminal. In this way, in the embodiment of the application, multiple play scenes in the cloud desktop can be covered for multimedia redirection, decoding and other operations in the cloud desktop server are not needed, and the occupation of resources of the cloud desktop server can be effectively reduced.

In the above or the following embodiments, the cloud desktop protocol server may further include a location management component. Fig. 5 is an internal exemplary structural schematic diagram of a cloud desktop protocol server according to an exemplary embodiment of the present application. Referring to fig. 5, a location management component in a cloud desktop protocol server may receive location information corresponding to a play frame provided by a redirection server; the method can also capture the position information corresponding to the application windows of other applications in the cloud desktop; and calculating the shielding information of the play picture according to the position information corresponding to the play picture and the position information corresponding to each application window.

Referring to fig. 2, the redirection client in the multimedia playing end may obtain the location information corresponding to the playing frame and provide the location information to the redirection server, and the redirection server may retransmit the received location information corresponding to the playing frame to the location management component in this embodiment. The position information corresponding to the playing picture refers to the position information of the playing picture in the application window of the multimedia playing end. For example, for a player, the play screen occupies only a partial area in the application window of the player; for a browser, the play screen occupies only a partial area in the application window of the browser.

In addition, in this embodiment, the location management component may capture location information of application windows corresponding to respective applications in the cloud desktop by calling a specified interface provided in the operating system. For example, for a cloud desktop employing a windows operating system, the location management component may invoke a WinEventHook interface to obtain location information for application windows corresponding to respective applications in the cloud desktop. Wherein, window coordinate information and display sequence between other windows can be included in the position information of the application window. An application window with an upper display order may obscure an application window with a lower display order. In this way, the position management component can determine the target application window with the display sequence in front of the multimedia playing end, and calculate the shielding information of the playing picture according to the target application window, the application window of the multimedia playing end and the position information of the playing picture. It should be understood that the occlusion information may be coordinate information of an occluded area in the play frame, or may be coordinate information of an unoccluded area in the play frame, which is not limited in this embodiment.

In this embodiment, the cloud desktop protocol server may acquire a desktop image of the cloud desktop, and send position information and shielding information corresponding to the desktop image and the play frame to the cloud desktop protocol client. The desktop image may be obtained by capturing a view of the cloud desktop by an existing capturing tool in the cloud desktop, which is not explained here too much. It should be noted that, since the cloud desktop is not playing multimedia, there is no playing screen in the cloud desktop, and therefore, in this embodiment, the playing screen may be displayed as a specified pattern in the cloud desktop, and the specified pattern may include, but is not limited to, a pure white pattern, a pure black pattern, a mosaic pattern, or the like.

In this embodiment, for a cloud desktop protocol client, a play image of a local player may be obtained; cutting the playing picture according to the shielding information; splicing the cut play picture with the desktop image according to the position information of the play picture; and displaying the spliced picture in the cloud desktop terminal. It should be noted that, in this embodiment, in the process of playing the multimedia by the local player in the cloud desktop terminal, the front end of the local player is not displayed, but only the multimedia is played in the background to generate the playing picture, so that the local player in the cloud desktop terminal does not affect the front end rendering of the cloud desktop. In addition, the cloud desktop protocol client in this embodiment may also clip the play frame generated by the local player according to the shielding information received from the cloud desktop protocol server, that is, clip the area shielded by other application windows in the play frame. On the basis, the cut-out playing picture can be spliced to the position of the playing picture in the desktop image so as to generate a final rendering picture. And displaying the rendering picture according to a certain frame rate in the cloud desktop terminal so as to realize the effect of multimedia playing.

The implementation scheme of obtaining the position information corresponding to the playing picture by the redirecting client will be described below in connection with three exemplary types of multimedia terminals that are summarized in the foregoing embodiments.

If the multimedia playing end is any non-customized player, receiving the position information corresponding to the playing picture provided by the player plug-in unit arranged in the multimedia playing end, wherein the position information corresponding to the playing picture is monitored by the multimedia playing end and provided to the player plug-in unit. In this case, in this embodiment, the multimedia playing end may monitor the position information corresponding to the playing frame and provide the position information to the player plug-in this embodiment, and the player plug-in may provide the received position information corresponding to the playing frame to the redirection client in the multimedia playing end, so that the redirection client may resend the position information corresponding to the playing frame to the redirection server.

If the multimedia playing end is any browser, receiving position information corresponding to a playing picture provided by a browser plug-in unit arranged in the multimedia playing end, wherein the position information corresponding to the playing picture is monitored by the browser and is provided to the browser plug-in unit. In this case, in this embodiment, the multimedia playing end may monitor the position information corresponding to the playing frame and provide the position information to the browser plug-in this embodiment, and the browser plug-in may provide the received position information corresponding to the playing frame to the redirection client in the multimedia playing end, so that the redirection client may resend the position information corresponding to the playing frame to the redirection server.

If the multimedia playing end is a customized redirection special player, receiving position information corresponding to a playing picture monitored by the redirection special player, wherein program logic for position monitoring and sending is preset in the redirection special player. In this case, the dedicated redirection player itself may monitor the location information corresponding to the play frame and provide the location information to the redirection client therein, and then the redirection client resends the location information corresponding to the play frame to the redirection server.

In this way, the location information corresponding to the playing frames in the various multimedia playing terminals possibly used in the cloud desktop can reach the redirection server, and the redirection server can provide the location information corresponding to the playing frames obtained by the redirection server to the location management component in the embodiment, so that the location information can be used as one of the bases in the calculation process of the shielding information by the location management component. Accordingly, in this embodiment, the position management component may accurately calculate the shielding information corresponding to the playing frame, and provide the shielding information to the cloud desktop protocol client in the cloud desktop terminal, so that the cloud desktop protocol client accurately renders the playing frame generated by its local player in the desktop image of the cloud desktop according to the shielding information, thereby ensuring the front-end rendering effect in the multimedia playing process in a multimedia redirection manner.

In the above or below embodiments, the cloud desktop server and the cloud desktop terminal may interact with not only the undecoded media stream mentioned in the foregoing embodiments, but also the position information and the shielding information corresponding to the playing picture, and also the instruction stream involved in the multimedia playing process. The instruction stream contains various control instructions generated in the process of playing the multimedia.

In this embodiment, the cloud desktop protocol server may further obtain a first instruction stream generated in the multimedia playing end from the redirection server, where the first instruction stream is obtained by a redirection client in the multimedia playing end and provided to the redirection server; and sending the first instruction stream to the cloud desktop protocol client so that the cloud desktop protocol client can play and control the local player in the cloud desktop terminal according to the first instruction stream. The first instruction stream may include control instructions such as volume adjustment, play, pause, end-of-file (end-of-file) and the like, which occur in the multimedia playing end.

In this embodiment, the cloud desktop protocol server may further receive a second instruction stream sent by the cloud desktop protocol client, where the second instruction stream is monitored by the cloud desktop protocol client from a local player of the cloud desktop terminal; and performing play control on the multimedia playing end according to the second instruction stream. The second instruction stream may include control instructions such as seek, fast forward, fast backward, and pull stream occurring in a local player in the cloud desktop terminal, where the control instructions are redirected to a multimedia playing end in the cloud desktop, so as to perform play control on the multimedia playing end, thereby ensuring accuracy of front-end rendering.

In addition, in this embodiment, it is proposed that, in this embodiment, a serialization and anti-serialization technology is adopted between a cloud desktop protocol server and a cloud desktop protocol client to perform data interaction, in consideration of that a data link for redirection of multimedia is relatively long, and the types of related transmission contents are relatively many and the formats are relatively complex. By adopting the serialization and anti-serialization technologies, the content transmission can be efficiently performed between processes and between networks. Serialization is understood to mean the saving of an object into a byte sequence, and deserialization is understood to mean the use of a byte sequence in converting it back into the original object.

Fig. 6 is a logic schematic diagram of a data interaction manner between a cloud desktop server and a cloud desktop terminal according to an exemplary embodiment of the present application. Referring to fig. 6, in this embodiment, data transmission plug-ins may be deployed in the cloud desktop protocol server and the cloud desktop protocol client, respectively, and the data transmission plug-ins may operate as a thread. Illustratively, the data transfer plug-in may employ a serialization plug-in serializer plugin.

Referring to fig. 6, in this embodiment, the cloud desktop protocol server may respectively serialize the undecoded media stream, the position information and the shielding information of the play frame mentioned in the foregoing embodiment, and the first instruction stream mentioned in this embodiment through the data transmission plug-in, and then send the serialized first instruction stream to the cloud desktop protocol client. The cloud desktop protocol client can deserialize the content sent by the cloud desktop protocol server through the data transmission plug-in to restore the position information and the shielding information of the un-decoded media stream and the play picture.

In addition, the cloud desktop protocol client can sequence the second instruction stream acquired by the cloud desktop protocol client through the data transmission plug-in, and then send the second instruction stream to the cloud desktop protocol server. The cloud desktop protocol server can perform serialization processing through the data transmission plug-in to restore the second instruction stream.

In this way, the two processes of the cloud desktop protocol server and the cloud desktop protocol client can send various types of data to each other when they are in remote communication. Whatever type of data is transmitted over the network in the form of a binary sequence. The sender converts the data into byte sequences and then transmits the byte sequences on a network; the receiver can deserialize the byte sequence to recover the data. Therefore, the transmission of the above-mentioned various types of data between the cloud desktop protocol server and the cloud desktop protocol client can be realized.

It should be noted that, in this embodiment, other manners may be adopted to perform data interaction between the cloud desktop protocol server and the cloud desktop protocol client besides the above-mentioned serialization and deserialization manners, and the embodiment is not limited thereto.

Accordingly, in this embodiment, the instruction stream involved in the multimedia playing process may also be interacted between the cloud desktop protocol server and the cloud desktop protocol client. And various data needing to be interacted between the cloud desktop protocol server and the cloud desktop protocol client can be transmitted in a serialization and anti-serialization mode, so that different types of data needing to be transmitted between the cloud desktop protocol server and the cloud desktop protocol client can be transmitted according to a proper channel, timeliness and accuracy of data transmission are guaranteed, and smoothness of multimedia playing is further improved.

Fig. 7 is a flowchart of a method for playing multimedia on a cloud desktop according to another exemplary embodiment of the present application, where the method may be performed by a data processing device, and the data processing device may be implemented as software, hardware, or a combination of software and hardware, and the data processing device may be integrated in a cloud desktop server. The cloud desktop server comprises a cloud desktop protocol server and a multimedia playing end, wherein the cloud desktop protocol server comprises a redirection server. Referring to fig. 7, the method may include:

step 700, receiving an undecoded media stream sent by a redirection client running in a multimedia playing end according to a specified protocol by utilizing a redirection server contained in a cloud desktop protocol server, wherein the undecoded media stream is acquired from the multimedia playing end by the redirection client;

step 701, sending, by a cloud desktop protocol server, an undecoded media stream received by a redirection server to a cloud desktop protocol client in a cloud desktop terminal, so that the cloud desktop protocol client decodes the undecoded media stream and invokes a local player of the cloud desktop terminal to play the decoded multimedia;

In an alternative embodiment, the specified protocol employs a websocket communication protocol.

In an alternative embodiment, the method may further comprise:

if the multimedia playing end is any one of the non-customized players, the redirection client is utilized to receive an undecoded media stream provided by a player plug-in arranged in the multimedia playing end, wherein the undecoded media stream is an audio stream and a video stream generated by the player plug-in for demultiplexing a multimedia source file to be played; the audio stream and the video stream are sent to a redirection server; or alternatively, the process may be performed,

if the multimedia playing end is any browser, the redirection client is utilized to receive an undecoded media stream provided by a browser plug-in arranged in the multimedia playing end, wherein the undecoded media stream is a multimedia source file to be played downloaded by the browser plug-in according to a video address captured from a current webpage; sending the multimedia source file to be played to a redirection server; the method comprises the steps that a redirection server side performs demultiplexing processing on a multimedia source file to be played, which is sent by a redirection client side in a browser, so as to generate a corresponding audio stream and a video stream; or alternatively, the process may be performed,

if the multimedia playing end is a customized redirection special player, the redirection client end is utilized to receive an undecoded media stream provided by the redirection special player, and the undecoded media stream is an audio stream and a video stream which are generated by the redirection special player for demultiplexing the multimedia file to be played; the audio stream and the video stream are sent to a redirection server; the special redirection player is preset with program logic for acquiring the multimedia file to be played and performing demultiplexing processing on the multimedia file to be played.

In an alternative embodiment, the method may further comprise:

determining that the cloud desktop terminal cannot support the coding format corresponding to the multimedia source file to be played by utilizing the player plug-in, and not executing de-multiplexing and subsequent operations; and feeding back a rollback instruction to the non-customized player to trigger the non-customized player to play the multimedia; the non-customized player preferably invokes a player plug-in, or,

determining that the video address captured from the current webpage does not support file downloading by utilizing a browser plug-in, not executing the operation of sending the multimedia source file to be played to a redirection server, and feeding back a rollback instruction to the browser so as to trigger the browser to play the multimedia; after receiving the multimedia source file, the redirection server stops the subsequent processing of the multimedia source file to be played if the cloud desktop terminal is determined to be incapable of supporting the coding format corresponding to the multimedia source file to be played; and feeding back a rollback instruction to the browser to trigger the browser to play the multimedia; the browser preferentially invokes the browser plug-in, or,

and the special redirection player can also be used for determining that the cloud desktop terminal cannot support the coding format corresponding to the multimedia source file to be played, and playing the multimedia to be played without performing de-multiplexing and subsequent processing.

In an alternative embodiment, the cloud desktop protocol server may further include a location management component, and the method may further include:

receiving position information corresponding to a playing picture provided by a redirection server by using a position management component, wherein the position information corresponding to the playing picture is acquired by a redirection client in a multimedia playing terminal and provided to the redirection server; capturing position information corresponding to application windows of other applications in the cloud desktop through a designated interface; calculating shielding information of the play picture according to the position information corresponding to the play picture and the position information corresponding to each application window;

transmitting the position information and the shielding information corresponding to the desktop image and the playing picture of the cloud desktop to the cloud desktop protocol client through the cloud desktop protocol server so that the cloud desktop protocol client can acquire the playing picture of the local player; cutting the playing picture according to the shielding information; splicing the cut play picture with the desktop image according to the position information of the play picture; and displaying the spliced picture in the cloud desktop terminal.

In an alternative embodiment, the method may further comprise:

if the multimedia playing end is any non-customized player, the redirection client end in the multimedia playing end is utilized to receive the position information corresponding to the playing picture provided by the player plug-in unit arranged in the multimedia playing end, and the position information corresponding to the playing picture is monitored by the multimedia playing end and provided to the player plug-in unit; or alternatively, the process may be performed,

If the multimedia playing end is any browser, the redirection client end in the multimedia playing end is utilized to receive the position information corresponding to the playing picture provided by the browser plug-in unit arranged in the multimedia playing end, and the position information corresponding to the playing picture is monitored by the browser and is provided to the browser plug-in unit; or alternatively, the process may be performed,

if the multimedia playing end is a customized redirection special player, the redirection client end in the multimedia playing end is utilized to receive the position information corresponding to the playing picture monitored by the redirection special player, and the redirection special player is preset with the program logic for position monitoring and sending.

In an alternative embodiment, the method may further comprise:

the method comprises the steps that a cloud desktop protocol server is utilized, a first instruction stream generated in a multimedia playing end is obtained from a redirection server, and the first instruction stream is obtained from a redirection client in the multimedia playing end and is provided to the redirection server;

the method comprises the steps that a first instruction stream is sent to a cloud desktop protocol client, so that the cloud desktop protocol client can play and control a local player in a cloud desktop terminal according to the first instruction stream;

receiving a second instruction stream sent by the cloud desktop protocol client, wherein the second instruction stream is monitored by the cloud desktop protocol client from a local player;

And performing play control on the multimedia playing end according to the second instruction stream.

In an alternative embodiment, the cloud desktop protocol server and the cloud desktop protocol client use serialization and deserialization technologies to perform data interaction.

In an alternative embodiment, the playing screen in the desktop image is displayed as a designated pattern, and the designated pattern includes a pure white pattern, a pure black pattern, or a mosaic pattern.

It should be noted that, for the technical details of the embodiments of the method for playing multimedia on a cloud desktop, reference may be made to the description related to the cloud desktop server in the foregoing system embodiments, which is not repeated herein for the sake of brevity, but this should not cause a loss of protection scope of the present application.

Fig. 8 is a flowchart of another method for playing multimedia on a cloud desktop according to another exemplary embodiment of the present application, where the method may be performed by a data processing device, and the data processing device may be implemented as software, hardware, or a combination of software and hardware, and the data processing device may be integrated in a cloud desktop terminal. The cloud desktop terminal may include a cloud desktop protocol client, and referring to fig. 8, the method may include:

step 800, receiving, by a cloud desktop protocol client, an undecoded media stream sent by a cloud desktop protocol server in a cloud desktop server, where the undecoded media stream is received by a redirection server in the cloud desktop protocol server from a redirection client running in a multimedia playing end according to a specified protocol, and the redirection client obtains the undecoded media stream from the multimedia playing end;

Step 801, decoding the un-decoded media stream;

step 802, calling a local player to play the decoded multimedia;

the multimedia playing end operates in the cloud desktop server, comprises multiple types of players and multiple types of browsers, is compatible with the multiple types of players and the multiple types of browsers by a specified protocol, and supports a one-to-many communication mechanism.

In an alternative embodiment, the method may further comprise:

the method comprises the steps that a playing picture of a local player is obtained through a cloud desktop protocol client;

cutting the playing picture according to the shielding information;

splicing the cut play picture with the desktop image according to the position information of the play picture;

displaying the spliced picture in the cloud desktop terminal;

the shielding information is generated by calculating position management components contained in the cloud desktop protocol server according to position information corresponding to a playing picture provided by the redirection server and position information corresponding to application windows of the multimedia playing end and other applications in the cloud desktop.

In an alternative embodiment, the method may further comprise:

The cloud desktop protocol client sends the second instruction stream to the cloud desktop protocol server to enable the cloud desktop protocol server to play and control the multimedia playing end according to the second instruction stream;

the second instruction stream is monitored by the cloud desktop protocol client from the local player.

It should be noted that, for the technical details of the embodiments of the method for playing multimedia on a cloud desktop, reference may be made to the description related to the cloud desktop terminal in the foregoing system embodiments, which is not repeated herein for the sake of brevity, but this should not cause a loss of protection scope of the present application.

It should be noted that, in some of the above embodiments and the flows described in the drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that the operations may be performed out of the order in which they appear herein or performed in parallel, the sequence numbers of the operations, such as 801, 802, etc., are merely used to distinguish between the various operations, and the sequence numbers themselves do not represent any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different instruction streams, and do not represent a sequential order, and do not limit that "first" and "second" are different types.

Fig. 9 is a schematic structural diagram of a cloud desktop server according to another exemplary embodiment of the present application. As shown in fig. 9, the cloud desktop server includes: a memory 90 and a processor 91.

A processor 91 coupled to the memory 90 for executing a computer program in the memory 90 for:

receiving an undecoded media stream sent by a redirection client running in a multimedia playing end according to a specified protocol by utilizing a redirection server contained in a cloud desktop protocol server, wherein the undecoded media stream is acquired from the multimedia playing end by the redirection client;

the method comprises the steps that an undecoded media stream received by a redirection server side is sent to a cloud desktop protocol client side in a cloud desktop terminal through the cloud desktop protocol server side, so that the cloud desktop protocol client side decodes the undecoded media stream and calls a local player of the cloud desktop terminal to play decoded multimedia;

In an alternative embodiment, processor 91 may be further configured to:

In an alternative embodiment, the cloud desktop protocol server may further include a location management component, and the processor 91 may be further configured to:

In an alternative embodiment, processor 91 may be further configured to:

Further, as shown in fig. 9, the cloud desktop server further includes: a communication component 92, a power supply component 93, and the like. Only some of the components are schematically shown in fig. 9, which does not mean that the cloud desktop server only includes the components shown in fig. 9.

It should be noted that, for the technical details of the embodiments of the cloud desktop server, reference may be made to the related descriptions of the embodiments of the system described above with respect to the cloud desktop server, which are not repeated herein for the sake of brevity, but should not cause a loss of protection scope of the present application.

Fig. 10 is a schematic structural diagram of a cloud desktop terminal according to another exemplary embodiment of the present application. As shown in fig. 10, the cloud desktop terminal includes: a memory 10 and a processor 11.

A processor 11 coupled to the memory 10 for executing a computer program in the memory 10 for:

decoding the un-decoded media stream;

calling a local player to play the decoded multimedia;

In an alternative embodiment, the processor 11 may be further configured to:

cutting the playing picture according to the shielding information;

displaying the spliced picture in the cloud desktop terminal;

In an alternative embodiment, the processor 11 may be further configured to:

Further, as shown in fig. 10, the cloud desktop terminal further includes: communication component 12, power component 13, display component 14, audio component 15, and other components. Only some of the components are schematically shown in fig. 10, which does not mean that the cloud desktop server only includes the components shown in fig. 10.

It should be noted that, for the technical details of the embodiments of the cloud desktop terminal, reference may be made to the related descriptions of the embodiments of the system described above with respect to the cloud desktop terminal, which are not repeated herein for the sake of brevity, but should not cause a loss of protection scope of the present application.

Accordingly, the embodiment of the application further provides a computer readable storage medium storing a computer program, and the computer program can implement each step in the embodiment of the method, which can be executed by the cloud desktop server or the cloud desktop terminal when being executed.

The memory in fig. 9 and 10 described above is for storing a computer program and may be configured to store various other data to support operations on a computing platform. Examples of such data include instructions for any application or method operating on a computing platform, contact data, phonebook data, messages, pictures, videos, and the like. The memory may be implemented by any type of volatile or nonvolatile memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

The communication assembly of fig. 9 and 10 is configured to facilitate wired or wireless communication between the device in which the communication assembly is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as a mobile communication network of WiFi,2G, 3G, 4G/LTE, 5G, etc., or a combination thereof. In one exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further comprises a Near Field Communication (NFC) module to facilitate short range communications. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

The display assembly of fig. 10 described above includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation.

The power supply assembly of fig. 9 and 10 provides power to the various components of the device in which the power supply assembly is located. The power components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power for the devices in which the power components are located.

The audio component of fig. 10 described above may be configured to output and/or input audio signals. For example, the audio component includes a Microphone (MIC) configured to receive external audio signals when the device in which the audio component is located is in an operational mode, such as a call mode, a recording mode, and a speech recognition mode. The received audio signal may be further stored in a memory or transmitted via a communication component. In some embodiments, the audio assembly further comprises a speaker for outputting audio signals.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. that fall within the spirit and principles of the present application are intended to be included within the scope of the present application.

Claims

1. The cloud desktop multimedia playing system comprises a cloud desktop terminal and a cloud desktop server, wherein the cloud desktop server comprises a cloud desktop protocol server and a multimedia playing end, the cloud desktop terminal comprises a cloud desktop protocol client, and the cloud desktop protocol server comprises a redirection server;

2. The system of claim 1, the specified protocol employing a websocket communication protocol.

3. The system of claim 1, the redirect client of the multimedia player, further for:

if the multimedia playing end is any non-customized player, receiving an undecoded media stream provided by a player plug-in unit arranged in the multimedia playing end, wherein the undecoded media stream is an audio stream and a video stream generated by the player plug-in unit for demultiplexing a multimedia source file to be played; sending the audio stream and the video stream to the redirection server; or alternatively, the process may be performed,

If the multimedia playing end is any browser, receiving an undecoded media stream provided by a browser plug-in unit arranged in the multimedia playing end, wherein the undecoded media stream is a multimedia source file to be played, which is downloaded by the browser plug-in unit according to a video address captured from a current webpage; sending the multimedia source file to be played to the redirection server; the redirection server is further configured to: demultiplexing the multimedia source file to be played, which is sent by the redirection client in the browser, so as to generate a corresponding audio stream and video stream; or alternatively, the process may be performed,

if the multimedia playing end is a customized redirection special player, receiving an undecoded media stream provided by the redirection special player, wherein the undecoded media stream is an audio stream and a video stream generated by the redirection special player for demultiplexing the multimedia file to be played; sending the audio stream and the video stream to the redirection server; the special redirection player is preset with program logic for acquiring the multimedia file to be played and performing demultiplexing processing on the multimedia file to be played.

4. The system of claim 3, the non-customized player preferentially invoking the player plug-in, the player plug-in further to: if the cloud desktop terminal is determined to be incapable of supporting the coding format corresponding to the multimedia source file to be played, de-multiplexing and subsequent operation are not executed; and feeding back a rollback instruction to the non-customized player to trigger the non-customized player to play multimedia; or alternatively, the process may be performed,

the browser preferentially calls the browser plug-in, and the browser plug-in is further used for: if the video address captured from the current webpage does not support file downloading, the operation of sending the multimedia source file to be played to the redirection server is not executed any more, and a rollback instruction is fed back to the browser to trigger the browser to play multimedia; the redirection server is further configured to: after receiving the multimedia source file, if the cloud desktop terminal is determined to be incapable of supporting the coding format corresponding to the multimedia source file to be played, stopping subsequent processing of the multimedia source file to be played; and feeding back a rollback instruction to the browser to trigger the browser to play multimedia; or alternatively, the process may be performed,

The redirection-specific player is further configured to: and if the cloud desktop terminal is determined to be incapable of supporting the coding format corresponding to the multimedia source file to be played, playing the multimedia to be played without performing the demultiplexing and subsequent processing.

5. The system of claim 1, the cloud desktop protocol server further comprising a location management component for:

receiving position information corresponding to a playing picture provided by the redirection server, wherein the position information corresponding to the playing picture is acquired by a redirection client in the multimedia playing terminal and provided to the redirection server;

capturing position information corresponding to application windows of other applications in the cloud desktop through a designated interface;

calculating shielding information of the playing picture according to the position information corresponding to the playing picture and the position information corresponding to each application window;

the cloud desktop protocol server is further configured to: transmitting the desktop image of the cloud desktop, the position information and the shielding information corresponding to the playing picture to the cloud desktop protocol client;

the cloud desktop protocol client is further configured to: acquiring a playing picture of the local player; cutting the play picture according to the shielding information; splicing the cut play picture with the desktop image according to the position information of the play picture; and displaying the spliced picture in the cloud desktop terminal.

6. The system of claim 5, the redirect client further to:

if the multimedia playing end is any non-customized player, receiving position information corresponding to a playing picture provided by a player plug-in unit arranged in the multimedia playing end, wherein the position information corresponding to the playing picture is monitored by the multimedia playing end and provided to the player plug-in unit; or alternatively, the process may be performed,

if the multimedia playing end is any browser, receiving position information corresponding to a playing picture provided by a browser plug-in unit arranged in the multimedia playing end, wherein the position information corresponding to the playing picture is monitored by the browser and provided to the browser plug-in unit; or alternatively, the process may be performed,

and if the multimedia playing end is a customized special redirection player, receiving position information corresponding to a playing picture monitored by the special redirection player, wherein program logic for position monitoring and sending is preset in the special redirection player.

7. The system of claim 5, the cloud desktop protocol server further configured to:

acquiring a first instruction stream generated in the multimedia playing end from the redirection service end, wherein the first instruction stream is acquired by a redirection client in the multimedia playing end and is provided to the redirection service end;

The first instruction stream is sent to the cloud desktop protocol client for the cloud desktop protocol client to play and control the local player in the cloud desktop terminal according to the first instruction stream;

receiving a second instruction stream sent by the cloud desktop protocol client, wherein the second instruction stream is monitored by the cloud desktop protocol client from the local player;

8. The system of claim 7, wherein the cloud desktop protocol server and the cloud desktop protocol client perform data interaction by adopting serialization and anti-serialization technologies.

9. The system of claim 5, wherein the play-out in the desktop image is displayed as a specified pattern comprising a solid white pattern, a solid black pattern, or a mosaic pattern.

10. The multimedia playing method of the cloud desktop is suitable for a cloud desktop server, wherein the cloud desktop server comprises a cloud desktop protocol server and a multimedia playing end, and the cloud desktop protocol server comprises a redirection server, and the method comprises the following steps:

11. The multimedia playing method of the cloud desktop is suitable for a cloud desktop terminal, wherein the cloud desktop terminal comprises a cloud desktop protocol client, and the method comprises the following steps:

Decoding the un-decoded media stream;

calling a local player to play the decoded multimedia;

12. A cloud desktop server comprises a memory and a processor;

the memory is used for storing one or more computer instructions;

13. A cloud desktop terminal comprises a memory and a processor;

the memory is used for storing one or more computer instructions;

decoding the un-decoded media stream;

calling a local player to play the decoded multimedia;

14. A computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the method of multimedia playing of a cloud desktop of any of claims 10 or 11.