CN114157903A

CN114157903A - Redirection method, redirection device, redirection equipment, storage medium and program product

Info

Publication number: CN114157903A
Application number: CN202111461661.0A
Authority: CN
Inventors: 何�轩; 杨俊拯; 钟江; 张子煜
Original assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Current assignee: Guangdong Oppo Mobile Telecommunications Corp Ltd
Priority date: 2021-12-02
Filing date: 2021-12-02
Publication date: 2022-03-08
Anticipated expiration: 2041-12-02

Abstract

The embodiment of the application discloses a redirection method, a redirection device, redirection equipment, a storage medium and a program product, and belongs to the technical field of screen projection. The method comprises the following steps: creating a source terminal redirection instance based on the source terminal equipment identification and the target redirection object identification, wherein the target redirection object identification is used for representing a target redirection object; under the condition that the same source terminal redirection instance corresponds to at least two destination terminal redirection instances, generating redirection data corresponding to the source terminal redirection instance through a multiplexing redirection data generator, wherein the destination terminal redirection instance is created based on a destination terminal device identification and a target screen identification; and transmitting the redirection data to the destination device so that the destination device can display the target redirection object through the target screen based on the redirection data. By adopting the scheme provided by the embodiment of the application, the encoding and decoding expenses generated by data transmission between the source end equipment and the destination end equipment can be reduced.

Description

Redirection method, redirection device, redirection equipment, storage medium and program product

Technical Field

The embodiment of the application relates to the technical field of screen projection, in particular to a redirection method, a redirection device, redirection equipment, a storage medium and a program product.

Background

Redirection (redirect) refers to a technique for transferring certain capabilities from one end device to another, including video, audio, and control, among others. A common redirection technique in everyday life involves the device throwing a screen.

With the help of the screen projection function, the source end equipment can project the current displayed picture to the destination end equipment, thereby achieving better display effect. The screen projection scheme provided by the related technology is suitable for a simple screen projection scene, such as a one-to-one screen projection scene or a many-to-one screen projection scene. However, as the complexity of the screen-projected scenes increases, such as many-to-many screen-projected scenes, the data flow between the source device and the destination device increases, thereby resulting in increased encoding and decoding overhead.

Disclosure of Invention

The embodiment of the application provides a redirection method, a redirection device, redirection equipment, a storage medium and a program product. The technical scheme is as follows:

in one aspect, an embodiment of the present application provides a redirection method, where the method is used for a source device, and the method includes:

creating a source terminal redirection instance based on a source terminal device identification and a target redirection object identification, wherein the target redirection object identification is used for representing a target redirection object;

under the condition that the same source terminal redirection instance corresponds to at least two destination terminal redirection instances, generating redirection data corresponding to the source terminal redirection instance through a multiplexing redirection data generator, wherein the destination terminal redirection instance is created based on a destination terminal device identifier and a target screen identifier, the destination terminal device identifier is used for representing destination terminal equipment, and the target screen identifier is used for representing a target screen of the destination terminal equipment;

and transmitting the redirection data to the destination terminal equipment so that the destination terminal equipment can display the target redirection object through the target screen based on the redirection data.

In another aspect, an embodiment of the present application provides a redirection method, where the redirection method is used for a destination device, and the method includes:

creating a destination redirection instance based on a destination device identifier and a target screen identifier, wherein the target screen identifier is used for representing a target screen of the destination device;

receiving redirection data corresponding to a source end redirection instance transmitted by source end equipment, wherein the source end redirection instance is created based on a source end equipment identifier and a target redirection object identifier, and the target redirection object identifier is used for representing a target redirection object;

when at least two destination terminal redirection instances correspond to the same source terminal redirection instance, analyzing the redirection data through a multiplexing redirection data analyzer;

and displaying the target redirection object through the target screen based on the analysis result of the redirection data.

In another aspect, an embodiment of the present application provides a redirection apparatus, where the apparatus includes:

the device comprises a creating module, a determining module and a sending module, wherein the creating module is used for creating a source terminal redirection instance based on a source terminal device identification and a target redirection object identification, and the target redirection object identification is used for representing a target redirection object;

the multiplexing module is configured to generate, by a multiplexing redirection data generator, redirection data corresponding to the source redirection instance under the condition that the same source redirection instance corresponds to at least two destination redirection instances, where the destination redirection instances are created based on a destination device identifier and a destination screen identifier, where the destination device identifier is used to represent destination devices, and the destination screen identifier is used to represent a destination screen of the destination devices;

and the first transmission module is used for transmitting the redirection data to the destination terminal equipment so that the destination terminal equipment can display the target redirection object through the target screen based on the redirection data.

the system comprises a creating module, a display module and a display module, wherein the creating module is used for creating a destination redirection instance based on a destination device identifier and a target screen identifier, and the target screen identifier is used for representing a target screen of the destination device;

the first receiving module is used for receiving redirection data corresponding to a source end redirection instance transmitted by a source end device, the source end redirection instance is created based on a source end device identifier and a target redirection object identifier, and the target redirection object identifier is used for representing a target redirection object;

the analysis module is used for analyzing the redirection data through a multiplexing redirection data analyzer under the condition that at least two destination redirection instances correspond to the same source redirection instance;

and the display module is used for displaying the target redirection object through the target screen based on the analysis result of the redirection data.

In another aspect, an embodiment of the present application provides a redirection system, where the redirection system includes: at least one source end device and at least one destination end device;

the source device is used for creating a source redirection instance based on a source device identifier and a target redirection object identifier, and the target redirection object identifier is used for representing a target redirection object; under the condition that the same source terminal redirection instance corresponds to at least two destination terminal redirection instances, generating redirection data corresponding to the source terminal redirection instance through a multiplexing redirection data generator; transmitting the redirection data to destination equipment so that the destination equipment can display the target redirection object through the target screen based on the redirection data;

the destination device is configured to create a destination redirection instance based on a destination device identifier and a target screen identifier, where the target screen identifier is used to represent a target screen of the destination device; receiving redirection data corresponding to the source terminal redirection instance transmitted by the source terminal device; when at least two destination terminal redirection instances correspond to the same source terminal redirection instance, analyzing the redirection data through a multiplexing redirection data analyzer; and displaying the target redirection object through the target screen based on the analysis result of the redirection data.

In another aspect, an embodiment of the present application provides a computer device, which includes a processor and a memory; the memory stores at least one instruction for execution by the processor to implement the redirection method as described in the above aspect.

In another aspect, an embodiment of the present application provides a computer-readable storage medium, in which at least one program code is stored, and the program code is loaded and executed by a processor to implement the redirection method according to the above aspect.

In another aspect, embodiments of the present application provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the redirection method provided in the various alternative implementations of the above aspects.

In the redirection process, a source device creates a source redirection instance based on a source device identifier and a target redirection object identifier, a destination device creates a destination redirection instance corresponding to the source redirection instance based on a destination device identifier and a target screen identifier, the source device generates redirection data through a multiplexing redirection data generator and transmits the redirection data to the destination device under the condition that the same source redirection instance corresponds to at least two destination re-instances, the destination device receives the redirector data, analyzes the redirection data through a multiplexing redirection data analyzer, and displays a target redirection object through a target screen indicated by the destination redirection instance; by adopting the scheme provided by the embodiment of the application, the encoding cost of the source end equipment and the decoding cost of the target end equipment in the data stream transmission process between the source end equipment and the target end equipment are reduced through the multiplexing redirection data generator and the redirection data analyzer, and the data transmission performance of a complex redirection scene is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a schematic illustration of an implementation environment provided by an exemplary embodiment of the present application;

FIG. 2 is a schematic diagram of an implementation of the encoding, transmission, and decoding processes provided by an exemplary embodiment of the present application;

FIG. 3 is a schematic diagram of an implementation of the encoding, transmission, and decoding processes provided by another exemplary embodiment of the present application;

FIG. 4 is a flow chart of a redirection method provided by an exemplary embodiment of the present application;

FIG. 5 is a schematic diagram illustrating a hierarchical architecture of a redirection module in accordance with an exemplary embodiment of the present application;

FIG. 6 is a schematic diagram of a hierarchical architecture of a redirection module shown in another exemplary embodiment of the present application;

FIG. 7 is a flow chart of a redirection method provided by another exemplary embodiment of the present application;

FIG. 8 is a flow chart of a video codec and transmission process shown in an exemplary embodiment of the present application;

FIG. 9 is a flowchart illustrating an implementation of a method for frame-filling by-pass in accordance with an exemplary embodiment of the present application;

fig. 10 is a flow chart illustrating a method for asynchronous access redirection in accordance with an exemplary embodiment of the present application;

fig. 11 is a flowchart illustrating a redirection method during concurrent access according to an exemplary embodiment of the present application;

fig. 12 is a schematic diagram illustrating an implementation of a redirection method during concurrent access according to an exemplary embodiment of the present application;

fig. 13 is a block diagram of a redirection device according to an embodiment of the present application;

fig. 14 is a block diagram of a redirection device according to another embodiment of the present application;

FIG. 15 is a system architecture diagram of a redirection system as provided by one embodiment of the present application;

fig. 16 is a block diagram of a computer device according to an exemplary embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

FIG. 1 illustrates a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application. The implementation environment may include: at least one source end device 110 and at least one destination end device 120.

The source device 110 is an electronic device with redirection requirements, which may be a mobile terminal such as a smartphone, a tablet, a personal computer, etc. The source device 110 includes a tablet 111 and a smart phone 112, which are illustrated in fig. 1 as an example.

An object having a redirection requirement in the source device 110 is referred to as a redirection object, the redirection object may be a screen image or a redirection application, and the redirection content of the redirection object may include at least one of video, audio, and control. The redirection application may be a navigation application, a video application, a game application, and the like, and the specific type of the screen projection application is not limited in the embodiments of the present application. Optionally, when the redirection object is an application, the application may be in a background running state in the redirection process.

In this embodiment, the same source device may redirect at least two redirection objects at the same time. For example, the screen image and the video application may be projected at the same time, or the video application and the navigation application may be projected at the same time. The number of redirection objects that the source device supports simultaneous redirection has an upper limit, and the upper limit is related to the processing performance of the source device.

The destination device 120 is an electronic device for displaying the redirection result, and may be a smart phone, a tablet computer, a personal computer, a smart television, or a vehicle-mounted terminal (such as a car machine), and the like. In fig. 1, the destination device 120 is illustrated as including a first vehicle-mounted terminal 121 and a second vehicle-mounted terminal 122, where the first vehicle-mounted terminal 121 and the second vehicle-mounted terminal 122 are disposed in the same vehicle, and the first vehicle-mounted terminal 121 and the second vehicle-mounted terminal 122 are respectively controlled by different socs (System on Chip). For example, the first vehicle-mounted terminal 121 is a vehicle-mounted terminal disposed on the front row of the vehicle, and the second vehicle-mounted terminal 122 is a vehicle-mounted terminal disposed on the rear row of the vehicle.

Optionally, the destination device 120 has at least one screen. As shown in fig. 1, the first vehicle-mounted terminal 121 has a first screen 1211 and a second screen 1212 (both controlled by the first SoC 1213 of the first vehicle-mounted terminal 121), and the second vehicle-mounted terminal 122 has a third screen 1221 and a fourth screen 1222 (both controlled by the second SoC 1223 of the second vehicle-mounted terminal 122).

In some embodiments, when the destination device 120 has at least two screens, the at least two screens may simultaneously display different screen-projecting contents, or the same screen-projecting content, and the screen-projecting contents displayed in each screen may be a single screen-projecting object or multiple screen-projecting objects (multiple screen-projecting objects are displayed in a split screen).

The destination device 120 may have an audio playing function besides the screen-projecting content displaying function, and the audio played in the screen-projecting process is the audio of the screen-projecting object.

Besides the audio/video playing function, the destination device 120 can also implement a reverse control function on the screen projection content through the input/output component. Optionally, the input/output component may be a touch screen of the destination device 120, and an external control device of the destination device 120, such as a handle, a keyboard, a mouse, or a touch pad, which is not limited in this embodiment.

The many-to-many redirection in the embodiment of the present application is embodied in a multi-source device (multiple source devices participate in redirection), a multi-directional object (multiple redirection objects in the same source device participate in redirection), a multi-destination device (multiple destination devices participate in redirection), and a multi-screen (multiple screens of the same destination device participate in redirection). In one possible many-to-many screen projection scenario, as shown in fig. 1, the video application in the tablet computer 111 is projected to the first screen 1211 of the first vehicle-mounted terminal 121; the first game application is projected to the second screen 1212 of the first in-vehicle terminal 121. The navigation application in the smart phone 112 is projected to the first screen 1211 of the first vehicle-mounted terminal 121 and displayed in a split screen manner with the video application; the second game application in the smartphone 112 is projected to the third screen 1221 and the fourth screen 1222 of the second in-vehicle terminal 122, that is, the third screen 1221 and the fourth screen 1222 display the same projection content. In addition, since each screen of the first vehicle-mounted terminal 121 and the second vehicle-mounted terminal 122 has a touch function, a user can operate the screen-casting application on the screen, thereby implementing reverse control of the screen-casting application.

It should be noted that the connection manner adopted between the source device 110 and the destination device 120 may include at least one of a WlAN AP (Wireless Local Area Network Access Point) mode, a WiFi-P2P (Wireless-Peer-to-Peer) mode, a USB (Universal Serial Bus) wired connection, and a mobile data Network, which is not limited in this embodiment of the present application.

In addition, the foregoing embodiment is only schematically described by taking a screen projection from the mobile terminal to the vehicle-mounted terminal as an example (i.e., a vehicle-mounted device scene), and the scheme provided by the embodiment of the present application may be applied to other many-to-many redirection scenes, and the embodiment does not limit a specific application scene.

In this embodiment of the present application, before transmitting redirection data to a destination device, a source device needs to collect a redirection object, and encode the collected content by using a redirection data generator to generate redirection data, which may generate encoding overhead. Correspondingly, when the destination device receives the redirection data, the redirection data needs to be parsed by using a redirection data parser, so as to restore the redirection object, and decoding overhead is generated in the process. Taking the redirection object as a video as an example, the source device needs to collect a video picture first, and then performs video coding on the collected picture through the video encoder to obtain a video data stream, where coding overhead is generated in the process, and correspondingly, after receiving the video data stream, the destination device needs to decode through the video decoder to restore the video picture, and decoding overhead is generated in the process.

In a possible screen projection scene, the same screen projection object in the source device needs to be projected to different destination devices, or projected to different screens of the same destination device. In the related art, the method shown in fig. 2 is adopted, that is, when the screen projection application in the source device needs to be projected to the screen 1 and the screen 2 of the destination device 1 and the screen 3 of the destination device 2 at the same time, video encoding is performed by the video encoders 1, 2 and 3, video decoding is performed by the video decoders 1, 2 and 3, and video data stream transmission is performed in three paths, which may generate triple encoding and decoding overhead.

In this embodiment, the source device and the destination device may adopt a multiplexing strategy, that is, the source device encodes the redirection object through the multiplexed redirection data generator to generate redirection data, and the destination device parses the redirection data through the multiplexed redirection data parser to restore the redirection object. Compared with the related technology, the source end device and the destination end device respectively adopt multiplexing strategies, and the encoding and decoding expenses are reduced. As shown in fig. 3, similarly, when a screen projection application in a source device needs to be projected to a screen 1 and a screen 2 of a destination device 1 and a screen 3 of a destination device 2 at the same time, the source device multiplexes the video encoder 1 to perform video encoding to obtain a video data stream, after receiving the video data stream, the destination device 1 performs video decoding through the multiplexed video decoder 1, so that a picture obtained by decoding is displayed through the screen 1 and the screen 2, and after receiving the video data stream, the destination device 2 performs video decoding through the video decoder 2, so that the picture obtained by decoding is displayed through the screen 3. Compared with the related technology, the source end device and the destination end device respectively adopt the multiplexing strategy encoder and the multiplexing strategy decoder, so that the encoding overhead is reduced by 2 times, and the decoding overhead is reduced by 1 time. The following describes a scheme provided by an embodiment of the present application.

Referring to fig. 4, a flowchart of a redirection method according to an exemplary embodiment of the present application is shown, where the embodiment of the present application takes application of the method to the implementation environment shown in fig. 1 as an example, and the method includes:

step 401, a source device creates a source redirection instance based on a source device identifier and a target redirection object identifier, where the target redirection object identifier is used to represent a target redirection object.

Wherein the type of the redirection object comprises at least one of a screen mirror or a redirection application, and the type of the redirection content of the redirection object comprises at least one of audio, video, and control.

When redirection needs to be performed through the source device, the source device creates a source redirection instance based on the source device identifier (i.e., the device identifier of the source device) and the target redirection object identifier of the target redirection object to be redirected, that is, the source device and the redirection object are taken as a unit to form the source redirection instance.

When different source end devices redirect the same redirection object (for example, different source end devices redirect the same application), source end redirection instances created by different source end devices are different; when the same source device needs to redirect different redirection objects, source redirection instances created by the same source device based on different redirection objects are also different.

Optionally, the source device creates the source redirection instance when receiving the redirection operation, or the source device creates the source redirection instance when receiving the redirection instruction of the third-party device, that is, redirection by the source device may be triggered at the source device side, or may be triggered by the third-party device, where the third-party device and the source device are in the same scheduling domain, for example, the third-party device and the source device log in the same user account.

Regarding the specific content redirected by the source device, in some embodiments, the source device may redirect the screen image, and may also redirect an application installed and running (i.e., a redirected application). Correspondingly, when the target redirection object is a screen mirror image, the target redirection object identifier is a mirror image identifier; when the target redirection object is the redirection application, the target redirection object identifier is an application identifier of the redirection application.

Step 402, the destination device creates a destination redirection instance based on the destination device identifier and the target screen identifier, where the target screen identifier is used to represent a target screen of the destination device.

Corresponding to the source device, the destination device that performs the presentation needs to create a destination redirection instance corresponding to the source redirection instance. The destination device creates a destination redirection instance based on the destination device identifier (i.e. the device identifier of the destination device) and the target screen identifier of the target screen to be displayed, that is, the destination redirection instance is formed by taking the destination device + the target screen as a unit.

In the embodiment of the application, the destination device has at least one screen, different screens correspond to different screen identifiers, and the different screens are controlled by the same SoC in the destination device. For example, the destination device is a vehicle-mounted terminal having a main driving screen and a sub-driving screen.

When different destination-end devices display the same redirection object, the destination-end redirection instances created by the different destination-end devices are different; when the same destination device is displayed through different screens, destination redirection examples created by the same destination device based on different screens are also different.

Optionally, the destination device creates a destination redirection instance when receiving a redirection request sent by the source device, or creates a destination redirection instance when the destination device receives a redirection instruction of a third-party device, where the third-party device and the destination device are in the same scheduling domain, for example, the third-party device and the destination device log in the same user account.

In step 403, the source device generates redirection data corresponding to the source redirection instance through the multiplexed redirection data generator under the condition that the same source redirection instance corresponds to at least two destination redirection instances.

In a possible implementation manner, the at least two destination redirection instances corresponding to the same source redirection instance may be that the same redirection object of the same source device is redirected to at least two destination screens of the same destination device. Illustratively, as shown in fig. 3, the screen projection application of the same source device is projected to the screen 1 and the screen 2 of the destination device 1.

In another possible implementation, that the same source redirection instance corresponds to at least two destination redirection instances means that the same redirection object of the same source device is redirected to target screens of different destination devices. Illustratively, as shown in fig. 3, the screen projection application of the same source device is projected to the screen 1, the screen 2 of the destination device 1 and the screen 3 of the destination device 2.

In the foregoing case, after the source device completes creation of the source redirection instance, in order to reduce the encoding overhead of the source device, the source device adopts a multiplexing policy, that is, a multiplexing redirection data generator converts a redirection object into redirection data. Wherein the type of the redirection data generator is related to the media type corresponding to the target redirection object. In an embodiment of the present application, the media type corresponding to the target redirection object includes at least one of video, audio, and control.

Optionally, when the media type of the target redirection object is video, the redirection data generator is a video encoder; when the media type of the target redirection object is audio, the redirection data generator is an audio encoder, and when the media type of the target redirection object is control, the redirection data generator is a control encoder.

In addition, the redirection data may include at least one of a video data stream, an audio data stream, and a control data stream.

In some embodiments, a source device performs video coding on a picture of a target redirection object to obtain a video data stream, where the source device may perform coding in video coding formats such as H264 and H265, which are not limited in this embodiment of the present application; when the redirection data comprises an audio data stream, the source end device performs audio coding on the sound of the target redirection object to obtain the audio data stream, wherein the source end device can perform coding in an audio coding format such as AAC (advanced audio coding) and the like, which is not limited in the embodiment of the application; when the redirection data includes a control data stream, the source device may encode the control command according to an agreed command format to obtain the control data stream.

In step 404, the source device transmits redirection data to the destination device.

Regarding the transmission manner of the redirection data, optionally, the source device may transmit the redirection data in a wireless manner, for example, Wlan AP, WiFi-P2P or mobile data network; the redirection data may also be transmitted in a wired manner, such as a USB wired connection; the redirection data may also be transmitted in a wired and wireless hybrid manner, which is not limited in the embodiment of the present application.

In step 405, the destination device receives the redirection data transmitted by the source device.

In one possible implementation, the same destination device receives redirection data transmitted by the same source device.

In another possible embodiment, different destination devices receive the redirection data transmitted by the same source device.

Step 406, in the case that there are at least two destination redirection instances corresponding to the same source redirection instance, the destination device parses the redirection data through the multiplexed redirection data parser.

In a possible implementation manner, at least two destination redirection instances corresponding to the same source redirection instance may be that different destination devices display the same redirection object of the same source device; in another possible implementation, the same redirection object of the same source device may also be exposed by the same destination device through different screens.

At this time, in order to reduce the decoding overhead of the destination device, the destination device adopts a multiplexing strategy, that is, a multiplexing redirection data parser parses the redirection data.

Wherein the type of the redirection data parser is related to the media type corresponding to the target redirection object. Optionally, when the media type of the target redirection object is video, the redirection data parser is a video decoder, and when the media type of the target redirection object is audio, the redirection data parser is an audio decoding player. When the media type of the target redirection object is control, the redirection data parser is a control decoder.

In some embodiments, when the redirection data includes a video data stream, the destination device parses the video data stream through a video decoder, when the redirection data includes an audio data stream, the destination device parses the audio data stream through an audio decoding player, and when the redirection data includes a control data stream, the destination device parses the control data stream through a control decoder.

Step 407, the destination device displays the target redirection object through the target screen based on the analysis result of the redirection data.

Further, the destination device determines a target screen indicated by the destination redirection instance, so as to display the target redirection object through the target screen.

In some embodiments, after the destination device parses the video data stream, a video playing component plays a picture of the target redirection object; and after the destination terminal equipment analyzes the audio data stream, the audio playing component plays the sound of the target redirection object.

To sum up, in the embodiment of the present application, in a redirection process, a source device creates a source redirection instance based on a source device identifier and a target redirection object identifier, a destination device creates a destination redirection instance corresponding to the source redirection instance based on a destination device identifier and a target screen identifier, and in a case that the same source redirection instance corresponds to at least two destination re-instances, the source device generates redirection data through a multiplexed redirection data generator and transmits the redirection data to the destination device, and the destination device receives the redirector data, parses the redirection data through a multiplexed redirection data parser, and displays a target redirection object through a target screen indicated by the destination redirection instance; by adopting the scheme provided by the embodiment of the application, the encoding cost of the source end equipment and the decoding cost of the target end equipment in the data stream transmission process between the source end equipment and the target end equipment are reduced through the multiplexing redirection data generator and the redirection data analyzer, and the data transmission performance of a complex redirection scene is improved.

In order to realize multi-level many-to-many redirection, the redirection modules of the source end device and the destination end device are designed in a layered mode. As shown in fig. 5, the hierarchy of redirection modules includes:

equipment layer: the system comprises a multi-channel screen projection service and an Application Programming Interface (API) for providing screen projection of equipment for the outside.

Screen projection application layer: including a source redirection instance (when acting as a source device) or a destination redirection instance (when acting as a destination device) created by the screen cast manager.

A streaming media processing layer: the system comprises a video processing module, an audio processing module and a control processing module, and is used for processing data of a specific media type in a source end or destination end redirection instance.

Redirection instance layer: when the device is used as a source device, the device comprises a source video redirector, a source audio redirector and a source control redirector; when the device is used as a destination device, the device comprises a destination video redirector, a destination audio redirector and a destination control redirector. The transmission link between the source redirector and the destination redirector of a particular media type is used for transmitting redirection data of a particular media type.

Display layer: the method comprises screen projecting window body application, and different screen projecting window body applications correspond to different screens of destination terminal equipment.

In the embodiment of the application, the source device and the destination device adopt a multiplexing strategy, and the encoding and decoding overheads of the source device and the destination device in the data stream transmission process are reduced through a structure in a multiplexing redirection module. The multiplexing strategy of the source device and the destination device is described below with reference to a more detailed hierarchy of the redirection module as shown in fig. 6.

1. Source end screen projection example: and the same redirection object is multiplexed and is responsible for managing various types of media processing modules and redirectors under the screen projection application at the source end.

2. Example of destination screen projection: and the same source device and the redirection object are multiplexed and are responsible for managing various redirectors under the screen projection application at the destination.

3. Redirection context: the same redirection object is reused, and context data and state conversion of the redirection object are maintained.

4. A source end video processing module: the same media type is multiplexed under the source screen projection example, and the source screen projection example is responsible for maintaining and uniformly processing the service logic of a certain media type.

5. A video encoder: the video media types are multiplexed under the source screen projection example, so that the same redirection object can only carry out one path of coding and is responsible for distributing to a plurality of screen projection links on the lower layer.

6. A source end video redirector: each specific screen projecting instance and the physical channel thereof corresponding to the source end are responsible for receiving the screen projecting request of the destination end and transmitting the media stream to the destination end without multiplexing.

7. Destination video redirector: and each specific screen projection instance and the physical channel thereof corresponding to the destination end are responsible for initiating a screen projection request to the source end and receiving and processing the media stream to the source end. Multiplexing under the same source device and redirection object provides uniform media stream receiving and processing for a plurality of display screens.

8. And the video decoder multiplexes the same source end device and the redirection object, ensures that the same redirection object can only decode one path, and provides decoded pictures for a plurality of display screens.

9. The application of the target screen projection window body is as follows: the multi-path media streams from the same display screen are multiplexed in one-to-one correspondence with the display screen of the destination terminal. And the display is responsible for displaying multiple paths of video pictures on the display screen.

10. The audio acquisition encoder: the device-level audio acquisition encoder is multiplexed for the entire source device. And the audio acquisition encoder at the application level is multiplexed at a certain screen projection application level. And uniformly recording and encoding the audio, and providing data for a plurality of audio links.

11. The audio decoding player: the device-level audio is multiplexed in units of each source device, and the application-level audio is multiplexed in units of each source device + redirection object.

The multiplexing strategy and the redirection process of the source device and the destination device are further described below with reference to the above hierarchical structure.

Referring to fig. 7, a flowchart of a redirection method according to another exemplary embodiment of the present application is shown, where the embodiment of the present application takes application of the method to the implementation environment shown in fig. 1 as an example, and the method includes:

step 701, a source device creates a source redirection instance based on a source device identifier and a target redirection object identifier, where the target redirection object identifier is used to represent a target redirection object.

The step 401 may be referred to in the implementation manner of this step, and this embodiment is not described herein again.

Step 702, the destination device creates a destination redirection instance based on the destination device identifier and a target screen identifier, where the target screen identifier is used to represent a target screen of the destination device.

The step 402 may be referred to in the implementation manner of this step, and details of this embodiment are not described herein.

In step 703, the source device generates redirection data corresponding to the source redirection instance through the multiplexed redirection data generator under the condition that the same source redirection instance corresponds to at least two destination redirection instances.

The step 403 may be referred to in the implementation manner of this step, and details of this embodiment are not described herein.

Step 704, the source device creates a source redirector when the at least two destination redirection instances correspond to the same destination device, or creates at least two source redirectors when the at least two destination redirection instances correspond to the at least two destination devices.

In a possible implementation manner, after the source device creates the source redirection instance, the source redirector is further created based on the source redirection instance, where the source redirector is used to transmit redirection data corresponding to the source redirection instance. Optionally, the source device creates a source redirector based on the source redirection instance and the destination device identifier.

In one possible implementation, when the same source device casts a same redirection object onto multiple screens of the destination device, the source device creates a source redirector. For example, when a game application needs to be simultaneously projected onto a first screen and a second screen of a first destination device, the source device creates a source redirector based on the source device identifier, the first destination identifier, and the game application identifier.

In another possible implementation, when the same source device casts a screen of the same redirection object to screens of different destination devices, the source device creates multiple source redirectors. For example, when a game application needs to be dropped on a first destination device and a second destination device, the source device creates a first source redirector based on the source device identifier, the first destination identifier, and the game application identifier, and creates a second source redirector based on the source device identifier, the second destination identifier, and the game application identifier.

Optionally, when performing redirection, a media type of the redirection object may be set (which may be manually set by a user or automatically set by the source device based on the capability of the destination device), where the media type includes at least one of video, audio, and control. When the media type of the source terminal redirection instance comprises a video, the destination terminal equipment can display a target redirection object through a screen; when the media type of the source terminal redirection instance comprises audio, the destination terminal device can play the sound of the target redirection object through the audio playing component; when the media type of the source redirection instance includes control, the destination device may reverse control over the target redirection object.

Correspondingly, the source device needs to respectively create source redirectors corresponding to different media types according to the media types corresponding to the target redirection object. In another possible implementation, the source device creates at least one source redirector based on the source redirection instance, the destination device identifier, and the media type corresponding to the target redirection object, where different source redirectors correspond to different media types.

Illustratively, when the media type corresponding to the target redirection object comprises a video, the source device creates a source video redirector, where the source video redirector is used for redirecting a video data stream of the target redirection object; when the media type corresponding to the target redirection object comprises audio, the source device creates a source audio redirector, and the source audio redirector is used for redirecting the audio data stream of the target redirection object; when the media type corresponding to the target redirection object includes control, the source device creates a source control redirector, and the source control redirector is used for redirecting the control data stream of the target redirection object.

For example, when a game application needs to be projected to a first destination device, and the first destination device supports displaying a game application picture, playing a game application sound, and reversely controlling the game application, the source device creates a source video redirector based on the source device identifier, the first destination identifier, the game application identifier, and the video type, and creates a source audio redirector based on the source device identifier, the first destination identifier, the game application identifier, and the audio type; and creating a source control redirector based on the source device identifier, the first destination identifier, the game application identifier and the control type.

Step 705, the destination device creates a destination redirector, and the destination redirector is multiplexed by at least two destination redirection instances.

Corresponding to the source device, the destination device needs to create a destination redirector corresponding to the source redirector, so as to transmit redirection data corresponding to the source redirector instance through a link between the source redirector and the destination redirector in the following.

Regarding the manner in which the destination redirector is created, in one possible implementation, the destination device knows the source device and the target redirection object, and thus creates the destination redirector based on the source device identification, the target redirection object identification, and the destination device identification.

In one possible implementation, when the same redirection object of the source device is projected to different destination devices, different destination devices create different destination redirectors. For example, when the source device casts a screen game application to the first destination device and the second destination device, the first destination device creates a first destination redirector based on the source device identifier, the first destination identifier, and the game application identifier, and the second destination device creates a second destination redirector based on the source device identifier, the second destination identifier, and the game application identifier.

In another possible implementation, when multiple screens of a destination device of a source device cast the same redirection object, the destination device creates a single destination redirector, and at this time, multiple destination redirection instance identifiers multiplex the same destination redirector. For example, when the source device simultaneously casts a game application to a first screen and a second screen of the first destination device, the first destination device creates a first destination redirector based on the source device identifier, the first destination identifier, and the game application identifier, and at this time, the first destination redirection instance created based on the first destination device identifier and the first screen identifier and the second destination redirection instance created based on the first destination device identifier and the second screen identifier multiplex the first destination redirector.

Optionally, when the user sets the media type of the source end redirection instance in the redirection process, the source end device may create different source end redirectors for different media types, and the destination end device needs to correspond to destination end redirectors for different media types.

In another possible implementation, the destination device creates at least one destination redirector based on the source redirection instance, the destination device identifier, and the media type corresponding to the target redirection object, where different destination redirectors correspond to different media types.

Illustratively, when the media type corresponding to the target redirection object includes a video, the destination device creates a destination video redirector, configured to receive a video data stream transmitted by the source video redirector; when the media type corresponding to the target redirection object comprises audio, the destination device creates a destination audio redirector, and the destination audio redirector is used for receiving an audio data stream transmitted by the source audio redirector; when the media type corresponding to the target redirection object includes control, the destination device creates a destination control redirector for transmitting a control data stream to the source control redirector.

For example, when the source device screens a game application to the first destination device, and the first destination device supports displaying a game application picture, playing a game application sound, and reversely controlling the game application, the destination device creates a destination video redirector based on the source device identifier, the first destination identifier, the game application identifier, and the video type, and creates a destination audio redirector based on the source device identifier, the first destination identifier, the game application identifier, and the audio type; and creating a destination control redirector based on the source device identifier, the first destination identifier, the game application identifier and the control type.

In step 706, the source device transmits redirection data to the destination device through a link between the source redirector and the destination redirector.

After the creation of the redirector is completed, the source device transmits redirection data to a corresponding destination redirector (the destination redirector and the source redirector correspond to the same source redirection instance) through the source redirector.

In one possible implementation, when a source device and a destination device create different redirectors for different media types, the source device transmits redirection data for at least one media type to the destination device via at least one link between at least one source redirector and one destination redirector, where the link between different source redirectors and destination redirectors is used for transmitting redirection data for different media types.

For example, when the source device projects a game application to the destination device, and the destination device supports displaying a game application picture and playing a game application sound, the source device transmits a video data stream to the destination video redirector of the destination device through the source video redirector, and transmits an audio data stream to the destination audio redirector of the destination device through the source audio redirector.

In step 707, the destination device receives the redirection data transmitted by the source device through the link between the destination redirector and the source redirector.

In one possible embodiment, the destination device receives redirection data of at least one media type transmitted by the source device via at least one link between at least one destination redirector and at least one source redirector.

Step 708, in the case that there are at least two destination redirection instances corresponding to the same source redirection instance, the destination device parses the redirection data through the multiplexed redirection data parser.

The step 406 may be referred to in the implementation manner of this step, and details thereof are not described in this embodiment of the present application.

And step 709, the destination device displays the target redirection object through the target screen based on the analysis result of the redirection data.

In a possible implementation manner, when the destination device has at least two screens, the destination device determines a target screen according to a target screen identifier in the destination redirection instance, and then displays the target screen through the target screen.

In some possible application scenarios, multiple projection screens (from the same source device or different source devices) need to simultaneously display the same screen (the same screen and multiple displays) in the destination device; the multi-path screen projection needs to be displayed on different screens (multi-screen different display) in the destination equipment; the same screen projection object of the source device needs to be displayed on different screens of the destination device at the same time (multi-screen simultaneous display).

In order to ensure the accuracy of screen projection display, in some embodiments, each screen of the destination device corresponds to a respective screen projection window application, and after receiving the redirection data, the destination device displays the redirection data through the target screen projection window application corresponding to the target screen. Taking the android system as an example, the screen-projecting window body application may be an Activity component.

Optionally, the screen projection window body application is correspondingly provided with a window application agent for controlling the life cycle of the screen projection window body application.

And aiming at the scene with the same screen and multiple displays, the target terminal equipment needs to reuse the application of the screen projection window body, and the multiple redirection objects are displayed in a split screen mode in the same screen.

In a possible implementation, presenting after the destination device receives the redirection data may include the following two possible cases.

1. Starting a target screen projection window body application in response to the target screen projection window body application corresponding to the target screen being in an un-started state; and displaying the target redirection object through the target screen projection window body application.

Optionally, after the displayed target screen is determined, the destination device detects whether the target screen projection window application corresponding to the target screen is in a starting state, if the target screen projection window application is in a non-starting state, it is indicated that no other screen projection object is displayed through the target screen at present, so that the target screen projection window application is started, and then the target screen projection window application is displayed, at this time, the target screen only displays the picture of the target redirection object.

2. And responding to the starting state of the target screen projection window body application, and displaying the target redirection object by multiplexing the target screen projection window body application, wherein after the target screen projection window body application is multiplexed, each screen projection object is displayed in a target screen in a split screen mode.

Optionally, when it is detected that the target screen-projecting window application is in a starting state, it indicates that the target screen is currently displaying other screen-projecting objects. The target terminal device needs to multiplex the target screen-projecting window application (that is, multiple source terminal redirection instances multiplex the same screen-projecting window application), and then performs screen-division display on other screen-projecting objects and the target redirection objects currently displayed on the target screen. The destination device may perform split-screen control based on the screen projecting direction (horizontal screen or vertical screen) of each screen projecting object and the number of screen projecting objects, which is not limited in the embodiment of the present application.

In some embodiments, when each source redirection instance in the screen-casting widget application exits, the destination device ends the life cycle of the screen-casting widget application, that is, destroys the screen-casting widget application.

In the embodiment of the application, the source device generates the redirection data through the multiplexing redirection data generator, the destination device analyzes the redirection data through the multiplexing redirection data analyzer, the encoding and decoding expenses of the source device and the destination device are reduced, in addition, the source device and the destination device respectively establish a source redirector and a destination redirector, and redirect data transmission is performed through a link between the redirectors, so that the accurate transmission of the redirection data is ensured.

In this embodiment of the present application, taking a media type of a target redirection object as an example of a video, a multiplexing policy adopted by a source device and a destination device is described, as shown in fig. 8, a video encoding/decoding and transmission process may include the following steps.

Step 801, a source device creates a source redirection instance based on a source device identifier and a target redirection object identifier, where the target redirection object identifier is used to represent a target redirection object.

Step 802, the destination device creates a destination redirection instance based on the destination device identifier and the target screen identifier, where the target screen identifier is used to represent a target screen of the destination device.

Step 803, the source device performs video coding on the picture corresponding to the same source redirection instance through the multiplexing video encoder to generate a video data stream.

Optionally, when the same redirection object of the same source device needs to be projected to different screens of the same destination device or different screens of different destination devices, the source device performs video coding on the picture of the source redirection instance by using a multiplexing video encoder to obtain a video data stream, instead of performing video coding by using a plurality of video encoders respectively according to the number of screens.

Schematically, as shown in fig. 3, when a screen projection application in a source device needs to be projected to a screen 1 and a screen 2 of a destination device 1 and a screen 3 of the destination device 2 at the same time, the source device multiplexes a video encoder 1 to perform video encoding, and obtains a video data stream.

Step 804, the source device creates a source video redirector when the at least two destination redirection instances correspond to the same destination device, or creates at least two source video redirectors when the at least two destination redirection instances correspond to the at least two destination devices.

Illustratively, as shown in fig. 3, when a screen projection application in a source device needs to be projected onto a screen 1 and a screen 2 of a destination device 1, the source device creates one source video redirector, and when the screen projection application in the source device needs to be projected onto the destination device 1 and the destination device 2 at the same time, the source device creates two source video redirectors.

At step 805, the destination device creates a destination video redirector.

Illustratively, as shown in fig. 3, when it is required to screen-cast a screen-casting application in a source device to a screen 1 and a screen 2 of a destination device 1, and the destination device creates a destination video redirector, when it is required to screen-cast the screen-casting application in the source device to the screen 1 and the screen 2 of the destination device 1, and to screen 3 of the destination device 2, the destination device creates a first destination video redirector for the destination device 1 and a second destination video redirector for the destination device 2.

In step 806, the source device transmits a video data stream to the destination device through a link between the source video redirector and the destination video redirector.

Schematically, as shown in fig. 3, since a screen-casting application needs to be screen-cast to a destination device 1 and a destination device 2, the source device needs to create a first source video redirector for the destination device 1 and a second source video redirector for the destination device 2, and transmit a video data stream obtained by encoding to the first source video redirector and the second source video redirector, further transmit the video data stream to the destination device 1 through the first source video redirector and the first destination video redirector, and transmit the video data stream to the destination device 2 through the second source video redirector and the second destination video redirector.

In step 807, the destination device receives the video data stream transmitted by the source device through the link between the destination video redirector and the source video redirector.

Illustratively, as shown in fig. 3, a screen projection application of a source device is projected to a destination device 1 and a destination device 2, the destination device 1 receives a video data stream through a link between a first destination video redirector and the first source video redirector, and the destination device 2 receives a video data stream through a link between a second destination video redirector and the second source video redirector.

Step 808, in the case that there are at least two destination redirection instances corresponding to the same source redirection instance, the destination device performs video decoding on the video data stream through the multiplexed video decoder to obtain a picture corresponding to the source redirection instance.

Optionally, when the same redirection object needs to be projected to at least two screens of the same destination device, the destination device performs video decoding on the video data stream of the source redirection instance by using the multiplexing video decoder to obtain video pictures, instead of performing video decoding by using a plurality of video decoders respectively according to the number of screens.

Illustratively, as shown in fig. 3, after the destination device 1 receives the video data stream, the multiplexing video decoder 1 performs video decoding, so that the decoded picture is displayed through the screen 1 and the screen 2.

And step 809, the destination device displays the target redirection object through the target screen based on the analysis result of the redirection data.

Schematically, as shown in fig. 3, the target device 1 displays a video through the screen 1 and the screen 2, and the target device 2 displays the video through the screen 3.

In the embodiment of the application, the source device multiplexes the video encoder, and the destination device multiplexes the video decoder, thereby reducing the encoding and decoding overhead in the process of transmitting the video data stream between the source device and the destination device.

When the media type corresponding to the target redirection object is audio, the above steps may be referred to, and a brief description thereof is provided below.

In another possible scenario, when the media type corresponding to the target redirection object includes audio, the source device needs to perform audio acquisition on the redirection object, encode the acquired audio to obtain an audio data stream, and transmit the audio data stream to the destination device for the destination device to perform parsing and playing. In a possible implementation manner, when the type of the target redirection object is a screen mirror image, the source device performs audio acquisition and encoding through a multiplexing device-level audio acquisition encoder to obtain an audio data stream, wherein the device-level audio acquisition encoder is used for acquiring and encoding the audio of the device; and when the type of the target redirection object is the redirection application, the source end equipment acquires and encodes audio through multiplexing an application-level audio acquisition encoder to obtain an audio data stream, wherein the application-level audio acquisition encoder is used for acquiring and encoding the audio of the redirection application.

Further, the source device transmits the audio data stream to the destination device through a link between the source audio redirector and the destination audio redirector. And the destination end equipment creates an audio redirector and receives the audio data stream transmitted by the source end equipment through a link between the destination end audio redirector and the source end audio redirector. And after receiving the audio data stream, the destination device performs audio decoding on the audio data stream through a multiplexing audio decoder to obtain the audio corresponding to the source end redirection instance. In the embodiment of the application, the source device multiplexes the audio acquisition encoder, and the destination device multiplexes the audio decoder to reduce the encoding and decoding overhead in the audio data stream transmission process between the source device and the destination device.

In a possible implementation, when the number of the destination devices is at least two, the source device shunts the redirected data to the at least two destination devices through at least two links between the at least two source redirectors and the at least two destination redirectors. Exemplarily, as shown in fig. 3, since the screen-casting application needs to be screen-cast to the destination device 1 and the destination device 2, the source device needs to create a first source video redirector for the destination device 1 and a second source video redirector for the destination device 2, and split a video data stream obtained by encoding to the first source video redirector and the second source video redirector, and then transmit the video data stream to the destination device 1 through the first source video redirector and the first destination video redirector, and transmit the video data stream to the destination device 2 through the second source video redirector and the second destination video redirector, so as to implement split transmission of the video data stream. Due to the adoption of the shunting strategy, the IO (input/output) overhead of data stream transmission of the source end equipment and the destination end equipment is reduced.

In a possible implementation manner, a first destination device and a second destination device asynchronously send screen-casting requests to the source device, and the source device transmits a video data stream to the first destination device and the second destination device through a link between a first source video redirector and a first destination video redirector and a link between a second source video redirector and a second destination video redirector sequentially. However, in this process, due to the fact that the first route screen of the first destination device and the second route screen of the second destination device have different access times, abnormal situations such as screen omission are easily caused.

Firstly, a simple introduction is made to a video coding process by taking an android system MediaCodec coding module as an example.

In this embodiment, the media type of the target redirection object is video, and the MediaCodec coding module of the source device samples a video picture at a certain frequency and codes the video picture into a plurality of I (Intra-coded picture) frames and a plurality of P (forward-coded picture) frames. The I frame, i.e., the key frame, is an independent frame with information for completing display, contains the entire amount of data, can be independently decoded without depending on other frames, and serves as a reference value for other frames. The P frame is a non-key frame, and the difference value between the current frame and the adjacent previous frame (I frame and P frame) is stored, and the forward frame is required to be referred for imaging. If any frame data of an I frame before the P frame and a plurality of P frames is lost or has errors, the pictures in a subsequent period of time are lost or become wrong, and the pictures are lost until the next I frame is recovered.

In addition, the MediaCodec encoding module generates a configuration (config) frame during encoding of a video Picture, i.e., header information of a video stream, records relevant parameters for encoding of the current video stream, and stores the parameters in an SPS (Sequence parameter Set) string and a PPS (Picture parameter Set) string. In addition, it contains information parameters needed to initialize the video decoder, such as encoded Profile, level, width of the picture, etc.

Further, the abnormal situations such as screen splash and the like generated when the first destination device and the second destination device are accessed asynchronously are explained by combining the video encoding process. For example, as shown in fig. 9, when a first route of the first destination device is switched on, the source device triggers the video encoder, and the configuration frame, the I frame, and the P frame generated by the video encoder are all sent to the first source video redirector, so that the first route of the video stream includes the complete configuration frame, the complete I frame, and the complete P frame. Since the second destination device is accessed after the first destination device, the second routing screen is usually accessed in a certain P frame, and at this time, if the second source-end video redirector receives the video frame, the second source-end video redirector lacks the configuration frame and the I frame, which may cause the destination device to display abnormally. In one possible embodiment, the missing configuration frame may cause the destination device decoder to be unable to decode; in another possible implementation, if the I frame is missing and the P frame after the I frame causes the display screen of the destination device to be displayed in a screen-splash manner. Therefore, in the embodiment of the present application, the shunting frame complementing method is adopted for the second screen projection to solve the above problem. The present invention provides a method for splitting and frame-filling, which is applicable to a mobile communication system. In addition, it should be noted that, in the embodiment of the present application, at least two destination devices exist, and the embodiment of the present application only takes a first destination device and a second destination device as an example for description, and the first destination device is accessed before the second destination device. Referring to fig. 10, a flowchart of a method for redirecting when accessing asynchronously according to an exemplary embodiment of the present application is shown.

Step 1001, the second destination device sends a screen projection request to the source device through a link between the second destination video redirector and the second source video redirector.

In a possible implementation manner, the first destination device is accessed before the second destination device, that is, after the first destination device sends the first screen projection request to the source device through the link between the first destination video redirector and the first source video redirector, the second destination device sends the second screen projection request to the source device through the link between the second destination video redirector and the second source video redirector.

In step 1002, in response to a screen-casting request of the second destination device, the source device obtains a configuration frame from a redirection context of the source redirection instance through the second source video redirector.

In this embodiment of the present application, a first destination device first sends a screen casting request to a source device, and after receiving the screen casting request sent by the first destination device, the source device starts a video encoder, synchronously reports a configuration frame generated by the video encoder to a redirection context, and sends all frames, that is, a configuration frame, an I frame, and a P frame, to a first source video redirector. And the source end equipment transmits the video data stream to the first destination end video redirector through a link between the first source end video redirector and the first destination end video redirector.

The source device maintains the key data and state of each screen projection by multiplexing the redirection context. The key data may include various video frames, screen projection parameters of the source device, screen projection parameters of the destination device, and the like, which is not limited in this embodiment of the present application. The state is used to indicate a connection state of the destination device and the source device, such as an access state, a screen projection state, an exit state, and the like, which is not limited in this embodiment of the present application.

When the second destination device sends a screen projection request to the source device, after the source device receives the screen projection request sent by the second destination device, the second source video redirector judges that the second source video redirector is not the first path screen projection, so that the source device obtains a configuration frame from the redirection context through the second source video redirector, and the configuration frame is generated by a video encoder during the first path screen projection.

In step 1003, the source device transmits a configuration frame to the second destination device through a link between the second source video redirector and the second destination video redirector.

Further, after the source device obtains the configuration frame, the source device transmits the configuration frame to a second destination video redirector of the second destination device through a link between the second source video redirector and the second destination video redirector separately.

In step 1004, the second destination device receives the configuration frame transmitted by the source device.

Further, the second destination device receives the configuration frame through the second destination video redirector.

In step 1005, the second destination device performs video decoding configuration based on the configuration frame.

Further, the video decoder of the second destination device configures the decoding parameters of the video decoder through the configuration frame. Wherein the decoding parameter may be the width of the image, etc.

In response to the screen-cast request of the second destination device, the source device retrieves a rotated frame from the redirection context through the second source video redirector, step 1006.

In one possible implementation, since horizontal and vertical screen rotation may occur during the video picture projection process, a rotation (Rotate) frame may be generated during the video encoder encoding the video picture. Since the second destination device is accessed after the first destination device, when the second drop screen is accessed, usually at a certain P frame, if the second source video redirector receives the video frame at this time, the rotated frame will be lost, which may cause the picture display direction of the destination device and the picture display direction of the source device to be inconsistent, and may differ by 90 °, 180 °, or 270 °.

In this embodiment of the present application, a first destination device first sends a screen projection request to a source device, and the source device receives the screen projection request sent by the first destination device, starts a video encoder, synchronously reports a rotating frame generated by the video encoder to a redirection context, and sends all frames, that is, a configuration frame, an I frame, a P frame, and the rotating frame, to a first source video redirector.

When the second destination device sends a screen projection request to the source device, after the source device receives the screen projection request sent by the second destination device, the second source video redirector judges that the second source video redirector is not the first path screen projection, so that the source device obtains a rotating frame from the redirection context through the second source video redirector, and the rotating frame is generated by the video encoder during the first path screen projection.

Step 1007, the source device transmits the rotating frame to the second destination device through the link between the second source video redirector and the second destination video redirector.

Further, after the source device obtains the rotation frame, the source device transmits the rotation frame to the second destination device through a link between the second source video redirector and the second destination video redirector.

In step 1008, the second destination device receives the rotation frame transmitted by the source device.

Further, the second destination device receives the rotated frame through the second destination video redirector.

In step 1009, the second destination device configures a display direction of the frame based on the rotating frame.

The video decoder of the second destination device configures the display direction of the destination device picture, which is identical to the display direction of the source device picture, by rotating the frame.

Step 1010, using the first key frame after the asynchronous access request as a starting point, the source device transmits a video data stream to the second destination device through a link between the second source video redirector and the second destination video redirector.

In a possible implementation, the source device sends a requestkeyframe request to the video encoder, and requests a first I frame after an asynchronous access request of a second destination device to arrive, and when the I frame arrives, a P frame is skipped until the I frame arrives, and the source device transmits a video data stream to the second destination device through a link between a second source video redirector and a second destination video redirector.

In another possible implementation, the source device does not need to send a request to the video encoder, and if a first I frame after an asynchronous access request of a second destination device does not arrive, waits for an I frame to arrive, and a P frame is skipped until the I frame arrives, and the source device transmits a video data stream to the second destination device through a link between a second source video redirector and a second destination redirector.

In step 1011, the second destination device receives the video data stream transmitted by the source device.

Further, the second destination device receives the video data stream transmitted by the source device through the second destination video redirector.

The foregoing method of shunting frame complementing is schematically illustrated in conjunction with fig. 10. The first route screen of the first destination device is accessed first, and the source device triggers the video encoder to generate a configuration frame, an I frame, a P frame and a rotation frame, and simultaneously synchronously reports the configuration frame and the rotation frame to the redirection context. Further, all frames are sent to the first source end video redirector, and then the source end device transmits the video data stream through a link between the first source end video redirector and the first destination end video redirector.

And a second cast screen of the second destination device is accessed at a certain P frame, and at this time, a second source video redirector of the source device directly acquires the configuration frame and the rotation frame from the redirection context, and transmits the configuration frame and the rotation frame to the second destination through a link between the second source video redirector and the second destination video redirector. Further, the source device sends a RequestKeyFrame request to the video encoder to request that the next I frame arrives as soon as possible, and when the I frame arrives, the P frame is skipped until the I frame arrives, and then the source device sends a video data stream to the second destination device through a link between the second source video redirector and the second destination video redirector.

In the embodiment of the application, when the first destination end device and the second destination end device are asynchronously accessed, a method of shunting and supplementing frames is adopted for second screen projection of the second destination end device, that is, a configuration frame and a rotation frame are obtained from a redirection context through a second source end video redirector and are transmitted to the second destination end video redirector, and a source end device and the like send a video data stream to the second destination end device by taking a first key frame after the asynchronous access request of the second destination end device as a starting point, so that abnormal situations such as screen splash and the like of a display picture of the destination end device are avoided.

In another possible implementation manner, the first destination device and the second destination device synchronously send a screen projection request to the source device, and at this time, the first destination device and the second destination device access concurrently, so that the data and the state shared by the first destination device and the second destination device are modified simultaneously in competition, thereby affecting the screen projection quality. In the embodiment of the present application, the screen projection parameters are negotiated by using the first way screen projection as a main negotiation object, so as to solve the above problem. Referring to fig. 11, a redirection method during concurrent access is described by taking a first destination device and a second destination device as examples, it should be noted that at least two destination devices exist in the embodiment of the present application, and the embodiment of the present application only takes the first destination device and the second destination device as examples for description. The first destination device is a first access, and the second destination device is a second access.

Step 1101, the second destination device sends an access request to the source device through a link between the second destination video redirector and the second source video redirector.

The access request is used for establishing connection between the second destination-end video redirector and the second source-end video redirector and creating a data stream channel.

Step 1102, in response to an access request of a second destination device, a source device obtains an instance identifier from a redirection context of a source redirection instance through a second source video redirector.

In this embodiment of the present application, a destination device serving as a first access sends an access request to a source device through a link between a first destination video redirector and a first source video redirector, and further, the source device receives the first access request, and the first source video redirector creates an instance identifier and synchronously reports the instance identifier to a redirection context. And simultaneously, the source end equipment sends the instance identifier to the first destination end equipment through a link between the first source end video redirector and the first destination end video redirector. The source device manages the key data and state of each video stream by multiplexing the redirection context.

Therefore, after the source device receives the access request sent by the second destination device as the second path, the source device directly obtains the instance identifier from the redirection context through the second source video redirector.

The instance identifier is determined based on the source device identifier and the target redirection object identifier of the target redirection object to be redirected.

Step 1103, the source device sends the instance identifier to the second destination device through a link between the second source video redirector and the second destination video redirector.

Further, after the source device obtains the instance identifier, the second source video redirector sends the instance identifier to the second destination video redirector through a link between the second source video redirector and the second destination video redirector.

In step 1104, the second destination device receives an instance identifier of the source redirection instance sent by the source device.

Further, the second destination video redirector receives the instance identifier, determines the source device based on the device identifier, confirms the content displayed by the destination device based on the target redirection object identifier, and further initializes the destination window application.

In a possible implementation manner, if the first destination device accessed in the first path has initialized the destination window application, the second destination device directly multiplexes the destination window application, and the destination window application does not need to be initialized again by the second destination video redirector.

In another possible implementation, if the first destination device accessed in the first path does not initialize the destination window application or the destination window application is destroyed, the second destination device needs to initialize the destination window application through the second destination video redirector, and then other accessed destination devices can directly reuse the destination window application.

In step 1105, the second destination device sends a parameter obtaining request to the source device through a link between the second destination video redirector and the second source video redirector.

Further, the second destination device sends a parameter obtaining request to the source device for capability negotiation.

In response to the parameter obtaining request of the second destination device, the source device obtains, through the second source video redirector, the target screen projection parameter from the redirection context of the source redirection instance, step 1106.

In this embodiment of the present application, a source end video redirector (including a first source end video redirector and a second source end video redirector) of a source end device receives a parameter obtaining request of a destination end device, and determines whether the source end video redirector is a first destination end device accessed by a first path or a second destination end device accessed by a second path. In a possible implementation manner, if the first path is a first destination device accessed, the first source video redirector determines a target screen projection parameter based on the source screen projection parameter and the first destination screen projection parameter, and synchronously reports the target screen projection parameter to the redirection context.

In another possible implementation, if the second destination device is a second access destination device, the second source video redirector directly obtains the target screen projection parameters from the screen re-projection context.

In addition, it should be noted that, in another possible implementation, the target screen projection parameter may also be determined in real time according to the source-end screen projection parameter and the current destination-end screen projection parameter. For example, if the first path is accessed to the first destination device, the first source video redirector determines the target screen projection parameter based on the source screen projection parameter and the first destination screen projection parameter, and if the second path is accessed to the second destination device, the second source video redirector determines the target screen projection parameter based on the source screen projection parameter and the second destination screen projection parameter.

The screen projection parameters may be screen resolution, refresh rate, delay, and the like, which are not limited in the embodiment of the present application.

In step 1107, the source device sends the target screen projection parameter to the second destination device through the link between the second source video redirector and the second destination video redirector.

Further, the source end device sends the finally determined target screen projection parameters to a second destination end device through a link between a second source end video redirector and a second destination end video redirector.

In step 1108, the second destination device receives the target screen projection parameter sent by the source device.

Further, a second destination video redirector of the second destination device receives the target screen projection parameters.

In step 1109, the second destination device initializes the video decoder based on the target projection parameters.

Further, after receiving the target screen projection parameter, the second destination video redirector initializes the video decoder according to the target screen projection parameter and sends a screen projection request to the source end device.

The above process is exemplarily described with reference to fig. 12.

And the second destination device sends an access request to the source device, and the second destination video redirector establishes connection to the second source video redirector to create a video data stream channel. Further, the second source-end video redirector obtains the instance identifier by multiplexing the redirection context and informs the second destination-end video redirector to use for binding the instance identifier. And the second destination video redirector acquires the instance identifier and requests to initialize the destination window application, and if the destination window application already exists, the second destination video redirector is directly multiplexed. Further, a second destination video redirector sends a parameter acquisition request to the second source video redirector, after receiving the parameter request, the second source video redirector judges whether the second destination video redirector is used as a first destination device accessed by a first path or is used as a second destination device accessed by a second path, and if the second destination device accessed by the first path is the destination device accessed by the first path, the second destination video redirector determines a target screen projection parameter by combining the source screen projection parameter and the first destination screen projection parameter, and synchronously reports the target screen projection parameter to a redirection context. And if the target screen projection parameter is the second destination device accessed by the second path, the second source end video redirector directly acquires the target screen projection parameter from the redirection context and sends the target screen projection parameter to the second destination end video redirector. Further, the second destination video redirector receives the target screen projection parameters, initializes the video decoder according to the parameters, and sends a screen projection request to the second source video redirector after the video decoder is initialized. If a screen projection request is sent by a first destination device accessed in a first path, a source device creates a video encoder, sends a configuration frame, an I frame, a P frame and a rotating frame generated by the video encoder to a first source video redirector, and synchronously reports the configuration frame and the rotating frame to a redirection context. Further, the source device sends the video data stream to the first destination video redirector through the first source video redirector. And if a screen projection request is sent to a second destination device accessed by a second path, the second source end video redirector directly acquires a configuration frame and a rotation frame from a redirection context, further, the source end device sends a video data stream to the second destination end video redirector through the second source end video redirector, and after receiving the video data stream, the second destination end video redirector analyzes the video data stream through a video decoder and sends the video data stream to rendering.

In this embodiment of the present application, when the first destination device and the second destination device access concurrently, the second destination device initializes the video decoder by obtaining, from the redirection context, the target screen projection parameter that has been determined based on the source screen projection parameter and the first destination screen projection parameter, so as to avoid modification of shared data when the first destination device and the second destination device access concurrently, and improve the quality of the display picture.

It should be noted that, in the foregoing embodiment, an interaction flow of the source device and the destination device in a redirection process is taken as an example for description, where a step of taking the source device as an execution main body may be separately implemented as a redirection method of the source device side, and a step of taking the destination device as an execution main body may be separately implemented as a redirection method of the destination device side, and this embodiment of the present application is not described herein again.

The following are embodiments of the apparatus of the present application that may be used to perform embodiments of the method of the present application. For details which are not disclosed in the embodiments of the apparatus of the present application, reference is made to the embodiments of the method of the present application.

Referring to fig. 13, a block diagram of a redirection device according to an embodiment of the present application is shown. The apparatus may include:

a creating module 1301, configured to create a source redirection instance based on a source device identifier and a target redirection object identifier, where the target redirection object identifier is used to represent a target redirection object;

a multiplexing module 1302, configured to generate, by a multiplexing redirection data generator, redirection data corresponding to a source redirection instance when the same source redirection instance corresponds to at least two destination redirection instances, where the destination redirection instance is created based on a destination device identifier and a target screen identifier, where the destination device identifier is used to represent a destination device, and the target screen identifier is used to represent a target screen of the destination device;

a first transmission module 1303, configured to transmit the redirection data to the destination device, so that the destination device displays the target redirection object through the target screen based on the redirection data.

Optionally, the first transmission module 1303 includes:

a creating unit, configured to create a source redirector if at least two destination redirection instances correspond to the same destination device, or create at least two source redirectors if at least two destination redirection instances correspond to at least two destination devices;

a transmission unit, configured to transmit the redirection data to the destination device through a link between the source redirector and the destination redirector, where the destination redirector is created by the destination device.

Optionally, the number of the destination devices is at least two;

the transmission unit is further configured to:

and shunting and transmitting the redirection data to at least two pieces of destination equipment through at least two links between at least two source end redirectors and at least two destination end redirectors.

Optionally, the media type corresponding to the target redirection object is a video;

the multiplexing module 1302 is further configured to:

performing video coding on the picture corresponding to the same source end redirection instance through a multiplexing video coder to generate a video data stream;

the transmission unit is further configured to:

and transmitting the video data stream to the destination device through a link between the source end video redirector and the destination end video redirector.

Optionally, the at least two destination devices include a first destination device and a second destination device;

the transmission unit is further configured to:

transmitting the video data stream to the first destination device through a link between a first source-end video redirector and a first destination-end video redirector;

and transmitting the video data stream to the second destination device through a link between a second source end video redirector and a second destination end video redirector.

Optionally, the first destination device and the second destination device are accessed asynchronously, and the first destination device is accessed before the second destination device;

the device further comprises:

a first obtaining module, configured to, in response to a screen-casting request of the second destination device, obtain, by the second source-side video redirector, a configuration frame from a redirection context of the source-side redirection instance, where the configuration frame is reported to the redirection context by the video encoder;

a second transmission module, configured to transmit the configuration frame to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector, so that the second destination device performs video decoding configuration based on the configuration frame.

Optionally, the transmission unit is further configured to:

and transmitting the video data stream to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector by taking the first key frame after the asynchronous access request as a starting point.

Optionally, the apparatus further comprises:

a second obtaining module, configured to, in response to the screen-casting request, obtain, by the second source-side video redirector, a rotating frame from the redirection context, where the rotating frame is reported to the redirection context by the video encoder when the source-side device rotates;

a third transmitting module, configured to transmit the rotated frame to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector, so that the second destination device configures a display direction of a picture based on the rotated frame.

Optionally, the first destination device and the second destination device are concurrently accessed, and the first destination device is a first access and the second destination device is a second access;

the device further comprises:

a third obtaining module, configured to, in response to an access request of the second destination device, obtain, through the second source-side video redirector, an instance identifier from a redirection context of the source-side redirection instance;

a first sending module, configured to send the instance identifier to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector;

a fourth obtaining module, configured to, in response to a parameter obtaining request of the second destination device, obtain, by the second source-side video redirector, a target screen projection parameter from a redirection context of the source-side redirection instance, where the target screen projection parameter is determined by the first source-side video redirector based on a source-side screen projection parameter and a destination-side screen projection parameter of the first destination device, and report the target screen projection parameter to the redirection context;

and the second sending module is configured to send the target screen projection parameter to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector.

Optionally, the media type corresponding to the target redirection object is audio;

a multiplexing module 1302, further configured to:

under the condition that the type of the target redirection object is a screen mirror image, audio acquisition and encoding are carried out through a multiplexing equipment-level audio acquisition encoder to obtain an audio data stream;

under the condition that the type of the target redirection object is application, audio acquisition and encoding are carried out through a multiplexing application-level audio acquisition encoder to obtain an audio data stream;

the transmission unit is further configured to:

and transmitting the audio data stream to the destination device through a link between the source audio redirector and the destination audio redirector.

To sum up, in the embodiment of the present application, in a redirection process, a source device creates a source redirection instance based on a source device identifier and a target redirection object identifier, a destination device creates a destination redirection instance corresponding to the source redirection instance based on a destination device identifier and a target screen identifier, the source device generates redirection data and transmits the redirection data to the destination device through a multiplexed redirection data generator under the condition that the same source redirection instance corresponds to at least two destination re-instances, the destination device receives the redirector data, parses the redirection data through a multiplexed redirection data parser, and displays a target redirection object through a target screen indicated by the destination redirection instance; by adopting the scheme provided by the embodiment of the application, the encoding cost of the source end equipment and the decoding cost of the target end equipment in the data stream transmission process between the source end equipment and the target end equipment are reduced through the multiplexing redirection data generator and the redirection data analyzer, and the data transmission performance of a complex redirection scene is improved.

Referring to fig. 14, a block diagram of a redirection device according to another embodiment of the present application is shown. The apparatus may include:

a creating module 1401, configured to create a destination redirection instance based on a destination device identifier and a target screen identifier, where the target screen identifier is used to represent a target screen of the destination device;

a first receiving module 1402, configured to receive redirection data corresponding to a source redirection instance transmitted by a source device, where the source redirection instance is created based on a source device identifier and a target redirection object identifier, and the target redirection object identifier is used to represent a target redirection object;

a multiplexing module 1403, configured to parse, by using a multiplexing redirection data parser, the redirection data when there are at least two destination redirection instances corresponding to the same source redirection instance;

a displaying module 1404, configured to display the target redirection object through the target screen based on the analysis result of the redirection data.

Optionally, the first receiving module 1402 includes:

a creating unit, configured to create a destination redirector, where the destination redirector is multiplexed by at least two of the destination redirection instances;

a receiving unit, configured to receive the redirection data transmitted by the source device through a link between the destination redirector and the source redirector, where the source redirector is created by the source device.

the creating unit is further configured to:

creating a destination video redirector;

the receiving unit is further configured to:

receiving a video data stream transmitted by the source-end device through a link between the destination-end video redirector and the source-end video redirector;

the multiplexing module 1403 is further configured to:

and carrying out video decoding on the video data stream through a multiplexing video decoder to obtain a picture corresponding to the source end redirection instance.

Optionally, the apparatus further comprises:

the first sending module is used for sending a screen projection request to the source-end equipment through a link between the destination-end video redirector and the source-end video redirector;

a second receiving module, configured to receive a configuration frame transmitted by the source device, where the configuration frame is obtained by the source video redirector from a redirection context of the source redirection instance when there is access of another destination device before the destination device;

and the first configuration module is used for carrying out video decoding configuration based on the configuration frame.

Optionally, the apparatus further comprises:

a third receiving module, configured to receive a rotating frame transmitted by the source device, where the rotating frame is obtained from the redirection context by the source video redirector when there is access of another destination device before the destination device, and the rotating frame is reported to the redirection context by the video encoder when the source device rotates;

and the second configuration module is used for configuring the display direction of the picture based on the rotating frame.

Optionally, the apparatus further comprises:

a second sending module, configured to send an access request to the source device through a link between the destination video redirector and the source video redirector;

a fourth receiving module, configured to receive an instance identifier of the source redirection instance sent by the source device, where the instance identifier is obtained by the source video redirector from a redirection context of the source redirection instance when there is access of another destination device before the destination device;

a third sending module, configured to send a parameter obtaining request to the source device through a link between the destination video redirector and the source video redirector;

a fifth receiving module, configured to receive a target screen projection parameter sent by the source device, where the target screen projection parameter is obtained by the source video redirector from a redirection context of the source redirection instance when there is access of another destination device before the destination device;

an initialization module for initializing the video decoder based on the target screen projection parameters.

the creating unit is further configured to:

creating a destination audio redirector;

the receiving unit is further configured to:

receiving an audio data stream transmitted by the source-end equipment through a link between the destination-end video redirector and the source-end video redirector;

the multiplexing module 1403 is configured to:

and carrying out audio decoding on the audio data stream through a multiplexing audio decoder to obtain the audio corresponding to the source end redirection instance.

Referring to fig. 15, a block diagram of a system architecture of a redirection system according to an embodiment of the present application is shown. The system comprises: at least one source end device 1510 and at least one destination end device 1520.

A source device 1510, configured to create a source redirection instance based on a source device identifier and a target redirection object identifier, where the target redirection object identifier is used to represent a target redirection object; under the condition that the same source terminal redirection instance corresponds to at least two destination terminal redirection instances, generating redirection data corresponding to the source terminal redirection instance through a multiplexing redirection data generator; transmitting redirection data to the destination device so that the destination device can display the target redirection object through the target screen based on the redirection data;

a destination device 1520, configured to create a destination redirection instance based on a destination device identifier and a target screen identifier, where the target screen identifier is used to represent a target screen of the destination device; receiving redirection data corresponding to a source terminal redirection instance transmitted by source terminal equipment; analyzing the redirected data through a multiplexing redirected data analyzer under the condition that at least two destination terminal redirected instances correspond to the same source terminal redirected instance; and displaying the target redirection object through the target screen based on the analysis result of the redirection data.

It should be noted that, for a specific process of performing redirection on the source device 1510 and the destination device 1520, reference may be made to the foregoing method embodiment, and details of this embodiment are not described herein again.

Referring to fig. 16, a block diagram of a computer device according to an exemplary embodiment of the present application is shown. The computer device 1600 may be implemented as a source device or a destination device in the various embodiments described above. Computer device 1600 may include one or more of the following components: a processor 1610 and a memory 1620.

Processor 1610 may include one or more processing cores. The processor 1610, using various interfaces and circuitry, connects various components throughout the computer device 1600, performs various functions of the computer device 1600 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 1620, and invoking data stored in the memory 1620. Alternatively, the processor 1610 may be implemented in hardware using at least one of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1610 may integrate one or a combination of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Neural-Network Processing Unit (NPU), a modem, and the like. Wherein, the CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing contents required to be displayed by the touch display screen; the NPU is used for realizing an Artificial Intelligence (AI) function; the modem is used to handle wireless communications. It is understood that the modem may not be integrated into the processor 1610, but may be implemented by one chip.

The Memory 1620 may include a Random Access Memory (RAM) and a Read-Only Memory (ROM). Optionally, the memory 1620 comprises a non-transitory computer-readable storage medium. The memory 1620 may be used to store instructions, programs, code sets, or instruction sets. The memory 1620 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, and the like; the stored data area may store data (such as audio data, a phonebook) created from use of the computer device 1600, and the like.

In addition, those skilled in the art will appreciate that the configuration of computer device 1600 illustrated in the above-described figures does not constitute a limitation of computer devices, which may include more or fewer components than illustrated, or some components may be combined, or a different arrangement of components. For example, the computer device 1600 further includes a display screen, a camera assembly, a microphone, a speaker, a radio frequency circuit, an input unit, a sensor (such as an acceleration sensor, an angular velocity sensor, a light sensor, etc.), an audio circuit, a WiFi module, a power supply, a bluetooth module, etc., which are not described herein again.

An embodiment of the present application further provides a computer-readable storage medium, which stores at least one program code, and the program code is loaded and executed by a processor to implement the redirection method according to the above embodiments.

Embodiments of the present application provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the redirection method provided in the various alternative implementations of the above aspects.

It should be understood that reference to "a plurality" herein means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. In addition, the step numbers described herein only exemplarily show one possible execution sequence among the steps, and in some other embodiments, the steps may also be executed out of the numbering sequence, for example, two steps with different numbers are executed simultaneously, or two steps with different numbers are executed in a reverse order to the order shown in the figure, which is not limited by the embodiment of the present application.

The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A redirection method, wherein the method is used for a source device, and the method includes:

2. The method of claim 1, wherein the transmitting the redirection data to the destination device comprises:

creating a source redirector under the condition that at least two destination redirection instances correspond to the same destination device, or creating at least two source redirectors under the condition that at least two destination redirection instances correspond to at least two destination devices;

and transmitting the redirection data to the destination device through a link between the source redirector and the destination redirector, wherein the destination redirector is created by the destination device.

3. The method of claim 2, wherein the destination devices are at least two;

the transmitting the redirection data to the destination device through the link between the source redirector and the destination redirector includes:

4. The method of claim 2, wherein the media type corresponding to the target redirection object is video;

the generating redirection data corresponding to the source end redirection instance by the multiplexing redirection data generator includes:

5. The method of claim 4, wherein the at least two destination end devices include a first destination end device and a second destination end device;

the transmitting the video data stream to the destination device through the link between the source video redirector and the destination video redirector includes:

6. The method of claim 5, wherein the first destination device and the second destination device are accessed asynchronously, and wherein the first destination device is accessed before the second destination device;

the method further comprises the following steps:

responding to a screen-casting request of the second destination device, acquiring a configuration frame from a redirection context of the source redirection instance through the second source video redirector, wherein the configuration frame is reported to the redirection context by the video encoder;

and transmitting the configuration frame to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector, so that the second destination device performs video decoding configuration based on the configuration frame.

7. The method of claim 6, wherein transmitting the video data stream to the second destination device over a link between a second source-side video redirector and a second destination-side video redirector comprises:

8. The method of claim 6, further comprising:

in response to the screen-casting request, obtaining, by the second source video redirector, a rotated frame from the redirection context, the rotated frame being reported by the video encoder to the redirection context as the source device rotates;

and transmitting the rotation frame to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector, so that the second destination device configures a display direction based on the rotation frame.

9. The method according to claim 5, wherein the first destination device and the second destination device are concurrently accessed, and the first destination device is a first access and the second destination device is a second access;

the method further comprises the following steps:

obtaining, by the second source-side video redirector, an instance identifier from a redirection context of the source-side redirection instance in response to an access request of the second destination-side device;

sending the instance identifier to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector;

responding to a parameter acquisition request of the second destination device, acquiring a target screen projection parameter from a redirection context of the source terminal redirection instance through the second source terminal video redirector, wherein the target screen projection parameter is determined by the first source terminal video redirector based on a source terminal screen projection parameter and a destination terminal screen projection parameter of the first destination device, and reporting to the redirection context;

and sending the target screen projection parameters to the second destination device through a link between the second source-end video redirector and the second destination-end video redirector.

10. The method of claim 2, wherein the media type corresponding to the target redirection object is audio;

11. A redirection method, wherein the method is used for a destination device, and wherein the method comprises:

12. The method of claim 11, wherein receiving redirection data corresponding to a source redirection instance transmitted by a source device comprises:

creating a destination redirector, said destination redirector being multiplexed by at least two of said destination redirection instances;

and receiving the redirection data transmitted by the source end equipment through a link between the destination end redirector and the source end redirector, wherein the source end redirector is created by the source end equipment.

13. The method of claim 12, wherein the media type corresponding to the target redirection object is video;

the creating of the destination redirector comprises the following steps:

creating a destination video redirector;

the receiving, by the link between the destination redirector and the source redirector, the redirection data transmitted by the source device includes:

parsing the redirection data by a multiplexed redirection data parser comprising:

14. The method of claim 13, wherein before receiving the video data stream transmitted by the source device, the method further comprises:

sending a screen projection request to the source-end equipment through a link between the destination-end video redirector and the source-end video redirector;

receiving a configuration frame transmitted by the source device, wherein the configuration frame is obtained by the source video redirector from a redirection context of the source redirection instance when other destination device accesses exist before the destination device;

and performing video decoding configuration based on the configuration frame.

15. The method of claim 14, further comprising:

receiving a rotating frame transmitted by the source device, wherein the rotating frame is obtained from the redirection context by the source video redirector under the condition that other destination devices are accessed before the destination device, and the rotating frame is reported to the redirection context by the video encoder when the source device rotates;

and configuring the display direction of the picture based on the rotating frame.

16. The method of claim 13, further comprising:

sending an access request to the source end equipment through a link between the destination end video redirector and the source end video redirector;

receiving an instance identifier of the source end redirection instance sent by the source end device, wherein the instance identifier is obtained by the source end video redirector from a redirection context of the source end redirection instance when other destination end devices are accessed before the destination end device;

sending a parameter acquisition request to the source-end equipment through a link between the destination-end video redirector and the source-end video redirector;

receiving a target screen projection parameter sent by the source end device, wherein the target screen projection parameter is obtained by the source end video redirector from a redirection context of the source end redirection instance under the condition that other destination end devices are accessed before the destination end device;

initializing the video decoder based on the target projection parameters.

17. The method of claim 12, wherein the media type corresponding to the target redirection object is audio;

the creating of the destination redirector comprises the following steps:

creating a destination audio redirector;

18. A redirection apparatus, wherein the apparatus is used for a source device, and the apparatus comprises:

19. A redirection apparatus, wherein the apparatus is used for a destination device, the apparatus comprising:

the multiplexing module is used for analyzing the redirection data through a multiplexing redirection data analyzer under the condition that at least two destination redirection instances correspond to the same source redirection instance;

20. A redirection system, wherein the redirection system comprises: at least one source end device and at least one destination end device;

21. A computer device, wherein the computer device comprises a processor and a memory; the memory stores at least one instruction for execution by the processor to implement a redirection method as claimed in any one of claims 1 to 10, or to implement a redirection method as claimed in any one of claims 11 to 17.

22. A computer-readable storage medium, in which at least one program code is stored, which is loaded and executed by a processor to implement the redirection method according to any of claims 1 to 10 or to implement the redirection method according to any of claims 11 to 17.

23. A computer program product, characterized in that the computer program product comprises computer instructions, the computer instructions being stored in a computer readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, the processor executing the computer instructions to cause the computer device to perform the redirection method according to any of claims 1 to 10, or the redirection method according to any of claims 11 to 17.