CN116126260A

CN116126260A - Screen projection method and screen projection device

Info

Publication number: CN116126260A
Application number: CN202111339528.8A
Authority: CN
Inventors: 黄娟娟; 文汉成; 阮方燃; 曹泽华
Original assignee: Samsung Guangzhou Mobile R&D Center; Samsung Electronics Co Ltd
Current assignee: Samsung Guangzhou Mobile R&D Center; Samsung Electronics Co Ltd
Priority date: 2021-11-12
Filing date: 2021-11-12
Publication date: 2023-05-16

Abstract

The screen projection method is applied to first electronic equipment and comprises the following steps: determining, in response to receiving a screen-casting request for at least one second application, screen-casting configuration information of the first application and the at least one second application under the condition that the first application is screen-cast from the first electronic device to the second electronic device; and sending the screen projection configuration information, the screen projection data of the first application and the screen projection data of the at least one second application to the second electronic equipment so that the first application and the at least one second application are projected to the second electronic equipment, and carrying out the same-screen display according to the screen projection configuration information.

Description

Screen projection method and screen projection device

Technical Field

The disclosure relates to the technical field of electronic terminals, and more particularly, to a screen projection method and a screen projection device.

Background

With the development of communication technology, electronic terminals already have a screen-throwing function. For example, a conference moderator needs to screen multiple APPs (e.g., PPT, financial news, syn-florist, etc.) on an electronic terminal to other electronic terminals simultaneously in order to make cross App presentation and data presentation to conference participants. On the other hand, other notifications that are irrelevant to the meeting, other than the App that needs to make the screen, do not need to make the screen. However, existing electronic terminals generally only support screen projection of a single application (App), but cannot simultaneously screen a plurality of apps to other electronic terminals for on-screen display.

Disclosure of Invention

The disclosure provides a screen projection method and a screen projection device for supporting screen projection of a plurality of applications for on-screen display.

According to an aspect of the present disclosure, there is provided a screen projection method, which is applied to a first electronic device, and includes: determining, in response to receiving a screen-casting request for at least one second application, screen-casting configuration information of the first application and the at least one second application under the condition that the first application is screen-cast from the first electronic device to the second electronic device; and sending the screen projection configuration information, the screen projection data of the first application and the screen projection data of the at least one second application to the second electronic equipment so that the first application and the at least one second application are projected to the second electronic equipment, and carrying out the same-screen display according to the screen projection configuration information.

Optionally, the screen projection method further includes: receiving multi-application screen-casting information from the second electronic device, wherein the step of determining screen-casting configuration information of the first application and the at least one second application comprises: based on the multi-application screen projection information received from the second electronic device, screen projection configuration information of the first application and the at least one second application is determined.

Optionally, the step of receiving multi-application screen-cast information from the second electronic device includes: and in response to receiving the screen projection request, sending a multi-application screen projection negotiation request to the second electronic device, and receiving the multi-application screen projection information from the second electronic device.

Optionally, sending the multi-application screen-projection negotiation request to the second electronic device through a media negotiation channel, receiving the multi-application screen-projection information from the second electronic device, and sending the screen-projection configuration information to the second electronic device; and/or transmitting the screen projection data of the first application and the screen projection data of the at least one second application to the second electronic equipment through a streaming media channel.

Optionally, the multi-application screen-projection negotiation request includes information for requesting the second electronic device to simultaneously display a plurality of applications, and the multi-application screen-projection information includes size information of a display screen of the second electronic device.

Optionally, the step of determining the screen-drop configuration information of the first application and the at least one second application includes: generating layout information for displaying each application of the first application and the at least one second application on the second electronic device based on the size information of the display screen of the second electronic device and the screen throwing sequence information of each application of the first application and the at least one second application, wherein the layout information is used as screen throwing configuration information of the first application and the at least one second application.

Optionally, the step of determining the screen-drop configuration information of the first application and the at least one second application further comprises: displaying the layout information on a display screen of the first electronic device in the form of a preview; and generating modified layout information as screen configuration information of the first application and the at least one second application in response to a modification operation for the layout information displayed in the form of a preview.

Optionally, the layout information includes position information and size information of each window for displaying each application on a display screen of the second electronic device.

Optionally, the step of sending the screen projection data of the first application and the screen projection data of the at least one second application to the second electronic device includes: adjusting a size of a first virtual display for on-screen display of the first application to be the same as a size of a window for displaying the first application on the second electronic device, wherein the first virtual display is created when the first application is on-screen to the second electronic device; creating at least one second virtual display for on-screen display of the at least one second application, wherein the at least one second application corresponds one-to-one to the at least one second virtual display, and the size of each second virtual display is the same as the size of a window for displaying the corresponding second application on the second electronic device; encoding video content of the first application displayed through a first virtual display and video content of the at least one second application displayed through the at least one second virtual display into video streaming media, and recording audio content of the first application and the at least one second application into audio streaming media, wherein the screen projection data of the first application comprises video content and audio content of the first application, and the screen projection data of the at least one second application comprises video content and audio content of the at least one second application; and transmitting the coded video streaming media and the recorded audio streaming media to the second electronic device.

Optionally, the screen projection method further includes: displaying a predetermined application of the first application and the at least one second application on a display screen of the first electronic device, wherein an aspect ratio of a window for displaying the predetermined application on the display screen of the first electronic device is the same as an aspect ratio of a window for displaying the predetermined application on the second electronic device; and/or displaying each of the first application and the at least one second application on a display screen of the first electronic device based on the layout information.

Optionally, the screen projection method further includes: and displaying another application of the first application and the at least one second application on a display screen of the first electronic device in response to a display switching operation of switching the predetermined application displayed on the display screen of the first electronic device to the another application, wherein an aspect ratio of a window for displaying the another application on the display screen of the first electronic device is the same as an aspect ratio of a window for displaying the another application on the second electronic device.

Optionally, the screen projection method further includes: in response to receiving a screen-drop request for at least one third application with the first application and the at least one second application being screened onto the second electronic device, determining second screen-drop configuration information for the first application, the at least one second application, and the at least one third application; and sending second screen projection configuration information, the first application, the at least one second application and the at least one third application to the second electronic device so that the first application, the at least one second application and the at least one third application are projected to the second electronic device, and performing on-screen display according to the second screen projection configuration information.

According to another aspect of the present disclosure, there is provided a screen projection method, which is applied to a first electronic device, and includes: receiving, from a second electronic device, screen-casting configuration information about a first application and at least one second application, and receiving, from the second electronic device, screen-casting data of the first application and screen-casting data of the at least one second application, in a case where the first application of the second electronic device is screen-cast to the first electronic device; and displaying the first application and the at least one second application on a display screen of the first electronic device in a same screen mode based on the screen projection configuration information.

Optionally, the screen projection method further includes: and sending multi-application screen projection information to the second electronic device, wherein the screen projection configuration information is determined based on the multi-application screen projection information.

Optionally, the step of sending the multi-application screen-drop information to the second electronic device includes: in response to receiving a multi-application screen negotiation request from a second electronic device regarding a first application of the second electronic device being screened onto the first electronic device, the multi-application screen information is sent to the second electronic device.

Optionally, receiving the multi-application screen-projection negotiation request from the second electronic device through a media negotiation channel, transmitting the multi-application screen-projection information to the second electronic device, and receiving the screen-projection configuration information from the second electronic device; and/or receiving the screen projection data of the first application and the screen projection data of the at least one second application from the second electronic equipment through a streaming media channel.

Optionally, the multi-application screen-projection negotiation request includes information for requesting simultaneous display of a plurality of applications, and the multi-application screen-projection information includes size information of a display screen of the first electronic device.

Optionally, the screen projection configuration information includes position information and size information for displaying each window of each of the first application and the at least one second application on a display screen of the first electronic device on a screen.

Optionally, the screen projection method further includes: adjusting a size of a first control for displaying the first application to be the same as a size of a window for displaying the first application included in the screen-casting configuration information, wherein the first control is created when the first application of the second electronic device is screen-cast to the first electronic device; at least one second control for displaying the at least one second application is created, wherein the at least one second control corresponds to the at least one second application one to one, and the size of each second control is the same as the size of a window included in the screen-casting configuration information for displaying the corresponding second application.

Optionally, the step of displaying the first application and the at least one second application on the display screen of the first electronic device on the same screen based on the screen projection configuration information includes: arranging a first control for displaying the first application and the at least one second control for displaying the at least one second application on a screen of the first electronic device based on the screen-casting configuration information; decoding video content of the first application in the form of video streaming media received from the second electronic device, and displaying the decoded video content of the first application through a first control; decoding video content of the at least one second application in the form of video streaming media received from the second electronic device, and displaying the decoded video content of the at least one second application through the at least one second control, wherein the screen projection data of the first application comprises video content and audio content of the first application, and the screen projection data of the at least one second application comprises video content and audio content of the at least one second application.

Optionally, the screen projection method further includes: receiving second screen-casting configuration information about the first application, the at least one second application and at least one third application from the second electronic device and screen-casting data of the first application, the at least one second application and the at least one third application from the second electronic device, in case the first application and the at least one second application are screen-cast to the first electronic device; and displaying the first application, the at least one second application and the at least one third application on a display screen of the first electronic device on-screen based on the second screen-casting configuration information.

According to another aspect of the present disclosure, there is provided a computer-readable storage medium storing a computer program which, when executed by a processor, implements the screen projection method as described above.

According to another aspect of the present disclosure, there is provided a screen projection apparatus implemented in a first electronic device, and including: a screen-projection configuration information determining unit configured to: determining, in response to receiving a screen-casting request for at least one second application, screen-casting configuration information of the first application and the at least one second application under the condition that the first application is screen-cast from the first electronic device to the second electronic device; a transmission unit configured to: and sending the screen projection configuration information, the screen projection data of the first application and the screen projection data of the at least one second application to the second electronic equipment so that the first application and the at least one second application are projected to the second electronic equipment, and carrying out the same-screen display according to the screen projection configuration information.

According to another aspect of the present disclosure, there is provided a screen projection apparatus implemented in a first electronic device, and including: a receiving unit configured to: receiving, from a second electronic device, screen-casting configuration information about a first application and at least one second application, and receiving, from the second electronic device, screen-casting data of the first application and screen-casting data of the at least one second application, in a case where the first application of the second electronic device is screen-cast to the first electronic device; a display unit configured to: and displaying the first application and the at least one second application on a display screen of the first electronic device in a same screen mode based on the screen projection configuration information.

According to another aspect of the present disclosure, there is provided an electronic device including: a processor; and a memory storing a computer program which, when executed by the processor, implements the method of touch screen as described above.

According to the screen projection method and the screen projection device, a plurality of applications can be projected from the first electronic equipment to the second electronic equipment so as to be displayed on the same screen, and therefore the application range of the screen projection technology is expanded. In addition, when a plurality of applications are projected from the first electronic equipment to the second electronic equipment for on-screen display, the display windows of the applications are adjustable, so that the user experience of the screen projection technology is improved.

Drawings

The foregoing and other objects and features of embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings which illustrate, by way of example, the embodiments in which:

FIG. 1 is a flowchart illustrating a first example of a screen casting method according to an embodiment of the present disclosure;

fig. 2 is a diagram showing a first example of a preview window;

fig. 3 is a diagram showing a second example of a preview window;

fig. 4 is a diagram showing a third example of a preview window;

fig. 5 is a diagram showing a fourth example of a preview window;

fig. 6 is a diagram showing a fifth example of a preview window;

fig. 7 is a diagram showing a sixth example of a preview window;

FIG. 8 is a flowchart illustrating a second example of a screen casting method according to an embodiment of the present disclosure;

fig. 9 is a block diagram illustrating a first example of a screen projection device according to an embodiment of the present disclosure;

fig. 10 is a block diagram showing a second example of a screen projection device according to an embodiment of the present disclosure;

fig. 11 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

Detailed Description

The following detailed description is provided to assist the reader in obtaining a thorough understanding of the methods, apparatus, and/or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatuses, and/or systems described herein will be apparent after an understanding of the disclosure of the present application. For example, the order of operations described herein is merely an example and is not limited to those set forth herein, but may be altered as will be apparent after an understanding of the disclosure of the present application, except for operations that must occur in a particular order. Furthermore, descriptions of features known in the art may be omitted for clarity and conciseness.

The features described herein may be embodied in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein have been provided to illustrate only some of the many possible ways to implement the methods, devices, and/or systems described herein, which will be apparent after an understanding of the present disclosure.

As used herein, the term "and/or" includes any one of the listed items associated as well as any combination of any two or more.

Although terms such as "first," "second," and "third" may be used herein to describe various elements, components, regions, layers or sections, these elements, components, regions, layers or sections should not be limited by these terms. Rather, these terms are only used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Thus, a first member, first component, first region, first layer, or first portion referred to in the examples described herein may also be referred to as a second member, second component, second region, second layer, or second portion without departing from the teachings of the examples.

In the description, when an element (such as a layer, region or substrate) is referred to as being "on" another element, "connected to" or "coupled to" the other element, it can be directly "on" the other element, be directly "connected to" or be "coupled to" the other element, or one or more other elements intervening elements may be present. In contrast, when an element is referred to as being "directly on" or "directly connected to" or "directly coupled to" another element, there may be no other element intervening elements present.

The terminology used herein is for the purpose of describing various examples only and is not intended to be limiting of the disclosure. Singular forms also are intended to include plural forms unless the context clearly indicates otherwise. The terms "comprises," "comprising," and "having" specify the presence of stated features, amounts, operations, components, elements, and/or combinations thereof, but do not preclude the presence or addition of one or more other features, amounts, operations, components, elements, and/or combinations thereof.

Unless defined otherwise, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs after understanding this disclosure. Unless explicitly so defined herein, terms (such as those defined in a general dictionary) should be construed to have meanings consistent with their meanings in the context of the relevant art and the present disclosure, and should not be interpreted idealized or overly formal.

In addition, in the description of the examples, when it is considered that detailed descriptions of well-known related structures or functions will cause a ambiguous explanation of the present disclosure, such detailed descriptions will be omitted.

Fig. 1 is a flowchart illustrating a first example of a screen projection method according to an embodiment of the present disclosure. The screen projection method can be applied to a first electronic device, and in a first example, the first electronic device is an electronic device which initiates a multi-application screen projection request.

Referring to fig. 1, in step S101, in a case where a first application is screen-cast from a first electronic device to a second electronic device (i.e., an electronic device that receives a multi-application screen-cast request and performs multi-application on-screen display), screen-cast configuration information of the first application and at least one second application may be determined when the screen-cast request for the at least one second application is received.

In particular, multi-application drop information may be received from a second electronic device. In this case, the screen projection configuration information of the first application and the at least one second application may be determined based on the multi-application screen projection information received from the second electronic device. Here, the multi-application screen projection information may include size information of a display screen of the second electronic device. Further, the multi-application screen negotiation request may be sent to the second electronic device upon receiving the screen request for the at least one second application, and the multi-application screen information in response may be received from the second electronic device. Here, the multi-application screen-cast negotiation request may include information for requesting the second electronic device to simultaneously display a plurality of applications. In other words, at this time, the multi-application screen-casting information may be response information to the multi-application screen-casting negotiation request.

Since the multi-application screen-casting information includes size information of the display screen of the second electronic device, when determining screen-casting configuration information of the first application and the at least one second application, layout information for displaying each of the first application and the at least one second application on the second electronic device may be generated as the screen-casting configuration information of the first application and the at least one second application based on the size information of the display screen of the second electronic device and the screen-casting sequence information of each of the first application and the at least one second application. Alternatively, the layout information may be displayed on the display screen of the first electronic device in the form of a preview. In this way, the modified layout information may be generated as the on-screen configuration information of the first application and the at least one second application in response to the modification operation for the layout information displayed in the form of the preview. Note that the layout information may include position information and size information of each window for displaying each application on the display screen of the second electronic device.

For example, the first electronic device may calculate, in a default split-screen layout display manner, a manner (i.e., layout information) most suitable for simultaneously displaying a plurality of applications on a display screen of the second electronic device according to a screen-casting sequence of each of the first application and the at least one second application. Meanwhile, a preview interface of layout information can be generated on the first electronic device, a user of the first electronic device can exchange the positions of display windows of all applications through dragging operation on the preview interface, and the proportion of the display windows is adjusted by dragging the dividing line of the display windows, so that the split screen display effect desired by the user is finally achieved. The layout information thus modified may be used as the screen configuration information of the first application and the at least one second application.

The following illustrates a modification operation of layout information. The first electronic device is assumed to be a mobile phone or a tablet device, and the second electronic device is assumed to be a mobile phone, a tablet device, a smart television, a projector, or the like. The first application is a Word document, and the second application is a video screen play App. When the screen of the video playing App is required to be projected to the second electronic device while the Word document is projected to the second electronic device, the first electronic device may send a multi-application screen projection negotiation request to the second electronic device. When receiving the multi-application screen-projection negotiation request, the second electronic device can send multi-application screen-projection information to the first electronic device as a response. The multi-application screen projection information may include size information of a display screen of the second electronic device, e.g., 1080 x 1920 pixels. The first electronic device generates a mode suitable for simultaneously displaying the first application and the second application on the display screen of the second electronic device based on the size information of the display screen of the second electronic device and the screen throwing sequence information of the first application and the second application.

When there are multiple split-screen display modes for simultaneously displaying the first application and the second application, the split-screen display modes can be used as layout information and displayed on the first electronic device in a preview mode. Here, the ratio of the preview image (hereinafter referred to as "preview window") coincides with the ratio of the size information of the display screen of the second electronic device in order to ensure that the preview effect coincides with the actual display effect projected onto the second electronic device. Fig. 2 is a diagram showing a first example of a preview window. In fig. 2, although the scale of the display screen of the first electronic device is different from 1080:1920, but the scale of the preview window remains consistent with the scale of the screen of the second electronic device, i.e., 1080:1920. in the preview window, the ratio of the display windows of the first application and the second application is 1080:960 (i.e., the display windows of the first application and the second application are arranged from top to bottom and each account for 50% of the preview window). If the user is not satisfied with the layout information displayed in the preview window, the user may perform a modification operation, e.g., a switching operation, on the layout information displayed in the preview window. Fig. 3 is a diagram showing a second example of the preview window. In fig. 3, in response to a switching operation performed by a user, a split screen display manner of a first application and a second application is switched, where the proportions of display windows of the first application and the second application are 540:1920 (i.e., the display windows of the first application and the second application are arranged from left to right and each account for 50% of the preview window).

Alternatively, in addition to switching the up-down arrangement to the left-right arrangement, the user may exchange display windows of the first application and the second application through a drag operation to modify the preview window. Fig. 4 is a diagram showing a third example of a preview window, and fig. 5 is a diagram showing a fourth example of a preview window. As shown in fig. 4, the display windows of the first application and the second application may be swapped by dragging the display window of the first application to the display window of the second application. As shown in fig. 5, the display windows of the first application and the second application arranged from top to bottom are changed to be arranged from bottom to top. In addition, the user can drag the boundary between the display window of the first application and the display window of the second application to adjust the sizes of the display windows of the first application and the second application so as to modify the preview window. Fig. 6 is a diagram showing a fifth example of a preview window, and fig. 7 is a diagram showing a sixth example of a preview window. As shown in fig. 6, the sizes of the display windows of the first application and the second application may be changed by dragging the boundary between the display windows of the first application and the second application. As shown in fig. 7, the display window of the first application becomes smaller, and the display window of the second application becomes larger. Here, it should be noted that the ratio of the preview window is always maintained at 1080 regardless of how the display windows of the first application and the second application are resized: 1920 are unchanged.

Taking fig. 2 as an example, the screen configuration information of the first application and the second application may have the following form according to an embodiment of the present disclosure. First application: (0,0) 1080:960; second application: (0, 960) 1080:960. here, (0, 0) and (0, 960) represent start point coordinates of display windows of the first application and the second application, 1080:960 represents the ratio of the display windows of the first application to the second application.

Referring back to fig. 1, in step S102, the screen projection configuration information, the screen projection data of the first application, and the screen projection data of the at least one second application may be sent to the second electronic device, so that the first application and the at least one second application are projected onto the second electronic device, and the same screen display is performed according to the screen projection configuration information.

According to the embodiment of the disclosure, when the first application is projected from the first electronic device to the second electronic device, the first electronic device and the second electronic device are connected, and the connection mode includes but is not limited to WIFI direct connection. In addition, the connection may include a media negotiation path and a streaming media path. In this case, the multi-application screen-projection negotiation request may be transmitted to the second electronic device through the media negotiation channel, the multi-application screen-projection information may be received from the second electronic device, and the screen-projection configuration information may be transmitted to the second electronic device; and/or, the screen projection data of the first application and the screen projection data of the at least one second application can be sent to the second electronic device through the streaming media channel. It should be noted that the screen-cast data for each application is transmitted over a different streaming channel. For example, the screen-drop data of the first application is transmitted through the first streaming media channel. When the at least one second application comprises two applications, the two applications are transmitted via the second and third stream media channels, respectively.

In particular, a first virtual display for on-screen display of a first application is created when the first application is on-screen from a first electronic device to a second electronic device. In this way, when the screen projection data of the first application and the screen projection data of the at least one second application are transmitted to the second electronic device, the size of the first virtual display for screen projection display of the first application may be adjusted to be the same as the size of the window for displaying the first application on the second electronic device. At least one second virtual display for on-screen display of at least one second application may then be created, the at least one second application being in one-to-one correspondence with the created at least one second virtual display, and the size of each second virtual display being the same as the size of a window for displaying the corresponding second application on the second electronic device. Next, video content of a first application displayed through the first virtual display and video content of at least one second application displayed through the at least one second virtual display may be encoded as video streaming media, and audio content of the first application and the at least one second application may be recorded as audio streaming media, that is, the screen projection data of the first application includes video content and audio content of the first application, and the screen projection data of the at least one second application includes video content and audio content of the at least one second application. Finally, the encoded video stream and the recorded audio stream may be transmitted to a second electronic device.

For example, as shown in fig. 2, the sizes of the first virtual display and the second virtual display may be 1080×960, and as shown in fig. 3, the sizes of the first virtual display and the second virtual display may be 540×1920.

Alternatively, the first application and the predetermined application of the at least one second application may be displayed on the display screen of the first electronic device while the screen projection data of the first application and the screen projection data of the at least one second application are transmitted to the second electronic device for on-screen display. Here, the aspect ratio of the window for displaying the predetermined application on the display screen of the first electronic device is the same as the aspect ratio of the window for displaying the predetermined application on the second electronic device. For example, if a first application is displayed on a display screen of a first electronic device, and the scale of the display window of the first application on a second electronic device is 1080:1440, the first application uses 1080:1440 is displayed on the first electronic device. In this case, in response to a display switching operation of switching a predetermined application displayed on the display screen of the first electronic device to another application of the first application and the at least one second application, the other application may be displayed on the display screen of the first electronic device. Similarly, the aspect ratio of the window for displaying another application on the display screen of the first electronic device is the same as the aspect ratio of the window for displaying another application on the second electronic device. For example, if a first application is switched to a second application while the first application is displayed on the display screen of the first electronic device, the second application has a display window on the second electronic device of a ratio 1080:480, the second application uses 1080: the scale of 480 is displayed on the first electronic device.

In another aspect, each of the first application and the at least one second application may be displayed on a display screen of the first electronic device based on the layout information. For example, similar to fig. 2, the first application and the second application may be displayed simultaneously on a display screen of the first electronic device. In this way, when performing multi-application screening, the user may learn on the first electronic device that each application has been screened to the second electronic device.

Alternatively, when a screen-casting request for at least one third application is received in a case where the first application and the at least one second application are screen-cast to the second electronic device, second screen-casting configuration information of the first application, the at least one second application, and the at least one third application may be determined. And then, the second screen projection configuration information, the first application, at least one second application and at least one third application can be transmitted to the second electronic device, so that the first application, the at least one second application and the at least one third application are projected to the second electronic device, and in-screen display is performed according to the second screen projection configuration information. Here, the second drop configuration information may be sent to the second electronic device through a media negotiation channel and the corresponding third application is transmitted through a media channel different from the media channel transmitting the first application and the at least one media channel transmitting the at least one second application.

According to the screen projection method disclosed by the embodiment of the invention, a plurality of applications can be projected from the first electronic equipment to the second electronic equipment so as to perform the same-screen display, so that the application range of the screen projection technology is expanded. In addition, when a plurality of applications are projected from the first electronic equipment to the second electronic equipment for on-screen display, the display windows of the applications can be adjusted on the first electronic equipment, so that the user experience of the projection technology is improved.

Fig. 8 is a flowchart illustrating a second example of a screen projection method according to an embodiment of the present disclosure. The screen projection method can be applied to a first electronic device, and in a second example, the first electronic device is an electronic device for carrying out on-screen display on a plurality of applications of screen projection.

Referring to fig. 8, in step S801, in a case where a first application of a second electronic device is screen-cast to the first electronic device, screen-cast configuration information about the first application and at least one second application may be received from the second electronic device, and screen-cast data of the first application and screen-cast data of the at least one second application may be received from the second electronic device.

In particular, the multi-application drop information may be transmitted to the second electronic device. In this case, the screen-casting configuration information about the first application and the at least one second application may be determined based on the multi-application screen-casting information. Here, the multi-application screen projection information may include size information of a display screen of the first electronic device, and the screen projection configuration information may include position information and size information for displaying respective windows of respective ones of the first application and the at least one second application on the display screen of the first electronic device on-screen. Further, the multi-application screen-casting information may be transmitted to the second electronic device when a multi-application screen-casting negotiation request regarding the first application and at least one second application is received from the second electronic device in a case that the first application of the second electronic device is screen-cast to the first electronic device. Here, the multi-application screen-cast negotiation request may include information for requesting simultaneous display of a plurality of applications. In other words, at this time, the multi-application screen-casting information may be response information to the multi-application screen-casting negotiation request.

Next, in step S802, the first application and the at least one second application may be displayed on the display screen of the first electronic device on-screen based on the screen-casting configuration information.

As described above, when the first application of the second electronic device is projected onto the first electronic device, the first electronic device and the second electronic device will establish a connection, where the connection manner includes, but is not limited to, WIFI direct connection. In addition, the connection may include a media negotiation path and a streaming media path. In this case, the multi-application screen-casting negotiation request may be received from the second electronic device through the media negotiation channel, the multi-application screen-casting information may be transmitted to the second electronic device, and the screen-casting configuration information may be received from the second electronic device; and/or, the screen projection data of the first application and the screen projection data of at least one second application can be received from the second electronic device through the streaming media channel. It should be noted that the screen-cast data for each application is transmitted over a different streaming channel. For example, the screen-drop data of the first application is transmitted through the first streaming media channel. When the at least one second application comprises two applications, the two applications are transmitted via the second and third stream media channels, respectively.

In order to display the first application and the at least one second application on the display screen of the first electronic device on-screen, a first control for displaying the first application may be resized to be the same size as a window for displaying the first application included in the screen-casting configuration information. Here, the first control is created when a first application of the second electronic device is projected onto the first electronic device. At least one second control for displaying the at least one second application may then be created. The at least one second control corresponds to the at least one second application one-to-one, and the size of each second control is the same as the size of a window included in the drop configuration information for displaying the corresponding second application.

More specifically, a first control for displaying a first application and at least one second control for displaying at least one second application may first be arranged on a screen of a first electronic device based on the screen-casting configuration information. Then, the video content of the first application in the form of video streaming media received from the second electronic device may be decoded and the decoded video content of the first application may be displayed through the first control, and the video content of the at least one second application in the form of video streaming media received from the second electronic device may be decoded and the decoded video content of the at least one second application may be displayed through the at least one second control. Here, the video content of the first application may be included in the screen-casting data of the first application, and the video content of the at least one second application may be included in the screen-casting data of the at least one second application. In another aspect, the first application's drop data may also include audio content of the first application, and the at least one second application's drop data may also include audio content of the at least one second application. In this case, the audio content of each application may be decoded and the decoded audio content may be played based on the screen-in order of each application. Alternatively, the audio content of the corresponding application may be decoded based on the user settings and the decoded audio content may be played.

Alternatively, in case that the first application and the at least one second application are projected to the first electronic device, second projection configuration information about the first application, the at least one second application, and the at least one third application may be received from the second electronic device, and projection data of the first application, the at least one second application, and the at least one third application may be received from the second electronic device, and then the first application, the at least one second application, and the at least one third application may be displayed on-screen on a display screen of the first electronic device based on the second projection configuration information. Here, the second drop configuration information may be received from the second electronic device through a media negotiation channel, and the corresponding third application may be received from the second electronic device through a media channel different from the media channel through which the first application is transmitted and the at least one media channel through which the at least one second application is transmitted.

According to the screen projection method disclosed by the embodiment of the invention, a plurality of applications can be projected from the second electronic equipment to the first electronic equipment so as to perform the same-screen display, so that the application range of the screen projection technology is expanded. In addition, when a plurality of applications are projected from the second electronic equipment to the first electronic equipment for on-screen display, the display windows of the applications can be adjusted on the second electronic equipment, so that the user experience of the screen projection technology is improved.

Fig. 9 is a block diagram illustrating a first example of a screen projection device according to an embodiment of the present disclosure. The screen projection device may be implemented on a first electronic device, which in a first example is the electronic device that initiates the multi-application screen projection request.

Referring to fig. 9, the screen projection device 900 may include a screen projection configuration information determining unit 910 and a transmitting unit 920. The screen-projection configuration information determining unit 910 may determine screen-projection configuration information of the first application and the at least one second application in response to receiving a screen-projection request for the at least one second application in a case where the first application is projected from the first electronic device to the second electronic device. The sending unit 920 may send the screen projection configuration information, the screen projection data of the first application, and the screen projection data of the at least one second application to the second electronic device, so that the first application and the at least one second application are projected onto the second electronic device, and perform on-screen display according to the screen projection configuration information.

Optionally, the screen projection device 900 may further include a receiving unit (not shown). The receiving unit may receive multi-application screen projection information from the second electronic device, and the screen projection configuration information determining unit 910 may determine screen projection configuration information of the first application and at least one second application based on the multi-application screen projection information received from the second electronic device. Further, the transmitting unit 920 may transmit a multi-application screen-projection negotiation request to the second electronic device in response to the screen-projection request, and the receiving unit may receive multi-application screen-projection information from the second electronic device.

As described above, the multi-application screen negotiation request may be transmitted to the second electronic device through the media negotiation channel, the multi-application screen information may be received from the second electronic device, and the screen configuration information may be transmitted to the second electronic device, and/or the screen data of the first application and the screen data of the at least one second application may be transmitted to the second electronic device through the streaming media channel. The multi-application screen-cast negotiation request includes information for requesting the second electronic device to simultaneously display a plurality of applications, and the multi-application screen-cast information includes size information of a display screen of the second electronic device.

The screen-projection configuration information determining unit 910 may generate layout information of each of the first application and the at least one second application displayed on the second electronic device as the screen-projection configuration information of the first application and the at least one second application based on the size information of the display screen of the second electronic device and the screen-projection sequence information of each of the first application and the at least one second application. More specifically, the screen-drop configuration information determination unit 910 may display the layout information in the form of a preview on the display screen of the first electronic device, and may generate the modified layout information as the screen-drop configuration information of the first application and the at least one second application in response to a modification operation for the layout information displayed in the form of the preview. Here, the layout information includes position information and size information of each window for displaying each application on the display screen of the second electronic device.

The sending unit 920 may adjust a size of a first virtual display for the on-screen display of the first application to be the same as a size of a window for displaying the first application on the second electronic device, wherein the first virtual display is created when the first application is on-screen to the second electronic device. Subsequently, the transmitting unit 920 may create at least one second virtual display for the at least one second application to perform a screen-cast display, wherein the at least one second application corresponds to the at least one second virtual display one by one, and a size of each second virtual display is the same as a size of a window for displaying the corresponding second application on the second electronic device. Next, the transmitting unit 920 may encode video content of a first application displayed through the first virtual display and video content of at least one second application displayed through the at least one second virtual display into video streaming media, and record audio content of the first application and the at least one second application into audio streaming media, wherein the screen projection data of the first application includes video content and audio content of the first application, and the screen projection data of the at least one second application includes video content and audio content of the at least one second application. Finally, the transmitting unit 920 may transmit the encoded video streaming media and the recorded audio streaming media to the second electronic device.

Optionally, the screen projection device 900 may further include a first control unit (not shown). The first control unit may display a predetermined application of the first application and the at least one second application on a display screen of the first electronic device, wherein an aspect ratio of a window for displaying the predetermined application on the display screen of the first electronic device is the same as an aspect ratio of a window for displaying the predetermined application on the second electronic device, and/or the first control unit may display each of the first application and the at least one second application on the display screen of the first electronic device based on the layout information.

Optionally, the screen projection device 900 may further include a second control unit (not shown). The second control unit may display another application on the display screen of the first electronic device in response to a display switching operation of switching a predetermined application displayed on the display screen of the first electronic device to another application of the first application and the at least one second application, wherein an aspect ratio of a window for displaying the another application on the display screen of the first electronic device is the same as an aspect ratio of a window for displaying the another application on the second electronic device.

According to an embodiment of the present disclosure, the screen-projection configuration information determining unit 910 may determine the second screen-projection configuration information of the first application, the at least one second application, and the at least one third application in response to receiving a screen-projection request for the at least one third application in a case where the first application and the at least one second application are screen-projected to the second electronic device. The sending unit 920 may send the second screen projection configuration information, the first application, the at least one second application, and the at least one third application to the second electronic device, so that the first application, the at least one second application, and the at least one third application are projected to the second electronic device, and perform on-screen display according to the second screen projection configuration information.

Fig. 10 is a block diagram illustrating a second example of a screen projection device according to an embodiment of the present disclosure. The screen projection device can be implemented on a first electronic device, and in a second example, the first electronic device is an electronic device for performing on-screen display on a plurality of applications of a screen projection.

Referring to fig. 10, the screen projection device 1000 may include a receiving unit 1010 and a display unit 1020. The receiving unit 1010 may receive, in a case where a first application of a second electronic device is screen-cast to the first electronic device, screen-cast configuration information about the first application and at least one second application from the second electronic device, and screen-cast data of the first application and screen-cast data of the at least one second application from the second electronic device. The display unit 1020 may display the first application and the at least one second application on the display screen of the first electronic device on-screen based on the screen-casting configuration information.

Optionally, the screen projection device 1000 may further include a transmitting unit (not shown). The transmitting unit may transmit the multi-application screen projection information to the second electronic device, and the screen projection configuration information may be determined based on the multi-application screen projection information. The transmitting unit may transmit the multi-application screen-projection information to the second electronic device in response to receiving a multi-application screen-projection negotiation request regarding the first application and at least one second application from the second electronic device in a case where the first application of the second electronic device is screen-projected to the first electronic device.

As described above, the multi-application screen-cast negotiation request may be received from the second electronic device through the media negotiation channel, the multi-application screen-cast information may be transmitted to the second electronic device, and the screen-cast configuration information may be received from the second electronic device, and/or the screen-cast data of the first application and the screen-cast data of the at least one second application may be received from the second electronic device through the streaming media channel. The multi-application screen-cast negotiation request includes information for requesting simultaneous display of a plurality of applications, the multi-application screen-cast information including size information of a display screen of the first electronic device. The screen projection configuration information includes position information and size information for displaying respective windows of respective applications of the first application and the at least one second application on a display screen of the first electronic device on a screen.

Optionally, the screen projection device 1000 may further comprise a first control unit (not shown). The first control unit may adjust a size of a first control for displaying the first application to be the same as a size of a window for displaying the first application included in the screen-casting configuration information, wherein the first control is created when the first application of the second electronic device is screen-cast to the first electronic device. Furthermore, the first control unit may create at least one second control for displaying at least one second application, wherein the at least one second control corresponds one-to-one to the at least one second application, and a size of each second control is the same as a size of a window included in the screen projection configuration information for displaying the corresponding second application.

The display unit 1020 may arrange a first control for displaying a first application and at least one second control for displaying at least one second application on a screen of the first electronic device based on the screen-casting configuration information; decoding video content of a first application in the form of video streaming media received from a second electronic device, and displaying the decoded video content of the first application through a first control; the video content of at least one second application in the form of video streaming media received from the second electronic device is decoded and the decoded video content of the at least one second application is displayed through the at least one second control. Here, the screen projection data of the first application includes video content and audio content of the first application, and the screen projection data of the at least one second application includes video content and audio content of the at least one second application.

According to an embodiment of the present disclosure, the receiving unit 1010 may receive second screen-casting configuration information about the first application, the at least one second application, and the at least one third application from the second electronic device and screen-casting data of the first application, the at least one second application, and the at least one third application from the second electronic device in a case where the first application and the at least one second application are screen-cast to the first electronic device. The display unit 1020 may display the first application, the at least one second application, and the at least one third application on the display screen of the first electronic device on-screen based on the second projection configuration information.

Fig. 11 is a block diagram illustrating an electronic device according to an embodiment of the present disclosure. The electronic device can be the electronic device which initiates the multi-application screen-throwing request, and can also be the electronic device which performs the same screen display on a plurality of applications which throw the screen.

Referring to fig. 11, a mobile terminal 1100 may include a memory 1101 and a processor 1102. The memory 1101 may store computer executable instructions that, when executed by the processor 1102, may cause the processor 1102 to perform a screen-casting method according to an embodiment of the present disclosure.

The processor 1102 may include a Central Processing Unit (CPU), a Graphics Processor (GPU), a programmable logic device, a special purpose processor system, a microcontroller, or a microprocessor. By way of example, and not limitation, processors may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, and the like. The processor 1102 may execute instructions or code stored in the memory 1101, wherein the memory 1101 may also store data. The instructions and data may also be transmitted and received over a network via a network interface device, which may employ any known transmission protocol.

The memory 1101 may be integrated with the processor 1102, for example, RAM or flash memory disposed within an integrated circuit microprocessor or the like. In addition, the memory 1101 may include a stand-alone device, such as an external disk drive, a storage array, or other storage device usable by any database system. The memory 1101 and the processor 1102 may be operatively coupled or may communicate with each other, for example, through an I/O port, network connection, etc., such that the processor 1102 is able to read files stored in the memory 1101.

Alternatively, the mobile terminal 1100 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the mobile terminal 1100 may be connected to each other via buses and/or networks.

The screen projection method according to the embodiments of the present disclosure may be written as a computer program and stored on a computer-readable storage medium. The screen projection method described above may be implemented when the computer program is executed by a processor. Examples of the computer readable storage medium include: read-only memory (ROM), random-access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, nonvolatile memory, CD-ROM, CD-R, CD + R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD + R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, blu-ray or optical disk storage, hard Disk Drives (HDD), solid State Disks (SSD), card memory (such as multimedia cards, secure Digital (SD) cards or ultra-fast digital (XD) cards), magnetic tape, floppy disks, magneto-optical data storage, hard disks, solid state disks, and any other means configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and to provide the computer programs and any associated data, data files and data structures to a processor or computer to enable the processor or computer to execute the programs. In one example, the computer program and any associated data, data files, and data structures are distributed across networked computer systems such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner by one or more processors or computers.

Although a few embodiments of the present disclosure have been shown and described, it would be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the disclosure, the scope of which is defined in the claims and their equivalents.

Claims

1. A screen projection method, wherein the screen projection method is applied to a first electronic device, and the screen projection method comprises:

determining, in response to receiving a screen-casting request for at least one second application, screen-casting configuration information of the first application and the at least one second application under the condition that the first application is screen-cast from the first electronic device to the second electronic device;

and sending the screen projection configuration information, the screen projection data of the first application and the screen projection data of the at least one second application to the second electronic equipment so that the first application and the at least one second application are projected to the second electronic equipment, and carrying out the same-screen display according to the screen projection configuration information.

2. The method of screening according to claim 1, further comprising:

receiving multi-application screen-cast information from the second electronic device,

wherein the step of determining the screen-drop configuration information of the first application and the at least one second application comprises: based on the multi-application screen projection information received from the second electronic device, screen projection configuration information of the first application and the at least one second application is determined.

3. The method of screening according to claim 2, wherein the step of receiving multi-application screening information from the second electronic device comprises:

and in response to receiving the screen projection request, sending a multi-application screen projection negotiation request to the second electronic device, and receiving the multi-application screen projection information from the second electronic device.

4. The screen-casting method of claim 3, wherein the multi-application screen-casting negotiation request is sent to the second electronic device through a media negotiation channel, the multi-application screen-casting information is received from the second electronic device, and the screen-casting configuration information is sent to the second electronic device; and/or

And sending the screen projection data of the first application and the screen projection data of the at least one second application to the second electronic equipment through a streaming media channel.

5. The method of claim 3, wherein the multi-application screen negotiation request includes information for requesting the second electronic device to simultaneously display a plurality of applications, the multi-application screen negotiation information including size information of a display screen of the second electronic device.

6. The method of screening according to claim 5, wherein determining the screening configuration information of the first application and the at least one second application comprises:

generating layout information for displaying each application of the first application and the at least one second application on the second electronic device based on the size information of the display screen of the second electronic device and the screen throwing sequence information of each application of the first application and the at least one second application, wherein the layout information is used as screen throwing configuration information of the first application and the at least one second application.

7. The method of screening according to claim 6, wherein determining the screening configuration information of the first application and the at least one second application further comprises:

displaying the layout information on a display screen of the first electronic device in the form of a preview;

And generating modified layout information as screen configuration information of the first application and the at least one second application in response to a modification operation for the layout information displayed in the form of a preview.

8. The screen projection method of claim 6 wherein the layout information includes position information and size information for displaying respective windows of respective applications on a display screen of the second electronic device.

9. The method of screening according to claim 8, wherein the step of transmitting the screening data of the first application and the screening data of the at least one second application to the second electronic device comprises:

adjusting a size of a first virtual display for on-screen display of the first application to be the same as a size of a window for displaying the first application on the second electronic device, wherein the first virtual display is created when the first application is on-screen to the second electronic device;

creating at least one second virtual display for on-screen display of the at least one second application, wherein the at least one second application corresponds one-to-one to the at least one second virtual display, and the size of each second virtual display is the same as the size of a window for displaying the corresponding second application on the second electronic device;

Encoding video content of the first application displayed through a first virtual display and video content of the at least one second application displayed through the at least one second virtual display into video streaming media, and recording audio content of the first application and the at least one second application into audio streaming media, wherein the screen projection data of the first application comprises video content and audio content of the first application, and the screen projection data of the at least one second application comprises video content and audio content of the at least one second application;

and transmitting the coded video streaming media and the recorded audio streaming media to the second electronic device.

10. The method of screening according to claim 8, further comprising:

displaying a predetermined application of the first application and the at least one second application on a display screen of the first electronic device, wherein an aspect ratio of a window for displaying the predetermined application on the display screen of the first electronic device is the same as an aspect ratio of a window for displaying the predetermined application on the second electronic device; and/or

Based on the layout information, each of the first application and the at least one second application is displayed on a display screen of the first electronic device.

11. The method of screening according to claim 10, further comprising:

and displaying another application of the first application and the at least one second application on a display screen of the first electronic device in response to a display switching operation of switching the predetermined application displayed on the display screen of the first electronic device to the another application, wherein an aspect ratio of a window for displaying the another application on the display screen of the first electronic device is the same as an aspect ratio of a window for displaying the another application on the second electronic device.

12. The method of screening according to claim 3, further comprising:

in response to receiving a screen-drop request for at least one third application with the first application and the at least one second application being screened onto the second electronic device, determining second screen-drop configuration information for the first application, the at least one second application, and the at least one third application;

And sending second screen projection configuration information, the first application, the at least one second application and the at least one third application to the second electronic device so that the first application, the at least one second application and the at least one third application are projected to the second electronic device, and performing on-screen display according to the second screen projection configuration information.

13. A screen projection method, wherein the screen projection method is applied to a first electronic device, and the screen projection method comprises:

receiving, from a second electronic device, screen-casting configuration information about a first application and at least one second application, and receiving, from the second electronic device, screen-casting data of the first application and screen-casting data of the at least one second application, in a case where the first application of the second electronic device is screen-cast to the first electronic device;

and displaying the first application and the at least one second application on a display screen of the first electronic device in a same screen mode based on the screen projection configuration information.

14. The method of screening according to claim 13, further comprising:

Transmitting multi-application screen-casting information to the second electronic device,

wherein the screen projection configuration information is determined based on the multi-application screen projection information.

15. The method of screening according to claim 14, wherein the step of transmitting multi-application screening information to the second electronic device comprises:

in response to receiving a multi-application screen negotiation request from a second electronic device regarding a first application of the second electronic device being screened onto the first electronic device, the multi-application screen information is sent to the second electronic device.

16. The method of screening according to claim 15, wherein the multi-application screening negotiation request is received from the second electronic device through a media negotiation channel, the multi-application screening information is sent to the second electronic device, and the screening configuration information is received from the second electronic device; and/or

And receiving the screen projection data of the first application and the screen projection data of the at least one second application from the second electronic equipment through a streaming media channel.

17. The method of claim 15, wherein the multi-application screen negotiation request includes information requesting simultaneous display of a plurality of applications, the multi-application screen negotiation information including size information of a display screen of the first electronic device.

18. The method of screening according to claim 15, wherein the screening configuration information includes position information and size information for displaying respective windows of respective ones of the first application and the at least one second application on a display screen of the first electronic device on-screen.

19. The method of screening according to claim 18, further comprising:

adjusting a size of a first control for displaying the first application to be the same as a size of a window for displaying the first application included in the screen-casting configuration information, wherein the first control is created when the first application of the second electronic device is screen-cast to the first electronic device;

at least one second control for displaying the at least one second application is created, wherein the at least one second control corresponds to the at least one second application one to one, and the size of each second control is the same as the size of a window included in the screen-casting configuration information for displaying the corresponding second application.

20. The method of on-screen display of claim 19, wherein the step of on-screen displaying the first application and the at least one second application on the display screen of the first electronic device based on the on-screen configuration information comprises:

Arranging a first control for displaying the first application and the at least one second control for displaying the at least one second application on a screen of the first electronic device based on the screen-casting configuration information;

decoding video content of the first application in the form of video streaming media received from the second electronic device, and displaying the decoded video content of the first application through a first control;

decoding video content of the at least one second application in the form of a video stream received from the second electronic device, and displaying the decoded video content of the at least one second application through the at least one second control,

wherein the first application's drop data includes video content and audio content of the first application, and the at least one second application's drop data includes video content and audio content of the at least one second application.

21. The method of screening according to claim 13, further comprising:

receiving second screen-casting configuration information about the first application, the at least one second application and at least one third application from the second electronic device and screen-casting data of the first application, the at least one second application and the at least one third application from the second electronic device, in case the first application and the at least one second application are screen-cast to the first electronic device; and

And displaying the first application, the at least one second application and the at least one third application on a display screen of the first electronic device in a same screen based on the second screen configuration information.

22. A computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the screen projection method according to any one of claims 1-21.

23. A screen projection device, wherein the screen projection device is implemented in a first electronic device, and the screen projection device comprises:

a screen-projection configuration information determining unit configured to: determining, in response to receiving a screen-casting request for at least one second application, screen-casting configuration information of the first application and the at least one second application under the condition that the first application is screen-cast from the first electronic device to the second electronic device;

a transmission unit configured to: and sending the screen projection configuration information, the screen projection data of the first application and the screen projection data of the at least one second application to the second electronic equipment so that the first application and the at least one second application are projected to the second electronic equipment, and carrying out the same-screen display according to the screen projection configuration information.

24. A screen projection device, wherein the screen projection device is implemented in a first electronic device, and the screen projection device comprises:

a receiving unit configured to: receiving, from a second electronic device, screen-casting configuration information about a first application and at least one second application, and receiving, from the second electronic device, screen-casting data of the first application and screen-casting data of the at least one second application, in a case where the first application of the second electronic device is screen-cast to the first electronic device;

a display unit configured to: and displaying the first application and the at least one second application on a display screen of the first electronic device in a same screen mode based on the screen projection configuration information.

25. An electronic device, the electronic device comprising:

a processor; and

memory storing a computer program which, when executed by a processor, implements the method of screening according to any one of claims 1-21.