CN117837150A

CN117837150A - Display device, communication terminal and screen-throwing picture dynamic display method

Info

Publication number: CN117837150A
Application number: CN202280053896.2A
Authority: CN
Inventors: 宋子全; 马晓燕; 刘美玉; 庞秀娟; 张娜
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Hisense Visual Technology Co Ltd
Priority date: 2021-08-02
Filing date: 2022-06-29
Publication date: 2024-04-05
Also published as: WO2023011058A1

Abstract

The application provides a display device (200), a communication terminal (300) and a screen-throwing picture dynamic display method, wherein when a user triggers the change of the number of the communication terminals (300) connected with the display device (200), the method can plan the layout mode of the screen-throwing picture by detecting the connection number of the communication terminals (300). And then sending a notification message containing the display parameters of the screen throwing picture to the communication terminal so that the communication terminal (300) can generate screen throwing data conforming to the layout mode according to the display parameters and display the screen throwing data in the display equipment (200).

Description

Display device, communication terminal and screen-throwing picture dynamic display method

Cross Reference to Related Applications

The application requires that the application number be 202110881496.8 and filed on the 2021 and 08 and 02 days; filing at 2021, 08 and 02, application number 202110881335.9; the application number 202111265530.5, filed on month 10 and 28 of 2021, is incorporated by reference in its entirety.

Technical Field

The present disclosure relates to the field of display devices, and in particular, to a display device, a communication terminal, and a method for dynamically displaying a screen.

Background

The display device refers to a terminal device capable of outputting a specific display screen, and may be a terminal device such as a smart television, a communication terminal, a smart advertisement screen, and a projector. Taking intelligent electricity as an example, the intelligent television is based on the Internet application technology, has an open operating system and a chip, has an open application platform, can realize a bidirectional man-machine interaction function, and is a television product integrating multiple functions of video, entertainment, data and the like, and the intelligent television is used for meeting the diversified and personalized requirements of users.

The display device can also establish a screen-throwing connection relation with the communication terminal in a communication mode such as a wireless local area network, namely, the display device can receive screen-throwing data shared by the communication terminal and display the received screen-throwing data in real time, namely, display a current picture of the communication terminal or a media picture played on the communication terminal.

Disclosure of Invention

An embodiment of the present application provides a display device, including: a display, a communicator, and a controller. Wherein the display is configured to display a screen-casting interface; the communicator is configured to establish a screen-throwing connection with a communication terminal; the controller is configured to perform the following program steps: detecting the number of connections of the communication terminal; planning a layout mode of the screen throwing picture according to the connection quantity so as to obtain display parameters of the screen throwing picture; sending a notification message to a communication terminal corresponding to the screen projection picture, wherein the notification message comprises the display parameters, and the notification message is used for enabling the communication terminal to generate screen projection data according to the display parameters; and receiving the screen projection data fed back by the communication terminal, and displaying the screen projection data according to the layout mode.

An embodiment of the present application provides a display device, including:

the display is used for displaying a screen-throwing interface; the communicator is used for being connected with the terminal equipment in a screen throwing way; a controller configured to perform: receiving screen-throwing requests of at least M terminal devices, respectively receiving media configuration information from the M terminal devices, and matching layout among M screen-throwing windows adapted to a screen; wherein M is more than or equal to 1, and the media configuration information comprises the aspect ratio of video of terminal equipment screen throwing; and controlling a display to display M screen-throwing windows in the screen-throwing interface according to the layout, and displaying screen-throwing data sent by corresponding terminal equipment in each screen-throwing window.

An embodiment of the present application provides a display device, including: a display configured to display a user interface; a controller operable with a playback application and at least one drop protocol application, each of the at least one drop protocol application being connectable to the playback application and configured to: when a first screen-throwing request sent by a first mobile terminal is received, a screen-throwing connection between the first mobile terminal and the first mobile terminal is established through a first screen-throwing protocol application corresponding to the first screen-throwing request, and first screen-throwing content transmitted by the first mobile terminal through the screen-throwing connection is sent to a playing application; and creating a first playing window on the user interface through the playing application, wherein the first playing window is used for playing the first content to be projected.

Drawings

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control device in an embodiment of the present application;

fig. 2 is a hardware configuration block diagram of a display device in an embodiment of the present application;

FIG. 3 is a schematic diagram of a software configuration of a display device according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an icon control interface display of a display device application in an embodiment of the present application;

fig. 5 is a schematic structural diagram of a communication terminal according to some embodiments;

FIG. 6 is a software architecture diagram of a communication terminal according to some embodiments;

FIG. 7 is a user interface schematic diagram of a communication terminal according to some embodiments;

FIG. 8 is a schematic view of a screen effect according to some embodiments;

FIG. 9 is a schematic diagram of a screen operation flow according to some embodiments;

fig. 10 is a schematic diagram of a multi-communication terminal screen-drop connection according to some embodiments;

FIG. 11 is an effect diagram of dividing two display regions according to some embodiments;

FIG. 12 is a flowchart of a method for dynamic display of a projection screen according to some embodiments;

FIG. 13 is an effect diagram of dividing three display areas according to some embodiments;

FIG. 14 is a flow diagram of detecting a number of connections in response to a connection instruction according to some embodiments;

FIG. 15 is a flow diagram of detecting a number of connections in response to an exit instruction according to some embodiments;

FIG. 16 is a flow diagram of matching layout templates according to some embodiments;

FIG. 17 is a flow diagram of sending a hint message according to some embodiments;

FIG. 18 is a schematic diagram of a communication terminal generating screen projection data flow according to some embodiments;

FIG. 19 is a flow chart of a multi-way projection display method according to some embodiments;

FIG. 20 (a) is a layout diagram in a single vertical screen scenario with a number of screen shots of 1 and a video stream according to some embodiments;

FIG. 20 (b) is a layout diagram in a single-landscape scene with a number of drops of 1 and a video stream according to some embodiments;

FIG. 21 (a) is a first layout diagram in a two-bar screen scenario with a 2-lane number, according to some embodiments;

FIG. 21 (b) is a second layout diagram in a 2-drop screen scenario with dual cross-screen scenarios, according to some embodiments;

FIG. 21 (c) is a first layout diagram of a 2-drop road number in a horizontal screen, vertical screen scenario according to some embodiments;

FIG. 21 (d) is a second layout diagram of a 2-drop road number in a horizontal screen, vertical screen scenario according to some embodiments;

FIG. 21 (e) is a layout diagram in a two vertical screen scenario with a 2 number of screen shots according to some embodiments;

FIG. 22 (a) is a schematic diagram of version 1 of a layout in a double-landscape (aspect ratio example different) scenario with a 2-throw road count, according to some embodiments;

FIG. 22 (b) is a schematic diagram of version 2 of a layout in a double-landscape (aspect ratio example different) scenario with a 2 drop-off number of passes according to some embodiments;

FIG. 22 (c) is a schematic diagram of version 1 of a layout in a two vertical (aspect ratio different) scene with a 2 drop road count, according to some embodiments;

FIG. 22 (d) is a schematic diagram of version 2 of a layout in a two vertical (aspect ratio different) scene with a 2 drop road count, according to some embodiments;

FIG. 23 (a) is a screen shot of 3 and a cross screen, according to some embodiments schematic diagram of version 1 of layout in two vertical screen scenario;

FIG. 23 (b) is a schematic diagram of version 2 of the layout in a 3-drop screen, one-horizontal, two-vertical screen scenario, according to some embodiments;

FIG. 23 (c) is a schematic diagram of version 3 of the layout in a 3 drop road number and one horizontal, two vertical screen scenario, according to some embodiments;

FIG. 24 is a flow chart of interaction of a terminal device with a display device according to some embodiments;

FIG. 25 is a diagram of an icon control interface display for an application in a display device 200 according to some embodiments;

fig. 26 is a configuration diagram of a controller in a display device according to some embodiments;

FIG. 27 is a flow chart of a method for a controller to perform a screen projection in a display device according to some embodiments;

FIG. 28 is another flow diagram of a controller performing a screen projection method in a display device according to some embodiments;

FIG. 29 is a schematic diagram of a play window on a display in a display device according to some embodiments;

FIG. 30 is a schematic structural diagram of a screen-casting interaction between a plurality of mobile terminals and a display device, according to some embodiments;

fig. 31 is a timing diagram of a screen shot interaction between a first mobile terminal and a display device, according to some embodiments.

Detailed Description

For purposes of clarity and implementation of the present application, the following description will make clear and complete descriptions of exemplary implementations of the present application with reference to the accompanying drawings in which exemplary implementations of the present application are illustrated, it being apparent that the exemplary implementations described are only some, but not all, of the examples of the present application.

It should be noted that the brief description of the terms in the present application is only for convenience in understanding the embodiments described below, and is not intended to limit the embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.

Fig. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in fig. 1, a user may operate the display device 200 through the smart device 300 or the control apparatus 100.

In some embodiments, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or bluetooth protocol communication, and other short-range communication modes, and the display device 200 is controlled by a wireless or wired mode. The user may control the display device 200 by inputting user instructions through keys on a remote control, voice input, control panel input, etc.

In some embodiments, the smart device 300 (e.g., a communication terminal, tablet, computer, notebook, etc.) may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on a smart device.

In some embodiments, the display device 200 may also perform control in a manner other than the control apparatus 100 and the smart device 300, for example, the voice command control of the user may be directly received through a module configured inside the display device 200 device for acquiring voice commands, or the voice command control of the user may be received through a voice control device configured outside the display device 200 device.

In some embodiments, the display device 200 is also in data communication with a server 400. The display device 200 may be permitted to make communication connections via a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200.

Fig. 2 is a block diagram of a configuration of the control apparatus 100 according to some embodiments. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction of a user and convert the operation instruction into an instruction recognizable and responsive to the display device 200, and may mediate interactions between the user and the display device 200.

Fig. 3 shows a hardware configuration block diagram of the display device 200 in accordance with an exemplary embodiment.

In some embodiments, display apparatus 200 includes at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, memory, a power supply, a user interface.

In some embodiments the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, a first interface for input/output to an nth interface.

In some embodiments, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, or other network communication protocol chip or a near field communication protocol chip, and an infrared receiver.

In some embodiments, the user input interface may be used to receive control signals from the control device 100 (e.g., an infrared remote control, etc.).

In some embodiments, the detector 230 may include an optical receiver, an image collector, a sound collector, and the like.

In some embodiments, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc.

In some embodiments, a system of display devices may include a Kernel (Kernel), a command parser (shell), a file system, and an application program.

Referring to FIG. 4, in some embodiments, the system is divided into four layers, from top to bottom, an application layer (application layer), an application framework layer (Application Framework layer), an Android run layer (An Zhuoyun row) and a system library layer (system runtime layer), and a kernel layer, respectively.

Fig. 5 shows a schematic configuration diagram of the communication terminal 300.

The embodiment will be specifically described below taking the communication terminal 300 as an example. It should be understood that the communication terminal 300 shown in fig. 5 is only one example, and that the communication terminal 300 may have more or fewer components than shown in fig. 5, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

As shown in fig. 5, the communication terminal 300 includes: radio Frequency (RF) circuitry 310, memory 320, display unit 330, camera 340, sensor 350, audio circuitry 360, wireless fidelity (Wireless Fidelity, wi-Fi) circuitry 370, processor 380, bluetooth circuitry 381, and power supply 390.

Fig. 6 is a software configuration block diagram of the communication terminal 300 according to the embodiment of the present invention.

The layered architecture divides the software into several layers, each with distinct roles and branches. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, from top to bottom, an application layer, an application framework layer, an Zhuoyun row (Android run) and system libraries, and a kernel layer, respectively.

The workflow of the communication terminal 300 software and hardware is described below in connection with capturing a photo scene.

When touch screen 331 receives a touch operation, a corresponding hardware interrupt is issued to the kernel layer. The kernel layer processes the touch operation into the original input event (including information of touch coordinates, time stamp of touch operation, etc.). The original input event is stored at the kernel layer. The application framework layer acquires an original input event from the kernel layer, and identifies a control corresponding to the input event. Taking the touch operation as a touch click operation, taking a control corresponding to the click operation as an example of a control of a camera application icon, the camera application calls an interface of an application framework layer, starts the camera application, further starts a camera driver by calling a kernel layer, and captures a still image or video through the camera 340.

As shown in fig. 7, a user may open a corresponding application by touching an application icon on the user interface, or may open a corresponding folder by touching a folder icon on the user interface.

Based on the display device 200 and the communication terminal 300 described above, the display device 200 can be caused to play the content in the display communication terminal 300 by establishing a screen-cast connection, as shown in fig. 8. In some embodiments, as shown in fig. 9, in order to establish the screen-cast connection relationship, a user may issue a screen-cast connection request through the communication terminal 300 and transmit the screen-cast connection request to the display apparatus 200 through the WiFi network. The display device 200 completes the configuration of the transmission protocol according to the screen connection request, so as to establish a transmission channel of the screen data with the communication terminal 300.

The display device 200 may also implement a screen-drop connection with the communication terminal 300 through different connection means. For example, when the display apparatus 200 and the communication terminal 300 access the same wireless lan, a screen-cast connection may be established based on a WiFi network. For another example, when NFC (Near Field Communication ) components are provided on both the display device 200 and the communication terminal 300, a screen-casting connection relationship may be established through the NFC component. It is obvious that other wired or wireless connection manners may be adopted between the display device 200 and the communication terminal 300, so as to establish a projection screen connection relationship, for example, an RF radio frequency connection, an infrared connection, a cellular network, and other connection manners that can be imagined by those skilled in the art based on the connection manners provided in the foregoing embodiments.

After the screen-cast connection is established, the display device 200 may receive the related data of the communication terminal 300 through the screen-cast data channel, thereby playing the related content in the display device 200. Under different screen data transmission protocols, the screen data acquired by the display device 200 is in different forms. For example, under the DLNA (Digital Living Network Alliance ) protocol, the display device 200 may acquire a link address of a certain resource in the communication terminal 300 through a screen-cast data channel and acquire media content data by accessing the link address, thereby playing the media data. Meanwhile, the display device 200 may also obtain, through the screen-throwing data channel, a control instruction sent by the communication terminal 300, such as fast forward, pause, stop screen throwing, etc., when playing media asset data. For example, after the connection is established through the communication protocol, the communication terminal 300 such as a mobile phone and the display device 200 may perform negotiation between the device function and the network condition to select a suitable audio/video transmission format. After establishing an on-line protocol (Session) of the video stream, a series of real-time streaming protocol (RTSP) control commands are used to control the playback and termination of the video stream.

Under other screen connection protocols such as Miracast screen projection, airplay screen projection, NFC mirror screen projection, music broadcast screen projection, etc., the display device 200 may directly acquire the video data stream from the communication terminal 300. The video data stream may be a video data stream corresponding to a final display screen of the display unit 310 of the communication terminal 300, that is, the video data stream includes a media content screen, a UI interface, and other video screen contents that may be displayed in the display unit 310 in the communication terminal 300. The video data stream may be just video data that can be acquired in the communication terminal 300, such as media video data being played, or the like. At this time, the communication terminal 300 may serve as a signal source of the display device 200, and the display device 200 may continuously acquire video data through the on-screen data channel and display the video data on the display 275 in real time, thereby achieving the effect of displaying the video data in synchronization with the communication terminal 300.

The communication terminal 300 can acquire different screen projection data according to different screen projection protocols. For example, for a system application or directly on the system level, the communication terminal 300 may establish a screen-casting data channel using a Miracast screen-casting, an Airplay screen-casting, or a WiDi screen-casting protocol.

The Miracast screen can be suitable for Android equipment, namely the Android equipment can share video pictures in a Miracast screen-throwing mode. Generally, the intelligent device with the android of more than 5.0 is internally provided with a Miracast protocol, so that the direct wireless screen can be directly carried out through the WiFiP2P function between the screen-throwing device and the WiFi network card of the receiving end device under the condition of no need of installing any software. In addition, miracast screen-throwing also supports audio and video synchronous transmission. However, since the Android devices are too many and too miscellaneous, the Miracast cannot be fully compatible with all the Android devices, and the screen-throwing function based on the Miracast cannot be realized, and malicious or misscreen-throwing cannot be prevented.

The Airplay screen is a wireless playing technology which is put forward by apple corporation and is suitable for wireless screen projection of iOS and Mac systems. The built-in Airplay screen-throwing function of the apple series equipment is matched with a screen-throwing device supporting AirPlay server service for use, the effect is clear and stable, and the protocol has the screen-throwing code function. However, it is necessary to ensure that the apple device and the screen projector are used under the same local area network (same network segment), and cannot be used across network segments and across VLANs.

The WiDi screen-throwing technology is a wireless playing technology which is dominant by Intel and is also based on a WiFi Direct connection technology. The Windows8/10 notebook computer originally supports WiDi screen projection, can directly project the screen without installing any software, and supports the USB reverse control function.

The above three screen projection protocols can acquire not only a video data stream in the communication terminal 300 but also an audio data stream in the communication terminal 300, so that the video data and the audio data can be synchronously transmitted to the display apparatus 200, and the display apparatus 200 can synchronously play video and audio. Since the above-mentioned Miracast screen-throwing protocol and the air-lay screen-throwing protocol require system-level application authority, the Miracast screen-throwing protocol and the air-lay screen-throwing protocol are generally only suitable for system application, or the display device 200 and the communication terminal 300 of the same manufacturer directly establish screen-throwing connection through the built-in rule of the operating system.

The display device 200 may establish a screen-projection connection with the communication terminal 300 based on any of the screen-projection manners provided in the above embodiments. In some embodiments, the display device 200 may also establish a screen-projection connection relationship with a plurality of communication terminals through one or more of the screen-projection modes described above. For example, the display apparatus 200 may simultaneously establish a screen-casting connection with two communication terminals 300 and divide a display area in a display screen for respectively displaying screen-casting screens shared by the two communication terminals 300. That is, as shown in fig. 10, a screen shot shared by the terminal a may be displayed on the left side of the user interface displayed by the display device 200, and a screen shot shared by the terminal B may be displayed on the right side of the user interface.

In order to be able to display a screen shot shared by a plurality of communication terminals 300 on the display device 200, in some embodiments, when the display device 200 establishes a screen shot connection with a plurality of communication terminals 300, the display device 200 may divide a user interface for the screen shot display into areas, such as halves, thirds, quarters, etc. After a plurality of display areas are obtained by area division, the display apparatus 200 displays a screen shot respectively in each divided area.

In addition, the communication terminal 300 may be in different placement states according to different usage modes, such as a vertical screen state and a horizontal screen state for the communication terminal 300 such as a smart phone. The vertical screen state refers to a state in which the width of the displayed screen is smaller than the height, and the horizontal screen state refers to a state in which the width of the displayed screen is larger than the height. The communication terminal 300 may present different screen layouts in different placement states. Based on this, when the display device 200 displays the screen shots of the plurality of communication terminals 300 by the area division, each display area may be presented with different screen proportions depending on the placement state of the communication terminal 300.

For example, as shown in fig. 11, for the display device 200 with a display resolution of 4096×2160, in the process of vertically halving the center position of the user interface to obtain two display areas, the area division may obtain two display areas with a width of 2048 and a height of 2160, that is, a vertically shaped area with a width smaller than the height, and this form of display area is more suitable for displaying a screen in a vertical screen state. Therefore, when the communication terminal 300 corresponding to the area is in the landscape state, the projection screen cannot fill most of the display area, resulting in waste of the display area.

In addition, when the plurality of communication terminals 300 are placed in different states, the display device 200 often fills the display area with each projection screen by stretching and filling. The display mode can enable the screen projection picture to be greatly deformed, and the viewing experience of a user is affected.

In order to fully utilize the display area and reduce the stretching deformation of the picture, some embodiments of the present application further provide a method for displaying the projection picture dynamically, which is applied to the display device 200 and is used for improving the viewing experience of the user. Wherein, the display device 200 to which the screen-casting picture dynamic display method can be applied includes: a display 260, a communicator 220, and a controller 250. The display 260 is configured to display a projection screen interface. The communicator 220 is configured to establish a screen-drop connection with the communication terminal 300. The controller 250 is configured to execute the above-mentioned method for displaying a screen, as shown in fig. 12, and specifically includes the following:

the number of connections of the communication terminal 300 is detected. The display device 200 may detect an operation state in real time during operation, and detect the number of communication terminals 300 that establish a screen-casting connection with the display device 200 during a process of determining that a screen-casting screen needs to be displayed. The number of connections of the communication terminal 300 can be obtained by detecting and judging each communication terminal in the wireless local area network where the display device 200 is located, respectively. For example, when the number of connections of the communication terminal 300 needs to be detected, the controller 250 of the display device 200 may invoke all the communication terminals 300 accessed in the current wireless network through the communicator 220, and then grasp transmission data of each communication terminal and identify a screen-throwing data flag therefrom to detect whether the current communication terminal 300 establishes a communication connection with the display device 200.

The number of connections may also be obtained by detection from a data table maintained in real time by the display device 200. For example, in the database of the display device 200, a connection device list may be maintained in real time along with the screen-on connection process, and each communication terminal 300 in the connection device list may set a different status value according to whether the connection device is online and establishes a screen-on connection relationship, that is, when the communication terminal 300 is online and the connection is available, the status value "devices online=1"; and the status value "devices online=0" when the communication terminal 300 is offline or connection is unavailable. Based on this, the display apparatus 200 can traverse the number of communication terminals 300 of the status value "device online=1" when the number of connections needs to be detected.

The display device 200 may perform detection of the number of communication terminals 300 when it is determined that the current screen-drop interface layout needs to be changed. For example, the display device 200 may monitor the access state of each communication terminal 300 in the screen-on connection in real time, and when detecting that any communication terminal 300 newly accesses or exits the connection, the display device 200 may perform the step of detecting the number of connections of the communication terminal 300.

In some embodiments, the display apparatus 200 may also periodically detect the number of connections of the communication terminal 300. For example, the display device 200 may detect the number of connections of the communication terminals 300 by the heartbeat command of each communication terminal 300 after starting to establish the screen-drop connection. By detecting the number of connections of the communication terminal 300, the display device 200 can timely detect the screen-throwing connection state of the communication terminal 300, so that when the communication terminal goes offline due to exceeding the network range or other hardware fault reasons, the screen-throwing interface can be adjusted.

In some embodiments, the display apparatus 200 may also detect the number of connections of the communication terminal 300 by detecting a control instruction of a user and responding to a part of the control instruction. For example, when the display apparatus 200 detects that the user inputs a control instruction for adjusting the page layout manner, the display apparatus 200 may perform a step of detecting the number of connections of the communication terminal 300 to trigger the subsequent adjustment of the screen input data acquisition manner by the display apparatus 200.

After detecting the number of connections of the communication terminal 300, the display device 200 may plan a layout manner of the screen-casting screen according to the detected number of connections to obtain display parameters of the screen-casting screen. Various layout templates may be built in the display apparatus 200, and after the number of connections is acquired, the display apparatus 200 may match the templates of the response according to the number of connections.

The built-in layout templates may also be different for different display device 200 formats, as well as different screen-connected users. For example, for the screen-casting connection sharing the mobile phone main interface demonstration operation, since the mobile phone main interface operation is mostly in a vertical screen state, the layout mode template may be a layout template obtained by a mode of equally dividing in a vertical direction, that is, three vertical areas with a width smaller than a height are divided on the screen of the display device 200; for the screen connection of the operation of sharing the game interface, the game interface is in a horizontal screen state, so the layout mode template can be obtained by equally dividing the horizontal direction, that is, dividing three horizontal areas with width larger than height on the screen of the display device 200 for displaying the game images shared by the mobile phone.

In some embodiments, when the display apparatus 200 does not explicitly set the screen-in connection channel, in order to be able to fully utilize the display area, the display apparatus 200 may plan the screen layout in accordance with the principle of maximizing the display. For example, when 3 screen-shot regions are displayed, one portrait region and two landscape regions may be maintained. In this way, each display area may be maintained in a 16:9 or 9:16 shape to accommodate the screen ratio of the communication terminal 300.

In some embodiments, the display device 200 may also plan a layout manner based on the number of connections by adding a display window to the user interface after detecting the number of connections of the communication terminal 300. For example, when it is detected that the number of communication terminals 300 for which the current display device 200 establishes a screen-cast connection is 2, 2 display windows may be added to the user interface for respectively displaying screen-cast pictures shared by each communication terminal 300. Each display window may have a fixed shape and scale, and the display device 200 may be required to provide a display screen in accordance with the scale.

In some embodiments, when a plurality of screen shots are displayed using a plurality of display windows, if the number of communication terminals 300 establishing screen shots with the display device 200 is large, a specific hierarchical relationship may also be maintained between the plurality of display windows. For example, the hierarchical relationship of the display windows may be set in the order in which the screen-casting connections are established with the display device 200, that is, the screen-casting screen shared by the communication terminal 300 that finally establishes the screen-casting connection is displayed at the uppermost layer.

After the display device 200 plans the layout manner of the screen-throwing picture, the display device 200 may also determine the display parameters of the screen-throwing picture according to the planned layout manner. The display parameters are used to represent the content of the picture proportion, the size, etc. of each display area in the layout mode planned by the display device 200, and the requirements of the screen projection data calculated according to the proportion, the size, etc. of each display area, such as code rate, resolution, etc.

According to the determined display parameters, the display device 200 may transmit a notification message to the communication terminal 300 corresponding to the screen shot. Wherein the notification message includes display parameters, and the notification message is used to enable the communication terminal 300 to generate screen projection data according to the display parameters. For example, as shown in fig. 13, when the display device 200 detects that the number of connections of the current communication terminal 300 is 3, the display areas of three screen shots may be determined by a planned layout manner, that is: region one is a vertical region of resolution "1080×1920"; the second and third regions are lateral regions of resolution "1280×720". Based on the above display parameters, the display device 200 may generate a notification message and transmit the generated notification message to the communication terminal 300, so that the communication terminal 300 may generate the screen projection data at the above resolution.

In order to generate the screen projection data, the communication terminal 300 may incorporate a screen recording program, and after receiving the notification message sent by the display device 200, start the screen recording program to parse the notification message, and extract the display parameters. And setting picture parameters in the screen recording program according to the display parameters. If the display parameter is vertical and 1080×1920, the communication terminal 300 may control the recording process of the recording program, so that the resolution of the generated recording video picture is 1080×1920.

If the display resolution of the communication terminal 300 does not match the resolution set in the notification message, the communication terminal 300 may process the screen by filling or cutting the screen so that the generated screen projection data matches the format specified in the display parameter. For example, when the screen ratio of each display area set in the display device 200 is 9:16, but the screen ratio of the communication terminal 300 is 9:18, in order to be able to display completely, the communication terminal 300 may perform a reduction process on the display screen, so that the screen can be displayed completely in height, and at this time, black areas will be filled in two sides of the generated screen corresponding to the screen data, so that the overall screen is maintained at the screen ratio of 9:16.

After the communication terminal 300 generates the screen projection data, the generated screen projection data may be fed back to the display device 200, and the display device 200 may display the received screen projection data, that is, display the screen in the screen projection data according to the layout mode. For example, when the display device 200 transmits notification messages containing display parameters of "vertical, 1080×1920", "horizontal, 1280×720" to three communication terminals, respectively, the three communication terminals 300 may feed back three screen-projection data streams with resolutions of 1080×1920, 1280×720, and 1280×720, respectively, and the display device 200 parses and renders the received three screen-projection data streams, respectively, to finally form a screen-projection screen.

It should be noted that, in order to simultaneously display a plurality of screen shots on the display device 200, when receiving a plurality of screen shots, the display device 200 may create a data buffer for each communication terminal 300, where the data buffer is used to store the screen shots sent by each communication terminal, and each display control arranged on the display device 200 obtains video data from the corresponding data buffer to display the video data, and when a certain communication terminal 300 is disconnected, the display device 200 may also destroy and release the corresponding data buffer.

The method for dynamically displaying the screen projection images provided in the above embodiment may plan the layout manner of a plurality of screen projection images in the screen projection interface by detecting the connection number of the communication terminals 300, and send a notification message containing the display parameters of each area to each communication terminal 300, so that the communication terminal 300 may generate the screen projection data in a manner matched with the layout manner. The display mode can ensure that the picture presented by the final display equipment 200 can adapt to the layout requirement, alleviate the deformation of the picture and improve the user viewing experience.

In some embodiments, the display apparatus 200 may start detecting the number of connections of the communication terminal 300 when the communication terminal 300 is accessed. That is, as shown in fig. 14, the display device 200 may acquire a connection instruction for establishing a screen-throwing connection input by a user, and initiate a screen-throwing connection to the communication terminal 300 in response to the connection instruction.

Wherein the connection instruction can be input in different ways. For example, for the communication terminal 300 that establishes a screen-cast connection with the display apparatus 200 for the first time. The user may first connect to the wireless network where the display device 200 is located through the network setting, and then perform an interactive operation on the communication terminal 300, for example, click on a "screen-throwing" button, to initiate a screen-throwing connection. The display device 200 may display the pairing interface of the screen connection after receiving the screen connection, and at this time, the user needs to execute the interactive operation again based on the pairing interface of the screen connection, for example, selecting the corresponding communication terminal 300 in the screen device list, so that the display device 200 and the communication terminal 300 may establish the screen connection. It can be seen that for the first connected communication terminal 300, the connection instruction is a series of interaction combinations based on the display device 200 and the communication terminal 300 described above.

For another example, for the communication terminal 300 that does not establish the screen connection with the display device 200 for the first time, since the device information of the other party can be stored in the display device 200 and the communication terminal 300, the user can directly click the "screen-throwing" button on the communication terminal 300 to initiate the screen-throwing operation, and at this time, the display device 200 can also receive the screen-throwing connection instruction, and automatically respond to the connection instruction to implement mutual handshake with the communication terminal 300, so as to complete the screen-throwing connection.

After the screen-cast connection configuration is completed, the display apparatus 200 may also traverse the number of connections of the communication terminal 300. For example, after one communication terminal 300 initiates the establishment of the screen-cast connection, the display apparatus 200 may first wait for the configuration procedure of the screen-cast connection. When the configuration of the screen-cast connection is completed, the total number of currently connected communication terminals 300 is traversed. I.e. the number of detected connections of the communication terminal 300, comprises the communication terminal 300 that newly established the screen-cast connection.

In some embodiments, the display apparatus 200 may further perform detection of the number of connections when it is detected that the communication terminal 300 stops the screen-on connection, and as shown in fig. 15, the display apparatus 200 may acquire an exit instruction for breaking the screen-on connection input by the user in the step of detecting the number of connections of the communication terminal 300, break the screen-on connection of the communication terminal 300 in response to the exit instruction, and traverse the number of connections of the remaining communication terminal 300 after breaking the screen-on connection.

For example, when 3 communication terminals 300 simultaneously establish a screen-on connection with the display device 200, the user may click a "stop screen-on" button on one of the communication terminals 300, and the control communication terminal 300 no longer executes the screen-recording program. At this time, the display apparatus 200 may automatically disconnect the screen shot connection established with the communication terminal 300 when it is detected that the communication terminal 300 does not transmit the screen shot data any more. And detects that the number of connections of the remaining communication terminals 300 is 2 after the screen-break connection. Thereby re-planning the layout in the screen-throwing interface according to the use states of the 2 communication terminals 300.

It can be seen that, in the above embodiment, since the screen-throwing connection is established at the new communication terminal 300, or when any communication terminal 300 exits the screen-throwing, the number of screen-throwing pictures in the screen-throwing interface presented by the display device 200 is changed, and the corresponding layout manner is also changed. Therefore, the connection quantity detection mode can be automatically adapted to the layout mode of the pictures on the display device 200, and timely adjust the display content, thereby improving the user experience.

In some embodiments, after the display device 200 establishes a screen connection with the communication terminal 300, the display device 200 may also establish a screen data channel when initiating the screen connection to the communication terminal 300, and after the screen data channel is established successfully, create a display control to display the screen data received through the screen data channel through the display control.

Among them, the display control is a display control program built in the display apparatus 200, such as surface View or Texture View. The display controls may add numbers that appear as different display modes. For example, when 1 display control is included in the display device 200, the display controls may be displayed in a full screen manner. When the display device 200 contains 2 display controls, the two display controls are in the form of windows in two portrait layouts, left and right.

As shown in fig. 16, in some embodiments, in order to determine the display parameters, the display device 200 may first detect the screen size information of the display 260 when planning the layout manner of the projection screen according to the number of connections, and then match the layout templates in the database with the number of connections and the screen size information as indexes, thereby extracting the display parameters in the layout templates.

The screen size information of the display 260 is different for different types of display devices 200, and thus the layout pattern conforming to the screen specification thereof is also different. For example, for a smart television with a conventional display proportion, in order to fully utilize the display space, when 3 screen projection images are displayed, a vertical display area layout mode and two horizontal display area layout modes can be adapted; for the super-wide screen movie broadcasting equipment, when 3 screen projection pictures are displayed, a vertical trisection display area layout mode can be adopted.

In addition, in the display device 200 or the server 400, a database may be run and maintained, in which layout templates corresponding to various screen size specifications and the number of connections may be stored. Accordingly, the display apparatus 200 can determine an optimal layout pattern adapted to the current display state according to the screen size information and the number of connections.

In some embodiments, the display device 200 may also determine the final layout mode to be adopted according to the matching result of the layout template. In the step of matching the layout templates in the database, if the layout templates are not matched in the database, the layout mode can be determined directly by using the layout templates obtained by matching; if the layout template is not matched in the database, the display area can be divided into areas, and the proper layout mode is determined. That is, the display device 200 may acquire device parameters of the communication terminal 300, divide sub-areas of the same number as the connection number in the display area according to the device parameters, and generate display parameters by traversing the graphic parameters of each sub-area.

For example, when the layout template is not matched in the database, the display device 200 may first detect a device parameter of the communication terminal 300, such as a model number, a screen size specification, etc., for which the screen-cast connection is currently established. The display device 200 then determines the corresponding display area shape according to the screen size specifications of all the connected communication terminals 300. Obviously, in order to obtain a better display effect, the display area shape may be similar to the screen shape of the communication terminal 300. The display apparatus 200 performs division of the display area according to the determined shape of the display area, thereby maximizing the utilization of the display space on the premise of ensuring the integrity of the display contents.

In some embodiments, the display apparatus 200 may further extract a display parameter in the step of transmitting a notification message to the communication terminal 300 corresponding to the screen shot, and generate the notification message according to the display parameter. Wherein the display parameters include a display direction of each display window. The display direction is used for indicating the communication terminal to generate screen projection data according to the set direction. In order to adapt to the placement state of the communication terminal 300, the display device 200 may preferentially match the adapted communication terminal 300 according to the shape of each display area after determining the layout manner. For example, when the display direction of the display window is vertical, the screen projection screen of the communication terminal 300 in the vertical screen state may be preferentially displayed in this window; when the display direction of the display window is landscape, the screen of the communication terminal 300 in the landscape state may be preferentially displayed in this window.

Since the larger the number of screen shots included in the user interface, the smaller the area occupied by each display area, the worse the final screen effect presented in each display area, and when the display area is smaller, the display area cannot obtain a corresponding effect for the display content of high image quality, the display device 200 may further determine the screen quality of the screen shots according to the number of connections after detecting the number of connections of the communication terminal 300 in order to reduce the total data transmission amount. I.e. the display parameters further comprise a code rate and a resolution of each display window, so that the communication terminal 300 can generate the projection data with reference to the set code rate and resolution.

Obviously, when the number of communication terminals 300 that establish screen-cast connections with the display apparatus 200 is larger, the range of each display area is correspondingly smaller, the code rate and resolution may be set smaller so that the transmission data amount of the screen-cast data is smaller, and vice versa. The code rate can be in one-to-one correspondence with the number of connections, or in many-to-one correspondence. If the display apparatus 200 is simultaneously connected to 1 communication terminal 300 or 2 communication terminals 300, a transmission code rate may be set to 5Mb/s; if the display apparatus 200 is connected to 3 communication terminals 300 or 4 communication terminals 300 at the same time, a transmission code rate of 3Mb/s may be set.

The resolution may be the same as the number of pixels each display area occupies in the width and height directions after dividing the display area. For example, when the number of pixels occupied by the first display region is 1080×1920, the resolution of the corresponding screen data may be 1080×1920. When the number of pixels occupied by the second display area and the third display area is 1280×720, the resolution of the corresponding screen projection data may be 1280×720.

After determining the display parameters, the notification message generated according to the display parameters also needs to include the above parameter content, that is, the notification message includes the display direction, the code rate and the resolution, so that the communication terminal 300 can parse the display direction, the code rate and the resolution from the notification message after receiving the notification message, and generate the screen projection data according to the above parameters, thereby ensuring the consistency of the screen projection picture and each display area.

Because of the limitation of the layout, even when the placement state of the communication terminal 300 does not match the display direction corresponding to the display area during the generation of the screen data, the communication terminal 300 generates the screen data according to the shape of the display area. For example, when the display area is in a vertical shape and the corresponding communication terminal 300 is in a horizontal screen state, the communication terminal 300 still generates a screen projection screen according to the vertical shape, but in the process of generating screen projection data, black areas may be added at the top and bottom of the display screen respectively, so that the whole screen is presented as a vertical screen with a width smaller than the height, and the screen projection screen can be displayed in the display area in the vertical shape without stretching deformation.

However, when the display area does not match the placement state, most of the screen of the display area is easily displayed as a solid-color filled background, so in some embodiments, the display device 200 may prompt the user to adjust the placement state of the communication terminal 300 by prompting the user with a prompt message when detecting that the display area does not match the placement state. As shown in fig. 17, the display device 200 may perform the steps of:

step S171, detecting the effective picture range of each screen-throwing picture in the screen-throwing interface;

Step S172, calculating the display proportion of the effective picture range to the total range of the display window;

step S173, if the display ratio is less than or equal to the ratio threshold, that is, it is determined that the display area does not match the placement state, step S174 is performed, otherwise step S175 is performed;

step S174, a prompt message is sent to the communication terminal 300 to prompt the user to rotate the direction manually or automatically by an internal mechanism of the communication terminal 300;

step S175 ends the process.

For example, when the display area is in a portrait shape and the corresponding communication terminal 300 is in a landscape screen state, the display device 200 may detect that the display ratio is less than or equal to the ratio threshold, so that a prompt message may be sent to the communication terminal 300, for example, "the current display direction is not matched, please rotate the mobile phone to a portrait screen state", to prompt the user to adjust the communication terminal 300 to a landscape screen state, and increase the display ratio of the effective screen in the projection data.

Based on the above display device 200, there is also provided in some embodiments of the present application a communication terminal 300 including: display unit 330, communication unit, and processor 380. Wherein the display unit 330 is configured to display a terminal interface, and the communication unit is configured to establish a screen-throwing connection with the display device 200; as shown in fig. 18, the processor 380 is configured to perform the steps of:

Step 181, receiving a notification message sent by the display device 200, where the notification message includes display parameters of a screen-throwing picture, where the display parameters are obtained when the display device 200 plans a screen-throwing picture layout mode according to the number of connections of the current communication terminal 300;

step 182, generating screen projection data according to the display parameters;

step 183, feeding back the screen projection data to the display device 200.

Based on the display device 200 and the communication terminal 300, some embodiments of the present application further provide a method for dynamically displaying a projection screen, which includes the following steps:

the display device 200 detects the connection number of the communication terminal 300, and plans the layout mode of the screen-throwing picture according to the connection number so as to obtain the display parameters of the screen-throwing picture;

the display device 200 sends a notification message to the communication terminal 300 corresponding to the screen projection picture, wherein the notification message comprises the display parameters;

the communication terminal 300 receives the notification message sent by the display device 200 and generates screen projection data according to the display parameters;

the communication terminal 300 feeds back the screen-casting data to the display device 200;

the display device 200 displays the screen projection data according to the layout mode.

The method for dynamically displaying the screen projection screen according to the above embodiment can plan the layout mode of the screen projection screen by detecting the number of connections of the communication terminal 300 when the user triggers the number of the communication terminals 300 connected to the display device 200 to change. And then sends a notification message containing the display parameters of the screen to the communication terminal 300, so that the communication terminal 300 can generate the screen projection data conforming to the layout mode according to the display parameters, and display the screen projection data in the display device 200. The method can enable the display device 200 to support multi-path screen projection, dynamically adjust the display layout of the screen projection picture, and inform the communication terminal 300 to carry out screen recording adjustment, so that the displayed screen projection picture layout is more reasonable.

The above embodiments introduce the content of the hardware/software architecture, the functional implementation, etc. of the display device. In some application scenarios, the terminal device may push media data to the display device, for example, the user projects a video of interest to the display device for large-screen playing through some application programs with a screen projection function; or the terminal device and the display device establish a screen-casting connection through some protocols, such as DLNA (Digital Living Network Alliance ) protocol, and the like. After the terminal equipment and the display equipment are successfully connected, the terminal equipment can project the screen projection data to the display equipment, and the display equipment displays the screen projection data on a screen projection interface. The terminal equipment can be mobile terminals such as smart phones, tablet computers and notebook computers, and can also be desktop computers or other small-screen display equipment.

In some embodiments, when the terminal device plays the media data, the user may select a device for playing the media data by dropping a screen from the device list, and the display device supporting the function of dropping a screen may be identified and automatically added to the device list, and after the user selects the display device from the device list, the user may then project the media data from the terminal device to the display device for playing. When the screen-throwing mode is adopted to push the media data, the terminal equipment and the display equipment can be connected to the same Wifi network, and the mode that the terminal pushes the media data to the display equipment is not limited. Of course, in other technologies, the screen-throwing mode can also be performed across networks. The specific implementation manner of the screen projection is not limited in the application.

In some embodiments, terminal device a sends a screen projection request to the display device, which may query the user, in the form of a UI query pop-up, etc., whether to agree with the screen projection of terminal device a, and may provide several option controls, such as "this agree", "disagree", and "permanent agree". The user selects 'permanent agreement', namely, the permanent authorization is equivalent to the permanent authorization of the terminal equipment A, and the subsequent terminal equipment A does not need to display a UI inquiry popup window any more when initiating a screen throwing request, but directly establishes the screen throwing connection between the display equipment and the terminal equipment A; if the user selects 'this time agrees', namely, the user performs one-time authorization on the terminal equipment A, the display equipment and the terminal equipment A can establish screen-throwing connection, but the screen-throwing authority of the terminal equipment A is only valid for the next time, and the UI inquiry popup window still needs to be displayed again when the subsequent terminal equipment A initiates a screen-throwing request; if the user selects "disagree", namely, the terminal equipment A is not authorized for the time, the display equipment and the terminal equipment A do not establish screen projection connection, and the UI inquiry popup window still needs to be displayed again when the subsequent terminal equipment A initiates a screen projection request.

In some embodiments, the display device may maintain a trusted device list, and support the user to add, delete and modify device information with the screen-casting authority in the trusted device list, where the device information includes, but is not limited to, a device ID, a MAC address, etc., that is, the user may customize the authorization list on the display device side. When the display device receives a screen projection request sent by the terminal device B, inquiring whether the terminal device B is contained in the trusted device list according to device information carried in the screen projection request. If the terminal equipment B is contained in the trusted equipment list, establishing screen-casting connection between the terminal equipment B and the display equipment; if the trusted device list does not include the terminal device B, the display device does not respond to the screen-throwing request, namely the screen-throwing connection with the terminal device B is not established. It should be noted that, the authorization and connection manner of the terminal device for screen projection are not limited to the examples of the present application.

In the current application scenario, a one-to-one screen-throwing mode is adopted, that is, the display device is only connected with one terminal device in a screen throwing mode, and screen throwing data can be displayed exclusively on a screen. However, in other application scenarios, for example, when the user plays the screen-throwing game of the terminal device C, if the user wants to synchronously watch the course or the explanation of the game, a screen-throwing request of the game course/explanation video can be initiated to the display device through the terminal device D, and after the screen-throwing connection between the display device and the terminal device D is successful, the connection is equivalent to accessing two paths of screen-throwing data; for example, the family comprises a member 1, a member 2 and a member 3, the three members can respectively initiate a screen throwing request to the display equipment through the used terminals, after the display equipment responds to the three screen throwing requests, the screen of each terminal is displayed on a screen throwing interface, the three members can perform interactive communication and the like, and at the moment, the display equipment is connected with three paths of screen throwing data.

For an application scene that the display device is connected with a plurality of screen throwing windows, the screen interface can display the screen throwing windows, each screen throwing window corresponds to a terminal device which is successfully connected with the screen throwing device and displays screen throwing data sent by the terminal device, but the display effect of the screen throwing windows on the screen throwing interface influences the appearance of a user, and a plurality of display effects exist: for example, a plurality of screen throwing windows are not compactly distributed, so that a screen throwing interface is more white; the distribution of a plurality of screen throwing windows is disordered, and the whole layout is not regular; the size of part of the screen-throwing window is smaller, so that the user cannot see the details of the video in the window clearly; the video is displayed on a transverse screen of the terminal equipment, the screen direction is disordered after the video is projected to the display equipment, the video is displayed on a vertical screen, and the uniformity of the video display between the terminal equipment and the display equipment is poor. In this regard, the present application aims at planning and laying out multiple projection windows to improve the display effect of projection interfaces.

Referring to fig. 19, an embodiment of the present application provides a multi-path screen display method, where the method is configured to be executed by a controller 250 in a display device, that is, the controller 250 is an execution subject of the method, and the method includes the following steps:

Step S01, matching and adapting to the layout among M screen throwing windows of a screen according to the M screen throwing paths counted currently and media configuration information received from M terminal devices respectively;

and step S02, displaying M screen throwing windows in the screen throwing interface according to the layout, and displaying screen throwing data sent by the corresponding terminal equipment in each screen throwing window.

The M is used for representing the number of screen-throwing paths, that is, the number of terminal devices which establish screen-throwing connection with the display device, and the number of screen-throwing paths can dynamically change along with the connection and disconnection of the screen-throwing connection, so that the number of screen-throwing paths needs to be counted so as to accurately plan typesetting and layout of a screen-throwing interface. When M is 1, executing single-path screen projection; when M is more than or equal to 2, the screen is thrown in multiple ways; m can be zero, and the display device does not receive the screen projection data sent by the connection device, so that the screen projection interface is not displayed, and the layout on the screen projection interface is empty.

In some embodiments, each time the display device receives a screen-throwing request of a target terminal device, if a screen-throwing connection with the target terminal device is newly established, accumulating and adding one to the count value of the screen-throwing path number; in addition, the terminal equipment may disconnect the screen-throwing connection with the display equipment due to the factors of disconnection or manual screen-throwing exit of the terminal user, so that the display equipment cannot receive the screen-throwing data sent by the terminal equipment, and the display equipment end needs to cut off the screen-throwing, so that the count value of the screen-throwing road number is reduced by one when the screen-throwing connection with one terminal equipment is detected to be disconnected. According to the embodiment, the automatic counting of the screen throwing path number according to the screen throwing connection dynamic state of the display equipment can be realized.

In some embodiments, each time the display device receives the screen-throwing data of one target terminal device, the count value of the screen-throwing path number is added one in an accumulated manner.

In the application, the display device side receives the screen projection data from the M terminal devices and synchronously receives the media configuration information sent by the M terminal devices respectively.

In some embodiments, the screen-cast data relates to a media category, e.g., the media is a pure video category, then the screen-cast data includes a video stream that is screen-cast; if the media is in the audio-video category, the screen throwing data comprises a video stream and an audio stream; the screen data may be in the form of images (still images, moving pictures, etc.), text, or audio-only streams. When the on-screen data is an audio stream, play content information such as audio may be presented on the display interface.

In some embodiments, the screen shot data is related to a screen recording mode of the terminal device. For example, the recording mode may be recorded according to a fixed resolution, such as 1280×720, 1920×1080, and the like, and may also be recorded according to a size of a terminal screen, such as a 16:9 size, a 17:9 size, a 21:9 size, and the like.

In some embodiments, the media configuration information includes an aspect ratio of video projected by the terminal device, that is, a ratio of a horizontal width to a vertical height when the video is displayed, and the display screen direction can be identified through the aspect ratio, and when the aspect ratio is greater than or equal to 1, that is, the width is greater than or equal to the height, the display mode is presented as a horizontal screen display mode; when the aspect ratio is less than 1, i.e., the width is less than the height, then the portrait display mode is presented. The creation of the screen throwing window can be restrained according to the aspect ratio and the screen direction indicated in the media configuration information, so that the aspect ratio and the display screen direction of the screen throwing window and the corresponding terminal equipment are kept consistent, and the problem of disorder of media related configuration after screen throwing can be avoided.

In some embodiments, the media configuration information may also include an aspect ratio of the terminal device. When the aspect ratio is greater than or equal to 1, i.e., the width is greater than or equal to the height, then the display mode is presented as a landscape display mode; when the aspect ratio is less than 1, i.e., the width is less than the height, then the portrait display mode is presented.

In some embodiments, the media configuration information may further include a media category, and after the display device reads the media configuration information, the media category to be projected may be known, so that projection windows with different effects are displayed according to the media category. For example, for pure video, audio, and image media, the projection window may present visual projection data, and the projection window should match the aspect ratio and screen orientation of the video/image; for text type media, the drop screen window can visually display readable text information, but the aspect ratio and the screen orientation of the drop screen window are not limited; for pure audio media, the screen throwing window cannot display a visual picture, a voice playing control (such as a small loudspeaker) can be arranged in the screen throwing window, when the audio stream of the screen throwing channel is played, the voice playing control can be presented as a dynamic special effect in playing, and likewise, the aspect ratio and the screen direction of the screen throwing window corresponding to the pure audio stream are not limited.

Optionally, for the screen projection window with undefined aspect ratio and screen orientation under the special media category, a preset window layout can be adopted, wherein the preset window layout refers to a preset screen projection window with fixed aspect ratio (or fixed width and height) and screen orientation, after the media to be projected is identified as text or pure audio and other categories, the preset window layout is directly called, and then screen projection data is loaded into the window layout; the method can also be used for preferentially establishing an adaptive screen throwing window for media with limited aspect ratio and screen directions such as pure video, audio and video, images and the like, then establishing and displaying an adaptive screen throwing window in a white area in a screen throwing interface so as to display screen throwing data of text and pure audio categories, and the mode can be preferentially matched with the limited screen throwing window, and then occupy the screen display area to the greatest extent, so that a plurality of screen throwing windows are more compact, and the white area is reduced. It should be noted that, creation of the drop screen window and UI display setting under different media categories are not limited to the embodiments of the present application.

In step S01, M sets of size information may be calculated according to the screen resolution of the display device or the aspect ratio and the screen directions of the video of the screen cast by the M terminal devices, and the screen route number, where the size information includes width widthi and height widthi, i.e. a serial number for identifying size information/screen cast window i is configured, i is greater than or equal to 1 and less than or equal to M, and the i value has a mapping relationship with the device information of the terminal device. For example, m=3, the number 1 corresponds to the terminal device a, the number 2 corresponds to the terminal device B, and the number 3 corresponds to the terminal device C, when i is equal to 1, it indicates that the screen-projection window 1 created according to the 1 st group size information is used to display the screen-projection data of the terminal device a, so that it can be accurately identified which terminal device the screen-projection data from should be kept being projected by each screen-projection window.

In some examples, the sequence numbers may be assigned in a chronological order of the screen connection, such as the earlier the screen is accessed, the smaller the sequence number. For example, a pre-typesetting mode is adopted, screen-throwing pictures are typeset in advance at the moment of screen-throwing connection, and after screen-throwing data are received, the pictures are displayed according to the pre-typesetting mode.

In another example, the sequence numbers may be assigned in the order of receiving the screen-cast data, for example, typesetting is performed according to the data stream received in real time.

In the above exemplary implementation, the display device media configuration information may be video recorded at a fixed resolution, such as 1920×1080, 1280×720, and so on.

In the above-described exemplary implementation, the display device media configuration information may be video recorded according to the proportion of the screen, and the proportion of the current screen is not 16:9/9:16 of the large screen terminal, but other proportions such as 17:9, 21:16, etc. In one exemplary implementation, M projection windows are constructed according to M sets of size information, and the projection windows maintain consistent aspect ratio and display orientation with their corresponding terminal devices. Since each set of size information defines a particular width and height of the screen window, a screen window matching the size information may be constructed. After the sizes of the M screen throwing windows are all determined, a specific display position of each screen throwing window in the screen interface is also required to be planned.

In some embodiments, the location distribution information is calculated from media configuration information received from the terminal device, the location distribution information including coordinates (xi, yi) of a preset reference point Pointi on the M projection windows, the coordinates (xi, yi) of the preset reference point Pointi being used to locate the position of the projection window on the screen. Because the screen projection window is generally rectangular, the preset reference points can select certain key points in the rectangle, alternatively, the preset reference points are any one of four vertexes of the rectangle or the center point of the rectangle, and the principle is selected so as to facilitate calculation and positioning. And finally, positioning and arranging the M created screen throwing windows according to the calculated position distribution information, so that the sizes and positions of the M screen throwing windows are determined, and the layout is formed.

The following will be described in connection with specific UI examples. Fig. 20 (a) illustrates a screen projection interface in a single vertical screen scenario where the number of screen projection paths is 1 and the video stream is a single vertical screen, where the display device is only accessed to a single screen projection initiated by the terminal device a first, and when the screen resolution of the display device is 1920×1080 (i.e., the default aspect ratio of the screen of the display device is 16:9), the media configuration file sent by the terminal device a indicates that the aspect ratio is 9:16 (vertical screen) as an example, then the screen projection window 1 to be created is also a vertical screen, and the height of the screen projection window 1 is adapted to the screen height, then the width of the screen projection window 1 is 1080×9/16= 607.5, i.e., the width of the screen projection window 1 is windowheight1=607.5×1080. When determining the position of the screen-throwing window 1, a coordinate system is constructed by taking the top left corner vertex of the screen as an origin, taking the right as an x-axis forward direction and taking the vertical downward as a y-axis forward direction, and taking the top left corner vertex of the screen-throwing window 1 as a preset reference Point1, considering that a single window is better in view effect in a vertical screen centered display mode, the coordinates (x 1, y 1) = (656.25,0) of the preset reference Point1 are calculated. Since the coordinate position of the preset reference Point1 and the size of the screen-projection window 1 are determined, the position of the screen-projection window 1 in the entire screen-projection interface is uniquely determined, and the layout thus obtained is shown in fig. 20 (a).

Fig. 20 (b) illustrates a screen projection interface in a single-horizontal screen scenario where the number of screen projection paths is 1 and the video stream is a single-horizontal screen, where the display device is only accessed to a single-screen projection initiated by the terminal device a first, and when the screen resolution of the display device is 1920 x 1080 (i.e. the default aspect ratio of the display device screen is 16:9), the media configuration file sent by the terminal device a indicates that the aspect ratio is 16:9 (horizontal screen) as an example, then the screen projection window 1 to be created is also a horizontal screen, and because of the single-window display mode, and the screen projection window 1 is completely adapted to the screen in aspect ratio, the screen projection window 1 can be displayed in a full screen, that is, the window width 1=1920 x 1080, and the top left corner vertex of the screen projection window 1 is taken as the preset reference Point1, and then coordinates (x 1, y 1) = (0, 0) of the reference Point1 are preset, so that the resulting layout is as shown in fig. 20 (b).

Fig. 21 (a) and 21 (B) exemplarily show two screen-throwing interfaces in a screen-throwing path number of 2 and a double-horizontal screen scene, fig. 21 (a) is that screen-throwing windows of two horizontal screens are transversely arranged side by side, fig. 21 (B) is that screen-throwing windows of two horizontal screens are longitudinally arranged side by side, at this time, a display device is accessed to a two-way screen-throwing initiated by a terminal device a and a terminal device B, and taking a screen resolution of the display device as 1920 x 1080, and media configuration files sent by the terminal device a and the terminal device B indicate aspect ratio examples that the aspect ratio examples are 16:9 (horizontal screen), then the created screen-throwing window 1 (corresponding to the terminal device a) and the screen-throwing window 2 (corresponding to the terminal device B) are also horizontal screens.

For the example of fig. 21 (a), in order to occupy the screen area to the greatest extent, the sum of the widths of the screen window 1 and the screen window 2 is equal to the screen width, i.e. the window width1+window width 2=1920, in addition, in order to make the size of the double-screen window more uniform, window width 1=window width 2=960 may be set, then window width 1=window width 2=960×9/16=540, i.e. window width1×960×2×540 is obtained, the top left corner vertex of the screen window 1 is taken as the preset reference Point1, and in consideration of the display mode of the double-screen window in which the window is arranged in the horizontal screen 960 and longitudinally centered is better in view, thus the preset reference Point1 (reference Point 1) = 0, 270×2) is calculated, as shown in fig. 1 (fig. 270×2).

For fig. 21 (b), since the screen area is occupied to the greatest extent for the vertical juxtaposition, the sum of the heights of the screen window 1 and the screen window 2 is made equal to the screen height, i.e., windowheight1+windowheight2=1080, and furthermore, for making the size of the double-screen window more uniform, windowheight1=windowheight2=540 may be set, windowheight1=windowheight2=540×16/9=960, i.e., windowheight1=windowheight2=960×540, taking the upper left corner vertex of the screen projection window 1 as the preset reference Point1 and the upper left corner vertex of the screen projection window 2 as the preset reference Point2, the coordinates (x 1, y 1) = (480,0) of the preset reference Point1 and the coordinates (x 2, y 2) = (480, 540) of the preset reference Point2 are calculated in consideration of that the double window is better in view of the display mode of the horizontal screen juxtaposition and the horizontal centering, and the layout obtained by this is shown in fig. 21 (b).

In some embodiments, after the matching generates the layout, in the process of displaying a plurality of screen-throwing windows according to the layout on the screen-throwing interface and loading and displaying screen-throwing data of each path, there may be a newly accessed screen-throwing path at any time, or there may be at least one terminal device disconnected the screen-throwing connection, that is, the screen-throwing path number may be changed during this period, the layout needs to be updated according to the changed screen-throwing path number, so as to realize the adaptive adjustment of the layout along with the screen-throwing path number, where the adjustment refers to the number of the screen-throwing windows in the layout, the size of each screen-throwing window and the relative position distribution of each screen-throwing window.

For example, in the scenario of fig. 20 (B), the display device establishes a screen-throwing connection with the terminal device B, and the aspect ratio of the media configuration information sent by the terminal device B is indicated as 16:9 (horizontal screen), and when the controller 250 detects that the screen-throwing path number is changed from 1 to 2, the layout may be updated to the version shown in fig. 21 (a) or 21 (B); for another example, in the scenario of fig. 21 (a) or 21 (B), when the terminal device B suddenly breaks the screen connection with the display device and the controller 250 detects that the screen route number is changed from 2 to 1, the layout may be updated to the version shown in fig. 20 (B).

FIGS. 21 (c) and 21 (d) are two screen interfaces in a horizontal screen and a vertical screen scenario with a screen-throwing path number of 2 according to some embodiments, and FIG. 21 (c) shows a screen-throwing window of the horizontal screen on the left side and a screen-throwing window of the vertical screen on the right side; fig. 21 (d) shows a vertical screen window on the left side and a horizontal screen window on the right side. At this time, the display device has been accessed to the two-way screen casting initiated by the terminal device a and the terminal device B, and the screen resolution of the display device is 1920×1080, the media configuration file sent by the terminal device a indicates that the aspect ratio example is 16:9 (horizontal screen), the media configuration file sent by the terminal device B indicates that the aspect ratio example is 9:16 (vertical screen), then the created screen casting window 1 (corresponding to the terminal device a) is the horizontal screen, and the screen casting window 2 (corresponding to the terminal device B) is the vertical screen.

For fig. 21 (c), since two windows are arranged side by side one by one, in order to occupy the screen area maximally, the sum of the widths of the screen-throwing window 1 and the screen-throwing window 2 is made equal to the screen width, i.e., windowWidth1+windowwidth 2=1920, the height of the screen-throwing window 2 of the vertical screen is adapted to the screen height, i.e., windowHeight 2=1080, windowWidth 2=1080×9/16=607.5, windowWidth 1=1920-607.5 =1312.5, windowHeight 1=1312.5×9/16= 738.3, the window width1×window height 1=1312.5×738.3, window width2×window height 2=607.5×1080, the upper left corner vertex of the screen projection window 1 is taken as the preset reference Point1, the upper left corner vertex of the screen projection window 2 is taken as the preset reference Point2, and considering that the screen projection window 1 is better in view of the effect in the vertically centered display mode, the coordinates (x 1, y 1) = (0, 170.9) of the preset reference Point1, the coordinates (x 2, y 2) = (1312.5,0) of the preset reference Point2 are calculated, and the resulting layout is shown in fig. 21 (c).

Fig. 21 (d) differs from fig. 21 (c) only in that the horizontal screen and the vertical screen are distributed on different sides in the lateral direction, and thus the size of the screen-projecting window 1 and the screen-projecting window 2 are the same, and the coordinates of Point1 and Point2 are different. That is, window width1×window height 1= 1312.5*738.3,windowWidth2*window Height2 =607.5×1080, the upper left corner vertex of the screen window 1 is taken as the preset reference Point1, the upper left corner vertex of the screen window 2 is taken as the preset reference Point2, and the coordinates (x 2, y 2) = (0, 0) of the preset reference Point2 and the coordinates (x 1, y 1) = (607.5, 170.9) of the preset reference Point1 are calculated in consideration of the screen window 1 being better in view in the vertically centered display mode, whereby the resulting layout is shown in fig. 21 (d).

In some examples, for example, in the single-path vertical screen scenario in fig. 20 (a), the display device establishes a screen-throwing connection with the terminal device B again, and the aspect ratio is indicated as 16:9 (horizontal screen) in the media configuration information sent by the terminal device B, and when the controller 250 detects that the screen-throwing path number is changed from 1 to 2, the layout may be updated to a two-path one-horizontal screen one-vertical screen version similar to that shown in fig. 21 (c) or 21 (d).

In some examples, for example, in the single-path landscape screen scenario in fig. 20 (B), the display device establishes a screen-throwing connection with the terminal device B, and the aspect ratio is indicated as 9:16 (vertical screen) in the media configuration information sent by the terminal device B, and when the controller 250 detects that the screen-throwing path number is changed from 1 to 2, the layout may be updated to the two-path one-landscape screen one-vertical screen version shown in fig. 21 (c) or 21 (d).

In some examples, for example, in the scenario of fig. 21 (c) or 21 (d), when the terminal device B suddenly breaks the screen-throwing connection with the display device, and the controller 250 detects that the screen-throwing path number is changed from 2 to 1, the layout may be updated to a version of the single-cross-screen window (screen-throwing window 1) full-screen exclusive screen shown in fig. 20 (B).

In some examples, for example, in the scenario of fig. 21 (c) or 21 (d), when the terminal device a suddenly breaks the screen-casting connection with the display device, and the controller 250 detects that the screen-casting path number is changed from 2 to 1, the layout may be updated to a version of the exclusive screen similar to the single vertical screen window (screen-casting window 2) shown in fig. 20 (a).

Fig. 21 (e) illustrates a screen projection interface in a dual vertical screen scenario with a screen projection path number of 2, and in fig. 21 (e), two vertical screen projection windows are arranged side by side along a transverse direction, and in this case, if the screen resolution of the display device is 1920 x 1080, the aspect ratio of the media configuration file indications sent by the terminal device a and the terminal device B are both 9:16 (vertical screen), then the created screen projection window 1 (corresponding to the terminal device a) and the created screen projection window 2 (corresponding to the terminal device B) are both vertical screens.

With fig. 21 (e), since the screen-projecting window 1 and the screen-projecting window 2 are both vertical screens, the heights of both screen-projecting windows are adapted to the screen height, that is, windowheight1=windowheight2=1080, whereby windowWidth 1=windowwidth 2=1080×9/16=607.5, windowWidth1×windowwidth 1=windowwidth 2×windowheight2=607.5×1080 can be calculated. Taking the top left corner vertex of the screen projection window 1 as a preset reference Point1 and the top left corner vertex of the screen projection window 2 as a preset reference Point2, considering that the viewing effect is better when the two vertical screen windows are uniformly arranged along the transverse equidistant direction, calculating the distance value dx to be (1920-2 x 607.5)/3=235, and then, the coordinates (x 1, y 1) = (235,0) of the preset reference Point1 and the coordinates (x 2, y 2) = (2 x dx+607.5, 0) = (1077.5,0) of the preset reference Point2, so that the obtained layout is shown in fig. 21 (e).

In some examples, for example, in the single-channel vertical screen scenario in fig. 20 (a), the display device establishes a screen-throwing connection with the terminal device B again, and the aspect ratio is indicated as 9:16 (vertical screen) in the media configuration information sent by the terminal device B, and when the controller 250 detects that the screen-throwing number is changed from 1 to 2, the layout may be updated to the version of the two-channel double vertical screen shown in fig. 21 (e).

In some examples, for example, in the two-way two-vertical screen scenario of 21 (e), where terminal device a or terminal device B disconnects the screen-casting, and controller 250 detects that the screen-casting path number is changed from 2 to 1, the layout may be updated to a version similar to the single-vertical screen window exclusive screen shown in fig. 20 (a).

In some examples, in any of the scenarios in fig. 20 (a) to 20 (b) and fig. 21 (a) to 21 (e), if all the screen-throwing connections are disconnected, that is, the controller detects that the screen-throwing path number becomes 0, the screen-throwing interface is exited, and the UI display interface is restored to the front of screen throwing.

The construction and planning processes of the layout are described in detail by taking the single-path and double-path screen projection as an example, when the screen projection number M is greater than or equal to 3, the matching mechanism of the layout can adaptively refer to the example, and the matching of the layout can have the characteristics of self-adaptability, flexibility, diversity and the like, so that the application is not listed and repeated one by one. When the number of the screen throwing paths, the screen directions and the aspect ratio of each path of screen throwing media are the same, the layout is diversified due to different typesetting forms among the screen throwing windows, for example, the two paths of double-transverse-screen scenes at least comprise two versions of fig. 21 (a) and 21 (b), and the two paths of single-transverse-screen and single-vertical-screen scenes at least comprise two versions of fig. 21 (c) and 21 (d).

In some embodiments, when matching to multiple versions of a layout, the present application performs optimal screening based on the principle of maximum area, i.e., calculating the sum of areas of all of the projection windows in each version of the layout, screening the target layout with the largest sum of areas, i.e., smax=max { k=1, 2 …, q|sk }, where k is used to represent the version number of the layout, Q is the number of versions of the matched layout, 1.ltoreq.k.ltoreq.q,in the formula, si is the area of the ith screen projection window in the layout of the kth version, and M is the total number of screen projection windows in the layout of all versions. The maximum area sum Smax is calculated in the mode, then the version corresponding to Smax is used as the screened target layout, and the target layout is displayed on the screen projection interface, so that each screen projection window occupies the screen more fully and maximally, the typesetting among the screen projection windows is compact, the white area is reduced as much as possible, and the display effect of the screen projection interface is improved. Since the screen-throwing window is rectangular, the screen-throwing window is independentThe area of one projection window is equal to the product of its width and height.

As an example, in a two-way dual-horizontal screen scenario, the screen resolution of the display device is 1920×1080, the aspect ratio of the media configuration file indication sent by the terminal device a is 17:9 (horizontal screen), and the aspect ratio of the media configuration file indication sent by the terminal device B is 16:9 (horizontal screen), that is, the two-way screen media have different aspect ratio, two screen projection windows are constructed, where the screen projection window 1 corresponds to the terminal device a, and the screen projection window 2 corresponds to the terminal device B.

As shown in fig. 22 (a), version 1 of the layout is shown, since the layout is arranged side by side in the lateral direction, in order to occupy the screen area maximally, the sum of the widths of the screen window 1 and the screen window 2 is equal to the screen width, i.e. the size of the double screen window is more balanced, and in order to make the size of the double screen window uniform, the size of the window 1=window 2=960 is set, the window height 1=960×9/17=508, the window height 2=960×9/16=540, the window height 1=960×508, and the window height 2=960×540 are set, and then for version 1, the sum of the areas of the screen windows s1=960×508+960×540= 1006080.

Fig. 22 (b) shows a layout version 2, in which, for the longitudinal side-by-side arrangement, in order to occupy the screen area maximally, the sum of the heights of the screen window 1 and the screen window 2 is equal to the screen height, i.e. the size of the double screen window is more balanced, and in addition, for the size of the double screen window, the sum of the areas of the screen window s2=1020+540+540×17/9=1020, and the screen window 1×6/9=960 is obtained, and for the version 2, the sum of the areas of the screen window s2=1020+540×540+ 1069200. As is evident from the comparison, S2 is greater than S1, so version 2 is taken as the target layout for the final display.

Fig. 22 (c) illustrates a screen projection interface in a dual vertical screen scenario with a screen projection path number of 2, where in fig. 22 (c), two vertical screen projection windows are arranged side by side along a transverse direction, and the resolutions of two screen recordings of terminal equipment are different, for example, the resolution of the screen recording of terminal equipment a is 1920×1080, the resolution of the screen recording of terminal equipment B is 2040×1080, and the media configuration files sent by the terminal equipment a and the terminal equipment B indicate aspect ratios of 9:16 (vertical screen) and 9:17 (vertical screen), respectively, and then the created screen projection window 1 (corresponding to the terminal equipment a) and the screen projection window 2 (corresponding to the terminal equipment B) are vertical screens, but the transverse display sizes of the two screen projection windows are different.

For fig. 22 (c), since the screen projection window 1 and the screen projection window 2 are vertical screens, the heights of the two screen projection windows are adapted to the screen height, that is, windowheight1=windowheight2=1080, so that windowWidth 1=1080×9/16=607.5, windowWidth 2=1080×9/17= 571.7, the top left corner vertex of the screen projection window 1 is taken as a preset reference Point1, the top left corner vertex of the screen projection window 2 is taken as a preset reference Point2, and considering that the viewing effect is better when the two vertical screen windows are uniformly arranged along the transverse direction, the distance value dx is (1920-607.5-571.7)/3=247, the coordinates (x 1, y 1) = (247,0) of the preset reference Point1, and the coordinates (x 2, y 2) = (2×dx+5, 0) = (1101.5,0) of the preset reference Point2 are calculated, so that the layout is obtained as shown in fig. 22 (c).

Fig. 22 (d) illustrates another screen projection interface in a dual vertical screen scene with a screen projection path number of 2, where in fig. 22 (d), two screen projection windows of the vertical screens are arranged side by side in the lateral direction, the resolution of the screen recording of the terminal device a is 2040×1080, the resolution of the screen recording of the terminal device B is 1920×1080, and according to the calculation mode illustrated in fig. 22 (c), the layout illustrated in fig. 22 (d) is obtained.

In some embodiments, if there are Q versions of the layout, and each version has the same sum of the areas of the drop windows, i.e., s1=s2= … =sq, then any one version may be randomly selected as the target layout.

In some embodiments, the screen shots in the layout may or may not be ordered by the time of access to the screen. Taking three paths of screen casting as an example, assuming that a display device is sequentially connected with a terminal device A, a terminal device C and a terminal device B in a screen casting manner, media configuration information sent by the terminal device A indicates that the aspect ratio of the media configuration information is 16:9 (horizontal screen), media configuration information sent by the terminal device C indicates that the aspect ratio of the media configuration information is 9:16 (vertical screen), and media configuration information sent by the terminal device B indicates that the aspect ratio of the media configuration information is 9:16 (vertical screen), namely a scene corresponding to two horizontal screens and two vertical screens of three paths of screen casting. Three versions of the layout can be matched in this scenario, with version 1 being illustrated in fig. 23 (a), version 2 being illustrated in fig. 23 (b), and version 3 being illustrated in fig. 23 (c).

Wherein, version 1 of the example of fig. 23 (a) supports arranging a screen-throwing window 1 (corresponding to terminal equipment a), a screen-throwing window 2 (corresponding to terminal equipment C) and a screen-throwing window 3 (corresponding to terminal equipment B) in sequence along the transverse direction according to the sequence of accessing the screen, and presents a typesetting of transverse screen-vertical screen, and sets a unique transverse screen window as a first display; FIG. 23 (b) then presents a portrait-landscape-portrait layout, i.e., the only landscape window is set to be a landscape-centered presentation, with the dual portrait windows being symmetrically split on either side of the landscape window; fig. 23 (c) then presents a portrait-landscape layout, i.e., the only landscape window is set to the last display. The versions of the examples of fig. 23 (b) and 23 (c) do not support ordering in the chronological order of access to the screen.

Because the layouts of the three versions in the scene all conform to the principle of the maximum area, any one version can be output as the target layout, and in practical application, the visual viewing effect can be referred to, and the layout of a certain version is selected alternatively, for example, the layout of version 2 shown in fig. 23 (b) has the best symmetry, so that in the scene of three paths of one horizontal screen and two vertical screens, the screen projection display can be preferably performed according to the layout of version 2.

In some embodiments, referring to fig. 24, an interaction flow between a terminal device and a display device is further provided in the embodiments of the present application, including:

step S201, terminal equipment end: and when receiving the screen projection operation of the user on the media, sending a screen projection request to the display equipment. The screen projection request at least comprises equipment information of terminal equipment;

step S202, the display device side: when receiving a screen projection request, establishing screen projection connection with terminal equipment, and feeding back authorized information to the terminal equipment;

step S203, the display device side: accumulating the number of the screen throwing paths by one;

step S204, the terminal device side: when authorized information is received, sending screen throwing data and media configuration information to a display device;

in some embodiments, after receiving the screen-throwing request, the display device may query whether the terminal device has the screen-throwing authority. If the terminal equipment has the screen throwing authority, the display equipment feeds back the authorized information to the terminal equipment; if the terminal equipment does not have the screen-throwing authority, the display equipment feeds back unauthorized information to the terminal equipment, so that after the terminal equipment receives the unauthorized information, the screen-throwing data and the media configuration information are not sent. The method of inquiring the screen-throwing authority of the terminal device by the display device is not limited to inquiring whether the user authorizes the terminal device or not through a UI popup window, or inquiring a trusted device list maintained by the display device.

Step S205, the display device side: and receiving screen throwing data and media configuration information sent by the terminal equipment, and inquiring the screen throwing number M of the current counting. M is greater than or equal to 1;

step S206, the display device side: and acquiring media configuration data correspondingly transmitted by M terminal devices which are accessed by the screen at present. The media configuration data at least comprises the aspect ratio of the media video, and the screen direction information can be acquired through the aspect ratio;

step S207, the display device side: calculating M groups of size information according to the screen resolution or the display screen directions and aspect ratio of M terminal equipment screen-throwing media and the screen-throwing path number M;

step S208, the display device side: and constructing M screen throwing windows according to the M groups of size information. The screen throwing window is consistent with the media displayed by the screen throwing window in terms of the screen direction and the aspect ratio;

step S209, the display device side: position distribution information is calculated based on the screen resolution and M sets of size information. The position distribution information comprises coordinates of preset reference points on the M screen throwing windows, and the coordinates of the preset reference points are used for positioning the positions of the screen throwing windows on a screen;

step S210, the display device side: and positioning and arranging M screen throwing windows according to the position distribution information to obtain a layout. Optionally, if a layout of multiple versions is obtained, screening can be performed on each version based on the principle of maximum area, and a target layout with the largest sum of areas of the screen-throwing windows is output;

Step S211, the display device side: and displaying the layout in a screen-throwing interface, and displaying screen-throwing data sent by the corresponding terminal equipment in each screen-throwing window in the layout. Optionally, the screen throwing window has a one-to-one correspondence with the device ID of the terminal device, so that the display device accurately throws the screen throwing data into the appointed screen throwing window for display;

step S212, the display device side: monitoring the screen throwing path number in real time in the process of displaying the screen throwing interface;

step S213, the display device side: when the screen-throwing path number is detected to be changed, updating the layout;

step S214, the display device side: and when the screen-throwing path number is detected to be reduced to zero, the screen-throwing interface is exited.

According to the method and the device, based on the screen throwing path number and the aspect ratio of the multipath screen throwing media and configuration information such as screen directions, dynamic self-adaptive layout matching is carried out on the screen throwing interface, so that typesetting and layout of M screen throwing windows are more adaptive to a screen, the multipath screen throwing windows are displayed in the largest range in the screen, the distribution of the screen throwing windows is more compact, and the white left of the screen throwing interface is reduced. In addition, the attractiveness of the display of the screen-throwing interface can be considered, for example, the uniformity, symmetry and relative balance of the size among the screen-throwing windows can be considered, the size and the position distribution of each screen-throwing window can be set, the display effect of the screen-throwing interface can be remarkably improved through the obtained layout, and the user experience of watching the screen is further improved.

FIG. 25 illustrates an icon control interface display diagram for an application in the display device 200, according to some embodiments. In some embodiments, the display device may directly enter the preset vod program interface after being started, where the vod program interface may include at least a navigation bar 510 and a content display area located below the navigation bar 510, as shown in fig. 25, where the content displayed in the content display area may change with the change of the selected control in the navigation bar. The program in the application program layer can be integrated in the video-on-demand program and displayed through one control of the navigation bar, and can be further displayed after the application control in the navigation bar is selected.

In some embodiments, the display device may directly enter the display interface of the signal source selected last time after being started, or the signal source selection interface, where the signal source may be a preset video on demand program, or may be at least one of an HDMI interface, a live tv interface, etc., and after the user selects a different signal source, the display may display the content obtained from the different signal source.

With the development of mobile internet technology, mobile terminals such as mobile phones and tablet computers have increasingly abundant functions, and more users use the mobile terminals to watch videos, browse files and the like. However, since the screen size of mobile terminals such as mobile phones and tablet computers is small, and the screen size of display devices such as televisions and desktop monitors is large, many users still prefer to watch videos and browse files on such display devices.

Taking a television as an example, when a user wants to watch a video played in a mobile terminal on the television, a screen-throwing function of the mobile terminal can be used to establish a screen-throwing connection with the television, so that the video can be synchronously watched on the mobile terminal and the television. In some embodiments, when the screen is projected, firstly, the screen projection application in the mobile terminal is started, after the screen projection application in the mobile terminal is started, the display equipment capable of projecting the screen is automatically searched in the local area network to which the mobile terminal is currently connected, and then the equipment names of the display equipment capable of projecting the screen are displayed in a list mode on the display interface of the mobile terminal. According to the actual screen projection requirement, after the user selects the equipment name of the display equipment which needs to be projected on the screen from the display interface of the mobile terminal, the mobile terminal can establish screen projection connection with the display equipment which is clicked and selected by the user.

After the mobile terminal is connected with the display device, the mobile terminal can send the data displayed on the mobile terminal to the display device, so that the video in the mobile terminal can be watched on the display device. However, the screen projection method in the related art cannot simultaneously perform screen projection and play on the content transmitted through a plurality of screen projection protocols on one display device.

In order to solve the technical problem, a display device provided in an embodiment of the present application includes: a display and a controller, wherein the display is configured to display a user interface. The controller can run a playing application and at least one screen-throwing protocol application, the at least one screen-throwing protocol application can be connected to the playing application, and each screen-throwing protocol application can independently perform data communication with the playing application.

Each of the screen-projection protocol applications adopts a different screen-projection protocol, and as an example, the existing screen-projection protocols include: DLNA screen-throwing protocol, miracast screen-throwing protocol, airplay screen-throwing protocol and other screen-throwing protocols. Based on two types of contents to be screen-thrown by the mobile terminal (one type is media data of a third-party media resource on the mobile terminal, and the other type is a screen real-time picture of the mobile terminal), the DLNA screen-throwing protocol can support the pushing screen-throwing of the media data of the third-party media resource on the mobile terminal; the Miracast screen-throwing protocol can support mirror image screen throwing of a screen real-time picture of the mobile terminal; the Airplay screen-throwing protocol can support the pushing screen-throwing of the media data of the third-party media resource on the mobile terminal and can also support the mirror image screen-throwing of the screen real-time picture of the mobile terminal; other screen-drop protocols may be self-developed protocols by the respective display device manufacturers.

In some embodiments, one screen-casting protocol application may be developed independently in the controller for each existing screen-casting protocol. For example, referring to fig. 26, a DLNA screen-projection protocol application is developed for the DLNA screen-projection protocol; aiming at the Miracast screen-throwing protocol, developing a Miracast screen-throwing protocol application; aiming at an Airplay screen-throwing protocol, developing an Airplay screen-throwing protocol application; for other screen-casting protocols, corresponding other screen-casting protocol applications are developed.

Each screen-throwing protocol application exists in a background service form, and can independently communicate data with the playing application. The playing application is an independent player application, is not responsible for protocol interaction with the mobile terminal, and is only responsible for receiving and playing media data of various screen-throwing protocols.

Referring to the workflow diagram shown in fig. 27, in order to solve the problem that the screen projection method in the related art cannot simultaneously perform screen projection playing on display data on a plurality of mobile terminals on one display device, the controller in this embodiment is configured to perform the following steps:

step 701, when a first screen-throwing request sent by a first mobile terminal is received, setting up screen-throwing connection with the first mobile terminal through a first screen-throwing protocol application corresponding to the first screen-throwing request, and sending first screen-throwing content transmitted by the first mobile terminal through the screen-throwing connection to a playing application;

Step 702, creating, by the playing application, a first playing window on the user interface, where the first playing window is used to play the first content to be screened.

The first mobile terminal may be the only mobile terminal connected to the display device in a screen-projection manner, or may be any one of a plurality of mobile terminals connected to the display device in a screen-projection manner. The first screen request may be a screen request sent by the first mobile terminal to the display device through any screen protocol. The first screen-throwing protocol application is a screen-throwing protocol application which adopts the same screen-throwing protocol with the first screen-throwing request, and the first playing window can be the only one playing window created by the playing application or any one of a plurality of playing windows created by the playing application.

When the first mobile terminal sends a first screen projection request to the display device, the controller receives the screen projection request. Responding and processing the screen throwing request through a first screen throwing protocol application, establishing screen throwing connection, and when receiving first screen throwing content transmitted by the first mobile terminal through the screen throwing connection, creating a playing window to play through the first screen throwing protocol application, sending the first screen throwing content to the playing application through the first screen throwing protocol application, and creating a playing window (namely a first playing window) to play through the playing application. When a screen throwing request sent by another screen throwing protocol exists, the corresponding screen throwing protocol application responds to the screen throwing request, and the playing application still continues to run at the moment, so that the first playing window is not influenced, and the first content to be thrown is continuously played.

The screen-throwing protocol application is only responsible for protocol interaction with the mobile terminal, is not responsible for playing the screen-throwing content, and is only responsible for creating a corresponding playing window to play the received screen-throwing content, and is not responsible for protocol interaction with the mobile terminal. When a plurality of mobile terminals send a plurality of screen projection requests to a display device through a plurality of screen projection protocols, a controller can respectively and independently interact with the plurality of mobile terminals through the plurality of screen projection protocols, screen projection connection is established, then the content to be screen projected is forwarded to a playing application, a playing window is created for playing as long as the playing application receives one content to be screen projected, and if a plurality of content to be screen projected is received simultaneously or successively, a plurality of independent playing windows are correspondingly created for playing, so that the content to be screen projected, which is sent through the plurality of screen projection protocols, can be played simultaneously on the same display device.

In connection with fig. 28, it is further explained how content transmitted through a plurality of screen-casting protocols is played simultaneously on one display device. When the display device is playing the screen content of the first mobile terminal, if a screen-throwing request is received, in order to prevent the playing of the current screen-throwing content from being terminated, the controller is further configured to execute the following steps:

Step 801, when a second screen-throwing request sent by a second mobile terminal is received, a screen-throwing connection between the second mobile terminal and the second mobile terminal is established through a second screen-throwing protocol application corresponding to the second screen-throwing request, and second screen-throwing content to be transmitted by the second mobile terminal through the screen-throwing connection is sent to a playing application;

it should be noted that, the second mobile terminal and the first mobile terminal may be the same mobile terminal, or may be two different mobile terminals, the second screen-throwing protocol application is a screen-throwing protocol application adopting the same screen-throwing protocol as the second screen-throwing request, and the first screen-throwing request and the second screen-throwing request are two different screen-throwing requests sent by the user based on two different contents to be screened, where the two screen-throwing requests may be sent by adopting the same screen-throwing protocol, or may be sent by adopting different screen-throwing protocols. If the two screen-throwing requests are sent by adopting the same screen-throwing protocol, the first screen-throwing protocol application and the second screen-throwing protocol application are the same screen-throwing protocol application, and if the two screen-throwing requests are sent by adopting different screen-throwing protocols, the first screen-throwing protocol application and the second screen-throwing protocol application are two different screen-throwing protocol applications.

Step 802, creating, by the playing application, a second playing window on the user interface, where the second playing window is used to play the second content to be screened.

Referring to fig. 29, the second playing window and the first playing window are mutually independent playing windows, and can exist on the display at the same time, and the two playing windows may be partially overlapped or have no overlap.

In some embodiments, the playback application may adaptively set the size and position of the first playback window and the second playback window such that both playback windows are completely displayed on the display. Or, the playing application may adjust the sizes and positions of the first playing window and the second playing window according to the preference setting of the user.

Based on the above procedure, those skilled in the art can clearly know how to play the multiple contents to be displayed simultaneously when the display device receives more screen display requests, which is not described herein.

The content to be screened of the mobile terminal is generally divided into two types: one is media data of a third party media resource on the mobile terminal, and the other is a screen real-time picture of the mobile terminal.

Based on these two types, the screen-casting protocol application in the controller needs to forward the content to be screened to the playing application by different methods, which will be described in detail below. For ease of understanding, the following will be described only with respect to the first mobile terminal, but it is needless to say that the same principle can be used for any mobile terminal that issues a screen-cast request.

In some embodiments, before sending the first to-be-screened content transmitted by the first mobile terminal through the screening connection to the playing application, the controller is further configured to:

acquiring the first to-be-screen content through the first screen-projection protocol application; and if the first to-be-screened content belongs to the media data of the third-party media resource, the first to-be-screened content is a target webpage link address, and if the first to-be-screened content belongs to a screen real-time picture, the first to-be-screened content is a screen video stream of the first mobile terminal.

When a first screen-throwing request is received, the first screen-throwing protocol application responds, and carries out protocol command negotiation with the screen-throwing application which sends the first screen-throwing request, and after the negotiation is completed, the first screen-throwing protocol application acquires first screen-throwing content sent by the first mobile terminal.

For the media data of the third party media resource, the first mobile terminal sends a web page link address of the third party media resource to the first screen-throwing protocol application, where the web page link address is a URL (Uniform Resource Locator ) address corresponding to the first to-be-thrown content, and this embodiment is called a target web page link address, and is usually an http type of source address. For the on-screen real-time picture, the first mobile terminal sends a video stream of the on-screen real-time picture to the first screen throwing protocol application.

In some embodiments, if the first to-be-screen content received by the first screen-projection protocol application is the target web page link address, the target web page link address may be sent to the playing application by a broadcasting method of Android or a communication method between other processes.

After receiving the target webpage link address, the playing application creates a first playing window, connects with the target link address, and plays the video content acquired from the target link address in the first playing window.

As an example, if a screen projection application in a first mobile terminal sends a first screen projection request to a display device through a DLNA screen projection protocol, the DLNA screen projection protocol application in a display device controller responds to the first screen projection protocol application, and performs a protocol command negotiation with the screen projection application, after the negotiation is completed, a screen projection connection is established, if the screen projection content to be projected, which is sent by the screen projection application through the screen projection connection, is a URL address of a certain film source in a certain third party video software, the DLNA screen projection protocol application notifies the received URL address to a playing application through an android broadcasting or other inter-process communication mode, and after the playing application receives the URL address, a playing window (or a playing control) is created to play the film source corresponding to the URL address.

In some embodiments, if the first to-be-screen content received by the first to-be-screen protocol application is a screen video stream of the first mobile terminal, the first to-be-screen content may be sent to the playing application in a manner of sharing the memory. Specifically, the controller is configured to:

and creating a first shared memory through the first screen-throwing protocol application, wherein the first shared memory has a unique index name.

And storing the screen video stream of the first mobile terminal into the first shared memory in real time through the first screen projection protocol application, and sending a first reading instruction to the playing application, wherein the first reading instruction comprises an index name of the first shared memory.

When a first reading instruction is received, based on the unique index name, the playing application can read the screen video stream from the first shared memory in real time, and play the screen video stream read from the first shared memory in real time in the first playing window.

As an example, if the screen-throwing application in the first mobile terminal sends a first screen-throwing request to the display device through the Miracast screen-throwing protocol, the Miracast screen-throwing protocol application in the display device controller responds to the first screen-throwing protocol application, and performs command negotiation with the screen-throwing application, after the negotiation is completed, the Miracast screen-throwing protocol application creates a shared memory, namely a first shared memory, names the first shared memory, then notifies the name of the first shared memory to the playing application through broadcasting of android or other inter-process communication modes, meanwhile, the Miracast screen-throwing protocol application receives a screen video stream transmitted in real time from the first mobile terminal through other data transmission protocols, writes the screen video stream into the first shared memory, creates a playing window, and reads the real-time screen video stream from the first shared memory to play.

In some embodiments, if the first to-be-screen content received by the first to-be-screen protocol application is a screen video stream of the first mobile terminal, the first to-be-screen content may be sent to the playing application through other inter-process communication modes besides the mode of sharing the memory. The inter-process communication mode can be a communication mode such as a named pipe or a socket.

When the screen throwing application in the first mobile terminal stops the screen throwing operation, the first screen throwing protocol application in the display equipment controller is informed through the screen throwing protocol, and the first screen throwing protocol application informs the playing application to stop playing.

In some embodiments, if the first to-be-screen content is a real-time screen of the first mobile terminal, the first to-be-screen protocol application may delete the created first shared memory after stopping playing.

The screen projection method provided by the application is described below with reference to specific examples. Assuming that four mobile terminals send a screen throwing request to a display device, assuming that a first mobile terminal pushes a certain piece source of video software A to the display device through a DLNA screen throwing protocol, a second mobile terminal pushes a certain piece source of video software B to the display device through the DLNA screen throwing protocol, a third mobile terminal pushes a screen picture of the mobile terminal to the display device through a miracast screen throwing protocol, and a fourth mobile terminal pushes a screen picture of the mobile terminal to the display device through an airlay screen throwing protocol. Protocol applications in the display device controller: the DLNA protocol application, the miracast protocol application and the air protocol application respectively respond to the four screen-throwing requests. As shown in fig. 30, the interaction procedure between the four mobile terminals and the display device is as follows:

(1) The first mobile terminal pushes a certain film source of the video software A to the display equipment through a DLNA (digital living network alliance) screen-throwing protocol, the DLNA screen-throwing protocol application receives a screen-throwing request, and interacts with the first mobile terminal in a standard DLNA protocol, after the interaction is completed, the URL address of the certain film source of the video software A is received, the DLNA screen-throwing protocol application starts a playing application, the URL address is transmitted to the playing application in a process communication mode, and the playing application creates a playing window layer 1 (the representation form on a display user interface is a playing window) and is connected with the URL address to play the video.

(2) The second mobile terminal pushes a certain film source of the video software B to the display equipment through a DLNA (digital living network alliance) screen-throwing protocol, the DLNA screen-throwing protocol application receives a screen-throwing request, and interacts with the second mobile terminal through a standard DLNA protocol, after the interaction is completed, the DLNA screen-throwing protocol application receives a URL (uniform resource locator) address of the certain film source of the video software B, starts a playing application, transmits the URL address to the playing application in a process communication mode, and the playing application creates a playing window player 2 (the representation form on a display user interface is a second playing window) and is connected with the URL address to play the video.

(2) The third mobile terminal pushes a screen picture of the third mobile terminal to the display equipment through a miracast screen-throwing protocol, the miracast screen-throwing protocol application receives the screen-throwing request, command negotiation of miracast connection is carried out, transmission of screen picture data is carried out after the negotiation is completed, the miracast screen-throwing protocol application creates a shared memory which is named as miracast_sharemry 1, meanwhile, a playing application is informed of reading a video stream from the shared memory miracast_sharemry 1 in real time, and meanwhile, the playing application creates a playing window layer3 (the expression form on the display user interface is a third playing window) at a proper position of the display interface to play the video stream.

(4) The fourth mobile terminal pushes a screen picture of the fourth mobile terminal to the display equipment through an air-lay screen-throwing protocol, the air-lay screen-throwing protocol application receives the screen-throwing request, command negotiation of air-lay connection is carried out, transmission of screen picture data is carried out after the negotiation is completed, the air-lay screen-throwing protocol application creates a shared memory which is named as air-lay-sharemory 1, meanwhile, the playing application is informed to read a video stream from the shared memory air-sharemory 1 in real time, and meanwhile, the playing application creates a playing window layer4 (the expression form on the display user interface is a fourth playing window) at a proper position of the display interface to play the video stream.

According to the display device provided by the embodiment of the application, one play application is independently developed in the controller, when a plurality of mobile terminals send a plurality of screen projection requests to the display device through a plurality of screen projection protocols, the controller can independently interact with the plurality of mobile terminals through the plurality of screen projection protocol applications, screen projection connection is established, then the content to be screened is forwarded to the play application, the play application only needs to receive one content to be screened, a play window is created for playing, and if a plurality of content to be screened are received simultaneously or successively, a plurality of independent play windows are correspondingly created for playing, so that the content to be screened sent through the plurality of screen projection protocols can be played on the same display device at the same time.

In some embodiments, the present application discloses a screen projection method, fig. 31 is a multi-terminal interaction flow chart between a mobile terminal and a display device, and in combination with fig. 31, the screen projection method includes:

step 3101, a first mobile terminal sends a first screen-throwing request;

step 3102, a first screen-throwing protocol application responds to a first screen-throwing request and establishes screen-throwing connection with the first mobile terminal, wherein the first screen-throwing protocol application is a protocol application adopting the same screen-throwing protocol as the first screen-throwing request;

Step 3103, the first mobile terminal sends a first content to be projected to the first projection protocol application through the projection connection;

step 3104, the first screen-casting protocol application sends the first content to be screen-cast to a playing application;

at step 3105, the playing application creates a first playing window on the user interface displayed on the display, where the first playing window is used to play the first content to be projected.

According to the screen projection method, when the first mobile terminal sends the first screen projection request to the display device, the controller receives the screen projection request. Responding and processing the screen throwing request through a first screen throwing protocol application, establishing screen throwing connection, and when receiving first screen throwing content transmitted by the first mobile terminal through the screen throwing connection, creating a playing window through the first screen throwing protocol application to play, sending the first screen throwing content to the playing application through the first screen throwing protocol application, and creating a playing window (namely a first playing window) by the playing application to play. When a screen throwing request sent by another screen throwing protocol exists, the corresponding screen throwing protocol application responds to the screen throwing request, and the playing application still continues to run at the moment, so that the first playing window is not influenced, and the first content to be thrown is continuously played. Therefore, the method and the device can realize simultaneous playing of the to-be-screen content sent by a plurality of screen-throwing protocols on the display equipment.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.

Claims

A display device, comprising:

a display configured to display a screen-casting interface;

a communicator configured to establish a screen-drop connection with a communication terminal;

a controller configured to:

detecting the number of connections of the communication terminal;

planning a layout mode of the screen throwing picture according to the connection quantity so as to obtain display parameters of the screen throwing picture;

sending a notification message to a communication terminal corresponding to the screen projection picture, wherein the notification message comprises the display parameters, and the notification message is used for enabling the communication terminal to generate screen projection data according to the display parameters;

and receiving the screen projection data fed back by the communication terminal, and displaying the screen projection data according to the layout mode.
The display device of claim 1, the controller further configured to:

acquiring a connection instruction input by a user and used for establishing screen connection;

responding to the connection instruction, and initiating screen-throwing connection to the communication terminal;

and traversing the connection quantity of the communication terminal after the screen projection connection is completed.
The display device of claim 2, the controller further configured to:

establishing a screen-throwing data channel;

and after the screen-throwing data channel is successfully established, creating a display control, wherein the display control is used for displaying the screen-throwing data received through the screen-throwing data channel.
The display device of claim 1, the controller further configured to:

acquiring an exit instruction which is input by a user and used for breaking the screen connection;

responding to the exit instruction, and disconnecting the screen connection corresponding to the communication terminal;

traversing the number of connections of the remaining communication terminals.
The display device of claim 1, the controller further configured to:

detecting screen size information of the display;

matching a layout template in a database by taking the connection number and the screen size information as indexes;

And extracting the display parameters from the layout template.
The display device of claim 5, the controller further configured to:

if the layout template is not matched in the database, acquiring the equipment parameters of the communication terminal;

dividing subareas with the same number as the connection number in a display area according to the equipment parameters;

traversing the graphics parameters of each sub-region to generate the display parameters.
The display device of claim 1, the controller further configured to:

extracting the display parameters, wherein the display parameters comprise a display direction, a code rate and a resolution of each display window, and the display direction is used for indicating the communication terminal to generate screen projection data according to a set direction;

and generating a notification message according to the display parameters, wherein the notification message comprises the display direction, the code rate and the resolution.
The display device of claim 1, the controller further configured to:

detecting an effective picture range of each screen-throwing picture in a screen-throwing interface;

calculating the display proportion of the effective picture range to the total range of the display window;

and if the display proportion is smaller than or equal to the proportion threshold value, sending a prompt message to the communication terminal, wherein the prompt message is used for prompting the rotation direction of the communication terminal.
A communication terminal, comprising:

a display unit configured to display a terminal interface;

a communication unit configured to establish a screen-casting connection with the display device;

a processor configured to:

receiving a notification message sent by the display equipment, wherein the notification message comprises display parameters of a screen projection picture, and the display parameters are obtained when the display equipment plans a screen projection picture layout mode according to the current connection quantity of the communication terminals;

generating screen throwing data according to the display parameters;

and feeding back the screen throwing data to the display equipment.
A method for dynamically displaying a screen projection picture comprises the following steps:

the display equipment detects the connection quantity of the communication terminal, and plans the layout mode of the screen-throwing picture according to the connection quantity so as to obtain the display parameter of the screen-throwing picture;

the display equipment sends a notification message to a communication terminal corresponding to the screen projection picture, wherein the notification message comprises the display parameters;

the communication terminal receives the notification message sent by the display device and generates screen projection data according to the display parameters;

the communication terminal feeds back the screen throwing data to the display equipment;

and the display equipment displays the screen throwing data according to the layout mode.