CN113613072B

CN113613072B - Multi-channel screen-throwing display method and display equipment

Info

Publication number: CN113613072B
Application number: CN202110881335.9A
Authority: CN
Inventors: 马晓燕; 宋子全; 刘美玉; 张娜
Original assignee: Hisense Visual Technology Co Ltd
Current assignee: Hisense Visual Technology Co Ltd
Priority date: 2021-08-02
Filing date: 2021-08-02
Publication date: 2023-05-05
Anticipated expiration: 2041-08-02
Also published as: CN113613072A

Abstract

The invention discloses a multi-channel screen-throwing display method and a display device, which belong to the technical field of display devices, wherein the display device matches and adapts to the layout among M screen-throwing windows of a screen according to the current counted M screen-throwing paths and media configuration information respectively received from M terminal devices; the media configuration information comprises aspect ratio of video projected by the terminal equipment; and displaying M screen throwing windows in the screen throwing interface according to the layout, and displaying the screen throwing data sent by the corresponding terminal equipment in each screen throwing window. The method and the device realize dynamic self-adaptive layout matching of the screen-throwing interfaces based on the screen-throwing path number, enable typesetting and layout of M screen-throwing windows to be more adaptive to the screen, enable the multi-path screen-throwing windows to be displayed in the largest range in the screen, and improve the display effect of the screen-throwing interfaces.

Description

Multi-channel screen-throwing display method and display equipment

Technical Field

The invention relates to the field of display equipment, in particular to a multi-channel screen-projection display method and display equipment.

Background

In some application scenarios, a user may initiate screen projection to a display device through a terminal device such as a smart phone, a tablet computer, etc., so as to project a video currently played by the terminal device to a large-screen display device. The display device may be simultaneously multi-channel screen-cast, such as a user playing a screen-cast game while watching a video illustration of the screen-cast, and such as multiple users playing a game and interacting together through the multi-channel screen-cast. Aiming at a multi-path screen-throwing scene, how to consider the video configuration characteristics of each terminal device screen throwing, and improving the display effect of the multi-path screen throwing is a technical problem to be solved urgently.

Disclosure of Invention

In order to solve the problems in the background art, the invention provides a multi-path screen projection display method and display equipment, so as to realize dynamic self-adaptive layout matching of a screen projection interface, display the multi-path screen projection window in the maximum range in a screen, and improve the display effect of the screen projection interface.

The display device provided in the first aspect includes:

the display is used for displaying a screen-throwing interface;

the communicator is used for being connected with the terminal equipment in a screen throwing way;

a controller configured to perform:

receiving screen-throwing requests of at least M terminal devices, respectively receiving media configuration information from the M terminal devices, and matching layout among M screen-throwing windows adapted to a screen; wherein M is more than or equal to 1, and the media configuration information comprises the aspect ratio of video of terminal equipment screen throwing;

and controlling a display to display M screen-throwing windows in the screen-throwing interface according to the layout, and displaying screen-throwing data sent by corresponding terminal equipment in each screen-throwing window.

In an exemplary implementation of the first aspect, the controller is configured to match the layout as follows:

calculating M groups of size information according to the screen resolution of the display device or the display screen directions and the aspect ratio of the video projected by M terminal devices and the number of projection paths, wherein the size information comprises width and height;

According to the M groups of size information, M screen throwing windows are constructed; the aspect ratio of the screen-throwing window and the corresponding terminal equipment are kept consistent, and the display screen direction is horizontal or vertical;

calculating position distribution information according to the screen resolution and M groups of size information, wherein the position distribution information comprises coordinates of preset reference points on M screen throwing windows, and the coordinates of the preset reference points are used for positioning the positions of the screen throwing windows on a screen;

or calculating the position distribution information according to the resolution ratio of the screen recording in the M terminal devices and M groups of size information;

and positioning and arranging M screen throwing windows according to the position distribution information to obtain the layout.

In an exemplary implementation of the first aspect, the controller is further configured to perform:

newly establishing screen-throwing connection with target terminal equipment every time a screen-throwing request of the target terminal equipment is received, and accumulating and adding one to the screen-throwing path number;

and subtracting one from the number of screen throwing paths every time the screen throwing connection with one terminal device is detected to be disconnected.

When the screen-throwing path number is detected to be changed, updating the layout according to the changed N screen-throwing path numbers and media configuration information from N terminal devices which currently maintain the screen-throwing connection state;

and controlling the display to display N screen throwing windows in the screen throwing interface according to the updated layout.

when the layout is matched with a plurality of versions, calculating the sum of the areas of all the screen throwing windows in the layout of each version;

screening out the target layout with the largest sum of the areas;

and controlling a display to display the target layout in the screen projection interface.

The multi-path screen projection display method provided in the second aspect comprises the following steps:

receiving screen throwing requests of at least M terminal devices, and respectively receiving media configuration information from the M terminal devices, wherein M is more than or equal to 1, and the media configuration information comprises aspect ratio examples of video of the screen throwing of the terminal devices;

and displaying M screen throwing windows in the screen throwing interface according to the layout, and displaying the screen throwing data sent by the corresponding terminal equipment in each screen throwing window.

In an exemplary implementation of the second aspect, the layout is matched as follows:

In an exemplary implementation manner of the second aspect, the method further includes:

and displaying N screen throwing windows in the screen throwing interface according to the updated layout.

screening out the target layout with the largest sum of the areas;

and displaying the target layout in the screen projection interface.

According to the technical scheme, the display equipment can establish screen connection with at least one terminal equipment, the screen connection is carried out on the screen connection number (namely, the number of terminal equipment accessed by the screen) of the screen connection, the display equipment receives the screen connection data sent by each terminal equipment, and also receives media configuration information sent by each terminal equipment, wherein the media configuration information comprises aspect ratio examples of video of the screen connection, the relative proportion of the width and the height of the video stream in display can be known through the aspect ratio examples, the display screen direction of the video stream can be obtained, for example, the video stream with the aspect ratio example being larger than or equal to 1 is a transverse screen display, the aspect ratio example being smaller than 1 is a vertical screen display, and after the video stream is projected to the display equipment from the terminal, the aspect ratio examples of the video stream and the display screen direction are kept consistent, and the screen direction are set to be M paths (M is larger than or equal to 1), the screen connection is needed to be established at a screen connection interface, each screen window corresponds to one terminal equipment, the screen connection data can be obtained through the aspect ratio examples, the aspect ratio examples can be obtained, the screen connection is matched with the screen connection window, namely, the screen connection is enabled to be realized, the screen connection is adaptive to the screen connection window is realized, the aspect ratio is smaller than the aspect ratio, and the screen connection window is matched with the screen connection window, and the screen connection is enabled to be matched with the screen connection window.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following description will briefly explain the drawings required for the embodiments, and it is apparent that the drawings in the following description are only some embodiments of the present invention and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 illustrates a usage scenario of a display device according to some embodiments;

fig. 2 shows a hardware configuration block diagram of the control apparatus 100 according to some embodiments;

fig. 3 illustrates a hardware configuration block diagram of a display device 200 according to some embodiments;

FIG. 4 illustrates a software configuration diagram in a display device 200 according to some embodiments;

FIG. 5 illustrates a flow chart of a multi-way drop display method of some embodiments;

FIG. 6 (a) is a schematic diagram illustrating a layout of a single vertical screen scene with a number of screen shots of 1;

FIG. 6 (b) is a schematic diagram illustrating a layout in a single-landscape scene with a number of drops of 1;

FIG. 7 (a) is a schematic diagram illustrating a first layout in a double-landscape scene with a 2-landscape screen number;

FIG. 7 (b) is a schematic diagram illustrating a second layout in a double-landscape scene with a 2-landscape screen number;

FIG. 7 (c) is a schematic diagram illustrating a first layout of a screen-drop road number of 2 and a horizontal screen, vertical screen scene;

FIG. 7 (d) is a schematic diagram illustrating a second layout of a screen-drop road number of 2 and in a horizontal screen, vertical screen scenario;

FIG. 7 (e) is a schematic diagram illustrating a layout in a two vertical screen scenario with a 2-drop screen number;

FIG. 8 (a) is a schematic diagram illustrating version 1 of the layout in a two-bar (aspect ratio example different) scenario with a 2-drop road count;

FIG. 8 (b) is a schematic diagram illustrating version 2 of the layout in a two-bar (aspect ratio example different) scenario with a 2 drop road count;

FIG. 8 (c) is a schematic diagram illustrating version 1 of the layout in a two vertical (aspect ratio different) scenario with a 2 drop road count;

FIG. 8 (d) is a schematic diagram illustrating version 2 of the layout in a two vertical (aspect ratio different) scenario with a 2 drop road count;

FIG. 9 (a) is a schematic diagram illustrating version 1 of the layout in a one-horizontal, two-vertical screen scenario with a 3-drop road count;

FIG. 9 (b) is a schematic diagram illustrating version 2 of the layout in a 3-drop road number and one-horizontal, two-vertical screen scenario;

FIG. 9 (c) is a schematic diagram illustrating version 3 of the layout in a one-horizontal, two-vertical screen scenario with a number of screen shots of 3;

fig. 10 illustrates a flow chart of interaction of a terminal device with a display device.

Detailed Description

For purposes of clarity and implementation of the present application, the following description will make clear and complete descriptions of exemplary implementations of the present application with reference to the accompanying drawings in which exemplary implementations of the present application are illustrated, it being apparent that the exemplary implementations described are only some, but not all, of the examples of the present application.

It should be noted that the brief description of the terms in the present application is only for convenience in understanding the embodiments described below, and is not intended to limit the embodiments of the present application. Unless otherwise indicated, these terms should be construed in their ordinary and customary meaning.

The terms "first," second, "" third and the like in the description and in the claims and in the above-described figures are used for distinguishing between similar or similar objects or entities and not necessarily for limiting a particular order or sequence, unless otherwise indicated. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a product or apparatus that comprises a list of elements is not necessarily limited to all elements explicitly listed, but may include other elements not expressly listed or inherent to such product or apparatus.

The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code that is capable of performing the function associated with that element.

Fig. 1 is a schematic diagram of a usage scenario of a display device according to an embodiment. As shown in fig. 1, the display device 200 is also in data communication with a server 400, and a user can operate the display device 200 through the smart device 300 or the control apparatus 100.

In an exemplary implementation, the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes at least one of infrared protocol communication or bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled by a wireless or wired method. The user may control the display apparatus 200 by inputting a user instruction through at least one of a key on a remote controller, a voice input, a control panel input, and the like.

In one exemplary implementation, the smart device 300 may include any of a mobile terminal, tablet, computer, notebook, AR/VR device, etc.

In one exemplary implementation, the smart device 300 may also be used to control the display device 200. For example, the display device 200 is controlled using an application running on a smart device.

In one exemplary implementation, the smart device 300 and the display device may also be used for communication of data.

In an exemplary implementation, the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300, for example, the voice command control of the user may be directly received through a module configured inside the display device 200 for obtaining the voice command, or the voice command control of the user may be received through a voice control apparatus configured outside the display device 200.

In one exemplary implementation, the display device 200 is also in data communication with a server 400. The display device 200 may be permitted to make communication connections via a Local Area Network (LAN), a Wireless Local Area Network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200. The server 400 may be a cluster, or may be multiple clusters, and may include one or more types of servers.

In one exemplary implementation, software steps performed by one step execution body may migrate as needed to be performed on another step execution body with which data communication is performed. For example, software steps executed by the server may migrate to be executed on demand on a display device in data communication therewith, and vice versa.

Fig. 2 exemplarily shows a block diagram of a configuration of the control apparatus 100 in accordance with an exemplary embodiment. As shown in fig. 2, the control device 100 includes a controller 110, a communication interface 130, a user input/output interface 140, a memory, and a power supply. The control apparatus 100 may receive an input operation instruction of a user and convert the operation instruction into an instruction recognizable and responsive to the display device 200, and function as an interaction between the user and the display device 200.

In an exemplary implementation, the communication interface 130 is configured to communicate with the outside, including at least one of a WIFI chip, a bluetooth module, NFC, or an alternative module.

In one exemplary implementation, the user input/output interface 140 includes at least one of a microphone, a touchpad, a sensor, keys, or an alternative module.

Fig. 3 shows a hardware configuration block diagram of the display device 200 in accordance with an exemplary embodiment.

In one exemplary implementation, the display apparatus 200 includes at least one of a modem 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, a user interface.

In one exemplary implementation, the controller includes a central processor, a video processor, an audio processor, a graphics processor, RAM, ROM, a first interface to an nth interface for input/output.

In one exemplary implementation, the display 260 includes a display screen component for presenting pictures, and a driving component for driving image display, for receiving image signals from the controller output, for displaying video content, image content, and components of a menu manipulation interface, and a user manipulation UI interface, etc.

In one exemplary implementation, the display 260 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

In one exemplary implementation, the modem 210 receives broadcast television signals via wired or wireless reception and demodulates audio-video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

In one exemplary implementation, communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example: the communicator may include at least one of a Wifi module, a bluetooth module, a wired ethernet module, or other network communication protocol chip or a near field communication protocol chip, and an infrared receiver. The display apparatus 200 may establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

In one exemplary implementation, the detector 230 is used to collect signals of the external environment or interaction with the outside. For example, detector 230 includes a light receiver, a sensor for capturing the intensity of ambient light; alternatively, the detector 230 includes an image collector such as a camera, which may be used to collect external environmental scenes, user attributes, or user interaction gestures, or alternatively, the detector 230 includes a sound collector such as a microphone, or the like, which is used to receive external sounds.

In one exemplary implementation, the external device interface 240 may include, but is not limited to, the following: high Definition Multimedia Interface (HDMI), analog or data high definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, or the like. The input/output interface may be a composite input/output interface formed by a plurality of interfaces.

In an exemplary implementation, the controller 250 and the modem 210 may be located in separate devices, i.e., the modem 210 may also be located in an external device to the main device in which the controller 250 is located, such as an external set-top box or the like.

In one exemplary implementation, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored on the memory. The controller 250 controls the overall operation of the display apparatus 200. For example: in response to receiving a user command to select a UI object to be displayed on the display 260, the controller 250 may perform an operation related to the object selected by the user command.

In one exemplary implementation, the object may be any one of selectable objects, such as a hyperlink, an icon, or other operable control. The operations related to the selected object are: displaying an operation of connecting to a hyperlink page, a document, an image, or the like, or executing an operation of a program corresponding to the icon.

In one exemplary implementation, the controller includes at least one of a central processing unit (Central Processing Unit, CPU), a video processor, an audio processor, a graphics processor (Graphics Processing Unit, GPU), RAM Random Access Memory, RAM), ROM (Read-Only Memory, ROM), a first interface to nth interface for input/output, a communication Bus (Bus), and the like.

In one exemplary implementation, a user may input a user command through a Graphical User Interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the Graphical User Interface (GUI). Alternatively, the user may input the user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.

In one exemplary implementation, a "user interface" is a media interface for interaction and exchange of information between an application or operating system and a user that enables conversion between an internal form of information and a form acceptable to the user. A commonly used presentation form of the user interface is a graphical user interface (Graphic User Interface, GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It may be an interface element such as an icon, a window, a control, etc. displayed in a display screen of the electronic device, where the control may include at least one of a visual interface element such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a Widget, etc.

In one exemplary implementation, the user interface 280 is an interface (e.g., a physical key on a display device body, or the like) that may be used to receive control inputs.

In one exemplary implementation, a system of display devices may include a Kernel (Kernel), a command parser (shell), a file system, and an application program. The kernel, shell, and file system together form the basic operating system architecture that allows users to manage files, run programs, and use the system. After power-up, the kernel is started, the kernel space is activated, hardware is abstracted, hardware parameters are initialized, virtual memory, a scheduler, signal and inter-process communication (IPC) are operated and maintained. After the kernel is started, shell and user application programs are loaded again. The application program is compiled into machine code after being started to form a process.

Referring to fig. 4, in an exemplary implementation, the system is divided into four layers, from top to bottom, an application layer (Applications), an application framework layer (Application Framework), a An Zhuoyun row (Android run) and a system library layer (system runtime layer), and a kernel layer, respectively.

In an exemplary implementation, at least one application program is running in the application program layer, where the application programs may be a Window (Window) program of an operating system, a system setup program, or a clock program, etc.; or may be an application developed by a third party developer. In particular implementations, the application packages in the application layer are not limited to the above examples.

The framework layer provides an application programming interface (application programming interface, API) and programming framework for application programs of the application layer. The application framework layer includes a number of predefined functions. The application framework layer corresponds to a processing center that decides to let the applications in the application layer act. Through the API interface, the application program can access the resources in the system and acquire the services of the system in the execution.

As shown in fig. 4, the application framework layer in the embodiment of the present application includes a manager (manager), a Content Provider (Content Provider), and the like, where the manager includes at least one of the following modules: an Activity Manager (Activity Manager) is used to interact with all activities that are running in the system; a Location Manager (Location Manager) is used to provide system services or applications with access to system Location services; a Package Manager (Package Manager) for retrieving various information about an application Package currently installed on the device; a notification manager (Notification Manager) for controlling the display and clearing of notification messages; a Window Manager (Window Manager) is used to manage bracketing icons, windows, toolbars, wallpaper, and desktop components on the user interface.

In one exemplary implementation, the kernel layer is a layer between hardware and software. As shown in fig. 4, the kernel layer contains at least one of the following drivers: audio drive, display drive, bluetooth drive, camera drive, WIFI drive, USB drive, HDMI drive, sensor drive (e.g., fingerprint sensor, temperature sensor, pressure sensor, etc.), and power supply drive, etc.

The above embodiments introduce the content of the hardware/software architecture, the functional implementation, etc. of the display device. In some application scenarios, the terminal device may push media data to the display device, for example, the user projects a video of interest to the display device for large-screen playing through some application programs with a screen projection function; or the terminal device and the display device establish a screen-casting connection through some protocols, such as DLNA (Digital Living Network Alliance ) protocol, and the like. After the terminal equipment and the display equipment are successfully connected, the terminal equipment can project the screen projection data to the display equipment, and the display equipment displays the screen projection data on a screen projection interface. The terminal equipment can be mobile terminals such as smart phones, tablet computers and notebook computers, and can also be desktop computers or other small-screen display equipment.

In an exemplary implementation manner, when the terminal device plays the media data, a user may select a device for playing the media data by projecting a screen from the device list, and a display device supporting the screen projecting function may be identified and automatically added to the device list, and after the user selects the display device from the device list, the media data may be projected by the terminal device to the display device end for playing. When the screen-throwing mode is adopted to push the media data, the terminal equipment and the display equipment can be connected to the same Wifi network, and the mode that the terminal pushes the media data to the display equipment is not limited. Of course, in other technologies, the screen-throwing mode can also be performed across networks. The specific implementation manner of the screen projection is not limited in the application.

In one exemplary implementation, terminal device a sends a screen-cast request to the display device, which may ask the user if he agrees to terminal device a's screen-cast, e.g., in the form of a UI question pop-up, and may provide several option controls, such as "this agreement", "disagree", and "permanent agreement". The user selects 'permanent agreement', namely, the permanent authorization is equivalent to the permanent authorization of the terminal equipment A, and the subsequent terminal equipment A does not need to display a UI inquiry popup window any more when initiating a screen throwing request, but directly establishes the screen throwing connection between the display equipment and the terminal equipment A; if the user selects 'this time agrees', namely, the user performs one-time authorization on the terminal equipment A, the display equipment and the terminal equipment A can establish screen-throwing connection, but the screen-throwing authority of the terminal equipment A is only valid for the next time, and the UI inquiry popup window still needs to be displayed again when the subsequent terminal equipment A initiates a screen-throwing request; if the user selects "disagree", namely, the terminal equipment A is not authorized for the time, the display equipment and the terminal equipment A do not establish screen projection connection, and the UI inquiry popup window still needs to be displayed again when the subsequent terminal equipment A initiates a screen projection request.

In an exemplary implementation, the display device may maintain a trusted device list, and support the user to add, delete and modify device information with the screen-casting authority in the trusted device list, where the device information includes, but is not limited to, a device ID, a MAC address, etc., that is, the user may customize the authorization list on the display device side. When the display device receives a screen projection request sent by the terminal device B, inquiring whether the terminal device B is contained in the trusted device list according to device information carried in the screen projection request. If the terminal equipment B is contained in the trusted equipment list, establishing screen-casting connection between the terminal equipment B and the display equipment; if the trusted device list does not include the terminal device B, the display device does not respond to the screen-throwing request, namely the screen-throwing connection with the terminal device B is not established. It should be noted that, the authorization and connection manner of the terminal device for screen projection are not limited to the examples of the present application.

In the current application scenario, a one-to-one screen-throwing mode is adopted, that is, the display device is only connected with one terminal device in a screen throwing mode, and screen throwing data can be displayed exclusively on a screen. However, in other application scenarios, for example, when the user plays the screen-throwing game of the terminal device C, if the user wants to synchronously watch the course or the explanation of the game, a screen-throwing request of the game course/explanation video can be initiated to the display device through the terminal device D, and after the screen-throwing connection between the display device and the terminal device D is successful, the connection is equivalent to accessing two paths of screen-throwing data; for example, the family comprises a member 1, a member 2 and a member 3, the three members can respectively initiate a screen throwing request to the display equipment through the used terminals, after the display equipment responds to the three screen throwing requests, the screen of each terminal is displayed on a screen throwing interface, the three members can perform interactive communication and the like, and at the moment, the display equipment is connected with three paths of screen throwing data.

For an application scene that the display device is connected with a plurality of screen throwing windows, the screen interface can display the screen throwing windows, each screen throwing window corresponds to a terminal device which is successfully connected with the screen throwing device and displays screen throwing data sent by the terminal device, but the display effect of the screen throwing windows on the screen throwing interface influences the look and feel of a user: for example, a plurality of screen throwing windows are not compactly distributed, so that a screen throwing interface is more white; for another example, the plurality of screen-throwing windows are distributed in a chaotic manner, and the overall layout is not fully orderly; for example, the size of part of the screen-throwing window is smaller, so that the user cannot see the details of the video in the window clearly; for example, when the video is displayed on a horizontal screen at the terminal device, the screen direction is disordered after the video is projected to the display device, and the video is displayed on a vertical screen, the uniformity of the display device and the terminal device for displaying the video is poor, and the like, so that the problem of a plurality of display effects exists. In this regard, the present application aims at planning and laying out multiple projection windows to improve the display effect of projection interfaces.

Referring to fig. 5, in an exemplary implementation, a multi-way projection display method is provided, the method being configured to be executed by a controller 250 in a display device, i.e. the controller 250 is the execution subject of the method, the method comprising the following program steps:

Step S01, matching and adapting to the layout among M screen throwing windows of a screen according to the M screen throwing paths counted currently and media configuration information received from M terminal devices respectively;

and step S02, displaying M screen throwing windows in the screen throwing interface according to the layout, and displaying screen throwing data sent by the corresponding terminal equipment in each screen throwing window.

The M is used for representing the number of screen-throwing paths, that is, the number of terminal devices which establish screen-throwing connection with the display device, and the number of screen-throwing paths can dynamically change along with the connection and disconnection of the screen-throwing connection, so that the number of screen-throwing paths needs to be counted so as to accurately plan typesetting and layout of a screen-throwing interface. When M is 1, executing single-path screen projection; when M is more than or equal to 2, the screen is thrown in multiple ways; m can be zero, and the display device does not receive the screen projection data sent by the connection device, so that the screen projection interface is not displayed, and the layout on the screen projection interface is empty.

In some embodiments, each time the display device receives a screen-throwing request of a target terminal device, if a screen-throwing connection with the target terminal device is newly established, accumulating and adding one to the count value of the screen-throwing path number; in addition, the terminal equipment may disconnect the screen-throwing connection with the display equipment due to the factors of disconnection or manual screen-throwing exit of the terminal user, so that the display equipment cannot receive the screen-throwing data sent by the terminal equipment, and the display equipment end needs to cut off the screen-throwing, so that the count value of the screen-throwing road number is reduced by one when the screen-throwing connection with one terminal equipment is detected to be disconnected. According to the embodiment, the automatic counting of the screen throwing path number according to the screen throwing connection dynamic state of the display equipment can be realized.

In some embodiments, each time the display device receives the screen-throwing data of one target terminal device, accumulating and adding one to the count value of the screen-throwing path number; in addition, the terminal equipment can not receive the screen throwing data sent by the terminal equipment due to the factors of network disconnection or the fact that a terminal user manually quits the screen throwing, if the display equipment end needs to cut off the screen throwing, the count value of the screen throwing path number is reduced by one. According to the embodiment, the automatic counting of the screen throwing path number according to the screen throwing connection dynamic state of the display equipment can be realized.

In some exemplary optional embodiments, the display device side in the present application may receive, in addition to the screen projection data from the M terminal devices, media configuration information sent by the M terminal devices respectively.

In an exemplary implementation, the screen-cast data relates to a media category, e.g., the media is a pure video category, and the screen-cast data includes a video stream that is screen-cast; if the media is in the audio-video category, the screen throwing data comprises a video stream and an audio stream; the screen data may be in the form of images (still images, moving pictures, etc.), text, or audio-only streams. When the on-screen data is an audio stream, play content information such as audio may be presented on the display interface.

In some exemplary embodiments, the screen shot data is related to a screen recording mode of the terminal device. For example, the recording mode may be recorded according to a fixed resolution, such as 1280×720, 1920×1080, and the like, and may also be recorded according to a size of a terminal screen, such as a 16:9 size, a 17:9 size, a 21:9 size, and the like.

In an exemplary implementation, the media configuration information includes an aspect ratio of video projected by the terminal device, that is, a ratio of a horizontal width to a vertical height when the video is displayed, through which a display screen direction can be identified, and when the aspect ratio is greater than or equal to 1, that is, the width is greater than or equal to the height, the display mode is presented as a horizontal screen display mode; when the aspect ratio is less than 1, i.e., the width is less than the height, then the portrait display mode is presented. The creation of the screen throwing window can be restrained according to the aspect ratio and the screen direction indicated in the media configuration information, so that the aspect ratio and the display screen direction of the screen throwing window and the corresponding terminal equipment are kept consistent, and the problem of disorder of media related configuration after screen throwing can be avoided.

In an exemplary implementation, the media configuration information may also include an aspect ratio of the terminal device. When the aspect ratio is greater than or equal to 1, i.e., the width is greater than or equal to the height, then the display mode is presented as a landscape display mode; when the aspect ratio is less than 1, i.e., the width is less than the height, then the portrait display mode is presented.

In an exemplary implementation manner, the media configuration information may further include a media category, and after the display device reads the media configuration information, the display device may learn the media category about to be dropped, so as to display the drop window with different effects according to the media category. For example, for pure video, audio, and image media, the projection window may present visual projection data, and the projection window should match the aspect ratio and screen orientation of the video/image; for text type media, the drop screen window can visually display readable text information, but the aspect ratio and the screen orientation of the drop screen window are not limited; for pure audio media, the screen throwing window cannot display a visual picture, a voice playing control (such as a small loudspeaker) can be arranged in the screen throwing window, when the audio stream of the screen throwing channel is played, the voice playing control can be presented as a dynamic special effect in playing, and likewise, the aspect ratio and the screen direction of the screen throwing window corresponding to the pure audio stream are not limited.

Optionally, for the screen projection window with the undefined aspect ratio and screen orientation under the special media category, for example, a preset window layout can be adopted, wherein the preset window layout refers to a preset screen projection window with a fixed aspect ratio (or a fixed width and height) and screen orientation, after the media to be projected is identified as text or pure audio and other categories, the preset window layout is directly called, and then the screen projection data is loaded into the window layout; for another example, instead of presetting the window layout, an adaptive screen projection window can be created preferentially for media such as pure video, audio, video, and images with limited aspect ratio and screen orientation, then an adaptive screen projection window is created and displayed in a blank area in the screen projection interface to display screen projection data of text and pure audio categories, and the defined screen projection window can be preferentially matched in this way, and then the screen display area is maximally occupied, so that a plurality of screen projection windows are more compact, and the blank area is reduced. It should be noted that, creation of the drop screen window and UI display setting under different media categories are not limited to the embodiments of the present application.

In an exemplary implementation of step S10, M sets of size information may be calculated according to the screen resolution of the display device or the aspect ratio and the screen directions of the video projected from M terminal devices, and the number of projection paths, where the size information includes width _i And a height window height _i I.e. build window width _i *windowHeight _i And i is used for identifying the size information/the sequence number of the screen throwing window, i is more than or equal to 1 and less than or equal to M, and the value of i has a mapping relation with the equipment information of the terminal equipment. For example, m=3, the number 1 corresponds to the terminal device a, the number 2 corresponds to the terminal device B, and the number 3 corresponds to the terminal device C, when i is equal to 1, it indicates that the screen-projection window 1 created according to the 1 st group size information is used to display the screen-projection data of the terminal device a, so that it can be accurately identified which terminal device the screen-projection data from should be kept being projected by each screen-projection window.

In some examples, the sequence numbers may be assigned in a chronological order of the screen connection, such as the earlier the screen is accessed, the smaller the sequence number. For example, a pre-typesetting mode is adopted, screen-throwing pictures are typeset in advance at the moment of screen-throwing connection, and after screen-throwing data are received, the pictures are displayed according to the pre-typesetting mode.

In another example, typesetting may be performed in the order in which the screen-cast data is received, e.g., in terms of a data stream received in real-time.

In the above exemplary implementation, the display device media configuration information may be video recorded at a fixed resolution, such as 1920×1080, 1280×720, and so on.

In the above-described exemplary implementation, the display device media configuration information may be video recorded according to the resolution of the screen, and the current screen is not proportional to the large screen terminal by 16:9/9:16, but other proportional resolutions such as 17:9, 21:16, etc. In one exemplary implementation, M projection windows are constructed according to M sets of size information, and the projection windows maintain consistent aspect ratio and display orientation with their corresponding terminal devices. Since each set of size information defines a particular width and height of the screen window, a screen window matching the size information may be constructed. After the sizes of the M screen throwing windows are all determined, a specific display position of each screen throwing window in the screen interface is also required to be planned.

In an exemplary implementation, the location distribution information is calculated according to the media configuration information received from the terminal device, and the location distribution information includes preset reference points on M projection windows _i Coordinates (x) _i ，y _i ) The preset reference Point _i Coordinates (x) _i ，y _i ) For locating the position of the screen-drop window on the screen. Because the screen projection window is generally rectangular, the preset reference points can select certain key points in the rectangle, alternatively, the preset reference points are any one of four vertexes of the rectangle or the center point of the rectangle, and the principle is selected so as to facilitate calculation and positioning. And finally, positioning and arranging the M created screen throwing windows according to the calculated position distribution information, so that the sizes and positions of the M screen throwing windows are determined, and the layout is formed.

The above implementation will be described below in connection with specific UI examples. Fig. 6 (a) illustrates a screen-casting interface in a single vertical screen scenario with a screen-casting number of 1 and a video stream, where the display device is only accessed to a single screen-casting initiated by the terminal device a first, and the width of the screen-casting window 1 is 1080×9/16= 607.5, i.e. the width of the screen-casting window 1 is 1080×9/16= 607.5, when the screen resolution of the display device is 1920×1080 (i.e. the default aspect ratio of the screen of the display device is 16:9), and the media configuration file sent by the terminal device a indicates that the aspect ratio is 9:16 (vertical screen), then the screen-casting window 1 to be created is also vertical screen, and the height of the screen-casting window 1 is adapted to the screen height windowWidth ₁ *windowHeight ₁ =607.5×1080. When determining the position of the screen-throwing window 1, a coordinate system is constructed by taking the top left corner vertex of the screen as an origin, taking the right corner vertex as an x-axis forward direction and taking the vertical downward direction as a y-axis forward direction, and the top left corner vertex of the screen-throwing window 1 is taken as a preset reference Point ₁ The preset reference Point is calculated by considering that the single window is better in view effect in a vertical screen centered display mode ₁ Coordinates (x) ₁ ，y ₁ ) = (656.25,0). Point due to the preset reference Point ₁ Both the coordinate position of the projection window 1 and the size of the projection window 1 are determined, so that the position of the projection window 1 in the overall projection interface is uniquely determined, and the resulting layout is shown in fig. 6 (a).

Fig. 6 (b) illustrates a screen-drop interface in a single-horizontal screen scenario where the number of screen-drops is 1 and the video stream is a single-horizontal screen scenario where the display device is only accessed to the screen-drop screen initiated by the terminal device a first, and when the screen resolution of the display device is 1920×1080 (i.e. the default aspect ratio of the screen of the display device is 16:9), the media configuration file sent by the terminal device a indicates that the aspect ratio is 16:9 (horizontal screen) as an example, then the screen-drop window 1 to be created is also a horizontal screen, and because of the single-window display mode, the screen-drop window 1 is completely adapted to the screen in aspect ratio, the screen-drop window 1 can be displayed in full screen, i.e. the window width ₁ *windowHeight ₁ 1920×1080, using the top left corner vertex of the projection window 1 as the preset reference Point ₁ Presetting a reference Point ₁ Coordinates (x) ₁ ，y ₁ ) = (0, 0), the layout thus obtained is shown in fig. 6 (b).

Fig. 7 (a) and fig. 7 (B) exemplarily show two screen-throwing interfaces in a screen-throwing path number of 2 and a double-horizontal screen scene, fig. 7 (a) is that screen-throwing windows of two horizontal screens are transversely arranged side by side, fig. 7 (B) is that screen-throwing windows of two horizontal screens are longitudinally arranged side by side, at this time, a display device is accessed to a two-way screen-throwing initiated by a terminal device a and a terminal device B, and taking a screen resolution of the display device as 1920 x 1080, and media configuration files sent by the terminal device a and the terminal device B indicate aspect ratio examples that the aspect ratio examples are 16:9 (horizontal screen), then the created screen-throwing window 1 (corresponding to the terminal device a) and the screen-throwing window 2 (corresponding to the terminal device B) are also horizontal screens.

For the example of fig. 7 (a), since it is arranged side by side in the lateral direction, in order to occupy the screen area maximally, the sum of the widths of the screen-throwing window 1 and the screen-throwing window 2 is made equal to the screen width, i.e., window width ₁ +windowWidth ₂ =1920, and in addition, to make the size of the double-throw window more uniform, a window width can be set ₁ ＝windowWidth ₂ =960, then window height ₁ ＝windowHeight ₂ =960×9/16=540, i.e. window width ₁ *windowHeight ₁ ＝windowWidth ₂ *windowHeight ₂ 960×540, the top left corner vertex of the projection window 1 is used as the preset reference Point ₁ Point with the top left corner of the projection window 2 as a preset reference Point ₂ The preset reference Point is calculated by considering that the display mode of the double windows arranged in a horizontal screen and longitudinally centered is better in view effect ₁ Coordinates (x) ₁ ，y ₁ ) = (0, 270), preset reference Point ₂ Coordinates (x) ₂ ，y ₂ ) = (960, 270), the layout thus obtained is shown in fig. 7 (a).

For the example of fig. 7 (b), since it is arranged side by side in the longitudinal direction, in order to occupy the screen area maximally, the sum of the heights of the screen-projecting window 1 and the screen-projecting window 2 is made equal to the screen height, i.e., window height ₁ +windowHeight ₂ =1080, in addition to making the size of the double-throw window more uniform, a window height can be set ₁ ＝windowHeight ₂ =540, then window width ₁ ＝windowWidth ₂ =540×16/9=960, i.e. window width ₁ *windowHeight ₁ ＝windowWidth ₂ *windowHeight ₂ 960×540, the top left corner vertex of the projection window 1 is used as the preset reference Point ₁ Point with the top left corner of the projection window 2 as a preset reference Point ₂ Considering that the double window is better in view effect in a display mode of parallel transverse screens and transverse centering, the preset reference Point is calculated ₁ Coordinates (x) ₁ ，y ₁ ) = (480,0), preset reference Point ₂ Coordinates (x) ₂ ，y ₂ ) = (480, 540), the layout thus obtained is shown in fig. 7 (b).

In an exemplary implementation manner, after a layout is generated by matching, in the process of displaying a plurality of screen throwing windows according to the layout on a screen throwing interface and loading and displaying screen throwing data of each path, a newly accessed screen throwing path may exist at any time, or at least one terminal device may be disconnected, that is, the screen throwing path number may be changed during the period, the layout needs to be updated according to the changed screen throwing path number, so as to realize the adaptive adjustment of the layout along with the screen throwing path number, wherein the adjustment relates to the number of the screen throwing windows in the layout, the size of each screen throwing window and the relative position distribution of each screen throwing window.

For example, in the scenario of fig. 6 (B), the display device establishes a screen-throwing connection with the terminal device B, and the aspect ratio is indicated as 16:9 (horizontal screen) in the media configuration information sent by the terminal device B, and when the controller 250 detects that the screen-throwing path number is changed from 1 to 2, the layout may be updated to the version shown in fig. 7 (a) or fig. 7 (B); for another example, in the scenario of fig. 7 (a) or fig. 7 (B), when the terminal device B suddenly breaks the screen connection with the display device and the controller 250 detects that the screen number is changed from 2 to 1, the layout may be updated to the version shown in fig. 6 (B).

Fig. 7 (c) and 7 (d) exemplarily show two screen-throwing interfaces in a horizontal screen and a vertical screen scene with the screen-throwing path number of 2, and fig. 7 (c) shows a screen-throwing window of the horizontal screen on the left side and a screen-throwing window of the vertical screen on the right side; fig. 7 (d) shows a vertical screen window on the left side and a horizontal screen window on the right side. At this time, the display device has been accessed to the two-way screen casting initiated by the terminal device a and the terminal device B, and the screen resolution of the display device is 1920×1080, the media configuration file sent by the terminal device a indicates that the aspect ratio example is 16:9 (horizontal screen), the media configuration file sent by the terminal device B indicates that the aspect ratio example is 9:16 (vertical screen), then the created screen casting window 1 (corresponding to the terminal device a) is the horizontal screen, and the screen casting window 2 (corresponding to the terminal device B) is the vertical screen.

For the example of FIG. 7 (c), since it is a horizontal one and a vertical oneThe windows are arranged side by side, so as to occupy the screen area maximally, and the sum of the widths of the screen-throwing window 1 and the screen-throwing window 2 is equal to the screen width, namely the window width ₁ +windowWidth ₂ Let 1920 adapt the height of the vertical screen's projection window 2 to the screen height, i.e. window height ₂ =1080, then window width ₂ ＝1080*9/16＝607.5，windowWidth ₁ ＝1920-607.5＝1312.5，windowHeight ₁ 1312.5×9/16= 738.3, then window width ₁ *windowHeight ₁ ＝1312.5*738.3，windowWidth ₂ *windowHeight ₂ =607.5×1080, with the top left vertex of the projection window 1 as the preset reference Point ₁ Point with the top left corner of the projection window 2 as a preset reference Point ₂ The preset reference Point is calculated from the view that the projection window 1 is better in view of the viewing effect in the vertically centered display mode ₁ Coordinates (x) ₁ ，y ₁ ) = (0, 170.9), preset reference Point ₂ Coordinates (x) ₂ ，y ₂ ) = (1312.5,0), the layout thus obtained is shown in fig. 7 (c).

FIG. 7 (d) differs from FIG. 7 (c) only in that the horizontal and vertical screens are laterally distributed on different sides, so that the size of the screen-throwing window 1 and the screen-throwing window 2 are the same, and Point ₁ And Point (Point) ₂ Is different in coordinates. I.e. window width ₁ *windowHeight ₁ ＝1312.5*738.3，windowWidth ₂ *windowHeight ₂ =607.5×1080, with the top left vertex of the projection window 1 as the preset reference Point ₁ Point with the top left corner of the projection window 2 as a preset reference Point ₂ The preset reference Point is calculated from the view that the projection window 1 is better in view of the viewing effect in the vertically centered display mode ₂ Coordinates (x) ₂ ，y ₂ ) = (0, 0), preset reference Point ₁ Coordinates (x) ₁ ，y ₁ ) = (607.5, 170.9), the layout thus obtained is shown in fig. 7 (d).

In some examples, for example, in the single-channel vertical screen scenario in fig. 6 (a), the display device establishes a screen-throwing connection with the terminal device B again, and the aspect ratio is indicated as 16:9 (horizontal screen) in the media configuration information sent by the terminal device B, and when the controller 250 detects that the screen-throwing number is changed from 1 to 2, the layout may be updated to a two-channel horizontal screen-vertical screen version similar to that shown in fig. 7 (c) or fig. 7 (d).

In some examples, for example, in the single-path landscape screen scenario in fig. 6 (B), the display device establishes a screen-throwing connection with the terminal device B, and the aspect ratio is indicated as 9:16 (vertical screen) in the media configuration information sent by the terminal device B, and when the controller 250 detects that the screen-throwing path number is changed from 1 to 2, the layout may be updated to the two-path one-landscape screen one-vertical screen version shown in fig. 7 (c) or fig. 7 (d).

In some examples, for example, in the scenario of fig. 7 (c) or fig. 7 (d), when the terminal device B suddenly breaks the screen-throwing connection with the display device, and the controller 250 detects that the screen-throwing path number is changed from 2 to 1, the layout may be updated to a version of the single-cross-screen window (screen-throwing window 1) full-screen exclusive screen shown in fig. 6 (B).

In some examples, for example, in the scenario of fig. 7 (c) or fig. 7 (d), when the terminal device a suddenly breaks the screen-casting connection with the display device, and the controller 250 detects that the screen-casting path number is changed from 2 to 1, the layout may be updated to a version of the exclusive screen similar to the single vertical screen window (screen-casting window 2) shown in fig. 6 (a).

Fig. 7 (e) illustrates a screen projection interface in a dual vertical screen scenario with a screen projection path number of 2, where in fig. 7 (e), two vertical screen projection windows are arranged side by side along a transverse direction, and in case that the screen resolution of the display device is 1920 x 1080, the aspect ratio of the media configuration file indications sent by the terminal device a and the terminal device B are both 9:16 (vertical screen), then the created screen projection window 1 (corresponding to the terminal device a) and the created screen projection window 2 (corresponding to the terminal device B) are both vertical screens.

For the example of fig. 7 (e), since the screen-throwing window 1 and the screen-throwing window 2 are both vertical screens, the heights of both screen-throwing windows are adapted to the screen height, i.e., window height ₁ ＝windowHeight ₂ =1080, from which a window width can be calculated ₁ ＝windowWidth ₂ ＝1080*9/16＝607.5，windowWidth ₁ *windowHeight ₁ ＝windowWidth ₂ *windowHeight ₂ =607.5×1080. Point with the top left corner of the projection window 1 as a preset reference Point ₁ Point with the top left corner of the projection window 2 as a preset reference Point ₂ Considering that the viewing effect is better when the two vertical screen windows are uniformly distributed along the transverse direction at equal intervals, calculating the distance value d _x For (1920-2 x 607.5)/3=235, the reference Point is preset ₁ Coordinates (x) ₁ ，y ₁ ) = (235,0), preset reference Point ₂ Coordinates (x) ₂ ，y ₂ )＝(2*d _x +607.5,0) = (1077.5,0), the layout thus obtained is shown in fig. 7 (e).

In some examples, for example, in the single-channel vertical screen scenario in fig. 6 (a), the display device establishes a screen-throwing connection with the terminal device B again, and the aspect ratio is indicated as 9:16 (vertical screen) in the media configuration information sent by the terminal device B, and when the controller 250 detects that the screen-throwing number is changed from 1 to 2, the layout may be updated to the version of the two-channel double vertical screen shown in fig. 7 (e).

In some examples, for example, in the two-way two-vertical screen scenario of 7 (e), where terminal device a or terminal device B disconnects the screen-casting connection, and controller 250 detects that the number of screen-casting paths is changed from 2 to 1, the layout may be updated to a version similar to the single-vertical screen window exclusive screen shown in fig. 6 (a).

In some examples, in any of the scenarios in fig. 6 (a) to 6 (b) and fig. 7 (a) to 7 (e), if all the screen-throwing connections are disconnected, that is, the controller detects that the screen-throwing path number becomes 0, the screen-throwing interface is exited, and the UI display interface is restored to the front of screen throwing.

The construction and planning processes of the layout are described in detail by taking the single-path and double-path screen projection as an example, when the screen projection number M is greater than or equal to 3, the matching mechanism of the layout can adaptively refer to the example, and the matching of the layout can have the characteristics of self-adaptability, flexibility, diversity and the like, so that the application is not listed and repeated one by one. When the number of the screen throwing paths, the screen directions and the aspect ratio of each path of screen throwing media are the same, the layout is diversified due to different typesetting forms among the screen throwing windows, for example, the two paths of double-transverse-screen scenes at least comprise two versions of fig. 7 (a) and 7 (b), and the two paths of single-transverse-screen and single-vertical-screen scenes at least comprise two versions of fig. 7 (c) and 7 (d).

In one exemplary implementation, when matching to multiple versions of a layout, the present application performs optimal screening based on the principle of maximum area, i.e., calculates the sum of areas of all projection windows in each version of the layout, and screens the target layout with the largest sum of areas, i.e., S _max ＝Max{k＝1,2…,Q∣S _k K is used to represent the version number of the layout, Q is the number of versions of the matched layout, 1.ltoreq.k.ltoreq.Q,

s in the formula _i The area of the ith screen projection window in the k-th version of layout is defined, and M is the total number of screen projection windows in all versions of layout. In this way, the maximum area sum S is calculated _max Then S is carried out _max The corresponding versions are used as the screened target layout, and the target layout is displayed on the screen projection interface, so that each screen projection window occupies the screen more fully and maximally, the typesetting among each screen projection window is compact, the white area is reduced as much as possible, and the display effect of the screen projection interface is improved. Since the screen-projection window is rectangular, the area of a single screen-projection window is equal to the product of its width and height.

As an example, in a two-way dual-horizontal screen scenario, the screen resolution of the display device is 1920×1080, the aspect ratio of the media configuration file indication sent by the terminal device a is 17:9 (horizontal screen), and the aspect ratio of the media configuration file indication sent by the terminal device B is 16:9 (horizontal screen), that is, the two-way screen media have different aspect ratio, two screen projection windows are constructed, where the screen projection window 1 corresponds to the terminal device a, and the screen projection window 2 corresponds to the terminal device B.

As version 1 of the layout shown in fig. 8 (a), since it is arranged side by side in the lateral direction, to occupy the screen area maximally, the sum of the widths of the screen-throwing window 1 and the screen-throwing window 2 is made equal to the screen width, i.e., window width ₁ +windowWidth ₂ =1920, and in addition, to make the size of the double-throw window more uniform, a window width can be set ₁ ＝windowWidth ₂ =960, windowHeight ₁ ＝960*9/17＝508，windowHeight ₂ ＝960*9/16＝540，windowWidth ₁ *windowHeight ₁ ＝960*508，windowWidth ₂ *windowHeight ₂ For version 1, the sum of the areas of its projection windows S ₁ ＝960*508+960*540＝1006080。

FIG. 8 (b) shows version 2 of the layout, since it is arranged side by side in the portrait orientation, to maximize the screen area, the sum of the heights of the screen-drop window 1 and the screen-drop window 2 is made equal to the screen height, i.e., the window height ₁ +windowHeight ₂ =1080, in addition to making the size of the double-throw window more uniform, a window height can be set ₁ ＝windowHeight ₂ ＝540，windowWidth ₁ ＝540*17/9＝1020，windowWidth ₂ =540×16/9=960, the window width can be obtained ₁ *windowHeight ₁ ＝1020*540，windowWidth ₂ *windowHeight ₂ For version 2, the sum of the areas of its projection windows S ₂ =1020×540+960×540= 1069200. As is evident from comparison, S ₂ Greater than S ₁ Version 2 is thus taken as the target layout for the final display.

Fig. 8 (c) illustrates a screen-projection interface in a dual vertical screen scenario with a screen-projection path number of 2, where in fig. 8 (c), screen-projection windows of two vertical screens are arranged side by side along a transverse direction, and resolutions of two screen recordings of terminal equipment are different, for example, the resolution of the screen recording of terminal equipment a is 1920×1080, the resolution of the screen recording of terminal equipment B is 2040×1080, and media configuration files sent by terminal equipment a and terminal equipment B indicate aspect ratios of 9:16 (vertical screen) and 9:17 (vertical screen), respectively, and then the created screen-projection window 1 (corresponding to terminal equipment a) and screen-projection window 2 (corresponding to terminal equipment B) are vertical screens, but the transverse display sizes of the two screen-projection windows are different.

For the example of fig. 8 (c), since the screen projection window 1 and the screen projection window 2 are both vertical screens, the heights of the two screen projection windows are adapted to the screen heights, that is, windowheight1=windowheight2=1080, so that the coordinates (x 1, y 1) = (247,0) of the windowWidth 1=1080×9/16=607.5, windowWidth 2=1080×9/17= 571.7, the top left corner vertex of the screen projection window 1 is taken as the preset reference Point1, the top left corner vertex of the screen projection window 2 is taken as the preset reference Point2, and considering that the viewing effect is better when the two vertical screen windows are uniformly arranged in the transverse direction, the distance value dx is (1920-607.5-571.7)/3=247, the coordinates (x 1, y 1) = (247,0) of the preset reference Point1, and the coordinates (x 2, y 2) = (2×x+607.5, 0) = (1101.5,0) of the preset reference Point 2) are calculated, so that the layout of fig. 8 (c) is obtained.

Fig. 8 (d) illustrates another screen projection interface in a dual vertical screen scene with the screen projection path number of 2, where in fig. 8 (d), the screen projection windows of two vertical screens are arranged side by side in the lateral direction, the resolution of the screen recording of the terminal device a is 2040×1080, the resolution of the screen recording of the terminal device B is 1920×1080, and the layout shown in fig. 8 (d) is obtained according to the calculation mode shown in fig. 8 (c).

In one exemplary implementation, if there are Q versions of the layout, and each version has the same sum of the area of the drop windows, S ₁ ＝S ₂ ＝…＝S _Q Any one version can be randomly selected as the target layout.

In some exemplary implementations, the drop windows in the layout may or may not be ordered by the time of access to the drop. Taking three paths of screen casting as an example, assuming that a display device is sequentially connected with a terminal device A, a terminal device C and a terminal device B in a screen casting manner, media configuration information sent by the terminal device A indicates that the aspect ratio of the media configuration information is 16:9 (horizontal screen), media configuration information sent by the terminal device C indicates that the aspect ratio of the media configuration information is 9:16 (vertical screen), and media configuration information sent by the terminal device B indicates that the aspect ratio of the media configuration information is 9:16 (vertical screen), namely a scene corresponding to two horizontal screens and two vertical screens of three paths of screen casting. Three versions of the layout can be matched in this scenario, with version 1 being illustrated in fig. 9 (a), version 2 being illustrated in fig. 9 (b), and version 3 being illustrated in fig. 9 (c).

Wherein, version 1 of the example of fig. 9 (a) supports arranging a screen-throwing window 1 (corresponding to terminal equipment a), an opposite screen-throwing window 2 (corresponding to terminal equipment C) and a screen-throwing window 3 (corresponding to terminal equipment B) in sequence along the transverse direction according to the sequence of accessing the screen-throwing, and presents a typesetting of transverse screen-vertical screen, and sets a unique transverse screen window as a first display; FIG. 9 (b) is presented as a portrait-landscape-portrait layout, i.e., the only landscape window is set as a landscape-centered display, and the dual portrait windows are symmetrically arranged on both sides of the landscape window; fig. 9 (c) then presents a portrait-landscape layout, i.e., the only landscape window is set to the last display. The versions of the examples of fig. 9 (b) and 9 (c) do not support ordering in the chronological order of access to the screen.

In some exemplary implementations, since the layouts of the three versions in the above scenario all conform to the principle of maximum area, any one version may be output as the target layout, and in practical application, the visual viewing effect may be referred to, and a layout of a certain version may be selected, for example, the layout of version 2 shown in fig. 9 (b) has the best symmetry, so in the scenario of three paths of horizontal and vertical screens, the screen projection display may be preferably performed according to the layout of version 2.

In an exemplary implementation manner, referring to fig. 10, an interaction flow between a terminal device and a display device is further provided in an embodiment of the present application, including:

step S10, the terminal device end: and when receiving the screen projection operation of the user on the media, sending a screen projection request to the display equipment. The screen projection request at least comprises equipment information of the terminal equipment.

Step S20, the display device side: when receiving the screen projection request, establishing screen projection connection with the terminal equipment, and feeding back authorized information to the terminal equipment.

Step S30, the display device side: and adding one to the accumulated screen throwing number.

Step S40, the terminal device end: and when the authorized information is received, the screen throwing data and the media configuration information are sent to the display equipment.

In some embodiments, after receiving the screen-throwing request, the display device may query whether the terminal device has the screen-throwing authority. If the terminal equipment has the screen throwing authority, the display equipment feeds back the authorized information to the terminal equipment; if the terminal equipment does not have the screen-throwing authority, the display equipment feeds back unauthorized information to the terminal equipment, so that after the terminal equipment receives the unauthorized information, the screen-throwing data and the media configuration information are not sent. The method of inquiring the screen-throwing authority of the terminal device by the display device is not limited to inquiring whether the user authorizes the terminal device or not through a UI popup window, or inquiring a trusted device list maintained by the display device.

Step S50, the display device side: and receiving screen throwing data and media configuration information sent by the terminal equipment, and inquiring the screen throwing number M of the current counting. M is greater than or equal to 1.

Step S60, the display device side: and acquiring media configuration data correspondingly transmitted by M terminal devices which are accessed by the screen at present. The media configuration data should include at least aspect ratio examples of the media video through which the screen orientation information can be obtained.

Step S70, the display device side: and calculating M groups of size information according to the screen resolution or the display screen directions and aspect ratio of M terminal equipment screen-throwing media and the screen-throwing path number M.

Step S80, the display device side: and constructing M screen throwing windows according to the M groups of size information. The drop window is consistent with the media it displays in terms of screen orientation and aspect ratio.

Step S90, the display device side: position distribution information is calculated based on the screen resolution and M sets of size information. The position distribution information comprises coordinates of preset reference points on the M screen throwing windows, and the coordinates of the preset reference points are used for positioning the positions of the screen throwing windows on the screen.

Step S100, the display device side: and positioning and arranging M screen throwing windows according to the position distribution information to obtain a layout. Alternatively, if a layout of multiple versions is obtained, each version may be screened based on the principle of maximum area, and a target layout with the largest sum of areas of the screen-throwing windows may be output.

Step S110, the display device side: and displaying the layout in a screen-throwing interface, and displaying screen-throwing data sent by the corresponding terminal equipment in each screen-throwing window in the layout. Optionally, the screen throwing window has a one-to-one correspondence with the device ID of the terminal device, so that the display device accurately throws the screen throwing data into the designated screen throwing window for display.

Step S120, the display device side: and monitoring the screen-throwing path number in real time in the process of displaying the screen-throwing interface.

Step S130, the display device side: and updating the layout when detecting that the screen-throwing path number is changed.

Step S140, the display device side: and when the screen-throwing path number is detected to be reduced to zero, the screen-throwing interface is exited.

According to the technical scheme, dynamic self-adaptive layout matching of the screen throwing interfaces is achieved based on the screen throwing path number, aspect ratio of the multi-path screen throwing media, screen directions and other configuration information, so that typesetting and layout of M screen throwing windows are more adaptive to the screen, the multi-path screen throwing windows are displayed in the largest range in the screen, the distribution of the screen throwing windows is more compact, and the white left of the screen throwing interfaces is reduced. In addition, the attractiveness of the display of the screen-throwing interface can be considered, for example, the uniformity, symmetry and relative balance of the size among the screen-throwing windows can be considered, the size and the position distribution of each screen-throwing window can be set, the display effect of the screen-throwing interface can be remarkably improved through the obtained layout, and the user experience of watching the screen is further improved.

In this description, the same and similar parts among the display device embodiment, the UI embodiment and the method embodiment may be referred to each other, and the relevant contents are not repeated. The UI presentation of the layout in this application is merely exemplary, and is specific to the actual design and application.

It will be apparent to those skilled in the art that the techniques of embodiments of the present invention may be implemented in software plus a necessary general hardware platform. In a specific implementation, the present invention also provides a computer storage medium, where the program may be stored. When the computer storage medium is located in the display device 200, the program may include the aforementioned multi-pass screen display method that the controller 250 is configured to perform when executed. The computer storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or the like.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

The foregoing description, for purposes of explanation, has been presented in conjunction with specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed above. Many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles and the practical application, to thereby enable others skilled in the art to best utilize the embodiments and various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A display device, characterized by comprising:

the display is used for displaying a screen-throwing interface;

a controller configured to perform:

receiving a screen projection request sent by first terminal equipment, wherein the screen projection request comprises equipment information of the first terminal equipment;

if the trusted equipment list comprises the equipment information of the first terminal equipment, establishing screen-throwing connection with the first terminal equipment through the communicator; the trusted device list comprises device information with screen throwing authority;

adding one to the M value, wherein M represents the screen-throwing path number recorded currently, and M is more than or equal to 1;

According to media configuration information respectively received from M terminal devices connected with the screen, matching layout among M screen throwing windows adapted to the screen; wherein the media configuration information comprises aspect ratio of video projected by the terminal equipment;

controlling a display to display M screen-throwing windows in a screen-throwing interface according to the layout, and displaying screen-throwing data sent by corresponding terminal equipment in each screen-throwing window;

if the trusted equipment list comprises equipment information of the target terminal equipment, newly establishing screen-throwing connection with the target terminal equipment, accumulating the screen-throwing path number by one, and subtracting the screen-throwing path number by one when the screen-throwing connection with one terminal equipment is detected to be disconnected;

when the screen-throwing path number is detected to be changed, dynamically updating the layout according to the changed M value and media configuration information from M terminal devices which currently maintain the screen-throwing connection state;

and when the screen-throwing path number is detected to be reduced to zero, controlling the display to withdraw from the screen-throwing interface, and displaying a user interface before screen throwing.

2. The display device of claim 1, wherein the controller is configured to match the layout as follows:

3. The display device according to any one of claims 1-2, wherein the controller is further configured to perform:

Screening out the target layout with the largest sum of the areas;

4. The multipath screen projection display method is characterized by comprising the following steps of:

if the trusted equipment list comprises the equipment information of the first terminal equipment, establishing screen-casting connection with the first terminal equipment through a communicator; the trusted device list comprises device information with screen throwing authority;

matching the layout among M screen throwing windows adapted to a screen according to media configuration information respectively received from M terminal devices connected by the screen throwing, wherein the media configuration information comprises aspect ratio of video thrown by the terminal devices;

displaying M screen throwing windows in the screen throwing interface according to the layout, and displaying screen throwing data sent by corresponding terminal equipment in each screen throwing window;

5. The method of claim 4, wherein the layout is matched as follows:

6. The method according to any one of claims 4 to 5, further comprising:

screening out the target layout with the largest sum of the areas;

and displaying the target layout in the screen projection interface.