WO2022228021A1 - 显示设备及多设备投屏同屏显示的控制方法 - Google Patents

显示设备及多设备投屏同屏显示的控制方法 Download PDF

Info

Publication number
WO2022228021A1
WO2022228021A1 PCT/CN2022/084370 CN2022084370W WO2022228021A1 WO 2022228021 A1 WO2022228021 A1 WO 2022228021A1 CN 2022084370 W CN2022084370 W CN 2022084370W WO 2022228021 A1 WO2022228021 A1 WO 2022228021A1
Authority
WO
WIPO (PCT)
Prior art keywords
screen
display
screen projection
casting
devices
Prior art date
Application number
PCT/CN2022/084370
Other languages
English (en)
French (fr)
Inventor
贾亚洲
马晓燕
刘晋
王之奎
吴汉勇
初德进
张安祺
张永芳
陈俊宁
李斌
崔文华
齐消消
王秉清
Original Assignee
海信视像科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 海信视像科技股份有限公司 filed Critical 海信视像科技股份有限公司
Priority to CN202280031755.0A priority Critical patent/CN117581191A/zh
Publication of WO2022228021A1 publication Critical patent/WO2022228021A1/zh
Priority to US18/384,618 priority patent/US20240053944A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1601—Constructional details related to the housing of computer displays, e.g. of CRT monitors, of flat displays
    • G06F1/1605—Multimedia displays, e.g. with integrated or attached speakers, cameras, microphones
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14—Display of multiple viewports
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/485—End-user interface for client configuration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04803—Split screen, i.e. subdividing the display area or the window area into separate subareas
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/0492—Change of orientation of the displayed image, e.g. upside-down, mirrored
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/12—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels

Definitions

  • the present application relates to display device technology, and in particular, to a display device and a control method for multi-device projection and display on the same screen.
  • the mobile device can display the currently displayed content of the screen on the user interface of the display device based on a certain screen projection protocol, where the content output by the mobile device includes various media information and real-time operation pictures. Because the current display device can only allow one mobile device to perform screen projection at the same time, that is, the user interface can only display the screen content of one mobile device, and cannot display the screen projection of multiple mobile devices on the same screen.
  • Some embodiments of the present application provide a display device, including:
  • a display configured to display a user interface, the user interface includes a plurality of windows; a controller, the controller is configured to: in response to a screen projection request of the plurality of screen projection devices, communicate with the plurality of screen projection devices Establish a connection, wherein multiple screencasting devices support the same screencasting protocol; receive screencasting content sent by multiple screencasting devices; the screencasting content of the last connected screencasting device is a vertical media asset control multiple windows to display the screencasting content of the screencasting devices correspondingly; when the screencasting content of the screencasting device accessed last is a horizontal media asset, control the display to display the last access in full screen The screen projection content of the screen projection device.
  • Some embodiments of the present application provide a display device, including:
  • a display configured to display a user interface, the user interface includes a plurality of windows; a controller, the controller is configured to: establish a connection with a plurality of screen projection devices respectively by using a plurality of screen projection services, so that the display device Receive screen projection content from multiple screen projection devices, wherein the multiple screen projection devices support different screen projection protocols; when the current screen projection mode is the split-screen mode, the screen projection contents of the multiple screen projection devices are divided into multiple Each of the windows is displayed; when the current screencasting mode is the smart mode, if the screencasting content of the last connected screencasting device is a vertical media asset, the screencasting content of the plurality of screencasting devices will be displayed. Display in a plurality of said windows respectively, if the screencast content of the last accessed screencasting device is a horizontal media asset, the screencast content of the last accessed screencasting device will be displayed in full screen on the display exhibit.
  • the controller is further configured to: when the user interface includes a window, control the display to display the screen projection content of the last connected screen projection device in full screen.
  • the data channel is used for data interaction between the screen projection device and the display device.
  • the controller is further configured to: when the different screen-casting devices correspond to different members in the group, control the screen-casting device to bind a corresponding device identifier, where the device identifier is used for the zone different members of the group.
  • the plurality of windows when controlling a plurality of windows to display the screen projection contents of a plurality of the screen projection devices respectively, if the displayed content of the user interface is retained, the plurality of windows are controlled to display the user interface and a plurality of corresponding screen projection devices respectively.
  • the screen projection content of the screen projection device if the content displayed on the user interface is not retained, multiple windows are controlled to display the screen projection content of the screen projection device correspondingly.
  • the current user interface When the current user interface supports split-screen display, start a corresponding number of windows, and push the user interface and the screen-casting content of the multiple screen-casting devices to the multiple windows respectively through the activity manager interface; the current user interface does not support During split-screen display, a corresponding number of windows are activated, and the proportionally reduced user interface and the screen-casting contents of the plurality of screen-casting devices are respectively pushed to the plurality of the windows through the activity manager interface.
  • the controller when controlling a plurality of windows to display screen projection contents of a plurality of the screen projection devices correspondingly, the controller is further configured to:
  • the controller is further configured to reset the coordinates, width and height of the user interface when the user interface is scaled down, so that the user interface is scaled down.
  • Some embodiments of the present application provide a method for controlling multi-device screen projection and displaying on the same screen.
  • the method includes: in response to a screen projection request from multiple screen projection devices, establishing a connection with a plurality of the screen projection devices, wherein, Multiple screencasting devices support the same screencasting protocol; receive screencasting content sent by multiple screencasting devices; control multiple screencasting devices when the screencasting content of the last connected screencasting device is a vertical media asset
  • the windows respectively display the screen projection contents of a plurality of the screen projection devices; when the screen projection content of the last connected screen projection device is a horizontal media asset, control the display to display the last connected screen projection screen in full screen The projected content of the device.
  • Some embodiments of the present application provide a method for controlling multi-device screen projection and displaying on the same screen.
  • the method includes: using multiple screen projection services to establish connections with multiple screen projection devices respectively, so that the display device receives multiple projection screens.
  • the screen projection content of the device wherein multiple screen projection devices support different screen projection protocols; when the current screen projection mode is the split-screen mode, the screen projection contents of the plurality of screen projection devices are displayed in multiple windows respectively;
  • the current screencasting mode is the smart mode, if the screencasting content of the last connected screencasting device is a vertical media asset, the screencasting contents of the plurality of screencasting devices are respectively placed in the multiple windows. Displaying is performed, and if the screencasting content of the last accessed screencasting device is a horizontal media asset, the screencasting content of the last accessed screencasting device is displayed in full screen on the display.
  • FIG. 1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments
  • FIG. 2 is a block diagram of a hardware configuration of the control apparatus 100 according to some embodiments.
  • FIG. 3 is a block diagram of a hardware configuration of a display device 200 according to some embodiments.
  • FIG. 4 is a schematic diagram of software configuration in the display device 200 according to some embodiments.
  • 5a is a schematic diagram of a display effect of a two-split screen in a split-screen mode according to some embodiments
  • 5b is a schematic diagram of a display effect of a three-split screen in a split-screen mode according to some embodiments
  • 5c is a schematic diagram of a display effect of a four-split screen in a split-screen mode according to some embodiments.
  • 5d is a schematic diagram of a display effect of switching windows in a split-screen mode according to some embodiments.
  • 6a is a schematic diagram of a display effect of switching a split-screen mode by pressing a button according to some embodiments
  • 6b is a schematic diagram of a display effect of switching a split-screen mode by switching options according to some embodiments
  • FIG. 6c is a schematic diagram of a display effect of automatically switching a split-screen mode according to some embodiments.
  • FIG. 6d is a schematic diagram of a display effect of another automatic switching split-screen mode according to some embodiments.
  • FIG. 6e is a schematic diagram of a display effect of touch event distribution according to some embodiments.
  • 6f is a timing diagram of touch event distribution according to some embodiments.
  • Figure 6g is a timing diagram of input event distribution according to some embodiments.
  • FIG. 7 is a schematic diagram of a screen projection process of a multi-screen projection device under the same screen projection protocol according to some embodiments.
  • FIGS. 8a-8b are schematic diagrams of establishing a connection between a multi-screen projection device and a display device under the same screen projection protocol according to some embodiments;
  • 9a is a schematic diagram of a screen projection process of a multi-screen projection device under different screen projection protocols according to some embodiments.
  • Figure 9b is a schematic diagram of an application layer hub module according to some embodiments.
  • FIG. 10 is a schematic diagram of a display effect of single-device screen projection in full-screen mode according to some embodiments
  • 11-12 are schematic diagrams of display effects of single-device screen projection in split-screen mode according to some embodiments.
  • FIG. 13 is a schematic diagram of a display effect of single-device screen projection in smart mode according to some embodiments.
  • FIG. 14 is a schematic diagram of establishing a connection between a multi-screen projection device and a display device under the same screen projection protocol according to some embodiments;
  • 15-16 are schematic diagrams of display effects of dual-device screen projection in a split-screen mode according to some embodiments.
  • 17 is a schematic diagram of a display effect of four-device screen projection in a split-screen mode according to some embodiments.
  • FIG. 18 is a schematic diagram of a selected window in a split-screen mode according to some embodiments.
  • 19 is a schematic diagram of the effect of screen projection by multiple screen projection devices under the same screen projection protocol according to some embodiments.
  • 20 is a schematic diagram of the effect of screen projection by a multi-screen projection device under different protocols according to some embodiments
  • 21 is a schematic diagram of establishing a connection between a screen projection device and a display device according to some embodiments.
  • 22 is a schematic diagram of a focus transfer flow for switching windows in a split-screen mode according to some embodiments.
  • FIG. 23 is a timing diagram of a stage of a display device in a split screen mode according to some embodiments.
  • FIG. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment.
  • a user can operate the display device 200 through the smart device 300 or the control device 100 .
  • the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-range communication methods, and the display device 200 is controlled wirelessly or wiredly.
  • the user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, and the like.
  • a smart device 300 can also be used to control the display device 200 .
  • the display device 200 is controlled using an application running on the smart device.
  • the display device 200 can also be controlled in a manner other than the control apparatus 100 and the smart device 300.
  • the module for acquiring voice commands configured inside the display device 200 can directly receive the user's voice command for control.
  • the user's voice command control can also be received through a voice control device provided outside the display device 200 device.
  • the display device 200 is also in data communication with the server 400 .
  • the display device 200 may be allowed to communicate via local area network (LAN), wireless local area network (WLAN), and other networks.
  • the server 400 may provide various contents and interactions to the display device 200 .
  • the server 400 may be a cluster or multiple clusters, and may include one or more types of servers.
  • FIG. 2 exemplarily shows a configuration block diagram of the control apparatus 100 according to an exemplary embodiment.
  • the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface 140 , a memory, and a power supply.
  • the control device 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the display device 200 can recognize and respond to, and play an intermediary role between the user and the display device 200 .
  • FIG. 3 is a block diagram showing a hardware configuration of the display apparatus 200 according to an exemplary embodiment.
  • the display device 200 includes a tuner demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply At least one of a power supply and a user interface.
  • the display 260 includes a display screen component for presenting a picture, and a driving component for driving the image display, for receiving the image signal output from the controller, for displaying the video content, the image content and the menu manipulation interface Components and user manipulation UI interface.
  • the display 260 may be a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.
  • the controller 250 controls the operation of the display device and responds to user operations.
  • the controller 250 controls the overall operation of the display apparatus 200 .
  • the controller 250 may perform an operation related to the object selected by the user command.
  • the object may be any of the selectable objects, such as hyperlinks, icons, or other operable controls.
  • the operations related to the selected object include: displaying operations connected to hyperlinked pages, documents, images, etc., or executing operations of programs corresponding to the icons.
  • the user may input user commands on a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the graphical user interface (GUI).
  • GUI graphical user interface
  • the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.
  • a "user interface” is a medium interface for interaction and information exchange between an application program or an operating system and a user, which enables conversion between an internal form of information and a form acceptable to the user.
  • the commonly used form of user interface is Graphical User Interface (GUI), which refers to a user interface related to computer operations displayed in a graphical manner. It can be an icon, window, control and other interface elements displayed on the display screen of the electronic device, wherein the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, etc. visual interface elements.
  • GUI Graphical User Interface
  • FIG. 4 is a schematic diagram of software configuration in the display device 200 according to some embodiments.
  • the system is divided into four layers, from top to bottom, the applications layer (referred to as “application layer”) ”), the Application Framework layer (referred to as the “framework layer”), the Android runtime (Android runtime) and the system library layer (referred to as the “system runtime layer”), and the kernel layer.
  • at least one application program runs in the application program layer, and these application programs can be a window (Window) program, a system setting program, or a clock program that comes with the operating system;
  • the framework layer provides an application programming interface for the application program. (application programming interface, API) and programming frameworks.
  • the application framework layer includes some predefined functions.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer at least includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, etc.
  • the user can perform split-screen processing on the displayed images during the use of the display device 200 , that is, when the display device 200 enters the split-screen mode, the display 260 forms multiple windows.
  • the display 260 of the display device 200 can simultaneously display two or more windows in the user interface, and each window can independently display different contents, so that the user can watch different program images at the same time.
  • FIG. 5a is a schematic diagram showing the display effect of the two-split screen in the split-screen mode according to some embodiments.
  • the display 260 can display a user interface one Divided into two, a first window and a second window are displayed, wherein the first window can be used to display the playback picture of the first media resource, and the second window can be used to display the playback picture of the second media resource.
  • the two-split screen may also be referred to as a double-split screen.
  • 5b is a schematic diagram of a display effect of a three-split screen in a split-screen mode according to some embodiments.
  • the display 260 can divide the user interface into There are three areas, respectively displaying a first window, a second window and a third window, wherein the first window can be used to display the playback screen of the first media asset, and the second window can be used to display the playback of the second media asset screen, and the third window can be used to display the playback screen of the third media resource.
  • FIG. 5c is a schematic diagram of a display effect of a four-split screen in a split-screen mode according to some embodiments.
  • the display 260 can display the user
  • the interface is divided into four areas, which display the first window, the second window, the third window and the fourth window respectively.
  • the application list is displayed in the first window, the application list is displayed in the second window, and the application list is displayed in the Deploy related controls for multiple applications.
  • the play screen of the third media asset is displayed in the third window, and the play screen of the fourth media asset is displayed in the fourth window. That is to say, in the split-screen mode, different windows can display different types of interfaces at the same time, and the windows do not interfere with each other.
  • FIG. 6a is a schematic diagram showing the display effect of switching the split-screen mode by pressing a button according to some embodiments.
  • a split-screen mode switching button may be set on the control device 100 supporting the display device 200.
  • the display device 200 can be controlled to enter or exit the split-screen mode, that is, the display 260 of the display device 200 can be controlled to display multiple windows or a single window.
  • a split-screen mode switching option can also be set in the UI interface of the display device 200, and when the user clicks the switching option, the display device 200 can be controlled to enter or exit the split-screen mode.
  • FIG. 6b is a schematic diagram of the display effect of switching the split-screen mode by switching options according to some embodiments. As shown in FIG. 6b, it exemplarily shows a schematic diagram of the display effect of switching the split-screen mode by switching options according to some embodiments.
  • a split-screen mode switching option can also be set in the UI interface of the display device 200, and when the user clicks the switching option, the display device 200 can be controlled to enter or exit the split-screen mode.
  • the user may control the display device 200 to enter or exit the split screen mode by voice. For example, after the user inputs the wake-up word "Hi! XX", and inputs a voice instruction of "switch to split screen mode" to the display device 200, the display device 200 can enter the split screen mode.
  • the display device 200 can also detect the usage status of the user in real time, and determine whether the user has a requirement for multi-window display, so as to intelligently switch the split-screen mode or provide a split-screen mode switching option according to the user's requirement.
  • FIG. 6c is a schematic diagram of a display effect of automatic switching of split-screen mode according to some embodiments. As shown in FIG. 6c, in the process of displaying the media asset screen on the display device 200, if there is a video chat request, it can automatically switch to split-screen mode. screen mode, so that the display device 200 can display the media asset picture through the first window; meanwhile, display the video chat picture through the second window.
  • FIG. 6c is a schematic diagram of a display effect of automatic switching of split-screen mode according to some embodiments. As shown in FIG. 6c, in the process of displaying the media asset screen on the display device 200, if there is a video chat request, it can automatically switch to split-screen mode. screen mode, so that the display device 200 can display
  • 6d is a schematic diagram of a display effect of another automatic switching split-screen mode according to some embodiments.
  • multiple screens may also be displayed on the right side of the user interface.
  • Recommended program options after the user selects any recommended program option and confirms it, the user can control the display device 200 to enter the split screen mode, that is, the original media playback screen is displayed in the first window, and the playback screen corresponding to the recommended program is displayed in the second window.
  • the control device 100 of the display device 200 includes a touch screen, a remote control, a mouse, etc.
  • the touch screen or the mouse calculates the position based on coordinates, and the click event in Android is Touch (touch) event, the touch screen or the mouse can distribute the touch event to the application corresponding to the position, so that the focus is transferred to the corresponding application.
  • FIG. 6e is a schematic diagram of a display effect of touch event distribution according to some embodiments. As shown in FIG. 6e, the position of the current focus is on the second window, and the touch event points to the first window, then the display device 200 touch events are distributed to the first window.
  • FIG. 6f is a sequence diagram of touch event distribution according to some embodiments. As shown in FIG.
  • FIG. 6f is a sequence diagram of input event distribution according to some embodiments. With reference to FIG. 6g, the display device 200 receives an Input event.
  • the Input event has no coordinate information but only key value information, so the input trigger will obtain the current focus from the system service.
  • the task stack where the second window is located when it is queried that the current focus is on the second window application, the key is distributed to the task stack where the second window is located.
  • the display device 200 can also detect the usage status of the user in real time, and determine whether the user has a requirement for multi-window display, so as to intelligently switch the split-screen mode or provide a split-screen mode switching option according to the user's requirement. For example, when the display device 200 is displaying the media asset screen, if there is a video chat request, it can automatically switch to the split-screen mode, so that the display device 200 can display the media asset screen through the first window; meanwhile, display the video through the second window. Chat screen.
  • the display device 200 is provided with a screen-casting application, and the screen-casting application is connected to the screen-casting device based on a certain screen-casting protocol, so that the user can display the screen-casting content of the screen-casting device on the display using the screen-casting protocol.
  • a display device may, based on the same screen-casting service, allow multiple screen-casting devices that support the same screen-casting protocol to perform screen-casting to the same display device, and simultaneously display the projections of multiple screen-casting devices on the same user interface. screen interface.
  • the screen projection device may be a mobile phone, a tablet computer, a computer, a laptop computer, a smart TV, or the like.
  • the controller is configured to perform a control process of multi-device mirroring and displaying on the same screen.
  • the controller 250 establishes connections with a plurality of the screen projection devices in response to the screen projection requests of the plurality of screen projection devices, wherein the plurality of screen projection devices support the same screen projection protocol.
  • the controller 250 receives the screen projection content sent by the plurality of screen projection devices, and determines the form of the screen projection content.
  • the controller 250 determines the form of the subsequent screen projection display according to the form of the screen projection content of the last connected screen projection device.
  • the form of the screen projection content refers to vertical media assets and horizontal media assets. Form refers to multi-window display and full-screen display.
  • the display device 200 is provided with a screen-casting mode selection control in the screen-casting application. After the user clicks the selection control, the currently required screen-casting mode can be selected.
  • the controller 250 controls multiple windows to display a plurality of the screen-casting devices correspondingly. The projected content of the device.
  • the controller 250 controls the display to display the screen projection content of the last connected screen projection device in full screen.
  • the screen-casting application in the display device 200 also includes a full-screen mode and a split-screen mode.
  • the full-screen mode means that the display device 200 controls the display 260 to display the screen-casting content of the last-connected screen-casting device in full screen.
  • the split-screen mode means that regardless of whether the screencast content of the last connected screencasting device is in the form of horizontal media assets or vertical media assets, the screencasting contents of multiple screencasting devices are divided into multiple said windows. to show.
  • a screen-casting application in the display device 200 is provided with a screen-casting mode selection control.
  • the selection control displays a selection list, and the selection list is set with full-screen mode, Split screen mode and smart mode, users can select the current screen projection mode through the selection list.
  • the split-screen mode in the screen projection mode here refers to the split-screen mode for multi-window display in the display device 200 .
  • FIG. 7 exemplarily shows a schematic diagram of a screen projection process of a multi-screen projection device under the same screen projection protocol according to some embodiments.
  • two screen projection devices are used as an example for description.
  • the screen-casting device A and the screen-casting device B establish a related protocol connection with the display device 200 through the same screen-casting protocol to perform screen-casting-related operations.
  • relevant protocol information is carried in the communication process.
  • the screen projection device can obtain the device information and network information of the display device 200 .
  • the display device 200 monitors the connection between the screen projection devices A and B.
  • the display device 260 is displayed on the display 260 in full screen mode, split screen mode or smart mode according to the setting of the screen projection mode.
  • the display device 200 receives screen projection requests from a plurality of the screen projection devices through the same screen projection service, performs client Group (group) management, and divides instruction channels and data channels to undertake business services.
  • the channel carries out the interaction of related protocols, and the transmission of the video stream is carried out through the data channel.
  • the display device 200 monitors the connection of the screen projection devices by using the screen projection service, and performs group management on the screen projection devices, wherein different screen projection devices correspond to different members in the group.
  • the screencasting service receives the screencasting request from the screencasting device, it establishes a client group for the corresponding screencasting device.
  • Different screencasting devices correspond to different members of the client group, and each member has a corresponding screencasting device. information and protocol interaction information.
  • the controller 250 controls the screen projection devices to bind corresponding device identifiers, wherein the device identifiers are used to distinguish different members in the group.
  • the client group and split-screen processing center processing can realize that under the same screencasting protocol, multiple screencasting devices can display the screencasting interface in different Activity stacks in parallel. and do not interfere with each other.
  • FIG. 8a-8b are schematic diagrams of establishing connections between multiple screen projection devices and display devices under the same screen projection protocol according to some embodiments.
  • the display device 200 creates a client group for the screen projection service, and the screen projection service requires Create a corresponding sub-screening channel according to the number of screen-casting devices that send screen-casting requests. After receiving a screen-casting request from a screen-casting device, add a sub-member to the current client group.
  • FIG. 8a different screen projection devices correspond to different members of the client group, and each member has corresponding screen projection device information and protocol interaction information. As shown in FIG.
  • FIG. 9a is a schematic diagram of a screen projection process of multiple screen projection devices under different screen projection protocols according to some embodiments.
  • the display device 200 receives screen projection requests from a plurality of the screen projection devices respectively through different screen projection services.
  • Screen-casting device A and screen-casting device B establish relevant protocol connections with the display device through different screen-casting protocols to perform screen-casting-related operations.
  • relevant protocol information is carried in the communication process. For example, based on NFC (Near Field Communication), the screen projection device can obtain the device information and network information of the display device 200.
  • NFC Near Field Communication
  • the display device 200 receives the screen-casting request for monitoring the screen-casting device through screen-casting services of different screen-casting protocols, monitors the connection of the screen-casting device by using the screen-casting service, and builds an instruction channel and a data channel, wherein the instruction channel uses For instruction interaction between the screen projection device and the display device, the data channel is used for data interaction between the screen projection device and the display device.
  • the screencasting service A in the display device 200 receives the screencasting request from the screencasting device A
  • the screencasting service B receives the screencasting request from the screencasting device B.
  • the screencasting service is responsible for monitoring the connection of the screencasting device and negotiating with the screencasting device.
  • the display device 200 can receive the request sent by the user through the screen projection device, and through the data channel, the display device 200 can receive related video data and the like sent by the screen projection device.
  • the screen projection device establishes a connection with the display device 200 .
  • the display device 260 is displayed on the display 260 in full screen mode, split screen mode or smart mode according to the setting of the screen projection mode.
  • the display device 200 discovers the screen-casting device through screen-casting services of different protocols, and connects the screen-casting device after discovering the screen-casting device.
  • the controller 250 uses the NFC protocol service to receive support for the NFC protocol The screencasting request of the screencasting device, and use the NFC protocol service to monitor the connection of the screencasting device.
  • the controller 250 uses the Airplay protocol service to receive a screen projection request from a screen projection device supporting the Airplay protocol, and uses the Airplay protocol service to monitor the connection of the screen projection device.
  • the controller 250 uses the DLNA protocol service to receive a screen projection request from a screen projection device that supports the DLNA protocol, and uses the DLNA protocol service to monitor the connection of the screen projection device.
  • NFC screencasting is based on the NFC protocol for two-way device discovery and connection
  • Airplay screencasting is based on the Airplay protocol for two-way device discovery and connection
  • DLNA push is based on the DLNA protocol for screencasting device discovery and connection
  • other protocols are also In this way, the screen-casting devices are discovered and connected through the agreement of their respective protocols.
  • Airplay screen projection refers to a wireless technology developed by Apple, which can wirelessly transmit pictures, audio, and video on iOS devices such as mobile phones and tablets to devices that support AirPlay through WiFi.
  • Fig. 9b is a schematic diagram of an application layer hub module according to some embodiments.
  • the application layer hub module can manage two aspects. The first is connection management, which is responsible for sorting out the data of the screen-casting devices to which the display device 200 performs screen-casting connection, so as to prepare for the final display effect of the video stream of the screen-casting device. The second is video stream management, which is responsible for classifying and decoding video streams from different projection protocols.
  • the screen projection device A after the screen projection device A establishes a connection with the display device 200, it notifies the application layer hub module, so that the application layer hub module can perform connection management.
  • the application layer hub module sorts out all the currently connected device members, and then combines the user's setting of the screencasting mode to activate the split-screen processing center as required to perform different screencasting forms. presentation.
  • FIG. 10 is a schematic diagram showing the display effect of single-device screen projection in full-screen mode according to some embodiments.
  • the screen-casting device A passes the data channel After sending the screencast content to the display device 200, the display device 200 can directly display the screencast content on the display 260 in full screen.
  • the screencast content of the screencast device A is displayed on the display 260 in the full screen mode , and the screen projection content may be the operation interface of the screen projection device A.
  • the display device 200 when the display device 200 determines that the current screen projection mode is the split-screen mode, it can further determine whether to display the current user interface of the display device 200 synchronously according to the needs of the user, that is, to display the current user interface of the display device 200 synchronously.
  • the TOP (top-level) application (or foreground application) and the screen-casting interface of the screen-casting device are placed on the user interface displayed on the display 260 at the same time. For example, as shown in FIG. 10 , when the screencast content transmitted by screencasting device A is vertical media assets, and the screencasting interface displayed by screencasting device A is also vertical, when the display device 200 displays, there will be The black frame makes the user experience poor.
  • the TOP application of the display device 200 is displayed on the display 260 at the same time, the user can operate the TOP application and view the screen projection interface of the screen projection device through the display device 200, thereby improving user experience.
  • the screen projection content of multiple screen projection devices will be displayed in multiple windows respectively, that is to say, if the user does not need to display the TOP application synchronously, then Only display the screencast content of the screencasting device.
  • split-screen mode if the user wants to display the current user interface and the screen-casting interface simultaneously, it is necessary to first determine whether the current user interface supports split-screen mode through the split-screen processing center.
  • the current user interface supports split-screen display, a corresponding number of windows are activated, and the user interface and the screen projection contents of the plurality of screen projection devices are respectively pushed to the plurality of windows through the activity manager interface.
  • the current user interface does not support split-screen display, a corresponding number of windows are activated, and the proportionally reduced user interface and the screen projection content of the plurality of screen projection devices are respectively pushed to the plurality of windows through the activity manager interface.
  • the split screen processing center starts the display device 200 to enter the two split screen in split screen mode according to the number of connected screen projection devices obtained from the application layer central module, and calls the The system split-screen interface places the TOP application in the selected window and displays the screen-casting interface of the screen-casting device in another window.
  • 11-12 are schematic diagrams of display effects of single-device screen projection in split-screen mode according to some embodiments.
  • the TOP application and the screen projection interface of screen projection device A are displayed in two windows.
  • the split-screen processing center can forcibly scale down the current TOP application, as shown in Figure 12, place the scaled-down TOP application in one of the windows for display, and cast the screen.
  • the screen projection interface of the device is displayed in another window.
  • the screen-casting interface is restarted, and when the current user interface supports split-screen, the split-screen processing center first divides the screen into two halves, and converts the current user The position of the interface is set to the left, and the user interface is scaled down by resetting the coordinates, width and height of the user interface. At the same time, the position of the screen projection content of the screen projection device A is set to the right side, so that the split screen processing center starts the display device. 200 Enters a two-split screen in split-screen mode.
  • the user clicks a video in a split screen to play it in full screen, and at this time, the video is played in full screen in the split screen window to which it belongs.
  • the display device 200 simultaneously displays the TOP application and the screen projection interface in the user interface displayed by the display 260, as shown in FIG. 11, if the user clicks a certain video in the screen projection interface of the screen projection device A to perform For full-screen playback, the video is played in full-screen in the window where the screen-casting interface of screen-casting device A is located.
  • the display device 200 when the display device 200 determines that the current screen projection mode is the smart mode, the display device 200 will receive the screen projection content sent by the screen projection device, and determine the current projection content according to the display ratio of the current screen projection content.
  • the optimal display ratio of the screen content When the screencast content sent by the screencasting device A is a horizontal media asset, that is, when the screencasting data is a horizontal code stream, if a certain video has been pushed, at this time, in order to ensure the user's visual experience, the display device 200 will perform the screencasting. Content plays in full screen. 13 is a schematic diagram of the display effect of single-device screen projection in smart mode according to some embodiments. As shown in FIG.
  • the screen projection content sent by the screen projection device A is a horizontal media resource
  • the display device 200 controls the display 260 to full screen Show the property.
  • the screencast content sent by screencasting device A is a vertical media asset, that is, when the screencasting data is a vertical stream, such as a mobile phone operation page
  • the screencasting data is displayed in full screen, it will be displayed on both sides of the screencasting interface.
  • a black frame will be displayed.
  • the display device 200 chooses to enter the split-screen mode, only displays the screen projection interface in one window, and displays the current user interface of the display device in another window, that is, Simultaneously display the current user interface and the projection interface.
  • FIG. 14 is a schematic diagram of establishing a connection between a multi-screen projection device and a display device under the same screen projection protocol according to some embodiments. As shown in FIG. 14 , the display device 200 successfully accesses the screen projection device A and the screen projection device B.
  • the display device 200 after the display device 200 receives a screen-casting request from a screen-casting device A and establishes a connection with it, it first performs screen-casting display according to the above-mentioned screen-casting state, and then the display device 200 receives a second screen-casting request again.
  • the application-layer hub module After the screen-casting request from the screen-casting device B and the connection is established with it, the application-layer hub module is notified so that the application-layer hub module can perform connection management.
  • the application layer hub module sorts out all the currently connected device members, starts the split-screen processing center, and presents the following different screencasting forms.
  • the display device 200 determines that the current screen projection mode is the full screen mode through the screen projection mode setting, the screen projection content of the last connected screen projection device is displayed on the display. Display in full screen. That is, after the screencasting device B sends the screencasting content to the display device 200 through the data channel, the display device 200 can directly disconnect the previous screencasting, and directly display the screencasting content sent by the screencasting device B on the display 260 in full screen. . That is to say, the screen-casting interface of the previously connected screen-casting device A is disconnected first, and the user interface displayed on the display 260 is updated to the screen-casting interface of the currently connected screen-casting device B.
  • the display device 200 determines that the current screen projection mode is the split-screen mode through the screen projection mode setting, it can be determined whether it is necessary to synchronously display the user of the display device 200 according to the needs of the user. content displayed on the interface.
  • the display device 200 needs to first determine whether the content displayed on the user interface supports the split-screen mode through the split-screen processing center.
  • the split screen processing center starts the display device 200 to enter the three split screen by applying split screen layout according to the number of connected screen projection devices obtained from the application layer hub module.
  • the display device 200 is divided into three screens, which are divided into a first window, a second window and a third window.
  • the TOP application on the display device side is placed in the first window and the screen projection interface of the screen projection device A is placed In the third window and the screen projection interface of the screen projection device B is placed in the second window for display.
  • the split-screen processing center can forcibly scale down the displayed content of the user interface, and place it in one of the windows for display.
  • the interface is displayed on the other two windows respectively.
  • the display effect is not limited to the method shown in Figure 15, and other types of layout can also be performed.
  • the screen projection interface of screen projection device A is placed in the first window and screen projection device B.
  • the screen projection interface is placed in the third window and the TOP application is placed in the second window for display.
  • the display device 200 determines that the current screen projection mode is the smart mode through the screen projection mode setting, the display device 200 receives the screen projection content sent by the screen projection device B, and According to the display ratio of the screen projection content of the last connected screen projection device, the optimal display ratio of the screen projection content of the screen projection device B is determined.
  • the screen projection content sent by the screen projection device B is a horizontal code stream, such as pushing a certain video, in order to ensure the user's visual experience, the display device 200 plays the screen projection content in full screen at this time.
  • the display device 200 can make full use of the display area of the display 260, choose to enter the split-screen mode, only display the screen-casting interface of the screen-casting device B in one window, and display the current user interface of the display device and the screen-casting device.
  • the screen projection interface of A is displayed in the other two windows, that is, the TOP application of the display device and the screen projection interface of the two screen projection devices are simultaneously displayed.
  • the display device 200 may enter a two-screen state, and only display the screen projection interfaces of the two screen projection devices.
  • the realization of the three screen projection modes of the display device 200 is the full-screen mode, the split-screen mode, and the smart mode, depending on the split-screen processing center.
  • the overall coordination processing is performed according to the number of screen projection devices and the capability support of the current display device 200, to determine which mode the current display device 200 is started in, and to be responsible for the specific execution of the split-screen mode. It should be noted that the specific number of split screens and the presentation mode of the display device 200 can be customized, and the presentation modes described herein are not limited.
  • FIG. 17 is a schematic diagram showing the display effect of four-device screen projection in a split-screen mode according to some embodiments.
  • screen projection device A, screen projection device B, screen projection device C, and screen projection device D all report to the display device. 200 sends a screen projection request, and the application layer central module sorts out the number of projection screens connected to the display device 200, and notifies the split-screen processing center.
  • the split-screen processing center controls the display device to enter the quad-screen, displays the screen-casting interface of the screen-casting device A in the first window, and displays the screen-casting interface of the screen-casting device A in the second window.
  • the screen projection interface of the screen projection device B, the screen projection interface of the screen projection device D is displayed in the third window, and the screen projection interface of the screen projection device D is displayed in the fourth window.
  • FIG. 18 is a schematic diagram of a selected window in a split-screen mode according to some embodiments.
  • the display device 200 is in a three-screen split state, and the split-screen processing center controls the TOP application and the screen-casting interfaces of the two screen-casting devices respectively in displayed in three windows.
  • the typesetting effect in FIG. 18 is only a presentation mode implemented in business, and the user can also customize the typesetting according to requirements.
  • the user can control the corresponding screen-casting application interface through the remote control terminal, control the interface, and switch between the split-screen and full-screen modes.
  • both screen projection device A and screen projection device B establish connections with the display device 200 based on the NFC protocol.
  • screen projection device A and screen projection device B are both screen projection devices that support the same screen projection protocol, and both screen projection devices establish connections with the display device through the screen projection protocol service.
  • the device is discovered according to the protocol layer. After the device is discovered, the display device monitors the command and establishes the command channel on the mobile terminal, so as to carry out the interaction of the command protocol.
  • the display device monitors the data of the screen projection device, and displays the corresponding interface after obtaining the screen projection data of the screen projection device.
  • the screen projection protocol needs to manage the display device screen projection device with the client group.
  • the screen projection device can be managed by different group members in the group. The management is roughly divided into two aspects. One is connection management. The number of screen projection devices connected to the display device is obtained through the number of members in the group to prepare for the final display effect of the video stream. The other is video stream management. The video stream sent by each screen projection device is classified and decoded.
  • the display device 200 displays the screen projection content of the last connected screen projection device in full screen. If the current screen projection mode is split screen mode, it can be divided into the following two types of layout situations. Case 1: The user needs to display the TOP application synchronously. After coming to the three-way projection video stream interface, it can be displayed synchronously with the TOP application. Display in a split-screen manner; Case 2 is a situation where users do not need to display TOP applications synchronously, so the three-way projection video stream interface that comes over needs to be typeset. Typesetting, the presentation form is not limited to that shown in the current template.
  • the current screencasting mode is the smart mode, further determine the screencasting content of the last connected screencasting device. If the screencast content of the last connected screencasting device is a vertical media asset, then according to the processing method in the split-screen mode, the screencasting contents of the multiple screencasting devices are displayed in the multiple windows respectively. If the screen projection content of the last accessed screen projection device is a horizontal media asset, according to the processing method in the full screen mode, the screen projection content of the last accessed screen projection device is displayed in full screen on the display.
  • FIG. 20 is a schematic diagram of the effect of screen projection by a multi-screen projection device under different protocols according to some embodiments.
  • the display device 200 has screen projection services corresponding to three protocols. , NFC protocol service, Airplay protocol service, DLNA protocol service, to undertake the push of the mobile terminal.
  • Step 1 The screen-casting device A, screen-casting device B, and screen-casting device C respectively establish connections with the display device through corresponding protocol services.
  • the display device 200 receives the screen-casting device A supporting the NFC protocol by using the NFC screen-casting service. and use the NFC screen casting service to monitor the connection of the screencasting device A, use the Airplay screencasting service to receive the screencasting request of the screencasting device B that supports the Airplay protocol, and use the Airplay screencasting service to monitor the connection of the screencasting device B.
  • Use the DLNA projection service to receive the projection request from the projection device C that supports the DLNA protocol, and use the DLNA projection service to monitor the connection of the projection device C.
  • FIG. 21 is a schematic diagram of establishing a connection between a screen projection device and a display device according to some embodiments.
  • the device in the process of establishing an instruction channel and a data channel between the screen projection device and the display device, the device is first discovered according to the protocol layer. , after the device is discovered, the display device monitors the command and establishes the command channel on the mobile terminal, so as to carry out the interaction of the command protocol. After the interaction of the command protocol and the agreement, the data transmission, that is, the transmission of the video stream, the display device monitors the data of the screen projection device, and displays the corresponding interface after obtaining the screen projection data of the screen projection device.
  • Step 2 The respective screen projection protocols need to manage the screen projection devices of the display device with the application layer hub module, that is, pivot processing in the application layer hub module, which is roughly divided into two aspects for management, one is connection management, responsible for statistics and display. The number of screencasting devices connected to the screencasting device prepares for the final display effect of the video stream. The other is video stream management. The video stream management is responsible for classifying and decoding the video streams from different screencasting protocols, so that the display The device 200 displays the screencast content in the window.
  • connection management responsible for statistics and display.
  • the number of screencasting devices connected to the screencasting device prepares for the final display effect of the video stream.
  • the other is video stream management.
  • the video stream management is responsible for classifying and decoding the video streams from different screencasting protocols, so that the display The device 200 displays the screencast content in the window.
  • Step 3 Perform screen projection video stream display processing. If the current screen projection mode of the display device 200 is the full screen mode at this time, the display device 200 displays the screen projection content of the last connected screen projection device in full screen. If the current screen projection mode is split screen mode, it can be divided into the following two types of layout situations. Case 1: The user needs to display the TOP application synchronously. After coming to the three-way projection video stream interface, it can be displayed synchronously with the TOP application. Display in a split-screen manner; Case 2 is a situation where users do not need to display TOP applications synchronously, so the three-way projection video stream interface that comes over needs to be typeset. Typesetting, the presentation form is not limited to that shown in the current template.
  • the current screencasting mode is the smart mode, further determine the screencasting content of the last connected screencasting device. If the screencast content of the last connected screencasting device is a vertical media asset, then according to the processing method in the split-screen mode, the screencasting contents of the multiple screencasting devices are displayed in the multiple windows respectively. If the screen projection content of the last accessed screen projection device is a horizontal media asset, according to the processing method in the full screen mode, the screen projection content of the last accessed screen projection device is displayed in full screen on the display.
  • the display device 200 may receive a screen projection request from a multi-screen projection device under different screen projection protocols or a screen projection request from a multi-screen projection device under the same screen projection protocol, and display the multi-projection on the same screen on the user interface of the display.
  • the screen projection interface of the screen device improves the user experience.
  • Some embodiments of the present application provide a method for controlling multi-device screen projection and displaying on the same screen. , where multiple screencasting devices support the same screencasting protocol.
  • the current screen projection mode is the full screen mode
  • the screen projection content of the last connected screen projection device is displayed on the display in full screen.
  • the current screen projection mode is the split screen mode
  • the screen projection contents of the plurality of screen projection devices are displayed in a plurality of windows respectively.
  • the current screencasting mode is the smart mode, if the screencasting content of the last connected screencasting device is a vertical media asset, the screencasting contents of the plurality of screencasting devices are respectively placed in the multiple windows.
  • Displaying is performed, and if the screencasting content of the last accessed screencasting device is a horizontal media asset, the screencasting content of the last accessed screencasting device is displayed in full screen on the display.
  • the display device can display media content in multiple windows, access multiple screen projection devices, and display the projection content in different projection modes, which greatly improves the user experience.
  • the display when the user interface includes a window, the display is controlled to display the screen projection content of the last connected screen projection device in full screen.
  • the process when establishing a connection with multiple screen-casting devices, includes: receiving screen-casting requests from multiple screen-casting devices through the same screen-casting service; in response to the screen-casting requests, using the screen-casting request
  • the service monitors the connection of the screen-casting device, and manages the screen-casting device in a group, wherein different screen-casting devices correspond to different members in the group; use the members in the group to build an instruction channel and a data channel with the screen-casting device ; the command channel is used for command interaction between the screen projection device and the display device, and the data channel is used for data interaction between the screen projection device and the display device.
  • the screen-casting devices when different screen-casting devices correspond to different members in the group, the screen-casting devices are controlled to bind to corresponding device identifiers, wherein the device identifiers are used to distinguish different members in the group.
  • the plurality of windows when controlling a plurality of windows to display the screen projection contents of a plurality of the screen projection devices respectively, if the displayed content of the user interface is retained, the plurality of windows are controlled to display the user interface and a plurality of corresponding screen projection devices respectively.
  • the screen projection content of the screen projection device if the content displayed on the user interface is not retained, multiple windows are controlled to display the screen projection content of the screen projection device correspondingly.
  • controlling multiple windows to respectively display the user interface and the screen-casting content of the plurality of screen-casting devices including: supporting split-screen display on the current user interface When the current user interface does not support split-screen display, start the corresponding number of The user interface and the screen projection content of the plurality of screen projection devices are respectively pushed to the plurality of the windows through the activity manager interface.
  • an input selection operation of a media asset control in any window is received, wherein the media asset control uses for displaying the corresponding multimedia resource; according to the selection operation, if the corresponding multimedia resource is a horizontal media resource, the corresponding multimedia resource is played in full screen in the window to which the media resource control belongs.
  • the coordinates, width and height of the user interface are reset, so that the user interface is proportionally reduced.
  • Some embodiments of the present application provide a method for controlling multi-device screen-casting and displaying on the same screen, including: using multiple screen-casting services to respectively establish connections with multiple screen-casting devices, so that the display device receives the screen-casting from the multiple screen-casting devices content, where multiple screencasting devices support different screencasting protocols.
  • the current screen projection mode is the full screen mode
  • the screen projection content of the last connected screen projection device is displayed on the display in full screen.
  • the current screen projection mode is the split screen mode
  • the screen projection contents of the plurality of screen projection devices are displayed in a plurality of windows respectively.
  • the current screencasting mode is the smart mode
  • the screencasting content of the last connected screencasting device is a vertical media asset
  • the screencasting contents of the plurality of screencasting devices are respectively placed in the multiple windows. Displaying is performed, and if the screencasting content of the last accessed screencasting device is a horizontal media asset, the screencasting content of the last accessed screencasting device is displayed in full screen on the display.
  • Android itself has multiple windows. With the opening of the window, usually the last opened window has the focus. Android can directly distribute the directional Touch event to the application clicked by the user, but for the Input event, since the Input has no direction It can only be distributed to the task stack where the current focus is located.
  • the display device 200 is in the split-screen mode, the user interface displayed on the display 260 is divided into multiple windows. For multiple windows, the user can only control the content in the window located at the top level of the user interface through the control device 100 . , and cannot control the content of other windows. As a result, the experience is not as good as the user does not have selective control between multiple windows.
  • a display device in some embodiments.
  • the present application refers to the state that has the focus and responds to the user's operation on the content displayed in the window as the first state, that is to say , in the first state, the content displayed in the window can obtain the focus.
  • the first window displaying the application list control is the window in the first state, and the window is demarcated by the focus frame. The user's operation of the APP can be realized in the window.
  • the state that has the focus but does not respond to the user's operation on the content displayed in the window is called the second state, that is, the window in the second state can obtain the focus, and the window in the second state can also be accessed through the focus frame. Calibration is used to prompt the window position of the current second state.
  • the form of the focus frame in the first state and the second state is different, which may be different in color, different in line thickness, or different in the form of the line. etc.; the state in which the window does not acquire the focus becomes the third state, for example, the second window, the third window and the fourth window in FIG. 5c are all windows in the third state.
  • the first window when the display device 200 enters the split-screen mode, that is, when the display 260 displays multiple windows, at this time, according to the preset settings of the display device 200, the first window may be the window in the first state by default, or That is, after the display device 200 enters the split-screen mode, the user can control the content displayed in the first window through the control device 100 .
  • the window in the first state can be demarcated by the first mark.
  • the window in the first state is calibrated by a focus frame, and the color of the focus frame may be a thick red line, which is not limited here. When the user sees the window marked by the focus frame with the thick red line, he can know that the window is in the first state at this time.
  • the first mark is displayed when the user operates the content displayed in the window in the first state, and the first mark is hidden when the user does not operate the content displayed in the window in the first state. a mark. That is, in order to ensure the user's visual experience, the display device 200 may only prompt the user which window is currently marked as the first state by the blinking state of the first mark.
  • the user can send a trigger instruction to the display device 200, and the controller 250 in the display device 200 triggers the switching of the display device 200 after receiving the trigger instruction. windowed mode.
  • the controller 250 marks the first window in the first state as the second state according to the received trigger instruction, including: according to the trigger instruction, moving the focus between multiple windows, and moving the The first mark of the first window is replaced with a second mark.
  • the display device 200 enters a switching window mode, in which there are no windows in the first state among all windows on the display 260, but only windows in the second state. In other words, in this mode, the user You can only move the focus to switch between windows, and you cannot manipulate the content displayed in the window.
  • the marking can be performed by the second mark, and the first mark and the second mark are different forms of focus boxes. That is to say, the form of the focus frame used to mark the second state is different from the form of the focus frame used to mark the first state, for example, it can be selected as a thin green line.
  • a selection instruction is further issued according to the spatial arrangement of the multiple windows on the display 260 .
  • the controller 250 receives the selection instruction sent by the user, marks the window selected by the user as the second state, and marks the first window from the second state to the third state.
  • FIG. 5d is a schematic diagram of the display effect of switching windows in a split-screen mode according to some embodiments.
  • the window in the current first state is the first window. If the user wants to manipulate the media resources in the fourth window , the fourth window needs to be marked as the second state according to the relative position between the fourth window and the first window, and a selection command of "rightward” and then "downward” is sent through the remote control. After receiving the confirmation instruction sent by the user, the fourth window is marked as the first state.
  • the controller 250 marks each window to which the focus is moved to the second state through the second mark, and after the focus is removed, the controller 250 sets its first state to the second state.
  • the second state is reset to the third state. That is to say, in the window switching mode, the window where the focus is located is marked as the second state by the border.
  • the user can judge the movement track of the focus and the current window through the movement of the second marker.
  • the controller controls the focus to move to the second window, and at this time, the second window is marked as the second window through the focus frame of the thin green line.
  • the window in the second state is marked as the first state, and correspondingly, the window in the second state is marked as the first state.
  • the second mark of the window is replaced with the first mark, for example, the green thin line focus frame calibration of the window is changed to the red thick line focus frame calibration.
  • the confirmation command can be issued by the user through the "Confirm" direction key on the remote control. It may also be issued by the user through voice control, for example, after the user enters the wake word into the display device, the user continues to input "confirm current selection" to the display device 200 .
  • the process of controlling the focus to move among multiple windows according to the user's instruction has been described above. Of course, after selecting the window in the first state, the user can also control the focus to move within the window.
  • a selection instruction input by the user is received, and according to the selection instruction, the control focus is transferred between various application controls in the window in the first state.
  • Each application control to which the focus is moved is marked as the focused application by changing the background color.
  • some embodiments of the present application further provide a user interface control method.
  • the control method of the user interface includes: the controller 250 receives a trigger instruction input by the user for triggering switching of the window mode, and when the user interface displayed on the display includes multiple windows, according to the trigger instruction A window is designated as the second state. Further receive a selection instruction input by the user for selecting a window, and according to the selection instruction, demarcate the window selected by the user as the second state, and demarcate the first window from the second state to the third state, wherein , the third state refers to a state without focus. Finally, the confirmation instruction input by the user is received, and according to the confirmation instruction, the window in the second state is marked as the first state.
  • FIG. 22 is a schematic diagram of a focus transfer flow for switching windows in a split-screen mode according to some embodiments.
  • the display device 200 firstly receives the key value sent by the user, that is, obtains key event. Then, the controller 250 determines whether the key event is a trigger instruction sent by the user. If it is a trigger instruction, the display device 200 is triggered to enter the switching window mode, that is, the window in the current first state is demarcated as the second state, the window demarcated by the red thick line focus frame is changed to the window demarcated by the green thin line focus frame, and Further wait for the user to make a choice.
  • the key event is directly distributed to the application pointed to by the current focus task. Then, when the selection instruction sent by the user is received, the current focus task is obtained, that is, the window where the current focus is located is found. After the current focus position is obtained, the target window is calculated according to the selection instruction sent by the user. Finally, the target window is placed on the top layer, and the focus is obtained, that is, the target window is marked as the first state.
  • the windows in the first state and the second state are demarcated by a first mark and a second mark, respectively, wherein the first mark and the second mark are different forms of focus frame.
  • the marking the first window in the first state as the second state according to the triggering instruction includes: moving the focus between multiple windows according to the triggering instruction; moving the first window to the second state; The first mark of the window is replaced with the second mark.
  • the first mark is displayed when the user operates the displayed content in the window in the first state; the displayed content in the window in the first state is not affected by the user. In operation, the first marker is hidden.
  • marking the corresponding window to the second state according to the selection instruction includes: calculating a target window according to the selection instruction; placing the target window on the topmost layer of the display ; control focus is transferred to the target window, and the target window is marked as the second state.
  • the window to which the focus is moved is marked as the second state by the second mark, and after the focus is moved, the second state is reset to the third state.
  • a selection instruction input by the user is received; according to the selection instruction, the control focus is transferred among the respective application controls in the windows in the first state.
  • the display device 200 implements the startup and operation of the split-screen mode based on the data interaction among the split-screen window service (SplitWindowService), the split-screen window service (HSSplitWindowUsercase) and the Activity management service (ActivityManagerService). application.
  • SplitWindowService split-screen window service
  • HSSplitWindowUsercase split-screen window service
  • ActivityManagerService Activity management service
  • the split-screen window service controls the split-screen window service by calling the split-screen startup interface to start the split-screen mode.
  • the split-screen window service affects the application stack.
  • the activity management service notifies the split-screen window business that the split-screen mode is started, and the split-screen window business will focus on the first underlying page, that is, set the default focus window.
  • the split-screen window service operates on the stack at this time. If it is a three-split screen in the split-screen mode, three stacks are activated through the Activity management service and arranged in sequence in the display order.
  • the Activity management service notifies the split-screen window service through callbacks. The screen startup is completed, and the page where the first bottom-level Task is located gets the focus.
  • the split-screen window service starts and creates the window through the Activity management service, and moves the focus to the window. Gives the window the focus. Since the display interface is 1080P by default, it is necessary to adjust the size of the window, that is, reduce it proportionally, change the application size by setting the position and width and height, change it to 1080P, and place it in the corresponding position. Finally, for multi-window focus switching, the split-screen window service obtains the key event output by the user, and when receiving a trigger instruction, triggers the display device 200 to enter the switching window mode.
  • the split-screen window service sets the focus to the outer layer, that is, Select the task where the window is located, and mark the current window mode as switching.
  • the split-screen window service provides a calibration prompt for the current window to indicate the current focus position.
  • the application in the task task where the focus window is located is set to obtain the focus, and a label prompt is made to indicate the current position of the focus.
  • the window of the framework layer obtains window information, where the window information includes location, task, etc., and the activity of the framework layer responds to key operations.
  • the key movement needs to be processed.
  • the focus between the outer windows is moved, not the focus on the controls in the application in the window.
  • the window in the first state is selected, the focus is moved to the window, that is, the user can move the focus on the controls in the application in the window.
  • the window, position, and activity of the framework layer are involved, and the key is first distributed to the key. to the current window, and then distributed to the application. When moving up, down, left, and right, it will move the position of the focus of the control in the application.
  • the user when the smart TV is in the split-screen mode, the user can realize the focus switching process between different windows, and freely operate and click any application in any window, which greatly improves the user experience.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Hardware Design (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

本申请一些实施方式公开一种显示设备及多设备投屏同屏显示的控制方法，其中，显示设备，包括：显示器，被配置为显示用户界面，所述用户界面中包括多个窗口；控制器，所述控制器被配置为：响应于多个投屏设备的投屏请求，与多个所述投屏设备建立连接，其中，多个投屏设备支持同一投屏协议；接收多个所述投屏设备所发送的投屏内容；在最后接入的所述投屏设备的投屏内容为竖向媒资时，控制多个窗口分别对应展示多个所述投屏设备的投屏内容；在最后接入的所述投屏设备的投屏内容为横向媒资时，控制所述显示器全屏展示最后接入的所述投屏设备的投屏内容。

Description

显示设备及多设备投屏同屏显示的控制方法
本申请要求于2021年4月30日提交的、申请号为202110484270.4；于2021年6月24日提交的、申请号为202110702220.9；于2021年6月24日提交的、申请号为202110702282.X；于2021年5月27日提交的、申请号为202110586572.2的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及显示设备技术，具体而言，涉及一种显示设备及多设备投屏同屏显示的控制方法。
背景技术
移动设备可以基于一定的投屏协议将屏幕当前显示内容展示到显示设备的用户界面上，其中，移动设备输出的内容包括各类媒体信息和实时操作画面等。由于当前显示设备在同一时刻只能够允许一个移动设备进行投屏，即用户界面只能展示出一个移动设备的屏幕内容，无法对多移动设备的投屏进行同屏展示。
发明内容
本申请一些实施例提供了一种显示设备，包括：
显示器，被配置为显示用户界面，所述用户界面中包括多个窗口；控制器，所述控制器被配置为：响应于多个投屏设备的投屏请求，与多个所述投屏设备建立连接，其中，多个投屏设备支持同一投屏协议；接收多个所述投屏设备所发送的投屏内容；在最后接入的所述投屏设备的投屏内容为竖向媒资时，控制多个窗口分别对应展示多个所述投屏设备的投屏内容；在最后接入的所述投屏设备的投屏内容为横向媒资时，控制所述显示器全屏展示最后接入的所述投屏设备的投屏内容。
本申请一些实施例提供了一种显示设备，包括：
显示器，被配置为显示用户界面，所述用户界面中包括多个窗口；控制器，所述控制器被配置为：利用多个投屏服务分别与多个投屏设备建立连接，以使显示设备接收多个投屏设备的投屏内容，其中，多个投屏设备支持不同投屏协议；在当前投屏模式为分屏模式时，将多个所述投屏设备的投屏内容分别于多个所述窗口进行展示；在当前投屏模式为智能模式时，若最后接入的所述投屏设备的投屏内容为竖向媒资，则将多个所述投屏设备的投屏内容分别于多个所述窗口进行展示，若最后接入的所述投屏设备的投屏内容为横向媒资，则将最后接入的所述投屏设备的投屏内容于所述显示器上全屏展示。
在一些实施例中，所述控制器还被配置为：在所述用户界面包括一个窗口时，控制所述显示器全屏展示最后接入的所述投屏设备的投屏内容。
在一些实施例中，在与多个投屏设备建立连接时，包括：
通过同一投屏服务接收多个所述投屏设备的投屏请求；响应于所述投屏请求，利用所述投屏服务监听所述投屏设备连接，对所述投屏设备进行组管理，其中，不同的所述投屏设备对应组内不同成员；利用组内成员与所述投屏设备搭建指令通道和数据通道，其中，所述指令通道用于所述投屏设备和所述显示设备之间的指令交互，所述 数据通道用于所述投屏设备和所述显示设备之间的数据交互。
在一些实施例中，所述控制器还被配置为：在不同的所述投屏设备对应组内不同成员时，控制投屏设备绑定相应的设备标识，其中，所述设备标识用于区分组内不同成员。
在一些实施例中，所述在控制多个窗口分别对应展示多个所述投屏设备的投屏内容时，若保留用户界面所显示内容，则控制多个窗口分别对应展示用户界面及多个所述投屏设备的投屏内容；若不保留用户界面所显示内容，则控制多个窗口分别对应展示多个所述投屏设备的投屏内容。
在一些实施例中，在所述若保留用户界面所显示内容，则控制多个窗口分别对应展示用户界面及多个所述投屏设备的投屏内容，包括：
在当前用户界面支持分屏展示时，启动相应数量的窗口，将用户界面及多个所述投屏设备的投屏内容通过活动管理器接口分别推送至多个所述窗口；在当前用户界面不支持分屏展示时，启动相应数量的窗口，将等比例缩小后的用户界面及多个所述投屏设备的投屏内容通过活动管理器接口分别推送至多个所述窗口。
在一些实施例中，在控制多个窗口分别对应展示多个所述投屏设备的投屏内容时，所述控制器还被配置为：
接收输入的对任一窗口内媒资控件的选中操作，其中，所述媒资控件用于展示对应的多媒体资源；根据所述选中操作，若所述对应的多媒体资源为横向媒资时，在所述媒资控件所属的窗口内全屏播放对应的多媒体资源。
在一些实施例中，所述控制器还被配置为：在将用户界面等比例缩小时，重新设置所述用户界面的坐标和宽高，以使所述用户界面等比例缩小。
本申请一些实施例提供了一种多设备投屏同屏显示的控制方法，所述方法包括：响应于多个投屏设备的投屏请求，与多个所述投屏设备建立连接，其中，多个投屏设备支持同一投屏协议；接收多个所述投屏设备所发送的投屏内容；在最后接入的所述投屏设备的投屏内容为竖向媒资时，控制多个窗口分别对应展示多个所述投屏设备的投屏内容；在最后接入的所述投屏设备的投屏内容为横向媒资时，控制所述显示器全屏展示最后接入的所述投屏设备的投屏内容。
本申请一些实施例提供了一种多设备投屏同屏显示的控制方法，所述方法包括：利用多个投屏服务分别与多个投屏设备建立连接，以使显示设备接收多个投屏设备的投屏内容，其中，多个投屏设备支持不同投屏协议；在当前投屏模式为分屏模式时，将多个所述投屏设备的投屏内容分别于多个窗口进行展示；在当前投屏模式为智能模式时，若最后接入的所述投屏设备的投屏内容为竖向媒资，则将多个所述投屏设备的投屏内容分别于多个所述窗口进行展示，若最后接入的所述投屏设备的投屏内容为横向媒资，则将最后接入的所述投屏设备的投屏内容于所述显示器上全屏展示。
附图说明
图1为根据一些实施例的显示设备与控制装置之间操作场景的示意图；
图2为根据一些实施例的控制装置100的硬件配置框图；
图3为根据一些实施例的显示设备200的硬件配置框图；
图4为根据一些实施例的显示设备200中软件配置示意图；
图5a为根据一些实施例的分屏模式中二分屏的显示效果示意图；
图5b为根据一些实施例的分屏模式中三分屏的显示效果示意图；
图5c为根据一些实施例的分屏模式中四分屏的显示效果示意图；
图5d为一些实施例的在分屏模式下切换窗口的显示效果示意图；
图6a为根据一些实施例的通过按键切换分屏模式的显示效果示意图；
图6b为根据一些实施例的通过切换选项切换分屏模式的显示效果示意图；
图6c为根据一些实施例的一种自动切换分屏模式的显示效果示意图；
图6d为根据一些实施例的另一种自动切换分屏模式的显示效果示意图；
图6e为根据一些实施例的触控事件分发的显示效果示意图；
图6f为根据一些实施例的触控事件分发的时序图；
图6g为根据一些实施例的输入事件分发的时序图；
图7为根据一些实施例的同一投屏协议下多投屏设备投屏流程示意图；
图8a-图8b为根据一些实施例的同一投屏协议下多投屏设备与显示设备建立连接的示意图；
图9a为根据一些实施例的不同投屏协议下多投屏设备投屏流程示意图；
图9b为根据一些实施例的应用层中枢模块的示意图；
图10为根据一些实施例的全屏模式下单设备投屏的显示效果示意图；
图11-图12为根据一些实施例的分屏模式下单设备投屏的显示效果示意图；
图13为根据一些实施例的智能模式下单设备投屏的显示效果示意图；
图14为根据一些实施例的同一投屏协议下多投屏设备与显示设备建立连接的示意图；
图15-图16为根据一些实施例的分屏模式下双设备投屏的显示效果示意图；
图17为根据一些实施例的分屏模式下四设备投屏的显示效果示意图；
图18为根据一些实施例的分屏模式下选定窗口示意图；
图19为根据一些实施例的同一投屏协议下多投屏设备投屏的效果示意图；
图20为根据一些实施例的不同协议下多投屏设备投屏的效果示意图；
图21为根据一些实施例的投屏设备与显示设备建立连接的示意图；
图22为根据一些实施例的在分屏模式下切换窗口的焦点传递流程示意图；
图23为根据一些实施例的显示设备处于分屏模式阶段的时序图。
具体实施方式
为使本申请的目的、实施方式和优点更加清楚，下面将结合本申请示例性实施例中的附图，对本申请示例性实施方式进行清楚、完整地描述，显然，所描述的示例性实施例仅是本申请一部分实施例，而不是全部的实施例。
图1为根据实施例中显示设备与控制装置之间操作场景的示意图。如图1所示，用户可通过智能设备300或控制装置100操作显示设备200。在一些实施例中，控制装置100可以是遥控器，遥控器和显示设备的通信包括红外协议通信或蓝牙协议通信，及其他短距离通信方式，通过无线或有线方式来控制显示设备200。用户可以通过遥控器上按键、语音输入、控制面板输入等输入用户指令，来控制显示设备200。在一些实施例中，也可以使用智能设备300(如移动终端、平板电脑、计算机、笔记本电脑等)以控制显示设备200。 例如，使用在智能设备上运行的应用程序控制显示设备200。在一些实施例中，显示设备200还可以采用除了控制装置100和智能设备300之外的方式进行控制，例如，可以通过显示设备200设备内部配置的获取语音指令的模块直接接收用户的语音指令控制，也可以通过显示设备200设备外部设置的语音控制设备来接收用户的语音指令控制。在一些实施例中，显示设备200还与服务器400进行数据通信。可允许显示设备200通过局域网(LAN)、无线局域网(WLAN)和其他网络进行通信连接。服务器400可以向显示设备200提供各种内容和互动。服务器400可以是一个集群，也可以是多个集群，可以包括一类或多类服务器。
图2示例性示出了根据示例性实施例中控制装置100的配置框图。如图2所示，控制装置100包括控制器110、通信接口130、用户输入/输出接口140、存储器、供电电源。控制装置100可接收用户的输入操作指令，且将操作指令转换为显示设备200可识别和响应的指令，起用用户与显示设备200之间交互中介作用。
图3示出了根据示例性实施例中显示设备200的硬件配置框图。如图3所示，在一些实施例中，显示设备200包括调谐解调器210、通信器220、检测器230、外部装置接口240、控制器250、显示器260、音频输出接口270、存储器、供电电源、用户接口中的至少一种。在一些实施例中，显示器260包括用于呈现画面的显示屏组件，以及驱动图像显示的驱动组件，用于接收源自控制器输出的图像信号，进行显示视频内容、图像内容以及菜单操控界面的组件以及用户操控UI界面。显示器260可为液晶显示器、OLED显示器、以及投影显示器，还可以为一种投影装置和投影屏幕。在一些实施例中，控制器250，通过存储在存储器上中各种软件控制程序，来控制显示设备的工作和响应用户的操作。控制器250控制显示设备200的整体操作。例如：响应于接收到用于选择在显示器260上显示UI对象的用户命令，控制器250便可以执行与由用户命令选择的对象有关的操作。在一些实施例中，所述对象可以是可选对象中的任何一个，例如超链接、图标或其他可操作的控件。与所选择的对象有关操作有：显示连接到超链接页面、文档、图像等操作，或者执行与所述图标相对应程序的操作。在一些实施例中，用户可在显示器260上显示的图形用户界面(GUI)输入用户命令，则用户输入接口通过图形用户界面(GUI)接收用户输入命令。或者，用户可通过输入特定的声音或手势进行输入用户命令，则用户输入接口通过传感器识别出声音或手势，来接收用户输入命令。
在一些实施例中，“用户界面”，是应用程序或操作系统与用户之间进行交互和信息交换的介质接口，它实现信息的内部形式与用户可以接受形式之间的转换。用户界面常用的表现形式是图形用户界面(Graphic User Interface，GUI)，是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的一个图标、窗口、控件等界面元素，其中控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。
图4为根据一些实施例的显示设备200中软件配置示意图，参见图4，在一些实施例中，将系统分为四层，从上至下分别为应用程序(Applications)层(简称“应用层”)，应用程序框架(Application Framework)层(简称“框架层”)，安卓运行时(Android runtime)和系统库层(简称“系统运行库层”)，以及内核层。在一些实施例中，应用程序层中运行有至少一个应用程序，这些应用程序可以是操作系统自带的窗口(Window)程序、系统设置程序或时钟程序等；框架层为应用程序提供应用编程接口(application programming interface，API) 和编程框架。应用程序框架层包括一些预先定义的函数。内核层是硬件和软件之间的层。内核层至少包含以下驱动中的至少一种：音频驱动、显示驱动、蓝牙驱动等。
<分屏模式>
基于上述显示设备200，用户在使用显示设备200的过程中能够对显示的画面进行分屏处理，即显示设备200进入分屏模式时，显示器260形成多个窗口。在分屏模式下，显示设备200的显示器260可以在用户界面中同时显示两个或两个以上的窗口，每个窗口可以独立的显示不同内容，使得用户可以同时观看到不同的节目画面。
图5a为根据一些实施例的分屏模式中二分屏的显示效果示意图，如图5a所示，在显示设备200进入到分屏模式中的二分屏展示时，显示器260可将用户界面一分为二，显示第一窗口和第二窗口，其中第一窗口可以用于显示第一媒资资源的播放画面，第二窗口可以用于显示第二媒资资源的播放画面。其中，二分屏可也称为双分屏。图5b为根据一些实施例的分屏模式中三分屏的显示效果示意图，如图5b所示，在显示设备200进入到分屏模式中的三分屏展示时，显示器260可将用户界面划分为三个区域，分别显示第一窗口、第二窗口和第三窗口，其中第一窗口可以用于显示第一媒资资源的播放画面，第二窗口可以用于显示第二媒资资源的播放画面，第三窗口可以用于显示第三媒资资源的播放画面。
在一些实施例中，针对不同的使用场景，多窗口显示的画面内容可以不同。图5c为根据一些实施例的分屏模式中四分屏的显示效果示意图，例如，如图5c所示，在显示设备200进入到分屏模式中的四分屏展示时，显示器260可将用户界面划分为四个区域，分别显示第一窗口、第二窗口、第三窗口和第四窗口，在第一窗口中显示着应用列表，在第二窗口中显示中应用列表，在应用列表中排布多个应用程序的相关控件。在第三窗口中显示第三媒资的播放画面，在第四窗口中显示第四媒资的播放画面。也就是说，在分屏模式下，不同窗口可以同时显示不同类型的界面，各个窗口之间互不干扰。
在一些实施例中，用户可以通过特定的交互动作控制显示设备200进入分屏模式。图6a为根据一些实施例的通过按键切换分屏模式的显示效果示意图，如图6a所示，可以在显示设备200配套的控制装置100上设置分屏模式切换按键，当用户按下该分屏模式切换按键时，可以控制显示设备200进入或退出分屏模式，即控制显示设备200的显示器260显示多个窗口或单个窗口。还可以在显示设备200的UI界面中设置分屏模式切换选项，当用户点击该切换选项时，可以控制显示设备200进入或退出分屏模式。图6b为根据一些实施例的通过切换选项切换分屏模式的显示效果示意图，如图6b所示，示例性示出了根据一些实施例的通过切换选项切换分屏模式的显示效果示意图。还可以在显示设备200的UI界面中设置分屏模式切换选项，当用户点击该切换选项时，可以控制显示设备200进入或退出分屏模式。在一些实施例中，用户可以通过语音控制显示设备200进入或退出分屏模式。例如，在用户输入唤醒词“嗨！××”后，对显示设备200输入“切换至分屏模式”的语音指令，显示设备200即可进入分屏模式。
在一些实施例中，显示设备200还可以对用户的使用状态实时检测，并判断用户是否具有多窗口显示的需求，从而根据用户需求智能切换分屏模式或提供分屏模式切换选项。图6c为根据一些实施例的一种自动切换分屏模式的显示效果示意图，如图6c所示，在显示设备200展示媒资画面的过程中，如果有视频聊天请求，则可以自动切换至分屏模式，使得显示设备200可以通过第一窗口显示媒资画面；同时通过第二窗口显示视频聊天画面。 图6d为根据一些实施例的另一种自动切换分屏模式的显示效果示意图，如图6d所示，在显示设备200播放媒资画面的过程中，还可以在用户界面的右侧显示多个推荐节目选项，当用户选中任一推荐节目选项并确认后，可以控制显示设备200进入分屏模式，即原媒资播放画面通过第一窗口显示，推荐节目对应的播放画面通过第二窗口显示。
在一些实施例中，显示设备200的控制装置100有触控屏、遥控器、鼠标等多种，于安卓系统中，触控屏或者鼠标基于坐标计算位置，其点击事件在安卓中为Touch(触控)事件，触控屏或者鼠标可以将触控事件分发给与该位置相对应的应用，使得焦点传递至相应的应用。图6e为根据一些实施例的触控事件分发的显示效果示意图，如图6e所示，当前焦点的位置位于第二窗口，触控事件指向了第一窗口，则显示设备200件触控事件分发至第一窗口。图6f为根据一些实施例的触控事件分发的时序图，如图6f，显示设备200接收到一个含有(x，y)坐标信息的Touch事件，输入触发器(InputFlinger)会先获取系统服务中任务栈的窗口信息，即当前窗口信息，然后根据坐标判断此Touch事件是在哪个应用显示范围内触发的，也就是查找点击坐标对应的窗口应用，随后直接分发按键给坐标对应的应用，不考虑当前焦点所在的任务栈。对于遥控器设备而言，只能将按键分发给位于最顶层的应用。图6g为根据一些实施例的输入事件分发的时序图，结合图6g，显示设备200接收到一个Input事件，Input事件无坐标信息只有按键值信息，所以输入触发器会从系统服务中获取当前焦点所在的任务栈，在查询到当前焦点位于第二窗口应用时，将按键分发到第二窗口所在的任务栈中。
在一些实施例中，显示设备200还可以对用户的使用状态实时检测，并判断用户是否具有多窗口显示的需求，从而根据用户需求智能切换分屏模式或提供分屏模式切换选项。例如，在显示设备200展示媒资画面的过程中，如果有视频聊天请求，则可以自动切换至分屏模式，使得显示设备200可以通过第一窗口显示媒资画面；同时通过第二窗口显示视频聊天画面。显示设备200中设置有投屏应用，通过投屏应用基于一定的投屏协议与投屏设备进行连接，能够满足用户利用投屏协议将投屏设备的投屏内容展示在显示器上。例如，在一些实施例中，显示设备可以基于同一投屏服务，让多个支持同一投屏协议投屏设备向同一显示设备进行投屏，同步在同一用户界面上展示多个投屏设备的投屏界面。这里，投屏设备可以为手机、平板电脑、计算机、笔记本电脑，智能电视等。
在一些实施例中，控制器被配置为执行多设备投屏同屏显示的控制过程。首先，控制器250响应于多个投屏设备的投屏请求，与多个所述投屏设备建立连接，其中，多个投屏设备支持同一投屏协议。控制器250接收多个所述投屏设备所发送的投屏内容，并判断投屏内容的形式。控制器250根据最后接入的所述投屏设备的投屏内容的形式确定后续投屏展示的形式，这里，投屏内容的形式指的是竖向媒资和横向媒资，投屏展示的形式指的是多窗口展示和全屏展示。
在一些实施例中，显示设备200在投屏应用中设置有投屏模式的选择控件，当用户点击该选择控件后，可选择当前所需要的投屏模式。当用户选中了投屏模式中的智能模式时，在最后接入的所述投屏设备的投屏内容为竖向媒资时，控制器250控制多个窗口分别对应展示多个所述投屏设备的投屏内容。在最后接入的所述投屏设备的投屏内容为横向媒资时，控制器250控制所述显示器全屏展示最后接入的所述投屏设备的投屏内容。
在一些实施例中，除了上述智能模式以外，显示设备200中的投屏应用还包括全屏模式和分屏模式。其中，全屏模式指的是显示设备200控制所述显示器260全屏展示最后接 入的所述投屏设备的投屏内容。分屏模式指的是不管最后接入的投屏设备的投屏内容的形式是横向媒资还是竖向媒资，均将多个所述投屏设备的投屏内容分别于多个所述窗口进行展示。
在一些实施例中，在显示设备200中的投屏应用中，设置有投屏模式的选择控件，当用户点击该选择控件后，选择控件展示出选择列表，在选择列表中设置有全屏模式、分屏模式及智能模式，用户可通过选择列表选择当前所需要的投屏模式。需要说明的是，此处投屏模式中的分屏模式即针对于显示设备200中多窗口显示的分屏模式。
<同一投屏协议下多设备投屏同屏展示>
图7示例性示出了根据一些实施例的同一投屏协议下多投屏设备投屏流程示意图。如图7所示，以两个投屏设备为例进行说明。投屏设备A、投屏设备B通过同一投屏协议与显示设备200建立相关的协议连接，进行投屏相关操作。通过协议层约定，通讯过程中携带相关的协议信息，例如，基于NFC，投屏设备能够拿到显示设备200的设备信息、网络信息等。显示设备200在发现投屏设备A和B之后，监听投屏设备A和B的连接。如图7中所示，在显示设备200通过同一投屏服务与各个投屏设备建立连接后，再根据投屏模式的设置，于显示器260上按照全屏模式、分屏模式或者智能模式进行展示。
在一些实施例中，显示设备200通过同一投屏服务接收多个所述投屏设备的投屏请求，进行客户端Group(组)管理，分指令通道和数据通道进行业务承接，其中，通过指令通道进行相关协议的交互，通过数据通道进行视频流的传输。显示设备200响应于投屏请求，利用投屏服务监听投屏设备连接，对投屏设备进行组管理，其中，不同的所述投屏设备对应组内不同成员。投屏服务接收到投屏设备的投屏请求后，将对应的投屏设备进行客户端Group的建立，不同的投屏设备对应客户端Group的不同成员，每个成员内有对应的投屏设备信息以及协议交互信息。在不同的所述投屏设备对应组内不同成员时，控制器250控制投屏设备绑定相应的设备标识，其中，所述设备标识用于区分组内不同成员。多个投屏设备通过同一协议进行投屏展示时，通过客户端Group以及分屏处理中心处理可以实现在同一投屏协议下，多个投屏设备在不同的Activity栈内并行展示投屏界面，且相互不干扰。
图8a-图8b为根据一些实施例的同一投屏协议下多投屏设备与显示设备建立连接的示意图，在一些实施例中，显示设备200投屏服务建立一个客户端Group，投屏服务需要根据发送投屏请求的投屏设备的个数建立对应子投屏通道，接收到投屏设备的投屏请求后，在当前客户端Group内增加一个子成员。如图8a所示，不同的投屏设备对应客户端Group的不同成员，每个成员内有对应的投屏设备信息以及协议交互信息。如图8b所示，当显示设备200响应投屏设备A投屏请求，即投屏设备A与显示设备200连接成功后，下一步要进行视频流的传输，此时查询客户端Group内投屏设备的连接个数，启动分屏处理中心，进行不同投屏形态的呈现。
<不同投屏协议下多设备投屏同屏展示>
图9a为根据一些实施例的不同投屏协议下多投屏设备投屏流程示意图，如图9a所示，显示设备200通过不同投屏服务分别接收多个所述投屏设备的投屏请求，投屏设备A、投屏设备B通过不同投屏协议与显示设备建立相关的协议连接，进行投屏相关操作。通过协议层约定，通讯过程中携带相关的协议信息，例如，基于NFC(Near Field Communication， 进场通信)，投屏设备能够拿到显示设备200的设备信息、网络信息等。显示设备200响应于投屏请求，通过不同投屏协议的投屏服务接收监听投屏设备的投屏请求，利用投屏服务监听投屏设备连接，搭建指令通道和数据通道，其中，指令通道用于投屏设备和所述显示设备之间的指令交互，所述数据通道用于所述投屏设备和所述显示设备之间的数据交互。如，显示设备200中的投屏服务A接收投屏设备A的投屏请求，投屏服务B接收投屏设备B的投屏请求，投屏服务负责监听投屏设备连接，与投屏设备议建立指令通过和数据通道。通过指令通道，显示设备200能够接收用户通过投屏设备发送的请求，并且通过数据通道，显示设备200能够接收到投屏设备所发送的相关视频数据等。待指令及数据通道建立完成后，投屏设备与显示设备200建立连接。如图9a中所示，在显示设备200通过不同投屏服务与各个投屏设备建立连接后，再根据投屏模式的设置，于显示器260上按照全屏模式、分屏模式或者智能模式进行展示。
在一些实施例中，显示设备200通过不同协议的投屏服务进行投屏设备的发现，发现投屏设备后，进行投屏设备的连接。例如，在多个所述投屏设备分别支持NFC、Airplay(无线投屏技术)及DLNA(DIGITAL LIVING NETWORK ALLIANCE，数字生活网络联盟)投屏协议时，控制器250利用NFC协议服务接收支持NFC协议的投屏设备的投屏请求，并利用NFC协议服务监听投屏设备连接。又如，控制器250利用Airplay协议服务接收支持Airplay协议的投屏设备的投屏请求，并利用Airplay协议服务监听投屏设备连接。再如控制器250利用DLNA协议服务接收支持DLNA协议的投屏设备的投屏请求，并利用DLNA协议服务监听投屏设备连接。其中，NFC投屏是基于NFC协议双向进行设备的发现以及连接，Airplay投屏是基于Airplay协议双向进行设备的发现以及连接，DLNA推送是通过DLNA协议进行投屏设备的发现及连接，其他协议也如此，均是通过各自协议的约定进行投屏设备的发现及连接。其中，Airplay投屏是指苹果开发的一种无线技术，可以通过WiFi将手机、平板等iOS设备上的包括图片、音频、视频通过无线的方式传输到支持AirPlay设备。
在一些实施例中，在投屏设备与显示设备200建立连接后，显示设备200通过应用层中枢模块负责对投屏设备的管理。图9b为根据一些实施例的应用层中枢模块的示意图，如图9b所示，应用层中枢模块可以进行两个方面的管理。一是连接管理，其负责对于显示设备200进行投屏连接的投屏设备的数据进行梳理，为投屏设备视频流最终展示的效果做准备。二是视频流管理，其负责将各个不同投屏协议过来的视频流进行分类解码处理。
在一些实施例中，投屏设备A与显示设备200建立连接后，然后通知应用层中枢模块，以便应用层中枢模块进行连接管理。当投屏设备A要进行投屏内容发送时，应用层中枢模块对当前连接的所有设备成员进行梳理，再结合用户对投屏模式的设置，根据需求启动分屏处理中心，进行不同投屏形态的呈现。
<单设备投屏>
以显示设备200接入一个投屏设备A为例，即Group内投屏设备的连接个数为1个，分别描述在不同投屏模式下的投屏状态。图10为根据一些实施例的全屏模式下单设备投屏的显示效果示意图,在一些实施例中，当显示设备200判定出当前的投屏模式为全屏模式时，在投屏设备A通过数据通道向显示设备200发送投屏内容后，显示设备200可直接将投屏内容全屏显示于显示器260上，如图10所示，在全屏模式下将投屏设备A的投屏内容展示于显示器260上，其投屏内容可以为投屏设备A的操作界面。
在一些实施例中，当显示设备200判定出当前的投屏模式为分屏模式时，可以进一步根据用户的需求，判断是否将显示设备200当前用户界面进行同步展示，也就是将显示设备200的TOP(顶层)应用(或者前台应用)与投屏设备的投屏界面同时置于显示器260所显示的用户界面上。例如，如图10所示，当投屏设备A传输过来的投屏内容是竖向媒资，其所展示的投屏界面也是竖向的，那在显示设备200展示时，左右两侧会有黑框，使得用户体验较差。如若此时，将显示设备200的TOP应用同时展示于显示器260上，那用户通过显示设备200既可以操作TOP应用，又可以观看投屏设备的投屏界面，提高用户体验。当然，若不同步展示用户界面所显示内容，则将多个所述投屏设备的投屏内容分别于多个所述窗口进行展示，也就是说若是用户不需要同步展示TOP应用，那也可只展示投屏设备的投屏内容。
在分屏模式下，若是用户想要同步展示当前用户界面和投屏界面时，需要先通过分屏处理中心判断此时当前用户界面支不支持分屏模式。在当前用户界面支持分屏展示时，启动相应数量的窗口，将用户界面及多个所述投屏设备的投屏内容通过活动管理器接口分别推送至多个所述窗口。在当前用户界面不支持分屏展示时，启动相应数量的窗口，将等比例缩小后的用户界面及多个所述投屏设备的投屏内容通过活动管理器接口分别推送至多个所述窗口。例如，在TOP应用支持分屏时，分屏处理中心根据从应用层中枢模块获取到的已连接的投屏设备的数量为一个，启动显示设备200进入分屏模式下的二分屏，通过调用系统分屏接口将TOP应用置于选定的窗口显示，并将投屏设备的投屏界面置于另一个窗口显示。图11-图12为根据一些实施例的分屏模式下单设备投屏的显示效果示意图，如图11所示，将TOP应用和投屏设备A的投屏界面于两个窗口进行展示。在TOP应用不支持分屏时，分屏处理中心可以强制性的将当前TOP应用等比例缩小，如图12所示，将等比例缩小后的TOP应用放置于其中一个窗口进行展示，将投屏设备的投屏界面置于另一个窗口显示。
在一些实施例中，在接收到投屏设备A的投屏内容后，重新启动投屏界面，在当前用户界面支持分屏时，首先分屏处理中心将屏幕分为两半，并将当前用户界面的位置设置为左侧，通过重新设置用户界面的坐标和宽高以实现用户界面等比例缩小，同时投屏设备A的投屏内容的位置设置为右侧，这样分屏处理中心启动显示设备200进入分屏模式下的二分屏。
在一些实施例中，当显示设备200在分屏模式下，用户点击了某一个分屏里的视频进行全屏播放，此时，将视频在其所属的分屏窗口内全屏播放。相当于，显示设备200在显示器260所展示的用户界面中同步展示TOP应用和投屏界面时，如图11所示，若此时用户点击了投屏设备A投屏界面内的某一视频进行全屏播放，则将该视频在投屏设备A的投屏界面所在的窗口内全屏播放。
在一些实施例中，当显示设备200判定出当前的投屏模式为智能模式时，显示设备200会接收投屏设备所发送过来的投屏内容，根据当前投屏内容的显示比例，判定当前投屏内容的最优显示比例。在投屏设备A所发送的投屏内容是横向媒资，也就是投屏数据为横向码流时，如推送过某一视频，此时为了保证用户的视觉体验，显示设备200将该投屏内容全屏播放。图13为根据一些实施例的智能模式下单设备投屏的显示效果示意图，如图13所示，投屏设备A所发送的投屏内容为一横向媒体资源，则显示设备200控制显示器260全屏展示该媒体资源。在投屏设备A所发送的投屏内容是竖向媒资，也就是投屏数据 为竖向码流时，如手机操作页面，若是将该投屏数据全屏展示，在投屏界面的两侧会显示有黑框，此时，为了更好的用户体验，显示设备200选择进入分屏模式，仅将投屏界面置于一个窗口展示，并将显示设备当前用户界面于另一个窗口展示，即同步展示当前用户界面和投屏界面。
<多设备投屏>
以显示设备200接入两个投屏设备，即客户端Group内投屏设备的连接个数为2个，分别描述在不同投屏模式下的投屏状态。图14为根据一些实施例的同一投屏协议下多投屏设备与显示设备建立连接的示意图，如图14所示，显示设备200成功接入投屏设备A和投屏设备B。在一些实施例中，当显示设备200接收到一个投屏设备A的投屏请求并与其建立连接后，首先根据上述投屏状态进行投屏展示，然后，在显示设备200再次接收到第二个投屏设备B的投屏请求并与之建立连接后，通知应用层中枢模块，以便应用层中枢模块进行连接管理。当投屏设备B要进行投屏数据发送时，应用层中枢模块对当前连接的所有设备成员进行梳理，启动分屏处理中心，进行以下不同投屏形态的呈现。
在一些实施例中，参见图7，当显示设备200通过投屏模式设置中判定出当前的投屏模式为全屏模式时，将最后接入的所述投屏设备的投屏内容于所述显示器上全屏展示。即在投屏设备B通过数据通道向显示设备200发送投屏内容后，显示设备200可将之前的投屏直接断掉，直接将投屏设备B所发送的投屏内容全屏显示于显示器260上。也就是说，先把之前接入的投屏设备A的投屏界面断掉，在显示器260所显示的用户界面上更新为当前接入的投屏设备B的投屏界面。
在一些实施例中，参见图7，当显示设备200通过投屏模式设置中判定出当前的投屏模式为分屏模式时，可以根据用户的需求，判断是否需要同步展示出显示设备200端的用户界面所显示内容。在用户需要同步展示用户界面所显示内容和投屏界面时，显示设备200需要先通过分屏处理中心判断此时用户界面所显示内容支不支持分屏模式。在用户界面所显示内容支持分屏时，分屏处理中心根据从应用层中枢模块获取到的已连接的投屏设备的数量为两个，通过应用分屏排版，启动显示设备200进入三分屏状态，通过调用系统分屏接口将用户界面所显示内容置于选定的窗口显示，并将两个投屏设备的投屏界面分别置于另外两个窗口显示。如图15所示，显示设备200进行了三分屏，分为第一窗口、第二窗口和第三窗口，将显示设备端的TOP应用放置于第一窗口、投屏设备A的投屏界面放置于第三窗口以及将投屏设备B的投屏界面放置于第二窗口进行展示。在用户界面所显示内容不支持分屏时，分屏处理中心可以强制性的将用户界面所显示内容等比例缩小，并将其放置于其中一个窗口进行展示，将两个投屏设备的投屏界面分别置于另外两个窗口显示。需要说明的是，其展示效果不仅局限于图15中的方式，还可以进行其他方式的排版，如图16所示，将投屏设备A的投屏界面放置于第一窗口、投屏设备B的投屏界面放置于第三窗口以及将TOP应用放置于第二窗口进行展示。
在一些实施例中，针对于图15和图16中的排版方式，需要结合投屏设备所发送的投屏内容的横竖屏模式，如果两个投屏内容均是横向码流，则可选择图16中的排版方式，如果两个投屏内容一横一竖，则选择图15中的排版方式，当然也可以根据产品定义进行展示，不局限上述原则。其中，图15和图16的排版方式可进行切换，可根据投屏设备所发送的投屏内容的横竖向切换以实现自主切换。
在一些实施例中，参见图7，当显示设备200通过投屏模式设置中判定出当前的投屏 模式为智能模式时，显示设备200接收到投屏设备B所发送过来的投屏内容，并根据最后接入的所述投屏设备的投屏内容的显示比例，判定投屏设备B投屏内容的最优显示比例。在投屏设备B所发送的投屏内容为横向码流时，如推送某一视频，此时为了保证用户的视觉体验，显示设备200将该投屏内容全屏播放。在投屏设备B所发送的投屏内容为竖向码流时，如手机操作页面，若是将该投屏内容全屏展示，在投屏界面的两侧会显示有黑框，此时，为了更好的用户体验，显示设备200可充分利用显示器260的显示区域，选择进入分屏模式，仅将投屏设备B投屏界面置于一个窗口展示，并将显示设备的当前用户界面和投屏设备A的投屏界面于另外两个窗口展示，即同步展示显示设备的TOP应用和两个投屏设备的投屏界面。当然，若是此时用户不需要展示TOP应用，显示设备200可进入二分屏状态，仅将两个投屏设备的投屏界面进行展示。
在一些实施例中，显示设备200的全屏模式、分屏模式和智能模式三种投屏模式的实现依附于分屏处理中心，分屏处理中心可以根据当前显示设备200的能力支撑情况，进行分逻辑的处理，根据投屏设备的投屏个数以及当前显示设备200的能力支撑情况进行整体的协调处理，判断当前显示设备200以何种模式启动，以及负责分屏模式的具体执行动作。需要说明的是，显示设备200具体的分屏个数以及展现模式可以进行自定义，不局限本文所述的展现模式。
图17为根据一些实施例的分屏模式下四设备投屏的显示效果示意图，如图17所示，投屏设备A、投屏设备B、投屏设备C和投屏设备D均向显示设备200发送投屏请求，应用层中枢模块梳理接入显示设备200的投屏个数，并通知分屏处理中心。在此，以用户不需要同步展示当前用户界面所显示的内容为例，分屏处理中心控制显示设备进入四分屏，在第一窗口展示投屏设备A的投屏界面，在第二窗口展示投屏设备B的投屏界面，在第三窗口展示投屏设备D的投屏界面以及在第四窗口展示投屏设备D的投屏界面。
图18为根据一些实施例的分屏模式下选定窗口示意图，在图18中，显示设备200处于三分屏状态，分屏处理中心控制TOP应用和两个投屏设备的投屏界面分别于三个窗口中进行展示。需要说明的是，图18中的排版效果仅是业务上所实现的一种呈现模式，用户还可以根据需求进行自定义排版。另外，用户可以通过遥控终端进行对应投屏应用界面的操控，进行界面的操控以及分屏全屏模式的各种切换。
<同一投屏协议下多投屏设备投屏同屏展示的过程>
图19为根据一些实施例的同一投屏协议下多投屏设备投屏的效果示意图，如图19所示，投屏设备A和投屏设备B均基于NFC协议与显示设备200建立连接。首先，投屏设备A和投屏设备B均为支持同一投屏协议的投屏设备，两个投屏设备均与显示设备通过投屏协议服务建立连接。首先根据协议层进行设备的发现，设备发现后，显示设备进行指令监听，于移动端建立指令通道，从而进行指令协议的交互。指令协议交互后达成一致后会进行数据的传输，即视频流的传输，显示设备对投屏设备进行数据监听，在获取投屏设备的投屏数据后进行相应的界面展示。然后，投屏协议需要与客户端Group进行显示设备投屏设备的管理。通过Group内不同的组员实现对投屏设备的管理。大致分两个方面进行管理，一个是连接管理，通过组内成员个数获取与显示设备投屏连接的投屏设备的数量，为视频流最终展示的效果做准备，另一个是视频流管理，将各个投屏设备所发送的视频流进行分类解码处理。
最后，进行投屏视频流展示处理。若是此时显示设备200当前投屏模式为全屏模式， 则显示设备200将最后接入的投屏设备的投屏内容进行全屏展示。若是当前投屏模式为分屏模式，可以分为如下两种排版情况，情况1用户需要同步展示TOP应用的情况，这样过来三路投屏视频流界面后，可以与TOP应用同步展示，以四分屏的方式进行展示；情况2是用户不需要同步展示TOP应用的情况，这样过来的三路投屏视频流界面要进行排版处理，具体各自投屏界面的排版位置可以根据视频流先后顺序进行排版，展现形式不局限于当前模板所示。若是当前投屏模式为智能模式，进一步判断最后接入的投屏设备的投屏内容。若最后接入的投屏设备的投屏内容为竖向媒资，则按照分屏模式下的处理方式，将多个所述投屏设备的投屏内容分别于多个所述窗口进行展示，若最后接入的所述投屏设备的投屏内容为横向媒资，按照全屏模式下的处理方式，则将最后接入的所述投屏设备的投屏内容于所述显示器上全屏展示。
<不同投屏协议下多投屏设备投屏同屏展示的过程>
图20为根据一些实施例的不同协议下多投屏设备投屏的效果示意图，如图20所示，以NFC、Airplay、DLNA投屏为例，显示设备200有相应三个协议的投屏服务，NFC协议服务、Airplay协议服务、DLNA协议服务，用以承接移动端的推送。
步骤一：投屏设备A、投屏设备B和投屏设备C分别与显示设备通过相对应的协议服务建立连接，如，显示设备200通过利用NFC投屏服务接收支持NFC协议的投屏设备A的投屏请求，并利用NFC投屏服务监听投屏设备A连接，利用Airplay投屏服务接收支持Airplay协议的投屏设备B的投屏请求，并利用Airplay投屏服务监听投屏设备B连接，利用DLNA投屏服务接收支持DLNA协议的投屏设备C的投屏请求，并利用DLNA投屏服务监听投屏设备C连接。
图21为根据一些实施例的投屏设备与显示设备建立连接的示意图，如图21所示，投屏设备与显示设备之间建立指令通道和数据通道的过程，首先根据协议层进行设备的发现，设备发现后，显示设备进行指令监听，于移动端建立指令通道，从而进行指令协议的交互。指令协议交互后达成一致后会进行数据的传输，即视频流的传输，显示设备对投屏设备进行数据监听，在获取投屏设备的投屏数据后进行相应的界面展示。
步骤二：各自投屏协议需要与应用层中枢模块进行显示设备投屏设备的管理，也就是在应用层中枢模块进行枢纽处理，大致分两个方面进行管理，一个是连接管理，负责统计与显示设备投屏连接的投屏设备的数量，为视频流最终展示的效果做准备，另一个是视频流管理，视频流管理负责将各个不同投屏协议过来的视频流进行分类解码处理，以使显示设备200在窗口中展示出投屏内容。
步骤三：进行投屏视频流展示处理。若是此时显示设备200当前投屏模式为全屏模式，则显示设备200将最后接入的投屏设备的投屏内容进行全屏展示。若是当前投屏模式为分屏模式，可以分为如下两种排版情况，情况1用户需要同步展示TOP应用的情况，这样过来三路投屏视频流界面后，可以与TOP应用同步展示，以四分屏的方式进行展示；情况2是用户不需要同步展示TOP应用的情况，这样过来的三路投屏视频流界面要进行排版处理，具体各自投屏界面的排版位置可以根据视频流先后顺序进行排版，展现形式不局限于当前模板所示。若是当前投屏模式为智能模式，进一步判断最后接入的投屏设备的投屏内容。若最后接入的投屏设备的投屏内容为竖向媒资，则按照分屏模式下的处理方式，将多个所述投屏设备的投屏内容分别于多个所述窗口进行展示，若最后接入的所述投屏设备的投屏内容为横向媒资，按照全屏模式下的处理方式，则将最后接入的所述投屏设备的投屏内容 于所述显示器上全屏展示。
本申请中，显示设备200可以接收来自不同投屏协议下多投屏设备的投屏请求或者同一投屏协议下多投屏设备的投屏请求，并于显示器的用户界面上同屏展示多投屏设备的投屏界面，提高了用户体验。
本申请一些实施例提供一种多设备投屏同屏显示的控制方法，包括：利用同一投屏服务分别与多个投屏设备建立连接，以使显示设备接收多个投屏设备的投屏内容，其中，多个投屏设备支持同一投屏协议。在当前投屏模式为全屏模式时，将最后接入的所述投屏设备的投屏内容于所述显示器上全屏展示。在当前投屏模式为分屏模式时，将多个所述投屏设备的投屏内容分别于多个窗口进行展示。在当前投屏模式为智能模式时，若最后接入的所述投屏设备的投屏内容为竖向媒资，则将多个所述投屏设备的投屏内容分别于多个所述窗口进行展示，若最后接入的所述投屏设备的投屏内容为横向媒资，则将最后接入的所述投屏设备的投屏内容于所述显示器上全屏展示。本申请中，显示设备可以多窗口显示媒资内容，接入多个投屏设备，并在不同的投屏模式下进行投屏内容的展示，极大的提升了用户体验。
在一些实施例中，在所述用户界面包括一个窗口时，控制所述显示器全屏展示最后接入的所述投屏设备的投屏内容。
在一些实施例中，在与多个投屏设备建立连接时，包括：通过同一投屏服务接收多个所述投屏设备的投屏请求；响应于所述投屏请求，利用所述投屏服务监听所述投屏设备连接，对所述投屏设备进行组管理，其中，不同的所述投屏设备对应组内不同成员；利用组内成员与所述投屏设备搭建指令通道和数据通道；所述指令通道用于所述投屏设备和所述显示设备之间的指令交互，所述数据通道用于所述投屏设备和所述显示设备之间的数据交互。
在一些实施例中，在不同的所述投屏设备对应组内不同成员时，控制投屏设备绑定相应的设备标识，其中，所述设备标识用于区分组内不同成员。
在一些实施例中，所述在控制多个窗口分别对应展示多个所述投屏设备的投屏内容时，若保留用户界面所显示内容，则控制多个窗口分别对应展示用户界面及多个所述投屏设备的投屏内容；若不保留用户界面所显示内容，则控制多个窗口分别对应展示多个所述投屏设备的投屏内容。
在一些实施例中，在所述若保留用户界面所显示内容，则控制多个窗口分别对应展示用户界面及多个所述投屏设备的投屏内容，包括：在当前用户界面支持分屏展示时，启动相应数量的窗口，将用户界面及多个所述投屏设备的投屏内容通过活动管理器接口分别推送至多个所述窗口；在当前用户界面不支持分屏展示时，启动相应数量的窗口，将等比例缩小后的用户界面及多个所述投屏设备的投屏内容通过活动管理器接口分别推送至多个所述窗口。
在一些实施例中，在控制多个窗口分别对应展示多个所述投屏设备的投屏内容时，接收输入的对任一窗口内媒资控件的选中操作，其中，所述媒资控件用于展示对应的多媒体资源；根据所述选中操作，若所述对应的多媒体资源为横向媒资时，在所述媒资控件所属的窗口内全屏播放对应的多媒体资源。
在一些实施例中，在将用户界面等比例缩小时，重新设置所述用户界面的坐标和宽高，以使所述用户界面等比例缩小。
本申请一些实施例提供一种多设备投屏同屏显示的控制方法，包括：利用多个投屏服务分别与多个投屏设备建立连接，以使显示设备接收多个投屏设备的投屏内容，其中，多个投屏设备支持不同投屏协议。在当前投屏模式为全屏模式时，将最后接入的所述投屏设备的投屏内容于所述显示器上全屏展示。在当前投屏模式为分屏模式时，将多个所述投屏设备的投屏内容分别于多个窗口进行展示。在当前投屏模式为智能模式时，若最后接入的所述投屏设备的投屏内容为竖向媒资，则将多个所述投屏设备的投屏内容分别于多个所述窗口进行展示，若最后接入的所述投屏设备的投屏内容为横向媒资，则将最后接入的所述投屏设备的投屏内容于所述显示器上全屏展示。
<焦点切换>
安卓本身存在多窗口，随着窗口的打开，通常情况下是最后打开的窗口拥有焦点安卓对于具有指向性的Touch事件可以直接分发到用户所点击的应用，但是对于Input的事件，由于Input无指向性，所以只能分发到当前焦点所在的任务栈。目前，当显示设备200处于分屏模式时，显示器260所显示的用户界面被划分为多个窗口，对多个窗口来说，用户只能通过控制装置100控制位于用户界面顶层的窗口内的内容，而无法控制其他窗口内的内容。因此，由于用户无法在多个窗口之间进行选择性控制，导致体验效果不佳。
针对上述问题，在一些实施例中提供了一种显示设备，在此，为了便于描述，本申请将具有焦点且响应用户对窗口内所展示内容进行操作的状态称为第一状态，也就是说，在第一状态下，窗口内所展示内容能够获取焦点，如图5c所示，展示有应用列表控件的第一窗口为第一状态下的窗口，该窗口通过焦点框进行标定，在第一窗口内可实现用户对APP的操作。将具有焦点但不响应用户对窗口内所展示内容进行操作的状态称为第二状态，也就是说，第二状态下的窗口上能够获取焦点，第二状态下的窗口也可通过焦点框进行标定，用于提示当前第二状态的窗口位置，需要说明的是，第一状态与第二状态的焦点框的形式不同，可以是颜色不同，可以是线条粗细不同，还可以是线条的形式不同等；将窗口不获取焦点的状态成为第三状态，例如，图5c中的第二窗口、第三窗口和第四窗口均为第三状态下的窗口。
在一些实施例中，在显示设备200进入分屏模式时，即显示器260显示多个窗口时，此时，根据对显示设备200的预先设置，可默认第一窗口为第一状态下窗口，也就是说，在显示设备200进入分屏模式后，用户可通过控制装置100对第一窗口内所展示的内容进行操控。为了指示当前焦点位置，可通过第一标记对第一状态下的窗口进行标定。例如，将第一状态下的窗口通过焦点框进行标定，该焦点框的颜色可以为红色粗线条，在此不做限定。当用户看到通过红色粗线条的焦点框标定的窗口时，即可得知此时该窗口处于第一状态下。
在一些实施例中，在用户对第一状态下的窗口所展示内容进行操作时，显示所述第一标记，在用户对第一状态下的窗口所展示内容不进行操作时，隐藏所述第一标记。也就是说，为了保证用户的视觉体验，显示设备200可仅通过所述第一标记的闪烁状态来提示用户目前被标定为第一状态的窗口是哪个窗口。当用户想要切换窗口，对其他窗口内所展示的内容进行控制时，可以向显示设备200发送触发指令，显示设备200中的控制器250在接收到该触发指令后，触发显示设备200的切换窗口模式。
在一些实施例中，控制器250根据接收到的触发指令将第一状态下的第一窗口标定为第二状态，包括：根据所述触发指令，将焦点移动至多个窗口之间，将所述第一窗口的第 一标记更换为第二标记。此时，显示设备200进入切换窗口模式，在该模式下，显示器260上的所有窗口中不存在第一状态下的窗口，仅有第二状态下的窗口，换句话说，在该模式下用户只可以移动焦点进行窗口之间的切换，无法操作窗口内所展示的内容。
在一些实施例中，对于标定为第二状态的窗口来说，可通过所述第二标记进行标定，所述第一标记和所述第二标记为不同形式的焦点框。也就是说，用来标定第二状态的焦点框的形式与标定第一状态的焦点框的形式不同，例如，可以选为绿色细线条。当用户触发显示设备200的切换窗口模式后，通过显示器260上多个窗口的空间排布情况，进一步发出选择指令。控制器250接收用户发出的选择指令，将用户选择的窗口标定为第二状态，并将第一窗口由第二状态标定为第三状态。
在一些实施例中，在控制器250接收到用户发送的选择指令后，计算用户所选定的目标窗口，将目标窗口置于显示器260的最顶层，并将焦点传递至该目标窗口，使得目标窗口标定为第二状态。图5d为一些实施例的在分屏模式下切换窗口的显示效果示意图，如图5d所示，当前第一状态下的窗口为第一窗口，假如用户想要操控第四窗口内的媒资资源，则需要根据第四窗口与第一窗口之间的相对位置，通过遥控器发送“向右”，再“向下”的选择指令，将第四窗口标定为第二状态。在接收到用户发送的确认指令后，将第四窗口标定为第一状态。
在一些实施例中，在控制焦点传递至目标窗口时，焦点所移动至的每一窗口，控制器250均通过第二标记将其标定为第二状态，在焦点移走后，再将其第二状态重置为第三状态。也就是说，在切换窗口模式下，焦点所在的窗口均通过边框标定为第二状态。用户可通过第二标记的移动，判断焦点的移动轨迹以及当前所在的窗口。承接上述例子，图5d中，当用户通过遥控器发送“向右”的选择指令时，控制器控制焦点移动至第二窗口，此时将第二窗口通过绿色细线条的焦点框标定为第二状态，在接收到用户通过遥控器发送“向下”的选择指令时，将焦点移动至第四窗口，并将第四窗口通过绿色细线条的焦点框标定为第二状态，同时，还需将第二窗口重置为第三状态，也就是取消第二标记标定。
在一些实施例中，根据用户的选择指令将用户选择的窗口标定为第二状态后，在接收到用户发出的确认指令，将第二状态下的窗口标定为第一状态，相应的，将该窗口的第二标记更换为第一标记，例如，将该窗口的绿色细线条焦点框标定改为红色粗线条焦点框标定。确认指令可由用户通过遥控器上的“确认”方向键发出。还可以由用户通过语音控制来发出，例如，用户在向显示设备输入唤醒词之后，继续对显示设备200输入“确认当前选择”。以上对根据用户指令控制焦点在多个窗口之间移动的过程进行了说明。当然，用户还可在选定第一状态的窗口后，控制焦点在窗口内进行移动。
在一些实施例中，在显示设备200为分屏模式时，接收用户输入的选择指令，根据选择指令，控制焦点在第一状态下的窗口内的各个应用控件之间传递。焦点所移动至的每一应用控件，均通过背景颜色的改变标定为聚焦应用。当用户想要控制当前第一状态下窗口内的应用程序时，可通过遥控器上的“上、下、左、右”方向键发出选择指令，进而控制选中对应应用程序的控件，被选定的应用程序，可通过背景颜色加深来指示当前应用程序已被选定。
为了进一步解释上述多窗口切换的过程，本申请一些实施例还提供了一种用户界面的控制方法。该用户界面的控制方法包括：控制器250接收用户输入的用于触发切换窗口模式的触发指令，在显示器所显示的用户界面包括多个窗口时，根据所述触发指令将第一状 态下的第一窗口标定为第二状态。进一步接收用户输入的用于选择窗口的选择指令，根据所述选择指令，将用户选择的窗口标定为所述第二状态，并将所述第一窗口由第二状态标定为第三状态，其中，所述第三状态指的是不具有焦点的状态。最后接收用户输入的确认指令，根据所述确认指令，将第二状态下的窗口标定为第一状态。
下面结合附图对该用户界面的控制方法做进一步说明。
图22为根据一些实施例的在分屏模式下切换窗口的焦点传递流程示意图，如图22所示，在一些实施例中，对于显示设备200来说，首先接收用户发送的按键值，即获取按键事件。然后，控制器250判断该按键事件是否是接收到用户所发送的触发指令。若是触发指令，则触发显示设备200进入切换窗口模式，也就是将当前第一状态下的窗口标定为第二状态，将红色粗线条焦点框标定的窗口更改为绿色细线条焦点框标定的窗口，并进一步等待用户做出选择。若不是触发指令，则直接将按键事件分发至当前焦点任务所指向的应用。然后，在接收到用户发送的选择指令时，获取当前焦点任务，即寻找当前焦点所位于哪个窗口。在获取到当前焦点所在位置后，根据用户发送的选择指令计算目标窗口。最后将目标窗口置于顶层，获取到焦点，即将目标窗口标定为第一状态。
在一些实施例中，所述第一状态和所述第二状态下的窗口分别通过第一标记和第二标记进行标定，其中，所述第一标记和所述第二标记为不同形式的焦点框。
在一些实施例中，所述根据所述触发指令将第一状态下的第一窗口标定为第二状态，包括：根据所述触发指令，将焦点移动至多个窗口之间；将所述第一窗口的第一标记更换为第二标记。
在一些实施例中，在用户对所述第一状态下的窗口内的所展示内容进行操作时，显示所述第一标记；在用户对所述第一状态下的窗口内的所展示内容不进行操作时，隐藏所述第一标记。
在一些实施例中，所述根据所述选择指令，将相应的窗口标定为所述第二状态，包括：根据所述选择指令，计算目标窗口；将所述目标窗口置于所述显示器最顶层；控制焦点传递至所述目标窗口，将所述目标窗口标定为所述第二状态。
在一些实施例中，焦点所移动至的窗口，均通过所述第二标记标定为所述第二状态，且在所述焦点移走后，将所述第二状态重置为第三状态。
在一些实施例中，在所述用户界面包含多个窗口时，接收用户输入的选择指令；根据所述选择指令，控制焦点在所述第一状态下的窗口内的各个应用控件之间传递。
下面结合附图，解释一下本申请实施例所提供的显示设备处于分屏模式阶段的数据交互过程。
图23为根据一些实施例的显示设备处于分屏模式阶段的时序图。在一些实施例中，如图23所示，显示设备200基于分屏窗口服务(SplitWindowService)、分屏窗口业务(HSSplitWindowUsercase)和Activity管理服务(ActivityManagerService)之间的数据交互实现分屏模式的启动和应用。首先，对于分屏启动阶段来说，分屏窗口服务在接收到启动分屏模式的指令后，通过调用分屏启动接口控制分屏窗口业务进行分屏模式的启动，分屏窗口业务对应用栈进行操作，把对应的栈放到相应的位置，Activity管理服务通知分屏窗口业务分屏模式启动完成，分屏窗口业务将第一个底层页面获得焦点，即设置默认获得焦点的窗口。例如，分屏窗口业务此时对堆栈进行操作，如果是分屏模式中三分屏，则通过Activity管理服务启动3个堆栈并按显示顺序依次排列，Activity管理服务通过回调通知 分屏窗口业务分屏启动完成，此时将第一个底层Task任务所在的页面获得焦点。然后是分屏应用启动阶段，用户点击某个底层页面上的窗口，也就是在某个窗口中启动应用，分屏窗口服务通过Activity管理服务启动并创建该窗口，并且将焦点移动至该窗口，使该窗口获取焦点。由于显示界面默认都是1080P，因此需要将窗口的大小进行调整，即将其等比例缩小，通过设置位置和宽高的方式改变应用大小，将其改为1080P，并放置在相应的位置。最后，对于多窗口焦点切换来说，分屏窗口服务获得用户输出的按键事件，在接收到触发指令时触发显示设备200进入切换窗口模式，此时分屏窗口业务将焦点设置到外层，即选择窗口所在的task任务，并标注当前是切换窗口模式，当选定窗口后，分屏窗口业务对该当前窗口进行标定提示，用于指示当前焦点位置。在用户按下确认键后，将焦点窗口所在的task任务内的应用设置为获取焦点，进行标注提示以指示焦点当前位置。
当显示设备200处于切换窗口模式时，也就是焦点在多个窗口之间移动时，框架层的窗口获取窗口信息，此处窗口信息包括位置、所在task任务等，框架层的活动响应按键操作。按键移动需要做处理，这时移动的是外层各个窗口之间的焦点，而不是窗口内应用中控件上的焦点。当选定第一状态下的窗口后，将焦点移动至窗口内，也就是用户可在窗口内应用中控件上移动焦点，此时，框架层的窗口、位置、活动参与，按键首先将按键分发到当前窗口，然后再分发给应用，上下左右移动时，会移动应用内控件焦点的位置。
本申请中，在智能电视处于分屏模式时，用户能够实现在不同窗口之间的焦点切换过程，自由操作点击任一窗口内的任一应用程序，极大的提升了用户体验。
为了方便解释，已经结合具体的实施方式进行了上述说明。但是，上述在一些实施例中讨论不是意图穷尽或者将实施方式限定到上述公开的具体形式。根据上述的教导，可以得到多种修改和变形。上述实施方式的选择和描述是为了更好的解释原理以及实际的应用，从而使得本领域技术人员更好的使用实施方式以及适于具体使用考虑的各种不同的变形的实施方式。

Claims (16)

  1. 一种显示设备，包括：
    显示器，被配置为显示用户界面，所述用户界面中包括多个窗口；
    控制器，所述控制器被配置为：
    响应于多个投屏设备的投屏请求，与多个所述投屏设备建立连接，其中，多个投屏设备支持同一投屏协议；
    接收多个所述投屏设备所发送的投屏内容；
    在最后接入的所述投屏设备的投屏内容为竖向媒资时，控制多个窗口分别对应展示多个所述投屏设备的投屏内容；
    在最后接入的所述投屏设备的投屏内容为横向媒资时，控制所述显示器全屏展示最后接入的所述投屏设备的投屏内容。
  2. 根据权利要求1所述的显示设备，所述控制器还被配置为：
    在所述用户界面包括一个窗口时，控制所述显示器全屏展示最后接入的所述投屏设备的投屏内容。
  3. 根据权利要求1所述的显示设备，所述控制器还被配置为：
    在与多个所述投屏设备建立连接时，
    通过同一投屏服务接收多个所述投屏设备的投屏请求；
    响应于所述投屏请求，利用所述投屏服务监听所述投屏设备连接，对所述投屏设备进行组管理，其中，不同的所述投屏设备对应组内不同成员；
    利用组内成员与所述投屏设备搭建指令通道和数据通道，其中，所述指令通道用于所述投屏设备和所述显示设备之间的指令交互，所述数据通道用于所述投屏设备和所述显示设备之间的数据交互。
  4. 根据权利要求3所述的显示设备，所述控制器还被配置为：
    在不同的所述投屏设备对应组内不同成员时，控制投屏设备绑定相应的设备标识，其中，所述设备标识用于区分组内不同成员。
  5. 根据权利要求1所述的显示设备，所述控制器还被配置为：
    在控制多个窗口分别对应展示多个所述投屏设备的投屏内容时，
    若保留用户界面所显示内容，则控制多个窗口分别对应展示用户界面及多个所述投屏设备的投屏内容；
    若不保留用户界面所显示内容，则控制多个窗口分别对应展示多个所述投屏设备的投屏内容。
  6. 根据权利要求5所述的显示设备，所述控制器还被配置为：
    在所述若保留用户界面所显示内容，则控制多个窗口分别对应展示用户界面及多个所述投屏设备的投屏内容时，
    在当前用户界面支持分屏展示时，启动相应数量的窗口，将用户界面及多个所述投屏设备的投屏内容通过活动管理器接口分别推送至多个所述窗口；
    在当前用户界面不支持分屏展示时，启动相应数量的窗口，将等比例缩小后的用户界面及多个所述投屏设备的投屏内容通过活动管理器接口分别推送至多个所述窗口。
  7. 根据权利要求1所述的显示设备，所述控制器还被配置为：
    在控制多个窗口分别对应展示多个所述投屏设备的投屏内容时，
    接收输入的对任一窗口内媒资控件的选中操作，其中，所述媒资控件用于展示对应的多媒体资源；
    根据所述选中操作，若所述对应的多媒体资源为横向媒资时，在所述媒资控件所属的窗口内全屏播放对应的多媒体资源。
  8. 根据权利要求6所述的显示设备，所述控制器还被配置为：
    在将用户界面等比例缩小时，
    重新设置所述用户界面的坐标和宽高，以使所述用户界面等比例缩小。
  9. 一种多设备投屏同屏显示的控制方法，包括：
    响应于多个投屏设备的投屏请求，与多个所述投屏设备建立连接，其中，多个投屏设备支持同一投屏协议；
    接收多个所述投屏设备所发送的投屏内容；
    在最后接入的所述投屏设备的投屏内容为竖向媒资时，控制多个窗口分别对应展示多个所述投屏设备的投屏内容；
    在最后接入的所述投屏设备的投屏内容为横向媒资时，控制所述显示器全屏展示最后接入的所述投屏设备的投屏内容。
  10. 根据权利要求9所述的控制方法，还包括：
    在用户界面包括一个窗口时，控制所述显示器全屏展示最后接入的所述投屏设备的投屏内容。
  11. 根据权利要求9所述的控制方法，还包括：在与多个所述投屏设备建立连接时，
    通过同一投屏服务接收多个所述投屏设备的投屏请求；
    响应于所述投屏请求，利用所述投屏服务监听所述投屏设备连接，对所述投屏设备进行组管理，其中，不同的所述投屏设备对应组内不同成员；
    利用组内成员与所述投屏设备搭建指令通道和数据通道，其中，所述指令通道用于所述投屏设备和所述显示设备之间的指令交互，所述数据通道用于所述投屏设备和所述显示设备之间的数据交互。
  12. 根据权利要求11所述的控制方法，还包括：在不同的所述投屏设备对应组内不同成员时，控制投屏设备绑定相应的设备标识，其中，所述设备标识用于区分组内不同成员。
  13. 根据权利要求9所述的控制方法，还包括：在控制多个窗口分别对应展示多个所述投屏设备的投屏内容时，
    若保留用户界面所显示内容，则控制多个窗口分别对应展示用户界面及多个所述投屏设备的投屏内容；
    若不保留用户界面所显示内容，则控制多个窗口分别对应展示多个所述投屏设备的投屏内容。
  14. 根据权利要求13所述的控制方法，还包括：在所述若保留用户界面所显示内容，则控制多个窗口分别对应展示用户界面及多个所述投屏设备的投屏内容时，
    在当前用户界面支持分屏展示时，启动相应数量的窗口，将用户界面及多个所述投屏设备的投屏内容通过活动管理器接口分别推送至多个所述窗口；
    在当前用户界面不支持分屏展示时，启动相应数量的窗口，将等比例缩小后的用户界 面及多个所述投屏设备的投屏内容通过活动管理器接口分别推送至多个所述窗口。
  15. 根据权利要求9所述的控制方法，还包括：在控制多个窗口分别对应展示多个所述投屏设备的投屏内容时，
    接收输入的对任一窗口内媒资控件的选中操作，其中，所述媒资控件用于展示对应的多媒体资源；
    根据所述选中操作，若所述对应的多媒体资源为横向媒资时，在所述媒资控件所属的窗口内全屏播放对应的多媒体资源。
  16. 根据权利要求14的控制方法，包括：
    在将用户界面等比例缩小时，
    重新设置所述用户界面的坐标和宽高，以使所述用户界面等比例缩小。
PCT/CN2022/084370 2021-04-30 2022-03-31 显示设备及多设备投屏同屏显示的控制方法 WO2022228021A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202280031755.0A CN117581191A (zh) 2021-04-30 2022-03-31 显示设备及多设备投屏同屏显示的控制方法
US18/384,618 US20240053944A1 (en) 2021-04-30 2023-10-27 Display apparatus and method for controlling screen projections from multiple devices to same screen

Applications Claiming Priority (8)

Application Number Priority Date Filing Date Title
CN202110484270.4 2021-04-30
CN202110484270 2021-04-30
CN202110586572.2 2021-05-27
CN202110586572.2A CN114296579A (zh) 2021-04-30 2021-05-27 显示设备及用户界面的控制方法
CN202110702220.9 2021-06-24
CN202110702282.X 2021-06-24
CN202110702282.XA CN114327327B (zh) 2021-04-30 2021-06-24 显示设备及多设备投屏同屏显示的控制方法
CN202110702220.9A CN114296670B (zh) 2021-04-30 2021-06-24 显示设备及多设备投屏同屏显示的控制方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/384,618 Continuation US20240053944A1 (en) 2021-04-30 2023-10-27 Display apparatus and method for controlling screen projections from multiple devices to same screen

Publications (1)

Publication Number Publication Date
WO2022228021A1 true WO2022228021A1 (zh) 2022-11-03

Family

ID=80964027

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/CN2022/084370 WO2022228021A1 (zh) 2021-04-30 2022-03-31 显示设备及多设备投屏同屏显示的控制方法
PCT/CN2022/090561 WO2022228572A1 (zh) 2021-04-30 2022-04-29 显示设备及用户界面的控制方法

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/090561 WO2022228572A1 (zh) 2021-04-30 2022-04-29 显示设备及用户界面的控制方法

Country Status (3)

Country Link
US (1) US20240053944A1 (zh)
CN (7) CN114296580A (zh)
WO (2) WO2022228021A1 (zh)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114296580A (zh) * 2021-04-30 2022-04-08 海信视像科技股份有限公司 显示设备及用户界面的控制方法
CN115190353A (zh) * 2022-06-30 2022-10-14 京东方科技集团股份有限公司 投屏系统、方法、计算机设备、存储介质及程序产品
CN114866998B (zh) * 2022-07-06 2022-11-11 荣耀终端有限公司 一种蓝牙连接方法及蓝牙设备
CN115442574A (zh) * 2022-07-12 2022-12-06 北京罗克维尔斯科技有限公司 投屏选择控制器、控制系统、控制方法及车辆
CN115915076A (zh) * 2022-08-31 2023-04-04 深圳方位通讯科技有限公司 一种蓝牙外线功能实现方法、移动终端及存储介质
WO2024139245A1 (zh) * 2022-12-30 2024-07-04 海信视像科技股份有限公司 一种显示设备及分屏显示方法
WO2024169421A1 (zh) * 2023-02-17 2024-08-22 海信视像科技股份有限公司 显示设备和显示处理方法
JP7560625B1 (ja) 2023-08-07 2024-10-02 レノボ・シンガポール・プライベート・リミテッド 情報処理装置、及び制御方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017034298A1 (ko) * 2015-08-26 2017-03-02 엘지전자 주식회사 디지털 디바이스 및 상기 디지털 디바이스에서 데이터 처리 방법
CN106506535A (zh) * 2016-12-14 2017-03-15 北京火舞科技有限公司 一种多人同屏互动的交互系统
CN110381197A (zh) * 2019-06-27 2019-10-25 华为技术有限公司 多对一投屏中音频数据的处理方法、装置及系统
CN111796784A (zh) * 2020-06-12 2020-10-20 彭程 投屏方法、电子设备及投屏终端
CN112988102A (zh) * 2021-05-11 2021-06-18 荣耀终端有限公司 投屏方法和装置

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003518831A (ja) * 1999-12-22 2003-06-10 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ 複数ウインドウディスプレイシステム
CN100367187C (zh) * 2005-04-30 2008-02-06 艾威梯软件技术(北京)有限公司 一种同时支持多路蓝牙音频应用的方法
JP4315197B2 (ja) * 2006-12-27 2009-08-19 ソニー株式会社 情報処理装置、表示制御方法、およびプログラム
CN103309555B (zh) * 2012-10-31 2016-08-03 中兴通讯股份有限公司 基于多窗口的焦点切换的方法及装置
CN103561220A (zh) * 2013-10-28 2014-02-05 三星电子（中国）研发中心 一种电视终端及其多屏显示和控制的方法
CN103677727A (zh) * 2013-12-05 2014-03-26 深圳市中兴移动通信有限公司 移动终端和音频控制方法
CN105005427A (zh) * 2014-04-23 2015-10-28 中兴通讯股份有限公司 一种窗口管理方法及装置
CN105100907B (zh) * 2014-04-28 2018-05-15 宇龙计算机通信科技(深圳)有限公司 选择性投屏的方法及其装置
KR20160060846A (ko) * 2014-11-20 2016-05-31 삼성전자주식회사 디스플레이 장치 및 디스플레이 방법
US10681479B2 (en) * 2015-01-30 2020-06-09 Cassia Networks Inc. Methods, devices and systems for bluetooth audio transmission
KR101586321B1 (ko) * 2015-03-11 2016-01-19 엘지전자 주식회사 디스플레이 디바이스 및 그 제어 방법
CN106454506B (zh) * 2016-11-15 2019-05-07 四川长虹电器股份有限公司 智能电视多窗口环境下实现音频路由控制的方法
CN106792192B (zh) * 2016-12-21 2020-03-17 深圳Tcl数字技术有限公司 数字电视多窗口播放方法与装置
CN107147929A (zh) * 2017-06-22 2017-09-08 深圳Tcl数字技术有限公司 多窗口声音输出方法、电视机以及计算机可读存储介质
CN108206886A (zh) * 2017-09-08 2018-06-26 中兴通讯股份有限公司 一种音频播放方法和装置、及终端
CN110032349B (zh) * 2019-03-29 2021-10-22 联想(北京)有限公司 控制方法、展示方法、控制装置及计算机可读存储介质
CN112073797B (zh) * 2019-06-10 2022-06-14 海信视像科技股份有限公司 一种音量调节方法及显示设备
CN112073761B (zh) * 2019-06-10 2022-09-09 海信视像科技股份有限公司 蓝牙功能的实现方法及装置、双硬件系统显示设备
CN110308885B (zh) * 2019-06-25 2022-04-01 维沃移动通信有限公司 一种投屏方法及移动终端
CN110415716B (zh) * 2019-07-05 2021-11-26 达闼机器人有限公司 音频混合方法、装置、存储介质及电子设备
CN112463269B (zh) * 2019-09-06 2022-03-15 青岛海信传媒网络技术有限公司 用户界面显示方法及显示设备
CN111131866B (zh) * 2019-11-25 2021-06-15 华为技术有限公司 一种投屏音视频播放方法及电子设备
CN111277907A (zh) * 2020-01-16 2020-06-12 海信视像科技股份有限公司 显示设备以及用户界面的呈现方法
CN211207311U (zh) * 2020-02-28 2020-08-07 深圳市嘉利达专显科技有限公司 基于显示器的一屏多显技术
CN111542031B (zh) * 2020-04-22 2022-07-12 海信视像科技股份有限公司 显示设备及蓝牙设备配对方法
CN111654727A (zh) * 2020-06-28 2020-09-11 北京汗粮科技有限公司 一种向大屏终端投屏交互操作的方法
CN112437334A (zh) * 2020-11-10 2021-03-02 青岛海信传媒网络技术有限公司 一种显示设备
CN112423102A (zh) * 2020-11-18 2021-02-26 努比亚技术有限公司 一种小窗投屏控制方法、设备及计算机可读存储介质
CN112486386B (zh) * 2020-11-30 2022-06-24 维沃移动通信有限公司 投屏方法、投屏装置、电子设备和可读存储介质
CN114296580A (zh) * 2021-04-30 2022-04-08 海信视像科技股份有限公司 显示设备及用户界面的控制方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017034298A1 (ko) * 2015-08-26 2017-03-02 엘지전자 주식회사 디지털 디바이스 및 상기 디지털 디바이스에서 데이터 처리 방법
CN106506535A (zh) * 2016-12-14 2017-03-15 北京火舞科技有限公司 一种多人同屏互动的交互系统
CN110381197A (zh) * 2019-06-27 2019-10-25 华为技术有限公司 多对一投屏中音频数据的处理方法、装置及系统
CN111796784A (zh) * 2020-06-12 2020-10-20 彭程 投屏方法、电子设备及投屏终端
CN112988102A (zh) * 2021-05-11 2021-06-18 荣耀终端有限公司 投屏方法和装置

Also Published As

Publication number Publication date
CN114327327B (zh) 2024-08-27
CN114296580A (zh) 2022-04-08
CN114327199A (zh) 2022-04-12
CN114327327A (zh) 2022-04-12
CN114296670A (zh) 2022-04-08
CN114339383A (zh) 2022-04-12
CN114296579A (zh) 2022-04-08
CN114296670B (zh) 2023-09-15
WO2022228572A1 (zh) 2022-11-03
CN117581191A (zh) 2024-02-20
US20240053944A1 (en) 2024-02-15

Similar Documents

Publication Publication Date Title
WO2022228021A1 (zh) 显示设备及多设备投屏同屏显示的控制方法
CN114390359B (zh) 一种消息的展示方法及显示设备
CN113613072A (zh) 一种多路投屏显示方法及显示设备
CN112463750B (zh) 一种文件同步方法及显示设备
WO2023155529A1 (zh) 显示设备、智能家居系统及用于显示设备的多屏控制方法
CN113766305A (zh) 显示设备及镜像投屏音频输出控制方法
WO2021219002A1 (zh) 显示设备
WO2024066538A1 (zh) 显示设备和显示设备控制方法
CN113593488A (zh) 背光调整方法及显示设备
WO2024041033A1 (zh) 显示设备和用于显示设备的设备名称处理方法
CN115243082B (zh) 一种显示设备及终端控制方法
CN114928760B (zh) 控制装置的按键设置方法及显示设备
WO2023071348A1 (zh) 一种显示设备及外接设备的内容的显示方法
WO2022222724A1 (zh) 一种音频输出通道的切换控制方法及显示设备
WO2022083357A1 (zh) 显示设备及摄像头控制的方法
WO2021218096A1 (zh) 一种调整频道控件排序的方法和显示设备
CN116347148A (zh) 一种投屏方法及显示设备
CN114302070A (zh) 显示设备和音频输出方法
WO2023138222A1 (zh) 显示设备和直播方法
CN115278322B (zh) 显示设备、控制设备及显示设备的控制方法
CN114968150A (zh) 投屏控制方法及终端设备
CN115857854A (zh) 显示设备及投屏画面切换方法
CN117615188A (zh) 一种显示设备、终端及终端控制方法
CN117915141A (zh) 一种本机按键控制方法及显示设备
CN113992963A (zh) 一种显示设备及投屏方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22794494

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280031755.0

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22794494

Country of ref document: EP

Kind code of ref document: A1