US20220398059A1 - Multi-window display method, electronic device, and system - Google Patents

Multi-window display method, electronic device, and system Download PDF

Info

Publication number
US20220398059A1
US20220398059A1 US17/770,372 US202017770372A US2022398059A1 US 20220398059 A1 US20220398059 A1 US 20220398059A1 US 202017770372 A US202017770372 A US 202017770372A US 2022398059 A1 US2022398059 A1 US 2022398059A1
Authority
US
United States
Prior art keywords
window
electronic device
mobile phone
application
interface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US17/770,372
Other languages
English (en)
Other versions
US12061833B2 (en
Inventor
Dengkui Zhu
Chen Chen
Hui Xu
Wenyan CAO
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of US20220398059A1 publication Critical patent/US20220398059A1/en
Assigned to HUAWEI TECHNOLOGIES CO., LTD. reassignment HUAWEI TECHNOLOGIES CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ZHU, DENGKUI, XU, HUI, CAO, Wenyan, CHEN, CHEN
Application granted granted Critical
Publication of US12061833B2 publication Critical patent/US12061833B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04803—Split screen, i.e. subdividing the display area or the window area into separate subareas
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2370/00—Aspects of data communication
    • G09G2370/16—Use of wireless transmission of display information

Definitions

  • Embodiments of this application relate to the field of electronic technologies, and in particular, to a multi-window display method, an electronic device, and a system.
  • Mobile office is increasingly popular in an office scenario.
  • the mobile phone has become one of main devices in the mobile office. Therefore, there is an increasing requirement for interconnection between the mobile phone and a personal computer (personal computer, PC), for example, file exchange between the mobile phone and the PC, and projection of a screen of the mobile phone on a display screen of the PC.
  • PC personal computer
  • screen content and data can be easily shared between the PC and the mobile phone by using screen sharing software.
  • a user can receive a notification event on the mobile phone by using the PC, and use a plurality of functions of the mobile phone on the PC.
  • the user can receive a notification on the mobile phone by using the PC, for example, make a call by using the PC, easily answer a call by using a pop-up window, or send/receive information.
  • the user can view an incoming call alarm.
  • the user can share a file between the PC and the mobile phone.
  • the user can quickly share file and text content between the PC and the mobile phone by performing a drag-and-drop operation.
  • the user inserts a photo on the mobile phone into a word document of the PC, or inserts a file on the PC into the mobile phone when the user sends text information or an email.
  • the screen sharing software mainly displays, in one window of the PC through imaging, all content (for example, an application interface) displayed on the screen of the mobile phone. Multi-window display cannot be implemented.
  • This application provides a multi-window display method, an electronic device, and a system.
  • a plurality of windows of a mobile phone can be displayed on a PC, and a user can perform an operation on each mobile phone application by using a window that corresponds to the mobile phone application and that is displayed on the PC, thereby improving office efficiency such as file sharing, and improving user experience.
  • a multi-window display method applied to a system including a first electronic device and a second electronic device that communicate with each other, where the method includes: A first electronic device receives size information of a display area of the second electronic device. The first electronic device runs a first application, and displays a first interface of the first application. The first electronic device determines, based on the size information of the display area of the second electronic device and size information corresponding to the first interface, location information of a first window that is of the first interface and that is in the display area of the second electronic device. The first electronic device sends the location information of the first window and display content information of the first interface to the second electronic device.
  • the second electronic device displays the first interface in the first window based on the received location information of the first window and the received display content information of the first interface. If detecting a first operation, the second electronic device sends an instruction for starting a second application to the first electronic device in response to the first operation. The first electronic device starts the second application based on the received instruction for starting the second application. The first electronic device determines, based on the size information of the display area of the second electronic device and size information corresponding to a second interface of the second application, location information of a second window that is of the second interface and that is in the display area of the second electronic device. The first electronic device sends the location information of the second window and display content information of the second interface to the second electronic device. The second electronic device displays the second interface in the second window based on the received location information of the second window and the received display content information of the second interface. The first window and the second window are different windows.
  • displaying a plurality of windows may be understood as “displaying a plurality of interfaces”, and each window is used to display a different interface.
  • a PC displays a plurality of interfaces of a mobile phone by using different windows, such as the first interface and the second interface.
  • Each interface is an interface that can be controlled independently, such as an operation performed by a user on the first interface and an operation performed by the user on the second interface are independent of each other.
  • the first electronic device and the second electronic device may communicate with each other in a plurality of different manners, for example, communicate with each other in a connection manner or by using a future communications technology, to transmit data.
  • the connection manner may include a plurality of different connection manners such as a wired connection manner or a wireless connection manner.
  • the first electronic device and the second electronic device may be connected to each other by using a USB data cable.
  • a wireless connection between the first electronic device and the second electronic device may be a Wi-Fi connection, a near connection established through near field communication, a connection established through Bluetooth code scanning, or the like.
  • the first electronic device and the second electronic device may transmit data by using the future communications technology such as a 5G communications network by installing different applications or a same application on the mobile phone and the PC.
  • the connection manner between the mobile phone and the PC is not limited in this embodiment of this application.
  • this embodiment of this application sets no limitation on an action of receiving the size information of the display area of the second electronic device by the first electronic device and an action of running the first application and displaying the first interface of the first application by the first electronic device.
  • the first electronic device may first run the first application and display the first interface of the first application, and when the user displays the first interface of the first electronic device by using the second electronic device, an action of obtaining the size information of the display area of the second electronic device by the first electronic device from the second electronic device is triggered.
  • the second electronic device first sends the size information of the display area of the second electronic device to the first electronic device, and then the first electronic device runs the first application and displays the first interface of the first application, and performs another subsequent action.
  • the first electronic device may display the second interface of the second application, that is, the first electronic device runs the second application in the foreground, or the first electronic device does not display the second interface of the second application, that is, the first electronic device runs the second application in the background. This is not limited in this embodiment of this application.
  • first application and the second application may be applications installed on the first electronic device, or may be quick applications that can run instantly. This is not limited in this embodiment of this application.
  • an interface of the mobile phone may be displayed by using the PC.
  • the PC may display a plurality of windows, and each window is used to display a different application.
  • the user may start an application of the mobile phone on the PC, and display the application of the mobile phone by using an independent window.
  • office efficiency such as file sharing can be improved, and an application of the mobile phone can be used on the PC, thereby improving user experience.
  • the method further includes: The first electronic device sends first information of the first application and second information of the second application to the second electronic device.
  • the first information of the first application includes at least one type of information such as a name, an identifier ID, and an icon of the first application.
  • the second information of the second application includes at least one type of information such as a name, an identifier ID, and an icon of the second application.
  • the second electronic device After receiving the first information of the first application and the second information of the second application, the second electronic device automatically displays a third interface.
  • the third interface includes the first information of the first application and the second information of the second application. Alternatively, after detecting a second operation of a user, the second electronic device displays the third interface based on the first information of the first application and the second information of the second application.
  • the mobile phone may send information about all installed applications to the PC, and the PC directly automatically displays the applications of the mobile phone.
  • the user may trigger the PC to send an application obtaining request to the mobile phone, to request to obtain application information of the mobile phone, for example, the first information of the first application and the second information of the second application.
  • the application information of the mobile phone herein may include at least one type of information such as names, identifiers (identification, ID), and icons of all applications of the mobile phone.
  • the PC After receiving the application information, the PC displays each application in a mobile phone window on the PC based on the name, the ID, and the icon of the application, to display all the applications of the mobile phone.
  • the first window is close to a right edge of the display area of the second electronic device.
  • the second window is located on the left of the first window, and does not overlap the first window.
  • a mobile phone window can be prevented from shielding an original application program (for example, My Computer or Recycle Bin) of the PC displayed in an interface of the PC, and the mobile phone window can be prevented from shielding a shortcut menu option such as an input method control menu, a time and date setting menu, or power management of the PC displayed in a lower-right corner area in the interface of the PC.
  • an original application program for example, My Computer or Recycle Bin
  • a shortcut menu option such as an input method control menu, a time and date setting menu, or power management of the PC displayed in a lower-right corner area in the interface of the PC.
  • the second window is located at an upper layer of the first window.
  • the location information of the first window may further include display level information of the first window
  • the location information of the second window further includes display level information of the second window
  • the second window is displayed at an upper level of the first window.
  • the display level information of the first window is sent by the first electronic device to the second electronic device, and the second electronic device determines, based on the display level information of the first window, a layer at which the first window should be displayed.
  • the location information of the first window includes coordinate information of a first vertex of the first window, length information of the first window, and width information of the first window; or the location information of the first window includes coordinate information of four vertices of the first window.
  • the location information of the second window is similar to the location information of the first window. Specifically, the location information of the second window includes coordinate information of a second vertex of the second window, length information of the second window, and width information of the second window; or the location information of the second window includes coordinate information of four vertices of the second window.
  • size information of the first window is determined by the first electronic device based on the size information corresponding to the first interface.
  • the method further includes: The first electronic device obtains size information of a display area of the first electronic device, and determines that the size information of the display area of the first electronic device is the size information corresponding to the first interface, or that the size information corresponding to the first interface is a product of the size information of the display area of the first electronic device and a first multiple.
  • the first multiple is greater than 0 and is not equal to 1.
  • the first electronic device may determine the size information of the first window based on a size of the first interface displayed on the first electronic device. For example, a size of the first window is the same as the size of the first interface displayed on the first electronic device. Alternatively, the size of the first interface displayed on the first electronic device is scaled up or scaled down based on a specific proportion, to obtain the size information of the first window. In the method, it can be ensured that when content in the first interface is displayed in the first window of the second electronic device, a display effect similar to that obtained when the first electronic device is used is provided for the user, so that display is more natural, and user experience is improved.
  • size information of the first window is determined by the first electronic device based on the size information corresponding to the first interface.
  • the method further includes: The first electronic device obtains, from configuration information of the first application, size information existing when the first interface is displayed on the first electronic device, and determines, as the size information corresponding to the first interface, the size information existing when the first interface is displayed on the first electronic device.
  • the first electronic device obtains, from a network device, size information existing when the first interface is displayed on the second electronic device, and determines, as the size information corresponding to the first interface, the size information existing when the first interface is displayed on the second electronic device.
  • the first electronic device may obtain, in different manners, the size information of the first interface displayed on the first electronic device.
  • the first electronic device obtains the size information from the configuration information of the first application, that is, may locally query interface information of the first application on the first electronic device, or requests the network device to query interface information of the second electronic device, to obtain the size information of the first interface displayed on the first electronic device.
  • the first window includes the first interface and a first menu bar; and/or the second window includes the second interface and a second menu bar.
  • the second menu bar includes a close key. After the second electronic device displays the second interface in the second window, the method further includes: If the second electronic device detects an operation performed by the user on the close key, the second electronic device closes the second window, and sends an instruction for closing the second application to the first electronic device. The first electronic device closes the second application based on the instruction for closing the second application.
  • the method further includes: If the second electronic device detects an operation of selecting the first window by the user, the second electronic device switches the first window to an uppermost layer for display.
  • the second menu bar includes a maximize key. After the second electronic device displays the second interface in the second window, the method further includes: If the second electronic device detects an operation performed by the user on the maximize key, the second electronic device displays the second window in full screen.
  • the method further includes: If the first electronic device detects an instruction of the user for closing the second application, the first electronic device closes the second application, and notifies the second electronic device to close the second window. The second electronic device closes the second window based on the notification of the first electronic device.
  • the method further includes: The second electronic device detects an operation of selecting and dragging the second window, where the second window moves from a first location to a second location.
  • the second electronic device sends information about the second location and information about the second application to the first electronic device.
  • the first electronic device receives the information about the second location and the information about the second application that are sent by the second electronic device, determines the second application based on the information about the second application, and determines a display location of display content in the second interface based on the information about the second location.
  • a window that is of each application and that is displayed on the PC may receive an operation of the user, for example, an operation of closing, moving, scaling, maximizing, and minimizing the window.
  • the user may perform an operation on each mobile phone application in a window that corresponds to the mobile phone application and that is displayed on the PC. For example, the user taps/clicks a control such as Play and Next in a window that is of a music application and that is on the PC, to control a music play process.
  • an operation performed by the user on the mobile phone application on the PC does not affect use of another function of the user on the mobile phone, for example, the foregoing scenario in which the user plays music by using the mobile phone and views a video by using the PC at the same time.
  • the user may perform an operation on any control of a mobile phone application in a window that is of the application and that is displayed on the PC.
  • the operation may be transferred back to the mobile phone, and the mobile phone executes an operation result, and transfers an execution result to the PC, so that the operation result executed by the mobile phone can be synchronized in the window that is of the application and that is on the PC, thereby improving use efficiency and use experience of the mobile phone application.
  • the method further includes: The first electronic device sends audio data of the second application to the second electronic device.
  • the second electronic device plays the audio data of the second application.
  • a window corresponding to the music application can be displayed on the PC.
  • the mobile phone sends audio data to the PC, and the PC plays the audio data.
  • the music audio may be actually played on a device in which the music application is started. Specifically, if the user taps the music application on the mobile phone, a device that actually plays the music audio is the mobile phone. The user plays the music in a mobile phone window on the PC by using the music application. In this case, the device that actually plays the music audio is the PC.
  • the first electronic device sends the location information of the first window and display content information of the first interface to the second electronic device includes: The first electronic device encodes the location information of the first window and the display content information of the first interface, to generate a first video stream, and sends the first video stream to the second electronic device. The second electronic device receives the first video stream sent by the first electronic device, and decodes the first video stream to obtain the location information of the first window and the display content information of the first interface.
  • That the first electronic device sends the location information of the second window and display content information of the second interface to the second electronic device includes: The first electronic device encodes the location information of the second window and the display content information of the second interface, to generate a second video stream, and sends the second video stream to the second electronic device. The second electronic device receives the second video stream sent by the first electronic device, and decodes the second video stream to obtain the location information of the second window and the display content information of the second interface.
  • a video stream transmission process may be used:
  • the mobile phone encodes all information to be sent to the PC, to generate a video stream, for example, encodes interface content information of the music application and location information of the window that is of the music application and that is displayed on the PC, to generate a video stream, and sends the video stream to the PC.
  • the PC After receiving and decoding the video stream, the PC accurately displays interface content of the music application in the window of the music application.
  • an application program of the mobile phone can be displayed in a mobile phone window on the PC, or can be displayed in a menu list of the PC.
  • the application of the mobile phone can be tapped/clicked and run on the PC, and a function of the application can be used.
  • the user can use the application of the mobile phone by using large-screen experience of the PC.
  • system integration can be implemented between the mobile phone and the PC, so that the user can invoke an application installed on the mobile phone from a system menu of the PC.
  • a plurality of mobile phone applications can independently run in parallel on the PC at the same time.
  • An independent window can be displayed on the PC for each application.
  • a plurality of windows corresponding to a plurality of applications are displayed in an interface of the PC at the same time, and running a plurality of applications of the mobile phone does not affect use of another function of the user on the mobile phone. For example, when using a video application or a music application of the mobile phone on the PC, the user may send/receive an SMS message, make a call, or the like by using the mobile phone at the same time.
  • the user may operate the mobile phone by using a peripheral input device such as a mouse or a keyboard, a use habit of the user is totally reserved, and there is no need to frequently switch between the two devices.
  • An application of the mobile phone is run by using the mobile phone window of the PC, thereby improving efficiency of file query, sharing, and office, and improving user experience.
  • a multi-window display method applied to a system including a first electronic device and a second electronic device that communicate with each other, where the method includes: A first electronic device receives size information of a display area of the second electronic device. The first electronic device runs a first application, and displays a first interface of the first application. The first electronic device determines, based on the size information of the display area of the second electronic device and size information corresponding to the first interface, location information of a first window that is of the first interface and that is in the display area of the second electronic device. The first electronic device sends the location information of the first window and display content information of the first interface to the second electronic device.
  • the first electronic device receives an instruction that is sent by the second electronic device and that is for starting a second application, and starts the second application.
  • the first electronic device determines, based on the size information of the display area of the second electronic device and size information corresponding to a second interface of the second application, location information of a second window that is of the second interface and that is in the display area of the second electronic device.
  • the first window and the second window are different windows.
  • the first electronic device sends the location information of the second window and display content information of the second interface to the second electronic device.
  • the method further includes: The first electronic device sends first information of the first application and second information of the second application to the second electronic device.
  • the first information of the first application includes at least one type of information such as a name, an identifier ID, and an icon of the first application.
  • the second information of the second application includes at least one type of information such as a name, an identifier ID, and an icon of the second application.
  • the first window is close to a right edge of the display area of the second electronic device.
  • the second window is located on the left of the first window, and does not overlap the first window.
  • the second window is located at an upper layer of the first window.
  • the location information of the first window includes coordinate information of a first vertex of the first window, length information of the first window, and width information of the first window; or the location information of the first window includes coordinate information of four vertices of the first window.
  • size information of the first window is determined by the first electronic device based on the size information corresponding to the first interface.
  • the method further includes: The first electronic device obtains size information of a display area of the first electronic device, and determines that the size information of the display area of the first electronic device is the size information corresponding to the first interface, or that the size information corresponding to the first interface is a product of the size information of the display area of the first electronic device and a first multiple.
  • the first multiple is greater than 0 and is not equal to 1.
  • size information of the first window is determined by the first electronic device based on the size information corresponding to the first interface.
  • the method further includes: The first electronic device obtains, from configuration information of the first application, size information existing when the first interface is displayed on the first electronic device, and determines, as the size information corresponding to the first interface, the size information existing when the first interface is displayed on the first electronic device.
  • the first electronic device obtains, from a network device, size information existing when the first interface is displayed on the second electronic device, and determines, as the size information corresponding to the first interface, the size information existing when the first interface is displayed on the second electronic device.
  • the first window includes the first interface and a first menu bar; and/or the second window includes the second interface and a second menu bar.
  • the second menu bar includes a close key
  • the method further includes: The first electronic device receives an instruction that is sent by the second electronic device and that is for closing the second application, and if detecting an operation performed by a user on the close key, the second electronic device closes the second window, and sends the instruction for closing the second application to the first electronic device.
  • the first electronic device closes the second application in response to the instruction for closing the second application.
  • the method further includes: If the first electronic device detects an instruction of the user for closing the second application, the first electronic device closes the second application, and sends an instruction for closing the second window to the second electronic device.
  • the method further includes: The first electronic device sends audio data of the second application to the second electronic device.
  • a multi-window display method applied to a system including a first electronic device and a second electronic device that communicate with each other, where the method includes: The second electronic device receives location information of a first window and display content information of a first interface that are sent by the first electronic device, where the first interface is a displayed interface of the first application run on the first electronic device. The second electronic device displays the first interface in the first window of a display area of the second electronic device based on the location information of the first window and the display content information of the first interface. If detecting a first operation, the second electronic device sends an instruction for starting a second application to the first electronic device in response to the first operation.
  • the second electronic device receives location information of a second window and display content information of a second interface that are sent by the first electronic device, where the second interface is an interface of the second application, and the first window and the second window are different windows.
  • the second electronic device displays the second interface in the second window of the display area of the second electronic device based on the location information of the second window and the display content information of the second interface.
  • the method further includes: The second electronic device receives first information of the first application and second information of the second application that are sent by the first electronic device.
  • the first information of the first application includes at least one type of information such as a name, an identifier ID, and an icon of the first application.
  • the second information of the second application includes at least one type of information such as a name, an identifier ID, and an icon of the second application.
  • the second electronic device automatically displays a third interface, where the third interface includes the first information of the first application and the second information of the second application. Alternatively, after detecting a second operation of a user, the second electronic device displays the third interface based on the first information of the first application and the second information of the second application.
  • the first window is close to a right edge of the display area of the second electronic device.
  • the second window is located on the left of the first window, and does not overlap the first window.
  • the second window is located at an upper layer of the first window.
  • the location information of the first window includes coordinate information of a first vertex of the first window, length information of the first window, and width information of the first window; or the location information of the first window includes coordinate information of four vertices of the first window.
  • size information of the first window is determined by the first electronic device based on size information corresponding to the first interface.
  • the size information corresponding to the first interface is size information of a display area of the first electronic device, or the size information corresponding to the first interface is a product of size information of a display area of the first electronic device and a first multiple.
  • the first multiple is greater than 0 and is not equal to 1.
  • size information of the first window is determined by the first electronic device based on size information corresponding to the first interface.
  • the size information corresponding to the first interface is size information existing when the first interface is displayed on the first electronic device, or the size information corresponding to the first interface is size information that is obtained by the first electronic device from a network device and that exists when the first interface is displayed on the second electronic device.
  • the first window includes the first interface and a first menu bar; and/or the second window includes the second interface and a second menu bar.
  • the second menu bar includes a close key. After the second electronic device displays the second interface in the second window, the method further includes: If the second electronic device detects an operation performed by the user on the close key, the second electronic device closes the second window, and sends an instruction for closing the second application to the first electronic device.
  • the method further includes: If the second electronic device detects an operation of selecting the first window by the user, the second electronic device switches the first window to an uppermost layer for display.
  • the second menu bar includes a maximize key. After the second electronic device displays the second interface in the second window, the method further includes: If the second electronic device detects an operation performed by the user on the maximize key, the second electronic device displays the second window in full screen.
  • the method further includes: If the second electronic device receives an instruction that is sent by the first electronic device and that is for closing the second window, the second electronic device closes the second window in response to the instruction for closing the second window.
  • the method further includes: The second electronic device receives audio data that is of the second application and that is sent by the first electronic device, and plays the audio data of the second application.
  • a multi-window display method applied to a system including a first electronic device and a second electronic device that communicate with each other, where the method includes: The first electronic device runs a first application, and displays a first interface of the first application. The first electronic device sends size information corresponding to the first interface and display content information of the first interface to the second electronic device. The second electronic device determines location information of a first window based on size information of a display area of the second electronic device and the size information corresponding to the first interface, and displays the first interface in the first window based on the location information of the first window and the display content information of the first interface.
  • the second electronic device sends an instruction for starting a second application to the first electronic device in response to the first operation.
  • the first electronic device starts the second application based on the instruction for starting the second application.
  • the first electronic device sends size information corresponding to a second interface of the second application and display content information of the second interface to the second electronic device.
  • the second electronic device determines location information of a second window based on the size information of the display area of the second electronic device and the size information corresponding to the second interface, and displays the second interface in the second window based on the location information of the second window and the display content information of the second interface.
  • the first window and the second window are different windows.
  • the first electronic device sends the location information of the first window and the display content information of the first interface to the second electronic device, and the second electronic device displays the first window based on an indication of the first electronic device.
  • the first electronic device may not obtain the size information of the display area of the second electronic device, but sends only the display content information of the first interface of the first electronic device.
  • the second electronic device determines the location information of the first window, and displays the first window.
  • display of the second window may also be determined by the second electronic device, and details are not described herein again.
  • the second electronic device may determine display locations of the first window and the second window based on application use, interface content, and the like of the second electronic device, to be better adapted to a use process of the second electronic device, reduce impact on use of the second electronic device, and improve user experience.
  • this application provides an apparatus.
  • the apparatus is included in an electronic device, and the apparatus has a function of implementing behavior of the electronic device in the foregoing aspects and the possible implementations of the foregoing aspects.
  • the function may be implemented by hardware, or may be implemented by corresponding software executed by hardware.
  • the hardware or the software includes one or more modules or units corresponding to the function, such as a display module or unit, a detection module or unit, or a processing module or unit.
  • this application provides an electronic device, including a touchscreen.
  • the touchscreen includes a touch-sensitive surface, a display, a camera, one or more processors, a memory, a plurality of application programs, and one or more computer programs.
  • the one or more computer programs are stored in the memory, and the one or more computer programs include an instruction.
  • the electronic device is enabled to perform the multi-window display method in any possible implementation of any one of the foregoing aspects.
  • this application provides an electronic device, including one or more processors and one or more memories.
  • the one or more memories are coupled to the one or more processors.
  • the one or more memories are configured to store computer program code.
  • the computer program code includes a computer instruction.
  • a system including a first electronic device and a second electronic device that perform the multi-window display method in any possible implementation of any one of the foregoing aspects.
  • this application provides a computer storage medium, including a computer instruction.
  • the computer instruction When the computer instruction is run on an electronic device, the electronic device is enabled to perform the multi-window display method in any possible implementation of any one of the foregoing aspects.
  • this application provides a computer program product.
  • the computer program product When the computer program product is run on an electronic device, the electronic device is enabled to perform the multi-window display method in any possible implementation of any one of the foregoing aspects.
  • FIG. 1 is a schematic structural diagram of a first electronic device according to an embodiment of this application.
  • FIG. 2 is a schematic structural diagram of a second electronic device according to an embodiment of this application.
  • FIG. 3 A and FIG. 3 B are a structural block diagram of software of a mobile phone and a PC according to an embodiment of this application;
  • FIG. 4 is a schematic diagram of displaying a mobile phone interface by a PC according to an embodiment of this application.
  • FIG. 5 A , FIG. 5 B , and FIG. 5 C are another schematic diagram of displaying a mobile phone interface by a PC according to an embodiment of this application;
  • FIG. 6 is another schematic diagram of displaying a mobile phone interface by a PC according to an embodiment of this application.
  • FIG. 7 is another schematic diagram of displaying a mobile phone interface by a PC according to an embodiment of this application.
  • FIG. 8 is a schematic flowchart of a multi-window display method according to an embodiment of this application.
  • FIG. 9 A , FIG. 9 B , and FIG. 9 C are a schematic diagram of creating a virtual area by a mobile phone according to an embodiment of this application;
  • FIG. 10 is another schematic flowchart of a multi-window display method according to an embodiment of this application.
  • FIG. 11 A , FIG. 11 B , and FIG. 11 C are another schematic diagram of creating a virtual area by a mobile phone according to an embodiment of this application;
  • FIG. 12 is another schematic flowchart of a multi-window display method according to an embodiment of this application.
  • FIG. 13 is another schematic flowchart of a multi-window display method according to an embodiment of this application.
  • A/B may represent A or B.
  • and/or describes only an association relationship for describing associated objects and represents that three relationships may exist.
  • a and/or B may represent the following three cases: Only A exists, both A and B exist, and only B exists.
  • a plurality of means two or more than two.
  • first and second are only used for distinguishing.
  • a first electronic device and “a second electronic device” are only used to indicate different electronic devices
  • a first window” and “a second window” are only used to indicate different display windows
  • a first operation” and “a second operation” are only used to indicate operations performed at different moments or for different purposes.
  • Embodiments of this application provide a multi-window display method, and the method may be applied to a system including two electronic devices that communicate with each other.
  • the two electronic devices are respectively referred to as a first electronic device and a second electronic device
  • the second electronic device may display an interface of the first electronic device.
  • the second electronic device displays a main interface of the first electronic device by using one window
  • the second electronic device displays interfaces of a plurality of applications of the first electronic device by using a plurality of windows.
  • a plurality of windows of the first electronic device can be displayed on the second electronic device by using an application program.
  • the application program may be installed only on the second electronic device.
  • the application program is installed on both the first electronic device and the second electronic device.
  • the application program is embodied as a first application program on the first electronic device, and is embodied as a second application program on the second electronic device.
  • the first application program is applicable to the first electronic device, and the second application program is applicable to the second electronic device.
  • the first application program is program code that is preset on the first electronic device, and may have no corresponding desktop application icon.
  • the second application program may be embodied as a plurality of possible forms such as a desktop application icon on the second electronic device, so that the second electronic device can display the main interface of the first electronic device by using one window or display the interfaces of the plurality of applications of the first electronic device by using a plurality of windows.
  • the first electronic device may be an electronic device such as a mobile phone, a smart terminal, or a wearable device.
  • the second electronic device may be an electronic device with a display function such as a PC, a tablet computer, a mobile phone, a smart terminal, a vehicle-mounted device, an augmented reality (augmented reality, AR)/virtual reality (virtual reality, VR) device, a notebook computer, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA), a display, or a television.
  • augmented reality augmented reality, AR
  • VR virtual reality
  • a notebook computer an ultra-mobile personal computer
  • UMPC ultra-mobile personal computer
  • netbook a personal digital assistant (personal digital assistant, PDA)
  • PDA personal digital assistant
  • the first electronic device and the second electronic device may be electronic devices of a same type or different types.
  • a mobile phone is used as the first electronic device
  • a PC is used as the second electronic device
  • a plurality of windows of the mobile phone are displayed on the PC.
  • FIG. 1 is a schematic structural diagram of a mobile phone 100 according to an embodiment of this application.
  • the mobile phone 100 may include a processor 110 , an external memory interface 120 , an internal memory 121 , a universal serial bus (universal serial bus, USB) interface 130 , a charging management module 140 , a power management module 141 , a battery 142 , an antenna 1 , an antenna 2 , a mobile communications module 150 , a wireless communications module 160 , an audio module 170 , a speaker 170 A, a phone receiver 170 B, a microphone 170 C, a headset jack 170 D, a sensor module 180 , a key 190 , a motor 191 , an indicator 192 , a camera 193 , a display screen 194 , a subscriber identity module (subscriber identification module, SIM) card interface 195 , and the like.
  • SIM subscriber identity module
  • the sensor module 180 may include a pressure sensor 180 A, a gyro sensor 180 B, a barometric pressure sensor 180 C, a magnetic sensor 180 D, an acceleration sensor 180 E, a distance sensor 180 F, an optical proximity sensor 180 G, a fingerprint sensor 180 H, a temperature sensor 180 J, a touch sensor 180 K, an ambient light sensor 180 L, a bone conduction sensor 180 M, and the like.
  • the structure illustrated in this embodiment of this application does not constitute a specific limitation on the mobile phone 100 .
  • the mobile phone 100 may include more or fewer components than those shown in the figure, or combine some components, or split some components, or have different component arrangements.
  • the components shown in the figure may be implemented by hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural-network processing unit (neural-network processing unit, NPU).
  • Different processing units may be independent components, or may be integrated into one or more processors.
  • the controller may be a nerve center and a command center of the mobile phone 100 .
  • the controller may generate an operation control signal based on instruction operation code and a timing signal, and read an instruction and control execution of the instruction.
  • a memory may be further disposed in the processor 110 , and is configured to store an instruction and data.
  • the memory in the processor 110 is a cache memory.
  • the memory may store an instruction or data that is recently used or cyclically used by the processor 110 . If the processor 110 needs to use the instruction or the data again, the processor 110 may directly invoke the instruction or the data from the memory, to avoid repeated access and reduce a waiting time of the processor 110 , thereby improving system efficiency.
  • the processor 110 may include one or more interfaces.
  • the interface may include an inter-integrated circuit (inter-integrated circuit, I2C) interface, an inter-integrated circuit sound (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver/transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (general-purpose input/output, GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, a universal serial bus (universal serial bus, USB) interface, and/or the like.
  • I2C inter-integrated circuit
  • I2S inter-integrated circuit sound
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous receiver/transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • the I2C interface is a two-way synchronization serial bus, and includes a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL).
  • the processor 110 may include a plurality of groups of I2C buses.
  • the processor 110 may be coupled to the touch sensor 180 K, a charger, a flash light, the camera 193 , and the like by using different I2C bus interfaces.
  • the processor 110 may be coupled to the touch sensor 180 K by using the I2C interface, so that the processor 110 communicates with the touch sensor 180 K by using the I2C bus interface, to implement a touch function of the mobile phone 100 .
  • the I2S interface may be used for audio communication.
  • the PCM interface may also be used for audio communication, and samples, quantizes, and encodes an analog signal.
  • the UART interface is a universal serial data bus, and is used for asynchronous communication.
  • the bus may be a two-way communications bus. The bus converts to-be-transmitted data between serial communication and parallel communication.
  • the MIPI interface may be configured to connect the processor 110 to peripheral components such as the display screen 194 and the camera 193 .
  • the MIPI interface includes a camera serial interface (camera serial interface, CSI), a display serial interface (display serial interface, DSI), and the like.
  • the GPIO interface may be configured by using software.
  • the GPIO interface may be configured as a control signal, or may be configured as a data signal.
  • the USB interface 130 is an interface that conforms to a USB standard specification, and may be specifically a mini USB interface, a micro USB interface, a USB type C interface, or the like.
  • the USB interface 130 may be configured to connect to another electronic device, for example, the USB interface may be used to connect to a PC.
  • an interface connection relationship between the modules illustrated in this embodiment of this application is merely an example for description, and does not constitute a limitation on the structure of the mobile phone 100 .
  • the mobile phone 100 may alternatively use an interface connection manner different from that in the foregoing embodiment, or use a combination of a plurality of interface connection manners.
  • the charging management module 140 is configured to receive charging input from the charger.
  • a wireless communication function of the mobile phone 100 may be implemented by using the antenna 1 , the antenna 2 , the mobile communications module 150 , the wireless communications module 160 , the modem processor, the baseband processor, and the like.
  • the antenna 1 and the antenna 2 are configured to transmit and receive an electromagnetic wave signal.
  • the mobile communications module 150 can provide a solution for wireless communication including 2G/3G/4G/5G and the like to be applied to the mobile phone 100 .
  • the wireless communications module 160 can provide a solution for wireless communication including a wireless local area network (wireless local area networks, WLAN) (such as a wireless fidelity (wireless fidelity, Wi-Fi) network), Bluetooth (bluetooth, BT), a global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), a near field communication (near field communication, NFC), an infrared (infrared, IR) technology, and the like to be applied to the mobile phone 100 .
  • the wireless communications module 160 may be one or more components integrating at least one communications processor module.
  • the wireless communications module 160 receives an electromagnetic wave by using the antenna 2 , performs frequency modulation and filtering processing on an electromagnetic wave signal, and sends a processed signal to the processor 110 .
  • the wireless communications module 160 may further receive a to-be-sent signal from the processor 110 , perform frequency modulation on the signal, amplify the signal, and convert the signal into an electromagnetic wave for radiation by using the antenna 2 .
  • the mobile phone 100 implements a display function by using the GPU, the display screen 194 , the application processor, and the like.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor.
  • the GPU is configured to perform mathematical and geometric calculation, and is configured to render an image.
  • the processor 110 may include one or more GPUs that execute a program instruction to generate or change display information.
  • the display screen 194 is configured to display an image, a video, and the like.
  • the display screen 194 includes a display panel.
  • the display panel may use a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (organic light-emitting diode, OLED), a passive-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light emitting diode, AMOLED), a flexible light-emitting diode (flex light-emitting diode, FLED), a mini-LED, a micro-LED, a micro-OLED, a quantum dot light-emitting diode (quantum dot light emitting diodes, QLED), or the like.
  • the mobile phone 100 may include one or N display screens 194 .
  • N is a positive integer greater than 1.
  • the mobile phone 100 can implement a photographing function by using the ISP, the camera 193 , the video codec, the GPU, the display screen 194 , the application processor, and the like.
  • the ISP is configured to process data fed back by the camera 193 .
  • the camera 193 is configured to capture a static image or a video. An optical image is generated for an object by using a lens, and is projected onto a photosensitive element.
  • the mobile phone 100 may include one or N cameras 193 . N is a positive integer greater than 1.
  • the external memory interface 120 may be configured to connect to an external storage card such as a micro SD card, to extend a storage capability of the mobile phone 100 .
  • the internal memory 121 may be configured to store computer executable program code, and the executable program code includes an instruction.
  • the processor 110 runs the instruction stored in the internal memory 121 to execute various function applications of the mobile phone 100 and process data.
  • the internal memory 121 may include a program storage area and a data storage area.
  • the program storage area may store an operating system, an application program required by at least one function (such as a sound play function and an image display function), and the like.
  • the data storage area may store data (such as audio data and an address book) created in a process of using the mobile phone 100 , and the like.
  • the mobile phone 100 can implement an audio function such as music play and recording by using the audio module 170 , the speaker 170 A, the phone receiver 170 B, the microphone 170 C, the headset jack 170 D, the application processor, and the like.
  • an audio function such as music play and recording by using the audio module 170 , the speaker 170 A, the phone receiver 170 B, the microphone 170 C, the headset jack 170 D, the application processor, and the like.
  • the mobile phone 100 further includes a plurality of sensors, such as the plurality of sensors shown in FIG. 1 .
  • the pressure sensor 180 A is configured to sense a pressure signal, and may convert the pressure signal into an electrical signal.
  • the gyro sensor 180 B may be configured to determine a movement posture of the mobile phone 100 .
  • the barometric pressure sensor 180 C is configured to measure barometric pressure.
  • the magnetic sensor 180 D includes a Hall sensor.
  • the acceleration sensor 180 E may detect an acceleration value in each direction (generally, three axes) of the mobile phone 100 .
  • the acceleration sensor 180 E may detect a value and a direction of gravity, and may be further configured to recognize a posture of the mobile phone 100 , and is applied to an application such as switching between landscape orientation and vertical orientation and a pedometer.
  • the distance sensor 180 F is configured to measure a distance.
  • the optical proximity sensor 180 G may include, for example, a light-emitting diode (LED) and an optical detector such as a photodiode.
  • the ambient light sensor 180 L is configured to sense ambient light brightness.
  • the fingerprint sensor 180 H is configured to collect a fingerprint.
  • the temperature sensor 180 J is configured to detect a temperature.
  • the touch sensor 180 K is also referred to as a “touch panel”.
  • the touch sensor 180 K may be disposed in the display screen 194 , and the touch sensor 180 K and the display screen 194 constitute a touchscreen that is also referred to as a “touchscreen”.
  • the touch sensor 180 K is configured to detect a touch operation on or near the touch sensor 180 K.
  • the touch sensor may transfer the detected touch operation to the application processor to determine a type of a touch event.
  • Visual output related to the touch operation may be provided by using the display screen 194 .
  • the touch sensor 180 K may be disposed on a surface of the mobile phone 100 at a location different from that in the display screen 194 .
  • the bone conduction sensor 180 M may obtain a vibration signal.
  • the key 190 includes a power key, a volume key, and the like.
  • the key 190 may be a mechanical key, or may be a touch key.
  • the mobile phone 100 may receive key input, and generate key signal input related to a user setting and function control of the mobile phone 100 .
  • the motor 191 may generate a vibration prompt.
  • the indicator 192 may be an indicator light that may be configured to indicate a charging state and a power change, or may be configured to indicate a message, a missed call, a notification, and the like.
  • the SIM card interface 195 is configured to connect to a SIM card.
  • FIG. 2 is a schematic structural diagram of a PC 200 according to an embodiment of this application.
  • the PC 200 may include a processor 210 , a memory 220 , a communications module 230 , a display screen 240 , and the like.
  • the structure of the PC 200 shown in this embodiment of this application does not constitute a specific limitation on the PC 200 .
  • the PC 200 may include more or fewer components than those shown in the figure, or combine some components, or split some components, or have different component arrangements.
  • the components shown in the figure may be implemented by hardware, software, or a combination of software and hardware.
  • the processor 210 may include one or more processing units, and the processor 210 may be configured to control and manage an action of the PC 200 .
  • the memory 220 is configured to store program code and data.
  • the communications module 230 may be used for communication between the internal modules of the PC 200 , communication between the PC 200 and another external device, or the like. For example, if the PC 200 communicates with another electronic device in a wired connection manner, the communications module 230 may include an interface such as a USB interface.
  • a USB interface refer to the description of the USB interface 130 in FIG. 1 .
  • the communications module 230 may include an audio component, a radio frequency circuit, a Bluetooth chip, a wireless fidelity (wireless fidelity, Wi-Fi) chip, a near field communication (near-field communication, NFC) module, and the like, and may implement interaction between the PC 200 and another electronic device.
  • the display screen 240 is configured to display an image, a video, and the like.
  • the PC 200 may further include a peripheral device 250 such as a mouse, a keyboard, a speaker, and a microphone.
  • the processor 210 may execute a computer executable instruction stored in the memory 220 , so that the PC 200 and a mobile phone 100 can form a system including electronic devices that communicate with each other, thereby displaying a plurality of windows of the mobile phone 100 by using the display screen 240 of the PC 200 .
  • FIG. 3 A and FIG. 3 B are a structural block diagram of software of a mobile phone 100 and a PC 200 according to an embodiment of this application.
  • the block diagram includes a software structure of the mobile phone 100 and a software structure of a mobile phone window display module of the PC 200 .
  • the software structure of the mobile phone 100 may use a layered architecture, an event driven architecture, a micro kernel architecture, a micro service architecture, or a cloud architecture.
  • an Android system with a layered architecture is used as an example to describe the software structure of the mobile phone 100 .
  • the Android system is used as an example.
  • the layered structure software is divided into several layers, and each layer has a clear role and task.
  • the layers communicate with each other by using a software interface.
  • the Android system includes an application program (application, APP) layer, an application program framework (framework) layer, a system library, and a kernel layer from top to bottom.
  • the application program layer may include a series of application program packages.
  • the application program layer may include an application program such as messaging, music, videos, and navigation. This is not limited in this embodiment of this application.
  • the application program framework layer may include a plurality of service programs or some predefined functions.
  • the application program framework layer provides an application programming interface (application programming interface, API) and a programming framework for an application program at the application program layer.
  • API application programming interface
  • an application program framework layer may include a link discovery service Android package (Android package, APK), a multi-screen framework service APK, and a multicast source discovery protocol (multicast source discovery protocol, MSDP) service APK.
  • Android package Android package, APK
  • APK link discovery service
  • APK multi-screen framework service
  • MSDP multicast source discovery protocol
  • the link discovery service APK is used to discover the PC 200 , and is connected to the PC 200 .
  • the link discovery service APK may include a software development kit (software development kit, SDK) that implements the function.
  • SDK may provide some files of an application programming interface (application programming interface, API) for a program design language, or may implement communication between hardware modules.
  • the MSDP-SDK shown in FIG. 3 A may provide an interface for an MSDP service to notify a data transmission path, to implement a process of transmitting a video stream for the MSDP service.
  • the mobile phone 100 and the PC 200 may communicate with each other in a plurality of different manners, for example, communicate with each other in a connection manner or by using a future communications technology, to transmit data.
  • the connection between the mobile phone 100 and the PC 200 may include a plurality of different connection manners such as a wired connection manner or a wireless connection manner.
  • a wired connection between the mobile phone 100 and the PC 200 may be established by using a USB data cable.
  • a wireless connection between the mobile phone 100 and the PC 200 may be a Wi-Fi connection, a near connection established by using a “one hop” function and a near field communication (near field communication, NFC) function supported by the mobile phone 100 and the PC 200 , a connection established by the mobile phone 100 and the PC 200 through Bluetooth code scanning, or the like.
  • Data may be transmitted by using the future communications technology such as a 5G communications network by installing different applications or a same application on the mobile phones 100 and PC 200 .
  • NFC near field communication
  • a connection is established between the mobile phone 100 and the PC 200 through NFC.
  • the mobile phone 100 has an NFC function, and first enables the NFC function of the mobile phone 100 .
  • the link discovery service APK of the mobile phone 100 may be used to scan and discover another device that enables the NFC function in a specific range. If the PC 200 also enables the NFC function, when the mobile phone 100 is close to the PC 200 , the link discovery service APK of the mobile phone 100 may send a connection request to an NFC function module of the PC 200 , and the NFC function module of the PC 200 accepts the connection request, and establishes a path with the mobile phone 100 , so that a process of establishing a connection between the mobile phone 100 and the PC 200 is completed.
  • having the NFC function by an electronic device may be implemented by using an NFC chip included in the electronic device, or an NFC device may be externally connected to the electronic device to implement the NFC function by presetting an application program related to the NFC function on the electronic device. This is not limited in this embodiment of this application.
  • a communication bandwidth and speed gradually increase, and data may also be transmitted between the mobile phone 100 and the PC 200 without establishing a near field communication connection.
  • a window of the mobile phone 100 may be displayed on the PC 200 through 5G communication.
  • data is transmitted by using the 5G communications network by installing different applications or a same application on the mobile phone 100 and the PC 200 .
  • the link discovery service APK of the mobile phone 100 may not provide a function of discovering the PC 200 and establishing a connection to the PC 200 , and the like. It should be understood that a communication manner between the mobile phone 100 and the PC 200 is not limited in this embodiment of this application. In a subsequent embodiment, an example in which an NFC connection is established between the mobile phone 100 and the PC 200 is used for detailed description.
  • a multi-window manager in the multi-screen frame service APK is used to manage a window program of the mobile phone 100 .
  • the multi-window manager may obtain a size of a to-be-displayed window on the mobile phone 100 , and determine content of the to-be-displayed window.
  • the to-be-displayed window of the mobile phone 100 may include a window that is being displayed in an interface of the mobile phone 100 , and may further include a window of one or more application programs running in the background of the mobile phone 100 .
  • a message notification manager may enable an application program to display notification information in a status bar of the mobile phone 100 , and may be used to convey a message of a notification type.
  • a message displayed in the status bar may disappear automatically after a short stay without user interaction.
  • the message displayed in the status bar may be used to notify a user that download is completed, and may be used to prompt for other message content.
  • the message notification manager may further provide a notification in a form of a chart or a scroll bar text in the status bar at the top of the system, for example, a notification of an application program running in the background, and may further provide a notification in a form of a dialog window on a screen of the mobile phone 100 .
  • the message notification manager may further provide different types of notifications, such as an alert tone, a vibration of the mobile phone 100 , and flashing of an indicator light.
  • the MSDP service APK is used to transmit data after the connection (for example, the NFC connection) is established between the mobile phone 100 and the PC 200 , for example, transmit a video stream between the mobile phone 100 and the PC 200 .
  • the MSDP service APK may implement functions such as video stream encoding, audio stream encoding, window coordinate synchronization, audio device virtualization, and reverse control.
  • the video stream encoding may implement encapsulation of to-be-transmitted video data into a packet that conforms to a data transmission protocol between the mobile phone 100 and the PC 200 .
  • the video stream packet may include information such as a type, a format, a sequence number, and a timestamp of transmission media, and whether additional data exists, to transmit a video stream in real time.
  • the audio stream encoding may implement encapsulation of to-be-transmitted audio data into a packet that conforms to the data transmission protocol between the mobile phone 100 and the PC 200 , to transmit an audio stream in real time.
  • the window coordinate synchronization may be used to determine coordinates of a to-be-displayed window of the mobile phone 100 on a display screen of the PC 200 .
  • the audio device virtualization may be used to create virtual audio for audio in the mobile phone 100 , and then send the virtual audio to the PC 200 , to play the audio on the PC 200 .
  • the reverse control may be used to synchronously perform a corresponding operation on the mobile phone 100 and the PC 200 based on an operation instruction transmitted between the mobile phone 100 and the PC 200 .
  • the PC 200 displays a play window of a video application of the mobile phone 100 .
  • the pause operation may be transferred back to the mobile phone 100 , and a reverse control module of the mobile phone 100 also controls the mobile phone 100 to pause play.
  • the reverse control may be further used to transfer an operation on the mobile phone 100 to the PC 200 . It should be understood that, for different systems, there are different communication manners between the mobile phone 100 and the PC 200 .
  • Some or all of the software modules described above may participate in a multi-window display process of the PC 200 in this embodiment of this application.
  • the application program framework layer may implement functions such as link discovery and MSDP service invoking between the PC 200 and the mobile phone 100 .
  • a multi-screen framework service at the application program framework layer may further implement functions such as file dragging and sharing between the PC 200 and the mobile phone 100 , display in a single window mode, and coordinated application lifecycle management.
  • file dragging and sharing between the PC 200 and the mobile phone 100 and display of one window of the mobile phone 100 by the PC 200 refer to an existing solution. Details are not described herein.
  • Coordinated application lifecycle management is a software tool used to configure and manage an application program, and specifies a lifecycle of the application program.
  • the system library may include a plurality of function modules, for example, a surface manager (surface manager), a media library (media libraries), a three-dimensional graphics processing library (for example, OpenGL ES), and a 2D graphics engine (for example, SGL).
  • the surface manager is used to manage a display subsystem of the mobile phone 100 , and provide fusion of 2D and 3D layers for a plurality of application programs.
  • the media library supports playback and recording in a plurality of commonly used audio and video formats, and static image files.
  • the media library may support a plurality of audio and video encoding formats such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.
  • the three-dimensional graphics processing library is used to implement drawing of three-dimensional graphics, image rendering, synthesis, layer processing, and the like.
  • the 2D graphics engine is a drawing engine for 2D graphics.
  • An Android runtime (Android runtime) includes a kernel library and a virtual machine.
  • the Android runtime is responsible for scheduling and management of the Android system.
  • the kernel layer is a layer between hardware and software.
  • the kernel layer includes at least a display driver, an audio driver, and the like.
  • the kernel layer of the mobile phone 100 may include a wireless fidelity (wireless fidelity, WIFI) module configured to establish a wireless connection.
  • the kernel layer may further include a near field communication (near-field communication, NFC) module configured to implement a connection between the mobile phone 100 and the PC 200 through NFC.
  • NFC near field communication
  • the kernel layer may not include a Wi-Fi module, an NFC module, or the like, or the kernel layer may include a Wi-Fi module, an NFC module, or the like that does not participate in the multi-window display process of the PC 200 in this embodiment of this application.
  • the mobile phone 100 may have different division manners or include more function modules. This is not limited in this embodiment of this application.
  • FIG. 3 B further shows the mobile phone window display module of the PC 200 .
  • the mobile phone window display module is configured to display interface content of the mobile phone 100 .
  • the PC 200 may include a multi-layer architecture, such as a pre-research layer, a PC software layer, an operating system (operating system, OS) layer, and a basic read-only memory (read only memory, ROM) multi-screen frame layer.
  • An executable file (executable file, exe) may be run at the PC software layer.
  • an exe program used to display a mobile phone window may be preset in the PC 200 .
  • the exe program for displaying a mobile phone window may be embodied in a form of an app on the PC 200 , or the exe program for displaying a mobile phone window runs, so that the PC 200 has a function of displaying a window of the mobile phone 100 .
  • a function module provided by the PC 200 for displaying the window of the mobile phone 100 is referred to as the “mobile phone window display module”. The following focuses on the mobile phone window display module of the PC 200 that is configured to display the window of the mobile phone 100 .
  • the mobile phone window display module of the PC 200 may provide a function corresponding to a related service at the application program framework layer of the mobile phone 100 , for example, provide a multi-screen framework service and an MSDP service.
  • the MSDP service APK corresponds to the MSDP service of the mobile phone 100 , and is used to transmit data after the PC 200 establishes a connection to the mobile phone 100 .
  • the MSDP service of the PC 200 may implement functions such as video stream decoding, audio stream decoding, window coordinate synchronization, audio device virtualization, and reverse control.
  • the MSDP service APK of the PC 200 may receive a video stream sent by the mobile phone 100 , and decode the video stream.
  • the video stream decoding can be used to decapsulate an encapsulation packet received by the PC 200 , and obtain complete video data obtained through decoding.
  • the audio stream decoding may be used to decapsulate a packet received by the PC 200 to obtain audio data.
  • the window coordinate synchronization may be used to determine, based on information included in the video stream, coordinates and an area that are on the display screen of the PC 200 and that are used to display a mobile phone window.
  • the audio device virtualization is used to play the virtual audio by using an audio device of the PC 200 based on the received virtual audio sent by the mobile phone 100 , to play the audio on the PC 200 .
  • the reverse control may be used to synchronously perform a corresponding operation on the mobile phone 100 and the PC 200 based on an operation instruction transmitted between the mobile phone 100 and the PC 200 .
  • the PC 200 displays a play window of a video application of the mobile phone 100 .
  • the pause operation may be transferred to the PC 200 , and a reverse control module of the PC 200 obtains an instruction to also control the PC 200 to pause play.
  • the reverse control may be further used to transfer an operation on the PC 200 to the mobile phone 100 .
  • the MSDP service APK obtains and parses a video frame, and transfers, to the multi-screen frame service APK, information that is about a mobile phone window and that is included in the video frame.
  • the information about the mobile phone window may include coordinate information of the mobile phone window, display content information of the mobile phone window, and the like.
  • the multi-screen frame service APK may include a multi-window manager, a message notification manager, a mobile phone window display, an application lifecycle manager, and the like.
  • the multi-window manager may determine, based on the coordinate information of the mobile phone window, to display one or more windows of the mobile phone 100 on the display screen of the PC 200 .
  • the mobile phone window display performs display on the display screen of the PC 200 based on the display content information of the mobile phone window, and the mobile phone window display may be used to display one or more windows of the mobile phone 100 .
  • the message notification manager may be used to display notification information on the PC 200 , for example, display the notification information in a manner of a message pop-up window, an alert tone, or the like.
  • the application lifecycle manager is used to provide, for an application program developer, an interface for conveniently accessing application program information, start an application program by invoking a single application programming interface (application programmers interface, API), and process various runtime states for each running application program.
  • the application lifecycle manager further maintains a program list for an application program installed in the system, and updates the list at any time when a monitored application program changes.
  • Transmitted data may include the video stream data described above, and may further include transmission of an intention-type instruction, for example, transmission of an operation instruction (for example, an instruction of an event such as starting, closing, moving, or scaling an application) such as a “touch event”.
  • the transmitted data may further include a parsable data format defined between the PC 200 and the mobile phone 100 , and the like. This is not limited in this embodiment of this application.
  • FIG. 3 A and FIG. 3 B a module and a software architecture that participate in the multi-window display process are described by using an example in which a plurality of windows of the mobile phone 100 are displayed on the PC 200 .
  • the mobile phone 100 may be used as the second electronic device, and the MSDP service APK of the mobile phone 100 may provide a video stream decoding function.
  • the PC 200 may be used as the first electronic device, and the MSDP service APK of the PC 200 may provide a video stream encoding function. This is not limited in this embodiment of this application.
  • the mobile phone may be connected to a large-screen device such as a PC, a display, or a television by using a data cable, so that the large-screen device can easily access the mobile phone.
  • the user may use two screens to process a plurality of tasks at the same time. For example, the user may view content such as a video, an image, and a document on a larger display screen, and may continue to use the mobile phone to send/receive an SMS message, answer a call, or the like at the same time.
  • the interface of the mobile phone needs to be projected to the large-screen device by using a connection of the data cable.
  • system integration cannot be implemented between the mobile phone and the PC, and an application installed on the mobile phone cannot be invoked from a system menu of the PC.
  • the mobile phone 100 having the hardware structure shown in FIG. 1 and the software architecture shown in FIG. 3 A and the PC 200 having the software structures shown in FIG. 2 and FIG. 3 B are used as an example in the following embodiment to describe the multi-window display method provided in the embodiments of this application.
  • FIG. 4 is a schematic diagram of displaying a mobile phone interface by a PC according to an embodiment of this application. As shown in (a) in FIG. 4 , after a mobile phone is connected to the PC, a mobile phone window of the PC 200 may display a same interface as the mobile phone 100 .
  • FIG. 4 shows a home screen 401 of the unlocked mobile phone 100 .
  • a plurality of application programs such as albums, settings, videos, and music are displayed on the home screen 401 .
  • the home screen 401 may further include more other application programs. This is not limited in this embodiment of this application.
  • FIG. 4 further shows a main interface 402 of the PC 200 , and the main interface 402 includes a plurality of apps (such as My Computer and Recycle Bin) of the PC 200 and a mobile phone window 10 .
  • the mobile phone window 10 may be used to display interface content of the mobile phone 100 .
  • the mobile phone window 10 may further include a gray menu bar 11 shown in the figure.
  • the gray menu bar 11 may include a hide to sidebar key 11 - 1 , a hide to menu bar key 11 - 2 , a maximize window key 11 - 3 , and a close window key 11 - 4 .
  • the hide to sidebar key 11 - 1 may be used to hide the mobile phone window 10 to a right frame of a display screen of the PC 200
  • the hide to menu bar key 11 - 2 may be used to hide the mobile phone window 10 to a menu area 12 .
  • the gray menu bar 11 may include some or all of the keys listed above, and may further include more function keys. This is not limited in this embodiment of this application.
  • the maximize window key 11 - 3 may be displayed in different forms based on an operation of the user, so that a function can be changed. For example, when the mobile phone window 10 is displayed as a small window, the user taps/clicks the maximize window key 11 - 3 , and the mobile phone window 10 is maximized in response to the tapping/clicking operation of the user. In this case, the maximize key 11 - 3 may be changed to a minimize window key such as a minimize window key 11 - 5 shown in (b) in FIG. 7 . The user may tap/click the minimize key 11 - 5 , and the mobile phone window 10 is scaled down and displayed as the original small window in response to the tapping/clicking operation of the user.
  • an area including the gray menu bar 11 may be collectively referred to as the “mobile phone window 10 ”.
  • the main interface 402 may further include the menu area 12 .
  • the menu area 12 may include a start menu 13 , a file management shortcut menu, a browser, an input method control menu, a time and date setting menu, and power management of the PC 200 . This is not limited in this embodiment of this application.
  • an interface currently displayed on the mobile phone 100 may be displayed in the mobile phone window 10 of the PC 200 .
  • the user taps a music application on the home screen 401 of the mobile phone.
  • Interfaces of the mobile phone 100 and the PC 200 may be shown in (b) in FIG. 4 .
  • the interface 401 of the mobile phone 100 displays an interface of the music application
  • the mobile phone window 10 of the PC 200 also displays the interface of the music application, so that display content in the interface 401 of the mobile phone 100 and display content in the mobile phone window 10 of the PC 200 are consistent. Details are not described herein.
  • the mobile phone window 10 is displayed on the PC 200 in the following manner: After a connection is established between the mobile phone 100 and the PC 200 , the mobile phone window 10 may automatically pop up and be displayed in the main interface 402 of the PC; or the mobile phone window 10 may be displayed in the main interface 402 of the PC after an operation of the user.
  • a display occasion and a display manner of the mobile phone window 10 are not limited in this embodiment of this application.
  • FIG. 5 A , FIG. 5 B , and FIG. 5 C are another schematic diagram of displaying a mobile phone interface by a PC according to an embodiment of this application.
  • a user may perform an operation on the PC 200 , and the mobile phone window 10 is displayed in a main interface 502 of the PC in response to the operation of the user.
  • the user may tap/click a start menu 13 on the PC 200 , and a menu window 20 is displayed in the main interface 502 of the PC 200 in response to the tapping/clicking operation of the user.
  • the menu window 20 includes a list of a plurality of applications of the PC, such as Word, Power Point, drawing, a calculator, or other office software, or another application such as a browser.
  • the menu window 20 may further include a shortcut of the application.
  • the application may be an “EMUJI desktop (EMUJI desktop)”, and the menu window 20 may include an icon of the EMUJI desktop.
  • the related application used to display a plurality of windows of the mobile phone 100 may be downloaded from an application store and installed on the PC 200 . Details about a method for and a process of installing the application are not described in this embodiment of this application.
  • the user may tap/click an EMUJI desktop menu bar in the menu window 20 , and a mobile phone application window 30 is displayed in the main interface 502 of the PC 200 in response to the tapping/clicking operation of the user, and all applications of the mobile phone 100 may be displayed in the mobile phone application window 30 .
  • the mobile phone application window 30 may be shown in FIG. 5 C , and an application program of the mobile phone 100 is displayed in a window of the EMUJI desktop.
  • the mobile phone application window 30 may be displayed as the mobile phone window 10 , and an application list of the mobile phone 100 is displayed in the mobile phone window 10 . This is not limited in this embodiment of this application.
  • the mobile phone 100 may send information about all installed applications to the PC 200 .
  • the user may trigger the PC 200 to send an application obtaining request to the mobile phone 100 , to request to obtain application information of the mobile phone 100 .
  • the application information herein may include information such as names, identifiers (identification, ID), and icons of all applications at the application program layer.
  • the PC 200 displays each application in the mobile phone application window 30 based on the name, the ID, and the icon of the application, to display all the applications of the mobile phone 100 .
  • the display may be shown in FIG. 5 C .
  • the identifier of the application may be a package name, the package name includes a string, and the PC 200 may determine a music application based on the package name.
  • the mobile phone window 10 pops up, and the mobile phone window 10 is consistent with an interface 501 of the mobile phone 100 in terms of display.
  • the window 30 of the EMUJI desktop may pop up in the main interface 502 of the PC 200 .
  • All application programs installed on the mobile phone may be displayed in the window 30 of the EMUJI desktop, and the user may perform an operation of tapping/clicking any application program in the window 30 of the EMUJI desktop.
  • the interface of the mobile phone 100 may be displayed in the interface of the PC 200 by using the mobile phone window 10 shown in FIG. 5 A , FIG. 5 B , and FIG. 5 C , or by using the operation method described in FIG. 5 A , FIG. 5 B , and FIG. 5 C : invoking an application (for example, the EMUJI desktop) from the start menu 13 of the PC 200 , and then using the application to display the mobile phone window 10 or the window 30 that is of the EMUJI desktop and that includes all the application programs of the mobile phone 100 .
  • an application for example, the EMUJI desktop
  • the interface of the mobile phone 100 may be displayed in the interface of the PC 200 based on a requirement of the user.
  • a plurality of windows corresponding to different applications may be displayed in the main interface 402 of the PC 200 .
  • FIG. 5 A , FIG. 5 B , and FIG. 5 C are another schematic diagram of displaying a mobile phone interface by a PC according to an embodiment of this application. It should be understood that a user may start the application by performing a tapping operation on the mobile phone, or start the application by performing a tapping/clicking operation in a mobile phone window 10 of the PC. This is not limited in this embodiment of this application.
  • mapping/clicking operations may be different operations for different devices.
  • the user may run the music application by performing a touching and tapping operation.
  • the user may run the music application by performing a clicking operation or double-clicking operation by using a peripheral input device such as a mouse or a keyboard.
  • a peripheral input device such as a mouse or a keyboard.
  • the PC is a device with a touchscreen, the user may run the music application through touching and tapping. This is not limited in this embodiment of this application.
  • the user may tap the music application on a home screen 401 of the mobile phone 100 .
  • a window 40 of the music application may automatically pop up on the PC 200 in response to the tapping operation of the user.
  • the window 40 of the music application may be used to display a main interface of the music application.
  • the user may tap/click a music application 14 in the mobile phone window 10 of the PC 200 , and a window 40 of the music application may automatically pop up in a main interface 402 of the PC 200 in response to the tapping/clicking operation of the user.
  • a window 40 of the music application may automatically pop up in a main interface 402 of the PC 200 in response to the tapping/clicking operation of the user. This is not limited in this embodiment of this application.
  • the window 40 of the music application may directly pop up in the mobile phone window 10 of the PC 200 in response to the tapping/clicking operation of the user, and the original mobile phone window 10 may retain as the home screen of the mobile phone 100 .
  • display content in the interface of the mobile phone 100 is not limited in this embodiment of this application.
  • the mobile phone 100 may be in a screen-off state, or display an unlocked home screen, or is consistent with display content in the mobile phone window 10 of the PC 200 .
  • the main interface of the music application may directly pop up and be displayed in the mobile phone window 10 of the PC 200 in response to the tapping/clicking operation of the user.
  • the user may select and drag the main interface of the music application, so that the window 40 of the music application reappears on the PC 200 .
  • the user may tap/click to select any location in the mobile phone window 10 , and drag the mobile phone window 10 to any area outside the mobile phone window 10 in a direction shown by a dashed line. Therefore, the interface of the PC 200 is shown in (a) in FIG. 6 , that is, the window 40 of the music application is independently displayed, and the home screen 401 of the mobile phone continues to be displayed in the mobile phone window 10 .
  • FIG. 7 is another schematic diagram of displaying a mobile phone interface by a PC according to an embodiment of this application.
  • a corresponding window may be displayed on the PC 200 for each application.
  • the user has run a music application on the PC 200 , and a window 40 of the music application is displayed in a main interface 402 of the PC 200 .
  • the mobile phone may return to a home screen 401 of the mobile phone 100 , and the music application may run in the background of the mobile phone 100 .
  • the user may further tap any other application on the home screen 401 of the mobile phone 100 .
  • the user may tap a video application 15 on the home screen 401 of the mobile phone 100 , and a window 50 of the video application may further automatically pop up on the PC 200 in response to the tapping operation of the user.
  • An interface of the mobile phone 100 is displayed as an interface of the video application, the user may return to the home screen 401 of the mobile phone 100 , and the video application may run in the background of the mobile phone 100 .
  • the user may tap/click a video application 15 in a mobile phone window 10 of the PC 200 , and a window 50 of the video application may automatically pop up on the PC 200 in response to the tapping/clicking operation of the user.
  • An interface of the mobile phone 100 may be displayed as the home screen 401 , that is, any possible interface of the mobile phone 100 is displayed without an impact of the operation of the user.
  • Any possible interface herein may be a lock screen interface, an interface of any application of the mobile phone that is used by the user, or the like.
  • the user may perform a related operation in a window that is of each application and that appears on the PC 200 .
  • the window 50 of the video application may be a main interface of the video application, and includes a list of a plurality of videos.
  • the user may tap/click a video, for example, the user taps/clicks “The Pursuit of Happiness”.
  • the window 50 of the video application may be displayed as the interface shown in (a) in FIG. 7 , and the video is played by using the window 50 of the video application.
  • this embodiment of this application sets no limitation on sizes and locations of windows that are of a plurality of applications and that are displayed on the PC 200 .
  • a process of determining a size and a location of a window that is of an application and that is displayed on the PC 200 refer to related descriptions in a subsequent step 804 .
  • the user may select a window of an application, and drag the window to any location on a display screen of the PC 200 .
  • the user may perform a corresponding operation on a window of each application by using a key included in a gray menu bar 11 , such as a hide to sidebar key 11 - 1 , a hide to menu bar key 11 - 2 , a maximize window key 11 - 3 , and a close window key 11 - 4 , or the user may drag a frame of the window 50 of the video application to scale down or scale up the window 50 of the video application. Details are not described herein again.
  • the window may be closed on the PC 200 in response to the close operation of the user.
  • An application that is of the mobile phone 100 and that corresponds to the window may run in the background, or an application corresponding to the window exits running and switches to a close state. This is not limited in this application.
  • the user when the user opens the video application on the PC 200 to display the window 50 of the video application, the user taps/clicks a close key in a gray menu bar in the window 50 of the video application, and the window may be closed on the PC 200 in response to the close operation of the user, and the video application of the mobile phone 100 may run in the background or exit running.
  • the user may maximize a window in different manners. For example, as shown in (a) in FIG. 7 , the user taps/clicks a maximize window key 11 - 3 included in the gray menu bar 11 of the window 50 of the video application, to maximize the window 50 of the video application to a window that is displayed in full screen and that is shown in (b) in FIG. 7 . Alternatively, the user taps/clicks a full-screen control 16 included in the window 50 of the video application, to maximize the window 50 of the video application to a window shown in (b) in FIG. 7 . This is not limited in this embodiment of this application.
  • the user may tap/click a window scaling-down control 17 included in the gray menu bar 11 , to restore the window 50 of the video application to a window that is not displayed in full screen and that is shown in (a) in FIG. 7 .
  • the user taps/clicks a minimize window key 11 - 5 included in the window 50 of the video application, to restore the window 50 of the video application to a window that is not displayed in full screen and that is shown in (a) in FIG. 7 .
  • This is not limited in this embodiment of this application.
  • an application program of the mobile phone 100 can be displayed in a mobile phone window on the PC 200 , or can be displayed in a menu list of the PC 200 .
  • the application of the mobile phone 100 can be tapped/clicked and run on the PC 200 , and a function of the application can be used.
  • the user can use the application of the mobile phone 100 by using large-screen experience of the PC 200 .
  • system integration can be implemented between the mobile phone and the PC, so that the user can invoke an application installed on the mobile phone from a system menu of the PC.
  • a plurality of mobile phone applications can independently run in parallel on the PC 200 at the same time.
  • An independent window can be displayed on the PC 200 for each application.
  • a plurality of windows corresponding to a plurality of applications are displayed in an interface of the PC 200 at the same time, and running a plurality of applications of the mobile phone 100 does not affect use of another function of the user on the mobile phone.
  • the user may send/receive an SMS message, make a call, or the like by using the mobile phone at the same time.
  • the user may operate the mobile phone by using a peripheral input device such as a mouse or a keyboard, a use habit of the user is totally reserved, and there is no need to frequently switch between the two devices.
  • An application of the mobile phone 100 is run by using the mobile phone window of the PC 200 , thereby improving efficiency of file query, sharing, and office, and improving user experience.
  • the multi-window display method provided in the embodiments of this application may be applied to a system including two electronic devices that communicate with each other.
  • the system includes a first electronic device and a second electronic device that is configured to display an interface of the first electronic device.
  • the mobile phone 100 serves as the first electronic device
  • the PC 200 serves as the second electronic device.
  • the PC 200 may display a plurality of windows of the mobile phone 100 .
  • FIG. 8 to FIG. 12 the following describes a specific implementation process of displaying a plurality of windows of the mobile phone 100 by the PC 200 .
  • FIG. 8 is a schematic flowchart of a multi-window display method according to an embodiment of this application. As shown in FIG. 8 , the method 800 may include the following steps.
  • a mobile phone 100 establishes a connection to a PC 200 .
  • the mobile phone 100 and the PC 200 need to communicate with each other.
  • a communication manner refer to the foregoing description. Details are not described herein again.
  • the link discovery service APK of the mobile phone 100 in FIG. 3 A may participate in a process of establishing a connection to the PC 200 by the mobile phone 100 . Details are not described herein again.
  • the mobile phone 100 obtains size information of a display screen of the PC 200 .
  • this embodiment of this application sets no limitation on an occasion and a manner of obtaining the size information of the display screen of the PC 200 by the mobile phone 100 .
  • the PC 200 may send information such as a size of the display screen to the mobile phone, and the mobile phone 100 may obtain the information such as the size of the display screen of the PC 200 .
  • the PC 200 may be triggered, based on an operation of a user, to send information such as a size of the display screen to the mobile phone, and the mobile phone 100 may obtain the information such as the size of the display screen of the PC 200 .
  • the PC 200 may send the information such as the size of the display screen to the mobile phone.
  • data exchange between the mobile phone 100 and the PC 200 may be transmitted by using different channels based on a process in which a connection is established between the two parties.
  • data between the mobile phone 100 and the PC 200 may be transmitted by discovering and establishing a Wi-Fi p2p path.
  • the information such as the size of the display screen of the PC 200 may be transmitted to the mobile phone 100 by using a Bluetooth channel. This is not limited in this embodiment of this application.
  • the multi-screen framework service APK at the application program framework layer of the mobile phone 100 may participate in step 801 .
  • the mobile phone 100 creates a virtual area based on the obtained size information of the display screen of the PC 200 .
  • the mobile phone 100 determines a first area of a mobile phone window 10 in the virtual area.
  • the mobile phone 100 may create a virtual area.
  • a size of the virtual area is the same as the size of the display screen of the PC 200 .
  • FIG. 9 A , FIG. 9 B , and FIG. 9 C are a schematic diagram of creating a virtual area by a mobile phone 100 according to an embodiment of this application.
  • a start point at an upper left corner of the display screen is used as a coordinate origin, and the size of the display screen is represented by using pixels in a horizontal direction (an x-axis direction) and a vertical direction (a y-axis direction).
  • all numbers are location coordinates of pixels. It should be understood that a larger quantity of pixels occupied in a direction indicates a more refined and delicate display effect.
  • Steps 803 and 804 may correspond to a process that is executed by the mobile phone 100 and that is shown in FIG. 9 A .
  • the size that is of the display screen of the PC 200 and that is obtained by the mobile phone 100 may be 3000 ⁇ 2000.
  • the display screen of the PC 200 includes 3000 pixels in the horizontal direction (the x-axis) and 2000 pixels in the vertical direction (the y-axis).
  • the mobile phone 100 creates the virtual area (an area shown by a dashed line) based on the size (3000 ⁇ 2000) information of the display screen of the PC 200 .
  • a coordinate origin of the virtual area is O (0, 0), and the virtual area includes 3000 pixels in the x-axis direction and 2000 pixels in the y-axis direction.
  • the mobile phone 100 determines, in the virtual area, the first area in which the mobile phone window 10 is located. In other words, the mobile phone 100 determines coordinates of a vertex O 1 of the first area, pixels included in the x-axis direction, and pixels included in the y-axis direction.
  • the coordinates of the vertex of the first area, a quantity of pixels included in the x-axis direction, and a quantity of pixels included in the y-axis direction may be preset in a system, or the first area is determined by the mobile phone 100 based on the size of the display screen of the PC 200 according to a preset rule. This is not limited in this embodiment of this application.
  • the mobile phone 100 may specify coordinates of an initial vertex, a quantity of pixels included in the x-axis direction, and a quantity of pixels included in the y-axis direction for each to-be-displayed window. For example, after the mobile phone 100 sets the coordinates of the initial vertex, the quantity of pixels included in the x-axis direction, and the quantity of pixels included in the y-axis direction for the mobile phone window 10 , the mobile phone window 10 may be displayed at a location close to a right frame of the display screen of the PC 200 . As shown in (a) in FIG.
  • the mobile phone window 10 is displayed at the location close to the right frame of the display screen of the PC 200 , and the mobile phone window 10 is located in the middle between an upper frame and a lower frame of the display screen of the PC 200 .
  • Setting of the initial coordinates can prevent the mobile phone window 10 from shielding an original application program (for example, My Computer or Recycle Bin) of the PC displayed in an interface 402 of the PC 200 , and can prevent the mobile phone window 10 from shielding a shortcut menu option such as an input method control menu, a time and date setting menu, or power management of the PC displayed in a lower-right corner area in the interface 402 of the PC 200 .
  • an original application program for example, My Computer or Recycle Bin
  • a shortcut menu option such as an input method control menu, a time and date setting menu, or power management of the PC displayed in a lower-right corner area in the interface 402 of the PC 200 .
  • the mobile phone 100 may specify the quantity of pixels included in the x-axis direction and the quantity of pixels included in the y-axis direction for each to-be-displayed window based on a display area of the mobile phone, to determine a size of the window to be displayed on the PC 200 .
  • the display area of the mobile phone herein is a maximum area that may be used to display an interface of the mobile phone on a display screen of the mobile phone 100 . As shown in FIG. 9 A , the display area of the mobile phone 100 is n ⁇ m.
  • the mobile phone window 10 on the PC 200 may have a same size as the display area (n ⁇ m) of the mobile phone.
  • the size of the mobile phone window 10 includes n pixels in the x-axis direction and m pixels in the y-axis direction.
  • the mobile phone 100 may specify the size as the size of the mobile phone window 10 displayed on the PC 200 .
  • the mobile phone window 10 on the PC 200 is used to display a home screen of the mobile phone 100 , and the mobile phone window 10 may be of a size obtained by magnifying the display area (n ⁇ m) of the mobile phone by 1.5 times.
  • the mobile phone 100 specifies that the mobile phone window 10 includes 1.5 ⁇ n pixels in the x-axis direction and 1.5 ⁇ m pixels in the y-axis direction.
  • the coordinates of the vertex O 1 of the first area are (2500, 500), and the first area includes 400 pixels in the x-axis direction and 1000 pixels in the y-axis direction.
  • the process that is of creating the virtual area and the first area by the mobile phone 100 and that is described above may be performed by the processor 110 of the mobile phone 100 in FIG. 1 , or is correspondingly completed by a module corresponding to the multi-screen frame service APK of the mobile phone 100 in the software architecture in FIG. 3 A .
  • the process of creating the virtual area by the mobile phone 100 may be understood as determining information about the virtual area
  • the process of determining the first area by the mobile phone 100 may be understood as determining information about the first area.
  • the process is an internal execution process of the mobile phone 100 , does not need to be displayed on the mobile phone 100 .
  • the process is completed by the mobile phone 100 , and is not displayed on the PC 200 .
  • the mobile phone 100 determines the information O 1 (2500, 500) about the first area, where the first area includes 400 pixels in the x-axis direction and 1000 pixels in the y-axis direction.
  • the mobile phone 100 encodes the information about the first area and display content information of the first area, to generate a first video stream.
  • the mobile phone 100 sends the first video stream to the PC 200 .
  • the first video stream includes the information about the first area, and may specifically include information such as the coordinates of the vertex, a length, and a width of the first area on the display screen of the PC 200 .
  • the information about the first area may further include display level information of the mobile phone window 10 .
  • a display layer herein is used to indicate a layer at which the window is displayed on the PC 200 in a current display method.
  • the PC 200 includes three mobile phone windows, a window displayed at an uppermost layer is displayed at the first layer, and a window displayed at a lowermost layer is displayed at the third layer.
  • the layer information may not be sent by the mobile phone 100 to the PC 200 , but the PC 200 manages layer display of all windows based on a time of receiving information about each window and an operation performed by the user on the PC 200 . For example, when a window 40 of a music application is displayed at a layer above a window 50 of a video application, and the user selects the window 50 of the video application on the PC, the PC 200 determines that the window 50 of the video application is displayed at the uppermost layer.
  • the mobile phone window 10 may be displayed at the uppermost layer of the PC 200 by default. Because display content in the mobile phone window 10 may correspond to the home screen 401 of the mobile phone 100 or include an application program of the mobile phone 100 , the mobile phone window 10 is displayed at the uppermost layer of the PC 200 , to facilitate use of the user. In addition, the mobile phone window 10 is located at the location close to the right frame of the display screen of the PC 200 , and does not shield an original application that is of the PC 200 and that is used by the user.
  • the display content information that is of the first area and that is included in the first video stream is used by the PC 200 to display content of the mobile phone 100 in the mobile phone window 10 .
  • the PC 200 may fill the first area based on the display content information of the first area, so that content displayed in the first area on the display screen of the PC 200 is an interface of the mobile phone 100 .
  • the display content information of the first area includes a wallpaper of the home screen of the mobile phone 100 , icons of all application programs, a window including a time, a date, and weather, and an operator name, a network signal identifier, power display information, and the like in a notification bar at the top of the interface of the mobile phone 100 .
  • the display content information of the first area includes a wallpaper of the home screen of the mobile phone 100 , icons of all application programs, a window including a time, a date, and weather, and an operator name, a network signal identifier, power display information, and the like in a notification bar at the top of the interface of the mobile phone 100 .
  • the display content information of the first area includes a wallpaper of the interface of the music application, and each control, a song name, a play progress, a singer image, and the like in the interface of the music application.
  • the PC 200 may further display a window related to an application that currently runs on the PC 200 , for example, the display screen of the PC 200 displays a web page window opened by the user on the PC 200 .
  • the mobile phone window 10 is displayed at the uppermost layer, to facilitate an operation of the user.
  • the user may tap/click any location of the web page window to display the web page window at the uppermost layer of the PC 200 , and move the mobile phone window 10 down by one layer (display the mobile phone window 10 at the second layer).
  • the user may tap/click a hide to sidebar key 11 - 1 or a hide to menu bar key 11 - 2 in a gray menu bar 11 of the mobile phone window 10 to hide the mobile phone window 10 .
  • the mobile phone window 10 is temporarily not displayed on the display screen of the PC 200 .
  • the MSDP service APK may implement a function including video stream encoding and the like.
  • the multi-screen frame service APK of the mobile phone 100 may send the determined information about the first area and the determined display content information of the first area to the MSDP service APK, and the MSDP service APK encodes the information about the first area and the display content information of the first area, to generate the first video stream.
  • the encoding process of the video stream is not described herein again.
  • the first video stream may be transmitted by using a dedicated video stream channel provided by the MSDP service, or may be transmitted by using a channel established in the process in which the mobile phone 100 establishes a connection to the PC 200 in step 701 .
  • a dedicated video stream channel provided by the MSDP service
  • the first video stream has a data format that can be parsed by both the mobile phone 100 and the PC 200 . Details are not described herein.
  • Step 805 may correspond to a process that is executed by the mobile phone 100 and that is shown in FIG. 9 B , and the process is also a video stream encoding process inside the mobile phone 100 .
  • the PC 200 receives and decodes the first video stream, to obtain the information about the first area and the display content information of the first area.
  • the PC 200 displays the mobile phone window 10 in a corresponding area based on the information about the first area, and displays corresponding content in the mobile phone window 10 based on the display content information of the first area.
  • the PC 200 receives the first video stream sent by the mobile phone 100 , and decodes the first video stream. It should be understood that a decoding manner corresponds to an encoding manner of the mobile phone 100 .
  • the process may be executed by the MSDP service APK of the PC 200 , and the MSDP service APK sends, to the multi-screen frame service APK, the information that is about the first area and that is obtained after the first video stream is decoded.
  • the multi-screen frame service APK determines the information such as the coordinates of the vertex, the length, and the width of the mobile phone window 10 on the display screen of the PC 200 based on the information about the first area, to determine a display area of the mobile phone window 10 .
  • the PC 200 further obtains the display content information of the first area from the first video stream.
  • the PC 200 may fill the mobile phone window 10 based on the display content information of the first area, so that content displayed in the mobile phone window 10 on the display screen of the PC 200 is an interface of the mobile phone 100 .
  • the interface may be the home screen of the mobile phone 100 or an interface of any application of the mobile phone 100
  • the display content information of the first area includes the wallpaper of the interface of the music application, and each control, the song name, the play progress, the singer image, and the like in the interface of the music application.
  • the MSDP service APK sends the obtained display content information of the first area to the multi-screen frame service APK.
  • the mobile phone window display of the multi-screen frame service APK module displays corresponding interface content based on the display content information of the first area.
  • the display screen of the PC 200 may display the mobile phone window 10 , and the home screen of the mobile phone 100 may be displayed in the mobile phone window 10 , or any interface of the mobile phone 100 in a use process of the user may be synchronously displayed in the mobile phone window 10 .
  • the PC 200 may display the mobile phone window 10 in the main interface 402 .
  • Display content in the mobile phone window 10 may continuously display the home screen of the mobile phone 100 .
  • display content in the mobile phone window 10 of the PC 200 may be synchronized with that on the mobile phone 100 , and is the interface of the music application.
  • Steps 807 and 808 may correspond to a process that is executed by the mobile phone 100 and that is shown in FIG. 9 C .
  • the PC 200 corresponds the first area to a corresponding location on the display screen of the PC 200 based on the coordinates of the vertex, the length, and the width of the first area in the virtual area.
  • the location corresponding to the first area is the display area of the mobile phone window 10 .
  • coordinates of the mobile phone window 10 on the display screen of the PC 200 are coordinates of the first area in the virtual area.
  • the mobile phone 100 may encode the information about the virtual area, the information about the first area, and the display content information of the first area together, to generate the first video stream, and send the first video stream to the PC 200 .
  • the PC 200 directly determines a location of the mobile phone window 10 on the display screen of the PC 200 based on the location of the first area in the virtual area, and displays the display content of the first area in the mobile phone window 10 .
  • the mobile phone 100 encodes information about the display area (n ⁇ m) of the mobile phone to generate the first video stream, and sends the first video stream to the PC 200 .
  • the PC 200 determines, based on the information about the display area (n ⁇ m) of the mobile phone, a location and a size of the mobile phone window 10 that is used to display the home screen of the mobile phone.
  • a location and a size of the mobile phone window 10 that is used to display the home screen of the mobile phone.
  • the mobile phone 100 determines the coordinates of the vertex of the first area, the quantity of pixels in the x-axis direction, and the quantity of pixels in the y-axis direction. Details are not described herein again.
  • the PC 200 may also determine, based on information that is sent by the mobile phone 100 and that is about the interface of the music application displayed on the mobile phone, a location and a size of the window 40 that is used to display the music application.
  • coordinates of the window 40 of the music application may be sent by the PC 200 to the mobile phone 100 , and the mobile phone 100 may update information about the coordinates of the window 40 of the music application.
  • information is exchanged between the mobile phone 100 and the PC 200 in real time.
  • the PC 200 may notify the mobile phone 100 of information such as the changed location and size of the window.
  • the mobile phone 100 may accurately obtain information about the window of the music application at any time.
  • a window coordinate synchronization module of the MSDP service APK of the mobile phone 100 described in FIG. 3 A may participate in the process.
  • the mobile phone 100 may learn of, in real time, a location of each application window displayed on the PC 200 , a location of a function control included in each application, and the like, so that the user can use and operate an application of the mobile phone 100 on the PC 200 .
  • FIG. 10 is another schematic flowchart of a multi-window display method according to an embodiment of this application.
  • a process shown in FIG. 10 may be executed after the process shown in FIG. 8 .
  • the process shown in FIG. 10 may be continued.
  • the process 1000 may correspond to the process shown in (a) in FIG. 6 .
  • a user performs an operation on a music application of a mobile phone on a mobile phone 100 side, and in response to the operation performed by the user on the mobile phone 100 , the PC 200 may display a window 40 of the music application in addition to the original mobile phone window 10 .
  • the method 1000 may include the following steps.
  • the display screen of the PC 200 displays the mobile phone window 10 .
  • the mobile phone 100 receives a first operation of the user, and runs the music application.
  • the first operation may be a tapping operation performed by the user on the mobile phone 100 on the music application.
  • the music application is opened and entered on the mobile phone 100 , that is, the mobile phone 100 may display an interface of the music application. Therefore, running the music application herein may be running the music application in the foreground, that is, a display screen of the mobile phone 100 displays the interface of the music application. 1003 .
  • the mobile phone 100 determines a second area in the virtual area in step 803 , and obtains display content information of the second area.
  • step 803 the virtual area created by the mobile phone 100 has a same size as the display screen of the PC 200 , and the mobile phone 100 has determined a first area in the virtual area for the mobile phone window 10 .
  • step 1003 the mobile phone 100 determines the second area in the created virtual area for the window 40 of the music application.
  • FIG. 11 A , FIG. 11 B , and FIG. 11 C are another schematic diagram of creating a virtual area by a mobile phone 100 according to an embodiment of this application. Steps 1003 and 1004 may correspond to a process that is executed by the mobile phone 100 and that is shown in FIG. 11 A .
  • the mobile phone 100 creates a virtual area shown by a dashed line.
  • a coordinate origin of the virtual area is 0 (0, 0), and the virtual area includes 3000 pixels in an x-axis direction and 2000 pixels in a y-axis direction.
  • the mobile phone 100 determines a location and a size of the mobile phone window 10 in various possible manners. Similarly, the mobile phone 100 may further determine, for each application, a location and a size of a window for displaying the application.
  • coordinates of an initial vertex of a window for displaying each application on the PC 200 may be set by the mobile phone 100 , and a quantity of pixels included in the x-axis direction and a quantity of pixels included in the y-axis direction may also be set by the mobile phone 100 .
  • the mobile phone 100 may specify coordinates of an initial vertex, a quantity of pixels included in the x-axis direction, and a quantity of pixels included in the y-axis direction for the to-be-displayed window 40 of the music application based on information about a display area (n ⁇ m) of the mobile phone 100 .
  • the window 40 of the music application on the PC 200 is used to display the interface of the music application of the mobile phone, and the window 40 of the music application may have a same size as the display area (n ⁇ m) of the mobile phone.
  • the window 40 of the music application includes n pixels in the x-axis direction and m pixels in the y-axis direction.
  • an x-axis coordinate of the vertex O 2 of the window 40 of the music application is obtained by subtracting n from an x-axis coordinate of O 1 .
  • Ay-axis coordinate of the vertex O 2 may be the same as a y-axis coordinate of O 1 , or a y-axis coordinate is obtained by subtracting m from a value obtained by adding a y-axis coordinate of O 1 and a y-axis size of the mobile phone window 10 .
  • the mobile phone 100 determines the second area of the window 40 of the music application.
  • the second area includes 400 pixels in the x-axis direction and 500 pixels in the y-axis direction, where 400 is equal to n, and 500 pixels are 0.5 ⁇ m.
  • the mobile phone 100 may further determine that coordinates of the vertex O 2 are (2100, 1000), to ensure that the window 40 of the music application is close to the mobile phone window 10 .
  • a window 50 of a video application on the PC 200 is used to display an interface of the video application of the mobile phone.
  • the window 50 of the video application may have a same size as the display area (n ⁇ m) of the mobile phone, m in the display area (n ⁇ m) of the mobile phone is used as a quantity of pixels included in the x-axis direction, and n is used as a quantity of pixels included in the y-axis direction.
  • the size of the window 50 of the video application can be better adapted to a process in which the user views a video, thereby improve viewing experience of the user.
  • the mobile phone 100 may specify the quantity of pixels included in the x-axis direction and the quantity of pixels included in the y-axis direction for the to-be-displayed window 40 of the music application based on the information about the display area (n ⁇ m) of the mobile phone 100 and a preset proportion relationship.
  • the mobile phone 100 specifies that the window 40 of the music application includes 1.5 ⁇ n pixels in the x-axis direction and 1.5 ⁇ m pixels in the y-axis direction.
  • the mobile phone 100 may determine, for the to-be-displayed window 40 of the music application based on the interface that is of the music application and that is displayed when the mobile phone runs the music application, the quantity of pixels included in the x-axis direction and the quantity of pixels included in the y-axis direction, to determine a size of a window to be displayed on the PC 200 .
  • the interface that is of the music application and that is displayed when the mobile phone runs the music application herein may be less than the display area n ⁇ m of the mobile phone 100 .
  • a size of the interface that is of the music application and that is displayed when the mobile phone runs the music application is p ⁇ q.
  • values of n and m may be obtained by the mobile phone 100 based on information about the display screen, and values of p and q may be information obtained when an installation package of the music application is installed on the mobile phone 100 , or information configured by the mobile phone 100 for the music application. This is not limited in this embodiment of this application.
  • the size of the window 40 of the music application may be the same as the size (p ⁇ q) of the interface of the music application displayed on the mobile phone 100 .
  • the window 40 of the music application includes p pixels in the x-axis direction and q pixels in the y-axis direction.
  • an x-axis coordinate of the vertex O 2 of the window 40 of the music application is obtained by subtracting p from an x-axis coordinate of O 1 .
  • Ay-axis coordinate of the vertex O 2 may be the same as a y-axis coordinate of O 1 , or a y-axis coordinate is obtained by subtracting q from a value obtained by adding a y-axis coordinate of O 1 and a y-axis size of the mobile phone window 10 .
  • the mobile phone 100 determines, according to a preset rule, the quantity of pixels included in the x-axis direction and the quantity of pixels included in the x-axis direction. For example, if the mobile phone 100 determines the coordinates of the initial vertex, the quantity of pixels included in the x-axis direction and the quantity of pixels included in the y-axis direction may be determined according to the preset rule.
  • a rule for setting initial coordinates of the window 40 of the music application that starts and runs first may be as follows: A right frame of the window 40 of the music application is close to a left frame of the mobile phone window 10 , and the window 40 does not shield the content in the mobile phone window 10 .
  • a rule for setting initial coordinates of the window 50 of the video application that starts and runs second may be as follows: A right frame of the window 50 of the video application is close to a left frame of the window 40 of the music application, and the window 50 does not shield content in the window 40 of the music application. Alternatively, a right frame of the window 50 of the video application that starts and runs second is still close to the left frame of the mobile phone window 10 , and the window 50 of the video application that starts and runs second is located at an upper layer of the window 40 of the music application that starts and runs first.
  • the window 50 of the video application is displayed at the upper layer of the window 40 of the music application. If the user taps/clicks the window 50 of the music application, the window 50 of the music application is displayed at an upper layer of the window 50 of the video application.
  • the window 40 of the music application is displayed at a location close to the left frame of the mobile phone window 10 , and the window 40 of the music application is located in the middle between an upper frame and a lower frame of the display screen of the PC 200 .
  • Setting of the initial coordinates may prevent the window 40 of the music application from shielding the mobile phone window 10 , so that the user continues to use an application program in the mobile phone window 10 .
  • the mobile phone 100 may further obtain the display content information of the second area, and the display content information is used by the PC 200 to display interface content of the music application of the mobile phone 100 in the window 40 of the music application.
  • the PC 200 may fill the second area based on the display content information of the second area, so that content displayed in the second area of the display screen of the PC 200 is the interface content of the music application of the mobile phone 100 .
  • the information about the second area may further include identifier information of the music application.
  • the identifier information of the music application is used to identify that the window is a window of the music application, and both the PC 200 and the mobile phone 100 may determine that the window is a window of the music application.
  • the PC 200 may determine that the operation is for the music application, to subsequently transfer the operation back to the mobile phone 100 accurately.
  • the mobile phone 100 and the PC 200 may determine, in another manner, an application corresponding to each window, or determine an application corresponding to an operation performed by the user in the window, to accurately provide a service.
  • the information about the second area further includes display level information.
  • a display layer herein is used to indicate a layer at which the window 40 of the music application is displayed on the PC 200 in a current display method. For example, if the user currently runs the music application, the music application is displayed at the first layer, that is, displayed at an uppermost layer of the display screen of the PC 200 . Similarly, when the user successively taps and runs the music application, the video application, and a chat application of the mobile phone 100 , a window of the chat application on the display screen of the PC 200 is displayed at the first layer, the window of the video application is displayed at the second layer, and the window of the music application is displayed at the third layer. In addition, when the user taps the window of the music application again, the window of the music application is displayed at the uppermost layer, and details are not described in this embodiment of this application.
  • the mobile phone 100 encodes the identifier information of the music application, the information about the second area, and the display content information of the second area, to generate a second video stream.
  • step 1005 may correspond to a process that is executed by the mobile phone 100 and that is shown in FIG. 11 B .
  • the process is also a video stream encoding process inside the mobile phone 100 .
  • the multi-screen frame service APK of the mobile phone 100 sends the determined identifier information of the music application, the determined information about the second area, and the determined display content information of the second area to the MSDP service APK.
  • the MSDP service APK encodes the information about the second area and the display content information of the second area, to generate the second video stream.
  • the mobile phone 100 sends the second video stream to the PC 200 .
  • the PC 200 receives and decodes the second video stream, to obtain the identifier information of the music application, the information about the second area, and the display content information of the second area.
  • the PC 200 displays the window 40 of the music application in a corresponding area based on the information about the second area and the display content information of the second area.
  • Steps 1006 and 1007 may correspond to a process that is executed by the mobile phone 100 and that is shown in FIG. 11 C .
  • the mobile phone window 10 and the window 40 of the music application may be displayed on the display screen of the PC 200 .
  • the PC 200 may display the mobile phone window 10 and the window 40 of the music application in the main interface 402 .
  • Display content in the mobile phone window 10 may be the home screen of the mobile phone 100
  • display content in the window 40 of the music application may be the interface of the music application of the mobile phone 100 .
  • the window 40 of the music application is displayed on the PC 200 .
  • the music application runs in the background, and does not affect a process of displaying the window 40 of the music application on the PC 200 .
  • the user taps the icon of the music application on the home screen 401 of the mobile phone 100 , and plays music.
  • the mobile phone window 10 displayed on the PC 200 may be displayed as the interface of the music application or the window 40 of the music application may be displayed on the PC 200 .
  • the music audio may be actually played on a device in which the music application is started. Specifically, if the user taps the music application on the mobile phone 100 , a device that actually plays the music audio is the mobile phone 100 .
  • the user plays music by tapping/clicking a music application 14 in the mobile phone window 10 of the PC 200 .
  • the window 40 of the music application is displayed on the PC 200
  • the mobile phone window 10 may also be displayed as the interface of the music application.
  • the device that actually plays the music audio is the PC 200 .
  • a device that actually plays the music audio when the music application runs is the mobile phone 100 .
  • the PC 200 may display the interface 402 shown in (a) in FIG. 7 .
  • a device that actually plays a video is the PC 200 .
  • the mobile phone 100 sends the information about video content in the window 50 of the video application by using the video stream, after the information about the video content is obtained by the PC 200 , the video is played by a play device such as a display or a speaker of the PC 200 .
  • the video application runs in the background for the mobile phone 100 , and therefore may not affect original music that is being played on the mobile phone 100 .
  • the devices for playing the music and playing the video are the PC 200 .
  • an audio preemption principle refer to a solution in the prior art. For example, if a priority of the video application that runs later is higher, audio play of the music is automatically paused. This is not limited in this embodiment of this application.
  • the foregoing describes a process of displaying two windows (the mobile phone window 10 and the window 40 of the music application) by the PC 200 .
  • the user may further run more applications of the mobile phone 100 , such as a video application, a chat application, and an office software application.
  • a plurality of windows may be displayed on the PC 200 , and each window corresponds to a different mobile phone application. Details are not described herein.
  • content of the mobile phone 100 may be displayed by using the PC 200 .
  • a plurality of windows may be displayed on the PC 200 , and each window is used to display a different application.
  • Each window may receive an operation of the user, for example, an operation of closing, moving, scaling, maximizing, and minimizing the window.
  • the user may perform an operation on each mobile phone application in a window that corresponds to the mobile phone application and that is displayed on the PC 200 .
  • the user taps/clicks a control such as Play and Next in the window that is of the music application and that is on the PC 200 , to control a music play process.
  • a control such as Play and Next in the window that is of the music application and that is on the PC 200
  • an operation performed by the user on the mobile phone application on the PC 200 does not affect use of another function of the user on the mobile phone 100 , for example, the foregoing scenario in which the user plays music by using the mobile phone 100 and views a video by using the PC 200 at the same time.
  • office efficiency such as file sharing can be improved, and user experience is improved.
  • the mobile phone 100 encodes, by using a video stream transmission process, identifier information of different running applications, and information such as coordinates of a vertex, a length, and a width of a window for displaying the application on the PC 200 , to generate a video stream.
  • the mobile phone 100 encodes the identifier information of the music application, information such as coordinates of a vertex, a length, and a width of the window 40 for displaying the music application on the PC 200 , identifier information of the video application, and information such as coordinates of a vertex, a length, and a width of the window 50 for displaying the video application on the PC 200 , to generate one video stream, and sends the video stream to the PC 200 .
  • the PC 200 distinguishes between different information, to accurately display the interface content of the music application in the window 40 of the music application, and display the interface content of the video application in the window 50 of the video application.
  • the one video stream is a video stream sent by using one video stream transmission channel. If a video transmission channel meets a transmission condition, encoding may be performed to generate a plurality of video streams. For example, identifier information of the music application and information such as coordinates of a vertex, a length, and a width of the window 40 for displaying the music application on the PC 200 are encoded to generate one video stream that is sent by using one video stream transmission channel. Identifier information of the video application and information such as coordinates of a vertex, a length, and a width of the window 50 for displaying the video application on the PC 200 are encoded to generate another video stream that is sent by using another video stream transmission channel.
  • the mobile phone 100 sends location information of a first window and display content information of a first interface to the PC 200 , and the PC 200 displays the first window based on an indication of the mobile phone 100 .
  • the mobile phone 100 may not obtain size information of a display area of the PC 200 , but send only the display content information of the first interface of the mobile phone 100 .
  • the PC 200 determines the location information of the first window, and displays the first window. Similarly, display of a second window may also be determined by the PC 200 .
  • the PC 200 may determine display locations of the first window and the second window based on application use, interface content, and the like of the PC 200 , to be better adapted to a use process of the PC 200 , reduce impact on use of the PC 200 , and improve user experience.
  • FIG. 10 , FIG. 11 A , FIG. 11 B , and FIG. 11 C the following is described above: After the display screen of the PC 200 has displayed the mobile phone window 10 , the user may tap the music application in the interface of the mobile phone 100 , and may display the window 40 of the music application on the PC 200 . In addition, the user may tap/click the music application 14 in the mobile phone window 10 of the PC 200 , or may display the window 40 of the music application on the PC 200 .
  • FIG. 12 is another schematic flowchart of a multi-window display method according to an embodiment of this application. As shown in FIG. 12 , the method 1200 may include the following steps.
  • a display screen of a PC 200 displays a mobile phone window 10 .
  • the PC 200 receives a second operation of a user, and determines operation information of the second operation.
  • the second operation may be an operation performed on an application to start the application.
  • a first operation and the second operation may be different operations performed for different devices.
  • the user may run the music application by performing a clicking operation or double-clicking operation by using a peripheral input device such as a mouse or a keyboard.
  • a peripheral input device such as a mouse or a keyboard.
  • the PC 200 is a device with a touchscreen, the user may run the music application through touching and tapping.
  • An operation type of the second operation is not limited in this embodiment of this application.
  • the operation information of the second operation includes an operation event for an application in the mobile phone window 10 , and the operation information of the second operation may further include at least one type of information such as a name, an ID, and an icon of the music application.
  • the second operation is an operation event (“touch event”).
  • touch event the PC 200 determines the name, the ID, and the icon of the music application in which the “touch event” occurs and coordinates of the “touch event” on the PC, that is, determines the operation information of the second operation.
  • the PC 200 sends the operation information of the second operation to a mobile phone 100 .
  • the mobile phone 100 determines, based on the received operation information of the second operation, to run the music application.
  • the PC 200 transfers the “touch event” to the mobile phone 100 .
  • a system of the mobile phone 100 determines the “touch event”, and sends the “touch event” to the corresponding music application.
  • the music application of the mobile phone 100 determines that the “touch event” is used to tap and run the music application. It should be understood that, for the mobile phone 100 , the music application may run in the foreground. For example, an interface of the music application is directly displayed on a home screen 401 of the mobile phone 100 . Alternatively, the music application may run in the background. For example, an interface of the music application is not displayed in the interface of the mobile phone 100 , and the music application runs only in the background.
  • the mobile phone 100 sends the second video stream to the PC 200 .
  • the PC 200 receives and decodes the second video stream, to obtain the identifier information of the music application, the information about the second area, and the display content information of the second area.
  • the PC 200 displays a window 40 of the music application in a corresponding area based on the information about the second area and the display content information of the second area.
  • steps 1205 to 1209 refer to steps 1003 to 10007 in FIG. 10 and the process described in FIG. 11 A , FIG. 11 B , and FIG. 11 C in which the mobile phone 100 encodes the second video stream and sends the second video stream to the PC 200 , and the display screen of the PC 200 displays the window 40 of the music application. Details are not described herein again. In this case, the music application of the mobile phone 100 may run on the PC 200 based on the operation of the user.
  • the PC 200 does not need to install an application of the mobile phone 100 , and different applications of the mobile phone 100 may be displayed by using different windows.
  • the user may run the application of the mobile phone 100 on the PC 200 , and display one window for each application, and the PC 200 may display a plurality of windows at the same time.
  • the mobile phone application window 30 may be displayed in the main interface 402 of the PC 200 , and the mobile phone application window 30 may include all applications of the mobile phone 100 .
  • FIG. 13 is another schematic flowchart of a multi-window display method according to an embodiment of this application. As shown in FIG. 13 , the method 1300 may include the following steps.
  • a mobile phone 100 sends application information to a PC 200 .
  • the PC 200 displays all applications of the mobile phone 100 based on the obtained application information.
  • the mobile phone 100 may automatically send, to the PC 200 , information about all applications at the application program layer shown in FIG. 3 A .
  • the mobile phone 100 may automatically send, to the PC 200 , information about all applications at the application program layer shown in FIG. 3 A .
  • FIG. 4 when tapping/clicking an “EMUJI desktop” menu in a menu window 20 , a user triggers the PC 200 to send an application obtaining request to the mobile phone 100 , to request to obtain the application information of the mobile phone 100 .
  • the application information herein may include information such as names, identifiers (identification, ID), and icons of all the applications at the application program layer.
  • the PC 200 displays each application in a mobile phone application window 30 based on the name, the ID, and the icon of the application, to display all the applications of the mobile phone 100 .
  • the display may be shown in FIG. 5 C .
  • the identifier of the application may be represented as a package name, the package name includes a string, and the PC 200 may determine a music application based on the package name.
  • the PC 200 displays an application list of the mobile phone 100 by using the mobile phone application window 30 . All application programs in the application list are application programs installed on the mobile phone 100 .
  • the PC 200 may not install an application program of the mobile phone 100 , and only displays an icon of the application list of the mobile phone 100 by using the mobile phone application window 30 .
  • the PC 200 receives a third operation of the user, and determines operation information of the third operation.
  • the PC 200 sends the operation information of the third operation to the mobile phone 100 .
  • the user may tap/click any application in the application list presented on the EMUJI desktop in a main interface 402 of the PC by using a peripheral input device such as a mouse or a keyboard.
  • a window of the tapped/clicked application may automatically pop up on the PC, to run the application.
  • the third operation may be a tapping/clicking operation performed by the user on a control in an application window on the PC 200 , and is used to execute a function of the control.
  • the operation information of the third operation further includes at least one type of information such as a name, an ID, and an icon of a control in an interface of the music application.
  • the operation information of the third operation may include an event type of the third operation and identifier (identification, ID) information of a function control on which the third operation is performed.
  • the PC 200 may send, to the mobile phone 100 , a “touch event” event of tapping/clicking the play control, for example, coordinates of the window 40 of the music application in which the “touch event” occurs, and a name, an ID, and an icon of the play control in the window 40 of the music application and coordinates of the play control on the PC 200 .
  • a “touch event” event of tapping/clicking the play control for example, coordinates of the window 40 of the music application in which the “touch event” occurs, and a name, an ID, and an icon of the play control in the window 40 of the music application and coordinates of the play control on the PC 200 .
  • the PC 200 transfers the “touch event” to the mobile phone 100 .
  • a system of the mobile phone 100 determines, based on the coordinates of the window 40 of the music application, that the “touch event” occurs in the music application, and sends the “touch event” to the corresponding music application.
  • the music application determines, based on the name, the ID, and the icon of the play control and the coordinates of the play control on the PC 200 , that the “touch event” is used to tap the play control. It should be understood that the process may be a background running process of the mobile phone 100 .
  • a first operation, a second operation, and the third operation may be different operations performed for different devices.
  • Running the music application is used as an example.
  • the user may run the music application by performing a touching and tapping operation.
  • the user may run the music application by performing a clicking operation or double-clicking operation by using a peripheral input device such as a mouse or a keyboard.
  • a peripheral input device such as a mouse or a keyboard.
  • the PC is a device with a touchscreen, the user may run the music application through touching and tapping.
  • Operation types of the first operation, the second operation, and the third operation are not limited in this embodiment of this application.
  • the music application of the mobile phone 100 determines an operation result based on the operation information of the third operation, and executes the operation result.
  • the mobile phone 100 sends the operation result to the PC 200 .
  • the PC 200 receives the operation result sent by the mobile phone 100 , and synchronizes the operation result.
  • a process of sending the operation result to the PC 200 by the mobile phone 100 in step 1306 may correspond to the process of creating a virtual area and sending the first video stream to the PC 200 by the mobile phone 100 in FIG. 9 A , FIG. 9 B , and FIG. 9 C and FIG. 11 A , FIG. 11 B , and FIG. 11 C .
  • the process of sending the operation result may also be completed by using a video stream sending process.
  • the mobile phone 100 re-determines the first video stream based on the music application currently running (running in the background or running in a current interface of the mobile phone) on the mobile phone 100 , so that the PC 200 displays the window 40 of the music application, that is, the operation of tapping/clicking the music play control is completed in the window 40 of the music application on the PC 200 , for example, if music play is paused, the music play control changes to a pause state in the window 40 of the music application of the PC 200 .
  • a reverse control module of the MSDP service APK of each of the mobile phone 100 and the PC 200 may participate in the process of transferring an operation back, and it is ensured that a corresponding operation is performed synchronously on the mobile phone 100 and the PC 200 .
  • the user may perform an operation on any control of a mobile phone application in a window that is of the application and that is displayed on the PC 200 .
  • the operation may be transferred back to the mobile phone 100 , and the mobile phone 100 executes an operation result, and transfers an execution result to the PC 200 , so that the operation result executed by the mobile phone 100 can be synchronized in the window that is of the application and that is on the PC 200 , thereby improving use efficiency and use experience of the mobile phone application.
  • the electronic device such as a mobile phone and a PC includes corresponding hardware and/or software modules for performing the functions.
  • the embodiments of this application can be implemented in a form of hardware or a combination of hardware and computer software.
  • function modules of the electronic device may be obtained through division according to the foregoing method examples.
  • the function modules may be obtained through division corresponding to various functions, or two or more functions may be integrated into one processing module.
  • the integrated module may be implemented in a form of hardware. It should be noted that, in the embodiments, division into modules is an example, and is merely logical function division. In actual implementation, another division manner may be used.
  • An embodiment further provides a computer readable storage medium.
  • the computer readable storage medium stores a computer instruction.
  • the computer instruction runs on an electronic device, the electronic device is enabled to perform the related method steps, to implement the multi-window display method in the foregoing embodiment.
  • An embodiment further provides a computer program product.
  • the computer program product runs on a computer, the computer is enabled to perform the related steps, to implement the multi-window display method in the foregoing embodiment.
  • an embodiment of this application further provides an apparatus.
  • the apparatus may be specifically a chip, a component, or a module.
  • the apparatus may include a processor and a memory that are connected to each other.
  • the memory is configured to store a computer executable instruction.
  • the processor may execute the computer executable instruction stored in the memory, so that the chip performs the multi-window display method in the foregoing method embodiments.
  • the electronic device, the computer storage medium, the computer program product, or the chip provided in the embodiments is configured to perform a corresponding method provided above. Therefore, for beneficial effects that can be achieved by the electronic device, the computer storage medium, the computer program product, or the chip, refer to beneficial effects of the corresponding method provided above. Details are not described herein again.
  • the disclosed apparatus and method may be implemented in other manners.
  • the described apparatus embodiment is merely an example.
  • division into the modules or units is merely logical function division.
  • a plurality of units or components may be combined or integrated into another apparatus, or some features may be ignored or not performed.
  • the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented by using some interfaces.
  • the indirect couplings or communication connections between the apparatuses or units may be implemented in electronic, mechanical, or other forms.
  • the units described as separate parts may or may not be physically separate, and parts displayed as units may be one or more physical units, may be located in one place, or may be distributed on different places. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.
  • function units in the embodiments of this application may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units may be integrated into one unit.
  • the integrated unit may be implemented in a form of hardware, or may be implemented in a form of a software function unit.
  • the integrated unit When the integrated unit is implemented in a form of a software function unit and sold or used as an independent product, the integrated unit may be stored in a readable storage medium. Based on such an understanding, the technical solutions of the embodiments of this application essentially, or the part contributing to the prior art, or all or some of the technical solutions may be implemented in a form of a software product.
  • the software product is stored in a storage medium and includes several instructions for instructing a device (which may be a single-chip microcomputer, a chip, or the like) or a processor (processor) to perform all or some of the steps of the methods described in the embodiments of this application.
  • the foregoing storage medium includes: any medium that can store program code, such as a USB flash drive, a removable hard disk, a read only memory (read only memory, ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disc.
  • program code such as a USB flash drive, a removable hard disk, a read only memory (read only memory, ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Telephone Function (AREA)
  • Digital Computer Display Output (AREA)
US17/770,372 2019-12-26 2020-11-19 Multi-window display method, electronic device, and system Active US12061833B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN201911366233.2A CN113050841A (zh) 2019-12-26 2019-12-26 显示多窗口的方法、电子设备和系统
CN201911366233.2 2019-12-26
PCT/CN2020/130112 WO2021129253A1 (zh) 2019-12-26 2020-11-19 显示多窗口的方法、电子设备和系统

Publications (2)

Publication Number Publication Date
US20220398059A1 true US20220398059A1 (en) 2022-12-15
US12061833B2 US12061833B2 (en) 2024-08-13

Family

ID=76505339

Family Applications (1)

Application Number Title Priority Date Filing Date
US17/770,372 Active US12061833B2 (en) 2019-12-26 2020-11-19 Multi-window display method, electronic device, and system

Country Status (5)

Country Link
US (1) US12061833B2 (zh)
EP (1) EP4040277A4 (zh)
JP (1) JP2023503679A (zh)
CN (2) CN115793916A (zh)
WO (1) WO2021129253A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220197430A1 (en) * 2020-12-23 2022-06-23 Seiko Epson Corporation Image display system, method for controlling image display system, and method for controlling display apparatus
US20230229375A1 (en) * 2020-11-16 2023-07-20 Honor Device Co., Ltd. Electronic Device, Inter-Device Screen Coordination Method, and Medium
US20230342106A1 (en) * 2020-12-28 2023-10-26 Huawei Technologies Co., Ltd. Collaborative display method, terminal device, and computer-readable storage medium
US20230393890A1 (en) * 2022-06-03 2023-12-07 Snap Inc. Auto-recovery for ar wearable devices
CN117873735A (zh) * 2024-03-11 2024-04-12 湖南马栏山视频先进技术研究院有限公司 一种虚拟化环境下gpu调度系统
US12032867B2 (en) * 2021-10-18 2024-07-09 Samsung Electronics Co., Ltd. Source device and sink device for sharing expanded screen, and methods of operating the same

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115543163A (zh) * 2021-06-30 2022-12-30 华为技术有限公司 一种投屏显示方法及电子设备
CN113391782A (zh) * 2021-06-30 2021-09-14 深圳市斯博科技有限公司 计算机控制移动终端的方法、系统、电子设备及存储介质
CN115617427A (zh) * 2021-07-13 2023-01-17 华为技术有限公司 应用的跨平台显示方法、可读介质和电子设备
CN115835409A (zh) * 2021-09-17 2023-03-21 荣耀终端有限公司 协同窗口处理方法、系统及电子设备
CN113849289A (zh) * 2021-09-30 2021-12-28 联想(北京)有限公司 一种控制方法和电子设备
CN116069224A (zh) * 2021-10-29 2023-05-05 华为终端有限公司 输入控制方法及电子设备
CN115146192B (zh) * 2022-09-02 2023-01-24 荣耀终端有限公司 内容接续方法和相关装置

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120040720A1 (en) * 2010-08-13 2012-02-16 Lg Electronics Inc. Mobile terminal, display device and controlling method thereof
US20140298355A1 (en) * 2013-04-01 2014-10-02 Samsung Electronics Co., Ltd. App operating method and device and app output device supporting the same
US20160054757A1 (en) * 2012-02-29 2016-02-25 Z124 Unified desktop docking flow
US20180101199A1 (en) * 2016-10-11 2018-04-12 Samsung Electronics Co., Ltd. Electronic device having a plurality of displays and operating method thereof
US20190012769A1 (en) * 2017-07-07 2019-01-10 Intelligent Waves Llc System, method and computer program product for remoting orientation changes
US20190296930A1 (en) * 2018-03-20 2019-09-26 Essential Products, Inc. Remote control of an assistant device using an adaptable user interface
US20200019431A1 (en) * 2016-12-23 2020-01-16 Samsung Electronics Co., Ltd. Electronic device and control method thereof
US20210042002A1 (en) * 2019-08-06 2021-02-11 Samsung Electronics Co., Ltd. Electronic device and method for sharing data thereof
US20220164091A1 (en) * 2012-09-10 2022-05-26 Samsung Electronics Co., Ltd. Method for connecting mobile terminal and external display and apparatus implementing the same

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9152582B2 (en) 2010-10-01 2015-10-06 Z124 Auto-configuration of a docked system in a multi-OS environment
WO2008085205A2 (en) 2006-12-29 2008-07-17 Prodea Systems, Inc. System and method for providing network support services and premises gateway support infrastructure
KR101198271B1 (ko) * 2012-01-10 2012-11-07 알서포트 주식회사 컴퓨터를 통한 터치스크린식 이동통신단말기의 제어에 있어서 컴퓨터 및 피제어 이동통신단말기의 화면상태 관리방법
KR101952682B1 (ko) * 2012-04-23 2019-02-27 엘지전자 주식회사 이동 단말기 및 그 제어방법
JP2015012512A (ja) * 2013-06-28 2015-01-19 株式会社東芝 情報処理装置および情報処理方法
KR102144553B1 (ko) 2013-08-30 2020-08-13 삼성전자주식회사 다중 디스플레이 방법, 저장 매체 및 전자 장치
CN104376090B (zh) 2014-11-20 2017-11-07 北京安奇智联科技有限公司 计算设备的浏览器与移动终端的屏幕同步
KR101594105B1 (ko) 2015-03-06 2016-02-16 주식회사 와이젯 사용자 장치 간에 여러 화면을 분배하고 입력 인터페이스를 공유하는 멀티 스크린 구현 방법 및 장치
CN104965702B (zh) 2015-06-15 2017-08-25 广东欧珀移动通信有限公司 一种智能终端的多窗口运行方法、装置及智能终端
US11360997B2 (en) 2015-12-21 2022-06-14 Sap Se Data synchronization error resolution based on UI manipulation actions
CN105828139A (zh) * 2016-04-06 2016-08-03 乐视控股（北京）有限公司 一种视频投屏方法及装置
CN110086938A (zh) * 2019-04-29 2019-08-02 百度在线网络技术（北京）有限公司 投屏显示的控制方法、系统、电子设备及计算机可读介质
CN110515576B (zh) * 2019-07-08 2021-06-01 华为技术有限公司 显示控制方法及装置
CN115629730A (zh) * 2019-07-23 2023-01-20 华为技术有限公司 显示方法及相关装置

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120040720A1 (en) * 2010-08-13 2012-02-16 Lg Electronics Inc. Mobile terminal, display device and controlling method thereof
US20160054757A1 (en) * 2012-02-29 2016-02-25 Z124 Unified desktop docking flow
US20220164091A1 (en) * 2012-09-10 2022-05-26 Samsung Electronics Co., Ltd. Method for connecting mobile terminal and external display and apparatus implementing the same
US20140298355A1 (en) * 2013-04-01 2014-10-02 Samsung Electronics Co., Ltd. App operating method and device and app output device supporting the same
US20180101199A1 (en) * 2016-10-11 2018-04-12 Samsung Electronics Co., Ltd. Electronic device having a plurality of displays and operating method thereof
US20200019431A1 (en) * 2016-12-23 2020-01-16 Samsung Electronics Co., Ltd. Electronic device and control method thereof
US20190012769A1 (en) * 2017-07-07 2019-01-10 Intelligent Waves Llc System, method and computer program product for remoting orientation changes
US20190296930A1 (en) * 2018-03-20 2019-09-26 Essential Products, Inc. Remote control of an assistant device using an adaptable user interface
US20210042002A1 (en) * 2019-08-06 2021-02-11 Samsung Electronics Co., Ltd. Electronic device and method for sharing data thereof

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
ETA Prime, "Samsung DEX Is Awesome! - Gaming,Emulation,Work and Linux On Dex", (Aug. 10, 2019), URL: https://www.youtube.com/watch?v=zcJmOTAghpM, pg. 1-3 (Year: 2019) *
Lu's Tech Source, "Using Samsung Dex on a PC", (Nov. 10, 2019), URL: https://www.youtube.com/watch?v=cy-3WZV-E04, pg. 1-23 (Year: 2019) *
Lu's Tech Source, "Using Samsung Dex on a PC", (Nov. 10, 2019), URL: https://www.youtube.com/watch?v=cy-3WZV-E04, pg. 1-28 (Year: 2019) *
Reddit, "Closing Apps (DEX Mode)", (2021), <URL: https://www.reddit.com/r/SamsungDex/comments/nvfvck/closing_apps_dex_mode/?sort=old>, pg. 1-2 (Year: 2021) *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230229375A1 (en) * 2020-11-16 2023-07-20 Honor Device Co., Ltd. Electronic Device, Inter-Device Screen Coordination Method, and Medium
US20220197430A1 (en) * 2020-12-23 2022-06-23 Seiko Epson Corporation Image display system, method for controlling image display system, and method for controlling display apparatus
US11934616B2 (en) * 2020-12-23 2024-03-19 Seiko Epson Corporation Image display system, method for controlling image display system, and method for controlling display apparatus
US20230342106A1 (en) * 2020-12-28 2023-10-26 Huawei Technologies Co., Ltd. Collaborative display method, terminal device, and computer-readable storage medium
US12106007B2 (en) * 2020-12-28 2024-10-01 Huawei Technologies Co., Ltd. Collaborative display method, terminal device, and computer-readable storage medium
US12032867B2 (en) * 2021-10-18 2024-07-09 Samsung Electronics Co., Ltd. Source device and sink device for sharing expanded screen, and methods of operating the same
US20230393890A1 (en) * 2022-06-03 2023-12-07 Snap Inc. Auto-recovery for ar wearable devices
US12001878B2 (en) * 2022-06-03 2024-06-04 Snap Inc. Auto-recovery for AR wearable devices
CN117873735A (zh) * 2024-03-11 2024-04-12 湖南马栏山视频先进技术研究院有限公司 一种虚拟化环境下gpu调度系统

Also Published As

Publication number Publication date
CN115793916A (zh) 2023-03-14
EP4040277A4 (en) 2022-12-21
CN113050841A (zh) 2021-06-29
WO2021129253A1 (zh) 2021-07-01
EP4040277A1 (en) 2022-08-10
JP2023503679A (ja) 2023-01-31
US12061833B2 (en) 2024-08-13

Similar Documents

Publication Publication Date Title
US12061833B2 (en) Multi-window display method, electronic device, and system
WO2021227770A1 (zh) 应用窗口显示方法和电子设备
WO2021057830A1 (zh) 一种信息处理方法及电子设备
US20220342850A1 (en) Data transmission method and related device
WO2022052772A1 (zh) 多窗口投屏场景下的应用界面显示方法及电子设备
US11853526B2 (en) Window display method, window switching method, electronic device, and system
WO2021121052A1 (zh) 一种多屏协同方法、系统及电子设备
KR20160045370A (ko) 디바이스 간에 화면 공유 방법 및 이를 이용하는 디바이스
WO2022083296A1 (zh) 一种显示方法及电子设备
CN112527174B (zh) 一种信息处理方法及电子设备
CN112527222A (zh) 一种信息处理方法及电子设备
CN114286165A (zh) 一种显示设备、移动终端及投屏数据传输方法
CN114286152A (zh) 显示设备、通信终端及投屏画面动态显示方法
CN112114733A (zh) 一种截屏、录屏方法、移动终端及计算机存储介质
WO2023005751A1 (zh) 渲染方法及电子设备
CN115904290A (zh) 显示设备、通信终端及投屏控制方法
WO2021052488A1 (zh) 一种信息处理方法及电子设备
WO2021248988A1 (zh) 跨终端录屏方法、终端设备及存储介质
CN111984167B (zh) 一种快捷命名的方法及显示设备
CN116719587B (zh) 屏幕显示方法、电子设备及计算机可读存储介质
CN113079332B (zh) 移动终端及其录屏方法
CN112363683B (zh) 一种网页应用支持多图层显示的方法及显示设备
WO2023284625A1 (zh) 应用的跨平台显示方法、可读介质和电子设备
WO2024001871A1 (zh) 一种操控方法和电子设备
WO2024022137A1 (zh) 一种投屏方法及设备

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

AS Assignment

Owner name: HUAWEI TECHNOLOGIES CO., LTD., CHINA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ZHU, DENGKUI;CHEN, CHEN;XU, HUI;AND OTHERS;SIGNING DATES FROM 20221223 TO 20230817;REEL/FRAME:064643/0622

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

ZAAB Notice of allowance mailed

Free format text: ORIGINAL CODE: MN/=.

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE