WO2023284625A1 - 应用的跨平台显示方法、可读介质和电子设备 - Google Patents

应用的跨平台显示方法、可读介质和电子设备 Download PDF

Info

Publication number
WO2023284625A1
WO2023284625A1 PCT/CN2022/104441 CN2022104441W WO2023284625A1 WO 2023284625 A1 WO2023284625 A1 WO 2023284625A1 CN 2022104441 W CN2022104441 W CN 2022104441W WO 2023284625 A1 WO2023284625 A1 WO 2023284625A1
Authority
WO
WIPO (PCT)
Prior art keywords
application
window
system platform
desktop
mobile phone
Prior art date
Application number
PCT/CN2022/104441
Other languages
English (en)
French (fr)
Inventor
徐亮
王欢
王勇
李英浩
李�杰
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023284625A1 publication Critical patent/WO2023284625A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces

Definitions

  • the present application relates to the field of computer technology, in particular to a cross-platform display method for applications, readable media and electronic equipment.
  • Embodiments of the present application provide a cross-platform display method for an application, a readable medium, and an electronic device.
  • the embodiment of the present application provides a cross-platform display method for applications, including: the first system platform acquires the first size of the first window of the first application running on the second system platform, and A third window corresponding to the first application is created in the platform, wherein the third window is used to display the display interface of the first application running on the second system platform; the first system platform acquires the first window of the second application running on the second system platform second size of the second window, and create a fourth window corresponding to the second application on the first system platform, wherein the fourth window is used to display the display interface of the second application running on the second system platform.
  • the first system platform may be the second operating system running on the second electronic device hereinafter, for example, system, also known as a desktop system; the second system platform may be a second operating system running on the first electronic device hereinafter, for example, system, also known as the Android system; the second system platform may also be the second operating system running on the virtual machine on the third electronic device hereinafter, for example, system, also known as the Android system.
  • the second electronic device and the third electronic device may be computers, and the first electronic device may be a mobile phone.
  • the first window of the first application may be the Android application window of the "calculator” application below.
  • the third window of the first application may be a desktop application window of the "Calculator” application below.
  • the first size of the first window of the first application may be the parameters of the Android application window of the "calculator” application hereinafter, wherein the parameters of the Android application window include but are not limited to the identification of the Android application window and the size of the Android application window And window area position coordinates, etc.
  • the second window of the second application may be the Android application window of the "Settings” application below.
  • the fourth window of the second application may be the desktop application window of the "Settings” application below.
  • the second size of the second window of the second application may be a parameter of the Android application window of the "calculator” application hereinafter.
  • the mobile phone can send the parameters of the Android application window of "calculator" to the computer, wherein the parameters of the Android application window include the size of the Android application window and the coordinates of the window area position;
  • the parameters of the window create a desktop application window and send the window handle of the desktop application window to the mobile phone, where the window handle of the desktop application window is used to uniquely identify the desktop application window;
  • the mobile phone uses the received window handle to generate drawing information and send the drawing information to the computer
  • the computer can call the graphics library of the computer to render the desktop application window based on the received drawing information; as shown in Figure 1B below, the desktop application window of the "calculator” application displays the content of the Android application "calculator". It can be understood that according to the above method, as shown in FIG. 1C below, the computer can start multiple Android applications, create multiple Android application windows and simultaneously display the contents of multiple Android applications.
  • the above method further includes: when the first system platform detects that the user opens the first application or the second application on the first system platform, from the second system platform The platform captures the first or second size.
  • the computer can simultaneously display the Android application of the mobile phone.
  • the computer can create a shortcut to the Android application on the desktop of the computer so that the computer can display the Android application synchronously.
  • the computer can obtain the parameters of the started Android application by detecting the user's operation of creating a shortcut to the Android application on the desktop of the computer.
  • the computer can also create a display button of an Android application on the display interface.
  • the computer can also obtain the parameters of the Android application by detecting the operation of the display button of the Android application created on the display window of the computer opened by the user.
  • the mobile phone can start the Android application based on the parameters of the Android application. Wherein, in the process of starting the Android application, the mobile phone will create the Android application window of the application, and the mobile phone will send the parameters of the Android application window to the computer.
  • the above method further includes: the second system platform does not display the first window and the second window.
  • the Android application can be started on the virtual screen of the mobile phone, and can also be started on the display screen of the mobile phone.
  • the computer can start the Android application, and the displayed content of the started Android application displayed on the computer.
  • the content of the Android application "Calculator" is only displayed on the computer. Therefore, the distributed application display of the cross-platform application can be realized without affecting the normal use of the mobile phone.
  • the display screen of the mobile phone may not display the Android application window of the "Video Player” application, and the desktop application window executes the drawing information of the "Video Player” application to achieve video image rendering and consumes computer resources.
  • the Android application window does not execute this command so as not to consume the GPU resources of the mobile phone, so that the display interface of the mobile phone application can be displayed on the computer without consuming the GPU resources of the mobile phone.
  • the above method further includes: creating a third window corresponding to the first application on the first system platform, or creating a fourth window corresponding to the second application on the first system platform , including: the first system platform sets a first identifier for the first application, or the first system platform sets a second identifier for the second application; wherein the first identifier is used to identify the display interface of the third window for displaying the first application, The second identifier is used to identify the display interface of the fourth window for displaying the second application.
  • the first identifier may be the window handle of the desktop application window of the "calculator” application
  • the second identifier may be the window handle of the desktop application window of the "settings” application.
  • the window handle of the desktop application window of the "Calculator” application is used to uniquely identify the desktop application window of the "Calculator” application
  • the window handle of the desktop application window of the "Settings” application is used to uniquely identify the desktop application window of the "Settings” application window.
  • the computer creates a desktop application window based on the parameters of the Android application window, and assigns a window handle to the created desktop application window.
  • the allocated window handle is used to uniquely identify the created desktop application window. For example, the computer creates 10 desktop application windows, and the computer assigns a window handle to each desktop application window; operate.
  • the foregoing method further includes: the first system platform sending the first identification or the second identification to the second system platform.
  • the mobile phone continues to create an Android application window, and associates the Android application window with the window handle of the desktop application window.
  • the Android application window is associated with the window handle of the desktop application window, and the mobile phone can use the drawing information generated by the window handle, so that the generated drawing information can be directly executed on the computer.
  • the above method further includes: the first system platform displays the first application running on the second system platform in the third window in the following manner: The platform receives the first drawing information, wherein the first drawing information includes a first identifier; the first system platform displays in a third window the drawing drawn according to the first drawing information according to the first identifier included in the first drawing information, the first The first display interface of the application.
  • the first drawing information can be the drawing information of the "calculator” application
  • the computer can call the image library of the computer according to the received drawing information of the "calculator” application, such as EGL, OpenGL, etc., and the first display interface can be called by the computer.
  • the image library performs image rendering, and the acquired display interface of the "calculator” application is displayed on the desktop application window of the "calculator".
  • the mobile phone only needs to send the parameters and drawing information of the Android application window to the computer, and the computer creates a desktop application window based on the received parameters and drawing information of the Android application window and calls the graphics library to draw the window display content, so as to realize the startup on the computer and Displays the content of an Android application.
  • the application distributed display of this application does not need to transmit a large amount of video image data between devices to achieve distributed display.
  • the above method further includes: the first system platform displays the second application running on the second system platform in the fourth window in the following manner: The platform receives the second drawing information, wherein the second drawing information includes the second identification; the second system platform displays in the fourth window the drawing drawn according to the second drawing information according to the second identification included in the second drawing information, and the second The second display interface of the application.
  • the second drawing information may be the drawing information of the "Settings” application
  • the computer may call the image library of the computer according to the received drawing information of the "Settings” application, such as EGL, OpenGL, etc.
  • the second display interface may be that the computer calls the image library Image rendering is performed to obtain a display interface for displaying the "Calculator" application on the desktop application window of the "Settings".
  • the above method further includes: the first system platform detects the user's first operation on the third window on the first system platform; the first system platform generates operation information, and sends The operation information is sent to the second system platform, wherein the operation information includes the first identification; the first system platform receives third drawing information from the second system platform, wherein the third drawing information includes the first identification, and the third drawing information is the first
  • the second system platform is generated by performing the operation corresponding to the operation information; the first system platform displays the third display interface in the third window based on the third drawing information, wherein the third display interface is changed after the first display interface corresponds to the first operation display interface.
  • the display content in the desktop application window can be the numbers or arithmetic symbols in the desktop application window of the "Calculator” application;
  • the status of the desktop application window of the "Calculator” application can be open, closed, minimized, etc.
  • the computer creates a desktop application window based on the received parameters of the Android application window and drawing information, and calls the graphics library to draw the window display content, thereby realizing the display of the desktop application window of the "calculator" application in Figure 1B. Therefore, the absolute coordinates of the window area of the Android application window and the absolute coordinates of the window area of the desktop application window have a mapping relationship, that is, the mobile phone can redraw the content displayed in the Android application window based on the operation position coordinates in the operation information to generate an operation drawing information.
  • the third drawing information may be operation drawing information of the "calculator” application below.
  • the desktop system of the computer detects the “zoom in” operation of the desktop application window, and the desktop system of the computer generates the window area position coordinates of the "zoom in” operation
  • the desktop system sends the “magnified” control operation information to the mobile phone agent process of the mobile phone;
  • the mobile phone agent process of the mobile phone adjusts the Android application by calling the resize Task interface of the ActivityTaskManagerService based on the received window area position coordinates and the operation position coordinates The position coordinates of the window.
  • the Android system of the mobile phone adjusts the area coordinates of the Android application window of the "calculator” application, and generates operation drawing information.
  • the operation drawing information is used to draw the content displayed in the enlarged desktop application window of the "Calculator” application.
  • the third drawing information may also be operation drawing information of the "settings" application below.
  • the Android system of the mobile phone detects the operation on setting option 4 based on the area position coordinates of the button icon of "setting option 4".
  • the Android system of the mobile phone performs a control operation on the task corresponding to the task identifier associated with the Android application window of the "Settings" application by calling the InputManagerService service interface, and generates operation drawing information.
  • set option 4 to Airplane Mode and the button icon is the status switch button of Airplane Mode.
  • the Android system of the mobile phone turns on the flight mode by calling the InputManagerService service interface, and generates operation drawing information, and the generated operation drawing information is used to draw the display window after the flight mode is turned on.
  • the above method further includes: the first operation includes at least one of the following operations: an operation of changing the size of the first display interface; an operation of controls in the first display interface .
  • the user's first operation on the third window on the first system platform may be the display content (for example, controls of the display interface) on the desktop application window of the "calculator" application by the user through the mouse, keyboard and touch, etc.
  • Window status and window size control operations may include the size of the desktop application window, the location coordinates of the window area, the operation location coordinates, and the like.
  • the foregoing method further includes: the first system platform and the second system platform have different operating systems.
  • the foregoing method further includes: the first system platform and the second system platform run on different electronic devices.
  • the foregoing method further includes: the first system platform is an operating system on a virtual machine running on the electronic device, and the second system platform is an operating system of the electronic device.
  • the implementation of this application provides a cross-platform display method for applications, which is characterized in that it includes: the first application runs on the second system platform; the first system platform acquires the first application's information from the second system platform The first size of the first window; the first system platform creates a third window corresponding to the first application on the first system platform and a first identifier, wherein the first identifier is used to identify the third window for displaying the first application interface; the second system platform generates first drawing information after receiving the first identification from the first system platform, wherein the first drawing information includes the first identification; the first system platform receives the first drawing information from the second system platform, and According to the first identification included in the first drawing information, the first display interface of the first application drawn according to the first drawing information is displayed in the third window.
  • the above method further includes: when the first system platform detects that the user opens the first application in the first system platform, the first system platform sends the first application information to to the second system platform; the second system platform runs the first application according to the first application information, and creates the first window of the first application in the second system platform; the second system platform uses the first window of the first application A dimension is sent to the first system platform.
  • the above method further includes: the second system platform does not display the first window.
  • the above method further includes: the first system platform detects the user's first operation on the third window on the first system platform; the first system platform generates operation information, and sends The operation information is sent to the second system platform, wherein the operation information includes the first identification; the second system platform generates third drawing information by performing the operation corresponding to the operation information, and sends the third drawing information to the first system platform; the first The system platform displays a third display interface in the third window based on the third drawing information, wherein the third display interface is a display interface changed from the first display interface corresponding to the first operation.
  • the above method further includes: the first operation includes at least one of the following operations: an operation of changing the size of the first display interface; an operation of controls in the first display interface .
  • the above method further includes: the first system platform and the second system platform have different operating systems, wherein the first system platform and the second system include at least one of the following: The first system platform and the second system platform run on different electronic devices; the first system platform is an operating system running on a virtual machine on the electronic device, and the second system platform is an operating system of the electronic device.
  • the implementation of this application provides a cross-platform display method for applications, which is characterized in that it includes: the first application runs on the second system platform; the second system platform sends the first application information to the first system platform The first size of the first window; after the second system platform receives the first identification from the first system platform, it generates the first drawing information, wherein the first identification is used for identification: the third window in the first system platform is used for displaying The display interface of the first application, and the first drawing information includes the first identifier, and the first drawing information can be used by the first system platform to draw the first display interface of the first application displayed in the third window.
  • the embodiments of the present application provide a readable medium, on which instructions are stored, and when the instructions are executed on the electronic equipment, the electronic equipment executes the above-mentioned first to third aspects and the first to third aspects.
  • the embodiment of the present application provides an electronic device, including:
  • memory for storing instructions to be executed by one or more processors of the electronic device
  • the processor is one of the processors of the electronic device, configured to execute the cross-platform display method applied in any one of the above first to third aspects and various possible implementations of the first to third aspects.
  • FIG. 1A shows a schematic diagram of establishing a distributed display connection between a mobile phone and a computer according to some embodiments of the present application
  • Fig. 1B shows a schematic diagram of a distributed display of a "settings" application according to some embodiments of the present application
  • Fig. 1C shows a schematic diagram of a distributed display of multiple applications according to some embodiments of the present application
  • Fig. 1D shows a schematic diagram of "zooming in” a display window of an application according to some embodiments of the present application
  • Fig. 2 shows a scene diagram of an application cross-platform display system according to some embodiments of the present application
  • Fig. 3 shows a cross-platform application distributed display interaction diagram of an application cross-platform display system according to some embodiments of the present application
  • Fig. 4 shows another cross-platform application distributed display interaction diagram of a cross-platform display system according to some embodiments of the present application
  • Fig. 5 shows a schematic diagram of a monitoring function for starting a mobile phone agent process according to some embodiments of the present application
  • Fig. 6 shows a schematic diagram of a monitoring function for starting a desktop agent process according to some embodiments of the present application
  • Fig. 7 shows a schematic diagram of successfully establishing a distributed display connection between a mobile phone and a computer according to some embodiments of the present application
  • Fig. 8 shows a schematic diagram of displaying an Android application on a computer display interface according to some embodiments of the present application
  • Fig. 9 shows another schematic diagram of displaying an Android application on a computer display interface according to some embodiments of the present application.
  • Fig. 10 shows a schematic diagram of distributed display of a "video player" application according to some embodiments of the present application
  • Fig. 11 shows an interaction diagram of performing a "zoom in” operation on a desktop application window of an Android application according to some embodiments of the present application
  • Fig. 12 shows a schematic diagram of before and after resizing of a desktop application window of a "calculator" application according to some embodiments of the present application
  • Fig. 13A shows a schematic display of a desktop application window of the "Settings" application before adjustment according to some embodiments of the present application
  • Fig. 13B shows a schematic display of an adjusted desktop application window of the "Settings" application according to some embodiments of the present application
  • Fig. 14 shows an interactive diagram for controlling the display content of the desktop application window of the Android application according to some embodiments of the present application
  • Fig. 15 shows another scene diagram of distributed display of cross-platform applications according to some embodiments of the present application.
  • Fig. 16 shows another cross-platform distributed display interaction diagram according to some embodiments of the present application.
  • Fig. 17 shows another schematic diagram of the monitoring function for starting a desktop agent process according to some embodiments of the present application.
  • Fig. 18 shows a schematic diagram of successfully establishing a distributed display connection between a virtual machine and a computer according to some embodiments of the present application
  • Fig. 19 shows a schematic diagram of distributed display of another "video player" application according to some embodiments of the present application.
  • Fig. 20 shows a schematic structural diagram of an electronic device according to some embodiments of the present application.
  • Fig. 21 shows a software structure block diagram of an electronic device according to some embodiments of the present application.
  • Fig. 22 shows a schematic structural diagram of another electronic device according to some embodiments of the present application.
  • Fig. 23 shows a software structural block diagram of another electronic device according to some embodiments of the present application.
  • Illustrative embodiments of the present application include, but are not limited to, cross-platform display methods of applications, readable media, and electronic devices.
  • the present application provides a cross-platform display method for applications, which can be applied to two electronic devices installed with different operating systems, and can also be applied to the situation where two operating systems are installed on one device.
  • the computer 200 can start the Android application "Calculator” by sending the parameters of the Android application “Calculator” to the mobile phone 100;
  • the parameters generate the Android application window of "calculator", and the mobile phone 100 can send the parameters of the Android application window of "Calculator” to the computer 200, wherein the parameters of the Android application window include the size of the Android application window and the window area position coordinates;
  • 200 creates a desktop application window based on the received parameters of the Android application window of "calculator” and sends the window handle of the desktop application window to the mobile phone 100, wherein the window handle of the desktop application window is used to uniquely identify the desktop application window;
  • the mobile phone 100 uses the received
  • the window handle generates the drawing information and sends the drawing information to the computer 200;
  • the computer 200 can call the graphics library of the computer 200 to render the desktop application window based on the received drawing information; as shown
  • the computer 200 can also perform control operations on the displayed content or the size of the window in the desktop application window 2003-2.
  • the computer 200 "zooms in" the desktop application window 2003-2 to generate operation information; the computer 200 can send the operation information to the mobile phone 100, wherein the operation information can include the size of the desktop application window, the position coordinates of the window area, and the operation information. location coordinates, etc.; the mobile phone 100 can adjust the Android application window of the "calculator” based on the operation information, and generate operation drawing information; the mobile phone 100 sends the generated operation drawing information to the computer 200; the computer 200 calls the graphics of the computer 200 based on the operation drawing information
  • the functions in the library perform image rendering on the desktop application window; as shown in FIG. 1D , so as to realize the "magnification" of the desktop application window 2003-2.
  • FIG. 2 shows a scene diagram of a cross-platform display system 20 for an application according to some embodiments of the present application.
  • the application cross-platform display system 20 includes a first electronic device 100 and a second electronic device 200 ; the first electronic device 100 can be established with the second electronic device 200 .
  • the first electronic device 100 can run the first operating system 11 ; the second electronic device 200 can run the second operating system 21 .
  • the display interface 110 of the first electronic device 100 may be a user operation interface of the first operating system 11 .
  • the display interface 210 of the second electronic device 200 may be a user operation interface of the second operating system 21 .
  • the first operating system 11 may be a Hongmeng operating system (harmony operating system, HarmonyOS), an Apple operating system (iPhone operating system, iOS), operating system, phone operating system, operating system, operating system etc.
  • the second operating system 21 can be operating system, Operating system, MAC operating system (operation system, OS), etc.
  • the first electronic device 100 may communicate with the second electronic device 200 through wireless communication methods such as Bluetooth, WIFI, or NFC.
  • the first electronic device 100 may also communicate with the second electronic device 200 through wired communication, for example, the first electronic device 100 communicates with the second electronic device 200 through a data line and a Universal Serial Bus (Universal Serial Bus, USB) interface connect.
  • USB Universal Serial Bus
  • the first electronic device 100 and the second electronic device 200 may be various computing devices capable of communicating with each other, for example, the first electronic device 100 or the second electronic device 200 may include, but not limited to, a laptop Computers, desktop computers, tablet computers, mobile phones, servers, wearable devices, head-mounted displays, mobile email devices, in-vehicle devices, portable game consoles, portable music players, reader devices, embedded or coupled to a or multi-processor televisions, or other electronic devices capable of accessing the Internet.
  • a laptop Computers desktop computers, tablet computers, mobile phones, servers, wearable devices, head-mounted displays, mobile email devices, in-vehicle devices, portable game consoles, portable music players, reader devices, embedded or coupled to a or multi-processor televisions, or other electronic devices capable of accessing the Internet.
  • the first electronic device 100 is the mobile phone 100
  • the first operating system is operating system
  • the second electronic device 200 is a computer 200
  • the second operating system is The operating system is taken as an example to illustrate the solution of this application.
  • the operating system is referred to as the Android system
  • the operating system is referred to as the desktop system.
  • FIG. 3 shows an interaction diagram of a cross-platform application distributed display system 20 of an application cross-platform display system 20 according to some embodiments of the present application.
  • the process of the cross-platform display method of the application cross-platform display system 20 includes the following steps:
  • S301 The mobile phone 100 establishes a distributed display connection with the computer 200.
  • the computer 200 can learn the verification code of the mobile phone 100, and verify the verification code of the mobile phone 100, thereby realizing the establishment of a distribution network with the mobile phone 100. to display the connection.
  • the verification code of the mobile phone 100 may be the IP address of the mobile phone 100 , or a random code generated by the mobile phone 100 . The process of verifying the verification code of the mobile phone 100 by the computer 200 will be described in detail below, and will not be repeated here.
  • S302 The computer 200 detects the operation of starting the Android application, and obtains parameters of the Android application of the started Android application.
  • the parameters of the Android application include a package name and a class name of the Android application
  • the mobile phone 100 can open the Android application according to the parameters of the Android application.
  • the computer 200 may detect the user's operation of opening the Android application on the computer 200, and the computer 200 generates an instruction to open the Android application.
  • the computer 200 can simultaneously display the Android application of the mobile phone 100 .
  • the computer 200 can create a shortcut of the Android application on the desktop of the computer 200 so that the computer 200 can display the Android application synchronously.
  • the computer 200 can obtain the parameters of the started Android application by detecting the user's operation of creating a shortcut to the Android application on the desktop of the computer 200 .
  • the computer 200 may create a display button of the Android application on the display interface.
  • the computer 200 can also obtain the parameters of the Android application by detecting the user's operation of opening the display button of the Android application created on the display window of the computer 200 .
  • S303 The computer 200 sends the parameters of the Android application to the mobile phone 100.
  • S304 The mobile phone 100 starts the application based on the parameters of the Android application, and creates an Android application window.
  • the mobile phone 100 can start the Android application based on the parameters of the Android application. Wherein, in the process of starting the Android application, the mobile phone 100 will create the Android application window of the application. During the process of creating the Android application window, the mobile phone 100 first sends the parameters of the Android application window to the computer 200 . The computer 200 can create a desktop application window based on the parameters of the Android application window. Wherein, the parameters of the Android application window include but not limited to the identification of the Android application window, the size of the Android application window, and the location coordinates of the window area.
  • the Android application can be started on the virtual screen of the mobile phone 100, and can also be started on the display screen of the mobile phone 100.
  • the computer 200 can start the Android application. application, and the displayed content of starting the Android application is only displayed on the computer 200 .
  • the content of the Android application “calculator” is only displayed on the computer 200 . In this way, cross-platform application and distributed application display can be realized without affecting the normal use of the mobile phone 100 .
  • S305 The mobile phone 100 sends the parameters of the Android application window to the computer 200.
  • S306 The computer 200 creates a desktop application window based on the parameters of the Android application window.
  • the computer 200 creates a desktop application window based on parameters of the Android application window, and assigns a window handle to the created desktop application window.
  • the allocated window handle is used to uniquely identify the created desktop application window.
  • the computer 200 creates 10 desktop application windows, and the computer 200 assigns a window handle to each desktop application window; application window.
  • S307 The computer 200 sends the window handle of the desktop application window to the mobile phone 100.
  • S308 The mobile phone 100 continues to create the Android application window, and associates the Android application window with the window handle of the desktop application window.
  • the mobile phone 100 after receiving the window handle of the desktop application window, the mobile phone 100 continues to create an Android application window, and associates the Android application window with the window handle of the desktop application window.
  • the Android application window is associated with the window handle of the desktop application window, and the mobile phone 100 can use the drawing information generated by the window handle, so that the generated drawing information can be directly displayed on the computer 200. implement.
  • S309 The mobile phone 100 uses the window handle to create drawing information.
  • the drawing information is used to call an embedded graphics library (Embedded-System Graphics Library, EGL), an open graphics library (Open Graphics Library, OpenGL), etc. to draw the display content of the Android application.
  • EGL embedded-System Graphics Library
  • OpenGL Open Graphics Library
  • the drawing information generated by the mobile phone 100 can be executed on the mobile phone 100 to realize the image rendering of the Android application window, and complete the display of the display content of the Android application on the desktop application window.
  • the drawing information generated by the mobile phone 100 may not be executed on the mobile phone 100 , so as not to consume the GPU performance of the mobile phone 100 without affecting the normal use of the mobile phone 100 . This will be described in detail below, and will not be repeated here.
  • the drawing information is used to call the embedded graphics library (Embedded-System Graphics Library, EGL) or the open image library (Open Graphics Library, OpenGL) to draw the display content of the Android application, wherein, the mobile phone 100 sends the drawing information to the computer 200 , the computer 200 executes the drawing information to realize the display of the rendered image on the desktop application window of the computer 200, thereby realizing the display of the Android application on the desktop application window of the computer 200. Specifically, the mobile phone 100 sends the drawing information to the computer 200, and the computer 200 executes the drawing information to use OpenGL to draw the image.
  • EGL embedded-System Graphics Library
  • OpenGL Open Graphics Library
  • the mobile phone 100 creates a rendering surface, that is, when EGLSurface uses an interface, in the option of NativeWindowType, the mobile phone 100 replaces the window handle (for example, native_window) sent by the computer 200 with the window handle of the Android application window, thereby realizing the drawing information created by the mobile phone 100 It can be executed on the computer 200, and when the computer 200 opens multiple desktop application windows of the Android application, the desktop application window that needs to redraw the image is determined according to the window handle information contained in the drawing information.
  • the window handle for example, native_window
  • the computer 200 when the computer 200 finishes drawing using OpenGL, use the eglSwapBuffers method to exchange front and rear buffers, and display the drawing content on the display screen of the computer 200; Determined, destroy EGLDisplay, EGLSurface, EGLContext, and release the EGL environment.
  • S310 The mobile phone 100 sends drawing information to the computer 200 .
  • S311 The computer 200 executes the drawing information to display the content of the Android application on the desktop application window.
  • the computer 200 calls the image library of the computer 200 according to the drawing information, such as EGL, OpenGL, etc., to implement image rendering on the desktop application window, and complete the display of the display content of the Android application on the desktop application window.
  • drawing information such as EGL, OpenGL, etc.
  • step S301 to step S311 it can be seen that the mobile phone 100 only needs to send the parameters and drawing information of the Android application window to the computer 200, and the computer 200 creates a desktop application window based on the received parameters and drawing information of the Android application window and calls the graphics library to draw the window display content , so as to start and display the content of the Android application on the computer 200 .
  • the application distributed display of this application does not need to transmit a large amount of video image data between devices to achieve distributed display.
  • S312 The computer 200 detects the control operation of the desktop application window, and generates operation information.
  • the computer 200 detects the user's control operation on the desktop application window, and generates operation information.
  • the user controls and operates the display content (for example, a control of a display interface), window state, window size, etc. in the desktop application window by using a mouse, a keyboard, and touch.
  • the operation information may include the size of the desktop application window, the location coordinates of the window area, the operation location coordinates, and the like.
  • the display content in the desktop application window (for example, the control of the display interface) can be the numbers or operation symbols in the desktop application window 2003-2;
  • the desktop application window size can be the desktop application window 2003- 2
  • the position of the display interface 210, the size of the display area of the desktop application window 2003-2, etc.; the status of the desktop application window 2003-2 can be open, closed, minimized, etc.
  • S313 The computer 200 sends the operation information to the mobile phone 100.
  • S314 The mobile phone 100 generates operation drawing information based on the operation information.
  • the operation information sent by the computer 200 to the mobile phone 100 includes the task identification of the Android application window, and through the task identification, the mobile phone 100 can determine that the image needs to be redrawn during the multi-window display shown in FIG. 1C Android application window.
  • the computer 200 creates a desktop application window based on the received Android application window parameters and drawing information, and calls a graphics library to draw the window display content, thereby realizing the display of the desktop application window 2003-2 in FIG. 1B . Therefore, the absolute coordinates of the window area of the Android application window and the absolute coordinates of the window area of the desktop application window have a mapping relationship, that is, the mobile phone 100 can redraw the content displayed in the Android application window based on the operation position coordinates in the operation information to generate an operation Drawing information.
  • the content of drawing information generated in step S310 please refer to the content of drawing information generated in step S310, which will not be repeated here.
  • S315 The mobile phone 100 sends the operation drawing information to the computer 200.
  • the computer 200 creates a desktop application window based on the received parameters of the Android application window and drawing information, and calls the graphics library to draw the window display content, thereby realizing the display of the desktop application window 2003-2 in FIG. 1B.
  • the mobile phone 100 can start and run the "Settings" application on the virtual screen of the mobile phone 100, so as to realize distributed application display of cross-platform applications without affecting the use of the mobile phone 100 .
  • the computer 200 detects that the control operation of the desktop application window generates operation information, the mobile phone 100 generates operation drawing information based on the operation information generated by the computer 200, and the computer 200 redraws the desktop application window based on the operation drawing information, and the desktop application window Displays the content of the Android application after the control operation.
  • the user can operate the numbers or arithmetic symbols in the desktop application window 2003-2 shown in FIG. position, the desktop application window 2003-2 shown in FIG. 1B can also be enlarged, reduced, closed, and minimized.
  • the computer 200 may include the desktop agent process 201, and the mobile phone 100 may include the mobile phone agent process 101;
  • FIG. 4 shows another cross-platform application distributed display interaction of the cross-platform display system 20 picture.
  • the specific process of cross-platform application distributed display is as follows:
  • S401 The mobile phone 100 detects an operation of starting monitoring in the mobile phone proxy process 101.
  • the mobile phone agent process 101 is used to provide the IP address of the mobile phone 100, and the mobile phone 100 opens the monitoring function through the mobile phone agent process 101; Establish connection with Android 11.
  • the mobile phone proxy process 101 can be an application installed on the mobile phone 100.
  • the display interface of the mobile phone proxy process 101 can display a button for closing monitoring, a button for starting monitoring, and a button of the mobile phone 100. IP address etc.
  • the user can start the mobile phone agent process 101 by clicking the "mobile phone agent” icon on the display interface 101 of the mobile phone 100; after starting the mobile phone agent process 101, as shown in Figure 5 (b), The user can start the monitoring function of the mobile agent process 101 by clicking the "start monitoring" button on the display interface 102 .
  • the user can also check the system notification of the mobile phone 100 to determine whether the monitoring function of the mobile phone agent process 101 has been turned on. For example, as shown in FIG. 5( d), when the monitoring function of the mobile phone agent process 101 is turned on, a system notification window 103 will appear.
  • the listening address 10.1.1.1 in the display interface 102 is the IP address of the mobile phone 100
  • the desktop agent process 201 of the computer 200 can be based on the IP address of the mobile phone 100 and the mobile phone agent process 101 of the mobile phone 100
  • the IP address of the mobile phone 100 may also be set to 11.1.1.1. It should be noted that, in the specific implementation process, the present application does not specifically limit the specific content of the IP address of the mobile phone 100 .
  • the user can also start the mobile phone agent process 101 of the mobile phone 100 in other ways, and enable the monitoring function of the mobile phone agent process 101, which is not specifically limited in this application according to actual application conditions.
  • the computer 200 detects that the desktop agent process 201 of the computer 200 starts a monitoring operation.
  • the desktop proxy process 201 can be used to input the IP address of the mobile phone 100, the desktop proxy process 201 can also be used to provide the communication connection mode between the mobile phone 100 and the computer 200, and the desktop proxy process 201 can also be used to Turn on the monitoring function of the computer 200 .
  • the desktop proxy process 201 can be software installed on the computer 200, and the desktop proxy process 201 is opened, and the display interface 220 of the desktop proxy process 201 can display an input box for inputting the IP address of the mobile phone 100,
  • the display interface 220 of the desktop agent process 201 can also be used to display the communication connection mode between the mobile phone 100 and the computer 200, for example, a USB connection or a wifi connection; the display interface 220 of the desktop agent process 201 can also display a button for starting monitoring, and the desktop agent process 201 can also be used to display whether the connection between the desktop system 21 and the Android system 11 is successful.
  • the user can start the desktop agent process 201 by clicking the shortcut icon 211 of the desktop agent on the display interface 210 of the computer 200; after starting the desktop agent process 201, as shown in FIG.
  • the user can input the IP address of the mobile phone 100 on the "input box" 221, and then click the "Connect” button 222 to start the monitoring function of the desktop proxy process 201.
  • the user can also start the desktop proxy process 201 of the computer 200 in other ways, and enable the monitoring function of the desktop proxy process 201; this application does not specifically limit this according to actual application conditions.
  • S403 Send/return a request between the mobile agent process 101 of the mobile phone 100 and the desktop agent process 201 of the computer 200, and establish a distributed display connection.
  • the mobile phone 100 and the computer 200 may be wired or wirelessly connected to a network.
  • the user can open the monitoring function of the mobile phone agent process 101 of the mobile phone 100 and obtain the IP address of the mobile phone 100; then, the user inputs the IP address of the mobile phone 100 on the desktop system 21 and clicks "Connect " button, after the desktop proxy process 201 learned the IP address of the mobile phone 100, the desktop proxy process 201 of the computer 200 and the mobile phone proxy process 101 of the mobile phone 100 sent/returned a request, wherein the request sent/returned could be a socket request (socket request) .
  • the socket request is used for two-way communication between different electronic devices in a wireless network or a wired network, and between applications of different electronic devices.
  • the mobile agent process 101 of the mobile phone 100 and the desktop agent process 201 of the computer 200 send/return requests to establish a distributed display connection.
  • the display interface 210 will display the "connection successful" icon 224, that is, the desktop proxy process 201 successfully opens the monitoring function;
  • the mobile phone proxy process 101 of 100 fails to establish a distributed display connection with the desktop proxy process 201 of the computer 200, and the display interface 210 will display a "connection failed" icon, that is, the desktop proxy process 201 fails to open the monitoring function.
  • the user inputs the IP address of the mobile phone 100 on the display interface of the computer 200 .
  • the desktop agent process 201 of the computer 200 obtains the IP address of the mobile phone 100 and sends a socket request to the mobile phone agent process 101 of the mobile phone 100 based on the IP address of the mobile phone 100 .
  • the user may also input the verification code of the mobile phone 100 on the display interface of the computer 200 .
  • the desktop agent process 201 of the computer 200 obtains the verification code of the mobile phone 100 , and sends a socket request to the mobile phone agent process 101 of the mobile phone 100 based on the verification code of the mobile phone 100 .
  • the verification code of the mobile phone 100 may be the IP address of the mobile phone 100 or a random code generated by the mobile phone 100 . According to the actual application, this application does not specifically limit the type of the verification code of the mobile phone 100 obtained by the computer 200 .
  • the computer 200 detects the operation of starting the “Settings” application of the mobile phone 100 on the desktop agent process 201 .
  • the following description takes the user starting the “Settings” application of the mobile phone 100 on the desktop agent process 201 as an example. It can be understood that other applications on the mobile phone 100 can also be started on the desktop proxy process 201, and the specific process of the startup process can refer to the process of starting the "settings” application of the mobile phone 100 on the desktop proxy process 201, which will not be repeated here.
  • an application button that can start the mobile phone 100 can be created on the display interface 220 of the desktop agent process 201 of the computer 200, and the user clicks the application button on the display interface 220 of the desktop agent process 201 of the computer 200 Start the application of the mobile phone 100; and, if there is no application button of the mobile phone 100 on the display interface 220 of the computer 200, the user can also input the application package name and/or class name of the mobile phone 100 in the input box of the display interface 220 of the computer 200 Start the application of the mobile phone 100 .
  • the user starts the “Settings” application of the mobile phone 100 by clicking the “Settings” button 225 on the display interface 220 of the desktop agent process 201 of the computer 200 .
  • the user can also enter the package name and class name respectively in the "input box” 226 and/or "input box” 227 of the computer 200 and click the "start” button 227 to open the application of the mobile phone 100.
  • the user can also enter the package name com.android.settings of the “settings” application of the mobile phone 100 in the “input box” 22 of the display interface 220, and then click the “start” button 228 to start the “settings” application of the mobile phone 100.
  • a desktop shortcut of the application of the mobile phone 100 may also be created on the display interface 210 of the computer 200 , and the user may open the application of the mobile phone 100 by clicking the desktop shortcut icon. For example, as shown in FIG. 9 , the user starts the “Settings” application of the mobile phone 100 by clicking the desktop shortcut icon 212 of the “Settings” application on the display interface 210 .
  • the process for the computer 200 to create a desktop shortcut for the application of the mobile phone 100 on the display interface 210 may include: the desktop agent process 201 of the computer 200 sends an application query instruction to the mobile phone agent process 101 of the mobile phone 100; After the mobile phone agent process 101 of the mobile phone 100 receives the application query instruction, the mobile phone proxy process 101 of the mobile phone 100 obtains all installed application package names on the mobile phone 100 by calling the getAllPackages interface of the PackageManagerService service of the Android system 11, and then the mobile phone agent process 101 of the mobile phone 100 Obtain the applicationPackageManager through getPackageManager(), then query the application package name through the getApplicationIcon interface of the applicationPackageManager, and obtain the application icon according to the application package name; the mobile phone agent process 101 of the mobile phone 100 traverses all application package names, and obtains the icon of each application on the mobile phone 100; The mobile phone agent process 101 sends the parameters of the icon of each application on the mobile phone 100 to the desktop agent process 201 of the computer 200; Create
  • S405 The desktop agent process 201 of the computer 200 sends the parameters of the started application of the mobile phone 100 to the mobile phone agent process 101.
  • the desktop agent process 201 of the computer 200 sends the parameters of the Android application to be started to the mobile phone agent process 101 of the mobile phone 100, and the mobile phone agent process 101 of the mobile phone 100 can open the application of the mobile phone 100 through the parameters of the application. In this way, the user can start the application of the mobile phone 100 on the computer 200, thereby making the user experience more convenient.
  • the parameters of the Android application include the package name of the application and the class name of the application.
  • the user starts the "settings" application of the mobile phone 100 on the desktop agent process 201, and the desktop agent process 201 may send the package name com.android.settings of the "settings" application to the mobile phone agent process 101.
  • S406 The mobile phone agent process 101 of the mobile phone 100 calls the interface of the Android system 11, and starts the “settings” application of the mobile phone 100.
  • the mobile phone proxy process 101 invokes the startActivity interface of the ActivityTaskManagerService of the Android system 11 to start the "settings" application of the mobile phone 100 according to the received parameters of the "settings” application.
  • S407 The Android system 11 of the mobile phone 100 creates an Android application window of the "Settings" application.
  • the agent process 1002 of the mobile phone 100 calls the interface of the Android system 11 to start the "Settings” application of the mobile phone 100; during the startup process of the "Settings” application, the Android system 11 will create the “Settings” application Android application window and send the parameters of the Android application window of the created "settings” application to the mobile phone agent process 101 of the mobile phone 100.
  • the “Settings” application of the mobile phone 100 can be launched on the virtual screen of the mobile phone 100 , or can be launched on the display screen of the mobile phone 100 .
  • the Android system 11 of the mobile phone 100 specifies the startup parameter android.activity.windowingMode as the freeform mode, and the Android system 11 of the mobile phone 100 specifies to start in the FLAG_ACTIVITY_NEW_TASK mode, that is, the Android system 11 of the mobile phone 100 is created with " Tasks associated with the Android app window of the Settings app.
  • S408 The Android system 11 of the mobile phone 100 sends the parameters of the Android application window of the "settings" application to the mobile agent process 101 of the mobile phone 100.
  • the Android system 11 of the mobile phone 100 can first send the parameters of the Android application window of the "settings” application to the mobile phone agent process 101 of the mobile phone 100, and the mobile phone agent process 101 of the mobile phone 100 will "set” the Android application of the application. Parameters for the window.
  • the desktop agent process 201 of the computer 200 can create the desktop application window of the "settings” application according to the parameters of the Android application window of the "settings” application.
  • the parameters of the Android application window may include the task identifier of the Android application window, the width and height of the Android application window, the area position coordinates of the Android application window, and the like.
  • the area position coordinates of the Android application window can be expressed as ⁇ A l , A r , A r , A b ⁇ , where A l is the position coordinate of the left boundary of the Android application window area, and A t is the position of the upper boundary of the Android application window area Coordinates, Ar is the position coordinate of the right boundary of the Android application window area, A l is the position coordinate of the lower boundary of the Android application window.
  • S409 The mobile agent process 101 of the mobile phone 100 sends the parameters of the Android application window of the application to the desktop agent process 201 of the computer 200.
  • the desktop agent process 201 of the computer 200 creates a desktop application window of the “Settings” application on the desktop system 21 .
  • the desktop agent process 201 of the computer 200 sends the parameters of the Android application window of the "Settings” application based on the mobile phone agent process 101 of the mobile phone 100, and the desktop agent process 201 creates the "Settings" application window on the desktop system 21. ” app’s desktop app window.
  • the desktop agent process 201 of the computer 200 searches the desktop application window 2003-1 of the currently created "settings” application to see if there is a task identification of the Android application window associated with the window "settings” application; if there is no window associated with the task identification , then the desktop agent process 201 creates a new parent window associated with the task identifier, and the desktop agent process 201 creates a desktop application window 2003-1 of the "Settings” application under the newly created parent window; if there is a window associated with the task identifier, then the computer 200 The desktop agent process 201 creates a desktop application window of the “Settings” application under the parent window associated with the task ID.
  • the desktop agent process 201 of the computer 200 determines the area position coordinates, window size, etc.
  • the task identifier of the Android application window is used to uniquely identify the Android application window.
  • the desktop application window created by the computer 200 based on the parameters of the Android application window is associated with the task identifier of the Android application window.
  • the parameters of the Android application window include the task identifier of the Android application window. Therefore, before creating the desktop application window, the desktop agent process 201 checks whether there is a task identifier of the Android application window associated with the window "settings" application in the currently created desktop application window, thereby preventing the application of a mobile phone 100 from being displayed on the computer 200 Two desktop app windows for the Settings app.
  • the desktop agent process 201 of the computer 200 realizes the synchronous adjustment of the created desktop application window by searching for the task identifier of the Android application window . This will be described in detail below, and will not be repeated here.
  • the desktop system 21 of the computer 200 sends the window handle of the desktop application window of the “Settings” application to the mobile phone agent process 101 of the mobile phone 100 .
  • the window handle of the desktop application window is used to uniquely identify the desktop application window.
  • the window handle of the desktop application window of the “Settings” application can be used to identify the desktop application window.
  • the window handle Operates on the desktop app window of the Settings app.
  • S412 The mobile phone proxy process 101 of the mobile phone 100 sends the window handle of the desktop application window of the “Settings” application on the computer 200 to the Android system 11 of the mobile phone 100.
  • S413 The Android system 11 of the computer 200 associates the window handle of the desktop application window 2003-1 of the “Settings” application with the Android application window of the mobile phone 100.
  • the Android system 11 of the computer 200 associates the window handle of the desktop application window of the "Settings” application with the Android application window of the mobile phone 100, so that the operation of the Android application window of the "Settings” application by the mobile phone 100 can be synchronized to On the desktop application window 2003-1 of the computer 200.
  • S414 The Android system 11 of the mobile phone 100 uses the window handle to create drawing information.
  • the drawing information may be drawing information created by the Android system 11 using the window handle of the desktop application window of the “Settings” application.
  • the drawing information may be drawing information created by the Android system 11 using the window handle of the desktop application window of the "Settings” application and the window handle of the Android application window.
  • the drawing information is used to execute on the desktop system 21 of the computer 200 and render an image on the desktop application window identified by the window handle.
  • the Android system 11 of the mobile phone 100 receives the window handle of the desktop application window of the "Settings” application of the mobile phone 100
  • the Android system 11 associates the window handle of the desktop application window 2003-1 with " Android App Window for the Settings app, and continue creating the Android App Window for the Settings app.
  • the Android system 11 continues to create the Android application window of the "Settings” application, it needs to generate drawing information to perform image rendering on the display content of the Android application window.
  • the drawing information is used to call the Embedded-System Graphics Library (EGL), Open Graphics Library (OpenGL), etc. to draw the display content of the Android application.
  • the Android system 11 of the mobile phone 100 uses the associated window handle as the parameter of the Android application window to create drawing information, when the drawing information is executed in the desktop application window, it will not appear that the desktop that executes the drawing information cannot be found.
  • the problem of the application window, and compared to the execution and rendering of the drawing information on the Android system 11, the execution and rendering of the drawing information on the desktop application window belongs to the current screen rendering (on-screen rendering), which can reduce a buffer operation , improving the GPU performance of the computer 200 during image rendering operations.
  • the way that the drawing information is executed on the desktop application window and the image is rendered belongs to the current screen rendering (on-screen rendering), and the way that the drawing information is executed and the image is rendered on the Android system 11 belongs to the off-screen rendering method (off-screen rendering).
  • S415 The Android system 11 of the mobile phone 100 sends the drawing information to the desktop agent process 201 of the computer 200 .
  • S416 The desktop agent process 201 of the computer 200 executes the drawing information.
  • the desktop agent process 201 of the computer 200 executes the drawing information, calls the image library in the graphics service through the graphics device interface, such as EGL, OpenGL, etc., thereby realizing the creation of the desktop of the "settings" application in step S410
  • the display image rendering on the application window completes the display of the display content of the Android application on the desktop application window. For example, the content displayed in the desktop application window 2003-2 of the "Settings" application as shown in FIG. 1B.
  • the computer 200 can also start other Android applications and Displays content from other Android apps.
  • the display interface 210 of the computer 200 may simultaneously display the desktop application window 2003-1 of the "Settings" application, the desktop application window 2003-2 of the "Calculator” application, and the desktop application window 2003-2 of the "Mail” application. Windows 2003-3.
  • the Android system 11 of the mobile phone 100 receives the window handle of the desktop application window 2003-1, and associates the window handle of the desktop application window 2003-1 with the Android application window of the "settings" application of the mobile phone 100. It can be understood that, for multiple applications on the mobile phone 100, the Android system 11 can associate the window handles of the desktop application windows 2003 of the applications on the mobile phone 100 with the Android application windows one by one, so that the windows can be opened and displayed on the desktop system 21.
  • the desktop application windows 2003 of multiple applications of the mobile phone 100 are displayed at the same time. For example, as shown in FIG.
  • the display interface 210 of the computer 200 simultaneously displays the desktop application window 2003-1 of the "Settings” application, the desktop application window 2003-2 of the "Calculator” application, and the desktop application window 2003-2 of the "Mail” application of the mobile phone 100.
  • the desktop agent process 201 of the computer 200 can also receive drawing information in real time, and render the display content of the desktop application window, so that dynamic image content can be displayed on the desktop application window of the computer 200 .
  • the desktop application window 2003-4 displayed on the display interface 210 of the computer 200 can be the desktop application window 2003-4 of the "video player" application of the mobile phone 100 that is opened;
  • the Android application window of the application can be displayed on the virtual screen of the mobile phone 100, and what is displayed on the display screen of the mobile phone 100 is the data of the performance of the mobile phone captured in real time. From the content displayed in real time on the display screen of the mobile phone 100 shown in FIG. GPU resources.
  • the desktop agent process 201 of the computer 200 can create a desktop application window of a "video player" application on the desktop system 21; the Android system 11 of the mobile phone 100 can also The window handle of the desktop application window of the "video player” application is associated with the Android application window of the "video player” application of the mobile phone 100.
  • the Android system 11 of the mobile phone 100 can use the window handle of the desktop application window of the "video player” application to create drawing information, and send the drawing information to the desktop system 21 of the computer 200, and the desktop system 21 of the computer 200 executes the drawing information and Renders an image on the desktop application window of the Video Player application.
  • the application window displayed on the display interface 210 of the computer 200 is the desktop application window 2003-4 of the "video player" application of the mobile phone 100 and this window shows that the video is being played;
  • the window handle of the desktop application window 2003-4 can be associated with the Android application window of the "video player", and the Android system 11 uses the associated window handle as a parameter of the Android application window of the "video player” to create drawing information.
  • the Android application window of the application is associated with the window handle of the desktop application window, and the desktop application window of the application is created based on the parameters of the Android application window, wherein the Android application window The parameter of contains the task ID of the Android application window. Therefore, the drawing information created by the Android system 11 using the window handle of the desktop application window of the "Settings" application can not only run on the desktop system 21, but the desktop system 21 can also determine according to the drawing information which needs to display the image rendered using the drawing information.
  • Desktop Application Windows 2003-3 Moreover, when the desktop agent process 201 of the computer 200 detects the control operation of the desktop application window 2003-3, it generates operation information. The control of the Android application on the computer 200 will be described in detail below, and will not be repeated here.
  • the desktop agent process 201 of the computer 200 can call the graphics library of the computer 200 to render the desktop application window based on the received drawing information; thus, the created desktop application window 2003-2 can display the content of the Android application "calculator".
  • the Android system 11 of the mobile phone 100 uses the received window handle to generate drawing information, which can also display multiple Android application desktop application windows on the display interface of the computer 200, so as to simultaneously open and display multiple Android applications.
  • the desktop application window 2003-2 displays the contents of the Android application "calculator”.
  • the computer 200 can start multiple Android applications, create multiple Android application windows and simultaneously display the contents of multiple Android applications.
  • the user can control and operate the display content (for example, controls of the display interface), window status, and window size in the desktop application window through the mouse, keyboard, and touch, so that the Android application not only It can be displayed on the computer 200, and the Android application of the mobile phone 100 can also be controlled on the computer 200.
  • the display content for example, controls of the display interface
  • window status for example, window status
  • window size in the desktop application window through the mouse, keyboard, and touch
  • Fig. 11 shows an interaction diagram of performing a "zoom in” operation on a desktop application window of an Android application.
  • the desktop system 21 of the computer 200 detects a control operation of a desktop application window, and generates window area position coordinates and operation position coordinates.
  • the desktop application window 2003-2 of the "calculator" application of the mobile phone 100 is displayed on the display interface 210 of the computer 200.
  • Magnification operation after the desktop system 21 of the computer 200 detects the control operation of the desktop application window, the desktop system 21 of the computer 200 adjusts the area position coordinates of the desktop application window; the desktop system 21 of the computer 200 will adjust the area position of the desktop application window before and after
  • the coordinates and the position coordinates at which the desktop system 21 of the computer 200 detects the control operation of the desktop application window are sent to the mobile phone agent process 101 of the mobile phone 100 .
  • the desktop system 21 of the computer 200 detects the control operation of the desktop application window, and the desktop system 21 of the computer 200 adjusts the window size of the desktop application window 2003-2 of the "calculator" application. And the desktop system 21 of the computer 200 generates the window area position coordinates and the operation position coordinates.
  • the area coordinates of the desktop application window 2003-2 are adjusted from [W l , W t , W r , W b ] to [W' l , W' t , W' r , W' b ], where W l is the position coordinates of the left boundary of the desktop application window area of the "calculator” application before adjustment, and W t is the desktop application window of the "calculator” application before adjustment.
  • W l is the position coordinates of the left boundary of the desktop application window area of the "calculator” application before adjustment
  • W t is the desktop application window of the "calculator” application before adjustment
  • W r is the position coordinates of the right boundary of the desktop application window area of the "Calculator” application before adjustment
  • W b is the position coordinates of the lower boundary of the desktop application window area of the "Calculator” application before adjustment
  • W' l is the position coordinate of the left boundary of the desktop application window area of
  • FIG. 12 is a schematic diagram of the desktop application window 2003-2 of the "Calculator" application before and after resizing.
  • the original area coordinates of the desktop application window 2003-2 are [W l , W t , W r , W b ].
  • the area coordinates of the desktop application window 2003-2 are adjusted to [W' l , W' t , W' r , W' b ]. It can be understood that the desktop system 21 of the computer 200 only adjusts the size of the window, and the desktop system 21 of the computer 200 can render the enlarged display content image of the desktop application window 2003-2 based on the operation drawing information sent by the mobile phone 100 .
  • the desktop system 21 of the computer 200 detects the "zoom in” operation of the desktop application window 2003-2 of the "calculator” application of the mobile phone 100, and generates the position coordinates of the "zoom in” window area and the operation position coordinates as an example for illustration. It can be understood that the desktop system 21 of the computer 200 can also detect the size of other application windows of the mobile phone 100 to operate. For details, refer to the desktop system 21 of the computer 200 detecting the desktop application window 2003-2 of the "calculator” application of the mobile phone 100. The "zooming in” operation of this application will not be repeated here.
  • the desktop system 21 of the computer 200 sends the window area position coordinates and the operation position coordinates to the mobile phone agent process 101 of the mobile phone 100 .
  • S1103 The mobile agent process 101 of the mobile phone 100 calls the Android interface, and readjusts the size and position coordinates of the Android application window of the "calculator" application.
  • the mobile phone agent process 101 of the mobile phone 100 resets the task associated with the Android application window of the "calculator” application by calling the resize Task interface of the ActivityTaskManagerService based on the received window area position coordinates and operating position coordinates.
  • the size and position coordinates of the corresponding task are identified, so as to realize the adjustment of the Android application window of the "calculator” application.
  • the window area position coordinates and the operation position coordinates include the task identifier of the Android application window of the “calculator” application of the mobile phone 100 .
  • S1104 The Android system 11 of the mobile phone 100 adjusts the window area position coordinates of the Android application window based on the window area position coordinates and the operation position coordinates, and generates operation drawing information.
  • the user performs a "zoom in” operation on the desktop application window 2003-2 of the "calculator” application on the mobile phone 100, and the desktop system 21 of the computer 200 detects the "zoom in” of the desktop application window 2003-2.
  • Zoom in” operation the desktop system 21 of the computer 200 generates the window area position coordinates and the operation position coordinates of the "zoom in” operation, and the desktop system 21 sends the control operation information of "zoom in” to the mobile phone agent process 101 of the mobile phone 100;
  • the agent process 101 adjusts the position coordinates of the Android application window by calling the resize Task interface of the ActivityTaskManagerService based on the received window area position coordinates and the operation position coordinates.
  • the position coordinates of the “zoomed-in” window area and the coordinates of the operation position include the task identifier of the Android application window of the “calculator” application of the mobile phone 100 .
  • the Android system 11 of the mobile phone 100 identifies the task associated with the Android application window of the "Calculator” application by calling the resize Task interface of the ActivityTaskManagerService based on the position coordinates of the "zoomed-in" window area and the operation position coordinates.
  • the corresponding task executes the control operation and adjusts the area position coordinates of the Android application window of the "Calculator”application; the area position coordinates of the Android application window of the "Calculator” application are adjusted by [A l , A t , A r , A b ] is [A' l , A' t , A' r , A' b ], where A l is the position coordinate of the left border of the Android application window area of the "Calculator” application before adjustment, and A t is the "Calculator” application before adjustment.
  • A' l is the position coordinates of the left border of the Android application window area of the "calculator” application after adjustment
  • a t is the position coordinates of the upper border of the Android application window area of the "calculator” application
  • Ar is the position coordinate of the right border of the Android application window area of the "Calculator” application
  • a b is the position coordinate of the lower border of the Android application window area of the "Calculator” application.
  • the Android system 11 of the mobile phone 100 calls the system service interface to adjust the area coordinates of the Android application window based on the window area position coordinates and the operating position coordinates, and the adjusted area coordinates of the Android application window [A' l , A' t , A' r , A' b ] need to satisfy the calculation relationship of the following formula (1) and formula (2):
  • the Android system 11 of the mobile phone 100 adjusts the area coordinates of the Android application window of the "Calculator” application, and generates operation drawing information.
  • the operation drawing information is used to draw the content displayed in the enlarged desktop application window of the "Calculator” application.
  • S1105 The Android system 11 of the mobile phone 100 sends the operation drawing information to the desktop agent process 201 of the computer 200 .
  • S1106 The desktop agent process 201 of the computer 200 executes the operation drawing information.
  • the desktop agent process 201 of the computer 200 draws the enlarged desktop application window 2003-2 of the "calculator" application on the desktop system 21 based on the operation drawing information.
  • the drawing information generated in step S415 of FIG. 4 refer to the content of the drawing information generated in step S415 of FIG. 4 , which will not be repeated here.
  • the computer 200 can also detect the user's "zooming out” operation on the desktop application window.
  • the specific process please refer to the process described in FIG. 11 above, and details will not be repeated here.
  • the following describes the control operation on the computer 200 for the display content (for example, the controls of the display interface) of the Android application of the mobile phone 100 .
  • the computer 200 can detect that the user clicks the button icon 218 of the desktop application window 2003-1, and adjust the display content of the desktop application window 2003-1 as shown in FIG. 13B. content, and the option 4 of the mobile phone 100 is changed from the closed state to the open state.
  • the interaction diagram of the control of the operation of the desktop application window of the Android application will be described in detail below in conjunction with FIG. 14 .
  • Fig. 14 shows an interaction diagram for controlling the display content of the desktop application window of the Android application.
  • the specific process of controlling the display content of the desktop application window of the Android application is as follows:
  • the desktop system 21 of the computer 200 detects the control operation of the desktop application window, and generates an operation position coordinate.
  • the computer 200 can create a desktop application window 2003-1 of the "Settings” application, and display the content of the Android application "Settings” on the desktop application window 2003-1. displayed on the display interface 210 of the computer 200.
  • the desktop system 21 of the computer 200 can detect that the user clicks the button icon 218 of the “Settings” application with the mouse, and generates an operation location icon.
  • the generated operation location icon may be the location coordinates of the area where the button icon 218 is located.
  • S1402 The desktop system 21 of the computer 200 sends the operation position coordinates to the mobile phone agent process 101 of the mobile phone 100 .
  • S1403 The mobile phone agent process 101 of the mobile phone 100 sends the operating position coordinates to the Android system 11 of the mobile phone 100.
  • S1404 The Android system 11 of the mobile phone 100 generates operation drawing information based on the operation position coordinates.
  • the relative position coordinates of the area position coordinates of the desktop application window of the “Settings” application and the area position coordinates of the Android application window have a mapping relationship.
  • the Android system 11 of the mobile phone 100 detects an operation on setting option 4 based on the area position coordinates of the button icon 218 of "setting option 4".
  • the Android system 11 of the mobile phone 100 performs a control operation on the task corresponding to the task identifier associated with the Android application window of the "Settings" application by calling the InputManagerService service interface, and generates operation drawing information.
  • setting option 4 is the flight mode
  • the button icon 218 is the state switch button of the flight mode.
  • the Android system 11 of the mobile phone 100 enables the flight mode by calling the InputManagerService service interface, and generates operation drawing information, and the generated operation drawing information is used to draw the display window after the flight mode is turned on.
  • S1405 The Android system 11 of the mobile phone 100 sends the operation drawing information to the desktop agent process 201 of the computer 200 .
  • the desktop agent process 201 of the computer 200 draws on the desktop system 21 the desktop application window 2003-1 of the “Settings” application after setting option 4 is closed based on the operation drawing information.
  • the desktop agent process 201 of the computer 200 draws on the desktop system 21 the desktop application window 2003-1 of the “Settings” application after setting option 4 is closed based on the operation drawing information.
  • the drawing information generated in step S415 of FIG. 4 which will not be repeated here.
  • cross-platform display method of the application shown in FIG. 3 can be applied to the electronic device 100 and the electronic device 200 respectively installed with different operating systems.
  • the cross-platform display method of the application may also be applicable to a scenario where two operating systems are running on one electronic device.
  • Fig. 15 shows another scenario of distributed display of cross-platform applications according to some embodiments of the present application.
  • the scenario shown in FIG. 15 includes a third electronic device 300 ; wherein, the third electronic device 300 runs the second operating system 21 , and the third electronic device 300 also installs a virtual machine 400 running the first operating system 11 .
  • the first operating system 11 and the second operating system 21 please refer to the above description, and details are not repeated here.
  • the user can double-click the "Mobile Application Engine” application shortcut icon 321 on the display interface 320 of the third electronic device 300 by using a mouse or touch, etc., to open the running first engine shown in Figure 15(b).
  • a user operation interface 410 of the virtual machine 400 of the operating system 11 is shown in Figure 15(a).
  • the third electronic device 300 may be an electronic device capable of simultaneously running the first operating system 11 and the second operating system 21, for example, the third electronic device 300 may include but not limited to a mobile phone, a notebook computer , tablet computer, desktop computer, laptop computer, Ultra-mobile Personal Computer (Ultra-mobile Personal Computer, UMPC), handheld computer, netbook, personal digital assistant (Personal Digital Assistant, PDA) and other electronic equipment.
  • a mobile phone a notebook computer , tablet computer, desktop computer, laptop computer, Ultra-mobile Personal Computer (Ultra-mobile Personal Computer, UMPC), handheld computer, netbook, personal digital assistant (Personal Digital Assistant, PDA) and other electronic equipment.
  • UMPC Ultra-mobile Personal Computer
  • PDA Personal Digital Assistant
  • the following takes the third electronic device 300 as the computer 300, the first operating system 11 as the Android system 11, and the second operating system 21 as the desktop system 21 as an example to illustrate another cross-platform application distributed display process;
  • FIG. The embodiment of the application shows another interactive diagram of cross-platform distributed display; as shown in Figure 16, the specific process of cross-platform distributed display is as follows:
  • the virtual machine 400 detects an operation of starting monitoring in the mobile phone agent process 101 .
  • the virtual machine of the Android system 11 is installed on the desktop system 21 of the computer 300, and the user can click the "mobile application engine” application shortcut of the display interface 320 of Figure 15(a) through a mouse or touch.
  • the icon 321 opens the display interface 410 of the virtual machine 400 of the Android system 11 .
  • the user can click the “mobile phone agent” icon on the display interface 401 of the virtual machine 400 to start the mobile phone agent process 101 by using a mouse or touch.
  • step S401 for the specific content of enabling the monitoring function of the mobile phone agent process 101, refer to step S401, which will not be repeated here.
  • the computer 300 detects that the desktop agent process 201 of the computer 300 starts a monitoring operation.
  • the user can start the desktop agent process 201 by clicking the shortcut icon 331 of the desktop agent on the display interface 330 of the computer 300.
  • the shortcut icon 331 of the desktop agent on the display interface 330 of the computer 300.
  • the user can start the desktop proxy process 331 by clicking the shortcut icon 331 of the desktop proxy on the display interface 320 of the computer 300; after starting the desktop proxy process, as shown in FIG. 17 (b), the user can click the button 341 of "USB” on the display interface 340, and the button 341 option of "USB” indicates that the desktop agent process 201 can communicate with the mobile phone agent process 101 through the connection mode of USB, and then, The user clicks the "Connect” button 342, and the desktop agent process 201 of the computer 300 starts the monitoring operation.
  • the user can also start the desktop proxy process 201 of the computer 300 in other ways, and enable the monitoring function of the desktop proxy process 201; this application does not specifically limit this according to actual application conditions.
  • S1603 Send/return a request between the mobile phone proxy process 101 of the virtual machine 400 and the desktop proxy process 201 of the computer 300, and establish a distributed display connection.
  • the mobile phone agent process 101 and the desktop agent process 201 can be connected through the network card address, as shown in Figure 17(b), when the user clicks on the display interface 340
  • the desktop agent process 201 is configured as Android debug bridge (android debug bridge, adb) port mapping, then, the user clicks the "Connect" button 342, the desktop agent process 201 of the computer 200 and the mobile phone 100
  • the mobile phone agent process 101 sends/returns a request, wherein the request sent/returned may be a socket request (socket request).
  • the local loopback address may be 127.0.0.1.
  • the display interface 340 will display a “successful connection” icon 343 .
  • step S1604 The computer 300 detects the operation of starting the application of the virtual machine 400 on the desktop proxy process 201. For details, refer to step S404, which will not be repeated here.
  • step S1605 The desktop agent process 201 of the computer 300 sends the parameters of the started Android application of the virtual machine 400 to the mobile phone agent process 101. For details, refer to step S405, which will not be repeated here.
  • step S1606 The mobile phone agent process 101 of the virtual machine 400 calls the interface of the Android system 11 to start the application of the virtual machine 400. Refer to step S406 for specific content, which will not be repeated here.
  • step S1607 The Android system 11 of the virtual machine 400 creates an Android application window of an application. For specific content, refer to step S407, which will not be repeated here.
  • S1608 The Android system 11 of the virtual machine 400 sends the parameters of the Android application window to the mobile phone agent process 101 of the virtual machine 400.
  • S1609 The mobile phone proxy process 101 of the virtual machine 400 sends the parameters of the Android application window of the application to the desktop proxy process 201 of the computer 300 .
  • the desktop agent process 201 of the computer 300 creates a desktop application window of the application on the desktop system 21 based on the parameters of the Android application window. For specific content, refer to step S410, which will not be repeated here.
  • S1611 The desktop system 21 of the computer 300 sends the window handle of the desktop application window of the application to the mobile phone agent process 101 of the virtual machine 400.
  • S1612 The mobile phone proxy process 101 of the virtual machine 400 sends the window handle of the desktop application window of the application on the computer 300 to the Android system 11 of the virtual machine 400.
  • S1613 The Android system 11 of the computer 300 associates the window handle of the desktop application window of the application with the Android application window of the virtual machine 400.
  • the Android system 11 receives the window handle of the desktop application window and associates the window handle of the desktop application window with the Android application window of the application of the virtual machine 400 .
  • Multiple applications of the virtual machine 400 may be displayed on multiple desktop application windows, wherein each Android application window of the application of the virtual machine 400 is associated with each desktop application window one by one.
  • step S1614 The Android system 11 of the virtual machine 400 uses the window handle to create drawing information. For details, refer to step S414, which will not be repeated here.
  • S1615 The Android system 11 of the virtual machine 400 sends the drawing information to the desktop agent process 201 of the computer 300 .
  • step S1616 The desktop agent process 201 of the computer 300 executes the drawing information created using the window handle on the desktop system 21 . For details, refer to step S416, which will not be repeated here.
  • the display interface 320 of the computer 300 displays a desktop application window 3003 of the “video player” application of the virtual machine 400 .
  • the desktop application window displayed on the display interface 320 of the computer 300 may be the desktop application window 3003 of the "video player" application of the opened virtual machine 400;
  • the drawing information is executed on the desktop application window and the rendered image belongs to the current screen rendering (on-screen rendering), which can reduce a buffer operation and improve the GPU performance of the computer 300 image rendering operation.
  • the way that the drawing information is executed on the desktop application window and the image is rendered belongs to the current screen rendering (on-screen rendering)
  • the way that the drawing information is executed and the image is rendered on the Android system 11 belongs to the off-screen rendering method (off-screen rendering).
  • the desktop of the computer 300 can also display the display windows of multiple Android applications of the virtual machine 400 at the same time.
  • the user can also control the display content (for example, the control of the display interface), the window state and the window size, etc. in the desktop application window of the computer 300 through the mouse, keyboard and touch, etc.
  • the Android application realizing the virtual machine 400 can not only be displayed on the computer 300 , but also can control the Android application of the mobile phone 100 on the computer 300 .
  • the application of the virtual machine 400 is displayed on the desktop application window of the desktop system 21, which can realize opening and simultaneously displaying the desktop application windows of multiple applications of the virtual machine 400; and when the desktop application window needs to render the image in real time when playing video , the drawing information generated by the Android system 11 of the virtual machine 400 can be executed on the desktop system 21, thereby realizing the off-screen rendering mode to render the image displayed on the desktop application window, thereby reducing a buffer operation and improving the image rendering operation of the computer 300 GPU performance; and the user can also perform operations such as zooming in, zooming out, and moving the desktop application window displaying the application of the virtual machine 400, making window operations more convenient and improving user experience.
  • FIG. 20 shows a schematic structural diagram of the electronic device 100 .
  • the electronic device 100 may include a processor 110, a memory 121, a universal serial bus (universal serial bus, USB) interface 130, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication Module 160, audio module 170, display screen 194, camera 193 and so on.
  • a processor 110 a memory 121
  • a universal serial bus universal serial bus, USB
  • a power management module 141 an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication Module 160, audio module 170, display screen 194, camera 193 and so on.
  • the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100 .
  • the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or decompose certain components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.
  • application processor application processor, AP
  • modem processor graphics processing unit
  • GPU graphics processing unit
  • image signal processor image signal processor
  • ISP image signal processor
  • controller video codec
  • digital signal processor digital signal processor
  • baseband processor baseband processor
  • neural network processor neural-network processing unit
  • the USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like.
  • the electronic device 100 may communicate with the electronic device 200 through the USB interface 130 .
  • the power management module 141 is used for connecting the battery, the charging management module and the processor 110 .
  • the power management module 141 receives the input of the battery and/or the charging management module, and supplies power to the processor 110 , the memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 .
  • the wireless communication function of the electronic device 100 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, a baseband processor, and the like.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas.
  • Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network.
  • the antenna may be used in conjunction with a tuning switch.
  • the wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (bluetooth, BT), etc. s solution.
  • the wireless communication module 160 may be one or more devices integrating at least one communication processing module.
  • the wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 .
  • the wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.
  • the above performance parameters may include available network bandwidth of the Wi-Fi network, where the available network bandwidth of the Wi-Fi network is the network bandwidth used for transmitting tasks.
  • the electronic device 100 and the electronic device 200 can communicate and connect through the wireless communication module 160; Sending the parameters of the application, the electronic device 100 may also send the parameters of the Android application window to the electronic device 200 , based on the parameters, the electronic device 200 may create a window to display the content of the application of the electronic device 100 .
  • the electronic device 100 realizes the display function through the GPU, the display screen 194 , and the application processor.
  • the GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.
  • Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
  • the display screen 194 is used to display images, videos and the like.
  • the started Android application can be started on the display screen 194 , and the Android application window and rendered image created by the electronic device 100 can be displayed on the display screen 194 .
  • the memory 121 may be used to store computer-executable program codes including instructions.
  • the memory 121 may include an area for storing programs and an area for storing data.
  • the structure shown in the embodiment of the present invention does not constitute a specific limitation of the electronic device 100 .
  • the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • FIG. 21 is a block diagram of the software structure of the electronic device 100 according to the embodiment of the present invention.
  • the layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces.
  • the Android system is divided into four layers, which are respectively the application program layer, the application program framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer from top to bottom.
  • the application layer can consist of a series of application packages.
  • the application package may include application programs such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.
  • application programs such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.
  • the application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer.
  • the application framework layer includes some predefined functions.
  • the application framework layer may include window manager, content provider, view system, phone manager, resource manager, notification manager and so on.
  • the window manager may include the mobile phone proxy service 101 of the electronic device 100 for managing window programs.
  • the window manager can call the interface to create the desktop application window according to the parameters of the Android application.
  • the window manager can also send the parameters and drawing information of the Android application window to the internal mobile phone proxy service 101, and the mobile phone proxy service 101 sends the parameters and drawing information of the Android application window to other electronic devices.
  • Content providers are used to store and retrieve data and make it accessible to applications.
  • Said data may include video, images, audio, calls made and received, browsing history and bookmarks, phonebook, etc.
  • the view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on.
  • the view system can be used to build applications.
  • a display interface can consist of one or more views.
  • a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.
  • the resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.
  • the resource manager can also be used to store the Overlay configuration file.
  • the notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction.
  • the notification manager is used to notify the download completion, message reminder, etc.
  • the notification manager can also be a notification that appears on the top status bar of the system in the form of a chart or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window.
  • prompting text information in the status bar issuing a prompt sound, vibrating the electronic device, and flashing the indicator light, etc.
  • the Android Runtime includes core library and virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.
  • the core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.
  • the application layer and the application framework layer run in virtual machines.
  • the virtual machine executes the java files of the application program layer and the application program framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • a system library can include multiple function modules. For example: surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.
  • the surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.
  • the media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing, etc.
  • 2D graphics engine is a drawing engine for 2D drawing.
  • the kernel layer is the layer between hardware and software.
  • the kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.
  • FIG. 22 shows a structural block diagram of an electronic device 200 or an electronic device 300 according to an embodiment of the present application.
  • the electronic device 200 or electronic device 300 may include a processor 201, a memory 202, a wireless communication processing module 203, a power switch 204, a wired LAN communication processing module 205, a display screen 206 and an audio module 207. in:
  • Processor 201 may be used to read and execute computer readable instructions.
  • the processor 201 may mainly include a controller, an arithmetic unit, and a register.
  • the controller is mainly responsible for instruction decoding, and sends out control signals for the operations corresponding to the instructions.
  • the arithmetic unit is mainly responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logic operations, and can also perform address operations and conversions.
  • the register is mainly responsible for saving the register operands and intermediate operation results temporarily stored during the execution of the instruction.
  • the hardware architecture of the processor 201 may be an application specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture, and so on.
  • ASIC application specific integrated circuit
  • the processor 201 may be used to analyze signals received by the wireless communication processing module 203 or the wired LAN communication processing module 205 , such as Android application window parameters and drawing information sent by the electronic device 100 .
  • the processor 201 can create a desktop application window according to the parameters of the Android application window; the processor 201 can also execute drawing information to render an image, so that the rendered image can be displayed on the desktop application window.
  • the memory 202 is coupled with the processor 201 for storing various software programs and/or sets of instructions.
  • the memory 202 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices or other non-volatile solid-state storage devices.
  • the memory 202 can store operating systems, such as embedded operating systems such as uCOS, VxWorks, and RTLinux.
  • the memory 202 can also store a communication program that can be used to communicate with the electronic device 100, one or more servers, or additional devices.
  • the power switch 204 can be used to control the power to supply power to various components of the electronic device 200 or the electronic device 300 .
  • the wired LAN communication processing module 205 can be used to communicate with other devices in the same LAN through the wired LAN, and can also be used to connect to the WAN through the wired LAN, and can communicate with devices in the WAN.
  • the display screen 206 may be used to display images, videos, and the like.
  • the display screen 206 may adopt a liquid crystal display (liquid crystal display, LCD), an organic light-emitting diode (organic light-emitting diode, OLED) display screen, an active-matrix organic light-emitting diode (active-matrix organic light emitting diode, AMOLED) Display, flexible light-emitting diode (FLED) display, quantum dot light emitting diodes (QLED) display, etc.
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • AMOLED active-matrix organic light emitting diode
  • FLED flexible light-emitting diode
  • QLED quantum dot light emitting diodes
  • the audio module 210 can be used to output audio signals through the audio output interface, so that the electronic device 200 or the electronic device 300 can support audio playback.
  • the audio module can also be used to receive audio data through the audio input interface.
  • the audio module 207 includes, but is not limited to: a microphone, a speaker, a receiver, and the like.
  • the structure shown in FIG. 22 does not constitute a specific limitation on the electronic device 200 or the electronic device 300 .
  • the electronic device 200 or the electronic device 300 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components.
  • the illustrated components can be realized in hardware, software or a combination of software and hardware.
  • FIG. 23 is a block diagram of the software structure of the electronic device 200 or the electronic device 300 according to the embodiment of the present invention.
  • the software structure of the electronic device 200 or the electronic device 300 is respectively an application layer, a framework layer, a core library layer and a driver layer from top to bottom.
  • the driver layer may include, but is not limited to, a CPU driver, a GPU driver, a display controller driver, and the like.
  • the core library layer is the core part of the operating system.
  • the core library layer includes but is not limited to input/output services, graphics services for CPU, and GPU graphics processing. Graphics services may include, but are not limited to, 2D engines, 3D engines, Frame Buffers, OpenGL, EGL, and the like.
  • the framework layer includes, but is not limited to, graphics services, system services, web services, and user services; graphics services may include, but is not limited to, canvases, views, and so on.
  • the application layer includes but is not limited to desktop, media player, browser, etc.
  • the electronic device 100 executes the drawing information to display the rendered image on the display screen mainly by drawing each application window into a buffer (Buffer), and then compositing the Buffer of these application windows on the display screen , in this case, there is one more screen buffer composition.
  • a buffer For example, in the application window display of the Android application of the mobile phone 100, what is displayed on the display screen is a synthesized display window, for example, what is displayed on the display screen of the mobile phone 100 is window 1 of the "settings" application, time or battery status bar Window 2 and so on are composited display windows. Therefore, the display screen of the mobile phone 100 needs a buffer so as to realize the buffer synthesis of these windows (window 1, window 2, etc.), and they can be displayed on the display screen after synthesis.
  • the electronic device 200 directly presents the drawing result of the window on the display screen of the computer 200, which is different from displaying application windows on the display screen of the mobile phone 100.
  • the content synthesized by multiple application windows is displayed on the display screen, while only the content of one application window is displayed on one desktop application window of the computer 200, so there is no need to synthesize multiple application windows, thereby saving the buffer (buffer) of the screen synthesis application window .
  • the application program of the desktop agent service 201 transmits the drawing information to the framework layer, and the framework layer applies to the system for a canvas of the drawing layer according to the drawing information, wherein the size of the canvas is based on the to-be The width and height parameters of the drawing layer are determined; the system applies for calling the layer memory allocation unit to allocate a canvas for drawing the layer; at the same time, it obtains the drawing information and passes it to the rendering driver of the core library layer to make the rendering driver Drive the drawing engine to call the OpenGL or EGL drawing layer based on the drawing information, so as to complete the image drawing of the desktop application window of the Android application.
  • the drawing engine includes a drawing engine of a graphics processing unit GPU, including a 2D drawing engine, a 3D drawing engine, and the like.
  • the electronic device 200 receives the drawing information sent by the electronic device 100, and the drawing information includes primitive information, primitive coordinates, and the like.
  • primitives are the most basic materials used by graphics software to operate and organize pictures.
  • a picture is composed of primitives, which are a group of the simplest and most common collection of graphics or characters, such as points, line segments, and triangles. , polygons, etc.
  • each unit/module mentioned in each device embodiment of this application is a logical unit/module.
  • a logical unit/module can be a physical unit/module, or a physical unit/module.
  • a part of the module can also be realized with a combination of multiple physical units/modules, the physical implementation of these logical units/modules is not the most important, the combination of functions realized by these logical units/modules is the solution The key to the technical issues raised.
  • the above-mentioned device embodiments of this application do not introduce units/modules that are not closely related to solving the technical problems proposed by this application, which does not mean that the above-mentioned device embodiments do not exist other units/modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Digital Computer Display Output (AREA)

Abstract

本申请涉及计算机技术领域，公开了一种应用的跨平台显示方法、可读介质和电子设备。包括：第一系统平台获取第二系统平台上运行的第一应用的第一窗口的第一尺寸，并在第一系统平台中创建对应第一应用的第三窗口，其中第三窗口用于显示第二系统平台上运行的第一应用的显示界面；第一系统平台获取第二系统平台上运行的第二应用的第二窗口的第二尺寸，并在第一系统平台中创建对应第二应用的第四窗口，其中第四窗口用于显示第二系统平台上运行的第二应用的显示界面。从而实现在第一系统平台上显示多个第二系统平台上运行的第二应用的显示界面。

Description

应用的跨平台显示方法、可读介质和电子设备
本申请要求于2021年07月13日提交中国专利局、申请号为202110791240.8、申请名称为“应用的跨平台显示方法、可读介质和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别涉及一种应用的跨平台显示方法、可读介质和电子设备。
背景技术
随着通信技术的发展，跨平台运行应用变得越来越普遍；例如，用户可以将
Figure PCTCN2022104441-appb-000001
系统上的应用在
Figure PCTCN2022104441-appb-000002
系统上运行。
例如，可以通过安装安卓虚拟机或者采用投屏技术将
Figure PCTCN2022104441-appb-000003
系统上的应用运行在
Figure PCTCN2022104441-appb-000004
系统上，但是，安装安卓虚拟机或者投屏技术都很难实现在
Figure PCTCN2022104441-appb-000005
系统上同时显示多个安卓应用的内容；并且用户也无法对在
Figure PCTCN2022104441-appb-000006
系统上显示的安卓应用的窗口的大小或位置等进行控制操作。
发明内容
本申请实施例提供了一种应用的跨平台显示方法、可读介质和电子设备。
第一方面，本申请实施例提供了一种应用的跨平台显示方法，包括：第一系统平台获取第二系统平台上运行的第一应用的第一窗口的第一尺寸，并在第一系统平台中创建对应第一应用的第三窗口，其中第三窗口用于显示第二系统平台上运行的第一应用的显示界面；第一系统平台获取第二系统平台上运行的第二应用的第二窗口的第二尺寸，并在第一系统平台中创建对应第二应用的第四窗口，其中第四窗口用于显示第二系统平台上运行的第二应用的显示界面。
例如，第一系统平台可以是下文中的运行于第二电子设备上的第二操作系统，例如，
Figure PCTCN2022104441-appb-000007
系统，也称桌面系统；第二系统平台可以是下文中运行于第一电子设备上的第二操作系统，例如，
Figure PCTCN2022104441-appb-000008
系统，也称安卓系统；第二系统平台也可以是下文中运行于第三电子设备上的虚拟机上的第二操作系统，例如，
Figure PCTCN2022104441-appb-000009
系统，也称安卓系统。其中，第二电子设备和第三电子设备可以是电脑，第一电子设备可以是手机。
例如，第一应用的第一窗口可以是下文的“计算器”应用的安卓应用窗口。第一应用的第三窗口可以是下文的“计算器”应用的桌面应用窗口。第一应用的第一窗口的第一尺寸可以是下文中的“计算器”应用的安卓应用窗口的参数，其中，安卓应用窗口的参数包括但不限于安卓应用窗口的标识、安卓应用窗口的尺寸以及窗口区域位置坐标等。
例如，第二应用的第二窗口可以是下文的“设置”应用的安卓应用窗口。第二应用的第四窗口可以是下文的“设置”应用的桌面应用窗口。第二应用的第二窗口的第二尺寸可以是下文中的“计算器”应用的安卓应用窗口的参数。
例如，手机可以将“计算器”的安卓应用窗口的参数发送给电脑，其中，安卓应用窗口的参数包含安卓应用窗口的尺寸以及窗口区域位置坐标；电脑基于接收的“计算器”应用的安卓应用窗口的参数创建桌面应用窗口并将桌面应用窗口的窗口句柄发送手机，其中，桌面应用窗口的窗口句柄用于唯一标识桌面应用窗口；手机使用接收的窗口句柄生成绘图信息并将绘图信息发送给电脑；电脑可以基于接收的绘图信息调用电脑的图形库对桌面应用窗口进行图像渲染；如下文中的图1B所示，从而实现“计算器”应用的桌面应用窗口显示安卓应用“计算器”的内容。可以理解，根据上述方法，如下文中的图1C所示，电脑可以启动多个安卓应用，创建多个安卓应用窗口并同时显示多个安卓应用的内容。
在上述第一方面的一种可能的实现中，上述方法还包括：第一系统平台在检测到用户在第一系统平台中打开第一应用或者第二应用的情况下，从所述第二系统平台获取所述第一尺寸或第二尺寸。
例如，电脑可以同步显示手机的安卓应用。例如，手机与电脑建立分布式连接后，电脑可以通过在电脑的桌面上创建安卓应用的快捷方式从而实现电脑同步显示安卓应用。电脑可以通过检测到用户打开电脑的桌面上创建安卓应用的快捷方式的操作，获得启动的安卓应用的参数。
例如，电脑还可以在显示界面上创建安卓应用的显示按钮。电脑还可以通过检测到用户打开电脑的显示窗口上创建的安卓应用的显示按钮的操作，获得安卓应用的参数。
例如，手机可以基于安卓应用的参数，启动安卓应用。其中，在启动安卓应用的过程中，手机会创建该应用的安卓应用窗口，手机将安卓应用窗口的参数发送给电脑。
在上述第一方面的一种可能的实现中，上述方法还包括：第二系统平台不显示第一窗口和第二窗口。
例如，安卓应用可以启动在手机的虚拟屏上，也可以启动在手机的显示屏上，当安卓应用启动在手机的虚拟屏上时，可以实现电脑启动安卓应用，并且启动安卓应用的显示内容只在电脑上显示。例如，如图1B所示，安卓应用“计算器”的内容只在电脑上显示。从而使得在不影响手机正常使用的情况下实现跨平台应用分布式应用显示。
可以理解，由下文的图10所示，手机的显示屏可以不显示“视频播放器”应用的安卓应用窗口，桌面应用窗口执行“视频播放器”应用的绘图信息实现视频图像渲染并消耗电脑的GPU资源，安卓应用窗口不执行该指令从而不消耗手机的GPU资源，从而实现既可以在电脑上显示手机的应用的显示界面，又可以不消耗手机的GPU资源。
在上述第一方面的一种可能的实现中，上述方法还包括：在第一系统平台中创建对应第一应用的第三窗口，或者在第一系统平台中创建对应第二应用的第四窗口，包括：第一系统平台为第一应用设置第一标识，或者第一系统平台为第二应用设置第二标识；其中第一标识用于标识第三窗口用于显示第一应用的显示界面，第二标识用于标识第四窗口用于显示第二应用的显示界面。
例如，第一标识可以是“计算器”应用的桌面应用窗口的窗口句柄，第二标识可以是“设置”应用的桌面应用窗口的窗口句柄。其中，“计算器”应用的桌面应用窗口的窗口句柄用于唯一标识“计算器”应用的桌面应用窗口；“设置”应用的桌面应用窗口的窗口句柄用于唯一标识“设置”应用的桌面应用窗口。
可以理解，电脑基于安卓应用窗口的参数，创建桌面应用窗口，并为创建的桌面应用窗口分配窗口句柄。其中，分配的窗口句柄用于唯一标识创建的桌面应用窗口。例如，电脑创建10个桌面应用窗口，电脑给每个桌面应用窗口分配一个窗口句柄；当电脑对桌面应用窗口进行操作时，需要先获取该桌面应用窗口的窗口句柄才可以对该桌面应用窗口进行操作。
在上述第一方面的一种可能的实现中，上述方法还包括：第一系统平台向第二系统平台发送第一标识或第二标识。
例如，手机接收电脑发送的桌面应用窗口的窗口句柄后，继续创建安卓应用窗口，并将安卓应用窗 口与桌面应用窗口的窗口句柄关联。在手机继续创建安卓应用窗口的过程中，安卓应用窗口与桌面应用窗口的窗口句柄关联，手机可以使用窗口句柄生成的绘图信息，从而使得生成的绘图信息可以直接在电脑上被执行。
在上述第一方面的一种可能的实现中，上述方法还包括：第一系统平台通过以下方式在第三窗口中显示第二系统平台上运行的第一应用：第一系统平台从第二系统平台接收第一绘图信息，其中第一绘图信息包括第一标识；第一系统平台根据第一绘图信息所包括的第一标识，在第三窗口中显示根据第一绘图信息所绘制的，第一应用的第一显示界面。
例如，第一绘图信息可以是“计算器”应用的绘图信息，电脑可以根据接收的“计算器”应用的绘图信息调用电脑的图像库，例如EGL、OpenGL等，第一显示界面可以是电脑调用图像库进行图像渲染，获得的在“计算器”的桌面应用窗口上用于显示“计算器”应用的显示界面。
可以理解，手机只需要向电脑发送安卓应用窗口的参数以及绘图信息，电脑基于接收的安卓应用窗口的参数以及绘图信息创建桌面应用窗口并调用图形库绘制窗口显示内容，从而实现在电脑上启动并显示安卓应用的内容。相较于现有的投屏技术的跨平台应用分布式显示，本申请的应用分布式显示不需要在设备之间传输大量视频图像数据即可实现分布式显示。
在上述第一方面的一种可能的实现中，上述方法还包括：第一系统平台通过以下方式在第四窗口中显示第二系统平台上运行的第二应用：第一系统平台从第二系统平台接收第二绘图信息，其中第二绘图信息包括第二标识；第二系统平台根据第二绘图信息所包括的第二标识，在第四窗口中显示根据第二绘图信息所绘制的，第二应用的第二显示界面。
例如，第二绘图信息可以是“设置”应用的绘图信息，电脑可以根据接收的“设置”应用的绘图信息调用电脑的图像库，例如EGL、OpenGL等，第二显示界面可以是电脑调用图像库进行图像渲染，获得的在“设置”的桌面应用窗口上用于显示“计算器”应用的显示界面。
在上述第一方面的一种可能的实现中，上述方法还包括：第一系统平台检测到用户对第一系统平台上的第三窗口的第一操作；第一系统平台生成操作信息，并将操作信息发送给第二系统平台，其中，操作信息包括第一标识；第一系统平台从第二系统平台接收第三绘图信息，其中第三绘图信息包括第一标识，并且第三绘图信息是第二系统平台，通过执行操作信息对应的操作生成的；第一系统平台基于第三绘图信息在第三窗口中显示第三显示界面，其中第三显示界面为第一显示界面对应第一操作改变后的显示界面。
例如，如下文的图1B所示，桌面应用窗口中的显示内容可以是“计算器”应用的桌面应用窗口中的数字或运算符号等；桌面应用窗口大小可以是“计算器”应用的桌面应用窗口在显示界面的位置、“计算器”应用的桌面应用窗口的显示区域大小等；“计算器”应用的桌面应用窗口状态可以是打开、关闭、最小化等。
如图1B所示，电脑基于接收的安卓应用窗口的参数以及绘图信息创建桌面应用窗口并调用图形库绘制窗口显示内容，从而实现图1B中的“计算器”应用的桌面应用窗口的显示。因此，安卓应用窗口的窗口区域的绝对坐标与桌面应用窗口的窗口区域的绝对坐标具有映射关系，即手机可以基于操作信息中的操作位置坐标对安卓应用窗口显示的内容进行重新绘制，生成操作绘图信息。
例如，第三绘图信息可以是下文的“计算器”应用的操作绘图信息。例如，用户对手机的“计算器”应用的桌面应用窗口进行“放大”操作，电脑的桌面系统检测到桌面应用窗口的“放大”操作，电脑的桌面系统生成“放大”操作的窗口区域位置坐标以及操作位置坐标，桌面系统将“放大”的控制操作信息发送给手机的手机代理进程；手机的手机代理进程基于接收的窗口区域位置坐标以及操作位置坐标，通过调用 ActivityTaskManagerService的resize Task接口调整安卓应用窗口的位置坐标。手机的安卓系统调整“计算器”应用的安卓应用窗口的区域坐标，并生成操作绘图信息。该操作绘图信息用于绘制放大后的“计算器”应用的桌面应用窗口显示的内容。
例如，第三绘图信息也可以是下文的“设置”应用的操作绘图信息。例如，“设置”应用的桌面应用窗口的区域位置坐标与安卓应用窗口的区域位置坐标的相对位置坐标具有映射关系。如下文的图13A所示，手机的安卓系统基于“设置选项4”的按钮图标的区域位置坐标检测到对设置选项4的操作。手机的安卓系统通过调用InputManagerService服务接口对“设置”应用的安卓应用窗口关联的任务标识对应的任务执行控制操作，并生成操作绘图信息。例如，设置选项4为飞行模式，按钮图标为飞行模式的状态开关按钮。手机的安卓系统通过调用InputManagerService服务接口打开飞行模式，并生成操作绘图信息，生成的操作绘图信息用于绘制飞行模式打开后的显示窗口。
在上述第一方面的一种可能的实现中，上述方法还包括：第一操作包括以下操作中的至少一种：改变第一显示界面的大小的操作；对第一显示界面中的控件的操作。
例如，用户对第一系统平台上的第三窗口的第一操作可以是用户通过鼠标、键盘以及触摸等对“计算器”应用的桌面应用窗口中的显示内容(例如，显示界面的控件)、窗口状态以及窗口大小等进行的控制操作。其中，操作信息可以包括桌面应用窗口的尺寸、窗口区域位置坐标以及操作位置坐标等。
在上述第一方面的一种可能的实现中，上述方法还包括：第一系统平台和第二系统平台具有不同的操作系统。
在上述第一方面的一种可能的实现中，上述方法还包括：第一系统平台和第二系统平台运行于不同的电子设备上。
在上述第一方面的一种可能的实现中，上述方法还包括：第一系统平台为运行于电子设备上的虚拟机上的操作系统，第二系统平台为电子设备的操作系统。
第二方面，本申请实施里提供了一种应用的跨平台显示方法，其特征在于，包括：第二系统平台上运行有第一应用；第一系统平台从第二系统平台获取第一应用的第一窗口的第一尺寸；第一系统平台在第一系统平台中创建对应第一应用的第三窗口和第一标识，其中第一标识用于标识第三窗口用于显示第一应用的显示界面；第二系统平台从第一系统平台接收到第一标识后，生成第一绘图信息，其中第一绘图信息包括第一标识；第一系统平台从第二系统平台接收第一绘图信息，并根据第一绘图信息所包括的第一标识，在第三窗口中显示根据第一绘图信息所绘制的，第一应用的第一显示界面。
在上述第二方面的一种可能的实现中，上述方法还包括：第一系统平台在检测到用户在第一系统平台中打开第一应用的情况下，第一系统平台将第一应用信息发送给第二系统平台；第二系统平台根据第一应用信息运行第一应用，并在第二系统平台中创建第一应用的第一窗口；第二系统平台将第一应用的第一窗口的第一尺寸发送给第一系统平台。
在上述第二方面的一种可能的实现中，上述方法还包括：第二系统平台不显示第一窗口。
在上述第二方面的一种可能的实现中，上述方法还包括：第一系统平台检测到用户对第一系统平台上的第三窗口的第一操作；第一系统平台生成操作信息，并将操作信息发送给第二系统平台，其中，操作信息包括第一标识；第二系统平台通过执行操作信息对应的操作生成第三绘图信息，并将第三绘图信息发送给第一系统平台；第一系统平台基于第三绘图信息在第三窗口中显示第三显示界面，其中第三显示界面为第一显示界面对应第一操作改变后的显示界面。
在上述第二方面的一种可能的实现中，上述方法还包括：第一操作包括以下操作中的至少一种： 改变第一显示界面的大小的操作；对第一显示界面中的控件的操作。
在上述第二方面的一种可能的实现中，上述方法还包括：第一系统平台和第二系统平台具有不同的操作系统，其中，第一系统平台和第二系统包括以下至少一种：第一系统平台和第二系统平台运行于不同的电子设备上；第一系统平台为运行于电子设备上的虚拟机上的操作系统，第二系统平台为电子设备的操作系统。
第三方面，本申请实施里提供了一种应用的跨平台显示方法，其特征在于，包括：第二系统平台上运行有第一应用；第二系统平台向第一系统平台发送第一应用的第一窗口的第一尺寸；第二系统平台从第一系统平台接收到第一标识后，生成第一绘图信息，其中第一标识用于标识：第一系统平台中的第三窗口用于显示第一应用的显示界面，并且第一绘图信息包括第一标识，并且第一绘图信息能够被第一系统平台用于绘制在第三窗口中显示的第一应用的第一显示界面。
第四方面，本申请实施例提供了一种可读介质，可读介质上存储有指令，该指令在电子设备上执行时使电子设备执行上述第一方面至第三方面以及第一方面至第三方面的各种可能实现中的任意一种应用的跨平台显示方法。
第五方面，本申请实施例提供了一种电子设备，包括：
存储器，用于存储由电子设备的一个或多个处理器执行的指令，以及
处理器，是电子设备的处理器之一，用于执行上述第一方面至第三方面以及第一方面至第三方面的各种可能实现中的任意一种应用的跨平台显示方法。
附图说明
图1A根据本申请的一些实施例，示出了一种手机与电脑建立分布式显示连接示意图；
图1B根据本申请的一些实施例，示出了一种“设置”应用的分布式显示示意图；
图1C根据本申请的一些实施例，示出了一种多个应用的分布式显示示意图；
图1D根据本申请的一些实施例，示出了一种应用的显示窗口“放大”示意图；
图2根据本申请的一些实施例，示出了一种应用的跨平台显示系统的场景图；
图3根据本申请的一些实施例，示出了一种应用的跨平台显示系统的跨平台应用分布式显示交互图；
图4根据本申请的一些实施例，示出了另一种应用的跨平台显示系统的跨平台应用分布式显示交互图；
图5根据本申请的一些实施例，示出了一种启动手机代理进程的监听功能的示意图；
图6根据本申请的一些实施例，示出了一种启动桌面代理进程的监听功能的示意图；
图7根据本申请的一些实施例，示出了一种手机与电脑成功建立分布式显示连接的示意图；
图8根据本申请的一些实施例，示出了一种电脑显示界面显示安卓应用的示意图；
图9根据本申请的一些实施例，示出了另一种电脑显示界面显示安卓应用的示意图；
图10根据本申请的一些实施例，示出了一种“视频播放器”应用的分布式显示示意图；
图11根据本申请的一些实施例，示出了一种对安卓应用的桌面应用窗口进行“放大”操作的交互图；
图12根据本申请的一些实施例，示出了一种“计算器”应用的桌面应用窗口大小调整前后的示意图；
图13A根据本申请的一些实施例，示出了一种调整前的“设置”应用的桌面应用窗口的显示示意图；
图13B根据本申请的一些实施例，示出了一种调整后的“设置”应用的桌面应用窗口的显示示意图；
图14根据本申请的一些实施例，示出了一种对安卓应用的桌面应用窗口的显示内容进行控制操作的交互图；
图15根据本申请的一些实施例，示出了另一种跨平台应用分布式显示的场景图；
图16根据本申请的一些实施例，示出了另一种跨平台分布式显示的交互图；
图17根据本申请的一些实施例，示出了另一种启动桌面代理进程的监听功能的示意图；
图18根据本申请的一些实施例，示出了一种虚拟机与电脑成功建立分布式显示连接的示意图；
图19根据本申请的一些实施例，示出了另一种“视频播放器”应用的分布式显示示意图；
图20根据本申请的一些实施例，示出了一种电子设备的结构示意图；
图21根据本申请的一些实施例，示出了一种电子设备的软件结构框图；
图22根据本申请的一些实施例，示出了另一种电子设备的结构示意图；
图23根据本申请的一些实施例，示出了另一种电子设备的软件结构框图。
具体实施方式
本申请的说明性实施例包括但不限于应用的跨平台显示方法、可读介质和电子设备。
为了解决上述问题，本申请提供了一种应用的跨平台显示方法，可以适用于分别安装不同操作系统的两个电子设备，也可以适用于一个设备安装两个操作系统的情况。
例如，如图1A所示，针对安装安卓系统的手机100以及安装
Figure PCTCN2022104441-appb-000010
系统的电脑200，电脑200和手机100建立连接后，电脑200可以通过向手机100发送安卓应用“计算器”的参数来启动安卓应用“计算器”；手机100可以基于接收的“计算器”的参数生成“计算器”的安卓应用窗口，手机100可以将“计算器”的安卓应用窗口的参数发送给电脑200，其中，安卓应用窗口的参数包含安卓应用窗口的尺寸以及窗口区域位置坐标；电脑200基于接收的“计算器”的安卓应用窗口的参数创建桌面应用窗口并将桌面应用窗口的窗口句柄发送手机100，其中，桌面应用窗口的窗口句柄用于唯一标识桌面应用窗口；手机100使用接收的窗口句柄生成绘图信息并将绘图信息发送给电脑200；电脑200可以基于接收的绘图信息调用电脑200的图形库对桌面应用窗口进行图像渲染；如图1B所示，从而实现桌面应用窗口2003-2显示安卓应用“计算器”的内容。可以理解，根据上述方法，如图1C所示，电脑200可以启动多个安卓应用，创建多个安卓应用窗口并同时显示多个安卓应用的内容。
进一步地，电脑200还可以对桌面应用窗口2003-2中的显示内容或者窗口大小等进行控制操作。例如，电脑200对桌面应用窗口2003-2进行“放大”操作，生成操作信息；电脑200可以将操作信息发送给手机100，其中，操作信息可以包括桌面应用窗口的尺寸、窗口区域位置坐标以及操作位置坐标等；手机100可以基于操作信息调整“计算器”的安卓应用窗口，并生成操作绘图信息；手机100将生成的操作绘图信息发送给电脑200；电脑200基于操作绘图信息调用电脑200的图形库中的函数对桌面应用窗口进行图像渲染；如图1D所示，从而实现对桌面应用窗口2003-2的“放大”。
为使本申请的目的、技术方案和优点更加清楚，下面结合图1至图23详细说明本申请的技术方案。
图2根据本申请的一些实施例示出了一种应用的跨平台显示系统20的场景图。
具体地，如图2所示，应用的跨平台显示系统20包括第一电子设备100、第二电子设备200；第一电子设备100可以与第二电子设备200建立。其中，第一电子设备100可以运行第一操作系统11；第二电子设备200可以运行第二操作系统21。
如图2所示，第一电子设备100的显示界面110可以是第一操作系统11的用户操作界面。第二电子设备200的显示界面210可以是第二操作系统21的用户操作界面。
在一些实施例中，第一操作系统11可以是鸿蒙操作系统(harmony operating system，HarmonyOS)、苹果操作系统(iPhone operation system，iOS)、
Figure PCTCN2022104441-appb-000011
操作系统、
Figure PCTCN2022104441-appb-000012
phone操作系统、
Figure PCTCN2022104441-appb-000013
操作系统、
Figure PCTCN2022104441-appb-000014
操作系统等。第二操作系统21可以是
Figure PCTCN2022104441-appb-000015
操作系统、
Figure PCTCN2022104441-appb-000016
操作系统、MAC操作系统(operation system，OS)等。
在一些实施例中，第一电子设备100可以通过、蓝牙、WIFI、或NFC等无线通信方式与第二电子设备200通信连接。第一电子设备100也可以通过有线通信方式与第二电子设备200通信连接，例如，第一电子设备100通过数据线以及通用串行总线(Universal Serial Bus，USB)接口与第二电子设备200通信连接。
在一些实施例中，第一电子设备100和第二电子设备200可以是能够互相通信的各种计算设备，例如，第一电子设备100或第二电子设备200可以包括但不限于，膝上型计算机、台式计算机、平板计算机、手机、服务器、可穿戴设备、头戴式显示器、移动电子邮件设备、车机设备、便携式游戏机、便携式音乐播放器、阅读器设备、其中嵌入或耦接有一个或多个处理器的电视机、或能够访问网络的其他电子设备。
下文以第一电子设备100为手机100，第一操作系统为
Figure PCTCN2022104441-appb-000017
操作系统，第二电子设备200为电脑200，第二操作系统为
Figure PCTCN2022104441-appb-000018
操作系统为例说明本申请的方案。下文
Figure PCTCN2022104441-appb-000019
操作系统简称安卓系统，
Figure PCTCN2022104441-appb-000020
操作系统简称桌面系统。
图3根据本申请的一些实施例示出了一种应用的跨平台显示系统应用的跨平台显示系统20的跨平台应用分布式显示交互图。
如图3所示，应用的跨平台显示系统应用的跨平台显示系统20的应用的跨平台显示方法的过程包括以下步骤：
S301：手机100与电脑200建立分布式显示连接。
在本申请的一些实施例中，手机100与电脑200建立有线通信连接或无线通信后，电脑200可以获知手机100的验证码，并对手机100的验证码进行验证，从而实现与手机100建立分布式显示连接。其中，手机100的验证码可以是手机100的IP地址，也可以是手机100生成的随机码等。电脑200对手机100的验证码进行验证的过程下文做详细描述，在此不做赘述。
S302：电脑200检测到启动安卓应用的操作，获得启动的安卓应用的安卓应用的参数。
在本申请的一些实施例中，安卓应用的参数包括安卓应用的包名、类名，手机100可以根据安卓应用的参数打开安卓应用。例如，电脑200可以检测到用户在电脑200上打开安卓应用的操作，电脑200从而生成打开安卓应用的指令。
在本申请的一些实施例中，电脑200可以同步显示手机100的安卓应用。例如，手机100与电脑200建立分布式连接后，电脑200可以通过在电脑200的桌面上创建安卓应用的快捷方式从而实现电脑200同步显示安卓应用。电脑200可以通过检测到用户打开电脑200的桌面上创建安卓应用的快捷方式的操作，获得启动的安卓应用的参数。
在本申请的其他一些实施例中，电脑200可以在显示界面上创建安卓应用的显示按钮。电脑200还可以通过检测到用户打开电脑200的显示窗口上创建的安卓应用的显示按钮的操作，获得安卓应用的参数。
S303：电脑200将安卓应用的参数发送给手机100。
S304：手机100基于安卓应用的参数启动应用，创建安卓应用窗口。
在本申请的一些实施例中，手机100可以基于安卓应用的参数，启动安卓应用。其中，在启 动安卓应用的过程中，手机100会创建该应用的安卓应用窗口。在创建安卓应用窗口的过程中，手机100先将安卓应用窗口的参数发送给电脑200。电脑200可以基于安卓应用窗口的参数创建桌面应用窗口。其中，安卓应用窗口的参数包括但不限于安卓应用窗口的标识、安卓应用窗口的尺寸以及窗口区域位置坐标等。
在本申请的实施例中，安卓应用可以启动在手机100的虚拟屏上，也可以启动在手机100的显示屏上，当安卓应用启动在手机100的虚拟屏上时，可以实现电脑200启动安卓应用，并且启动安卓应用的显示内容只在电脑200上显示。例如，如图1B所示，安卓应用“计算器”的内容只在电脑200上显示。从而使得在不影响手机100正常使用的情况下实现跨平台应用分布式应用显示。
S305：手机100向电脑200发送安卓应用窗口的参数。
S306：电脑200基于安卓应用窗口的参数，创建桌面应用窗口。
在本申请的实施例中，电脑200基于安卓应用窗口的参数，创建桌面应用窗口，并为创建的桌面应用窗口分配窗口句柄。其中，分配的窗口句柄用于唯一标识创建的桌面应用窗口。
例如，电脑200创建10个桌面应用窗口，电脑200给每个桌面应用窗口分配一个窗口句柄；当电脑200对桌面应用窗口进行操作时，需要先获取该桌面应用窗口的窗口句柄才可以对该桌面应用窗口进行操作。
S307：电脑200向手机100发送桌面应用窗口的窗口句柄。
S308：手机100继续创建安卓应用窗口，并将安卓应用窗口与桌面应用窗口的窗口句柄关联。
在本申请的一些实施例中，手机100接收到桌面应用窗口的窗口句柄后，继续创建安卓应用窗口，并将安卓应用窗口与桌面应用窗口的窗口句柄关联。
例如，在手机100继续创建安卓应用窗口的过程中，安卓应用窗口与桌面应用窗口的窗口句柄关联，手机100可以使用窗口句柄生成的绘图信息，从而使得生成的绘图信息可以直接在电脑200上被执行。
S309：手机100使用窗口句柄创建绘图信息。
在本申请的一些实施例中，绘图信息用于调用嵌入式图形库(Embedded-System Graphics Library，EGL)、开放图像库(Open Graphics Library，OpenGL)等绘制安卓应用的显示内容。
在本申请的一些实施例中，手机100生成绘图信息既可以在手机100上执行实现安卓应用窗口的图像渲染，完成安卓应用的显示内容在桌面应用窗口上的显示。在本申请的其他一些实施例中，手机100生成绘图信息也可以不在手机100上执行，从而实现在不影响手机100正常使用的情况下也不消耗手机100的GPU性能。下文对此作详细描述，在此不做赘述。
例如，绘图信息用于调用嵌入式图形库(Embedded-System Graphics Library，EGL)或开放图像库(Open Graphics Library，OpenGL)绘制安卓应用的显示内容，其中，手机100通过将绘图信息发送给电脑200，电脑200执行绘图信息实现电脑200的桌面应用窗口显示渲染图像，从而实现安卓应用在电脑200的桌面应用窗口的显示。具体地，手机100将绘图信息发送给电脑200，电脑200执行绘图信息，以使用OpenGL进行绘制图像。其中，手机100创建渲染表面，即EGLSurface使用接口时，在NativeWindowType的选项中，手机100将电脑200发送的窗口句柄(例如，native_window)替换安卓应用窗口的窗口句柄，从而实现手机100创建的绘图信息可以在电脑200上执行，并且在电脑200打开多个安卓应用的桌面应用窗口的情况下，根据绘图信息包含的窗口句柄的信息，确定需要重新绘制图像的桌面应用窗口。
在一些实施例中，当电脑200使用OpenGL绘制结束后，使用eglSwapBuffers方法交换前后缓冲，将绘制内容显示到电脑200的显示屏上；当绘制结束不再需要使用EGL时，电脑200取消 eglMakeCurrent的绑定，销毁EGLDisplay、EGLSurface、EGLContext，释放EGL环境。
S310：手机100向电脑200发送绘图信息。
S311：电脑200执行绘图信息在桌面应用窗口上显示安卓应用的内容。
在本申请的一些实施例中，电脑200根据绘图信息调用电脑200的图像库，例如EGL、OpenGL等，实现对桌面应用窗口进行图像渲染，完成安卓应用的显示内容在桌面应用窗口上的显示。具体内容下文做详细描述，在此不做赘述。
由步骤S301至步骤S311可知，手机100只需要向电脑200发送安卓应用窗口的参数以及绘图信息，电脑200基于接收的安卓应用窗口的参数以及绘图信息创建桌面应用窗口并调用图形库绘制窗口显示内容，从而实现在电脑200上启动并显示安卓应用的内容。相较于现有的投屏技术的跨平台应用分布式显示，本申请的应用分布式显示不需要在设备之间传输大量视频图像数据即可实现分布式显示。
S312：电脑200检测到桌面应用窗口的控制操作，生成操作信息。
在本申请的实施例中，电脑200检测到用户对桌面应用窗口的控制操作，并生成操作信息。例如，用户通过鼠标、键盘以及触摸等对桌面应用窗口中的显示内容(例如，显示界面的控件)、窗口状态以及窗口大小等进行控制操作。其中，操作信息可以包括桌面应用窗口的尺寸、窗口区域位置坐标以及操作位置坐标等。
例如，如图1B所示，桌面应用窗口中的显示内容(例如，显示界面的控件)可以是桌面应用窗口2003-2中的数字或运算符号等；桌面应用窗口大小可以是桌面应用窗口2003-2在显示界面210的位置、桌面应用窗口2003-2的显示区域大小等；桌面应用窗口2003-2状态可以是打开、关闭、最小化等。
S313：电脑200将操作信息发送给手机100。
S314：手机100基于操作信息，生成操作绘图信息。
在本申请的一些实施例中，电脑200向手机100发送的操作信息包含安卓应用窗口的任务标识，通过该任务标识，在图1C所示的多窗口显示时，手机100可以确定需要重新绘制图像的安卓应用窗口。
如图1B所示，电脑200基于接收的安卓应用窗口的参数以及绘图信息创建桌面应用窗口并调用图形库绘制窗口显示内容，从而实现图1B中的桌面应用窗口2003-2的显示。因此，安卓应用窗口的窗口区域的绝对坐标与桌面应用窗口的窗口区域的绝对坐标具有映射关系，即手机100可以基于操作信息中的操作位置坐标对安卓应用窗口显示的内容进行重新绘制，生成操作绘图信息。具体内容参考步骤S310生成绘图信息的内容，在此不做赘述。
S315：手机100将操作绘图信息发送给电脑200。
S316：电脑200执行操作绘图信息，重新渲染桌面应用窗口显示的图像。具体内容参考步骤S311生成绘图信息的内容，在此不做赘述。
可以理解，根据上述描述的内容，电脑200基于接收的安卓应用窗口的参数以及绘图信息创建桌面应用窗口并调用图形库绘制窗口显示内容，从而实现图1B中的桌面应用窗口2003-2的显示。并且如图1B所示，手机100可以将“设置”应用启动并运行在手机100的虚拟屏上，从而实现在不影响手机100使用的情况下实现跨平台应用分布式应用显示。
进一步地，电脑200检测到桌面应用窗口的控制操作生成操作信息，手机100基于电脑200生成的操作信息生成操作绘图信息，电脑200基于操作绘图信息对桌面应用窗口进行重新绘制，在桌面应用窗口上显示控制操作后的安卓应用的内容。从而实现用户通过鼠标、键盘、触摸等可以对图1B所示的桌面应用窗口2003-2中的数字或运算符号进行操作，也可以移动图1B所示的桌面应用窗口2003-2在显 示界面21的位置，还可以放大、缩小、关闭、最小化图1B所示的桌面应用窗口2003-2。
在本申请的其他实施例中，电脑200可以包括桌面代理进程201，手机100可以包括手机代理进程101；图4示出了另一种应用的跨平台显示系统20的跨平台应用分布式显示交互图。如图4所示，跨平台应用分布式显示具体过程如下：
S401：手机100检测到手机代理进程101中启动监听的操作。
在本申请的一些实施例中，手机代理进程101用于提供手机100的IP地址，并且手机100通过手机代理进程101开启监听功能；手机100开启监听后，桌面系统21可以通过手机100的IP地址与安卓系统11建立连接。
在本申请的一些实施例中，手机代理进程101可以是安装在手机100的应用，打开手机代理进程101，手机代理进程101的显示界面可以显示关闭监听的按钮、启动监听的按钮以及手机100的IP地址等。
例如，如图5(a)所示，用户可以通过点击手机100的显示界面101上的“手机代理”图标启动手机代理进程101；启动手机代理进程101后，如图5(b)所示，用户可以通过点击显示界面102上的“启动监听”按钮开启手机代理进程101的监听功能。其中，如图5(c)所示,用户还可以通过查看手机100的系统通知确定手机代理进程101的监听功能是否已经打开。例如，如图5(d)所示,当手机代理进程101的监听功能打开时，会出现系统通知窗口103。
例如，如图5(b)所示，显示界面102中的监听地址10.1.1.1为手机100的IP地址，电脑200的桌面代理进程201可以根据手机100的IP地址与手机100的手机代理进程101建立连接，在本申请的其他实施例中，手机100的IP地址也可以设置为是11.1.1.1。需要说明的是，在具体实现过程中，本申请对手机100的IP地址的具体内容不做具体限定。
在本申请的其他实施例中，用户也可以通过其他方式启动手机100的手机代理进程101，并开启手机代理进程101的监听功能，根据实际应用情况本申请对此不做具体限定。
S402：电脑200检测到电脑200的桌面代理进程201中启动监听的操作。
在本申请的一些实施例中，桌面代理进程201可以用于输入手机100的IP地址，桌面代理进程201还可以用于提供手机100与电脑200的通信连接方式，桌面代理进程201还可以用于开启电脑200的监听功能。
在本申请的一些实施例中，桌面代理进程201可以是安装在电脑200的软件，打开桌面代理进程201，桌面代理进程201的显示界面220可以显示用于输入手机100的IP地址的输入框，桌面代理进程201的显示界面220还可以用于显示手机100与电脑200的通信连接方式，例如，USB连接或wifi连接；桌面代理进程201的显示界面220还可以显示启动监听的按钮，桌面代理进程201还可以用于显示桌面系统21与安卓系统11是否连接成功。
在本申请的实施例中，如图6(a)所示，用户可以通过点击电脑200的显示界面210上桌面代理的快捷方式图标211启动桌面代理进程201；启动桌面代理进程201后，如图6(b)所示，用户可以在“输入框”221上输入手机100的IP地址，然后点击“连接”按钮222开启桌面代理进程201的监听功能。
在本申请的其他实施例中，用户也可以通过其他方式启动电脑200的桌面代理进程201，并开启桌面代理进程201的监听功能；根据实际应用情况本申请对此不做具体限定。
S403：手机100的手机代理进程101与电脑200的桌面代理进程201之间发送/返回请求，建立分布式显示连接。
在本申请的一些实施例中，手机100与电脑200可以通过有线方式或无线方式进行网络连接。基于手机100与电脑200建立的网络连接，用户可以开启手机100的手机代理进程101的监听功能并获得手机100的IP地址；然后，用户在桌面系统21上输入手机100的IP地址并点击“连接”按钮，桌面代理进程201获知手机100的IP地址后，电脑200的桌面代理进程201与手机100的手机代理进程101发送/返回请求，其中发送/返回的请求可以socket请求(套接字请求)。socket请求用于无线网络或者有线网络中不同电子设备之间，不同电子设备的应用程序之间进行双向通信。
手机100的手机代理进程101与电脑200的桌面代理进程201之间发送/返回请求，建立分布式显示连接。如图7所示，手机100的手机代理进程101与电脑200的桌面代理进程201成功建立连接后，显示界面210会显示“连接成功”图标224，即桌面代理进程201成功打开监听功能；当手机100的手机代理进程101与电脑200的桌面代理进程201未成功建立分布式显示连接，显示界面210会显示“连接失败”图标，即桌面代理进程201未成功打开监听功能。
在上述S401至S403中，用户在电脑200的显示界面输入手机100的IP地址。电脑200的桌面代理进程201获知手机100的IP地址，并基于手机100的IP地址向手机100的手机代理进程101发送socket请求。在本申请的其他实施例中，用户也可以在电脑200的显示界面输入手机100的验证码。电脑200的桌面代理进程201获知手机100的验证码，并基于手机100的验证码向手机100的手机代理进程101发送socket请求。可以理解的是，手机100的验证码可以是手机100的IP地址，也可以是手机100生成的随机码。根据实际的应用，本申请对电脑200获知的手机100的验证码的类型不做具体限定。
S404：电脑200检测到桌面代理进程201上启动手机100的“设置”应用的操作。
为了方便描述，下文以用户在桌面代理进程201上启动手机100的“设置”应用为例进行说明。可以理解，手机100上其他应用也可以在桌面代理进程201上启动，启动过程具体过程可以参考桌面代理进程201上启动手机100的“设置”应用的过程，在此不做赘述。
在本申请的一些实施例中，可以在电脑200的桌面代理进程201的显示界面220上创建可以启动手机100的应用按钮，用户通过点击电脑200的桌面代理进程201的显示界面220上的应用按钮启动手机100的应用；并且，若电脑200的显示界面220上没有手机100的应用按钮时，用户也可以通过在电脑200的显示界面220的输入框中输入手机100的应用包名和/或类名启动手机100的应用。
例如，如图8所示，用户通过点击电脑200的桌面代理进程201的显示界面220上的“设置”按钮225启动手机100的“设置”应用。用户也可以通过在电脑200的“输入框”226和/或“输入框”227中分别输入包名和类名并点击“启动”按钮227打开手机100的应用。例如，用户也可以通过在显示界面220的“输入框”22中输入手机100的“设置”应用的包名com.android.settings，然后点击“启动”按钮228启动手机100的“设置”应用。
在本申请的其他一些实施例中，还可以在电脑200的显示界面210上创建手机100的应用的桌面快捷方式，用户可以通过点击桌面快捷方式图标打开手机100的应用。例如，如图9所示，用户通过点击显示界面210的“设置”应用的桌面快捷方式图标212启动手机100的“设置”应用。
例如，在一些实施例中，电脑200在显示界面210上创建手机100的应用的桌面快捷方式的过程可以包括：电脑200的桌面代理进程201向手机100的手机代理进程101发送应用查询指令；手机100的手机代理进程101收到应用查询指令后，手机100的手机代理进程101通过调用安卓 系统11的PackageManagerService服务的getAllPackages接口获取手机100上所有安装的应用包名，然后手机100的手机代理进程101通过getPackageManager()获取applicationPackageManager，再通过applicationPackageManager的getApplicationIcon接口查询应用包名，根据应用包名获取应用图标；手机100的手机代理进程101遍历所有的应用包名，获取手机100上每个应用的图标；手机代理进程101将手机100上每个应用的图标的参数发送给电脑200的桌面代理进程201；电脑200的桌面代理进程201调用微软32位平台的应用程序接口(Application Programming Interface，API)为手机100的每个应用创建桌面快捷方式。
S405：电脑200的桌面代理进程201将启动的手机100的应用的参数发送给手机代理进程101。
在一些实施例中，电脑200的桌面代理进程201将需要启动的安卓应用的参数发送给手机100的手机代理进程101，手机100的手机代理进程101可以通过该应用的参数打开手机100的应用。从而实现用户可以在电脑200上启动手机100的应用，进而使得用户体验更加便捷。
在本申请的一些实施例中，安卓应用的参数包括应用的包名、应用的类名。例如，用户在桌面代理进程201上启动手机100的“设置”应用，桌面代理进程201可以将“设置”应用的包名com.android.settings发送给手机代理进程101。
S406：手机100的手机代理进程101调用安卓系统11的接口，启动手机100的“设置”应用。
在本申请的一些实施例中，例如，手机代理进程101根据接收的“设置”应用的参数调用安卓系统11的ActivityTaskManagerService的startActivity接口启动手机100的“设置”应用。
S407：手机100的安卓系统11创建“设置”应用的安卓应用窗口。
例如，在本申请的一些实施例中，手机100代理进程1002调用安卓系统11的接口，启动手机100的“设置”应用；在“设置”应用启动过程中，安卓系统11会创建“设置”应用的安卓应用窗口并将创建的“设置”应用的安卓应用窗口的参数发送给手机100的手机代理进程101。在本申请的实施例中，手机100的“设置”应用可以启动在手机100的虚拟屏上，也可以启动在手机100的显示屏。
在本申请的一些实施例中，手机100的安卓系统11指定启动参数android.activity.windowingMode为freeform模式，并且手机100的安卓系统11指定以FLAG_ACTIVITY_NEW_TASK方式启动，即手机100的安卓系统11创建与“设置”应用的安卓应用窗口关联的任务。
S408：手机100的安卓系统11向手机100的手机代理进程101发送“设置”应用的安卓应用窗口的参数。
在一些实施例中，手机100的安卓系统11可以先将“设置”应用的安卓应用窗口的参数发送给手机100的手机代理进程101，手机100的手机代理进程101将“设置”应用的安卓应用窗口的参数。电脑200的桌面代理进程201可以根据“设置”应用的安卓应用窗口的参数创建“设置”应用的桌面应用窗口。
在本申请的一些实施例中，安卓应用窗口的参数可以包括安卓应用窗口的任务标识，安卓应用窗口的宽和高，安卓应用窗口的区域位置坐标等。安卓应用窗口的区域位置坐标可以表示为{A l，A r，A r，A b}，其中，A l为安卓应用窗口区域左边界的位置坐标，A t为安卓应用窗口区域上边界的位置坐标，A r为安卓应用窗口区域右边界的位置坐标，A l为安卓应用窗口下边界的位置坐标。
S409：手机100的手机代理进程101向电脑200的桌面代理进程201发送应用的安卓应用窗口的参数。
S410：电脑200的桌面代理进程201在桌面系统21上创建“设置”应用的桌面应用窗口。
在本申请的一些实施例中，例如，电脑200的桌面代理进程201基于手机100的手机代理进程101发送“设置”应用的安卓应用窗口的参数，桌面代理进程201在桌面系统21上创建“设置”应用的桌面应用窗口。
具体地，电脑200的桌面代理进程201查找当前所创建的“设置”应用的桌面应用窗口2003-1中是否有窗口关联“设置”应用的安卓应用窗口的任务标识；如果没有窗口关联该任务标识，则桌面代理进程201新建一个父窗口关联此任务标识，并且桌面代理进程201在新建的父窗口下创建“设置”应用的桌面应用窗口2003-1；如果有窗口关联该任务标识，则电脑200的桌面代理进程201在关联了此任务标识的父窗口下创建“设置”应用的桌面应用窗口。
可以理解，电脑200的桌面代理进程201基于“设置”应用的安卓应用窗口的参数，确定需要创建“设置”应用的桌面应用窗口在电脑200的显示界面210中的区域位置坐标，窗口大小等。
在一些实施例中，安卓应用窗口的任务标识用于唯一标识安卓应用窗口。电脑200基于安卓应用窗口的参数创建的桌面应用窗口与安卓应用窗口的任务标识关联。其中，安卓应用窗口的参数包括安卓应用窗口的任务标识。因此，桌面代理进程201在创建桌面应用窗口之前，通过查找当前创建的桌面应用窗口中是否有窗口关联“设置”应用的安卓应用窗口的任务标识，从而防止一个手机100的应用在电脑200上显示两个“设置”应用的桌面应用窗口。并且针对已创建的桌面应用窗口，当安卓应用窗口显示的图像或者窗口大小等调整时，电脑200的桌面代理进程201通过查找该安卓应用窗口的任务标识实现对已创建的桌面应用窗口的同步调整。下文对此做详细描述，在此不做赘述。
S411：电脑200的桌面系统21将“设置”应用的桌面应用窗口的窗口句柄发送给手机100的手机代理进程101。
在一些实施例中，桌面应用窗口的窗口句柄用于唯一标识该桌面应用窗口。例如，“设置”应用的桌面应用窗口的窗口句柄可以用于标识桌面应用窗口，桌面系统21对“设置”应用的桌面应用窗口进行操作时，通过获取“设置”应用的桌面应用窗口的窗口句柄对“设置”应用的桌面应用窗口进行操作。
S412：手机100的手机代理进程101将电脑200的“设置”应用的桌面应用窗口的窗口句柄发送给手机100的安卓系统11。
S413：电脑200的安卓系统11将“设置”应用的桌面应用窗口2003-1的窗口句柄关联手机100的安卓应用窗口。
在一些实施例中，电脑200的安卓系统11将“设置”应用的桌面应用窗口的窗口句柄关联手机100的安卓应用窗口，可以使得手机100对“设置”应用的安卓应用窗口进行操作可以同步到电脑200的桌面应用窗口2003-1上。
S414：手机100的安卓系统11使用窗口句柄创建绘图信息。
在本申请的一些实施例中，绘图信息可以是安卓系统11使用“设置”应用的桌面应用窗口的窗口句柄创建的绘图信息。绘图信息可以是安卓系统11使用“设置”应用的桌面应用窗口的窗口句柄以及安卓应用窗口的窗口句柄创建的绘图信息。绘图信息用于在电脑200的桌面系统21上执行并在该窗口句柄标识的桌面应用窗口上渲染图像。
例如，在本申请的一些实施例中，手机100的安卓系统11接收到手机100的“设置”应用的桌面应用窗口的窗口句柄后，安卓系统11将桌面应用窗口2003-1的窗口句柄关联“设置”应用的安卓应用窗口， 并继续创建“设置”应用的安卓应用窗口。安卓系统11在继续创建“设置”应用的安卓应用窗口时，需要生成绘图信息对安卓应用窗口的显示内容进行图像渲染。该绘图信息用于调用嵌入式图形库(Embedded-System Graphics Library，EGL)、开放图像库(Open Graphics Library，OpenGL)等绘制安卓应用的显示内容。
由上述描述可知，由于手机100的安卓系统11使用关联的窗口句柄作为安卓应用窗口的参数来创建绘图信息，当该绘图信息在桌面应用窗口执行时不会出现找不到执行该绘图信息的桌面应用窗口的问题，并且相较于绘图信息在安卓系统11上执行并渲染图像，该绘图信息在桌面应用窗口上执行并渲染图像属于当前屏幕渲染(on-screen Rending),可以减少一个缓冲区操作，提升电脑200图像渲染操作时的GPU性能。其中，绘图信息在桌面应用窗口上执行并渲染图像方式属于当前屏幕渲染(on-screen Rending)，绘图信息在安卓系统11上执行并渲染图像的方式属于离屏渲染方式(off-screen Rending)。
S415：手机100的安卓系统11将绘图信息发送给电脑200的桌面代理进程201。
S416：电脑200的桌面代理进程201执行绘图信息。
在本申请的一些实施例中，电脑200的桌面代理进程201执行绘图信息，通过图形设备接口调用图形服务中的图像库，例如EGL、OpenGL等，从而实现在步骤S410创建“设置”应用的桌面应用窗口上显示图像渲染，完成安卓应用的显示内容在桌面应用窗口上的显示。例如，如图1B所示的“设置”应用的桌面应用窗口2003-2显示的内容。
基于步骤S404至S416中描述的电脑200的桌面代理进程201启动应用创建桌面应用窗口2003-2以及桌面应用窗口2003-2上的显示内容的图像渲染的过程，电脑200还可以启动其他安卓应用并显示其他安卓应用的内容。例如，如图1C所示，电脑200的显示界面210上可以同时显示“设置”应用的桌面应用窗口2003-1、“计算器”应用的桌面应用窗口2003-2、“邮件”应用的桌面应用窗口2003-3。
手机100的安卓系统11接收桌面应用窗口2003-1的窗口句柄，并将桌面应用窗口2003-1的窗口句柄与手机100的“设置”应用的安卓应用窗口关联。可以理解的是，针对手机100的多个应用，安卓系统11可以将手机100的应用的桌面应用窗口2003的窗口句柄与安卓应用窗口进行一一进行关联，从而可以实现在桌面系统21上打开并同时显示手机100的多个应用的桌面应用窗口2003。例如，如图1C所示，电脑200的显示界面210同时显示手机100的“设置”应用的桌面应用窗口2003-1、“计算器”应用的桌面应用窗口2003-2、“邮件”应用的桌面应用窗口2003-3。
在本申请的其他一些实施例中，电脑200的桌面代理进程201还可以实时接收绘图信息，对桌面应用窗口的显示内容进行渲染，从而使得可以在电脑200的桌面应用窗口上显示动态图像内容。如图10所示，电脑200的显示界面210上显示的桌面应用窗口2003-4可以是打开的手机100的“视频播放器”应用的桌面应用窗口2003-4；手机100的“视频播放器”应用的安卓应用窗口可以显示在手机100的虚拟屏上，而手机100的显示屏上显示的是实时抓取的手机性能的数据。由图10所示的手机100的显示屏实时显示的内容可知，桌面应用窗口2003-4执行绘图信息实现视频图像渲染并消耗电脑200的GPU资源，安卓应用窗口不执行该指令从而不消耗手机100的GPU资源。
在本申请的一些实施例中，由步骤S404至步骤S413可知，电脑200的桌面代理进程201可以在桌面系统21上创建“视频播放器”应用的桌面应用窗口；手机100的安卓系统11也可以将“视频播放器”应用的桌面应用窗口的窗口句柄关联手机100的“视频播放器”应用的安卓应用窗口上。手机100的安卓系统11可以使用“视频播放器”应用的桌面应用窗口的窗口句柄创建绘图信息，并将该绘图信息发送给电脑200的桌面系统21，电脑200的桌面系统21执行该绘图信息并在“视频 播放器”应用的桌面应用窗口上渲染图像。
例如，电脑200的显示界面210显示的应用窗口为手机100的“视频播放器”应用的桌面应用窗口2003-4并且该窗口显示正在播放视频；由上述步骤可知，手机100的“视频播放器”的桌面应用窗口2003-4的窗口句柄可以与“视频播放器”的安卓应用窗口关联，安卓系统11使用关联的窗口句柄作为“视频播放器”的安卓应用窗口的参数来创建绘图信息。
可以理解的是，例如，针对一个安卓应用“设置”，该应用的安卓应用窗口关联桌面应用窗口的窗口句柄，该应用的桌面应用窗口是基于安卓应用窗口的参数创建的，其中，安卓应用窗口的参数包含安卓应用窗口的任务标识。因此，安卓系统11使用“设置”应用的桌面应用窗口的窗口句柄创建的绘图信息不仅可以在桌面系统21上运行，桌面系统21还可以根据该绘图信息确定需要显示使用该绘图信息渲染的图像的桌面应用窗口2003-3。并且，当电脑200的桌面代理进程201检测到桌面应用窗口2003-3的控制操作，生成操作信息。下文对在电脑200上对安卓应用进行控制进行做详细说明，在此不做赘述。
由图4的应用的跨平台显示系统20的跨平台应用分布式显示交互流程可知，手机100的安卓系统11使用接收的窗口句柄生成绘图信息并将绘图信息发送给电脑200的桌面代理进程201，电脑200桌面代理进程201可以基于接收的绘图信息调用电脑200的图形库对桌面应用窗口进行图像渲染；从而实现创建的桌面应用窗口2003-2显示显示安卓应用“计算器”的内容。并且手机100的安卓系统11使用接收的窗口句柄生成绘图信息还可以使得在电脑200的显示界面上可以显示多个安卓应用的桌面应用窗口，实现同时打开并显示多个安卓应用。例如，如图1B所示，从而实现桌面应用窗口2003-2显示安卓应用“计算器”的内容。可以理解，根据上述方法，如图1C所示，电脑200可以启动多个安卓应用，创建多个安卓应用窗口并同时显示多个安卓应用的内容。
在本申请的其他一些实施例中，用户可以通过鼠标、键盘以及触摸等对桌面应用窗口中的显示内容(例如，显示界面的控件)、窗口状态以及窗口大小等进行控制操作，实现安卓应用不仅可以在电脑200上显示，还可以在电脑200上对手机100的安卓应用进行控制。
下面详细介绍在电脑200上对手机100的安卓应用进行控制的几种类型。
例如，如图1B所示，用户可以通过鼠标、触摸等对桌面应用窗口中的窗口“放大”操作，电脑200可以检测到用户对桌面应用窗口的“放大”操作。图11示出了一种对安卓应用的桌面应用窗口进行“放大”操作的交互图。
S1101：电脑200的桌面系统21检测到桌面应用窗口的控制操作，生成窗口区域位置坐标以及操作位置坐标。
例如，如图1B所示，电脑200的显示界面210上显示手机100的“计算器”应用的桌面应用窗口2003-2，用户可以通过鼠标或者触摸等点击放大图标对桌面应用窗口2003-2进行放大操作；电脑200的桌面系统21检测到桌面应用窗口的控制操作后，电脑200的桌面系统21调整桌面应用窗口的区域位置坐标；电脑200的桌面系统21将调整前后的桌面应用窗口的区域位置坐标以及电脑200的桌面系统21检测到桌面应用窗口的控制操作的位置坐标发送给手机100的手机代理进程101。
在本申请的一些实施例中，电脑200的桌面系统21检测到桌面应用窗口的控制操作，电脑200的桌面系统21对“计算器”应用的桌面应用窗口2003-2的窗口大小进行调整。并且电脑200的桌面系统21生成窗口区域位置坐标以及操作位置坐标。
例如，电脑200的桌面系统21检测到桌面应用窗口的控制操作后，桌面应用窗口2003-2的区域坐标由[W l，W t，W r，W b]调整为[W’ l，W’ t，W’ r，W’ b]，其中，W l为调整前的“计算器”应用的 桌面应用窗口区域左边界的位置坐标，W t为调整前的“计算器”应用的桌面应用窗口区域上边界的位置坐标，W r为调整前的“计算器”应用的桌面应用窗口区域右边界的位置坐标，W b为调整前的“计算器”应用的桌面应用窗口区域下边界的位置坐标；W’ l为调整后的“计算器”应用的桌面应用窗口区域左边界的位置坐标，W’ t为调整后的“计算器”应用的桌面应用窗口区域上边界的位置坐标，W’ r为调整后的“计算器”应用的桌面应用窗口区域右边界的位置坐标，W’ b为调整后的“计算器”应用的桌面应用窗口区域下边界的位置坐标。
图12为“计算器”应用的桌面应用窗口2003-2大小调整前后的示意图。如图12(a)所示，桌面应用窗口2003-2的原始区域坐标为[W l，W t，W r，W b]。如图12(b)所示，用户桌面应用窗口2003-2点击窗口的“放大”图标后，桌面应用窗口2003-2区域坐标调整为[W’ l，W’ t，W’ r，W’ b]。可以理解，电脑200的桌面系统21只是对窗口的大小进行调整，电脑200的桌面系统21可以基于手机100发送的操作绘图信息对放大后的桌面应用窗口2003-2的显示内容图像进行渲染。
下文以电脑200的桌面系统21检测到手机100的“计算器”应用的桌面应用窗口2003-2的“放大”操作，并生成“放大”窗口区域位置坐标以及操作位置坐标为例进行说明。可以理解，电脑200的桌面系统21也可以检测到手机100的其他应用窗口的大小进行操作，具体内容参考电脑200的桌面系统21检测到手机100的“计算器”应用的桌面应用窗口2003-2的“放大”操作，本申请在此不做一一赘述。
S1102：电脑200的桌面系统21将窗口区域位置坐标以及操作位置坐标发送给手机100的手机代理进程101。
S1103：手机100的手机代理进程101调用安卓接口，重新调整“计算器”应用的安卓应用窗口大小及位置坐标。
在本申请的一些实施例中，手机100的手机代理进程101基于接收的窗口区域位置坐标以及操作位置坐标，通过调用ActivityTaskManagerService的resize Task接口，重新设置“计算器”应用的安卓应用窗口关联的任务标识对应的任务的大小及位置坐标，从而实现调整“计算器”应用的安卓应用窗口。其中，窗口区域位置坐标以及操作位置坐标包含手机100的“计算器”应用的安卓应用窗口的任务标识。
S1104：手机100的安卓系统11基于窗口区域位置坐标以及操作位置坐标调整安卓应用窗口的窗口区域位置坐标，并生成操作绘图信息。
例如，在本申请的一些实施例中，用户对手机100的“计算器”应用的桌面应用窗口2003-2进行“放大”操作，电脑200的桌面系统21检测到桌面应用窗口2003-2的“放大”操作，电脑200的桌面系统21生成“放大”操作的窗口区域位置坐标以及操作位置坐标，桌面系统21将“放大”的控制操作信息发送给手机100的手机代理进程101；手机100的手机代理进程101基于接收的窗口区域位置坐标以及操作位置坐标，通过调用ActivityTaskManagerService的resize Task接口调整安卓应用窗口的位置坐标。其中，“放大”的窗口区域位置坐标以及操作位置坐标包含手机100的“计算器”应用的安卓应用窗口的任务标识。
在本申请的一些实施例中，手机100的安卓系统11基于“放大”的窗口区域位置坐标以及操作位置坐标，通过调用ActivityTaskManagerService的resize Task接口对“计算器”应用的安卓应用窗口关联的任务标识对应的任务执行控制操作，调整“计算器”应用的安卓应用窗口的区域位置坐标；“计算器”应用的安卓应用窗口的区域位置坐标由[A l，A t，A r，A b]调整为[A’ l，A’ t，A’ r，A’ b]，其中，A l为调整前的“计算器”应用的安卓应用窗口区域左边界的位置坐标，A t为调整前的“计算器”应用的 安卓应用窗口区域上边界的位置坐标，A r为调整前的“计算器”应用的安卓应用窗口区域右边界的位置坐标，A b为调整前的“计算器”应用的安卓应用窗口区域下边界的位置坐标；A’ l为调整后的“计算器”应用的安卓应用窗口区域左边界的位置坐标，A t为“计算器”应用的安卓应用窗口区域上边界的位置坐标，A r为“计算器”应用的安卓应用窗口区域右边界的位置坐标，A b为“计算器”应用的安卓应用窗口区域下边界的位置坐标。
在本申请的一些实施例中，手机100的安卓系统11基于窗口区域位置坐标以及操作位置坐标调用系统服务接口调整安卓应用窗口的区域坐标，调整后的安卓应用窗口的区域坐标[A’ l，A’ t，A’ r，A’ b]需要满足如下公式(1)和公式(2)的计算关系：
Figure PCTCN2022104441-appb-000021
Figure PCTCN2022104441-appb-000022
可以理解，手机100的安卓系统11调整“计算器”应用的安卓应用窗口的区域坐标，并生成操作绘图信息。该操作绘图信息用于绘制放大后的“计算器”应用的桌面应用窗口显示的内容。
S1105：手机100的安卓系统11将操作绘图信息发送给电脑200的桌面代理进程201。
S1106：电脑200的桌面代理进程201执行操作绘图信息。
例如，电脑200的桌面代理进程201基于操作绘图信息在桌面系统21上绘制放大后的“计算器”应用的桌面应用窗口2003-2。具体内容参考图4的步骤S415生成绘图信息的内容，在此不做赘述。
可以理解，电脑200也可以检测到用户对桌面应用窗口的“缩小”操作，具体过程请参考上述图11描述的过程，在此不做赘述。
下面介绍在电脑200上对手机100的安卓应用的显示内容(例如，显示界面的控件)进行控制操作。例如，如图13所示，如图13A所示，电脑200可以检测到用户点击桌面应用窗口2003-1的按钮图标218的操作，调整桌面应用窗口2003-1的显示内容为图13B所示的内容，并且手机100的选项4由关闭状态变更为打开状态。下面结合图14详细描述安卓应用的桌面应用窗口的操作的控制的交互图。
图14示出了一种对安卓应用的桌面应用窗口的显示内容进行控制操作的交互图。如图14所示，实现安卓应用的桌面应用窗口的显示内容的控制操作具体过程如下：
S1401：电脑200的桌面系统21检测到桌面应用窗口的控制操作，生成操作位置坐标。
例如，根据图4所示描述的过程，电脑200可以创建“设置”应用的桌面应用窗口2003-1，并在桌面应用窗口2003-1上显示安卓应用“设置”的内容，安卓应用“设置”在电脑200的显示界面210上显示。如图13A所示，电脑200的桌面系统21可以检测到用户通过鼠标点击“设置”应用的按钮图标218，生成操作位置图标。其中，生成的操作位置图标可以是按钮图标218所在区域的位置坐标。
S1402：电脑200的桌面系统21将操作位置坐标发送给手机100的手机代理进程101。
S1403：手机100的手机代理进程101将操作位置坐标发送给手机100的安卓系统11。
S1404：手机100的安卓系统11基于操作位置坐标生成操作绘图信息。
在本申请的一些实施例中，根据图4的分布式显示过程，“设置”应用的桌面应用窗口的区域位置坐标与安卓应用窗口的区域位置坐标的相对位置坐标具有映射关系。例如，如图13A所示，手机100的安卓系统11基于“设置选项4”的按钮图标218的区域位置坐标检测到对设置选项4的操作。手机100的安卓系统11通过调用InputManagerService服务接口对“设置”应用的安卓应用窗口关联的任务标识对应的任务执行控制操作，并生成操作绘图信息。例如，设置选项4为飞行模 式，按钮图标218为飞行模式的状态开关按钮。手机100的安卓系统11通过调用InputManagerService服务接口打开飞行模式，并生成操作绘图信息，生成的操作绘图信息用于绘制飞行模式打开后的显示窗口。
S1405：手机100的安卓系统11将操作绘图信息发送给电脑200的桌面代理进程201。
S1406：电脑200的桌面代理进程201执行操作绘图信息。
例如，如图13B所示，电脑200的桌面代理进程201基于操作绘图信息在桌面系统21上绘制关闭设置选项4后的“设置”应用的桌面应用窗口2003-1。具体内容参考图4的步骤S415生成绘图信息的内容，在此不做赘述。
可以理解，图3所示的应用的跨平台显示方法可以适用于分别安装不同操作系统的电子设备100以及电子设备200。在本申请的其他实施例中，应用的跨平台显示方法也可以适用于一个电子设备上运行两个操作系统的场景。
图15根据本申请的一些实施例示出了另一种跨平台应用分布式显示的场景。图15所示的场景包括第三电子设备300；其中，第三电子设备300运行第二操作系统21，并且第三电子设备300还安装了运行第一操作系统11的虚拟机400。第一操作系统11与第二操作系统21的描述请参考上文描述，在此不做赘述。
如图15(a)所示，用户可以通过鼠标或触摸等双击第三电子设备300的显示界面320的“移动应用引擎”应用快捷方式图标321，打开图15(b)所示的运行第一操作系统11的虚拟机400的用户操作界面410。
在本申请的一些实施例中，第三电子设备300可以是能够同时运行第一操作系统11和第二操作系统21的电子设备，例如，第三电子设备300可以包括但不限于手机、笔记本电脑、平板电脑、桌上型电脑、膝上型电脑、超级移动个人计算机(Ultra-mobile Personal Computer,UMPC)、手持计算机、上网本、个人数字助理(Personal Digital Assistant，PDA)等电子设备。
下文以第三电子设备300为电脑300，第一操作系统11为安卓系统11，第二操作系统21为桌面系统21为例进行说明另一种跨平台应用分布式显示过程；图16为根据本申请的实施例，示出了另一种跨平台分布式显示的交互图；如图16所示，跨平台应用分布式显示具体过程如下：
S1601：虚拟机400检测到手机代理进程101中启动监听的操作。
在本申请的一些实施例中，电脑300的桌面系统21上安装安卓系统11的虚拟机，用户可以通过鼠标或触摸等点击图15(a)的显示界面320的“移动应用引擎”应用快捷方式图标321打开安卓系统11的虚拟机400的显示界面410。如图15(b)所示，用户可以通过鼠标或触摸等点击虚拟机400的显示界面401上的“手机代理”图标启动手机代理进程101。其中，开启手机代理进程101的监听功能的具体内容参考步骤S401，在此不做赘述。
S1602：电脑300检测到电脑300的桌面代理进程201中启动监听的操作。
在本申请的一些实施例中，如图17(a)所示，用户可以通过点击电脑300的显示界面330上桌面代理的快捷方式图标331启动桌面代理进程201，具体过程参考步骤S402，在此不做赘述。
在本申请的一些实施例中，如图17(a)所示，用户可以通过点击电脑300的显示界面320上桌面代理的快捷方式图标331启动桌面代理进程331；启动桌面代理进程后，如图17(b)所示，用户可以点击显示界面340上的“USB”的按钮341，“USB”的按钮341选项表示桌面代理进程201可以通过USB的连接方式与手机代理进程101通信连接，然后，用户点击“连接”按钮342，电脑300的桌面代理进程201启动监听的操作。
在本申请的其他实施例中，用户也可以通过其他方式启动电脑300的桌面代理进程201，并 开启桌面代理进程201的监听功能；根据实际应用情况本申请对此不做具体限定。
S1603：虚拟机400的手机代理进程101与电脑300的桌面代理进程201之间发送/返回请求，建立分布式显示连接。
在本申请的一些实施例中，由于虚拟机400启动在桌面系统21上，手机代理进程101和桌面代理进程201可以通过网卡地址连接，如图17(b)所示，当用户点击显示界面340上的“USB”的按钮341时，桌面代理进程201配置成安卓调试桥(android debug bridge，adb)端口映射，然后，用户点击“连接”按钮342，电脑200的桌面代理进程201与手机100的手机代理进程101发送/返回请求，其中发送/返回的请求可以socket请求(套接字请求)。其中，本地回环地址可以是127.0.0.1。
如图18所示，当虚拟机400的手机代理进程101与电脑300的桌面代理进程201成功建立连接时，显示界面340会显示“连接成功”图标343。
S1604：电脑300检测到桌面代理进程201上启动虚拟机400的应用的操作，具体内容参考步骤S404，在此不做赘述。
S1605：电脑300的桌面代理进程201将启动的虚拟机400的安卓应用的参数发送给手机代理进程101。具体内容参考步骤S405，在此不做赘述。
S1606：虚拟机400的手机代理进程101调用安卓系统11的接口，启动虚拟机400的应用。具体内容参考步骤S406，在此不做赘述。
S1607：虚拟机400的安卓系统11创建应用的安卓应用窗口。具体内容参考步骤S407，在此不做赘述。
S1608：虚拟机400的安卓系统11向虚拟机400的手机代理进程101发送安卓应用窗口的参数。
S1609：虚拟机400的手机代理进程101向电脑300的桌面代理进程201发送应用的安卓应用窗口的参数。
S1610：电脑300的桌面代理进程201基于安卓应用窗口的参数，在桌面系统21上创建应用的桌面应用窗口。具体内容参考步骤S410，在此不做赘述。
S1611：电脑300的桌面系统21将应用的桌面应用窗口的窗口句柄发送给虚拟机400的手机代理进程101。
S1612：虚拟机400的手机代理进程101将电脑300的应用的桌面应用窗口的窗口句柄发送给虚拟机400的安卓系统11。
S1613：电脑300的安卓系统11将应用的桌面应用窗口的窗口句柄关联虚拟机400的安卓应用窗口。
在本申请的一些实施例中，安卓系统11通过接收桌面应用窗口的窗口句柄，并将桌面应用窗口的窗口句柄与虚拟机400的应用的安卓应用窗口关联。可以实现虚拟机400的多个应用在多个桌面应用窗口上显示，其中，每个虚拟机400的应用的安卓应用窗口与每个桌面应用窗口一一关联。
S1614：虚拟机400的安卓系统11使用窗口句柄创建绘图信息。具体内容参考步骤S414，在此不做赘述。
S1615：虚拟机400的安卓系统11将绘图信息发送给电脑300的桌面代理进程201。
S1616：电脑300的桌面代理进程201在桌面系统21上执行使用窗口句柄创建的绘图信息。具体内容参考步骤S416，在此不做赘述。
例如，如图19所示，电脑300的显示界面320上显示虚拟机400的“视频播放器”应用的桌面应用窗口3003。
可以理解，电脑300的显示界面320上显示的桌面应用窗口可以是打开的虚拟机400的“视频播放器”应用的桌面应用窗口3003；相较于绘图信息在安卓系统11上执行并渲染图像，该绘图信息在桌面应用窗口上执行并渲染图像属于当前屏幕渲染(on-screen Rending),可以减少一个缓冲区操作，提升电脑300图像渲染操作时的GPU性能。其中，绘图信息在桌面应用窗口上执行并渲染图像方式属于当前屏幕渲染(on-screen Rending),绘图信息在安卓系统11上执行并渲染图像的方式属于离屏渲染方式(off-screen Rending)。
在本申请的一些实施例中，电脑300的桌面也可以同时显示虚拟机400的多个安卓应用的显示窗口，每个窗口的创建和图像渲染的过程参考图16的内容，在此不做赘述。
在本申请的其他一些实施例中，用户也可以通过鼠标、键盘以及触摸等对电脑300的桌面应用窗口中的显示内容(例如，显示界面的控件)、窗口状态以及窗口大小等进行控制操作，实现虚拟机400的安卓应用不仅可以在电脑300上显示，还可以在电脑300上对手机100的安卓应用进行控制。具体的过程请参考图11或图13描述的过程，在此不做赘述。
可以理解，虚拟机400的应用在桌面系统21的桌面应用窗口上显示，既可以实现打开并同时显示虚拟机400的多个应用的桌面应用窗口；并且当桌面应用窗口播放视频需要实时渲染图像时，虚拟机400的安卓系统11生成的绘图信息可以在桌面系统21上执行，从而实现离屏渲染方式渲染桌面应用窗口显示的图像，从而可以减少一个缓冲区操作，提升电脑300图像渲染操作时的GPU性能；并且用户还可以对显示虚拟机400的应用的桌面应用窗口进行放大、缩小、移动等操作，对窗口操作更加便捷，进而提升用户体验。
图20示出了电子设备100的一种结构示意图。
如图20所示，电子设备100可以包括处理器110，存储器121，通用串行总线(universal serial bus，USB)接口130，电源管理模块141，天线1，天线2，移动通信模块150，无线通信模块160，音频模块170，显示屏194，以及摄像头193等。
可以理解的是，本发明实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者分解某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
USB接口130是符合USB标准规范的接口，具体可以是Mini USB接口，Micro USB接口，USB Type C接口等。在本申请的一些实施例中，电子设备100可以通过USB接口130与电子设备200进行通信连接。
电源管理模块141用于连接电池，充电管理模块与处理器110。电源管理模块141接收电池和/或充电管理模块的输入，为处理器110，存储器121，显示屏194，摄像头193，和无线通信模块160等供电。
电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块150，无线通信模块160，调制解调处理器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
无线通信模块160可以提供应用在电子设备100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。
无线通信模块160经由天线2接收电磁波，将电磁波信号调频以及滤波处理，将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号，对其进行调频，放大，经天线2转为电磁波辐射出去。上文中的性能参数可以包括Wi-Fi网络的可利用的网络带宽，其中，Wi-Fi网络的可利用的网络带宽为用于传输子任的网络带宽。
可以理解，在本申请一些实施例中，电子设备100和电子设备200可以通过无线通信模块160进行通信连接；电子设备100和电子设备200进行跨平台分布式显示，电子设备200可以向电子设备100发送应用的参数，电子设备100也可以向电子设备200发送安卓应用窗口的参数，基于该参数，电子设备200可以创建窗口显示电子设备100的应用的内容。
电子设备100通过GPU，显示屏194，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏194和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
显示屏194用于显示图像，视频等。在一些实施例中，启动的安卓应用可以在显示屏194上启动，则电子设备100创建的安卓应用窗口以及渲染图像可以在显示屏194上显示。
存储器121可以用于存储计算机可执行程序代码，所述可执行程序代码包括指令。存储器121可以包括存储程序区和存储数据区。
可以理解的是，本发明实施例示意的结构并不构成电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
图21是本发明实施例的电子设备100的软件结构框图。
分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。在一些实施例中，将Android系统分为四层，从上至下分别为应用程序层，应用程序框架层，安卓运行时(Android runtime)和系统库，以及内核层。
应用程序层可以包括一系列应用程序包。
图21所示，应用程序包可以包括相机，图库，日历，通话，地图，导航，WLAN，蓝牙，音乐，视频，短信息等应用程序。
应用程序框架层为应用程序层的应用程序提供应用编程接口(application programming interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。
图21所示，应用程序框架层可以包括窗口管理器，内容提供器，视图系统，电话管理器，资源管理器，通知管理器等。
例如，窗口管理器可以包含电子设备100的手机代理服务101，用于管理窗口程序。窗口管理器可以根据安卓应用的参数调用接口创建桌面应用窗口。窗口管理器还可以将安卓应用窗口的参数、绘图信息发送给内部的手机代理服务101，手机代理服务101将安卓应用窗口的参数、绘图信息发送给其他电子设备。
内容提供器用来存放和获取数据，并使这些数据可以被应用程序访问。所述数据可以包括视频，图像，音频，拨打和接听的电话，浏览历史和书签，电话簿等。
视图系统包括可视控件，例如显示文字的控件，显示图片的控件等。视图系统可用于构建应用程序。显示界面可以由一个或多个视图组成的。例如，包括短信通知图标的显示界面，可以包括显示文字的视图以及显示图片的视图。
资源管理器为应用程序提供各种资源，比如本地化字符串，图标，图片，布局文件，视频文件等等。
在本发明的实施例中，资源管理器还可以用于存储Overlay配置文件。
通知管理器使应用程序可以在状态栏中显示通知信息，可以用于传达告知类型的消息，可以短暂停留后自动消失，无需用户交互。比如通知管理器被用于告知下载完成，消息提醒等。通知管理器还可以是以图表或者滚动条文本形式出现在系统顶部状态栏的通知，例如后台运行的应用程序的通知，还可以是以对话窗口形式出现在屏幕上的通知。例如在状态栏提示文本信息，发出提示音，电子设备振动，指示灯闪烁等。
Android Runtime包括核心库和虚拟机。Android runtime负责安卓系统的调度和管理。
核心库包含两部分：一部分是java语言需要调用的功能函数，另一部分是安卓的核心库。
应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的java文件执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
系统库可以包括多个功能模块。例如：表面管理器(surface manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，2D图形引擎(例如：SGL)等。
表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了2D和3D图层的融合。
媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。
三维图形处理库用于实现三维图形绘图，图像渲染，合成，和图层处理等。
2D图形引擎是2D绘图的绘图引擎。
内核层是硬件和软件之间的层。内核层至少包含显示驱动，摄像头驱动，音频驱动，传感器驱动。
图22根据本申请的实施例示出了一种电子设备200或电子设备300的结构框图。
具体地，如图22所示，电子设备200或电子设备300可以包括处理器201、存储器202、无线通信处理模块203、电源开关204、有线LAN通信处理模块205、显示屏206和音频模块207。其中：
处理器201可用于读取和执行计算机可读指令。具体实现中，处理器201可主要包括控制器、运算器和寄存器。其中，控制器主要负责指令译码，并为指令对应的操作发出控制信号。运算器主要负责执行定点或浮点算数运算操作、移位操作以及逻辑操作等，也可以执行地址运算和转换。寄存器主要负责保存指令执行过程中临时存放的寄存器操作数和中间操作结果等。具体实现中，处理器201的硬件架构可以是专用集成电路(ASIC)架构、MIPS架构、ARM架构或者NP架构等等。
在一些实施例中，处理器201可以用于解析无线通信处理模块203或有线LAN通信处理模块205接收到的信号，如电子设备100发送的安卓应用窗口参数、绘图信息等。处理器201可以根据安卓应用窗口参数创建桌面应用窗口；处理器201还可以执行绘图信息渲染图像，从而使得渲染的图像可以在桌面应用窗口上显示。
存储器202与处理器201耦合，用于存储各种软件程序和/或多组指令。具体实现中，存储器202可包括高速随机存取的存储器，并且也可包括非易失性存储器，例如一个或多个磁盘存储设备、闪存设备或其他非易失性固态存储设备。存储器202可以存储操作系统，例如uCOS、VxWorks、RTLinux等嵌入式操作系统。存储器202还可以存储通信程序，该通信程序可用于与电子设备100，一个或多个服务器，或附加设备进行通信。
电源开关204可用于控制电源向电子设备200或电子设备300的各个部件供电。
有线LAN通信处理模块205可用于通过有线LAN和同一个LAN中的其他设备进行通信，还可用于通过有线LAN连接到WAN，可与WAN中的设备通信。
显示屏206可用于显示图像，视频等。显示屏206可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)显示屏，有源矩阵有机发光二极体(active-matrix organic light emitting diode，AMOLED)显示屏，柔性发光二极管(flexible light-emitting diode，FLED)显示屏，量子点发光二极管(quantum dot light emitting diodes，QLED)显示屏等等。可以理解，显示屏206上可以显示安卓应用的桌面应用窗口的显示内容。
音频模块210可用于通过音频输出接口输出音频信号，这样可使得电子设备200或电子设备300支持音频播放。音频模块还可用于通过音频输入接口接收音频数据。音频模块207包括但不限于：麦克风、扬声器、受话器等。
可以理解的是，图22示意的结构并不构成对电子设备200或电子设备300的具体限定。在本申请另外一些实施例中，电子设备200或电子设备300可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
图23是本发明实施例的电子设备200或电子设备300的软件结构框图。
在一些实施例中，电子设备200或电子设备300的软件结构从上至下分别为应用层，框架层，核心库层以及驱动层。
如图23所示，驱动层可以包括但不限于CPU驱动、GPU驱动、显示控制器驱动等。驱动层上面是核心库层，核心库层是操作系统的核心部分，核心库层包括但不限于输入/输出服务、实现CPU、GPU图形处理的图形服务等。图形服务可以包括但不限于2D引擎、3D引擎、、帧缓冲区(Frame Buffer)、OpenGL、EGL等。框架层包括但不限于图形服务、系统服务、网页服务和用户服务等；图形服务可以包括但不限于画布、视图等。应用层包括但不限于桌面、媒体播放器、浏览器等。
在一些实施例中，电子设备100执行绘图信息在显示屏上显示渲染图像主要是通过将每个应用窗口绘制到一个缓冲区(Buffer)中，然后在显示屏上对这些应用窗口的Buffer进行合成，这样的话，就多了一个屏幕的缓冲合成。例如，在手机100的安卓应用的应用窗口显示中，显示屏上显示的是合成后的显示窗口，例如，手机100的显示屏上显示的是“设置”应用的窗口1，时间或电池状态栏的窗口2等合成的显示窗口。因此，手机100的显示屏需要一个缓冲区从而实现对这些窗口(窗口1、窗口2等)进行缓冲合成，合成后才能在显示屏上显示。
相较于电子设备100执行绘图信息在显示屏上显示渲染图像，电子设备200直接将窗口的绘制结果呈现到电脑200的显示屏上，与手机100的显示屏上显示应用窗口不同，手机100的显示屏上显示多个应用窗口合成的内容，而电脑200的一个桌面应用窗口上只显示一个应用窗口的内容，不需要合成多个应用窗口，从而节省显示屏合成应用窗口的缓冲区(buffer)。
在一些实施例中，桌面代理服务201的应用程序将绘图信息传递给框架层，在框架层依据该绘图信息向系统申请绘制图层的画布，其中该画布的大小依据所述绘图信息包含的待绘制图层的宽高参数确定；系统申请调用图层内存分配单元为其分配用于绘制图层的画布；同时将其获取所述绘图信息传递给核心库层的渲染驱动，用于使渲染驱动驱使绘图引擎基于所述绘图信息调用OpenGL或EGL绘制图层，从而完成安卓应用的桌面应用窗口的图像绘制。其中，绘图引擎包括图形处理器GPU的绘图引擎，包括2D绘图引擎、3D绘图引擎等。
例如，在一些实施例中，电子设备200接收电子设备100发送的绘图信息，该绘图信息中包含了图元信息、图元坐标等。其中，图元是图形软件用于操作和组织画面的最基本的素材，一幅画面由图元组成，图元是一组最简单的、最通用的集合图形或者字符，比如点、线段、三角形、多边形等。
在附图中，可以以特定布置和/或顺序示出一些结构或方法特征。然而，应该理解，可能不需要这样的特定布置和/或排序。而是，在一些实施例中，这些特征可以以不同于说明性附图中所示的方式和/或顺序来布置。另外，在特定图中包括结构或方法特征并不意味着暗示在所有实施例中都需要这样的特征，并且在一些实施例中，可以不包括这些特征或者可以与其他特征组合。
需要说明的是，本申请各设备实施例中提到的各单元/模块都是逻辑单元/模块，在物理上，一个逻辑单元/模块可以是一个物理单元/模块，也可以是一个物理单元/模块的一部分，还可以以多个物理单元/模块的组合实现，这些逻辑单元/模块本身的物理实现方式并不是最重要的，这些逻辑单元/模块所实现的功能的组合才是解决本申请所提出的技术问题的关键。此外，为了突出本申请的创新部分，本申请上述各设备实施例并没有将与解决本申请所提出的技术问题关系不太密切的单元/模块引入，这并不表明上述设备实施例并不存在其它的单元/模块。
需要说明的是，在本专利的示例和说明书中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
虽然通过参照本申请的某些优选实施例，已经对本申请进行了图示和描述，但本领域的普通技术人员应该明白，可以在形式上和细节上对其作各种改变，而不偏离本申请的精神和范围。

Claims (20)

  1. 一种应用的跨平台显示方法，其特征在于，包括：
    第一系统平台获取第二系统平台上运行的第一应用的第一窗口的第一尺寸，并在所述第一系统平台中创建对应所述第一应用的第三窗口，其中所述第三窗口用于显示所述第二系统平台上运行的所述第一应用的显示界面；
    第一系统平台获取第二系统平台上运行的第二应用的第二窗口的第二尺寸，并在所述第一系统平台中创建对应所述第二应用的第四窗口，其中所述第四窗口用于显示所述第二系统平台上运行的所述第二应用的显示界面。
  2. 根据权利要求1所述的方法，其特征在于，所述第一系统平台在检测到用户在所述第一系统平台中打开第一应用或者第二应用的情况下，从所述第二系统平台获取所述第一尺寸或第二尺寸。
  3. 根据权利要求1所述的方法，其特征在于，所述第二系统平台不显示所述第一窗口和第二窗口。
  4. 根据权利要求1所述的方法，其特征在于，在所述第一系统平台中创建对应所述第一应用的第三窗口，或者在所述第一系统平台中创建对应所述第二应用的第四窗口，包括：
    所述第一系统平台为所述第一应用设置第一标识，或者所述第一系统平台为所述第二应用设置第二标识；
    其中第一标识用于标识所述第三窗口用于显示所述第一应用的显示界面，第二标识用于标识所述第四窗口用于显示所述第二应用的显示界面。
  5. 根据权利要求4所述的方法，其特征在于，还包括：
    所述第一系统平台向所述第二系统平台发送所述第一标识或第二标识。
  6. 根据权利要求5所述的方法，其特征在于，所述第一系统平台通过以下方式在所述第三窗口中显示所述第二系统平台上运行的所述第一应用；
    所述第一系统平台从所述第二系统平台接收第一绘图信息，其中所述第一绘图信息包括所述第一标识；
    所述第一系统平台根据所述第一绘图信息所包括的第一标识，在所述第三窗口中显示根据所述第一绘图信息所绘制的，所述第一应用的第一显示界面。
  7. 根据权利要求5所述的方法，其特征在于，所述第一系统平台通过以下方式在所述第四窗口中显示所述第二系统平台上运行的所述第二应用：
    所述第一系统平台从所述第二系统平台接收第二绘图信息，其中所述第二绘图信息包括所述第二标识；
    所述第二系统平台根据所述第二绘图信息所包括的第二标识，在所述第四窗口中显示根据所述第二绘图信息所绘制的，所述第二应用的第二显示界面。
  8. 根据权利要求1所述的方法，其特征在于，还包括：
    所述第一系统平台检测到用户对所述第一系统平台上的所述第三窗口的第一操作；
    所述第一系统平台生成操作信息，并将所述操作信息发送给所述第二系统平台，其中，所述操作信息包括第一标识；
    所述第一系统平台从所述第二系统平台接收第三绘图信息，其中所述第三绘图信息包括所述第一标识，并且所述第三绘图信息是所述第二系统平台，通过执行所述操作信息对应的操作生成的；
    所述第一系统平台基于所述第三绘图信息在所述第三窗口中显示第三显示界面，其中第三显示界 面为第一显示界面对应所述第一操作改变后的显示界面。
  9. 根据权利要求8所述的方法，其特征在于，所述第一操作包括以下操作中的至少一种：
    改变所述第一显示界面的大小的操作；
    对所述第一显示界面中的控件的操作。
  10. 根据权利要求1所述的方法，其特征在于，所述第一系统平台和所述第二系统平台具有不同的操作系统。
  11. 根据权利要求1所述的方法，其特征在于，所述第一系统平台和所述第二系统平台运行于不同的电子设备上。
  12. 根据权利要求1所述的方法，其特征在于，所述第一系统平台为运行于电子设备上的虚拟机上的操作系统，所述第二系统平台为所述电子设备的操作系统。
  13. 一种应用的跨平台显示方法，其特征在于，包括：
    第二系统平台上运行有第一应用；
    第一系统平台从所述第二系统平台获取第一应用的第一窗口的第一尺寸；
    第一系统平台在所述第一系统平台中创建对应所述第一应用的第三窗口和第一标识，其中第一标识用于标识所述第三窗口用于显示所述第一应用的显示界面；
    第二系统平台从所述第一系统平台接收到所述第一标识后，生成第一绘图信息，其中所述第一绘图信息包括所述第一标识；
    所述第一系统平台从所述第二系统平台接收所述第一绘图信息，并根据所述第一绘图信息所包括的第一标识，在所述第三窗口中显示根据所述第一绘图信息所绘制的，所述第一应用的第一显示界面。
  14. 根据权利要求13所述的方法，其特征在于，所述第一系统平台在检测到用户在所述第一系统平台中打开第一应用的情况下，所述第一系统平台将所述第一应用信息发送给所述第二系统平台；
    所述第二系统平台根据所述第一应用信息运行第一应用，并在第二系统平台中创建所述第一应用的第一窗口；
    所述第二系统平台将所述第一应用的第一窗口的第一尺寸发送给所述第一系统平台。
  15. 根据权利要求13所述的方法，其特征在于，所述第二系统平台不显示所述第一窗口。
  16. 根据权利要求13所述的方法，其特征在于，还包括：
    所述第一系统平台检测到用户对所述第一系统平台上的所述第三窗口的第一操作；
    所述第一系统平台生成操作信息，并将所述操作信息发送给所述第二系统平台，其中，所述操作信息包括所述第一标识；
    所述第二系统平台通过执行所述操作信息对应的操作生成第三绘图信息，并将第三绘图信息发送给所述第一系统平台；
    所述第一系统平台基于所述第三绘图信息在所述第三窗口中显示第三显示界面，其中第三显示界面为所述第一显示界面对应所述第一操作改变后的显示界面。
  17. 根据权利要求16所述的方法，其特征在于，所述第一操作包括以下操作中的至少一种：
    改变所述第一显示界面的大小的操作；
    对所述第一显示界面中的控件的操作。
  18. 根据权利要求13所述的方法，其特征在于，所述第一系统平台和所述第二系统平台具有不同的操作系统，其中，所述第一系统平台和所述第二系统包括以下至少一种：
    所述第一系统平台和所述第二系统平台运行于不同的电子设备上；
    所述第一系统平台为运行于电子设备上的虚拟机上的操作系统，所述第二系统平台为所述电子设备的操作系统。
  19. 一种可读介质，其特征在于，所述可读介质上存储有指令，该指令在电子设备上执行时使电子设备执行权利要求1至18中任一项所述的应用的跨平台显示方法。
  20. 一种电子设备，其特征在于，包括：
    存储器，用于存储由电子设备的一个或多个处理器执行的指令，以及
    处理器，是电子设备的处理器之一，用于执行权利要求1至18中任一项所述的应用的跨平台显示方法。
PCT/CN2022/104441 2021-07-13 2022-07-07 应用的跨平台显示方法、可读介质和电子设备 WO2023284625A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110791240.8A CN115617427A (zh) 2021-07-13 2021-07-13 应用的跨平台显示方法、可读介质和电子设备
CN202110791240.8 2021-07-13

Publications (1)

Publication Number Publication Date
WO2023284625A1 true WO2023284625A1 (zh) 2023-01-19

Family

ID=84855396

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/104441 WO2023284625A1 (zh) 2021-07-13 2022-07-07 应用的跨平台显示方法、可读介质和电子设备

Country Status (2)

Country Link
CN (1) CN115617427A (zh)
WO (1) WO2023284625A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110004827A1 (en) * 2008-02-29 2011-01-06 The Regents Of The University Of California Scalable, Cross-Platform Method for Multi-Tile Display Systems
CN112256177A (zh) * 2020-10-23 2021-01-22 南京高盛魔笛信息科技有限公司 一种小触控屏和大显示屏协同实现高效协作的系统架构
CN112995727A (zh) * 2019-12-17 2021-06-18 华为技术有限公司 一种多屏协同方法、系统及电子设备
CN113050841A (zh) * 2019-12-26 2021-06-29 华为技术有限公司 显示多窗口的方法、电子设备和系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110004827A1 (en) * 2008-02-29 2011-01-06 The Regents Of The University Of California Scalable, Cross-Platform Method for Multi-Tile Display Systems
CN112995727A (zh) * 2019-12-17 2021-06-18 华为技术有限公司 一种多屏协同方法、系统及电子设备
CN113050841A (zh) * 2019-12-26 2021-06-29 华为技术有限公司 显示多窗口的方法、电子设备和系统
CN112256177A (zh) * 2020-10-23 2021-01-22 南京高盛魔笛信息科技有限公司 一种小触控屏和大显示屏协同实现高效协作的系统架构

Also Published As

Publication number Publication date
CN115617427A (zh) 2023-01-17

Similar Documents

Publication Publication Date Title
WO2021057830A1 (zh) 一种信息处理方法及电子设备
CN113553014B (zh) 多窗口投屏场景下的应用界面显示方法及电子设备
US20220398059A1 (en) Multi-window display method, electronic device, and system
WO2022100315A1 (zh) 应用界面的生成方法及相关装置
US8477143B2 (en) Buffers for display acceleration
US11853526B2 (en) Window display method, window switching method, electronic device, and system
WO2021013019A1 (zh) 一种图片处理方法及装置
US20220253207A1 (en) Display method and electronic device
CN114816167B (zh) 应用图标的显示方法、电子设备及可读存储介质
WO2023051111A1 (zh) 多个应用组合且同时启动多个应用的方法及电子设备
WO2022052677A1 (zh) 界面显示方法及电子设备
WO2021052488A1 (zh) 一种信息处理方法及电子设备
WO2021254113A1 (zh) 一种三维界面的控制方法和终端
WO2023284625A1 (zh) 应用的跨平台显示方法、可读介质和电子设备
CN114461312B (zh) 显示的方法、电子设备及存储介质
WO2021253922A1 (zh) 字体切换方法及电子设备
CN117707656B (zh) 一种用户操作的响应方法、设备及存储介质
CN116048317B (zh) 一种显示方法及装置
WO2024022137A1 (zh) 一种投屏方法及设备
CN116700554B (zh) 信息的显示方法、电子设备及可读存储介质
WO2023030276A1 (zh) 一种显示方法、装置、设备及存储介质
WO2024104131A1 (zh) 一种多窗口场景下的窗口获焦方法、设备及系统
WO2023202444A1 (zh) 一种输入方法及装置
WO2024001871A1 (zh) 一种操控方法和电子设备
WO2024067599A1 (zh) 应用显示的方法和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22841261

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22841261

Country of ref document: EP

Kind code of ref document: A1