WO2024082871A1 - 一种投屏系统和电子设备 - Google Patents

一种投屏系统和电子设备 Download PDF

Info

Publication number
WO2024082871A1
WO2024082871A1 PCT/CN2023/118031 CN2023118031W WO2024082871A1 WO 2024082871 A1 WO2024082871 A1 WO 2024082871A1 CN 2023118031 W CN2023118031 W CN 2023118031W WO 2024082871 A1 WO2024082871 A1 WO 2024082871A1
Authority
WO
WIPO (PCT)
Prior art keywords
electronic device
application
message
screen
interface
Prior art date
Application number
PCT/CN2023/118031
Other languages
English (en)
French (fr)
Inventor
李文远
王冬伟
Original Assignee
荣耀终端有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 荣耀终端有限公司 filed Critical 荣耀终端有限公司
Publication of WO2024082871A1 publication Critical patent/WO2024082871A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units

Definitions

  • the embodiments of the present application relate to the fields of communication technology and terminal technology, and in particular to a screen projection system and electronic device.
  • the collaborative work of multiple electronic devices brings convenience to users' lives and work.
  • a mobile phone receives a message, it can not only display the message notification on the mobile phone, but also transfer the message notification to the PC for display. This makes it convenient for users to view messages across devices and improves the efficiency of collaborative work between devices. Of course, it is not enough to simply transfer message notifications between devices. After users view the message notifications from their mobile phones on the PC, they often need to process the message notifications through the PC, which requires casting the phone screen to the PC.
  • anomalies may occur when the user starts screen projection on the PC.
  • the present application provides a screen projection system and electronic device. After a message from a first electronic device is transferred to a second electronic device, the user is allowed to directly start the application corresponding to the message on the second electronic device and project the screen on the second electronic device when the first electronic device is in a locked or off state.
  • a screen projection system which includes a first electronic device and a second electronic device, the first electronic device including a first application; the first electronic device is used to receive first data of the first application; the first electronic device is used to send a first message including the first data to the second electronic device; the second electronic device displays a first control in response to receiving the first message; the second electronic device is used to receive a first operation of the user on the first control at a first time; the second electronic device sends a screen projection request to the first electronic device in response to the first operation; the first electronic device sends image information of the first application to the second electronic device in response to receiving the screen projection request, and the image information of the first application includes the first data; the second electronic device is used to display a window according to the received image information of the first application, and the first interface of the first application is displayed in the window at a second time, and the first interface includes the image information of the first application; the second time is later than the first time, and during the time interval between the first time and the second
  • the user When the first electronic device is in a locked or off-screen state, the user does not need to perform any operation on the first electronic device, but can directly operate on the second electronic device, that is, the first operation on the first control mentioned above, the first operation can be an operation of clicking the first control, thereby pulling up the first application on the first electronic device and casting the screen on the second electronic device.
  • the first application is an application installed on the first electronic device, and whether the second electronic device has the first application installed does not affect the implementation of the above system.
  • the first electronic device sends the first message to the second electronic device; in response to receiving the first message, the second electronic device is used to display the first control; in the first time interval, the first electronic device is in a locked screen or The second electronic device is also used to receive a first operation of the user on the first control in the first time interval; in response to the first operation, the second electronic device is also used to send a screen projection request to the first electronic device; in response to receiving the screen projection request, the first electronic device is also used to send data of the first application corresponding to the first message to the second electronic device; in response to receiving the data of the first application corresponding to the first message, the second electronic device is also used to display a screen projection window, which displays a first interface in the first time interval, and the first interface includes the first message.
  • the first electronic device in the second time interval, is also used to display the main interface; the first electronic device is also used to receive a second operation of the user; in response to receiving the second operation, the first electronic device is in a locked screen or off screen state in the first time interval, and the start time of the first time interval is the end time of the second time interval; the first electronic device is also used to receive a third operation of the user; in response to receiving the third operation, the first electronic device displays the main interface in a third time interval, and the start time of the third time interval is the end time of the first time interval; in the third time interval, the second electronic device is also used to display the first interface in the projection window.
  • the first electronic device is locked or turned off when displaying the main interface; then, when the first electronic device is locked or turned off, a first message from the first application is received and transferred to the second electronic device.
  • the first interface is projected to the second electronic device; then, the first electronic device is turned on or unlocked, that is, the above third operation, at which time the first electronic device displays the main interface, while at the same time the projection window of the second electronic device displays the first interface.
  • the first electronic device also includes a second application, and the first application is different from the second application; the first electronic device is also used to receive an operation by a user to open the second application; in response to the user's operation to open the second application, the first electronic device is also used to display a second interface of the second application in a fourth time interval, and the start time of the fourth time interval is the end time of the third time interval; in the fourth time interval, the second electronic device is also used to display the first interface in the projection window.
  • the first electronic device when the projection window of the second electronic device displays the first interface, the first electronic device can open other applications different from the first application, such as the second application. At this time, the projection window of the second electronic device still displays the first interface and will not change with the display interface on the first electronic device, thereby realizing heterogeneous screen projection.
  • the second electronic device in the fourth time interval, is also used to display a third interface in the projection window, where the third interface is the interface of the first application, and the third interface is different from the first interface.
  • the third interface and the first interface are both interfaces of the first application. That is to say, after the first application is projected to the second electronic device, the first application can be operated in the projection window of the second electronic device to switch different interfaces.
  • the first electronic device also includes a third application, which is different from the first application; the first electronic device is also used to receive a second message from the third application; the first electronic device is also used to send the second message to the second electronic device; in response to receiving the second message, the second electronic device is also used to display a second control; the second electronic device is also used to receive a fourth operation of the user on the second control in the first time interval; in response to the fourth operation, the second electronic device is also used to send a screen projection request to the first electronic device; in response to receiving the screen projection request, the first electronic device sends the data of the third application corresponding to the second message to the second electronic device; in response to receiving the data of the third application corresponding to the second message, During the first time interval, the second electronic device is further used to display the fourth interface in the projection window, and the fourth interface includes the second message.
  • the user can also switch the projection through the above operation and project the second application to the second electronic device.
  • the second electronic device is further used to display the first control on the desktop; or, the second electronic device is further used to display the first control in the message list window, and the first control is a message card corresponding to the first message.
  • the first control is a message card corresponding to the first message.
  • the message card also has another situation: when the first control is displayed on the desktop for a certain period of time and still does not receive any user operation, it disappears, and a message card corresponding to the first message is generated in the message list. If the user wants to operate the first message after the first control disappears, the user can also open the message list window and find the message card corresponding to the first message in the message list window, which can also achieve the effect of the user operating the first control.
  • the first operation is an operation of a user clicking on the first control; or, the first operation is an operation of a user dragging the first control.
  • the second electronic device is further used to receive the first operation and display a security authentication interface in the screen projection window; upon receiving an operation in which a user enters authentication information in the security authentication interface, the second electronic device is further used to send the authentication information to the first electronic device; upon receiving the authentication information, the first electronic device is further used to determine that the authentication information satisfies a preset value and send a message of successful authentication to the second electronic device; in response to receiving the message of successful authentication, the second electronic device sends a screen projection request to the first electronic device.
  • the security authentication mechanism will be triggered before the second electronic device starts screen projection to ensure the user's privacy security.
  • the second electronic device is further used to receive an operation of the user on the first interface on the first message, and the second electronic device sends an instruction to the first electronic device instructing the first electronic device to perform the operation.
  • the user can directly reply to the first application, make audio or video calls, and perform other operations on the second electronic device.
  • the second electronic device will send all these operation instructions to the first electronic device, which will be executed by the first electronic device, thereby realizing control of the projected application on the second electronic device.
  • a screen projection method is provided, which is applied to a first electronic device, wherein the first electronic device includes a first application; the first electronic device receives a first message of the first application; the first electronic device sends the first message to a second electronic device through a first communication connection; in a first time interval, the first electronic device is in a locked screen or screen-off state; in response to receiving a screen projection request sent by the second electronic device, the first electronic device sends data of the first application corresponding to the first message to the second electronic device through a second communication connection, so that in the first time interval, the second electronic device displays the first interface in the screen projection window; wherein the first interface includes the first message.
  • the method includes the first electronic device maintaining one or more activities of the first application in the OnResume state; the first electronic device controls one or more activities of the first application not to enter a sleep process.
  • the first electronic device keeps one or more activities of the first application in the OnResume state, and at the same time controls one or more activities of the first application not to enter the sleep process, so that the one or more applications can be visible to the user and the user can operate after the screen is projected to the second electronic device.
  • the first electronic device includes an asynchronous processor, an activity recorder, and a root window container.
  • the method also includes the asynchronous processor marking one or more activities of the first application with a first tag; the activity recorder determines that one or more activities of the first application have the first tag, and keeps the one or more activities of the first application in an OnResume state; the root window container determines that one or more activities of the first application have the first tag, and controls the one or more activities of the first application not to enter a sleep process.
  • the first electronic device can filter out the activities corresponding to the first application from the activities corresponding to other applications by marking the activities corresponding to the first application with a specific label, thereby realizing the above-mentioned screen projection operation only for the activities of the first application that need to be projected.
  • the first electronic device in the first time interval, includes a virtual display area and a visible display area, and before the first electronic device sends the first interface to the second electronic device, the first electronic device marks the virtual display area with a second label; the first electronic device draws a first image in the virtual display area, and the first image is the interface of the first application corresponding to the first message; the first electronic device draws a second image in the visible display area, and the second image is the interface corresponding to the first electronic device when the first electronic device is in the locked screen or screen-off state; the first electronic device determines that the virtual display area has the second label, obtains the first image to obtain the first interface, and the first interface does not include the second image.
  • the first electronic device marks the virtual display area with a specific label, thereby indicating that the image drawn in the virtual display area shall not be synthesized with the images in other display areas, so that the interface projected to the second electronic device is only related to the image in the virtual display area, and will not change with the changes of the images in other display areas, nor will the lock screen layer be covered on the projection interface.
  • the first electronic device includes a display screen
  • the method also includes that in a second time interval, the first electronic device displays a main interface; the first electronic device receives a second operation of the user; in response to receiving the second operation, the first electronic device is in a locked screen or screen-off state at a first time, and the first time is the end time of the second time interval and the start time of the first time interval; the first electronic device receives a third operation of the user; in response to receiving the third operation, the first electronic device draws the main interface in the visible display area and displays the main interface in a third time interval, and the start time of the third time interval is the end time of the first time interval; in the third time interval, the first electronic device determines that the virtual display area has the second tag, and does not obtain the main interface of the visible display area and send it to the second electronic device, so that the second electronic device displays the first interface in the projection window.
  • the method before the first electronic device receives the request to start screen projection, the method also includes the first electronic device receiving authentication information sent by the second electronic device through a third communication connection; the first electronic device determines that the authentication information meets a preset value, and sends a message of authentication success to the second electronic device through the third communication connection, so that the second electronic device sends a message to the first electronic device. Send screen casting request.
  • the method includes the first electronic device receiving an instruction from the second electronic device and executing an operation of the user on the first interface on the first message.
  • a method is provided, which is applied to a first electronic device and a second electronic device, wherein before the second electronic device establishes the first communication connection with the first electronic device, the first electronic device establishes a second communication connection with the second electronic device; the first electronic device receives the first message; the first device sends the second message to the second electronic device through the second communication connection based on the first message; in response to receiving the second message, the second electronic device displays the first control based on the second message; when the first electronic device is in a locked screen or screen-off state, the second electronic device receives the first operation.
  • the second electronic device determines that the first application is projected on the second electronic device for the first time, and the second electronic device establishes a third communication connection with the first electronic device, and the third communication connection is used for security authentication; the projection window displays the security authentication interface of the first electronic device.
  • the second electronic device determines that this is not the first time that the first application is projected on the second electronic device, and the current time is longer than a preset time from the last successful authentication, and the second electronic device establishes a third communication connection with the first electronic device; the projection window displays the security authentication interface of the first electronic device.
  • the second electronic device after the second electronic device receives the authentication information input by the user in the security authentication interface, the second electronic device sends the authentication information to the first electronic device through the third communication connection; the first electronic device determines that the authentication information meets the preset value, and sends a message of authentication success to the second electronic device through the third communication connection.
  • the second electronic device determines that the first application is set with an application lock, and the second electronic device does not display the projection window of the first electronic device.
  • the first electronic device includes a screen projection assistant module, a first virtualization service module, and a first screen projection service module.
  • the screen projection assistant module After the first electronic device receives the screen projection request, the screen projection assistant module binds the screen projection service; the screen projection assistant module sends a first indication message for initialization to the first virtualization service module; the first virtualization service module completes initialization according to the first indication message; the first virtualization service module sends a second indication message to start the screen projection service to the first screen projection service module; the first screen projection service module starts the screen projection service according to the second indication message; the first screen projection service module sends a message to the screen projection assistant module that the screen projection service has been started.
  • the first electronic device creates a virtual display area and marks the first label for the virtual display area.
  • the first screen projection service module starts the screen projection service, it creates the virtual display area and marks the first label for the virtual display area.
  • the first electronic device further includes a multi-screen management module and a system interface module
  • the second electronic device includes a second screen projection service module and a message center module.
  • the screen projection assistant module sends a message that the screen projection service has been started to the message center module through the first communication connection
  • the message center module sends a request to start application screen projection to the system interface module through the first communication connection
  • the system interface module based on the request to start application screen projection,
  • a third indication message for starting screen projection of the first application is sent to the multi-screen management module; upon receiving the third indication message, the multi-screen management module places all activities of the first application in a foreground visible state, and skips the dormant state for all activities of the first application; the multi-screen management module obtains the first display data corresponding to the first application; the screen projection management module sends the first display data to the second screen projection service module through the first communication connection
  • the multi-screen management module puts all activities of the first application in a foreground visible state, and skips the dormant state for all activities of the first application, and the multi-screen management module marks all activities of the first application with a second tag; the multi-screen management module puts the activities marked with the second tag in a foreground visible state, and skips the dormant state for the activities marked with the second tag.
  • the method before the multi-screen management module obtains the first display data corresponding to the first application, the method includes: the multi-screen management module sends a fourth indication message to the first application; the first application is started according to the fourth indication message; the first application sends a message to the multi-screen management module that the first application has been started; after receiving the message that the first application has been started, the multi-screen management module draws the first display data in the virtual display area.
  • the method includes: the second screen projection service module receives the first display data, and renders the first interface according to the first display data; the second screen projection service module sends the first interface to the message center module; after receiving the first interface, the message center controls the display screen of the second electronic device to display the first interface in the projection window.
  • the first electronic device also includes a first communication module
  • the second electronic device also includes a second communication module
  • the second electronic device establishes a first communication connection with the first electronic device, specifically including: the message center module sends a fifth indication information to the second communication module; the second communication module establishes the first communication connection with the first communication module according to the fifth indication information; the second communication module sends a message to the message center indicating that the first communication connection is successfully established.
  • the method before the screen projection assistant module is bound to the screen projection service, the method includes: the first communication module sends sixth indication information to the screen projection assistant module; and the screen projection assistant module is started according to the sixth indication information.
  • the second electronic device before the second electronic device sends a screen projection request to the first electronic device through the first communication connection, it specifically includes: the message center obtains a session key and the IP address of the first electronic device through the first communication connection; the message center sends a seventh indication message to the second screen projection service module, and the seventh indication message carries the session key and the IP address of the first electronic device; the second screen projection service module completes initialization according to the seventh indication message.
  • the following specifically includes: the second screen projection service module sends the screen projection request to the second communication module through the first communication connection; the second communication module sends the screen projection request to the first communication module through the first communication connection; the first communication module sends the screen projection request to the screen projection assistant module.
  • the steps specifically include: the first application sends the second message to the system interface module according to the first message; the system interface module sends the second message to the message center module through the second communication connection; after receiving the second message, the message center module controls the display screen of the second electronic device to display the first message according to the second message. Control; upon receiving the first operation of the user on the first control, the message center module sends a fifth indication information to the second communication module.
  • the method further includes: the message center module controls the display screen of the second electronic device to display the transition loading interface in the projection window.
  • the second electronic device also includes a second security authentication module
  • the method also includes: the message center sends an eighth indication message to the second security authentication module; the second security authentication module determines, based on the eighth indication message, that the first application is on the second electronic device for the first time, and the second security authentication module sends a ninth indication message to the second communication module; the second communication module establishes the third communication connection with the first communication module based on the ninth indication message.
  • the method after the message center module receives the first operation, the method also includes: the message center sends an eighth indication message to the second security authentication module; the second security authentication module determines, based on the eighth indication message, that this is not the first time that the first application is projected on the second electronic device, and the current time is longer than the preset value from the last successful authentication, and the second security authentication module sends the ninth indication message to the second communication module; the second communication module establishes the third communication connection with the first communication module based on the ninth indication message.
  • the third communication connection after the third communication connection is established, it specifically includes: the second communication module sends a message that the third communication connection is successfully established to the message center module; the message center sends a tenth indication message to the second security authentication module; and the second security authentication module controls the display screen of the second electronic device to display the security authentication interface in the projection window according to the tenth indication message.
  • the first electronic device also includes a first security authentication module.
  • the second security authentication module receives the operation of the user inputting authentication information in the security authentication interface, it specifically includes: the second security authentication module sends the authentication information to the first security authentication module through the third communication connection; the first security authentication module determines that the authentication information meets the preset value, and sends a successful authentication message to the second security authentication module through the third communication connection; the second security authentication module sends the successful authentication message to the message center module; after receiving the successful authentication message, the message center module controls the display screen of the second electronic device to display the transition loading interface in the projection window.
  • the multi-screen management module includes an asynchronous processor, an activity launcher, an activity stack manager, a lock screen controller, an activity recorder, a task stack, and a root window container.
  • the multi-screen management module puts all activities of the first application in a foreground visible state and skips the dormant state for all activities of the first application, specifically including: the asynchronous processor marks some or all activities of the first application with the second label; the asynchronous processor sends a request to the activity launcher to start an activity with the second label; in response to receiving the first command, the activity launcher sends a request to the activity stack manager to start application screen projection; in response to receiving the request to start application screen projection, the activity stack manager calls the lock screen controller to perform a first action, the first action including putting the activity marked with the second label in the foreground visible state; the lock screen controller puts the activity marked with the second label in the foreground visible state.
  • the lock screen controller sends a third command to the activity recorder; upon receiving the third command, the activity recorder controls the activity marked with the second tag to remain visible in the foreground; the activity recorder sends a fourth command to the task stack, the fourth command being used to instruct the task stack to check the status of the activity at the top of the task stack; upon receiving the fourth command, the task stack determines that the activity at the top of the task stack is in the foreground visible state, and sends a fifth command to the root window container; upon receiving the fifth command, the root window The container skips the dormant state for the activity marked with the second tag.
  • the first message corresponds to a first activity of the first application
  • the first activity is marked with the second tag by the asynchronous processor
  • the first activity is located at the top of the task stack.
  • the multi-screen management module also includes an activity stack management server and a layer synthesizer. Before the multi-screen management module obtains the first display data corresponding to the first application, it specifically includes: after receiving the first command, the activity launcher sends a fourth indication information to the first application; the first application is started according to the fourth indication information; the first application sends a message to the activity stack management server that the first application has been started; the activity stack management server sends a sixth command to the layer synthesizer; the layer synthesizer draws the application layer corresponding to the first application in the virtual display area; the layer synthesizer obtains the first display data according to the application layer.
  • an electronic device which includes one or more processors; one or more memories; the one or more memories store one or more computer programs, and the one or more computer programs include instructions, which, when executed by the one or more processors, enable the electronic device to execute any method that may be implemented in the above-mentioned second aspect.
  • a computer-readable storage medium comprising computer instructions.
  • the computer instructions When the computer instructions are executed on an electronic device, the electronic device executes the method described in the second aspect.
  • a computer program product in a sixth aspect, includes a computer program code.
  • the computer program code runs on an electronic device, the electronic device executes the method described in the second aspect.
  • a chip is provided for executing instructions. When the chip is running, the chip executes the method described in the second aspect above.
  • FIG1 is a schematic diagram of a message flow scenario provided in an embodiment of the present application.
  • FIG. 2 is a schematic diagram of an interface display of a heterogeneous screen projection provided in an embodiment of the present application.
  • FIG3 is a schematic diagram of an interface display of a heterogeneous screen projection provided in an embodiment of the present application.
  • FIG4 is a schematic diagram of the principle of a heterogeneous screen projection provided in an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a scenario of screen projection in the screen-off state provided in an embodiment of the present application.
  • Figure 6 is a schematic diagram of a screen projection scenario in a locked screen state provided in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of another scenario of screen projection in the screen-off state provided in an embodiment of the present application.
  • FIG8 is a schematic diagram of a screen projection operation provided in an embodiment of the present application.
  • FIG. 9 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.
  • FIG. 10 is a software architecture diagram of an electronic device provided in an embodiment of the present application.
  • FIG. 11 is a diagram showing a specific implementation process of a screen projection method provided in an embodiment of the present application.
  • FIG. 12 is a specific implementation process diagram of another screen projection method provided in an embodiment of the present application.
  • FIG. 13 is a specific implementation process diagram of another screen projection method provided in an embodiment of the present application.
  • FIG. 14 is a specific implementation process diagram of another screen projection method provided in an embodiment of the present application.
  • FIG. 15 is a specific implementation process diagram of another screen projection method provided in an embodiment of the present application.
  • FIG. 16 is a specific implementation process diagram of another screen projection method provided in an embodiment of the present application.
  • FIG. 17A is a schematic diagram of a principle provided in an embodiment of the present application.
  • FIG. 17B is another schematic diagram of a principle provided in an embodiment of the present application.
  • Figure 18 is a flowchart of a screen projection method provided in an embodiment of the present application.
  • an embodiment of the present application provides a system, including multiple electronic devices, such as device 1, device 2, and device 3.
  • the electronic devices in the above system ie, device 1, device 2, device 3 may belong to the same local area network.
  • the local area network can be a wireless communication network, such as wireless local area networks (WLAN), wireless fidelity point to point (Wi-Fi P2P), Bluetooth network, zigbee network, infrared technology (IR) or near field communication (NFC) network.
  • WLAN wireless local area networks
  • Wi-Fi P2P wireless fidelity point to point
  • Bluetooth Bluetooth network
  • zigbee network infrared technology
  • NFC near field communication
  • the above-mentioned local area network can also be a wired communication network.
  • VGA video graphics array
  • DVI digital visual interface
  • HDMI high definition multimedia interface
  • the above-mentioned local area network can also be a remote network.
  • the system composed of devices 1, 2 and 3 can also be called a trust ring.
  • the process of mutual trust between devices can refer to related technologies. For example, it can be judged based on one or more of whether the devices have logged in the same system account, whether they have been authorized, and whether near field communication is used. The details are not repeated here.
  • multiple electronic devices can securely share data, and the shared data includes but is not limited to message reminders, message records, user information of other electronic devices, device type, running status of each application, identification of each application, synchronization of identification of recently used applications, etc.
  • device 1 when device 1 receives a message notification, it can not only display the message notification on device 1, but also transfer the message notification to device 2 and/or device 3 for display. This makes it convenient for users to view messages across devices and improves the efficiency of collaborative work between devices.
  • device 2 can also transfer the received message notification to device 1 and/or device 3
  • device 3 can also transfer the received message notification to device 1 and/or device 3.
  • the information notification is forwarded to device 1 and/or device 2.
  • the screen projection scenario includes a source device and a target device.
  • the source device and the target device use a certain protocol to display the content on the source device on the target device, that is, the source device projects the screen to the target device.
  • Common protocols include DLNA (Digital Living Network Alliance) protocol, Chromecast protocol, Miracast protocol, Airplay protocol, etc.
  • the source device can be a mobile phone, tablet computer, PC, watch, etc.
  • the target device can be a tablet computer, PC, TV, etc.
  • the source device and the target device may also have other names, for example, the source device may be called an active screen projection device, and the target device may be called a passive screen projection device; or, the source device may be called a first electronic device, and the target device may be called a second electronic device, etc., which is not limited in this application.
  • the source device as a mobile phone and the target device as a PC as an example.
  • the mobile phone and the PC belong to the same local area network and complete mutual trust recognition, the mobile phone and the PC belong to the same trust ring, that is, a communication channel for message flow can be established between the mobile phone and the PC.
  • multiple types of applications may be installed in a mobile phone.
  • application messages pushed by various applications may be received.
  • the application messages come from a message server corresponding to the application.
  • a social application may be installed in the mobile phone, and the mobile phone may receive application messages pushed by the social application, such as a message “Are you going for a run today?” sent by Xiao Ming, a contact in the social application.
  • the mobile phone After receiving an application message pushed by an application, if the mobile phone is unlocked and the screen is on, the mobile phone can display a prompt message on the current interface. For example, when receiving an application message pushed by a social application, if the current interface of the mobile phone is the main interface 101, a notification bubble 102 is displayed in the main interface 101, and the notification bubble 102 is used to display the application message received by the mobile phone.
  • the mobile phone can transfer the above application message (for example, "Are you going for a run today?" pushed by a social application) to the PC through the communication channel used for transferring messages between the mobile phone and the PC.
  • the above application message for example, "Are you going for a run today?" pushed by a social application
  • a prompt message for the above application message can be displayed.
  • the above prompt message can be displayed on the desktop of the PC in the form of a message notification control.
  • the above message notification control can be a floating window, a message bubble, a message card, etc.
  • the relevant content can be that the application message comes from a mobile phone, or that the application message comes from a social application, or that the main content of the application message, etc. It can be understood that in the embodiment of the present application, there is no specific limitation on the display position of the message card 104.
  • the message card 104 displayed on the desktop 103 can remind the user to pay attention to the application message currently received by the mobile phone. In this way, even if the user is not currently using the mobile phone directly or the mobile phone is not around, the user can still understand the brief content of the application message to avoid missing the message.
  • the message card 104 may have a display time limit.
  • the display time limit is preset to time 1, and the PC starts to display the message card 104 in response to the application message forwarded by the mobile phone.
  • the PC may cancel the display.
  • the message card 104 is displayed.
  • the PC detects that the user is in contact with the display area 1 in the touch screen, and the PC determines that the user indicates to select the message card 104.
  • the display area 1 is the area on the touch screen where the message card 104 is displayed.
  • the PC is connected to a mouse, it is detected that the cursor corresponding to the mouse is located in the display area 1, and a confirmation instruction is input, and the PC determines that the user indicates to select the message card 104.
  • the user can process the application message by operating the PC, or can choose to process the application message by directly operating the mobile phone.
  • the method of directly operating the mobile phone to process application messages can refer to the relevant technology.
  • the mobile phone in response to the user clicking the notification bubble 102, the mobile phone can run the social application in the foreground and display the application interface containing the above social message. In this way, the user can reply to the application message by operating on the above application interface.
  • the mobile phone responds to the user's instruction to start the social application and displays the application interface of the social application. In this way, the user can view the above application message or reply to the application message by operating on the application interface provided by the social application.
  • choosing to operate the PC to process application messages will trigger screen projection between the mobile phone and the PC.
  • the user can select the message card 104 on the PC.
  • the PC can remotely control the mobile phone to run the social application.
  • the social application enters the running state, the PC can display the projection window on the desktop 103.
  • the user can control the mobile phone through the projection window in the PC, such as operating the mobile phone to process the above-mentioned application messages.
  • the above-mentioned screen projection window can be used to synchronously display the current interface of the mobile phone.
  • the screen projection window also displays the application interface, which is the same-source screen projection.
  • the current interface of the mobile phone can also be displayed asynchronously.
  • the mobile phone displays the main interface
  • the screen projection window displays the application interface, which is a different-source screen projection.
  • heterogeneous screen projection can be realized through data interaction between the PC and the mobile phone. As shown in FIG2 , after the PC determines that the user has selected the message card 104, the heterogeneous screen projection is started, the main interface 101 is still displayed on the display screen of the mobile phone, and the projection window 201 on the display screen of the PC displays the application interface 202 corresponding to the above application message.
  • the content displayed in the projection window will not change synchronously with the content displayed on the mobile phone display.
  • the mobile phone display switches to display information application interface 303, and the projection window 201 on the PC display still displays the application interface 201 corresponding to the above application message.
  • Figure 4 is a schematic diagram of the principle of heterogeneous screen projection in Figures 2 and 3 above.
  • the display information on the target device is not synchronized with the display information on the foreground of the source device.
  • the window of application B is displayed on the source device.
  • the window of application B is displayed in the created virtual display area (VirtualDisplay).
  • the content of the virtual display area is invisible to the user and can be used to project the screen to the target device for display.
  • the source device also includes a user-visible display area.
  • the user-visible display area displays the window of application A, that is, the source device displays the window of application A in the foreground.
  • the user-visible area on the source device switches from the window of application A to the window of application C, that is, the source device displays the window of application C in the foreground.
  • the window of application B is still displayed on the target device.
  • the heterogeneous projection method has better isolation.
  • the method can provide users with independent control screens (i.e., the display screen of the source device and the display screen of the target device) to handle different applications.
  • the source device operates application A, or switches the window of application A to the window of application C, and the user can still operate application B on the target device. Therefore, heterogeneous screen projection releases the control of the display information of the source device over the display information of the target device, thereby enhancing the reusability of multiple devices and allowing users to operate different applications on different devices through screen projection.
  • the content displayed in the projection window changes synchronously with the content displayed on the mobile phone display.
  • the heterogeneous screen projection scenario is described as an example.
  • the mobile phone display screen is in a black screen state.
  • the mobile phone receives an application message from a social application, the mobile phone can briefly light up the screen and display a notification bubble, or it can not display the notification bubble and continue to maintain a black screen state.
  • the above-mentioned black screen state refers to the state where the display screen of the mobile phone is not lit or only displays the screen saver image.
  • the mobile phone display screen can display the lock screen interface or also be in a black screen state.
  • the mobile phone receives an application message from a social application, the mobile phone can display a notification bubble on the lock screen interface.
  • the mobile phone can also transfer the application message to the PC.
  • the PC can still display the message card 104 on the desktop 103 .
  • the mobile phone while the PC displays the message card 104, even if the PC determines that the user indicates that the message card 104 is selected, the mobile phone continues to maintain a black screen state or displays a lock screen interface. It can be understood that while the black screen state is maintained or the lock screen interface is displayed, the mobile phone cannot run the social application in the foreground, that is, it cannot display the application interface of the social application.
  • the screen off described in this article specifically refers to the situation where the screen is locked and off.
  • the source device displays a black screen interface and the display screen is not lit, as shown in Figure 5;
  • the screen lock described in this article specifically refers to the situation where the screen is locked but the screen is on.
  • the source device displays the lock screen interface, and the lock screen interface may include a lock logo, as shown in Figure 6.
  • the projection window even if the PC displays the projection window corresponding to the mobile phone, the projection window also displays a black screen interface.
  • the projection window even if the PC displays the projection window corresponding to the mobile phone, the projection window also displays the lock screen interface.
  • the user can first light up the screen of the mobile phone and unlock it, and then indicate on the PC to select the message card 104 to start the heterogeneous screen projection. It can be understood that the screen projection scene after the mobile phone is unlocked can refer to FIG2, which will not be repeated here.
  • the user can first unlock the mobile phone and keep the screen on, and then select the message card 104 on the PC to start the heterogeneous screen projection. It can be understood that the screen projection scene after the mobile phone is unlocked can refer to FIG4, which will not be repeated here.
  • the user cannot directly start the application window screen projection showing the message on the PC, but must first go to the room to unlock the mobile phone and put it in the bright screen state before being able to start the screen projection window corresponding to the mobile phone on the PC.
  • the application projection window refers to the projection window displayed on the target device when the application on the source device is projected.
  • the user can start the application screen projection window on the target device.
  • the embodiments of the present application provide a screen projection method, electronic device and system to provide users with a convenient and efficient screen projection experience.
  • the user can directly operate on the target device to realize the projection of the application window corresponding to the flow message on the target device.
  • the function of multi-screen collaborative interaction across devices is realized through control and input collaboration.
  • the user can focus on the operation of the target device without having to operate the source device in a locked screen or off screen state, which reduces the user operation process of starting the screen projection, improves the efficiency of human-computer interaction in starting the screen projection, and enhances the user experience.
  • FIG. 7 is a schematic diagram of a screen projection method provided in an embodiment of the present application.
  • the embodiment of the present application can realize the process of projecting the screen from a mobile phone to a PC, from a mobile phone to a tablet computer, and from a tablet computer to a PC.
  • the following is an example of a source device being a mobile phone and a target device being a PC.
  • the mobile phone and the PC belong to the same local area network and complete mutual trust recognition, the mobile phone and the PC belong to the same trust circle, that is, a communication channel for message flow can be established between the mobile phone and the PC.
  • a social application may be installed in the mobile phone, and the mobile phone may receive application messages pushed by the social application, such as a message “Are you going for a run today?” sent by Xiao Ming, a contact in the social application.
  • the screen After receiving an application message pushed by an application, if the phone is in the screen-off state, the screen can be briefly lit and the message notification bubble can be displayed, or the message notification bubble can be not displayed and the screen can remain unlit. As shown in Figure 7, the phone displays a black screen interface. If the phone is in the lock screen state, the message notification can be briefly displayed. It should be noted that the following embodiments are described by taking the mobile phone in the screen-off state as an example, and the situation when the mobile phone is in the screen-locked state can be used as a reference and will not be repeated.
  • the mobile phone can transfer the above application message (for example, "Are you going for a run today?" pushed by a social application) to the PC through the communication channel used for transferring messages between the mobile phone and the PC.
  • the above application message for example, "Are you going for a run today?" pushed by a social application
  • the application can be installed on the PC (the corresponding computer version of the application, the mobile version and computer version of the application are collectively referred to as the application in this article, without distinguishing them in name) or not. In other words, whether the corresponding application is installed on the PC does not affect the flow of messages and subsequent heterogeneous screen projection.
  • message card 702 displays summary information of the application message "Are you running today?", device information “from mobile phone”, application information “social application”, and contact information "Xiao Ming", so that users can quickly and easily get the content of the application message through the message card.
  • the application message is a message sent by Xiao Ming and received by the social application on the mobile phone, which makes it easy for users to determine whether to start screen projection to reply to the application message and other operations.
  • the user can process the application message by operating the PC. Selecting to operate the PC to process the application message will trigger the screen projection between the mobile phone and the PC.
  • the target device may receive one or more application messages transferred from any source device in the same trust ring.
  • the PC may respond to user operations and display a message list window, which may centrally display received message notifications or centrally display received and unprocessed message notifications.
  • the above message notifications may also be displayed in the message list window in the form of message cards.
  • a message list window 801 can be displayed.
  • the message list window 801 displays application messages from mobile phone 1 (such as message card 802), application messages from mobile phone 2 (such as message card 803), etc.
  • message card 802 corresponds to the message pushed by the social application in mobile phone 1.
  • message card 803 corresponds to the message pushed by the information application in mobile phone 2 (for example, text messages).
  • the PC when the PC detects that the user has selected any message card in the message list window 801, it can trigger the device corresponding to the message card to cast the screen. For example, when the PC detects that the user has selected message card 802, it can trigger mobile phone 1 to cast the screen. In this way, mobile phone 1 is the source device and the PC is the corresponding target device. For another example, when the PC detects that the user has selected message card 803, it can trigger mobile phone 2 to cast the screen. In this way, mobile phone 2 is the source device and the PC is the corresponding target device.
  • the user can select a message card 702 on the PC.
  • the PC can detect that the user has selected the message card 702, the PC can remotely control the background operation of the social application on the mobile phone even if the mobile phone is in the screen-off state.
  • the mobile phone still displays a black screen interface.
  • the PC can also display a projection window 703 corresponding to the social application on the desktop 701.
  • the projection window 703 corresponding to the above-mentioned social application is used to display the application interface 704 corresponding to the application message.
  • the application interface refers to the interface for displaying the above-mentioned application message.
  • the screen projection window that displays the application interface can be called the application screen projection window.
  • the application interface 704 is a chat box interface corresponding to the application message, and the user can view the application message and reply in this interface.
  • the mobile phone can render the corresponding application interface on the virtual display area through data interaction between the PC and the mobile phone.
  • the application interface rendered on the virtual display area is not displayed in the foreground of the mobile phone.
  • the mobile phone display still displays a black screen interface.
  • the virtual display area is a service for the PC.
  • the content in the virtual display area can be displayed in the projection window 703 in the PC. In this way, when the mobile phone is in the screen-off state, the content displayed in the projection window on the PC is the application interface 704 in the virtual display area.
  • the display interface in the screen projection window 703 and the interface shown in the mobile phone display screen will not change synchronously. It is understandable that when the mobile phone is in the screen-off state, the mobile phone display screen always displays a black screen interface.
  • the application interface 704 includes at least an interface element for the user to perform a reply operation. After the application interface 704 is displayed in the projection window of the PC, the user can reply to the application message on the PC through the interface element.
  • the application interface 704 includes a return control 707, but does not include a home button.
  • the PC displays the upper-level application interface of the application interface 704 in the projection window 703.
  • the upper-level application interface is the main interface of the application, wherein the main interface does not include a return control.
  • the application projection window on the target device can display all application interfaces of the application, but not the application interfaces of other applications, including the mobile phone's desktop, so there is no need to set a return control on the main interface of the application.
  • a close control may be configured on the projection window 703.
  • the PC may cancel the display of the projection window 703 in response to the user's selection of the "close control".
  • the projection window 703 may also include a minimize control, a maximize control, etc., which are not limited here.
  • the PC closes the screen projection window 703, and the mobile phone display screen displays the content of the current social application.
  • a mobile phone when a mobile phone opens an application that is performing heterogeneous screen projection on a PC, the application will be launched to the foreground of the mobile phone and the heterogeneous screen projection will end.
  • the source device may optionally enable a security authentication function to protect the privacy of the source device with a locked or turned-off screen.
  • the target device e.g., PC
  • the target device can display a screen projection window, which is used to display a security authentication interface.
  • the unlock password can be an authentication password used to unlock the phone screen.
  • the security authentication method can also be fingerprint authentication by the user inputting a fingerprint, voice authentication by the user inputting a voice, etc., and there is no limitation to this.
  • the projection window displaying the security authentication interface can be called the security authentication window.
  • the PC After the PC detects that the user has selected the message card 702, if it is determined that the mobile phone has the conditions for screen projection, the PC can display the screen projection window 703.
  • the above-mentioned screen projection window 703 can first display the security authentication interface 705.
  • the PC controls the screen projection window 703 to switch to display the transition loading interface 706 of "prompting waiting information".
  • the projection window that displays the "prompt waiting information" transition loading interface can be called the transition loading window.
  • the mobile phone responds to the PC's instructions, starts the social application, and draws the application interface of the social application in the virtual display area, and obtains the drawing parameters of each display object in the virtual display area.
  • the above-mentioned virtual display area is a service for the PC, which is not visible on the display screen of the mobile phone.
  • the above-mentioned drawing parameters are related parameters for rendering the interface, which can also be called display data in some examples.
  • the mobile phone After the mobile phone completes drawing the above application interface, it sends the corresponding display data to the PC, instructing the PC to render the above application interface in the projection window 703, so that the projection window 703 switches to display the application interface 704 of the social application, that is, the mobile phone projects the content of the virtual display area to the projection window 703 of the PC. That is, after the PC receives the mobile phone unlock password entered by the user and the unlock password is verified, the mobile phone starts to perform heterogeneous projection to the PC.
  • GUI graphical user interface
  • the user can directly operate on the target device to realize the projection of the application window corresponding to the flow message on the target device.
  • the function of multi-screen collaborative interaction across devices is realized through control and input collaboration.
  • the user can focus on the operation of the target device without having to operate the source device in a locked screen or off screen state, which reduces the user operation of starting the projection, improves the efficiency of human-computer interaction in starting the projection, and enhances the user experience.
  • the mobile phone when the mobile phone is in the screen-off or locked state, there is such a scenario: when the user is using a PC to play games or work in the study or living room at home, the mobile phone is placed in another room; when a contact (Xiao Ming) on the social application sends a message to the user's mobile phone, the user can immediately know on the PC that his mobile phone has been disconnected. A message notification was received.
  • users are often focused on playing games or working on their PCs. They don’t want to get up and go to the room to get their phones to check or reply to messages. Instead, they want to start the application window on the PC that displays the message (that is, the screen projection window corresponding to the phone) and process the message through the PC.
  • the phone in the other room is in a locked screen state at this time, based on the existing technology, the user cannot directly start the application window on the PC that displays the message. Instead, they must first go to the room to unlock the phone and put it in a bright screen state before they can start the screen projection window corresponding to the phone on the PC.
  • the screen projection window corresponding to the mobile phone can be started on the PC to display the screen projection window, thereby realizing the window display of heterogeneous screen projection, thereby facilitating the user to process the message and improving the user experience.
  • the target device may receive one or more application messages transferred from any source device in the same trust ring.
  • the PC may respond to user operations and display a message list window, which may centrally display received message notifications or centrally display received and unprocessed message notifications.
  • the above message notifications may also be displayed in the message list window in the form of message cards.
  • FIG 8 is a schematic diagram of displaying a message list window.
  • the PC detects a user's sliding operation on the touch screen and can display a message list window 801.
  • the message list window 801 displays application messages from mobile phone 1 (such as message card 802), application messages from mobile phone 2 (such as message card 803), etc.
  • the message notifications displayed in the message list window 801 are all information recently received by the PC from other devices in the trust ring.
  • message card 802 corresponds to the message pushed by the social application in mobile phone 1.
  • Message card 803 corresponds to the message pushed by the information application in mobile phone 2 (for example, text messages).
  • the PC when the PC detects that the user has selected any message card in the message list window 801, it can trigger the device corresponding to the message card to cast the screen. For example, when the PC detects that the user has selected message card 802, it can trigger mobile phone 1 to cast the screen. In this way, mobile phone 1 is the source device and the PC is the corresponding target device. For another example, when the PC detects that the user has selected message card 803, it can trigger mobile phone 2 to cast the screen. In this way, mobile phone 2 is the source device and the PC is the corresponding target device.
  • the embodiments of the present application provide a method for screen projection, which can be applied to electronic devices such as mobile phones, tablet computers, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc.
  • UMPCs ultra-mobile personal computers
  • PDAs personal digital assistants
  • the embodiments of the present application do not impose any restrictions on the specific types of electronic devices.
  • FIG. 9 is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application.
  • the electronic device 100 may include a processor 110, an antenna 1, an antenna 2, a mobile communication module 120, a wireless communication module 130, an audio module 140, a speaker 140A, a receiver 140B, a microphone 140C, an earphone interface 140D, a sensor module 150, a display screen 160, etc.
  • the sensor module 150 may include a proximity light sensor 150A, a fingerprint sensor 150B, a touch sensor 150C, an ambient light sensor 150D, etc.
  • the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100.
  • the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently.
  • the components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
  • the processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and/or a neural-network processing unit (NPU), etc.
  • AP application processor
  • GPU graphics processor
  • ISP image signal processor
  • DSP digital signal processor
  • NPU neural-network processing unit
  • Different processing units may be independent devices or integrated in one or more processors.
  • the wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 120, the wireless communication module 130, the modem processor and the baseband processor.
  • the mobile communication module 120 can provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 .
  • the wireless communication module 130 can provide wireless communication solutions for application in the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.
  • WLAN wireless local area networks
  • Wi-Fi wireless fidelity
  • BT Bluetooth
  • GNSS global navigation satellite system
  • FM frequency modulation
  • NFC near field communication technology
  • IR infrared technology
  • the electronic device 100 implements the display function through a GPU, a display screen 160, and an application processor.
  • the GPU is a microprocessor for image processing, which connects the display screen 160 and the application processor.
  • the GPU is used to perform mathematical and geometric calculations for graphics rendering.
  • the processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.
  • the electronic device 100 can implement audio functions, such as voice calls, through the audio module 140 and the application processor.
  • the speaker 140A also called a "speaker" is used to convert an audio electrical signal into a sound signal.
  • the electronic device 100 can listen to music or listen to a hands-free call through the speaker 140A.
  • the receiver 140B also called a "earpiece" is used to convert audio electrical signals into sound signals.
  • the voice can be received by placing the receiver 140B close to the human ear.
  • Microphone 140C also called “microphone” or “microphone” is used to convert sound signals into electrical signals.
  • the user can speak by approaching the microphone 140C with his mouth to input the sound signal into the microphone 140C.
  • the earphone jack 140D is used to connect a wired earphone.
  • the proximity light sensor 150A may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode.
  • the electronic device 100 may use the proximity light sensor 150A to detect when a user holds the electronic device 100 close to their ear to make a call, so as to automatically turn off the screen to save power.
  • the proximity light sensor 150A may also be used for automatic unlocking and locking of the screen in the leather case mode and the pocket mode.
  • the fingerprint sensor 150B is used to collect fingerprints.
  • the electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc.
  • the touch sensor 150C is also called a "touch control device”.
  • the touch sensor 150C can be set on the display screen 160, and the touch sensor 150C and the display screen 160 form a touch screen, also called a "touch control screen”.
  • the touch sensor 150D is used to detect touch operations acting on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output related to the touch operation can be provided through the display screen 160.
  • the touch sensor 150C can also be set on the surface of the electronic device 100, which is different from the position of the display screen 160.
  • the ambient light sensor 150D is used to sense the ambient light brightness.
  • the electronic device 100 can adaptively adjust the brightness of the display screen 160 according to the perceived ambient light brightness.
  • the ambient light sensor 150D can also be used to automatically adjust the white balance when taking pictures.
  • the ambient light sensor 150B can also cooperate with the proximity light sensor 150A to detect whether the electronic device 100 is in a pocket to prevent accidental touch.
  • the layered architecture divides the software into several layers, each with clear roles and division of labor.
  • the layers communicate with each other through software interfaces.
  • the right half of Figure 10 is a software architecture diagram of the source device, such as a mobile phone.
  • the left half of Figure 10 is a software architecture diagram of the target device, such as a tablet (Pad) or PC.
  • the software framework of the source device may include an application layer, a capability service layer, an application framework layer (framework, FWK), an Android runtime (Android runtime) and a system library and a driver layer.
  • the application layer may include a series of application packages, such as social, information, call, camera and other applications (also referred to as applications).
  • exemplary, social applications include WeChat, QQ, etc.
  • information applications include text messages, etc.
  • the application is referred to as the application below.
  • the application on the source device can be a native application (such as an application installed in the source device when the operating system is installed before the source device leaves the factory), or it can be a third-party application (such as an application downloaded and installed by the user through the application store), which is not limited in the embodiments of this application.
  • the capability service layer provides capability support for implementing corresponding services. As shown in Figure 10, it may include a screen projection assistant module, a first virtualization service module, a first screen projection service module, a first device discovery authentication connection module, etc.
  • the screen projection assistant module can be a module used to interact with other electronic devices (such as target devices) to exchange screen projection related information, realize end-to-end logical control and connection management of screen projection, and generate a session key (SessionKey) to ensure the security of the communication session between the source device and the target device.
  • the screen projection assistant module may include functional modules such as a screen projection management module, a heterogeneous switching module, and a virtualization service initialization module.
  • the screen projection management module is responsible for managing screen projection related matters, such as adding and removing related processing logic settings of VirtualDisplay.
  • VirtualDisplay can also be called a virtual display area, a virtual screen, or a virtual Display.
  • the heterogeneous switching module is used to control the source device to switch from a heterogeneous screen projection mode to a homogeneous screen projection mode.
  • the virtualization service initialization module makes initialization settings for the virtualization service that starts heterogeneous screen projection.
  • the first virtualization service module is used to implement end-to-end logical control of screen projection of audio and video streams, and is responsible for the audio and video data streams generated by Speakers, Mics, Cameras, etc. transmitted between the source device and the target device.
  • the first screen projection service module may include a first display virtualization module and a first reverse control module, respectively providing The basic capabilities of screen projection (i.e., the content of the source device is projected to the target device for display) and reverse events (i.e., events that control the source device triggered by the target device).
  • the first screen projection service module receives instructions from the screen projection management module, and provides corresponding screen projection services according to the instructions, thereby realizing the projection of programs running on the local device to other electronic devices (such as the target device).
  • the first screen projection service module injects reverse control and input method into the display of the target device (hereinafter referred to as the external screen), and adds a virtual display tag (such as Flag1) when creating a virtual display area, so that the external screen only displays its own content, and does not mirror the content of the display of the source device (hereinafter referred to as the main screen), thereby realizing heterogeneous screen projection.
  • a virtual display tag such as Flag1
  • the first device discovery connection authentication module is responsible for discovery, connection, authentication and other tasks, and is used to control the driver layer to implement proximity discovery, authentication, connection and other functions between the source device and the target device, and may include a first communication module and a first security authentication module.
  • the first communication module provides a signaling transmission channel between the source device and the target device, and is used to send instructions to the driver layer to establish a communication connection and a screen projection connection between the source device and the target device. At the same time, the first communication module is also used to start the screen projection assistant module after the transmission channel is established.
  • the first security authentication module is used for security authentication, and is responsible for owner identification and initiating distributed unlocking authentication to obtain a trust ring and ensure the privacy and security of users.
  • the application framework layer provides application programming interfaces (API) and programming frameworks for applications in the application layer.
  • the application framework layer includes some predefined functions. As shown in Figure 10, the application framework layer may include an activity management service (ActivityManagerService, AMS), a window management service (WindowManagerService, WMS), a multi-screen framework module, a SystemUI module, etc.
  • AMS is responsible for managing Activities, starting, switching, and scheduling various components in the system, and managing and scheduling applications.
  • ActivityRecord application record
  • AMS For each Activity, there will be a corresponding application record (ActivityRecord) in AMS, which records the status of the Activity of the application.
  • ActivityRecord Each activity will create a corresponding ActivityRecord object in AMS, and AMS can use this ActivityRecord as an identifier to schedule the Activity process of the application; these ActivityRecord objects are managed in their respective task stacks, and each task stack can add multiple ActivityRecord objects; all task stacks are uniformly managed by ActivityStack, which is responsible for the order, stacking, and popping of TaskStack.
  • AMS defines data classes for saving processes (Process), activities (Activity), and tasks (Task).
  • the data class corresponding to the process can include process file information, process memory status information, and the activities and services contained in the process.
  • Activity information can be saved in ActivityStack.
  • ActivityStack is used to uniformly schedule application activities.
  • ActivityStack can specifically save the information of all running Activities (i.e. final ArrayList mHistory), such as interface configuration information.
  • the running Activity can be saved in newArrayList.
  • ActivityStack can also save the information of historically run Activities, such as interface configuration information. It should be noted that an Activity does not correspond to an application, but an ActivityThread corresponds to an application. Therefore, Android allows multiple applications to run at the same time, which actually allows multiple ActivityThreads to run at the same time.
  • each application process reports to AMS when it wants to start a new Activity or stop the current Activity.
  • AMS internally keeps records for all application processes.
  • AMS receives a start or stop report, it first updates the internal record and then notifies the corresponding client process to run or stop the specified Activity. Since AMS has records of all Activities internally, it can schedule these Activities and automatically close the background Activity according to the status of the Activity and system memory.
  • WMS carries data and attributes related to the "interface” and is used to manage the state related to the "interface", that is, to manage the graphical user interface (GUI) resources used on the mobile phone screen, such as managing window programs and event dispatching.
  • managing window programs means outputting to the physical screen or other display devices in an orderly manner with the assistance of the application server and WMS according to the display request of the application.
  • Event dispatching means dispatching user events from the keyboard, physical buttons, touch screen, mouse, trackball (TraceBoll), etc. to the corresponding controls or windows.
  • the management window program can also obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc., and is responsible for managing the display mode of the application window, including the coordinate size of the window display, the window display level, etc. Specifically, it may include: creating and destroying windows, displaying and hiding windows, layout of windows, management of focus, and management of input methods and wallpapers, etc.
  • WMS can ensure the visibility of the projection window when the mobile phone is locked and the screen is off.
  • the task stack management function of AMS is used to manage the life cycle of Activities and the scheduling between Activities.
  • the life cycle management function of WMS is used to manage the life cycle of windows. The two can cooperate with each other to activate the window through the callback (onResume) interface, and the activated window is displayed in the foreground; or, the window can be paused through the pause (onPause) interface to switch the window from the foreground to the background.
  • the top Window (top Activity) in the task stack as an example, the top Activity calls onResume to activate it, so the top Activity is displayed in the foreground.
  • the original top Activity calls onPause to pause the window so that the original top Activity is switched from the foreground to the background. Similar principles can be used to manage the life cycle of new windows.
  • the multi-screen framework module is responsible for implementing the heterogeneous screen projection logic, calling the PendingIntetRecord interface to start the Acvtivity corresponding to PendingIntet (specify the behavior send), and starting the application corresponding to the Activity to the virtual display area by adding a heterogeneous screen projection tag (such as Flag2).
  • PendingIntent is a capability provided by Android for external programs to call their own programs. Its life cycle is not related to the main program. When certain conditions are met or certain events are triggered, the specified behavior is executed.
  • the multi-screen framework module is also used to monitor sensing events and interact with the screen projection assistant module.
  • the multi-screen framework module can be used to translate, encapsulate, and process the original input events to obtain input events containing more information and send them to WMS, which stores the clickable areas (such as controls) of each application, the location information of the focus window, etc. Therefore, WMS can correctly distribute input events to the specified controls or focus windows.
  • the multi-screen framework module can respond to the received input events, forward the input events to the WMS, the WMS distributes them to the corresponding applications, the applications send registration requests to the AMS, and the applications are started.
  • the multi-screen framework module is also used to switch the application projected to the external screen back to the main screen when the user clicks on the recent task card or desktop icon.
  • the application projected to the external screen adopts a heterogeneous screen projection method
  • it is necessary to call the heterogeneous switching module mentioned above to switch the heterogeneous screen projection method to the homogeneous screen projection method otherwise the application cannot be displayed on the main screen.
  • the multi-screen framework module is also used to remove the security layer when the source device is in the locked screen state.
  • the multi-screen framework module can also be used to make relevant modifications to the power button of the mobile phone in the screen projection scenario.
  • the SystemUI module is a set of UI components that provide users with system-level information display and interaction. It is mainly used to implement status bar information display (such as battery, wifi signal, 3G/4G icon display), notification panel (such as system messages, third-party message notifications), recent taskbar display panel (such as display of recently used Application), etc.
  • the SystemUI module is responsible for the notification service, which is used to transfer the source device message notification flow to the target device.
  • the application framework layer may also include a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. (not shown in FIG. 10 ). The specific meanings thereof may be referred to in relevant technical documents and will not be described here.
  • the Android runtime is responsible for the scheduling and management of the system.
  • the Android runtime includes a core library and a virtual machine.
  • the core library consists of two parts: one is the function that the programming language (for example, the Java language) needs to call, and the other is the core library of the system.
  • the application layer and the application framework layer run in the virtual machine.
  • the virtual machine executes the programming files (for example, Java files) of the application layer and the application framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • the system library can include multiple functional modules. For example: Surface Manager, Media Libraries, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc.
  • the surface manager is used to manage the display subsystem and provides the fusion of 2D (2-Dimensional, 2D) and 3D (3-Dimensional, 3D) layers for multiple applications.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.
  • the 2D graphics engine is a drawing engine for 2D drawing.
  • the driver layer is the foundation of the Android operating system, and the final functions of the Android operating system are completed through the driver layer.
  • the driver layer includes the underlying driver, which is responsible for discovery, authentication, connection, etc.
  • the driver layer receives commands issued by the communication module in the application framework layer and performs actions such as connection and disconnection.
  • the driver layer includes WI-FI P2P chips, Bluetooth chips, etc., which can establish transmission channels such as WI-FI P2P and Bluetooth.
  • the driver layer can also include hardware drivers such as display drivers, camera drivers, audio drivers, sensor drivers, virtual card drivers, etc.
  • the software framework of the target device may include a PC manager, an operating system, and a driver layer.
  • Some functional modules of the target device and their functions can refer to the above-mentioned source device and will not be described here.
  • the PC manager of the target device includes a message center module, a second virtualization service module, a second screen projection service module, and a second device discovery connection authentication module.
  • the message center module is used to interact with the messages and signaling of the source device, receive message notifications from the source device, and initiate session instructions (openSession) to the source device.
  • the module is also used to issue instructions to start the screen projection service after a session is established.
  • the functions of the second virtualization service module, the second screen projection service module, and the second device discovery authentication connection module can respectively refer to the first virtualization service module, the first screen projection service module, and the first device discovery connection authentication module in the above-mentioned source device.
  • the second screen projection service module is used to receive and parse video streams and related information sent by other devices, so as to display applications running on other devices on the local device.
  • the operating system of the target device can be Windows OS (such as a personal computer) or Android OS (such as a tablet).
  • driver layer of the target device please refer to the above source device.
  • Figures 11 to 13 are timing diagrams of a method and device for projecting a screen provided in an embodiment of the present application. It can be understood that Figures 11 to 13 are a set of continuous timing diagrams, which illustrate the interaction process between the various functional modules shown in Figure 11.
  • the mobile phone end includes a SystemUI module, a first communication module, a screen projection assistant module, a first virtualization service module, a first screen projection service module, a first security authentication module, FWK, and an application;
  • the PC end includes a message center module, a second screen projection module, a second communication module, and a second security authentication module.
  • the other functional modules shown in Figure 10 also participate in the specific implementation process of a screen projection method and device provided in an embodiment of the present application, which is not shown in this timing diagram.
  • the mobile phone is in a locked screen or off screen state.
  • the source device and the target device establish a first communication connection, the application message flow of the source device is transferred to the target device, and a message notification control is displayed on the target device (see Figure 11)
  • the message center module of the PC receives an operation of selecting a mobile phone by a user.
  • S101 is only an example of establishing a first communication connection between a PC and a mobile phone.
  • the PC detects a device in the same local area network as the PC, displays all connectable devices for the user to select, and the user selects the mobile phone on the PC to start the subsequent connection behavior.
  • the mobile phone detects a device in the same local area network as the PC, displays all connectable devices for the user to select, and the user selects the PC on the mobile phone to start the subsequent connection behavior.
  • the mobile phone and the PC can also establish a near-field connection through the "touch and pay" function, the NFC function, etc., which is not limited here.
  • the message center module sends a message to the second communication module instructing to establish a communication connection with the mobile phone, and the message carries a device identification code of the mobile phone.
  • the user selects the mobile phone as the collaborative notification device from the nearby available collaborative notification devices found by the PC search.
  • the message center module of the PC can send an inter-process communication (IPC) message to the second communication module.
  • IPC inter-process communication
  • the IPC message can carry the device name, device identification code, MAC address, etc. of the mobile phone, which is used to instruct the second communication module to establish a communication connection with the mobile phone.
  • the second communication module establishes a first communication connection with the first communication module of the mobile phone.
  • the first communication connection may be a Bluetooth connection (BR connection), which is used to implement collaborative notifications between the PC and the mobile phone, that is, messages on the mobile phone may be transferred to the PC via a Bluetooth channel.
  • BR connection Bluetooth connection
  • the second communication module returns a message to the message center module indicating that the first communication connection is successfully established.
  • S105 The application of the mobile phone receives an application message.
  • the mobile phone After the PC and the mobile phone establish the first communication connection, the mobile phone receives an application message pushed by the application installed on the mobile phone.
  • the application message can have different forms of expression in the application on the mobile phone (such as the first data), the SystemUI module, and the message center module on the PC (such as the first message), collectively referred to here as application messages.
  • the application sends an application message to the SystemUI module.
  • the application message carries the message content and the application package name.
  • S107 The SystemUI module instructs the first communication module to forward the application message to the PC.
  • the SystemUI module will transfer all messages received by the mobile phone to the PC based on the first communication connection; it may also be transferred selectively, for example, messages with a special tag will be transferred to the PC, or messages from specific applications will be transferred to the PC, or messages judged to be of higher importance will be transferred to the PC, etc., which are not limited here.
  • the first communication module sends an application message to the second communication module based on the first communication connection, where the application message carries message content and an application package name.
  • the second communication module sends an application message to the message center module, where the application message carries message content and an application package name.
  • the message center module receives the application message and controls the display screen to display the corresponding message notification control.
  • the application message received on the mobile phone will flow to the PC. It is understandable that at this time, whether the mobile phone is in an unlocked screen state or a locked screen or a screen off state does not affect the flow of message notifications between the PC and the mobile phone.
  • the PC After the PC receives the application message, it displays the message notification control associated with the application message.
  • the mobile phone is in the screen-off state.
  • a message notification control associated with the application message such as a message card 702 , will be displayed on the PC.
  • the target device receives a user operation on the message notification control, the target device establishes a third communication connection with the source device, and the target device sends a request to start heterogeneous screen projection to the source device.
  • the application projection window is displayed on the target device (see Figures 12-13)
  • the message center module sends a message to the second communication module instructing the creation of a second session.
  • the message carries the service channel name (WI-FI P2P channel) and the device identification code of the mobile phone.
  • the service channel name can be a WI-FI P2P channel
  • the device identification code is the identification code of the mobile phone
  • the above-mentioned instruction to create a second session is used to instruct the second communication module to establish a WI-FI P2P channel between the PC and the mobile phone corresponding to the device identification code.
  • the second communication module establishes a third communication connection with the first communication module.
  • the third communication connection can be a WI-FI P2P connection, providing a data transmission channel for the screen projection service.
  • the types of the first communication connection, the second communication connection, and the third communication connection established between the PC and the mobile phone are not limited to Bluetooth connection and Wi-Fi P2P connection, and may also be other types of connection modes. As long as the requirements for data transmission between the two can be met.
  • the first communication module sends a message instructing the screen projection assistant module to start.
  • the first communication module is also responsible for interacting with the screen projection assistant.
  • the screen projection assistant module is in a closed state when the mobile phone has no screen projection demand.
  • the first communication module triggers the screen projection assistant module to start when there is a screen projection demand.
  • the second communication module returns a message to the message center module indicating that the third communication connection is successfully established.
  • the message center module Based on the third communication connection, the message center module negotiates parameters with the screen projection assistant to determine the session key and the IP address of the other party.
  • SessionKey also known as data encryption key or working key
  • SessionKey is a randomly generated encryption and decryption key to ensure a secure communication session between a user and other computers or between two computers. It can be obtained through negotiation between communication users. It is generally generated dynamically and only when session data encryption is required.
  • the message center module sends a message indicating startup to the second screen projection service module, and the message carries the session key and the IP address of the mobile phone.
  • S209 The second screen projection service module is started and initialized according to the session key and the IP address of the mobile phone.
  • the second screen projection service module returns a message indicating that initialization is completed to the message center module.
  • the screen projection service module provides screen projection capabilities and basic reverse event control capabilities. After the screen projection service on the PC side is successfully initialized, it provides capability support for receiving and parsing relevant information sent by the mobile phone side, so as to display the application running on the mobile phone side on the PC side. At the same time, the basic reverse event control capability provides support for heterogeneous screen projection. Therefore, before performing heterogeneous screen projection services, the screen projection service module needs to be initialized to ensure the normal provision of screen projection capabilities and basic reverse event control capabilities.
  • the message center module sends a message to the second screen projection service module to instruct the start of the screen projection service.
  • the second screen projection service module Based on the third communication connection, the second screen projection service module sends a message to the screen projection assistant module to start the screen projection service.
  • step S207 includes the second screen projection service module sending a message to the second communication module, the second communication module then transmits the message to the first communication module through the third communication connection, and the first communication module then sends the message to the screen projection assistant module.
  • the following "based on the third communication connection" includes the data transmission process between the first communication module and the second communication module, which will not be repeated below.
  • the screen projection management module in the screen projection assistant module is bound to the relevant screen projection services to manage screen projection affairs.
  • the screen projection assistant module sends a message indicating initialization to the first virtualization service module.
  • the virtualization service initialization module in the screen projection assistant module is used to instruct the virtualization service module to initialize.
  • S215 The first virtualization service module completes initialization.
  • the first virtualization service module is responsible for the audio and video virtualization in the screen projection service.
  • the interaction between the first virtualization service module on the mobile phone and the second virtualization service module on the PC can enable the mobile phone application projected to the PC to use the PC's Speaker, Mic, and Camera during the screen projection process, and transmit the generated audio and video data streams between the mobile phone and the PC, thereby realizing functions such as making and receiving audio and video calls during screen projection.
  • the first virtualization service module sends a message to the first screen projection service module to instruct the start of the screen projection service.
  • the first screen projection service module creates a virtual display and adds a virtual display tag to it.
  • the screen projection service module is an important functional module for implementing heterogeneous screen projection.
  • the specific implementation method is that the first screen projection service module on the mobile phone adds Flag1: VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY (virtual display tag) when creating a virtual display area, which is used to realize that the screen projection window on the PC only displays its own content, and does not mirror the content displayed on the mobile phone display, so as to ensure that the display content of the mobile phone will not affect the display content of the screen projection window of the PC.
  • VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY virtual display tag
  • the PC's projection window can maintain its own content without being synchronized with the display content of the phone.
  • the basic reverse event control capabilities provided by the projection service module can achieve reverse control of the mobile phone on the computer side.
  • the implementation of the heterogeneous projection method also requires the second projection service module on the PC side to interact with the first projection service module on the mobile phone side, which is not shown in Figure 12.
  • the Android system will cover the lock screen layer on the phone display on the VirtualDispaly by default, which means that the projection window on the PC display will also be covered by the lock screen layer, that is, the projection window on the PC also displays the lock screen layer, and the projection is abnormal.
  • the virtual reality area with Flag1 added will not mirror the content displayed on the phone display to the projection window of the PC, realizing heterogeneous projection, which solves the problem of the projection window showing a black screen or lock screen interface as shown in Figures 5 and 6.
  • the first screen projection service module sends a message to the screen projection assistant module indicating that the screen projection service has been started.
  • the first screen projection assistant Based on the third communication connection, the first screen projection assistant sends a message to the message center module indicating that the screen projection service has been started.
  • the message center module Based on the third communication connection, the message center module sends a message to SystemUI to start the application for screen projection, carrying the application package name.
  • S222 SystemUI sends a message to FWK to start application screen projection, and the message carries the application package name.
  • FWK is required to control the visibility of the heterogeneous projection window of the application. It is understandable that the specific implementation process of FWK involves the functional interaction between the embedded classes of FWK. Please refer to the following for the interaction process, which will not be repeated here.
  • S224: FWK sends the display data of the application screen projection to the first screen projection service module.
  • the first screen projection service module Based on the third communication connection, the first screen projection service module sends the display data of the application screen projection to the second screen projection service module.
  • the second screen projection service module controls the display screen to render the above display data to the screen projection window.
  • the second screen projection service module sends a message to the message center indicating that the rendering of the application screen projection interface of the screen projection window is complete.
  • the message center module controls the display screen to display the application projection window.
  • the PC displays the application screen projection window corresponding to the application message, and the screen projection is successful when the phone is in the locked screen state.
  • the mobile phone is in the screen-off state, and when the PC receives the user's operation of clicking the message card 702, the PC will display the projection window 703 corresponding to the above social application, which is used to display the application interface 704 corresponding to the application message.
  • the above application projection window can be a projection window 703 for displaying the application interface 704.
  • the Activity corresponding to the application message When the phone is locked or off, the Activity corresponding to the application message will enter the dormant state, and its life cycle will go from the OnResume state (visible and interactive to the user) to the OnPause state (inactive state), so the application interface cannot be displayed normally when projecting the screen to the PC.
  • the OnResume state visible and interactive to the user
  • OnPause state active state
  • the application interface cannot be displayed normally when projecting the screen to the PC.
  • FWK also includes embedded classes such as PendingIntentRecord (asynchronous processor), ActivityStarter (activity starter), ActivityTaskManagerServiceEx (activity stack management server), ActivityTaskSupervisor (activity stack manager), KeyguardController (lock screen controller), ActivityRecord (activity recorder), Task (task stack), RootWindowsContainer (root window container), SurfaceFlinger (layer synthesizer), etc.
  • PendingIntentRecord asynchronous processor
  • ActivityStarter activity starter
  • ActivityTaskManagerServiceEx activity stack management server
  • ActivityTaskSupervisor activity stack manager
  • KeyguardController lock screen controller
  • ActivityRecord activity recorder
  • Task task stack
  • RootWindowsContainer root window container
  • SurfaceFlinger layer synthesizer
  • steps S301 to S318 shown in FIG. 14 are between steps S222 and S224 shown in FIG. 13 .
  • S222 The SystemUI module sends a request to PendingIntentRecord to start application screen projection, and the request carries the application package name (SystemUI sends a message to FWK to start application screen projection, and the message carries the application package name).
  • S301 PendingIntentRecord marks all the Activities of the application with a projection tag.
  • Step S301 is equivalent to an Activity filter, which filters out all the Activities of the application, thereby marking all the Activities of the application that need to start projection with special marks.
  • the application can also include Activity 2 and Activity 3, which are also marked with the heterogeneous casting tag Flag2. Because after the application is cast, the user can control and execute other activities of the application in the application casting window, so all activities of the application need to be filtered out to perform the following related steps.
  • PendingIntentRecord sends a request to ActivityStarter to start an Activity marked with a screen projection tag, and the request carries the application package name.
  • ActivityTaskSupervisor calls KeyguardController to monitor whether the Activity is in the OnResume state. If it is, it means that the application screen projection window corresponding to the Activity is visible to the user and can interact with the user after it is launched on the PC (OnResume state); if the Activity is in the stopped state (OnPause state), it means that the application screen projection window corresponding to the Activity is in the background after it is launched on the PC, and the user cannot operate it.
  • the OnResume state and the OnPause state are two states in the life cycle of the Activity.
  • the Activity When the Activity is in the OnResume state, the Activity is in an activated state that is visible and can interact with the user, and the user can get the focus of the activity; when the Activity is in the OnPause state, the Activity is covered by another transparent or Dialog-style Activity. At this time, it still maintains a connection with the window manager, and the system continues to maintain its internal state. It is still visible, but it has lost focus and cannot interact with the user.
  • the AMS component When the phone is in the locked or off screen state, the AMS component will switch the Activity life cycle to the onPause state during the process of starting the Activity, causing the Activity to be unable to interact with the user, so the screen projection will be abnormal.
  • S305 KeyguardController identifies the Activity with the projection tag and sets it to the OnResume state.
  • KeyguardController is a lock screen controller. It identifies the Activity with the projection tag and returns "true” to indicate that the Activity is set to the OnResume state, ensuring that when the source device is in the lock screen state, the application projection window corresponding to the Activity with the projection tag is visible on the target device.
  • S306 KeyguardController sends a message to ActivityRecord instructing to keep the Activity with the projection tag in the OnResume state.
  • ActivityRecord sends a verification message to Task, instructing Task to verify the status of the Activity at the top of the task stack.
  • Task determines that the Activity at the top of the task stack is in the OnResume state.
  • an application may correspond to one or more Activities, and one or more Activities will be stored in one or more task stacks.
  • the task stack follows the last-in-first-out rule, and the Activity at the top of the task stack is executed first. In order to ensure that the application projection window corresponding to the Activity to be executed is visible in the foreground, it is necessary to ensure that the Activity at the top of the stack is in the OnResume state.
  • steps S308 and S309 are equivalent to a fault-tolerant mechanism, which checks whether the Activity at the top of the task stack is in the OnResume state, so these two steps are optional, that is, in other embodiments, S308 and S309 may not be included.
  • RootWindowsContainer traverses the Activities of all tasks, skips the Activities with the projection tag, and controls other Activities to enter the dormant state.
  • RootWindowsContainer needs to traverse all the activities corresponding to the tasks, filter out the Activities with the screen projection tag and return "false” to indicate that the Activity does not enter sleep; other Activities return "true” to enter the sleep process.
  • Activity1 and Activity2 in the same task stack are both added with heterogeneous screen projection tags, and RootWindowsContainer makes Activity1 and Activity2 skip the sleep process; while Activity3 and Activity4 that are not added with heterogeneous screen projection tags are put into sleep state.
  • Task checks the status of Activity1 to ensure that it is in the OnResume state. Then, after Activity1 is projected from a different source, it is visible to the user on the PC and can be operated by the user in the foreground.
  • Activity2 After executing Activity1, Activity2, which is not dormant at this time, is advanced to the top of the task stack. Task checks the status of Activity2 to ensure that it is in the OnResume state. Then Activity2 is visible to the user on the PC after being projected from another source and can be operated by the user in the foreground. The execution process of other Activities that have been marked with projecting from another source is similar. The Activity that has not been marked with the screen projection will be put into a dormant state by the RootWindowsContainer and will not be started to display on the external screen.
  • S312 ActivityStarter sends a message to the application to instruct it to start.
  • step S223 includes the interaction between FWK and the application layer, that is, ActivityStarter in FWK calls up the application of the application layer.
  • S314 The application sends a message indicating successful startup to ActivityTaskManagerServiceEx.
  • S317 SurfaceFlinger determines that the virtual display area has a virtual display tag, synthesizes the application layer on the virtual display area, and obtains corresponding display data.
  • the role of SurfaceFlinger is to receive graphics display data from multiple sources, synthesize them, and then send them to the display device. For example, when opening an application, there are three common layers of display, the status bar at the top, the navigation bar at the bottom or side, and the application interface. Each layer is updated and rendered separately. These interfaces are synthesized by SurfaceFlinger into an application layer display data refreshed to the hardware display.
  • SurfaceFlinger determines that the virtual display area has a virtual display tag, then when synthesizing the application layer, only the layer on the virtual display area is obtained, and the layers of other display areas will not be obtained to synthesize with it, so that the aforementioned heterogeneous screen projection method can be realized, that is, the synthesized application layer is only related to the layer of the virtual display area.
  • SurfaceFlinger will not obtain the layer of the visible display area, so even if the visible display area includes the lock screen interface or other application interfaces, it will not affect the interface displayed by the PC's projection window. Because SurfaceFlinger only obtains the layer of the virtual display area and passes it to the PC's projection window for display in some way.
  • S318 SurfaceFlinger sends display data to the first screen projection service module.
  • the above application layer may include the application interface corresponding to the currently executed Activity.
  • the above display data may include information such as different display objects in the application layer, display positions and sizes of the display objects in the virtual display screen.
  • the virtual display area is the virtual display area marked Flag1 created in step S218.
  • the content of the virtual display area is invisible to the user, and the mobile phone can use the virtual display area as a canvas that occupies a certain storage space.
  • Figure 17A is a process diagram for starting the first application projection window.
  • the PendingIntentRecord will mark all Activities of application 1 with a projection label, as shown in the task stack (created by Task) in Figure 17A.
  • Activity1 of application 1 Activity1 corresponds to the first message
  • the mobile phone draws the first application interface corresponding to Activity1 in the virtual display area.
  • SurfaceFlinger determines that the virtual display area has a virtual display label, then only obtains the first application interface layer in the virtual display area, and generates the corresponding first display data and sends it to the target device.
  • the target device After receiving the first display data, the target device draws and renders the application screen projection interface corresponding to the first message of the first application according to the first display data, and displays it in the screen projection window of the target device.
  • the projection window of the target device displays the application projection interface corresponding to the first display data.
  • the mobile phone When executing Activity2 of Application 1 (Activity2 is pushed to the top of the task stack), the mobile phone draws the second application interface corresponding to Activity2 in the virtual display area.
  • SurfaceFlinger determines that the virtual display area has a virtual display tag, then only obtains the second application interface layer in the virtual display area, and generates the corresponding second display data and sends it to the target device. Then the projection window of the target device displays the application projection interface corresponding to the second display data, that is, the application projection interface corresponding to the first display data is switched to the application projection interface corresponding to the second display data, completing the switching of different application interfaces of Application 1 in the projection window.
  • Figure 17B is a process diagram for starting the second application projection window.
  • the source device receives a second message from the second application (also called the second data). After the second message flows to the target device, the target device displays the message notification control corresponding to the second message. After the target device receives the operation of the source device clicking the message notification control, it sends a second request to the source device to start the projection.
  • the first projection service module receives the second request, creates a second virtual display area (VirtualDispaly2) and adds a virtual display tag.
  • Activity 3 of Application 2 Activity 3 corresponds to the second message
  • the mobile phone draws the third application interface corresponding to Activity 3 in the virtual display area.
  • SurfaceFlinger determines that the virtual display area has a virtual display tag, then only obtains the third application interface layer in the virtual display area, and generates corresponding third display data and sends it to the target device.
  • the target device After the target device receives the third display data, it draws and renders the application projection interface corresponding to the second message of the second application according to the third display data, and displays it in the projection window of the target device. Please refer to steps S225-S228 in Figure 13 and Figure 7 for details.
  • the projection window of the target device displays that the application projection interface corresponding to the second display data is switched to the application projection interface corresponding to the third display data, completing the switching of application interfaces of different applications (such as the first application and the second application) in the projection window.
  • the source device can optionally enable a security authentication function to protect the privacy of the source device with a locked or off screen.
  • the security authentication interface 705 can also be displayed, and the transition loading interface 706 can also be displayed.
  • the security authentication mechanism will be triggered first.
  • the specific implementation process please refer to Figure 14 and the following description.
  • the target device When the source device is in a locked screen or off screen state, the target device receives a user operation on a message notification control, and the source device establishes a second communication connection with the target device (see FIG. 15 )
  • step S201 in Figure 12 are both operations in which the user clicks on the message notification control.
  • steps shown in Figures 15 and 16 are added before triggering the establishment of the third communication connection in Figure 12.
  • the security authentication mechanism will be triggered before the screen projection is started from the PC to ensure the user's privacy security.
  • the security authentication mechanism Since the embodiment of the present application directly starts the screen projection on the PC side, the security authentication mechanism will be triggered on the PC side, and the password authentication is distributed on the mobile phone side, so the second security authentication module on the PC side needs to send the authentication information (password) received on the PC side to the first security authentication module on the mobile phone side for password authentication, and the mobile phone side will return the authentication result to the PC side. Therefore, in order to transmit the signaling required for security authentication between the mobile phone and the PC, a transmission channel needs to be established first.
  • the message center module sends a message instructing to create a first session to the second communication module.
  • the message carries a service channel name (Bluetooth channel) and a device identification code of the mobile phone.
  • the service channel name may be a Bluetooth channel
  • the device identification code may be an identification code of a mobile phone
  • the instruction for creating the first session is used to instruct the second communication module to establish a Bluetooth channel between the PC and the mobile phone corresponding to the device identification code.
  • the second communication module establishes a second communication connection (Bluetooth connection) with the first communication module.
  • the second communication connection may be a Bluetooth connection (BR connection), which provides a securely authenticated signaling transmission channel.
  • BR connection Bluetooth connection
  • the first communication module sends a message instructing the screen projection assistant to start.
  • the second communication module returns a message to the message center module indicating that the second communication connection is successfully established.
  • the message center module sends a message instructing the second screen projection service module to start.
  • the second screen projection service module returns a message indicating successful startup to the message center module.
  • the screen projection assistant on the mobile phone and the second screen projection service module on the PC have been started, providing the necessary functions for realizing the subsequent screen projection service.
  • the target device sends a security authentication request to the source device, and the source device returns a successful authentication result (see Figure 16)
  • the message center module sends a security authentication request to the second security authentication module of the PC.
  • the second security authentication module controls the display screen of the PC to display a security authentication window.
  • a security authentication interface 705 is displayed in the screen projection window 703 , and the user can enter a password in the security authentication window.
  • the security authentication method is not limited to entering a password for security authentication.
  • fingerprints are collected for verification
  • voice unlocking voices are collected for verification.
  • password unlocking is only used as an example.
  • the second security authentication module receives the password input by the user through the security authentication window.
  • the second security authentication module sends a password to the second communication module.
  • the second communication module sends a password to the first communication module based on the second communication connection.
  • the first communication module sends a password to the first security authentication module of the mobile phone.
  • the security authentication method can also be fingerprint authentication by the user inputting a fingerprint, voice authentication by the user inputting a voice, etc., and this is not limited.
  • the first security authentication module receives the password collected by the PC and compares it with the unlock password set by the user on the mobile phone. If the password collected by the PC matches the unlock password set on the mobile phone, the authentication is successful, otherwise the authentication fails.
  • the PC can prompt the user that the password is wrong in the security authentication window.
  • a security authentication request will be sent to the mobile phone again for password verification.
  • it can be set to suspend authentication if the number of authentication failures exceeds the threshold of the number of incorrect passwords.
  • the threshold of the number of incorrect passwords can be set by the user or the default value.
  • the first security authentication module sends a message of successful authentication to the first communication module.
  • the first communication module sends an authentication success message to the second communication module based on the second communication connection.
  • the second communication module sends a message of successful authentication to the second security authentication module.
  • the second security authentication module sends a message of successful authentication to the message center module.
  • the message center module controls the display screen to display a transition loading window.
  • the transition loading window shown in Figure 7 that is, the transition loading interface 706 is displayed in the screen projection window 703. It is understandable that if the authentication is successful, the application screen projection steps of Figures 12 to 15 will be started. It takes a certain amount of time to successfully start the application screen projection, so there is a certain time span between displaying the security authentication window and displaying the application screen projection window.
  • the transition loading window can be started and displayed on the target device, which can prompt the user that the security authentication has passed and the application screen projection is being started.
  • the transition loading interface displayed in the screen projection window 703 switches to the application interface.
  • the target device after the target device receives the user's instruction to project the application screen on the corresponding source device, it first determines whether security authentication is required. For example, whether the source device is projecting the screen on the target device for the first time. Another example is whether the source device has set an unlock password.
  • the following is a method for screen projection provided by an embodiment of the present application.
  • the method adds a judgment mechanism when the target device receives a user operation on a message notification control.
  • Figure 18, is an example diagram of a process with added security authentication judgment.
  • the method may include but is not limited to the following steps:
  • S1 The source device and the target device establish a first communication connection
  • S2 The source device receives the application message
  • S6 The source device establishes a second communication connection with the target device
  • the target device sends a security authentication request to the source device
  • S8 The source device sends a message of successful authentication to the target device
  • the target device establishes a third communication connection with the source device
  • the target device sends a request to start application screen projection to the source device
  • S11 The source device starts the application screen projection
  • the source device sends display data of the application screen projection to the target device;
  • a judgment mechanism for whether security authentication is required is added between step S5 and step S6.
  • the target device does not need to display the unlock verification interface in the projection window, but instructs the source device to directly perform heterogeneous projection.
  • the target device can directly display the "prompt waiting information". After the source device completes drawing the application interface to be projected, the target device controls the above-mentioned projection window to display the application interface to be projected.
  • the source device is projecting its screen to the target device for the first time
  • the source device has an unlock password set, that is, the user is also required to perform password authentication before directly using the source device.
  • the target device determines that the source device is qualified for projection, it can trigger security authentication.
  • security authentication please refer to the above text and will not be repeated here.
  • the security authentication mechanism will be triggered when the target device instructs the source device to perform heterogeneous screen projection.
  • the first time the source device starts screen projection on the target device may refer to the first time the source device starts heterogeneous screen projection on the same target device, or it may refer to the first time the heterogeneous screen projection is started during the period from when the target device is turned on to when it is shut down.
  • the target device is turned on for the first time, and the target device is shut down for the first time after being used for a period of time
  • the first startup to the first shutdown is the first usage period.
  • the target device is turned on for the second time, and the target device is shut down for the second time after being used for a period of time, and the second startup to the second shutdown is the second usage period.
  • the first time the source device starts heterogeneous screen projection on the target device belongs to the "first" startup referred to in the second explanation; during the second usage period, the first time the source device starts heterogeneous screen projection on the target device also belongs to the "first" startup referred to in the second explanation.
  • the source device in a scenario where it is not the first time that the source device is projecting the screen to the target device, that is, the source device has previously initiated heterogeneous screen projection on the same target device, or the source device has initiated heterogeneous screen projection on the target device during the period from when the target device was turned on to when it was shut down.
  • the second explanation is that during the first use period, the source device starts heterogeneous screen projection on the target device for the first time, and when the source device starts heterogeneous screen projection on the target device for the second time, the target device will determine that the source device has started heterogeneous screen projection on the target device; the same applies to the third, fourth, and other times. During the second use period, the source device starts heterogeneous screen projection on the target device for the first time.
  • the target device will determine that the source device has started heterogeneous screen projection on the target device for the first time; when the source device starts heterogeneous screen projection on the target device for the second time, the target device will determine that the source device has started heterogeneous screen projection on the target device; the same applies to the third, fourth, and other times. If the source device has started heterogeneous screen projection on the target device, in order to reduce the number of times the user enters the password while ensuring the privacy of the user, the target device can optionally first determine whether the time since the last security authentication exceeds the preset time threshold.
  • the preset time threshold can be a default value or a value set by the user, which indicates that within this time range, the screen projection connection between the source device and the target device still meets the protection requirements.
  • the need for user privacy and security Therefore, when the target device determines that the time since the last security authentication does not exceed the preset time threshold, the screen projection is directly started without displaying the unlock verification interface. This can avoid repeatedly requiring users to enter passwords, which causes cumbersome operations. It protects user privacy and security while improving the convenience of users using heterogeneous screen projection.
  • security authentication needs to be re-performed. It can be understood that when the target device determines that the time since the last security authentication is passed exceeds the preset time threshold, it is considered that the screen projection connection between the source device and the target device at this time no longer meets the requirements for protecting user privacy and security.
  • the security authentication process in this case is the same as the security authentication process of "the source device starts heterogeneous screen projection on the target device for the first time" mentioned above, and will not be repeated here.
  • the target device determines that the user instructs the source device to project the screen, and determines that the source device has the conditions for projection, it can trigger the security authentication mechanism.
  • the target device After passing the security authentication, the first heterogeneous screen projection is successfully started; the first heterogeneous screen projection ends at time T1; assuming that the preset time threshold set by the user is 1 hour, at time T2, the target device detects that the user instructs the same source device to perform heterogeneous screen projection again; the target device determines that the source device has started heterogeneous screen projection on the target device, and then determines whether the time from T2 to T1 is greater than 1 hour; the target device determines that the value of (T2-T1) is less than 1 hour, and directly starts the second heterogeneous screen projection.
  • screen projection failure may also occur.
  • screen projection fails.
  • the application corresponding to the message notification control is protected by an application lock
  • screen projection fails.
  • the channel used for screen projection is occupied by other devices
  • screen projection fails.
  • the network is unstable, screen projection fails, etc.
  • application lock refers to the protection set by the user of the source device for a certain application in order to protect the security or privacy of the application, and a pre-set password or fingerprint is required to open the application. Therefore, in the scenario where the source device is in a locked screen or off screen state, out of consideration for the security and privacy of the source device, in some possible embodiments, before the actual screen projection, it will be determined whether application a (for example, the application corresponding to the message card selected by the user on the PC) is optionally protected by the source device. If the source device sets an application lock for the application a, it means that the owner of the source device hopes that the application a needs to enter a password or fingerprint to unlock it every time it is accessed. In this case, for such applications with strong privacy protection, it is not suitable to start the screen projection directly on the target device side when the source device is in a locked screen or off screen state, which goes against the original intention of the user to set the application lock.
  • application a for example, the application corresponding to the message card selected by the user
  • the embodiments of the present application provide three possible scenarios in which the screen projection fails to start.
  • the PC After the PC detects that the user has selected the message card 702, it sends a query message 1 to the corresponding device (that is, the mobile phone), and the query message 1 is used to query whether the mobile phone is configured with an application lock for the information application. If the mobile phone is configured with an application lock for the information application, the mobile phone will send feedback information 1 to the PC, and the feedback information 1 indicates that the information application is currently protected by the application lock. In this way, after receiving the feedback information 1, the PC can display a reminder message 1. The reminder message 1 is used to remind the user that the application corresponding to the selected message card is protected by an application lock.
  • the reminder information can be displayed on the PC in the form of a floating window or a card, and the embodiments of the present application do not limit the display form of the reminder information.
  • the source device and the target device can establish a screen projection connection through WLAN or WI-FI P2P.
  • the establishment of the screen projection connection can refer to the above method of establishing a communication connection between the source device and the target device, which will not be repeated here.
  • the following embodiment establishes a screen projection connection in WI-FI P2P mode.
  • the PC after the PC detects that the user has selected the message card 702, it determines whether the screen projection communication channel (Wi-Fi P2P channel) between the source device (i.e., mobile phone) and the target device (i.e., PC) corresponding to the message card 702 has been occupied by other services. Among them, the source device and the target device establish a screen projection connection through the Wi-Fi P2P channel to transmit data for the screen projection service, thereby realizing the mirroring of content between two or more electronic devices.
  • Wi-Fi P2P channel Wi-Fi P2P channel
  • the channel for the source device and the opposite device to perform the screen projection service i.e., the screen projection channel
  • the source device and the target device have established other services based on WI-FI P2P, such as PC collaboration, device sharing, etc.
  • the channel for the source device and the opposite device to perform the screen projection service i.e., the screen projection channel
  • the screen projection channel is occupied, resulting in the source device being unable to project the screen to the target device based on WI-FI P2P.
  • the PC determines that the screen projection channel (Wi-Fi P2P) between the PC and the mobile phone is occupied, and the screen projection fails to start, and a reminder message 2 is displayed.
  • the reminder message 2 exemplarily prompts the user that "screen projection connection conflicts, please disconnect the connection with other devices and try again", thereby prompting the user to go to the source device (mobile phone) or the target device (PC) to disconnect the screen projection connection with other devices.
  • the reminder information can be displayed on the PC in the form of a floating window or a card, and the embodiment of the present application does not limit the display form of the reminder information.
  • the PC determines whether the network status of the source device (that is, the mobile phone) corresponding to the message card meets the screen projection requirements.
  • the network status includes the flow table matching status, the data packet execution port load status, the total number of lost/forwarded/received data packets or the failure of the data packet TTR and other similar situations.
  • any network status that may occur in the network it is not limited here. It can be understood that the network status will affect the service quality of the screen projection, and the PC can optionally have a minimum network status threshold that meets the screen projection conditions. If the PC determines that the current network status is lower than this minimum network status threshold, it is considered that the current network status cannot meet the screen projection requirements, resulting in a screen projection failure.
  • the screen projection fails to start, and a reminder message 3 is displayed.
  • the reminder message 3 can remind the user to try to trigger the source device to cast the screen after the network is stable. The user is prompted by an example that "the network is unstable, please try again" to prompt the user to adjust and optimize the network status.
  • step S5 of Figure 18 it is possible to add a judgment whether there is a situation that may cause the screen projection to fail, which may include but is not limited to the above three situations.
  • the PC judges in turn at each judgment node whether the application lock is set, whether the screen projection channel is occupied, and whether the network status meets the screen projection requirements.
  • this round of judgment is terminated.
  • the user adjusts according to the prompts, he can click the message notification control again to trigger a new round of judgment.
  • the judgment order of the judgment mechanism can be adjusted, and after a judgment node triggers the failure to start the screen projection, the judgment process can stay at the judgment node and continue to detect whether the conditions for entering the next judgment node are met.
  • the user adjusts according to the prompts, and when the conditions for entering the next judgment node are met, the judgment process automatically enters the next judgment node.
  • the PC can first check whether the corresponding application is protected by the application lock (which can be called application lock detection), then check whether the communication channel used for screen projection is occupied (which can be called channel detection), and finally check whether the network conditions meet the requirements (referred to as network detection).
  • the order of the above-mentioned multiple tests is not limited. That is, in some embodiments, the PC can also first check whether the communication channel used for screen projection is occupied, then check whether the corresponding application is protected by the application lock, and finally detect the network conditions.
  • the PC can also perform application lock detection, channel detection, and network detection simultaneously, and display corresponding reminder information when the obtained detection results indicate that at least one detection item fails to trigger screen projection. For example, if the obtained test results indicate that the application is protected by the application lock and the communication channel is occupied, the PC will simultaneously display reminder information 1 and reminder information 2.
  • the PC determines that the user has selected the message card 702
  • the PC also needs to perform application lock detection, channel detection and network detection on the mobile phone. If the detection results indicate that none of the above detection items triggers the failure of screen projection, the PC determines that the mobile phone has the conditions for screen projection, so that the PC can display the application interface 704 in the screen projection window 703. In the case of screen projection failure, the PC can display the corresponding reminder information.
  • the electronic device includes hardware and/or software modules corresponding to the execution of each function.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of the present application.
  • the embodiment of the present application can divide the functional modules of the electronic device according to the above method example.
  • each function can be divided into various functional modules, such as a detection unit, a processing unit, a display unit, etc., or two or more functions can be integrated into one module.
  • the above integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.
  • the electronic device provided in this embodiment is used to execute the above-mentioned screen projection method, and thus can achieve the same effect as the above-mentioned implementation method.
  • the electronic device may further include a processing module, a storage module and a communication module.
  • the processing module may be used to control and manage the actions of the electronic device.
  • the storage module may be used to support the electronic device to execute stored program codes and data, etc.
  • the communication module may be used to support the electronic device to communicate with other devices.
  • the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application.
  • the processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc.
  • the storage module can be a memory.
  • the communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.
  • the electronic device involved in this embodiment may be a device having the structure shown in FIG. 9 .
  • the present application also provides a computer-readable storage medium, which stores A computer program, when executed by a processor, enables the processor to execute the screen projection method of any of the above embodiments.
  • An embodiment of the present application also provides a computer program product.
  • the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement the screen projection method in the above-mentioned embodiment.
  • the embodiments of the present application also provide a device, which may be a chip, a component or a module, and the device may include a connected processor and a memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory, so that the chip executes the caller ID method in the above-mentioned method embodiments.
  • the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding method provided above, so the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.
  • the disclosed devices and methods can be implemented in other ways.
  • the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
  • Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium.
  • the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请实施例提供了一种投屏系统和电子设备，该系统包括第一电子设备和第二电子设备，第一电子设备上的应用消息流转到第二电子设备，并在第二电子设备上显示对应的消息通知控件；第二电子设备在第一时间接收用户对消息通知控件的操作，向第一电子设备发送投屏请求，进而该应用消息对应的界面在第二时间被显示在第二电子设备的投屏窗口内，该界面包括该应用消息对应的图像信息；第二时间晚于第一时间，在第一时间和第二时间的时间区间，第一电子设备处于锁屏或者息屏状态。从而实现在第一电子设备处于锁屏或者息屏状态时，用户可以直接在第二电子设备上进行操作，从而拉起第一电子设备上的应用在第二电子设备上投屏。

Description

一种投屏系统和电子设备
本申请要求于2022年10月19日提交国家知识产权局、申请号为202211282319.9、发明名称为“一种投屏系统和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及通信技术领域和终端技术领域，尤其涉及一种投屏系统和电子设备。
背景技术
多台电子设备协同工作给用户的生活和工作带来便利。以协同通知为例，手机接收到一条消息，不仅可以由手机显示消息通知，还可以将该消息通知流转到PC，由PC进行显示，这样，方便用户跨设备查看消息，提高设备间协同工作的效率。当然，设备间仅仅进行消息通知的流转是不够的，用户在PC上查看了来自手机的消息通知之后，往往需要通过PC处理该消息通知，这就需要将手机投屏至PC。
然而，在手机处于锁屏或者息屏状态的场景中，用户在PC上启动手机投屏会出现异常。
发明内容
本申请提供了一种投屏系统和电子设备，在第一电子设备的消息流转到第二电子设备后，允许用户在第一电子设备处于锁屏或者息屏状态时，直接在第二电子设备上启动该消息对应的应用在第二电子设备上投屏。
第一方面，提供了一种投屏系统，该系统包括第一电子设备和第二电子设备，该第一电子设备包括第一应用；该第一电子设备用于接收第一应用的第一数据；该第一电子设备用于向该第二电子设备发送包括该第一数据的第一消息；该第二电子设备响应于接收到所述第一消息，显示第一控件；该第二电子设备用于在第一时间接收用户对该第一控件的第一操作；该第二电子设备响应于该第一操作，向该第一电子设备发送投屏请求；该第一电子设备响应于接收到该投屏请求，向该第二电子设备发送该第一应用的图像信息，该第一应用的图像信息包括该第一数据；该第二电子设备，用于根据接收到的该第一应用的图像信息显示窗口，该第一应用的第一界面在第二时间被显示在该窗口内，该第一界面包括该第一应用的图像信息；第二时间晚于第一时间，在第一时间和第二时间的时间区间，该第一电子设备处于锁屏或者息屏状态。
其中，在第一电子设备处于锁屏或者息屏状态时，用户无需在第一电子设备上进行任何操作，可以直接在第二电子设备上进行操作，即上述对该第一控件的第一操作，第一操作可以是点击第一控件的操作，从而拉起第一电子设备上的第一应用在第二电子设备上投屏。其中，第一应用为安装在第一电子设备上的应用程序，第二电子设备是否安装第一应用不影响上述系统的实现。
该第一电子设备向该第二电子设备发送该第一消息；响应于接收到该第一消息，该第二电子设备用于显示第一控件；在第一时间区间，该第一电子设备处于锁屏或者 息屏状态；该第二电子设备还用于在该第一时间区间接收用户对该第一控件的第一操作；响应于该第一操作，该第二电子设备还用于向该第一电子设备发送投屏请求；响应于接收到该投屏请求，该第一电子设备还用于向该第二电子设备发送该第一消息对应的该第一应用的数据；响应于接收到该第一消息对应的该第一应用的数据，该第二电子设备还用于显示投屏窗口，该投屏窗口在该第一时间区间显示第一界面，该第一界面包括该第一消息。
本申请实施例中，在第二时间区间，该第一电子设备还用于显示主界面；该第一电子设备还用于接收用户的第二操作；响应于接收到该第二操作，该第一电子设备在该第一时间区间处于锁屏或者息屏状态，该第一时间区间的开始时间为该第二时间区间的结束时间；该第一电子设备还用于接收用户的第三操作；响应于接收到该第三操作，该第一电子设备在第三时间区间显示该主界面，该第三时间区间的开始时间为该第一时间区间的结束时间；在该第三时间区间，该第二电子设备还用于在该投屏窗口显示该第一界面。
在上述实现方式中，首先第一电子设备在显示主界面时被锁屏或者息屏；随后，在第一电子设备处于锁屏或者息屏时，接收到来自第一应用的第一消息，并流转到第二电子设备。经过上述投屏操作后第一界面投屏到第二电子设备；随后，第一电子设备被亮屏或解锁，即上述第三操作，此时第一电子设备显示主界面，而与此同时第二电子设备的投屏窗口显示的是第一界面。
结合第一方面，在第一方面的某些实现方式中，该第一电子设备还包括第二应用，该第一应用和该第二应用不同；该第一电子设备还用于接收用户打开该第二应用的操作；响应于该用户打开该第二应用的操作，该第一电子设备还用于在第四时间区间显示该第二应用的第二界面，该第四时间区间的开始时间为该第三时间区间的结束时间；在该第四时间区间，该第二电子设备还用于在该投屏窗口显示该第一界面。
在上述实现方式中，在第二电子设备的投屏窗口显示该第一界面时，第一电子设备可以打开不同于第一应用的其他应用，比如第二应用，此时第二电子设备的投屏窗口仍显示该第一界面，不会随着第一电子设备上的显示界面变化而发生变化，从而实现异源投屏。
在另一些可能的方式中，在该第四时间区间，该第二电子设备还用于在该投屏窗口显示第三界面，该第三界面为该第一应用的界面，该第三界面与该第一界面不同。
其中，第三界面和第一界面均为第一应用的界面，也就是说，第一应用投屏到第二电子设备之后，可以在第二电子设备的投屏窗口对第一应用进行操作，切换不同的界面。
本申请实施例中，该第一电子设备还包括第三应用，该第三应用与该第一应用不同；该第一电子设备还用于接收该第三应用的第二消息；该第一电子设备还用于向该第二电子设备发送该第二消息；响应于接收到该第二消息，该第二电子设备还用于显示第二控件；该第二电子设备还用于在该第一时间区间接收用户对该第二控件的第四操作；响应于该第四操作，该第二电子设备还用于向该第一电子设备发送投屏请求；响应于接收到该投屏请求，该第一电子设备向该第二电子设备发送所述第二消息对应的所述第三应用的数据；响应于接收到所述第二消息对应的所述第三应用的数据，在 所述第一时间区间，所述第二电子设备还用于在该投屏窗口显示该第四界面，该第四界面包括该第二消息。
在上述实现方式中，第一应用投屏到第二电子设备之后，如果此时第一电子设备又接收到了另一个应用的消息，例如第三应用的第二消息，用户同样可以通过上述操作实现投屏的切换，将第二应用投屏至第二电子设备。
结合第一方面，在第一方面的某些实现方式中，该第二电子设备还用于在桌面显示该第一控件；或者，该第二电子设备还用于在消息列表窗口显示该第一控件，该第一控件为该第一消息对应的消息卡片。
其中，第一控件为第一消息对应的消息卡片，该消息卡片除了在桌面上单独显示之外，还存在另一种情形：当第一控件在桌面显示了一定时间后仍未接收到用户操作则消失，并在消息列表里生成与第一消息对应的消息卡片。如果用户在第一控件消失后想要对第一消息进行操作，还可以打开消息列表窗口，在消息列表窗口里找到与第一消息对应的消息卡片，同样可以实现用户操作第一控件的效果。
在一些可能的实现方式中，该第一操作为用户点击该第一控件的操作；或者，该第一操作为用户拖动该第一控件的操作。
在另一些可能的实现方式中，该第二电子设备还用于接收到该第一操作，在该投屏窗口显示安全认证界面；接收到用户在该安全认证界面输入认证信息的操作，该第二电子设备还用于向该第一电子设备发送该认证信息；接收到该认证信息，该第一电子设备还用于确定该认证信息满足预设值，向该第二电子设备发送认证成功的消息；响应于接收到该认证成功的消息，该第二电子设备向该第一电子设备发送投屏请求。
在上述实现方式中，当第一电子设备锁屏或者息屏时是没有经过用户的安全认证的，这种情况下如果允许第二电子设备不通过安全认证直接启动投屏，那么用户的隐私安全就得不到保障。所以，在一些实施例中，在第二电子设备启动投屏之前会触发安全认证机制以保证用户的隐私安全。
结合第一方面，在第一方面的某些实现方式中，该第二电子设备还用于接收到用户在该第一界面对该第一消息的操作，该第二电子设备向该第一电子设备发送指示该第一电子设备执行该操作的指令。
其中，第一应用投屏到第二电子设备之后，用户可以直接在第二电子设备上对第一应用进行回复、拨打音视频电话等等操作，第二电子设备会将这些操作指令都发送给第一电子设备，由第一电子设备执行，从而实现在第二电子设备上对被投屏应用进行操控。
第二方面，提供了一种投屏的方法，该方法应用于该第一电子设备中，该第一电子设备包括第一应用；该第一电子设备接收该第一应用的第一消息；该第一电子设备通过第一通信连接向第二电子设备发送该第一消息；在第一时间区间，该第一电子设备处于锁屏或者息屏状态；响应于接收到该第二电子设备发送的投屏请求，该第一电子设备通过第二通信连接向该第二电子设备发送该第一消息对应的该第一应用的数据，以使得在该第一时间区间，该第二电子设备在该投屏窗口显示该第一界面；其中，该第一界面包括该第一消息。
结合第二方面，在第二方面的某些实现方式中，在该第一时间区间，该第一电子 设备接收到该投屏请求之后，该方法包括该第一电子设备将该第一应用的一个或多个活动保持在OnResume状态；该第一电子设备控制该第一应用的一个或多个活动不进入休眠流程。
其中，第一电子设备将第一应用的一个或多个活动保持在OnResume状态，同时控制该第一应用的一个或多个活动不进入休眠流程，从而实现该一个或多个应用在投屏至第二电子设备后能过对用户可见且用户可操作。
在一些可能的实现方式中，该第一电子设备包括异步处理器、活动记录器、根窗口容器，在该第一时间区间，该第一电子设备接收到该投屏请求之后，该方法还包括该异步处理器将该第一应用的一个或多个活动标记第一标签；该活动记录器确定该第一应用的一个或多个活动带有该第一标签，将该第一应用的一个或多个活动保持在OnResume状态；该根窗口容器确定该第一应用的一个或多个活动带有该第一标签，控制该第一应用的一个或多个活动不进入休眠流程。
其中，第一电子设备通过为该第一应用对应的活动标记上特定的标签，从而能够将第一应用对应的活动从其他应用对应的活动中过滤出来，实现只对需要投屏的第一应用的活动进行上述投屏操作。
结合第二方面，在第二方面的某些实现方式中，在该第一时间区间，该第一电子设备包括虚拟显示区域和可见显示区域，该第一电子设备向该第二电子设备发送该第一界面之前，该第一电子设备将该虚拟显示区域标记第二标签；该第一电子设备在该虚拟显示区域绘制第一图像，该第一图像为该第一消息对应的该第一应用的界面；该第一电子设备在该可见显示区域绘制第二图像，该第二图像为该第一电子设备处于该锁屏或者息屏状态时对应的界面；该第一电子设备确定该虚拟显示区域带有该第二标签，获取该第一图像得到第一界面，该第一界面不包括该第二图像。
其中，第一电子设备通过将虚拟显示区域标价上特定标签，从而指示该虚拟显示区域内所绘制的图像不与其他显示区域的图像进行合成，实现投屏到第二电子设备上的界面只与该虚拟显示区域的图像相关，不会跟随其他显示区域图像的变化而变化，也不会将锁屏图层等覆盖投屏界面上。
在一些可能的实现方式中，该第一电子设备包括显示屏，该方法还包括在第二时间区间，该第一电子设备显示主界面；该第一电子设备接收用户的第二操作；响应于接收到该第二操作，该第一电子设备在第一时间处于锁屏或者息屏状态，该第一时间为该第二时间区间的结束时间，该第一时间区间的开始时间；该第一电子设备接收用户的第三操作；响应于接收到该第三操作，该第一电子设备在该可见显示区域绘制主界面，在第三时间区间显示该主界面，该第三时间区间的开始时间为该第一时间区间的结束时间；在该第三时间区间，该第一电子设备确定该虚拟显示区域带有该第二标签，不获取该可见显示区域的主界面发送给该第二电子设备，以使得该第二电子设备在该投屏窗口显示该第一界面。
结合第二方面，在第二方面的某些实现方式中，该第一电子设备在接收到该启动投屏请求之前，该方法还包括该第一电子设备通过第三通信连接接收到该第二电子设备发送的认证信息；该第一电子设备确定该认证信息满足预设值，通过第三通信连接向该第二电子设备发送认证成功的消息，以使得该第二电子设备向该第一电子设备发 送投屏请求。
在一些可能的实现方式中，该方法包括该第一电子设备接收到该第二电子设备指令，执行用户在该第一界面对该第一消息的操作。
第三方面，提供了一种方法，该方法应用于第一电子设备和第二电子设备，在该第二电子设备与该第一电子设备建立该第一通信连接之前，该第一电子设备与该第二电子设备建立第二通信连接；该第一电子设备接收到该第一消息；该第一设备根据该第一消息，通过该第二通信连接向该第二电子设备发送该第二消息；响应于接收到该第二消息，该第二电子设备根据该第二消息，显示该第一控件；在该第一电子设备处于锁屏或者息屏状态时，该第二电子设备接收到该第一操作。
本申请实施例中，该第二电子设备接收到该第一操作之后，该第二电子设备确定该第一应用首次在该第二电子设备上投屏，该第二电子设备与该第一电子设备建立第三通信连接，该第三通信连接用于安全认证；该投屏窗口显示该第一电子设备的该安全认证界面。
结合第三方面，在第三方面的某些实施例中，该第二电子设备接收到该第一操作之后，该第二电子设备确定该第一应用非首次在该第二电子设备上投屏，且当前时间距离上一次认证成功的时间超过预设时长，该第二电子设备与该第一电子设备建立第三通信连接；该投屏窗口显示该第一电子设备的该安全认证界面。
在一些可能实施方式中，在该第二电子设备接收到用户在该安全认证界面输入认证信息的操作之后，该第二电子设备通过该第三通信连接向该第一电子设备发送该认证信息；该第一电子设备确定该认证信息满足预设值，通过该第三通信连接向该第二电子设备发送认证成功的消息。
在另一些可能的实施例中，该第二电子设备接收到用户对该第一控件的第一操作之后，该第二电子设备确定该第一应用设置有应用锁，该第二电子设备不显示该第一电子设备的投屏窗口。
结合第三方面，在第三方面的某些实施例中，该第一电子设备包括投屏助手模块、第一虚拟化服务模块、第一投屏服务模块，该第一电子设备接收到该投屏请求之后，该投屏助手模块绑定投屏服务；该投屏助手模块向该第一虚拟化服务模块发送进行初始化的第一指示信息；第一虚拟化服务模块根据该第一指示信息完成初始化；该第一虚拟化服务模块向该第一投屏服务模块发送启动投屏服务的第二指示信息；该第一投屏服务模块根据该第二指示信息启动投屏服务；该第一投屏服务模块向该投屏助手模块发送投屏服务已启动的消息。
在一些可能的实施方式中，该第一电子设备创建虚拟显示区域，为该虚拟显示区域标记该第一标签，该第一投屏服务模块启动投屏服务后，创建该虚拟显示区域，并为该虚拟显示区域标记该第一标签。
在另一些可能的实施方式中，该第一电子设备还包括多屏管理模块、系统界面模块，第二电子设备包括第二投屏服务模块、消息中心模块，该第一投屏服务模块向该投屏助手模块发送投屏服务已启动的消息之后，该投屏助手模块通过第一通信连接向该消息中心模块发送投屏服务已启动的消息；该消息中心模块通过第一通信连接向该系统界面模块发送启动应用投屏的请求；该系统界面模块根据该启动应用投屏的请求， 向该多屏管理模块发送启动第一应用投屏的第三指示信息；接收到该第三指示信息，该多屏管理模块将该第一应用的所有活动置于前台可见状态，并将该第一应用的所有活动跳过休眠状态；该多屏管理模块获取该第一应用对应的该第一显示数据；该投屏管理模块通过第一通信连接向该第二投屏服务模块发送该第一显示数据。
结合第三方面，在第三方面的某些实施例中，该多屏管理模块将该第一应用的所有活动置于前台可见状态，并将该第一应用的所有活动跳过休眠状态，该多屏管理模块将该第一应用的所有活动标记第二标签；该多屏管理模块将标记了该第二标签的活动置于前台可见状态，并将该标记了该第二标签的活动跳过休眠状态。
结合第三方面，在第三方面的某些实施例中，该多屏管理模块获取该第一应用对应的该第一显示数据之前，该方法包括：该多屏管理模块向该第一应用发送第四指示信息；该第一应用根据该第四指示信息启动；该第一应用向该多屏管理模块发送第一应用已启动的消息；接收到该第一应用已启动的消息，该多屏管理模块在该虚拟显示区域绘制该第一显示数据。
本申请实施例中，该多屏管理模块通过第一通信连接向该第二投屏服务模块发送该第一显示数据之后，该方法包括：该第二投屏服务模块接收到该第一显示数据，根据所述第一显示数据，渲染所述第一界面；该第二投屏服务模块向该消息中心模块发送该第一界面；接收到该第一界面，该消息中心控制该第二电子设备的显示屏在该投屏窗口显示该第一界面。
在一些可能的实施例中，该第一电子设备还包括第一通信模块，该第二电子设备还包括第二通信模块，该第二电子设备与该第一电子设备建立第一通信连接，具体包括：该消息中心模块向该第二通信模块发送第五指示信息；该第二通信模块根据该第五指示信息与该第一通信模块建立该第一通信连接；该第二通信模块向该消息中心发送第一通信连接建立成功的消息。
在另一些可能的实施例中，该投屏助手模块绑定投屏服务之前，该方法包括：该第一通信模块向该投屏助手模块发送第六指示信息；该投屏助手模块根据该第六指示信息启动。
结合第三方面，在第三方面的某些实施例中，该第二电子设备通过该第一通信连接向该第一电子设备发送投屏请求之前，具体包括：该消息中心通过该第一通信连接获取会话密钥和该第一电子设备的IP地址；该消息中心向该第二投屏服务模块发送第七指示信息，第七指示信息携带该会话密钥和该第一电子设备的IP地址；该第二投屏服务模块根据该第七指示信息完成初始化。
本申请实施例中，该第一电子设备接收到该投屏请求之前，具体包括：该第二投屏服务模块通过该第一通信连接向该第二通信模块发送该投屏请求；该第二通信模块通过该第一通信连接，向该第一通信模块发送该投屏请求；该第一通信模块向该投屏助手模块发送该投屏请求。
结合第三方面，在第三方面的某些实施例中，该第一电子设备接收到该第一消息之后，具体包括：该第一应用根据该第一消息，向该系统界面模块发送该第二消息；该系统界面模块通过该第二通信连接向该消息中心模块发送该第二消息；接收到该第二消息，该消息中心模块根据该第二消息，控制该第二电子设备的显示屏显示该第一 控件；接收到用户对该第一控件的该第一操作，该消息中心模块向该第二通信模块发送第五指示信息。
在一些可能的实现方式中，该消息中心模块接收到该第一操作之后，该方法还包括：该消息中心模块控制该第二电子设备的显示屏在该投屏窗口显示该过渡加载界面。
在另一些可能的实现方式中，该第二电子设备还包括第二安全认证模块，该消息中心模块接收到该第一操作之后，该方法还包括：该消息中心向该第二安全认证模块发送第八指示信息；该第二安全认证模块根据该第八指示信息，确定该第一应用首次在该第二电子设备上，该第二安全认证模块向该第二通信模块发送第九指示信息；该第二通信模块根据该第九指示信息与该第一通信模块建立该第三通信连接。
结合第三方面，在第三方面的某些实施例中，该消息中心模块接收到该第一操作之后，该方法还包括：该消息中心向该第二安全认证模块发送第八指示信息；该第二安全认证模块根据该第八指示信息，确定该第一应用非首次在该第二电子设备上投屏，且当前时间距离上次认证成功的时间超过该预设值，该第二安全认证模块向该第二通信模块发送该第九指示信息；该第二通信模块根据该第九指示信息与该第一通信模块建立该第三通信连接。
结合第三方面，在第三方面的某些实施例中，该第三通信连接建立之后，具体包括：该第二通信模块向该消息中心模块发送第三通信连接建立成功的消息；该消息中心向该第二安全认证模块发送第十指示信息；该第二安全认证模块根据第十指示信息，控制该第二电子设备的显示屏在该投屏窗口显示该安全认证界面。
本申请实施例中，该第一电子设备还包括第一安全认证模块，该第二安全认证模块接收到用户在该安全认证界面输入认证信息的操作之后，具体包括：该第二安全认证模块通过该第三通信连接向该第一安全认证模块发送该认证信息；第一安全认证模块确定该认证信息满足预设值，通过该第三通信连接向该第二安全认证模块发送认证成功的消息；第二安全认证模块向该消息中心模块发送该认证成功的消息；接收到该认证成功的消息，该消息中心模块控制该第二电子设备的显示屏在该投屏窗口显示该过渡加载界面。
结合第三方面，在第三方面的某些实施例中，该多屏管理模块包括异步处理器、活动启动器、活动栈管家、锁屏控制器、活动记录器、任务栈、根窗口容器，该多屏管理模块将该第一应用的所有活动置于前台可见状态，并将该第一应用的所有活动跳过休眠状态，具体包括：该异步处理器将该第一应用的部分或全部活动标记该第二标签；该异步处理器向该活动启动器发送启动带有该第二标签的活动的请求；响应于接收到该第一命令，该活动启动器向该活动栈管家发送启动应用投屏的请求；响应于接收该启动应用投屏的请求，该活动栈管家调用该锁屏控制器执行第一动作，该第一动作包括将该标记了该第二标签的活动置于前台可见状态；该锁屏控制器将该标记了该第二标签的活动置于前台可见状态。该锁屏控制器向该活动记录器发送第三命令；接收到该第三命令，该活动记录器控制该标记了该第二标签的活动保持前台可见状态；该活动记录器向该任务栈发送第四命令，该第四命令用于指示该任务栈校验位于该任务栈栈顶的活动的状态；接收到该第四命令，该任务栈确定该位于该任务栈栈顶的活动处于前台可见状态，向该根窗口容器发送第五命令；接收到该第五命令，该根窗口 容器将该标记了该第二标签的活动跳过休眠状态。
结合第三方面，在第三方面的某些实施例中，该第一消息对应该第一应用的第一活动，该第一活动被该异步处理器标记该第二标签，该第一活动位于该任务栈栈顶。
在一些可能的实现方式中，该多屏管理模块还包括活动栈管理服务器、图层合成器，该多屏管理模块获取该第一应用对应的该第一显示数据之前，具体包括：接收到该第一命令，该活动启动器向该第一应用发送第四指示信息；该第一应用根据该第四指示信息启动；该第一应用向该活动栈管理服务器发送第一应用已启动的消息；该活动栈管理服务器向该图层合成器发送第六命令；该图层合成器在该虚拟显示区域绘制该第一应用对应的应用图层；该图层合成器根据该应用图层获取该第一显示数据。
第四方面，提供了一种电子设备，该电子设备包括一个或多个处理器；一个或多个存储器；该一个或多个存储器存储有一个或者多个计算机程序，该一个或者多个计算机程序包括指令，当该指令被该一个或多个处理器执行时，使得该电子设备执行上述第二方面中任一项可能实现的方法。
第五方面，提供了一种计算机可读存储介质，包括计算机指令，当该计算机指令在电子设备上运行时，使得该电子设备执行上述第二方面所述的方法。
第六方面，提供了一种计算机程序产品，该计算机程序产品包括计算机程序代码，当该计算机程序代码在电子设备上运行时，使得该电子设备执行上述第二方面所述的方法。
第七方面，提供了一种芯片用于执行指令，当所述芯片运行时，所述芯片执行上述第二方面所述的方法。
附图说明
图1是本申请实施例提供的一种消息流转的场景示意图。
图2是本申请实施例提供的一种异源投屏的界面显示示意图。
图3是本申请实施例提供的一种异源投屏的界面显示示意图。
图4是本申请实施例提供的一种异源投屏的原理示意图。
图5是本申请实施例提供的一种息屏状态下投屏的场景示意图。
图6是本申请实施例提供的一种锁屏状态下投屏的场景示意图。
图7是本申请实施例提供的又一种息屏状态下投屏的场景示意图。
图8是本申请实施例提供的一种投屏的操作示意图。
图9是本申请实施例提供的一种电子设备的硬件结构示意图。
图10是本申请实施例提供的一种电子设备的软件架构图。
图11是本申请实施例提供的一种投屏方法的具体实现过程图。
图12是本申请实施例提供的又一种投屏方法的具体实现过程图。
图13是本申请实施例提供的又一种投屏方法的具体实现过程图。
图14是本申请实施例提供的又一种投屏方法的具体实现过程图。
图15是本申请实施例提供的又一种投屏方法的具体实现过程图。
图16是本申请实施例提供的又一种投屏方法的具体实现过程图。
图17A是本申请实施例提供的一种原理示意图。
图17B是本申请实施例提供的又一种原理示意图。
图18是本申请实施例提供的一种投屏方法的流程图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。其中，在本申请实施例的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；文本中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况，另外，在本申请实施例的描述中，“多个”是指两个或多于两个。
应当理解，本申请的说明书和权利要求书及附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
在本申请中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本申请所描述的实施例可以与其它实施例相结合。
示例性地，本申请实施例提供了一种系统，包括多台电子设备，如设备1、设备2及设备3。上述系统中的电子设备(也即，设备1、设备2、设备3)可以属于同一局域网络。
该局域网络可以是无线通信网络，如，无线局域网(wireless local areanetworks，WLAN)、无线保真点对点(wireless fidelity point to point，Wi-Fi P2P)、蓝牙网络、zigbee网络、红外技术(infrared，IR)或近场通信(near field communication，NFC)网络等。又示例性地，上述局域网络也可以是有线通信网络。例如，视频图像配接器(video graphics array，VGA)、数字视频接口(digital visual interface，DVI)、高清多媒体接口(high definition multimedia interface，HDMI)或数据传输线等建立的通信网络。再示例性地，上述局域网络还可以是远程网络。
在设备1、设备2和设备3彼此经过信任互认之后，由设备1、设备2和设备3组成的系统又可称为信任环。另外，设备间进行信任互认的过程可参考相关技术，例如，可以根据设备间是否登录有相同的系统账号、是否已进行授权以及是否采用近场通信方式等一项或多项进行判断，具体在此不再赘述。
可以理解地，在同一信任环中，多台电子设备之间可以安全地进行数据共享，共享的数据包括但不限于消息提醒、消息记录以及其他电子设备的用户信息、设备类型、各个应用的运行状态、各个应用的标识、最近使用应用的标识的同步等等。
这样，在信任环内，可以实现多样的协同服务，如，协同通话、协同通知、投屏等服务。
以协同通知为例，在信任环中，设备1接收到消息通知，不仅可以由设备1显示该消息通知，还可以将该消息通知流转到设备2和/或设备3，由设备2和/或设备3进行显示，这样，方便用户跨设备查看消息，提高设备间协同工作的效率。同样，设备2也可以将接收到的消息通知流转到设备1和/或设备3，设备3也可以将接收到的消 息通知流转到设备1和/或设备2。
当然，设备间仅仅进行消息的流转是不够的，如果用户在设备2上查看了来自设备1的消息通知之后，往往需要通过设备2处理该消息通知，这就需要将设备1投屏至设备2。
一般来说，投屏场景中包括源端设备和目标设备，源端设备与目标设备之间通过某种协议，实现源端设备上的内容在目标设备上显示，即源端设备向目标设备投屏。常见的协议如DLNA(Digital Living Network Alliance，数字生活网络联盟)协议和Chromecast协议、Miracast协议、Airplay协议等。其中，源端设备可以是手机、平板电脑、PC、手表等。目标设备可以是平板电脑、PC、电视机等。
可以理解的是，源端设备和目标设备还可以有其它称呼，比如，源端设备可以称为主动投屏设备，目标设备可以称为被动投屏设备；或者，源端设备可以称为第一电子设备，目标设备可以称为第二电子设备等等，本申请对此不作限定。
如图1所示，以源端设备为手机、目标设备为PC为例。在手机和PC属于同一局域网络且完成信任互认之后，手机与PC属于同一信任环，也就是，手机和PC之间可以建立用于流转消息的通信通道。
在一些实施例中，手机中可以安装有多类应用程序，手机运行期间，可以接收各应用程序推送的应用消息，该应用消息来自于应用程序对应的消息服务器。
示例性地，手机中可以安装有社交应用，手机可以接收社交应用推送的应用消息，如，社交应用中的联系人小明所发送的消息“你今天去跑步吗？”。
在接收到应用程序推送的应用消息之后，如果手机处于解锁且亮屏的状态，手机可以在当前界面显示提示信息。比如，在接收到社交应用推送的应用消息时，如果手机的当前界面为主界面101，那么在主界面101中显示通知气泡102，该通知气泡102用于显示手机所接收到的应用消息。
然后，手机可以通过与PC之间用于流转消息的通信通道，将上述应用消息(比如，社交应用推送的“你今天去跑步吗？”)流转到PC。
在一些实施例中，在PC接收到来自手机的应用消息之后，可以显示对上述应用消息的提示信息。其中，上述提示信息可以采用消息通知控件的形式显示于PC的桌面，示例性地，上述消息通知控件可以是悬浮窗口、消息气泡、消息卡片等，当然，本申请实施例中，对于其显示形态不作具体限定。比如，图1所示，在桌面103中显示消息卡片104，该消息卡片104用于显示上述应用消息的相关内容。其中，相关内容可以是该应用消息来自手机，也可以是该应用消息来自于社交应用，还可以是该应用消息的主要内容等。可以理解地，本申请实施例中对于消息卡片104的显示位置不做具体限定。
在图1所示的场景下，如果用户正在使用PC，那么桌面103中显示的消息卡片104可以提醒用户关注到手机当前接收到了应用消息。这样，即使用户当前未直接使用手机，或者手机不在身边，也能够了解该应用消息的简要内容，避免消息的遗漏。
在一些实施例中，上述消息卡片104可以具有显示时限。比如，预先设置显示时限为时长1，PC响应于手机转发的上述应用消息，开始显示消息卡片104。在消息卡片104显示的时长超过时长1，且未检测到用户选中该消息卡片104，PC可以取消显 示该消息卡片104。
示例性地，在PC的显示屏是触控屏时，PC检测到用户与触控屏中的显示区域1存在接触，PC确定用户指示选中消息卡片104。其中，上述显示区域1是触控屏上显示消息卡片104的区域。又示例性地，在PC已接入鼠标时，检测到鼠标对应的光标位于显示区域1内，并输入确认指令，PC确定用户指示选中消息卡片104。
在一些实施例中，在PC显示消息卡片104之后，用户可以通过操作PC的方式处理该应用消息，也可以选择直接操作手机的方式处理该应用消息。
其中，直接操作手机处理应用消息的方式可以参考相关技术。例如，手机可以响应于用户点击通知气泡102的操作，前台运行社交应用，并显示包含上述社交消息的应用界面。这样，用户可以通过在上述应用界面上的操作，回复该应用消息。再例如，手机响应用户指示启动社交应用的操作，显示社交应用的应用界面，这样，用户可以通过在社交应用提供的应用界面上操作，查看上述应用消息或回复该应用消息等。
另外，选择操作PC处理应用消息，就会触发手机与PC之间的投屏。示例性地，用户可以在PC上选中消息卡片104。在PC可以检测到用户选中消息卡片104之后，PC可以遥控手机运行社交应用。在社交应用进入运行状态之后，PC可以在桌面103上显示投屏窗口。在PC显示投屏窗口之后，用户可以通过PC中的投屏窗口，操控手机，如，操作手机处理上述应用消息。
其中，上述投屏窗口，可以用于同步显示手机的当前界面。比如，在手机的当前界面是社交应用的应用界面时，该投屏窗口也显示该应用界面，此为同源投屏。也可以不同步显示手机的当前界面，比如，手机显示主界面，该投屏窗口显示应用界面，此为异源投屏。
请参阅图2，图2介绍异源投屏在上述场景中的界面显示。
作为一种实现方式，在PC确定用户选中消息卡片104之后，通过PC和手机之间的数据交互可实现异源投屏。如图2所示，在PC确定用户选中消息卡片104之后启动异源投屏，手机的显示屏上仍显示主界面101，PC的显示屏上的投屏窗口201显示上述应用消息对应的应用界面202。
此后，投屏窗口所显示的内容不会随着手机显示屏中所显示的内容同步变化。比如，请参阅图3，手机的显示屏上切换为显示信息应用界面303，PC的显示屏上的投屏窗口201仍显示上述应用消息对应的应用界面201。
请参阅图4，图4为上述图2和图3中异源投屏的原理示意图。异源投屏场景下，源端设备向目标设备投屏时，目标设备上的显示信息与源端设备前台的显示信息是不同步的。源端设备上显示应用B的窗口，源端设备向目标设备投屏应用B的窗口时，在创建的虚拟显示区域(VirtualDisplay)中显示应用B的窗口。其中，虚拟显示区域的内容对用户不可见，可用于投屏到目标设备上显示。除了虚拟显示区域，源端设备上还包括用户可见显示区域，源端设备将应用B的窗口投屏到目标设备时，用户可见显示区域显示应用A的窗口，即源端设备前台显示应用A的窗口。假设源端设备上用户可见区域从应用A的窗口切换为应用C的窗口，即源端设备前台显示应用C的窗口，此时，目标设备上仍然显示应用B的窗口。
可以理解，与同源投屏相比，异源投屏方式具有更好的隔离性。例如，异源投屏 方式可以为用户提供独立的操控屏(即源端设备的显示屏和目标设备的显示屏)处理不同的应用。这样一来，用户在将应用B的窗口投屏到目标设备上显示之后，源端设备对应用A进行操作，或者由应用A的窗口切换为应用C的窗口，用户仍然可以在目标设备上针对应用B进行操作。所以异源投屏释放了源端设备的显示信息对目标设备的显示信息的控制，从而增强了多个设备的可复用性，允许用户通过投屏在不同设备上对不同应用进行操作。而同源投屏方式中，投屏窗口所显示的内容随着手机显示屏中所显示的内容同步变化。本申请实施例中，以异源投屏场景为例进行描述。
以上述图1和图2为例，描述了手机在亮屏、解锁的状态下，手机接收到消息通知之后，将其流转至PC，然后，用户操作PC上选中显示该消息通知的消息卡片，触发手机向PC进行异源投屏的过程。
在异源投屏场景中，如图5所示，在相关技术中，手机在息屏状态下，手机显示屏处于黑屏状态。此时，如果手机接收到来自社交应用的应用消息，手机可以短暂地亮屏并显示通知气泡，也可以不显示通知气泡，继续保持黑屏状态。可以理解地，上述黑屏状态是指手机的显示屏未点亮或者仅显示息屏屏保图像的状态。
如图6所示，手机在锁屏状态下，手机显示屏可以显示锁屏界面或者也处于黑屏状态。此时，如果手机接收到来自社交应用的应用消息，手机可以在锁屏界面上显示通知气泡。
在图5和图6所示的场景下，手机同样可以将该应用消息流转至PC。这样，PC依然可以在桌面103中显示消息卡片104。
当然，在图5和图6所示的场景下，PC显示消息卡片104期间，即使PC确定用户指示选中消息卡片104，手机继续保持黑屏状态或显示锁屏界面。可以理解地，在保持黑屏状态或显示锁屏界面期间，手机无法前台运行社交应用，也就是，也无法显示社交应用的应用界面。
需要说明的是，本文所述息屏特指锁屏且灭屏的情况，此时源端设备显示黑屏界面，显示屏未点亮，如图5所示；本文所述锁屏特指锁屏但亮屏的情况，此时源端设备显示锁屏界面，锁屏界面可以包括锁头标识，如图6所示。
对应地，如图5所示，即使PC显示手机对应的投屏窗口，该投屏窗口也同样显示黑屏界面。
对应地，如图6所示，即使PC显示手机对应的投屏窗口，该投屏窗口也同样显示锁屏界面。
可见，在图5和图6所示的场景中，无论投屏窗口显示黑屏界面，还是显示锁屏界面，用户无法通过PC遥控手机，并处理上述应用消息。也即，用户只能选择解锁手机，直接操作手机，完成针对应用消息的处理。
对应图5，用户如果想在PC上显示应用界面的投屏窗口，用户可以先将手机点亮屏幕并解锁，再在PC上指示选中消息卡片104从而启动异源投屏。可以理解的是，手机经过解锁后的投屏场景可参照图2，此处不予赘述。
对应图6，用户如果想在PC上显示应用界面的投屏窗口，用户可以先在手机上进行解锁并保持亮屏状态，再在PC上指示选中消息卡片104从而启动异源投屏。可以理解的是，手机经过解锁后的投屏场景可参照图4，此处不予赘述。
也就是，在手机处于息屏或者锁屏状态时，就存在这样一种场景：当用户在家中书房或客厅使用PC玩游戏或者办公，手机被放置在其他房间；在社交应用上的联系人(小明)给用户的手机发送一条消息时，用户可以立即在PC上得知自己的手机收到了一条消息通知。但这种场景下，用户往往专注于在PC上游戏或者办公，并不想起身去房间拿手机查看或回复消息，而想在PC上启动展示该条消息的应用窗口投屏(也即，手机对应的投屏窗口)，并通过PC处理该消息。但由于此时在其他房间的手机处于锁屏息屏状态，基于现有技术用户无法直接在PC上启动展示该条消息的应用窗口投屏，而必须先去房间将手机解锁并使其处于亮屏状态之后，才能够在PC上启动手机对应的投屏窗口。
可见，当手机处于锁屏或息屏状态时，用户在PC上启动投屏十分不便且耗费时间。当然，在相关技术中，手机同时处于息屏锁屏状态时，同样存在类似的问题，在此不再赘述。
由此可见，通常情况下，当源端设备处于锁屏或者息屏状态时，用户无法直接在目标设备上启动展示该消息的应用投屏窗口，而需要先将源端设备解锁并使其处于亮屏状态，才能在目标设备上启动展示该消息的应用投屏窗口，否则只能在目标设备上显示黑屏窗口或者锁屏界面窗口。这增加了用户的操作流程，并且当源端设备不在用户身边时则进一步降低了投屏效率，给用户的投屏操作带来了繁琐低效的体验。
需要说明的是，应用投屏窗口指的是源端设备上的应用投屏到目标设备上显示的投屏窗口。
为了解决上述问题，实现在源端设备锁屏或者息屏状态下，用户能够在目标设备启动应用投屏窗口，本申请实施例提供了一种投屏的方法、电子设备和系统，给用户提供便捷高效的投屏体验。
具体来说，在上述源端设备处于锁屏或者息屏的场景下，根据本实施例提供的一种投屏的方法，用户可以直接在目标设备上进行操作，实现该流转消息所对应的应用窗口在目标设备上投屏。基于该应用窗口在目标设备上的投屏，通过操控和输入协同来实现跨设备的多屏协同交互的功能。这样一来，用户可以专注于对目标设备的操作，无需对处于锁屏或者息屏状态下的源端设备进行操作，减少了启动投屏的用户操作流程，提高了启动投屏的人机交互效率，增强了用户体验。
示例性的，请参阅图7，图7为本申请实施例提供的一种投屏方法的示意图。本申请实施例可以实现手机投屏至PC、手机投屏至平板电脑、平板电脑投屏至PC等过程。下面以源端设备为手机，目标设备为PC为例进行说明。
在手机和PC属于同一局域网络且完成信任互认之后，手机与PC属于同一信任环，也就是，手机和PC之间可以建立用于流转消息的通信通道。
示例性地，手机中可以安装有社交应用，手机可以接收社交应用推送的应用消息，如，社交应用中的联系人小明所发送的消息“你今天去跑步吗？”。
在接收到应用程序推送的应用消息之后，如果手机处于息屏的状态，可以短暂地亮屏并显示消息通知气泡，也可以不显示消息通知气泡，继续保持屏幕未点亮状态。如图7所示，手机显示黑屏界面。如果手机处于锁屏的状态，可以短暂显示消息通知 气泡，也可以不显示消息通知气泡，继续保持锁屏界面。需要说明的是，下述实施例以手机处于息屏状态为例进行说明，当手机处于锁屏状态时的情况可予以参考，不再赘述。
然后，手机可以通过与PC之间用于流转消息的通信通道，将上述应用消息(比如，社交应用推送的“你今天去跑步吗？”)流转到PC。
可以理解的是，PC上可以安装该应用程序(该应用程序对应的电脑版本，本文将应用程序对应的手机版和电脑版统称为应用程序，不在名称上加以区分)，也可以未安装该应用程序。也就是说，PC上是否安装有对应的应用程序不影响消息的流转及后续的异源投屏。
示例性的，消息卡片702显示了应用消息的概要信息“你今天还跑步吗？”、设备信息“来自手机”、应用信息“社交应用”、联系人信息“小明”，让用户能够通过消息卡片方便快捷地获知应用消息的内容，且应用消息为手机上的社交应用收到的一条由小明发过来的消息，从而便于用户判断是否要启动投屏对应用消息进行回复等操作。
在一些实施例中，在PC显示消息卡片702之后，用户可以通过操作PC的方式处理该应用消息。选择操作PC处理应用消息，就会触发手机与PC之间的投屏。
在另一些实施例中，目标设备可以接收同一信任环中任意源端设备流转过来的一条或多条应用消息。示例性地，PC可以响应用户操作，显示消息列表窗口，该消息列表窗口可以集中显示已接收到的消息通知，或者集中显示已接收且未处理过的消息通知。作为一种实现方式，上述消息通知也可以采用消息卡片的形式显示于消息列表窗口。
以目标设备为包含触控屏的PC，源端设备为手机1和手机2为例。在PC检测到指定操作，请参阅图8，图8为本申请实施例提供的另一种启动投屏的操作示意图。PC检测到用户在触控屏上的下滑操作，可以显示消息列表窗口801。该消息列表窗口801中显示有来自手机1的应用消息(如消息卡片802)、来自手机2的应用消息(如消息卡片803)等。可以理解，消息列表窗口801中所显示的消息通知均为PC近期接收到的、来自信任环中其他设备的信息。其中，消息卡片802与手机1中社交应用推送的消息对应。消息卡片803与手机2中信息应用推送的消息(比如，短信)对应。
与显示于桌面的消息卡片702相同，在PC检测到用户选中消息列表窗口801中任意一条消息卡片时，均可触发该消息卡片对应的设备进行投屏。例如，PC检测到用户选中消息卡片802时，可以触发手机1进行投屏，这样，手机1为源端设备，PC为对应的目标设备。再例如，PC检测到用户选中消息卡片803时，可以触发手机2进行投屏，这样，手机2为源端设备，PC为对应的目标设备。
示例性地，用户可以在PC上选中消息卡片702。在PC可以检测到用户选中消息卡片702之后，即使手机在息屏状态下，PC也可以遥控手机后台运行社交应用。在社交应用进入后台运行状态之后，手机仍然显示黑屏界面。另外，PC也可以在桌面701上显示社交应用对应的投屏窗口703。其中，上述社交应用对应的投屏窗口703，用于显示应用消息所对应的应用界面704。其中，应用界面指的是展示上述应用消息的界面。
可理解，显示应用界面的投屏窗口可称为应用投屏窗口。可选地，应用界面704 包括应用消息的内容、操作控件、回复框等。示例性的，应用界面704为该条应用消息对应的聊天框界面，用户可以在该界面查看该条应用消息并进行回复。
作为一种实现方式，在PC确定用户选中消息卡片702之后，通过PC和手机之间的数据交互，手机可以在虚拟显示区域上渲染对应的应用界面。在虚拟显示区域上渲染的应用界面在手机前台不显示。也就是说，如图7所示，手机显示屏显示的仍然是黑屏界面。该虚拟显示区域是面向PC的服务，PC调用该虚拟显示区域时，可以将该虚拟显示区域中的内容显示到PC中的投屏窗口703。这样，实现了手机处于息屏状态时，PC上的投屏窗口所显示的内容是虚拟显示区域中的应用界面704。
在显示应用投屏窗口期间，用户不仅可以通过PC向手机发送控制信息，还可以通过PC查看手机对于控制信息的响应情况，从而，实现实时的遥控。如，遥控手机输入回复内容，并发送给指定联系人(小明)。再如，遥控手机接打音视频通话、复制、剪切、删除等等。
另需说明的是，在PC和手机之间采用异源投屏的场景下，投屏窗口703中的显示界面与手机显示屏中所示的界面不会同步变化。可以理解的是，在手机处于息屏的状态时，手机显示屏始终显示黑屏界面。
可选的，应用界面704至少包括用户进行回复操作的界面元素，在PC的投屏窗口显示应用界面704后，用户可以通过该界面元素在PC上对该应用消息进行回复。应用界面704包括返回控件707，不包括home键。PC响应于用户点击返回控件707的操作，在投屏窗口703显示应用界面704的上一级应用界面。示例性的，在一些实施例中，上一级应用界面为该应用的主界面，其中，主界面不包括返回控件。
由上文可知，异源投屏的投屏粒度是应用，可理解，在异源投屏中，目标设备上的应用投屏窗口可以显示该应用的所有应用界面，但不显示其他应用的应用界面，包括手机的桌面，所以在该应用的主界面无需设置返回控件。
当然，投屏窗口703上可以配置有关闭控件。PC可以响应于用户对“关闭控件”的选中操作，取消显示上述投屏窗口703。可选的，投屏窗口703还可以包括最小化控件、最大化控件等，在此不作限定。
进一步的，在一些实施例中，在手机处于息屏状态下启动异源投屏的场景中，还可以包括，当手机解锁后：
(1)检测到手机当前的显示界面发生变化时，保持PC上投屏窗口703的显示内容；
(2)检测到用户在应用界面704进行操作后，将用户操作所对应的指令发送至手机，基于指令对手机后台运行的社交应用进行操作，但不改变手机显示屏的显示内容；
(3)检测到用户在手机上打开社交应用，PC关闭投屏窗口703，手机显示屏显示当前社交应用的内容。
也就是说，在一些实施例中，当手机打开正在PC上进行异源投屏的应用时，该应用会被启动至手机前台，异源投屏结束。
在另一些实施例中，可选的，源端设备可以启用安全认证功能，保障已锁屏或息屏的源端设备的隐私性。也就是，在源端设备(如，手机)启用安全认证功能，且配 置有解锁密码的情况下，如果目标设备(如，PC)确定源端设备已具备投屏的条件，那么目标设备可以显示投屏窗口，此时的投屏窗口用于显示安全认证界面。其中，上述解锁密码可以是用于解除手机锁屏状态的认证密码。
可以理解的是，安全认证方式还可以为用户输入指纹进行指纹认证、用户输入语音进行语音认证等，对此不作限定。
可理解，显示安全认证界面的投屏窗口可称为安全认证窗口。
请继续参阅图7，PC检测到用户选中消息卡片702之后，如果确定手机已具备投屏条件，PC可以显示投屏窗口703。可选地，上述投屏窗口703可以先显示安全认证界面705。这样，在PC接收到用户从安全认证界面705输入的手机解锁密码，且解锁密码验证通过之后，可选的，PC控制投屏窗口703切换显示“提示等待的信息”的过渡加载界面706。
可理解，显示“提示等待的信息”过渡加载界面的投屏窗口可称为过渡加载窗口。
在投屏窗口703显示“提示等待的信息”过渡加载界面706期间，手机响应于PC的指示，启动社交应用，并在虚拟显示区域中绘制社交应用的应用界面，得到虚拟显示区域中各个显示对象的绘制参数。其中，上述虚拟显示区域是面向PC的服务，其在手机的显示屏上不可见。上述绘制参数是用于渲染界面的相关参数，在一些示例中也可以称为显示数据。
在手机完成上述应用界面的绘制之后，将对应的显示数据发送给PC，指示PC在投屏窗口703中渲染上述应用界面，这样，上述投屏窗口703切换显示社交应用的应用界面704，也即，手机将虚拟显示区域的内容投屏到PC的投屏窗口703中。也就是，在PC接收到用户输入的手机解锁密码，且解锁密码验证通过之后，手机开始向PC进行异源投屏。
可理解，本申请的说明书和权利要求书及附图中的术语“界面”和“应用界面”，是应用程序或操作系统与用户之间进行交互和信息交换的介质接口，它实现信息的内部形式与用户可以接受形式之间的转换。常用的表现形式是图形用户界面(graphic user interface，GUI)，是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的一个图标、窗口、控件等界面元素，其中控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。
可理解，在上述源端设备处于锁屏或者息屏的场景下，根据本实施例提供的一种投屏的方法，用户可以直接在目标设备上进行操作，实现该流转消息所对应的应用窗口在目标设备上投屏。基于该应用窗口在目标设备上的投屏，通过操控和输入协同来实现跨设备的多屏协同交互的功能。这样一来，用户可以专注于对目标设备的操作，无需对处于锁屏或者息屏状态下的源端设备进行操作，减少了启动投屏的用户操作，提高了启动投屏的人机交互效率，增强了用户体验。
示例性的，在手机处于息屏或者锁屏状态时，就存在这样一种场景：当用户在家中书房或客厅使用PC玩游戏或者办公，手机被放置在其他房间；在社交应用上的联系人(小明)给用户的手机发送一条消息时，用户可以立即在PC上得知自己的手机 收到了一条消息通知。但这种场景下，用户往往专注于在PC上游戏或者办公，并不想起身去房间拿手机查看或回复消息，而想在PC上启动展示该条消息的应用窗口投屏(也即，手机对应的投屏窗口)，并通过PC处理该消息。但由于此时在其他房间的手机处于锁屏息屏状态，基于现有技术用户无法直接在PC上启动展示该条消息的应用窗口投屏，而必须先去房间将手机解锁并使其处于亮屏状态之后，才能够在PC上启动手机对应的投屏窗口。
采用本申请提供的投屏的方法，即使面对前述场景，用户无需再去其他房间对手机进行操作，而是可以继续待在书房或客厅，只要在PC上点击手机对应的消息卡片，就能在PC上启动展示手机对应的投屏窗口，实现异源投屏的窗口显示，从而便于用户对该条消息进行处理，提高了用户体验。
在另一些实施例中，目标设备可以接收同一信任环中任意源端设备流转过来的一条或多条应用消息。示例性地，PC可以响应用户操作，显示消息列表窗口，该消息列表窗口可以集中显示已接收到的消息通知，或者集中显示已接收且未处理过的消息通知。作为一种实现方式，上述消息通知也可以采用消息卡片的形式显示于消息列表窗口。
以目标设备为包含触控屏的PC，源端设备为手机1和手机2为例。在PC检测到指定操作，请参阅图8，图8为显示消息列表窗口的示意图，PC检测到用户在触控屏上的下滑操作，可以显示消息列表窗口801。该消息列表窗口801中显示有来自手机1的应用消息(如消息卡片802)、来自手机2的应用消息(如消息卡片803)等。可以理解，消息列表窗口801中所显示的消息通知均为PC近期接收到的、来自信任环中其他设备的信息。其中，消息卡片802与手机1中社交应用推送的消息对应。消息卡片803与手机2中信息应用推送的消息(比如，短信)对应。
与显示于桌面的消息卡片702相同，在PC检测到用户选中消息列表窗口801中任意一条消息卡片时，均可触发该消息卡片对应的设备进行投屏。例如，PC检测到用户选中消息卡片802时，可以触发手机1进行投屏，这样，手机1为源端设备，PC为对应的目标设备。再例如，PC检测到用户选中消息卡片803时，可以触发手机2进行投屏，这样，手机2为源端设备，PC为对应的目标设备。
可以理解的是，本申请实施例提供了一种投屏的方法，可以应用于诸如手机、平板电脑、笔记本电脑、超级移动个人计算机(ultra-mobile personal computer，UMPC)、上网本、个人数字助理(personal digital Assistant，PDA)等电子设备。本申请实施例对电子设备的具体类型不作任何限制。
示例性的，请参阅图9，图9为本申请实施例提供的一种电子设备100的硬件结构示意图。
电子设备100可以包括处理器110，天线1，天线2，移动通信模块120，无线通信模块130，音频模块140，扬声器140A，受话器140B，麦克风140C，耳机接口140D，传感器模块150，显示屏160等。其中传感器模块150可以包括接近光传感器150A，指纹传感器150B，触摸传感器150C，环境光传感器150D等。
可以理解的是，本发明实施例示意的结构并不构成对电子设备100的具体限定。在本申请另一些实施例中，电子设备100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
处理器110可以包括一个或多个处理单元，例如：处理器110可以包括应用处理器(application processor，AP)，调制解调处理器，图形处理器(graphics processing unit，GPU)，图像信号处理器(image signal processor，ISP)，控制器，视频编解码器，数字信号处理器(digital signal processor，DSP)，基带处理器，和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，也可以集成在一个或多个处理器中。
电子设备100的无线通信功能可以通过天线1，天线2，移动通信模块120，无线通信模块130，调制解调处理器以及基带处理器等实现。
移动通信模块120可以提供应用在电子设备100上的包括2G/3G/4G/5G等无线通信的解决方案。
无线通信模块130可以提供应用在电子设备100上的包括无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(bluetooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信的解决方案。
电子设备100通过GPU，显示屏160，以及应用处理器等实现显示功能。GPU为图像处理的微处理器，连接显示屏160和应用处理器。GPU用于执行数学和几何计算，用于图形渲染。处理器110可包括一个或多个GPU，其执行程序指令以生成或改变显示信息。
电子设备100可以通过音频模块140，以及应用处理器等实现音频功能。例如语音通话等。
扬声器140A，也称“喇叭”，用于将音频电信号转换为声音信号。电子设备100可以通过扬声器140A收听音乐，或收听免提通话。
受话器140B，也称“听筒”，用于将音频电信号转换成声音信号。当电子设备100接听电话或语音信息时，可以通过将受话器140B靠近人耳接听语音。
麦克风140C，也称“话筒”，“传声器”，用于将声音信号转换为电信号。当拨打电话或发送语音信息时，用户可以通过人嘴靠近麦克风140C发声，将声音信号输入到麦克风140C。
耳机接口140D用于连接有线耳机。
接近光传感器150A可以包括例如发光二极管(LED)和光检测器，例如光电二极管。电子设备100可以利用接近光传感器150A检测用户手持电子设备100贴近耳朵通话，以便自动熄灭屏幕达到省电的目的。接近光传感器150A也可用于皮套模式，口袋模式自动解锁与锁屏。
指纹传感器150B用于采集指纹。电子设备100可以利用采集的指纹特性实现指纹解锁，访问应用锁，指纹拍照，指纹接听来电等。
触摸传感器150C，也称“触控器件”。触摸传感器150C可以设置于显示屏160，由触摸传感器150C与显示屏160组成触摸屏，也称“触控屏”。触摸传感器150D用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏160提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器150C也可以设置于电子设备100的表面，与显示屏160所处的位置不同。
环境光传感器150D用于感知环境光亮度。电子设备100可以根据感知的环境光亮度自适应调节显示屏160亮度。环境光传感器150D也可用于拍照时自动调节白平衡。环境光传感器150B还可以与接近光传感器150A配合，检测电子设备100是否在口袋里，以防误触。
下面结合图10介绍本申请实施例提供的一种电子设备的系统结构。
分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。需要说明的是，图10的右半部分是源端设备的软件架构图，比如可以是一个手机。图10的左半部分是目标设备的软件架构图，比如可以是一个平板电脑(Pad)或者PC。
在一些实施例中，如图10的右半部分所示，源端设备的软件框架可以包括应用程序层、能力服务层、应用程序框架层(framework，FWK)、安卓运行时(Android runtime)和系统库、驱动层。
(1)应用程序层
应用程序层可以包括一系列应用程序包，例如社交，信息，通话，相机等应用程序(也可以称为应用)。示例性的，社交应用包括微信、QQ等，信息应用包括短信等。为方便描述，以下将应用程序简称为应用。源端设备上的应用可以是原生的应用(如在源端设备出厂前，安装操作系统时安装在源端设备中的应用)，也可以是第三方应用(如用户通过应用商店下载安装的应用)，本申请实施例不予限定。
(2)能力服务层
能力服务层提供实现相应服务的能力支撑，如图10所示，可以包括投屏助手模块、第一虚拟化服务模块、第一投屏服务模块、第一设备发现认证连接模块等。
其中，投屏助手模块可以是用于与其他电子设备(如目标设备)交互投屏相关信息的模块，实现投屏端到端的逻辑控制及连接管理等，并用于生成会话密钥(SessionKey)以保证源端设备和目标设备之间通信会话安全。投屏助手模块可以包括投屏管理模块、异源切换模块、虚拟化服务初始化模块等功能模块。投屏管理模块用于负责管理投屏相关事务，比如添加和移除VirtualDisplay的相关处理逻辑设置。其中，VirtualDisplay也可称为虚拟显示区域、虚拟屏或虚拟Display等。异源切换模块用于控制源端设备由异源投屏方式切换为同源投屏方式。虚拟化服务初始化模块为启动异源投屏的虚拟化服务做初始化设置。
第一虚拟化服务模块用于实现对音视频流的投屏端到端逻辑控制，负责源端设备和目标设备之间传输的Speaker、Mic、Camera等产生的音频、视频数据码流。
第一投屏服务模块可以包括第一显示虚拟化模块和第一反向控制模块，分别提供 投屏能力(即源端设备的内容投屏到目标设备显示)和反向事件(即通过目标设备触发的对源端设备进行控制的事件)控制基础能力。第一投屏服务模块接收投屏管理模块的指令，按照指令提供相应的投屏服务，从而实现将本机运行的程序投屏至其他电子设备(如目标设备)。第一投屏服务模块将反控和输入法注入到目标设备的显示器(以下称为外屏)，并在创建虚拟显示区域时，添加虚拟显示标签(如，Flag1)，使得外屏只显示其本身的内容，而不镜像源端设备的显示器(以下称为主屏)的内容，从而实现异源投屏。
第一设备发现连接认证模块负责发现、连接、认证等工作，用于控制驱动层实现源端设备和目标设备之间的靠近发现、鉴权、连接等功能，可以包括第一通信模块和第一安全认证模块。第一通信模块，提供源端设备和目标设备之间信令传输通道，用于向驱动层下发建立源端设备和目标设备之间的通信连接和投屏连接的指令。同时，第一通信模块还用于在建立好传输通道后，启动投屏助手模块。第一安全认证模块用于安全认证，负责机主识别和启动分布式解锁认证，获得信任环，保证用户的隐私安全。
(3)应用程序框架层(FWK)
应用程序框架层为应用程序层的应用程序提供应用编程接口(Application Programming Interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。如图10所示，应用框架层可以包括活动管理服务(ActivityManagerService，AMS)、窗口管理服务(WindowManagerService，WMS)、多屏框架模块、SystemUI模块等t。
AMS负责管理Activity，负责系统中各组件的启动、切换、调度及应用程序的管理和调度等工作。对于每一个Activity，在AMS中都会有一个与之对应的应用记录(ActivityRecord)，这个ActivityRecord记录了该应用的Activity的状态。每个活动Activity在AMS中都会创建对应的ActivityRecord对象，AMS可以利用这个ActivityRecord作为标识，调度应用的Activity进程；这些ActivityRecord对象被管理在各自的任务栈中，每个任务栈可以添加多个ActivityRecord对象；所有的任务栈都被ActivityStack统一管理，负责TaskStack的顺序、入栈和出栈。具体的，AMS中定义了分别用来保存进程(Process)、活动(Activity)和任务(Task)的数据类。其中，进程(Process)对应的数据类可以包括进程文件信息、进程的内存状态信息和进程中包含的Activity、Service等。Activity信息可以保存在ActivityStack中。其中，ActivityStack用于统一调度应用程序Activity。ActivityStack具体可以保存所有正在运行的Activity(即final ArrayList mHistory)信息，如界面配置信息。例如正在运行的Activity可以保存在newArrayList中。ActivityStack还可以保存历史运行过的Activity的信息，如界面配置信息。需要注意，Activity并不对应一个应用程序，ActivityThread才对应一个应用程序。因此Android允许同时运行多个应用程序，实际是允许同时运行多个ActivityThread。在Android中，Activity调度的基本思路是这样的：各应用进程在要启动新的Activity或者停止当前的Activity时，向AMS报告。AMS在内部为所有应用进程都做了记录，当AMS接到启动或停止的报告时，首先更新内部记录，然后再通知相应客户进程运行或者停止指定的Activity。由于AMS内部有所有Activity的记录，因此能够调度这些Activity，并根据Activity和系统内存的状态自动关闭后台的Activity。
WMS承载着和“界面”有关的数据和属性，用于管理和“界面”有关的状态，即用于管理在手机屏幕上使用的图形用户界面(graphical user interface，GUI)资源，例如用于管理窗口程序和事件派发。其中，管理窗口程序是指根据应用程序的显示请求在应用服务端和WMS的协助下有序地输出给物理屏幕或其他显示设备。事件派发是指将来自键盘、物理按键、触摸屏、鼠标、轨迹球(TraceBoll)等的用户事件派发给相应的控件或窗口。管理窗口程序还可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等，并负责管理应用窗口的显示方式，包括窗口显示的坐标大小、窗口显示层级等。具体可包括：窗口的创建和销毁、窗口的显示与隐藏、窗口的布局、焦点的管理以及输入法和壁纸管理等。在本申请的一些实施例中，WMS可以保证在手机锁屏息屏状态下的投屏窗口可见性。
需要说明的是，AMS的任务栈管理功能用于管理Activity的生命周期以及Activity之间的调度。WMS的生命周期管理功能用于管理窗口的生命周期。二者相互配合可以通过回调(onResume)接口激活窗口，激活后的窗口在前台显示；或者，通过暂停(onPause)接口暂停窗口以使窗口从前台切换到后台显示。以任务栈中的最上层的窗口(top Activity)为例，top Activity调用onResume以激活，那么top Activity在前台显示，当新的窗口入栈所述任务栈中，且新的窗口位于原top Activity上层时，原top Activity调用onPause暂停窗口以使原top Activity从前台切换到后台显示，对于新的窗口可以使用类似的原理管理生命周期。
多屏框架模块负责实现异源投屏逻辑，调用PendingIntetRecord接口启动PendingIntet对应的Acvtivity(指定行为send),通过添加异源投屏标签(如，Flag2)将该Activity对应的应用启动至虚拟显示区域。其中，PendingIntent是Android提供的一种用于外部程序调起自身程序的能力，生命周期不与主程序相关，当满足某些条件或触发某些事件后执行指定的行为。多屏框架模块还用于监听感应事件并与投屏助手模块进行交互，比如，调用ActivityRecord监听Activity是否在前台且可见，并将监听结果通知投屏助手模块；调用Task监听虚拟显示区域是否栈空并将监听结果通知投屏助手模块。多屏框架模块可以用于对原始输入事件进行翻译、封装等处理，得到包含更多信息的输入事件，并发送到WMS，WMS中存储有每个应用程序的可点击区域(比如控件)、焦点窗口的位置信息等。因此，WMS可以正确的将输入事件分发到指定的控件或者焦点窗口。多屏框架模块可以对接收到的输入事件做出响应，将输入事件转发给WMS，WMS分发给相应的应用，应用向AMS发送注册请求，应用启动。
另外，多屏框架模块还用于在用户点击最近任务卡片或者桌面图标时将投至外屏的应用切换回主屏，此时如果投至外屏的应用采用的是异源投屏方式，在将该应用切换回主屏时需调用上文中的异源切换模块，将异源投屏方式切换为同源投屏方式，否则该应用无法在主屏上显示。多屏框架模块还用于在源端设备锁屏状态下，去除安全图层。在一些实施例中，多屏框架模块还可以用于在投屏场景下，对手机的电源键作相关修改。
其中，SystemUI模块是为用户提供系统级别的信息显示与交互的一套UI组件，主要用于实现状态栏信息显示(比如电池，wifi信号，3G/4G等icon显示)、通知面板(比如系统消息、第三方消息通知)、近期任务栏显示面板(比如显示近期使用的 应用)等。在本申请实施例中，SystemUI模块负责通知服务，用于将源端设备消息通知流转到目标设备。
可理解，应用程序框架层还可以包括内容提供器，视图系统，电话管理器，资源管理器，通知管理器等(图10中未示出)。其具体含义可以参考相关技术文档，在此不展开说明。
(4)安卓运行时和系统库
安卓运行时负责系统的调度和管理。安卓运行时包括核心库和虚拟机。其中，核心库包含两部分：一部分是编程语言(例如，java语言)需要调用的功能函数，另一部分是系统的核心库。应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的编程文件(例如，java文件)执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
系统库可以包括多个功能模块。例如：表面管理器(Surface Manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，二维图形引擎(例如：SGL)等。表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了二维(2-Dimensional，2D)和三维(3-Dimensional，3D)图层的融合。媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。三维图形处理库用于实现3D图形绘图，图像渲染，合成，和图层处理等。2D图形引擎是2D绘图的绘图引擎。
(5)驱动层
驱动层是Android操作系统的基础，Android操作系统最终的功能都是通过驱动层完成。如图10所示，驱动层包括底层驱动，负责发现、认证、连接等工作，比如，驱动层接收应用框架层中通信模块下发的命令，执行连接、断开等动作。具体而言，驱动层包括WI-FI P2P芯片、蓝牙芯片等，可建立WI-FI P2P、蓝牙等传输通道。当然，驱动层还可以包括硬件驱动比如显示屏驱动、摄像头驱动、音频驱动、传感器驱动、虚拟卡驱动等。
需要说明的是，本申请提供的图10所示的电子设备的软件结构示意图仅作为一种示例，并不限定Android操作系统不同分层中的具体模块划分，具体可以参考常规技术中对Android操作系统软件结构的介绍。另外，本申请提供的投屏的方法和装置还可以基于其他操作系统实现，本申请不再一一举例。
在一些实施例中，如图10的左半部分所示，目标设备的软件框架可以包括PC管家、操作系统、驱动层。目标设备的部分功能模块以及功能模块的作用可以参照上述源端设备，此处不再赘述。
具体来说，目标设备的PC管家包括消息中心模块、第二虚拟化服务模块、第二投屏服务模块、第二设备发现连接认证模块。
其中，消息中心模块用于与源端设备的消息及信令进行交互接收从源端设备流转过来的消息通知，并负责向源端设备发起会话指令(openSession)。同时，消息中心 模块还用于在建立会话(session)后，下发启动投屏服务的指令。
第二虚拟化服务模块、第二投屏服务模块、第二设备发现认证连接模块的功能可分别参考上述源端设备中的第一虚拟化服务模块、第一投屏服务模块、第一设备发现连接认证模块，其中不同的是，第二投屏服务模块用于接收并解析其他设备发送的视频流和相关信息，从而在本机显示在其他设备上运行的应用。
目标设备的操作系统可以是Windows OS(如个人计算机)，也可以是Android OS(如平板电脑)。
目标设备的驱动层请参考上述源端设备。
基于图10所示的电子设备的系统结构，以手机为源端设备、PC为目标设备举例，结合图11至图13说明前述实施例的具体实现方式。
请参阅图11至图13，图11至图13为本申请实施例提供的一种投屏的方法和装置的时序图。可以理解的是，图11至图13是一组连续的时序图，表明了图11所示的各个功能模块之间的交互流程。
示例性的，手机端包括SystemUI模块、第一通信模块、投屏助手模块、第一虚拟化服务模块、第一投屏服务模块、第一安全认证模块、FWK、应用；PC端包括消息中心模块、第二投屏模块、第二通信模块、第二安全认证模块。需要说明的是，图10所示的其他功能模块也参与了本申请实施例提供的一种投屏的方法和装置的具体实现过程，未在本时序图中示出。
需要说明的是，本申请实施例如无特别说明，手机处于锁屏或者息屏状态。
1、源端设备和目标设备建立第一通信连接，源端设备的应用消息流转到目标设备，并在目标设备上显示消息通知控件(请参阅图11)
S101：PC的消息中心模块接收到用户选中手机的操作。
可以理解的是，S101仅作为PC与手机建立第一通信连接的一种示例，PC检测到与其处于同一局域网的设备，显示所有可连接设备供用户选择，用户在PC上选中手机以启动后续的连接行为。启动PC和手机建立第一通信连接还可以有其他方式，比如手机检测到与其处于同一局域网的设备，显示所有可连接设备供用户选择，用户在手机上选中PC以启动后续的连接行为。又比如，手机与PC也可以通过“碰一碰”功能、NFC功能等建立近场连接，此处不做限定。
S102：消息中心模块向第二通信模块发送指示与手机建立通信连接的消息，该消息中携带手机的设备标识码。
其中，用户在PC搜索发现的附近可用协同通知设备中，选中了手机作为协同通知设备。PC的消息中心模块接收到用户输入的选中操作后，可以向第二通信模块发送一个进程间通信(inter-process communication，IPC)消息，可选地，该IPC消息可以携带手机的设备名称、设备标识码、MAC地址等，用于指示第二通信模块与手机建立通信连接。
S103：第二通信模块与手机的第一通信模块建立第一通信连接。
其中，示例性的，第一通信连接可以是蓝牙连接(BR连接)，用于实现PC与手机之间的协同通知，即手机上的消息可以通过蓝牙通道流转到PC上。
S104：第二通信模块向消息中心模块返回建立第一通信连接成功的消息。
S105：手机的应用接收到一条应用消息。
在PC和手机建立第一通信连接之后，手机上接收到由安装在手机上的应用推送的一条应用消息，该应用消息在手机上的应用(如，第一数据)、SystemUI模块以及PC上的消息中心模块(如，第一消息)中可以有不同的表现形式，此处统称为应用消息。
S106：应用向SystemUI模块发送应用消息，该应用消息携带消息内容和应用包名。
S107：SystemUI模块指示第一通信模块向PC转发应用消息。
示例性的，本申请实施例中，在PC与手机建立第一通信连接之后，SystemUI模块会基于第一通信连接将所有手机接收到的消息流转到PC上；也可以选择性地流转，比如带有某种特殊标签的消息会流转到PC，或者来自特定应用的消息会流转到PC，或者判断重要程度较高的消息会流转到PC，等等，此处不做限定。
S108：第一通信模块基于第一通信连接向第二通信模块发送应用消息，该应用消息携带消息内容和应用包名。
S109：第二通信模块向消息中心模块发送应用消息，该应用消息携带消息内容和应用包名。
S110：消息中心模块接收到应用消息，控制显示屏显示对应的消息通知控件。
其中，PC与手机建立第一通信连接之后，手机上收到的应用消息会流转到PC。可以理解的是，此时手机处于解锁亮屏状态或者锁屏、息屏状态都不影响消息通知在PC和手机之间的流转。PC接收到应用消息后，显示与所述应用消息相关联的消息通知控件。
示例性的，请返回参阅图7，手机处于息屏状态，当手机收到一条应用消息时，PC上会显示与该条应用消息相关联的消息通知控件，如消息卡片702。
2、当源端设备处于锁屏或者息屏状态时，目标设备接收到对消息通知控件的用户操作，目标设备与源端设备建立第三通信连接，目标设备向源端设备发送启动异源投屏请求，源端设备在锁屏或者息屏状态下，在目标设备显示应用投屏窗口(请参阅图12-图13)
S201：当手机处于锁屏或者息屏状态时，消息中心模块接收到用户点击消息通知控件的第一操作。
S202：消息中心模块向第二通信模块发送指示创建第二会话的消息，该消息携带业务通道名称(WI-FI P2P通道)和手机的设备标识码。
其中，示例性的，业务通道名称可以为WI-FI P2P通道，设备标识码为手机的标识码，上述创建第二会话的指令用于指示第二通信模块在PC和设备标识码对应的手机之间建立WI-FI P2P通道。
S203：第二通信模块与第一通信模块建立第三通信连接。
示例性的，第三通信连接可以是WI-FI P2P连接，提供投屏服务的数据传输通道。
需要说明的是，上述PC和手机之间所建立第一通信连接、第二通信连接、第三通信连接的类型不限于蓝牙连接和Wi-Fi P2P连接，也可以是其他类型的连接方式， 只要能满足两者之间传输数据的要求即可。
S204：第一通信模块向投屏助手模块发送指示启动的消息。
S205：投屏助手模块启动。
第一通信模块还负责与投屏助手的交互，投屏助手模块在手机无投屏需求时处于关闭状态，由第一通信模块在有投屏需求时触发投屏助手模块启动。
S206：第二通信模块向消息中心模块返回建立第三通信连接成功的消息。
S207：基于第三通信连接，消息中心模块与投屏助手进行参数协商，确定会话密钥和对端的IP地址。
参数协商包括业务侧协商SessionKey并获取对端设备的IP地址。其中，SessionKey也称为数据加密密钥或者工作密钥，是保证用户跟其它计算机或者两台计算机之间安全通信会话而随机产生的加密和解密密钥，它可由通信用户之间进行协商得到。它一般是动态地、仅在需要进行会话数据加密时产生。
S208：消息中心模块向第二投屏服务模块发送指示启动的消息，该消息携带会话密钥和手机的IP地址。
S209：第二投屏服务模块启动，根据会话密钥和手机的IP地址进行初始化。
S210：第二投屏服务模块向消息中心模块返回初始化已完成的消息。
由上文可知，投屏服务模块提供投屏能力和反向事件控制基础能力。PC端的投屏服务初始化成功后，为接收并解析手机端发送的相关信息提供能力支撑，以实现在PC端显示在手机端运行的应用。同时反向事件控制基础能力为实现异源投屏提供支撑。所以在进行异源投屏服务之前需要投屏服务模块进行初始化，以保证正常提供投屏能力和反向事件控制基础能力。
S211：消息中心模块向第二投屏服务模块发送指示启动投屏服务的消息。
S212：基于第三通信连接，第二投屏服务模块向投屏助手模块发送启动投屏服务的消息。
需要说明的是，如图所示，步骤S207包括第二投屏服务模块向第二通信模块发送消息，第二通信模块再通过第三通信连接将该消息传输给第一通信模块，第一通信模块再向投屏助手模块发送该消息。为了方便描述，以下“基于第三通信连接”均包含第一通信模块和第二通信模块之间的数据传输过程，下文不再赘述。
S213：投屏助手模块绑定投屏服务。
可理解，投屏助手模块中的投屏管理模块绑定相关的投屏服务以管理投屏事务。
S214：投屏助手模块向第一虚拟化服务模块发送指示初始化的消息。
可理解，投屏助手模块中的虚拟化服务初始化模块用于指示虚拟化服务模块进行初始化。
S215：第一虚拟化服务模块完成初始化。
第一虚拟化服务模块负责投屏服务中的音视频虚拟化，手机端的第一虚拟化服务模块与PC端的第二虚拟化服务模块(图13中未示出)交互可以实现在投屏过程中，被投屏到PC上的手机应用可以使用PC的Speaker、Mic、Camera，并在手机和PC之间传输所产生的音频、视频数据码流，以此实现投屏时接打音视频电话等功能。
S216：第一虚拟化服务模块向第一投屏服务模块发送指示启动投屏服务的消息。
S217：第一投屏服务模块启动。
S218：第一投屏服务模块创建虚拟显示器，并为其添加虚拟显示标签。
需要说明的是，投屏服务模块是实现异源投屏方式的重要功能模块。具体的实现方式是手机端的第一投屏服务模块在创建虚拟显示区域时添加Flag1：VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY(虚拟显示标签)，用于实现PC上的投屏窗口只显示其本身的内容，而不会镜像手机显示屏所显示的内容，以此保证手机的显示内容不会影响PC的投屏窗口的显示内容。
这样可以实现两个目的：一是在手机锁屏或者息屏状态下，手机显示黑屏或锁屏界面，PC的投屏窗口不会显示黑屏或锁屏界面；二是在手机使用过程中，手机显示其他应用界面或桌面，不会影响PC的投屏窗口的显示，PC的投屏窗口能够维持本身的内容而不与手机的显示内容同步。同时，投屏服务模块提供的反向事件控制基础能力能够实现在电脑端反向控制手机端。当然，异源投屏方式的实现还需要PC端的第二投屏服务模块与手机端的第一投屏服务模块配合交互，图12未示出。
需要说明的是，手机如果处于锁屏或者息屏状态，即便当前被执行的Activity所对应的应用图层已经绘制渲染至投屏窗口，但安卓系统会默认把手机显示屏上的锁屏图层覆盖在VirtualDispaly上面，表现就是，PC显示屏上的投屏窗口也会被锁屏图层覆盖，即PC上的投屏窗口也显示锁屏图层，投屏异常。而添加了Flag1的虚拟现实区域则不会将手机显示屏所显示的内容镜像到PC的投屏窗口，实现异源投屏，这就解决了图5和图6所示的投屏窗口显示黑屏或锁屏界面的问题。
S219：第一投屏服务模块向投屏助手模块发送已启动投屏服务的消息。
S220：基于第三通信连接，第一投屏助手向消息中心模块发送已启动投屏服务的消息。
S221：基于第三通信连接，消息中心模块向SystemUI发送启动应用进行投屏的消息，携带应用包名。
S222：SystemUI向FWK发送启动应用投屏的消息，该消息携带应用包名。
S223：FWK启动应用投屏。
需要说明的是，在手机锁屏或者息屏的状态下，为了能够保证PC上的异源投屏窗口能够正常显示应用界面，需要FWK控制应用的异源投屏窗口的可见性。可理解，FWK具体的实现过程涉及到FWK的内嵌类之间的功能交互，交互流程请参见下文，此处不予赘述。
S224：FWK向第一投屏服务模块发送应用投屏的显示数据。
S225：基于第三通信连接，第一投屏服务模块向第二投屏服务模块发送应用投屏的显示数据。
S226：第二投屏服务模块控制显示屏将上述显示数据渲染到投屏窗口。
S227：第二投屏服务模块向消息中心发送投屏窗口的应用投屏界面渲染完成的消息。
S228：消息中心模块控制显示屏显示应用投屏窗口。
至此，PC显示应用消息所对应的应用投屏窗口，在手机处于锁屏息屏状态下投屏成功。
示例性的，请返回参阅图7，手机处于息屏状态，当PC接收到用户点击消息卡片702的操作，PC上会显示上述社交应用对应的投屏窗口703，用于显示应用消息所对应的应用界面704。上述应用投屏窗口可以是用于显示应用界面704的投屏窗口703。
下面结合图14说明前述实施例中FWK启动应用投屏的具体实现过程。
由于手机处于锁屏或者息屏的状态时，应用消息对应的Activity会进入休眠状态，其生命周期会从OnResume状态(用户可见且可与用户交互)走到OnPause状态(非活动状态)，所以投屏至PC时无法正常显示应用界面。为了实现应用投屏，需要保证应用的Activity跳过休眠状态，并且当前执行的Activity保持在OnResume状态，以保证应用对应的投屏窗口的可见性。
请参阅图14，图14为图13中步骤S223的具体实现过程。如图14所示，FWK内部还包括PendingIntentRecord(异步处理器)、ActivityStarter(活动启动器)、ActivityTaskManagerServiceEx(活动栈管理服务器)、ActivityTaskSupervisor(活动栈管家)、KeyguardController(锁屏控制器)、ActivityRecord(活动记录器)、Task(任务栈)、RootWindowsContainer(根窗口容器)、SurfaceFlinger(图层合成器)等内嵌类。
可以理解的是，图14所示的步骤S301至步骤S318介于图13所示的步骤S222和步骤S224之间。
S222:SystemUI模块向PendingIntentRecord发送启动应用投屏的请求，该请求携带应用包名(SystemUI向FWK发送启动应用投屏的消息，该消息携带应用包名)。
S301：PendingIntentRecord将应用的所有Activity标记投屏标签。
需要说明的是，添加了投屏标签的Activity才会被启动进行投屏，步骤S301相当于一个Activity过滤器，过滤出应用的所有Activity，从而把需要启动投屏的应用的所有Activity打上特殊的标记。
示例性的，应用消息1对应Activity1，PendingIntentRecord将Activity1标记异源投屏标签Flag2：Intent.java--->FLAG_HW_ACTIVITY_PC_CAST_NOTIFICATION＝0x04000000。应用还可以包括Activity2、Activity3，也被标记异源投屏标签Flag2。因为应用被投屏后，用户可以在应用投屏窗口操控执行该应用的其他Activity，所以需要将应用的所有Activity都过滤出来执行下述相关步骤。
S302：PendingIntentRecord向ActivityStarter发送启动标有投屏标签的Activity的请求，该请求携带应用包名。
S303：ActivityStarter向ActivityTaskSupervisor发送指示启动应用投屏的请求。
S304：ActivityTaskSupervisor向KeyguardController发送指示将带有投屏标签的Activity设置为OnResume状态的消息。
需要说明的是，ActivityTaskSupervisor通过调用KeyguardController监听Activity是否处于OnResume状态，如果是的话则说明该Activity所对应的应用投屏窗口启动至PC后对用户可见且可与用户交互(OnResume状态)；如果Activity处于停止状态(OnPause状态)，则说明该Activity所对应的应用投屏窗口启动至PC后在后台，用户不能进行操作。其中，OnResume状态和OnPause状态为Activity的生命周期中的两 种状态。当Activity处于OnResume状态时，该Activity处于可见且能够与用户交互的激活状态，用户可以获得activity的焦点；当Activity处于OnPause状态时，该Activity被另一个透明或者Dialog样式的Activity覆盖，此时它依然与窗口管理器保持连接，系统继续维护其内部状态，它仍然可见，但它已经失去了焦点，故不可与用户交互。在手机处于锁屏或息屏状态时，AMS组件启动Activity的流程中，会将Activity的生命周期切换至onPause状态，导致Activity不能与用户交互，所以投屏出现异常。
S305：KeyguardController识别出带有投屏标签的Activity，将其设置为OnResume状态。
KeyguardController为锁屏控制器，识别出带有投屏标签的Activity并返回“ture”，以指示将该Activity设置为OnResume状态，保证在源端设备锁屏状态下，带有投屏标签的Activity所对应的应用投屏窗口在目标设备上可见。
S306：KeyguardController向ActivityRecord发送指示将带有投屏标签的Activity保持OnResume状态的消息。
S307：ActivityRecord控制带有投屏标签的Activity保持OnResume状态。
S308：ActivityRecord向Task发送指示校验的消息，指示Task校验位于任务栈栈顶的Activity状态。
S309：Task确定位于任务栈栈顶的Activity处于OnResume状态。
需要说明的是，一个应用可能对应一个或多个Activity，一个或多个Activity会被存放在一个或多个任务栈中。任务栈遵循后进先出的规律，位于任务栈栈顶的Activity最先被执行。为了保证即将被执行的Activity对应的应用投屏窗口前台可见，需要保证栈顶的Activity处于OnResume状态。
可以理解的是，步骤S308和S309相当于容错机制，校验位于任务栈栈顶的Activity是否处于OnResume状态，所以该两个步骤为可选项，即在另一些实施例中，也可以不包括S308和S309。
S310：Task指示RootWindowsContainer控制所有任务进入休眠状态。
S311：RootWindowsContainer遍历所有任务的Activity，跳过带有投屏标签的Activity，控制其他Activity进入休眠状态。
需要说明的是，除了带有投屏标签的Activity跳过休眠流程，其他activity都会被置于休眠状态。因此，RootWindowsContainer需要遍历所有任务对应的Activity，从中过滤出带有投屏标签的Activity并返回“false”，以指示该Activity不进入休眠；其他Activity则返回“true”，使其进入休眠流程。
示例性的，同一个任务栈中的Activity1和Activity2都被添加了异源投屏标记，RootWindowsContainer使Activity1和Activity2跳过休眠流程；而未添加异源投屏标记的Activity3和Activity4则被置于休眠状态。
当Activity1位于任务栈栈顶时，Task校验Activity1的状态，保证其处于OnResume状态，进而Activity1被异源投屏后在PC上对用户可见且在前台可被用户操作。
执行完Activity1，此时未休眠的Activity2递进到了任务栈栈顶，Task校验Activity2的状态，保证其处于OnResume状态，进而Activity2被异源投屏后在PC上对用户可见且在前台可被用户操作。其他被添加了异源投屏标记的Activity的执行流程依此类 推，此处不再赘述。而未被添加了异源投屏标记的Activity会被RootWindowsContainer置于休眠状态，不会被启动至外屏显示。
S312：ActivityStarter向应用发送指示启动的消息。
可以理解的是，ActivityStarter在步骤S304中发送启动应用投屏的请求时，也会发送启动应用的请求，因为要实现应用进行异源投屏，前提是应用在手机端启动。所以步骤S223包括FWK与应用程序层的交互，即FWK中的ActivityStarter调起应用程序层的应用。
S313：应用启动。
S314：应用向ActivityTaskManagerServiceEx发送启动成功的消息。
S316：ActivityTaskManagerServiceEx向SurfaceFlinger发送指示启动图层合成的消息。
S317：SurfaceFlinger确定虚拟显示区域带有虚拟显示标签，在虚拟显示区域上合成应用图层，得到对应的显示数据。
其中，SurfaceFlinger的作用是接收多个来源的图形显示数据，将他们合成，然后发送到显示设备。比如打开应用，常见的有三层显示，顶部的statusbar、底部或者侧面的导航栏以及应用的界面，每个图层是单独更新和渲染，这些界面都是由SurfaceFlinger合成为一个应用图层显示数据刷新到硬件显示。
在本申请实施例中，SurfaceFlinger确定虚拟显示区域带有虚拟显示标签，则在合成应用图层时只获取虚拟显示区域上的图层，而不会再获取其他显示区域的图层与它合成，这样就能实现前述的异源投屏方式，即合成的应用图层只与虚拟显示区域的图层相关。请返回参阅图4，SurfaceFlinger不会获取可见显示区域的图层，所以即使可见显示区域里包括锁屏界面或者其他应用界面，都不会影响PC的投屏窗口所显示的界面。因为SurfaceFlinger只获取虚拟显示区域的图层，并将其以某种方式传递到PC的投屏窗口进行显示。
S318：SurfaceFlinger向第一投屏服务模块发送显示数据。
可理解，上述应用图层可以包括当前执行的Activity对应的应用界面。上述显示数据可以包括应用图层中不同的显示对象、显示对象在虚拟显示屏中的显示位置、大小等信息。
需要说明的是，虚拟显示区域为步骤S218所创建的标记Flag1的虚拟显示区域。虚拟显示区域的内容对用户不可见，手机可将虚拟显示区域作为占用一定存储空间的画布使用。例如，请参阅图17A，图17A为启动第一应用投屏窗口的过程图。当启动投屏的第一请求所携带的应用包名对应应用1，那么PendingIntentRecord将应用1的所有Activity都打上投屏标签，如图17A中的任务堆栈(由Task创建)所示。在执行应用1的Activity1(Activity1对应第一消息)时，如图17A所示，手机在虚拟显示区域中绘制Activity1对应的第一应用界面，SurfaceFlinger确定虚拟显示区域带有虚拟显示标签，则只获取该虚拟显示区域中的第一应用界面图层，并生成对应的第一显示数据发送给目标设备。
目标设备收到第一显示数据后，根据第一显示数据绘制渲染与第一应用的第一消息相对应的应用投屏界面，在目标设备的投屏窗口显示。具体请参阅图13中的步骤 S225-S228以及图7。此时目标设备的投屏窗口显示与第一显示数据相对应的应用投屏界面。
在执行应用1的Activity2时(Activity2被推到了任务栈栈顶)，手机在虚拟显示区域中绘制Activity2对应的第二应用界面，SurfaceFlinger确定虚拟显示区域带有虚拟显示标签，则只获取该虚拟显示区域中的第二应用界面图层，并生成对应的第二显示数据发送给目标设备。随后目标设备的投屏窗口显示与第二显示数据相对应的应用投屏界面，即由与第一显示数据相对应的应用投屏界面切换为与第二显示数据相对应的应用投屏界面，完成应用1的不同应用界面在投屏窗口中的切换。
示例性的，在本申请的一些实施例中，请参阅图17B，图17B为启动第二应用投屏窗口的过程图。在本申请实施例提供的投屏的方法中，当目标设备的投屏窗口显示第一应用对应的应用投屏界面时(如，目标设备的投屏窗口显示与第二显示数据相对应的应用投屏界面)，此时源端设备又收到来自第二应用的第二消息(也称，第二数据)。第二消息流转至目标设备后，目标设备显示第二消息对应的消息通知控件。目标设备接收到源端设备点击该消息通知控件的操作后，向源端设备发送启动投屏的第二请求。在源端设备锁屏或者息屏状态下，第一投屏服务模块接收到第二请求，创建第二虚拟显示区域(VirtualDispaly2)并添加虚拟显示标签。在执行应用2的Activity3(Activity3对应第二消息)时，如图17B所示，手机在虚拟显示区域中绘制Activity3对应的第三应用界面，SurfaceFlinger确定虚拟显示区域带有虚拟显示标签，则只获取该虚拟显示区域中的第三应用界面图层，并生成对应的第三显示数据发送给目标设备。
目标设备收到第三显示数据后，根据第三显示数据绘制渲染与第二应用的第二消息相对应的应用投屏界面，在目标设备的投屏窗口显示。具体请参阅图13中的步骤S225-S228以及图7。示例性的，目标设备的投屏窗口显示与第二显示数据相对应的应用投屏界面切换为与第三显示数据相对应的应用投屏界面，完成不同应用(如，第一应用和第二应用)的应用界面在投屏窗口中的切换。
下面结合图15和图16说明在另一些实施例中，源端设备如果启动了安全认证功能的情况下，前述实施例的具体实现方式。
请返回参阅图7，在另一些实施例中，可选的，源端设备可以启用安全认证功能，保障已锁屏或息屏的源端设备的隐私性。在投屏窗口703显示应用界面704之前，还可以显示安全认证界面705，并还可以显示过渡加载界面706。这种情况下，上述步骤S201即消息中心模块接收到用户点击消息通知控件的第一操作后，会先触发安全认证机制，具体实现过程请参阅图14及以下说明。
1、当源端设备处于锁屏或者息屏状态时，目标设备接收到对消息通知控件的用户操作，源端设备与目标设备建立第二通信连接(请参阅图15)
S401：当手机处于锁屏或者息屏状态时，消息中心模块接收到用户点击消息通知控件的第二操作。
需要说明的是，此处的第二操作与图12中步骤S201中的第一操作均为用户点击消息通知控件的操作，在本实施例中，由于增加了安全认证机制，所以在触发图12中建立第三通信连接之前，增加了图15和图16所示步骤。
可以理解的是，当手机锁屏或者息屏时是没有经过用户的安全认证的，这种情况下如果允许PC不通过安全认证直接启动投屏，那么用户的隐私安全就得不到保障。所以，在一些实施例中，从PC端启动投屏之前会触发安全认证机制以保证用户的隐私安全。
由于本申请实施例是在PC端直接启动投屏，所以会在PC端触发安全认证机制，而密码认证分布在手机端，所以PC端的第二安全认证模块需要把在PC端接收到的认证信息(密码)发往手机端的第一安全认证模块进行密码认证，手机端会向PC端返回认证结果。所以为了在手机和PC之间需要进行安全认证的信令传输，需要先建立传输通道。
S402：消息中心模块向第二通信模块发送指示创建第一会话的消息，该消息携带业务通道名称(蓝牙通道)和手机的设备标识码。
其中，示例性的，业务通道名称可以为蓝牙通道，设备标识码为手机的标识码，上述创建第一会话的指令用于指示第二通信模块在PC和设备标识码对应的手机之间建立蓝牙通道。
S403：第二通信模块与第一通信模块建立第二通信连接(蓝牙连接)。
示例性的，第二通信连接可以是蓝牙连接(BR连接)，提供安全认证的信令传输通道。
S404：第一通信模块向投屏助手发送指示启动的消息。
S405：投屏助手启动。
S406：第二通信模块向消息中心模块返回建立第二通信连接成功的消息。
S407：消息中心模块向第二投屏服务模块发送指示启动的消息。
S408：第二投屏服务模块启动。
S409：第二投屏服务模块向消息中心模块返回启动成功的消息。
需要说明的是，此时，手机端的投屏助手和PC端的第二投屏服务模块已启动，为实现接下来的投屏服务提供必要的功能。
2、目标设备向源端设备发送安全认证请求，源端设备返回认证成功结果(请参阅图16)
S501：消息中心模块向PC的第二安全认证模块发送安全认证请求。
S502：第二安全认证模块控制PC的显示屏显示安全认证窗口。
示例性的，请返回参阅图7所示的安全认证窗口，即在投屏窗口703显示安全认证界面705，用户可在上述安全认证窗口输入密码。
可以理解的是，安全认证方式不限于输入密码进行安全认证，比如指纹解锁场景下，采集指纹进行验证；语音解锁场景下，采集语音进行验证。此处不做限定，仅以密码解锁作为一种示例。
S503：第二安全认证模块接收到用户通过安全认证窗口输入的密码。
S504：第二安全认证模块向第二通信模块发送密码。
S505：第二通信模块基于第二通信连接向第一通信模块发送密码。
S506：第一通信模块向手机的第一安全认证模块发送密码。
可以理解的是，安全认证方式还可以为用户输入指纹进行指纹认证、用户输入语音进行语音认证等，对此不作限定。第一安全认证模块接收到PC端采集的密码，与用户在手机端设置的解锁密码进行比对，如果PC端采集的密码与手机端设置的解锁密码匹配，则认证成功，否则认证失败。
S507：第一安全认证模块密码认证成功。
在手机端密码认证成功则向PC端返回密码认证成功结果；在手机端密码认证失败则向PC端返回密码认证失败结果，可选的，PC端接收到密码认证失败结果，可在安全认证窗口提示用户密码错误，用户重新输入密码后再次向手机端发起安全认证请求进行密码验证。当然，可以设置为认证失败的次数超过密码错误次数阈值则挂起认证，密码错误次数阈值可以为用户自行设置或者为默认值。
S508：第一安全认证模块向第一通信模块发动认证成功的消息。
S509：第一通信模块基于第二通信连接向第二通信模块发送认证成功的消息。
S510：第二通信模块向第二安全认证模块发送认证成功的消息。
S511：第二安全认证模块向消息中心模块发送认证成功的消息。
S512：消息中心模块控制显示屏显示过渡加载窗口。
示例性的，请返回参阅图7所示的过渡加载窗口，即在投屏窗口703中显示过渡加载界面706。可以理解的是，认证成功则启动图12至图15的应用投屏步骤，成功启动应用投屏需要一定的时间，所以在显示安全认证窗口至显示应用投屏窗口之间，存在一定的时间跨度。可选的，为了提高用户观感和体验，可以在安全认证成功后，启动过渡加载窗口在目标设备上显示，可提示用户已通过安全认证，正在启动应用投屏。当经过图12至图15的应用投屏步骤成功启动应用投屏后，投屏窗口703显示过渡加载界面切换为应用界面。
在另一些实施例中，目标设备接收到用户指示对应的源端设备进行应用投屏之后，先判断是否需要进行安全认证。比如，源端设备是否首次在目标设备投屏。再比如，源端设备是否设置解锁密码。
下面介绍本申请实施例提供的一种投屏的方法。该方法在目标设备接收到对消息通知控件的用户操作时增加了判断机制，请参考图18，图18为增加了安全认证判断的流程示例图。该方法可以包括但不限于以下步骤：
S1：源端设备和目标设备建立第一通信连接；
S2：源端设备接收到应用消息；
S3：源端设备的应用消息流转到目标设备；
S4：目标设备上显示与应用消息相关联的消息通知控件；
S5：在源端设备处于锁屏或者息屏状态时，目标设备接收到对消息通知控件的用户操作；
S6：源端设备与目标设备建立第二通信连接；
S7：目标设备向源端设备发送安全认证请求；
S8：源端设备向目标设备发送认证成功的消息；
S9：目标设备与源端设备建立第三通信连接；
S10：目标设备向源端设备发送启动应用投屏请求；
S11：源端设备启动应用投屏；
S12：源端设备向目标设备发送应用投屏的显示数据；
S13：目标设备显示应用投屏窗口。
作为一种实现方式，在步骤S5和步骤S6之间增加了是否需要进行安全认证的判断机制。在源端设备首次向目标设备投屏的场景下，如果源端设备未设置解锁密码，目标设备无需在投屏窗口中显示解锁验证界面，而是指示源端设备直接进行异源投屏。在此场景下，目标设备可以直接显示“提示等待的信息”。待源端设备将需投屏的应用界面绘制完成之后，目标设备再控制上述投屏窗口显示该需投屏的应用界面。
在源端设备首次向目标设备投屏的场景下，如果源端设备设置了解锁密码，也就是说，用户直接使用源端设备之前，也需要进行密码认证。这种情况下，目标设备确定源端设备具备投屏的条件之后，可以触发安全认证。关于安全认证的描述可参考上文，在此不再赘述。
另外，需要说明的是，源端设备首次向目标设备进行异源投屏，且设置了解锁密码，那么无论源端设备处于锁屏息屏状态还是解锁状态，当在目标设备指示源端设备进行异源投屏时都会触发安全认证机制。
需要说明的是，源端设备首次在目标设备上启动投屏，可以指的是源端设备在同一目标设备上第一次启动异源投屏，也可以指的是在目标设备开机至当次关机的使用期间内第一次启动异源投屏。第二种解释具体来说，假设目标设备第一次开机，使用一段时间后目标设备第一次关机，第一次开机至第一次关机为第一使用期间。目标设备第二次开机，使用一段时间后目标设备第二次关机，第二次开机至第二次关机为第二使用期间。在第一使用期间，源端设备第一次在目标设备上启动异源投屏属于第二种解释中所指的“首次”启动；在第二使用期间，源端设备第一次在目标设备上启动异源投屏也属于第二种解释中所指的“首次”启动。
作为一种实现方式，在源端设备非首次向目标设备投屏的场景下，也即，源端设备在同一目标设备上曾经启动过异源投屏的场景下，又或者是，在目标设备开机至当次关机的使用期间内，源端设备在目标设备上启动过异源投屏。
第二种解释具体来说，在第一使用期间，源端设备第一次在目标设备上启动异源投屏，待源端设备第二次在目标设备上启动异源投屏，此时目标设备将判定源端设备在目标设备上启动过异源投屏；此后的第三次、第四次等亦如此。在第二使用期间，源端设备第一次在目标设备上启动异源投屏，由上文可知，此时目标设备将判定源端设备首次在目标设备上启动异源投屏；待源端设备第二次在目标设备上启动异源投屏，此时目标设备将判定源端设备在目标设备上启动过异源投屏；此后的第三次、第四次等亦如此。如果源端设备在目标设备上启动过异源投屏，为了在保障用户隐私安全的前提下，减少用户输入密码的次数，可选的，目标设备可以先判断距离上次通过安全认证的时间是否超过预设时长阈值。
示例性地，距离上次通过安全认证的时间没有超过预设时长阈值时，可以不用重新进行安全认证。需要说明的是，预设时长阈值可以是默认值，也可以是用户自行设置的值，它表明在这个时间范围内，源端设备和目标设备之间的投屏连接仍满足保护 用户隐私安全的需求。所以当目标设备确定距离上次通过安全认证的时间没有超过预设时长阈值时则直接启动投屏，无需显示解锁验证界面，这样能够避免反复要求用户输入密码导致操作繁琐，在保障用户隐私安全的同时提高用户使用异源投屏的便捷性。
又示例性地，距离上次通过安全认证的时间超过预设时长阈值时，需要重新进行安全认证。可以理解地，当目标设备确定距离上次通过安全认证的时间超过预设时长阈值时，则认为此时源端设备和目标设备之间的投屏连接已经不满足保护用户隐私安全的需求，这种情况下的安全认证流程与上文中“源端设备首次在目标设备上启动异源投屏”的安全认证流程相同，在此不再赘述。
示例性的，假设源端设备处于锁屏息屏状态并设置了解锁密码，此前未在目标设备上启动过异源投屏。在目标设备确定用户指示源端设备进行投屏，且确定源端设备具备投屏条件时，可以触发安全认证机制，通过安全认证后，成功启动第一次异源投屏；在T1时间第一次异源投屏结束；假设用户所设的预设时长阈值为1小时，在T2时间目标设备再次检测到用户指示同一源端设备进行异源投屏；目标设备确定源端设备在目标设备上启动过异源投屏，进而判断T2距离T1的时间是否大于1小时；目标设备确定(T2-T1)的值小于1小时，直接启动第二次异源投屏。
在另一些实施例中，目标设备接收到用户指示对应的源端设备进行异源投屏之后(即，步骤S5之后)，也可能出现投屏失败的情况。比如，消息通知控件对应的应用受应用锁保护的情况下，导致投屏失败。再比如，用于投屏的通道被其他设备占用的情况下，导致投屏失败。再例如，网络不稳定的情况下，导致投屏失败等。
首先，应用锁指的是源端设备的用户为了保护某一应用的安全性或隐私性而针对该应用设置了保护，需要输入预先设置好的密码或指纹等才能打开该应用。所以，在源端设备处于锁屏息屏状态的场景中，出于对源端设备安全性和隐私性的考虑，一些可能的实施例中，实际投屏之前，会判断应用a(例如，用户在PC上选中的消息卡片所对应的应用)是否被源端设备可选地增加了一层保护。如果源端设备为该应用a设置了应用锁，说明该应用a是源端设备的机主希望在每次访问时都需要输入密码或指纹等进行解锁的，那么对于这类强隐私保护的应用就不适用在源端设备处于锁屏息屏状态下直接在目标设备一侧启动投屏，这违背了用户设置应用锁的初衷。
因此，本申请实施例提供了可能的三种启动投屏失败的场景。PC检测到用户选中消息卡片702之后，向对应的设备(也即，手机)发送查询信息1，该查询信息1用于查询手机是否为信息应用配置有应用锁。如果手机中为信息应用配置有应用锁，那么手机会向PC发送反馈信息1，该反馈信息1指示信息应用当前受应用锁保护。这样，PC在接收到反馈信息1之后，可以显示提醒信息1。该提醒信息1用于提示用户被选中的消息卡片对应的应用受应用锁保护。例如，可以是文字“信息应用受应用锁保护，请前往手机解除应用锁”。另外，提醒信息可以以悬浮窗或者卡片的形式显示在PC上，本申请实施例对提醒信息的显示形式不作限定。
其次，源端设备与目标设备可以通过WLAN或者WI-FI P2P等方式建立投屏连接，投屏连接的建立具体可参考上文源端设备和目标设备建立通信连接的方式，在此不予赘述。为了方便描述，以下实施例以WI-FI P2P方式建立投屏连接。
在本申请的一些实施例中，PC检测到用户选中消息卡片702之后，判断该消息卡片702对应的源端设备(也即，手机)与目标设备(也即，PC)的投屏通信通道(Wi-Fi P2P通道)是否已经被其他业务占用。其中，源端设备与目标设备通过Wi-Fi P2P通道建立投屏连接进行投屏业务的数据传输，从而实现两个或多个电子设备之间内容的镜像。当源端设备与目标设备已经建立了基于WI-FI P2P的其他业务，例如PC协同、设备分享等，那么源端设备与对端设备进行投屏业务的通道(也即，投屏通道)就被占用，导致源端设备无法基于WI-FI P2P向目标设备进行投屏。
例如，PC确定与手机之间的投屏通道(Wi-Fi P2P)被占用，启动投屏失败，显示提醒信息2，该提醒信息2示例性的提示用户“投屏连接冲突，请断开与其他设备之间的连接后再试”，以此提示用户前往源端设备(手机)或目标设备(PC)断开与其他设备之间的投屏连接。其中，提醒信息可以以悬浮窗或者卡片的形式显示在PC上，本申请实施例对提醒信息的显示形式不作限定。
再次，在本申请的一些实施例中，在PC检测到用户选中消息卡片702之后，判断该消息卡片对应的源端设备(也即，手机)的网络状态是否满足投屏要求。其中，网络状态包括流表项匹配状态、数据包执行端口负载状态、丢失/转发/收到数据包的总数量或数据包TTR失效等类似情况，只要是网络中可能出现的任何网络状态都包括在内，在此不作限定。可以理解，网络状态会影响投屏的服务质量，PC可选的存在满足投屏条件的最低网络状态阈值。如果PC确定当前网络状态低于这一最低网络状态阈值，则认为当前网络状态无法满足投屏要求，导致投屏失败。
在PC确定当前网络状态不满足投屏要求，启动投屏失败，显示提醒信息3。该提醒信息3可以提醒用户待网络稳定后再尝试触发源端设备进行投屏。示例性的提示用户“网络不稳定，请重试”，以此提示用户调整和优化网络状态。
可以理解的是，以上三种场景仅作为示例，不作为对本申请实施例的限定。PC判断对应应用是否被应用锁保护、投屏通道是否被占用、网络状态是否满足投屏要求不作为本申请实施例必须具备的操作，同时判断顺序也不作限定。
在一些实施例中，请返回参阅图18，可以在图18的步骤S5之后增加判断是否存在可能导致投屏失败的情况，可以包括但不限于以上三种情况。PC响应于用户点击消息卡片702的操作，在各个判断节点依次判断是否设置了应用锁、投屏通道是否被占用、网络状态是否满足投屏要求，示例性的，只要有一个判断结果触发上述启动投屏失败的场景，此轮判断终止。用户按照提示调整之后，可以再次点击消息通知控件，触发新的一轮判断。当然，在一些实施例中，判断机制的判断顺序可以调整，并且可以在某一判断节点触发启动投屏失败之后，判断流程停留在该判断节点，并持续检测是否满足进入下一判断节点的条件。用户按照提示进行调整，当满足进入下一判断节点的条件之后，判断流程自动进入下一判断节点。
可以理解地，PC可以先检验对应的应用是否受应用锁保护(可称为应用锁检测)，再检验用于投屏的通信通道是否被占用(可称为通道检测)，最后检验网络条件是否满足要求(简称网络检测)。实际上，本申请实施例中，不限定上述多项检测的先后顺序，也就是，在一些实施例中，PC还可以先检验用于投屏的通信通道是否被占用，再检验对应的应用是否受应用锁保护，最后检测网络条件。
在其他可能的实施例中，PC还可以同步进行应用锁检测、通道检测和网络检测，在得到的检测结果指示至少一项检测项目触发投屏失败时，显示对应的提醒信息。例如，如果得到的测结果指示应用受应用锁保护、通信通道被占用，PC会同步显示提醒信息1与提醒信息2。
另外，在PC桌面701显示消息卡片702时，如果PC确定用户选中消息卡片702，那么PC也需要针对手机进行应用锁检测、通道检测及网络检测。在检测结果指示上述检测项目均未触发投屏失败的情况下，PC确定手机已具备投屏的条件，这样，PC可以在投屏窗口703显示应用界面704。在投屏失败的情况下，PC可以显示对应的提醒信息。
上文详细介绍了本申请实施例提供的投屏的方法及电子设备的示例。可以理解的是，电子设备为了实现上述功能，其包含了执行各个功能相应的硬件和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以结合实施例对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对电子设备进行功能模块的划分，例如，可以对应各个功能划分为各个功能模块，例如检测单元、处理单元、显示单元等，也可以将两个或两个以上的功能集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
需要说明的是，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
本实施例提供的电子设备，用于执行上述投屏方法，因此可以达到与上述实现方法相同的效果。
在采用集成的单元的情况下，电子设备还可以包括处理模块、存储模块和通信模块。其中，处理模块可以用于对电子设备的动作进行控制管理。存储模块可以用于支持电子设备执行存储程序代码和数据等。通信模块，可以用于支持电子设备与其他设备的通信。
其中，处理模块可以是处理器或控制器。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，数字信号处理(digital signal processing，DSP)和微处理器的组合等等。存储模块可以是存储器。通信模块具体可以为射频电路、蓝牙芯片、Wi-Fi芯片等与其他电子设备交互的设备。
在一个实施例中，当处理模块为处理器，存储模块为存储器时，本实施例所涉及的电子设备可以为具有图9所示结构的设备。
本申请实施例还提供了一种计算机可读存储介质，计算机可读存储介质中存储了 计算机程序，当计算机程序被处理器执行时，使得处理器执行上述任一实施例的投屏方法。
本申请实施例还提供了一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得计算机执行上述相关步骤，以实现上述实施例中的投屏方法。
另外，本申请的实施例还提供一种装置，这个装置具体可以是芯片，组件或模块，该装置可包括相连的处理器和存储器；其中，存储器用于存储计算机执行指令，当装置运行时，处理器可执行存储器存储的计算机执行指令，以使芯片执行上述各方法实施例中的来电显示方法。其中，本实施例提供的电子设备、计算机可读存储介质、计算机程序产品或芯片均用于执行上文所提供的对应的方法，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
通过以上实施方式的描述，所属领域的技术人员可以了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (36)

  1. 一种投屏系统，其特征在于，所述系统包括第一电子设备和第二电子设备，所述第一电子设备包括第一应用：
    所述第一电子设备用于接收所述第一应用的第一数据；
    所述第一电子设备用于向所述第二电子设备发送包括所述第一数据的第一消息；
    所述第二电子设备，用于响应于接收到所述第一消息，显示第一控件；
    所述第二电子设备用于在第一时间接收用户对所述第一控件的第一操作；
    所述第二电子设备，用于响应于所述第一操作，向所述第一电子设备发送投屏请求；
    所述第一电子设备，用于响应于接收到所述投屏请求，向所述第二电子设备发送所述第一应用的图像信息，所述第一应用的图像信息包括所述第一数据；
    所述第二电子设备，用于根据接收到的所述第一应用的图像信息显示窗口，所述第一应用的第一界面在第二时间被显示在所述窗口内，所述第一界面包括所述第一应用的图像信息；
    所述第二时间晚于所述第一时间，在所述第一时间和所述第二时间的时间区间，所述第一电子设备处于锁屏或者息屏状态。
  2. 根据权利要求1所述的系统，其特征在于，
    在第一时间区间，所述第一电子设备还用于显示主界面；
    所述第一电子设备还用于接收用户的第二操作；
    响应于接收到所述第二操作，所述第一电子设备在第二时间区间处于锁屏或者息屏状态，所述第二时间区间的开始时间为所述第一时间区间的结束时间；
    所述第一电子设备还用于接收用户的第三操作；
    响应于接收到所述第三操作，所述第一电子设备在第三时间区间显示所述主界面，所述第三时间区间的开始时间为所述第二时间区间的结束时间；
    在所述第三时间区间，所述第二电子设备还用于在所述窗口显示所述第一界面。
  3. 根据权利要求1或2所述的系统，其特征在于，所述第二电子设备不包括所述第一应用。
  4. 根据权利要求1-3任一项所述的系统，其特征在于，所述第一电子设备还包括第二应用，所述第一应用和所述第二应用不同；
    所述第一电子设备还用于接收用户打开所述第二应用的操作；
    响应于所述用户打开所述第二应用的操作，所述第一电子设备还用于在第四时间区间显示所述第二应用的第二界面，所述第四时间区间的开始时间为所述第三时间区间的结束时间；
    在所述第四时间区间，所述第二电子设备还用于在所述窗口显示所述第一界面。
  5. 根据权利要求1-4任一项所述的系统，其特征在于，响应于接收到用户在所述第一界面的操作，所述第二电子设备还用于在所述窗口显示所述第一应用的第三界面，所述第三界面与所述第一界面不同。
  6. 根据权利要求1-3任一项所述的系统，其特征在于，所述第一电子设备还包括第三应用，所述第三应用与所述第一应用不同；
    所述第一电子设备还用于接收所述第三应用的第二数据；
    所述第一电子设备还用于向所述第二电子设备发送包括所述第二数据的第二消息；
    响应于接收到所述第二消息，所述第二电子设备还用于显示第二控件；
    所述第二电子设备还用于在第三时间接收用户对所述第二控件的第四操作；
    响应于所述第四操作，所述第二电子设备还用于向所述第一电子设备发送投屏请求；
    响应于接收到所述投屏请求，所述第一电子设备向所述第二电子设备发送所述第三应用的图像信息，所述第三应用的图像信息包括所述第二数据；
    响应于接收到所述第三应用的图像信息，在第四时间，所述第二电子设备在所述窗口显示所述第三应用的第四界面，所述第四界面包括所述第三应用的图像信息；
    所述第四时间晚于所述第三时间，在所述第三时间和所述第四时间的时间区间，所述第一电子设备处于锁屏或者息屏状态。
  7. 根据权利要求1-6任一项所述的系统，其特征在于，
    响应于接收到用户在所述第一界面对所述第一消息的操作，所述第二电子设备还用于向所述第一电子设备发送指示所述第一电子设备执行所述操作的指令。
  8. 根据权利要求1-7任一项所述的系统，其特征在于，所述第二电子设备还用于在桌面显示所述第一控件；或者，所述第二电子设备还用于在消息列表窗口显示所述第一控件，所述第一控件为所述第一消息对应的消息卡片。
  9. 根据权利要求1-8任一项所述的系统，其特征在于，所述第一控件包括所述第一消息的内容。
  10. 根据权利要求1-9任一项所述的系统，其特征在于，所述第一操作为用户点击所述第一控件的操作；或者，所述第一操作为用户拖动所述第一控件的操作。
  11. 根据权利要求1-10任一项所述的系统，其特征在于，响应于接收到所述第一操作，所述第二电子设备还用于在所述窗口显示安全认证界面；
    接收到用户在所述安全认证界面输入认证信息的操作，所述第二电子设备还用于向所述第一电子设备发送所述认证信息；
    接收到所述认证信息，所述第一电子设备还用于确定所述认证信息满足预设值，向所述第二电子设备发送认证成功的消息；
    响应于接收到所述认证成功的消息，所述第二电子设备向所述第一电子设备发送投屏的请求。
  12. 根据权利要求1-11任一项所述的系统，其特征在于，所述第一电子设备用于通过第一通信连接向所述第二电子设备发送所述第一消息；
    所述第二电子设备用于通过第二通信连接向所述第一电子设备发送所述投屏请求；
    所述第一电子设备用于通过所述第二通信连接向所述第二电子设备发送所述第一应用的图像信息；
    所述第一通信连接与所述第二通信连接不同。
  13. 根据权利要求11所述的系统，其特征在于，所述第二设备还用于通过第三通信连接向所述第一电子设备发送所述认证信息；
    所述第一电子设备还用于通过所述第三通信连接向所述第二电子设备发送所述认 证成功的消息；
    所述第三通信连接与所述第一通信连接不同，与所述第二通信连接不同。
  14. 一种投屏方法，其特征在于，所述方法应用于第一电子设备，所述第一电子设备包括第一应用：
    所述第一电子设备接收所述第一应用的第一数据；
    所述第一电子设备通过第一通信连接向第二电子设备发送包括所述第一数据的第一消息；
    响应于在第一时间接收到所述第二电子设备通过第二通信连接发送的投屏请求，所述第一电子设备通过所述第二通信连接向所述第二电子设备发送所述第一应用的图像信息，以使得在第二时间，所述第二电子设备在窗口显示第一界面；
    其中，所述第一应用的图像信息包括所述第一数据，所述第一界面包括所述第一应用的图像信息；
    所述第二时间晚于所述第一时间，在所述第一时间和所述第二时间的时间区间，所述第一电子设备处于锁屏或者息屏状态。
  15. 根据权利要求14所述的方法，其特征在于，在所述第一电子设备接收到所述投屏请求之后，所述方法包括：
    所述第一电子设备将所述第一应用的一个或多个活动设置为OnResume状态；
    所述第一电子设备控制所述第一应用的一个或多个活动不进入休眠流程。
  16. 根据权利要求15所述的方法，其特征在于，所述第一电子设备包括异步处理器、活动记录器、根窗口容器，在所述第一时间区间，所述第一电子设备接收到所述投屏请求之后，所述方法还包括：
    所述异步处理器将所述第一应用的一个或多个活动标记第一标签；
    所述异步处理器发送启动带有所述第一标签的活动的指令；
    接收到所述启动带有所述第一标签的活动的指令，所述活动记录器确定所述第一应用的一个或多个活动带有所述第一标签，将所述第一应用的一个或多个活动设置为OnResume状态；
    所述根窗口容器确定所述第一应用的一个或多个活动带有所述第一标签，控制所述第一应用的一个或多个活动不进入休眠流程。
  17. 根据权利要求14-16任一项所述的方法，其特征在于，所述第一电子设备包括虚拟显示区域和可见显示区域，所述第一电子设备向所述第二电子设备发送所述第一界面之前，所述方法还包括：
    所述第一电子设备将所述虚拟显示区域标记第二标签；
    所述第一电子设备在所述虚拟显示区域绘制第一图像，所述第一图像为所述第一消息对应的所述第一应用的界面；
    所述第一电子设备在所述可见显示区域绘制第二图像，所述第二图像为所述第一电子设备处于所述锁屏或者息屏状态时对应的界面；
    所述第一电子设备确定所述虚拟显示区域带有所述第二标签，获取所述第一图像得到第一界面，所述第一界面不包括所述第二图像。
  18. 根据权利要求14-17任一项所述的方法，其特征在于，所述方法还包括：
    在第一时间区间，所述第一电子设备显示主界面；
    所述第一电子设备接收用户的第二操作；
    响应于接收到所述第二操作，所述第一电子设备在第二时间区间处于锁屏或者息屏状态，所述第二时间区间的开始时间为所述第一时间区间的结束时间；
    所述第一电子设备接收用户的第三操作；
    响应于接收到所述第三操作，所述第一电子设备在所述可见显示区域绘制主界面，在第三时间区间显示所述主界面，所述第三时间区间的开始时间为所述第一时间区间的结束时间；
    在所述第三时间区间，所述第一电子设备确定所述虚拟显示区域带有所述第二标签，不获取所述可见显示区域的主界面发送给所述第二电子设备，以使得所述第二电子设备在所述窗口显示所述第一界面。
  19. 根据权利要求14-18任一项所述的方法，其特征在于，所述第一电子设备在接收到所述启动投屏请求之前，所述方法还包括：
    所述第一电子设备通过第三通信连接接收到所述第二电子设备发送的认证信息；
    所述第一电子设备确定所述认证信息满足预设值，通过第三通信连接向所述第二电子设备发送认证成功的消息，以使得所述第二电子设备向所述第一电子设备发送投屏请求。
  20. 根据权利要求14-19任一项所述的方法，其特征在于，所述方法包括：
    所述第一电子设备接收到所述第二电子设备发送的指令，执行用户在所述第一界面对所述第一消息的操作。
  21. 一种投屏方法，其特征在于，所述方法应用于第二电子设备，所述方法包括：
    所述第二电子设备接收到第一电子设备通过第一通信连接发送的第一消息，所述第一消息包括所述第一电子设备的第一应用的第一数据；
    所述第二电子设备响应于接收到所述第一消息，显示第一控件；
    所述第二电子设备在第一时间接收到用户对所述第一控件的第一操作；
    所述第二电子设备响应于所述第一操作，通过第二通信连接向所述第一电子设备发送投屏请求；
    所述第二电子设备接收所述第一电子设备通过所述第二通信连接发送的所述第一应用的图像信息，所述第一应用的图像信息包括所述第一数据；
    所述第二电子设备根据所述第一应用的图像信息显示窗口，所述第一应用的第一界面在第二时间被显示在所述窗口内，所述第一界面包括所述第一应用的图像信息；
    所述第二时间晚于所述第一时间，在所述第一时间和所述第二时间的时间区间，所述第一电子设备处于锁屏或者息屏状态。
  22. 根据权利要求21所述的方法，其特征在于，所述方法还包括：
    在第一时间区间，所述第二电子设备在所述窗口显示所述第一界面，所述第一时间区间为所述第一电子设备显示主界面的时间区间，所述第一时间区间的开始时间晚于所述第二时间；
    在第二时间区间，所述第二电子设备在所述窗口显示所述第一界面，所述第二时间区间为所述第一电子设备显示第二应用的第二界面的时间区间，所述第二时间区间 的开始时间为所述第一时间区间的结束时间。
  23. 根据权利要求21或22所述的方法，其特征在于，所述方法包括：
    所述第二电子设备不包括所述第一应用。
  24. 根据权利要求21-23任一项所述的方法，其特征在于，所述方法包括：
    响应于接收到用户在所述第一界面的操作，所述第二电子设备还用于在所述窗口显示所述第一应用的第三界面，所述第三界面与所述第一界面不同。
  25. 根据权利要求21-24任一项所述的方法，其特征在于，所述方法还包括：
    所述第二电子设备接收到第一电子设备通过第三通信连接发送的第二消息，所述第二消息包括所述第一电子设备的第三应用的第二数据；
    所述第二电子设备响应于接收到所述第二消息，显示第二控件；
    所述第二电子设备在第三时间接收到用户对所述第二控件的第二操作；
    所述第二电子设备响应于所述第二操作，通过第四通信连接向所述第一电子设备发送投屏请求；
    所述第二电子设备接收所述第一电子设备通过所述第四通信连接发送的所述第三应用的图像信息，所述第三应用的图像信息包括所述第二数据；
    所述第二电子设备根据所述第三应用的图像信息显示窗口，所述第三应用的第四界面在第四时间被显示在所述窗口内，所述第四界面包括所述第三应用的图像信息；
    所述第四时间晚于所述第三时间，在所述第三时间和所述第四时间的时间区间，所述第一电子设备处于锁屏或者息屏状态。
  26. 根据权利要求21-25任一项所述的方法，其特征在于，所述方法包括：
    响应于接收到用户在所述第一界面对所述第一消息的操作，所述第二电子设备向所述第一电子设备发送指示所述第一电子设备执行所述操作的指令。
  27. 根据权利要求21-26任一项所述的方法，其特征在于，所述方法包括：
    所述第二电子设备在桌面显示所述第一控件；或者，所述第二电子设备在消息列表窗口显示所述第一控件，所述第一控件为所述第一消息对应的消息卡片。
  28. 根据权利要求21-27任一项所述的方法，其特征在于，所述方法还包括：所述第一控件包括所述第一消息的内容。
  29. 根据权利要求21-28任一项所述的方法，其特征在于，所述方法还包括：所述第一操作为用户点击所述第一控件的操作；或者，所述第一操作为用户拖动所述第一控件的操作。
  30. 根据权利要求21-29任一项所述的方法，其特征在于，所述方法还包括：响应于接收到所述第一操作，所述第二电子设备在所述窗口显示安全认证界面；
    接收到用户在所述安全认证界面输入认证信息的操作，所述第二电子设备通过第三通信连接向所述第一电子设备发送所述认证信息；
    所述第二电子设备接收所述第一电子设备通过所述第三通信连接发送的认证成功的消息；响应于接收到所述认证成功的消息；
    所述第二电子设备接收到所述认证成功的消息，向所述第一电子设备发送投屏的请求。
  31. 根据权利要求30所述的方法，其特征在于，所述第二电子设备接收到所述第一 操作之后，所述方法还包括：
    所述第二电子设备确定所述第一应用首次在所述第二电子设备上投屏，且所述第一电子设备设置有安全认证，在所述投屏窗口显示所述安全认证界面。
  32. 根据权利要求30所述的方法，其特征在于，所述第二电子设备接收到所述第一操作之后，所述方法还包括：
    所述第二电子设备确定所述第一应用非首次在所述第二电子设备上投屏，且当前时间距离上一次认证成功的时间超过预设时长，所述第一电子设备设置有安全认证，在所述投屏窗口显示所述第一电子设备的所述安全认证界面。
  33. 根据权利要求21-32任一项所述的方法，其特征在于，所述第二电子设备接收到所述第一操作之后，所述方法还包括：
    所述第二电子设备确定所述第一应用设置有应用锁，所述第二电子设备不显示所述第一电子设备的投屏窗口。
  34. 一种电子设备，其特征在于，包括：
    一个或多个处理器；
    一个或多个存储器；
    所述一个或多个存储器存储有一个或者多个计算机程序，所述一个或者多个计算机程序包括指令，当所述指令被所述一个或多个处理器执行时，使得所述电子设备执行如权利要求14-20中任一项所述的方法。
  35. 一种电子设备，其特征在于，包括：
    一个或多个处理器；
    一个或多个存储器；
    所述一个或多个存储器存储有一个或者多个计算机程序，所述一个或者多个计算机程序包括指令，当所述指令被所述一个或多个处理器执行时，使得所述电子设备执行如权利要求21-33中任一项所述的方法。
  36. 一种计算机可读存储介质，其特征在于，包括计算机指令，当所述计算机指令在电子设备上运行时，使得所述电子设备执行如权利要求14-20或21-33中任一项所述的方法。
PCT/CN2023/118031 2022-10-19 2023-09-11 一种投屏系统和电子设备 WO2024082871A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211282319.9A CN117908814A (zh) 2022-10-19 2022-10-19 一种投屏系统和电子设备
CN202211282319.9 2022-10-19

Publications (1)

Publication Number Publication Date
WO2024082871A1 true WO2024082871A1 (zh) 2024-04-25

Family

ID=90682716

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/118031 WO2024082871A1 (zh) 2022-10-19 2023-09-11 一种投屏系统和电子设备

Country Status (2)

Country Link
CN (1) CN117908814A (zh)
WO (1) WO2024082871A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108259665A (zh) * 2017-12-29 2018-07-06 苏州车萝卜汽车电子科技有限公司 车内互动显示方法以及移动通信终端
CN112346685A (zh) * 2020-10-26 2021-02-09 努比亚技术有限公司 一种熄屏投屏控制方法、设备及计算机可读存储介质
WO2021164445A1 (zh) * 2020-02-19 2021-08-26 华为技术有限公司 一种通知处理方法、电子设备和系统
CN113741840A (zh) * 2020-09-10 2021-12-03 华为技术有限公司 多窗口投屏场景下的应用界面显示方法及电子设备
CN115048067A (zh) * 2021-02-26 2022-09-13 华为技术有限公司 一种投屏显示方法及电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108259665A (zh) * 2017-12-29 2018-07-06 苏州车萝卜汽车电子科技有限公司 车内互动显示方法以及移动通信终端
WO2021164445A1 (zh) * 2020-02-19 2021-08-26 华为技术有限公司 一种通知处理方法、电子设备和系统
CN113741840A (zh) * 2020-09-10 2021-12-03 华为技术有限公司 多窗口投屏场景下的应用界面显示方法及电子设备
CN112346685A (zh) * 2020-10-26 2021-02-09 努比亚技术有限公司 一种熄屏投屏控制方法、设备及计算机可读存储介质
CN115048067A (zh) * 2021-02-26 2022-09-13 华为技术有限公司 一种投屏显示方法及电子设备

Also Published As

Publication number Publication date
CN117908814A (zh) 2024-04-19

Similar Documents

Publication Publication Date Title
EP4024193A1 (en) Data transmission method and related devices
WO2021227770A1 (zh) 应用窗口显示方法和电子设备
WO2021057830A1 (zh) 一种信息处理方法及电子设备
WO2021129253A1 (zh) 显示多窗口的方法、电子设备和系统
US20220179455A1 (en) Display method and electronic device
US20240111473A1 (en) Distributed display method and terminal for application interface
US11314894B2 (en) Multi-android system implementation method, non-transitory computer-readable storage medium, and terminal
US20230422154A1 (en) Method for using cellular communication function, and related apparatus and system
CN114237779A (zh) 一种显示窗口的方法、切换窗口的方法、电子设备和系统
CN114286165A (zh) 一种显示设备、移动终端及投屏数据传输方法
WO2022028494A1 (zh) 一种多设备数据协作的方法及电子设备
WO2022127661A1 (zh) 应用共享方法、电子设备和存储介质
US20240086231A1 (en) Task migration system and method
WO2022089294A1 (zh) 一种设备间屏幕协同方法及设备
WO2021052488A1 (zh) 一种信息处理方法及电子设备
WO2024082871A1 (zh) 一种投屏系统和电子设备
WO2024093530A1 (zh) 一种投屏方法和电子设备
WO2023020048A1 (zh) 外屏显示方法及电子设备
WO2022188683A1 (zh) 灵活授权的访问控制方法、相关装置及系统
US20230362782A1 (en) Data Sharing Method, Electronic Device, and System
CN116743971B (zh) 一种同源投屏方法及电子设备
CN114286320A (zh) 一种显示设备、移动终端及蓝牙连接方法
CN111787157A (zh) 移动终端及其操作响应方法
CN111163220A (zh) 一种显示方法及通信终端
WO2023207218A1 (zh) 一种来电提醒系统、方法及电子设备