WO2024093530A1 - 一种投屏方法和电子设备 - Google Patents

一种投屏方法和电子设备 Download PDF

Info

Publication number
WO2024093530A1
WO2024093530A1 PCT/CN2023/117850 CN2023117850W WO2024093530A1 WO 2024093530 A1 WO2024093530 A1 WO 2024093530A1 CN 2023117850 W CN2023117850 W CN 2023117850W WO 2024093530 A1 WO2024093530 A1 WO 2024093530A1
Authority
WO
WIPO (PCT)
Prior art keywords
screen
application
display
display area
message
Prior art date
Application number
PCT/CN2023/117850
Other languages
English (en)
French (fr)
Inventor
王冬伟
Original Assignee
荣耀终端有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 荣耀终端有限公司 filed Critical 荣耀终端有限公司
Publication of WO2024093530A1 publication Critical patent/WO2024093530A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units

Definitions

  • the embodiments of the present application relate to the fields of communication technology and terminal technology, and in particular to a screen projection method and electronic device.
  • the collaborative work of multiple electronic devices brings convenience to users' lives and work.
  • a mobile phone receives a message, it can not only display the message notification on the mobile phone, but also transfer the message notification to the PC for display. This makes it convenient for users to view messages across devices and improves the efficiency of collaborative work between devices. Of course, it is not enough to simply transfer message notifications between devices. After users view the message notifications from their mobile phones on the PC, they often need to process the message notifications through the PC, which requires casting the phone screen to the PC.
  • An embodiment of the present application provides a screen projection method and an electronic device.
  • the second device can still normally display the application projection window provided by the first device.
  • an embodiment of the present application provides a screen projection method, which is applied to a first device and a second device, and the method includes: after the first device projects the screen to the second device, the second device displays a first window; when the first device is in a screen-on state, the second device displays first image information from the first device in the first window, and the first image information corresponds to an application interface of a first application in the first device; after the first device switches to a lock screen or screen-off state, the second device displays second image information from the first device in the first window, and the second image information also corresponds to the application interface of the first application in the first device.
  • the first device projects its screen to the second device when the screen is on. For example, heterogeneous screen projection is performed. After the heterogeneous screen projection is successful, the first device can draw the application interface of the first application into first image information, and send it to the second device, which is displayed in the first window by the second device. In this way, the user can use the first application installed in the first device by operating the second device. Afterwards, the first device can enter the screen-off or screen-lock state in response to the user's operation. The first device can draw the second image information based on the application interface of the first application, and send it to the second device, so that the second device can continue to display the application interface related to the first application in the first window.
  • first image information and second image information both correspond to the application interface of the first application, and can be image information drawn according to the application interface of the first application at different times, that is, the first image information and the second image information.
  • the contents displayed by the two image information may be the same or different.
  • the first device can continue to project the screen to the second device normally. At the same time, it also avoids excessive energy consumption caused by the long-term screen-on when the user is not using the first device, and prevents others from using the unlocked first device without the user's permission.
  • the method further includes: the first device configures the first application to not sleep during its life cycle, and configures the application interface of the first application to be allowed to be drawn in a first display area during its life cycle, where the first display area is an invisible display area in the first device; the first device sends second image information of the first application to the second device, instructing the second device to display the second image information in a first window, where the second image information is data drawn in the first display area.
  • the application interface of the first application is kept visible in the foreground, ensuring that after the first device turns off the screen or locks the screen, the user can continue to use the first application in the first device through the first window in the second device.
  • the method before the first device projects the screen to the second device, the method also includes: the first device receives a first message of the first application; the first device forwards the first message to the second device; the second device displays a first control, and the first control is used to display the first message; the second device sends a first request to the first device in response to the user's operation of the first control, and the first request includes a first identifier of the first application; the first device starts projecting the screen to the second device in response to the first request.
  • the method after initiating screen projection to the second device, the method also includes: the first device creates a first display area and starts a first application; after the application interface of the first application is drawn in the first display area, the first device generates first image information based on the image data in the first display area; the first device sends the first image information to the second device.
  • the method before launching the first application, the method also includes: the first device tags the first application with a first tag; before the first device configures the first application to not sleep during its life cycle, and configures the application interface of the first application to be allowed to be drawn in the first display area during its life cycle, the method also includes: the first device determines that the first application has a first tag.
  • the method before the first device switches to the lock screen or screen off state, the method also includes: the first device responds to the user's instruction to lock the screen or screen off operation, only controls the display screen of the first device to power off or only displays the lock screen layer on the display screen, and the service running state corresponding to the display screen remains unchanged.
  • the first display area is marked with a second label.
  • the method further includes: when the first device detects that the conditions for automatic screen off are met, determining that the first display area has the second label; the first device only controls the display screen of the first device to power off or only displays the lock screen layer on the display screen, and the service running status corresponding to the display screen remains unchanged.
  • the method before the first device switches to a lock screen or screen off state, the method further includes: the first device determines not to draw a lock screen layer in the first display area.
  • the method before the first device switches to a lock screen or screen off state, the method further includes: the first device determines that the first display area does not mirror content in a second display area, and the second display area is a display area visible in the first device.
  • the embodiment of the present application provides a screen projection method, which is applied to a first device, and the method comprises: the first device projects the screen to the second device; when the first device is in a screen-on state, the first device projects the screen to the second device; The second device sends first image information corresponding to the first application, instructing the second device to display the first image information on a first window; after the first device switches to a lock screen or screen-off state, the first device sends second image information corresponding to the first application to the second device, instructing the second device to display the second image information on the first window, and the first image information and the second image information are both application interfaces of the first application.
  • the method further includes: the first device configures the first application to not sleep during its life cycle, and configures the application interface of the first application to be allowed to be drawn in a first display area during its life cycle, and the first display area is an invisible display area in the first device; the first device generates the second image information based on the content drawn in the first display area.
  • the method before the first device projects the screen to the second device, the method also includes: the first device receives a first message of the first application; the first device forwards the first message to the second device; after receiving a first request sent by the second device, starting the screen projection to the second device, the first request is request information for screen projection for the first application, and the first request carries a first identifier of the first application.
  • the method before the first device sends the first image information corresponding to the first application to the second device, the method further includes: the first device creates a first display area and starts the first application;
  • the first device After the application interface of the first application is drawn in the first display area, the first device generates the first image information according to the image data in the first display area.
  • the method before launching the first application, the method also includes: the first device marks the first application with a first tag; before the first device configures the first application to not sleep during its life cycle, and configures the application interface of the first application to be allowed to be drawn in the first display area during its life cycle, the method also includes: the first device determines that the first application has the first tag.
  • the method before the first device switches to the lock screen or screen off state, the method also includes: the first device responds to the user's instruction to lock the screen or screen off operation, only controls the display screen of the first device to power off or only displays the lock screen layer on the display screen, and the service running state corresponding to the display screen remains unchanged.
  • the first display area is marked with a second label.
  • the method further includes: when the first device detects that a condition for automatic screen off is met, determining that the first display area has the second label; the first device only controls the display screen of the first device to power off or only displays the lock screen layer on the display screen, and the service running state corresponding to the display screen remains unchanged.
  • the method before the first device switches to a lock screen or screen off state, the method further includes: the first device determines not to draw a lock screen layer in the first display area.
  • the method before the first device switches to a lock screen or screen off state, the method further includes: the first device determines that the first display area does not mirror content in a second display area, and the second display area is a display area visible in the first device.
  • an embodiment of the present application provides an electronic device, the electronic device comprising one or more processors and a memory; the memory is coupled to the processor, the memory is used to store computer program code, the computer program code comprises computer instructions, when one or more processors execute the computer instructions, the one or more processors A processor is used to execute the method described in the first aspect and its possible embodiments; or, when one or more processors execute computer instructions, the one or more processors are used to execute the method described in the second aspect and its possible embodiments.
  • an embodiment of the present application provides a computer storage medium, comprising computer instructions, which, when executed on an electronic device, causes the electronic device to execute the method described in the first aspect and possible embodiments thereof; or, when executed on an electronic device, causes the electronic device to execute the method described in the second aspect and possible embodiments thereof.
  • FIG1 is a schematic diagram of a message flow scenario provided in an embodiment of the present application.
  • FIG. 2 is a schematic diagram of an interface display of a heterogeneous screen projection provided in an embodiment of the present application.
  • FIG3 is a schematic diagram of an interface display of a heterogeneous screen projection provided in an embodiment of the present application.
  • FIG4 is a schematic diagram of the principle of a heterogeneous screen projection provided in an embodiment of the present application.
  • Figure 5 is a schematic diagram of an interface display of a heterogeneous screen projection provided in an embodiment of the present application.
  • FIG6 is a schematic diagram of an interface display of a heterogeneous screen projection provided in an embodiment of the present application.
  • FIG. 7 is a flow chart of a judgment mechanism provided in an embodiment of the present application.
  • FIG8 is a schematic diagram of an interface display of a heterogeneous screen projection provided in an embodiment of the present application.
  • FIG. 9 is a software architecture diagram of an electronic device provided in an embodiment of the present application.
  • FIG. 10 is a diagram showing a specific implementation process of a screen projection method provided in an embodiment of the present application.
  • FIG. 11 is a specific implementation process diagram of another screen projection method provided in an embodiment of the present application.
  • FIG. 12 is a specific implementation process diagram of another screen projection method provided in an embodiment of the present application.
  • FIG. 13 is a specific implementation process diagram of another screen projection method provided in an embodiment of the present application.
  • FIG. 14 is a specific implementation process diagram of another screen projection method provided in an embodiment of the present application.
  • FIG. 15 is a specific implementation process diagram of another screen projection method provided in an embodiment of the present application.
  • FIG. 16 is a schematic diagram of a principle provided in an embodiment of the present application.
  • FIG. 17 is another schematic diagram of a principle provided in an embodiment of the present application.
  • an embodiment of the present application provides a system, including multiple electronic devices, such as device 1, device 2, and device 3.
  • the electronic devices in the above system ie, device 1, device 2, device 3 may belong to the same local area network.
  • the local area network can be a wireless communication network, such as wireless local area networks (WLAN), wireless fidelity point to point (Wi-Fi P2P), Bluetooth network, zigbee network, infrared technology (IR) or near field communication (NFC) network.
  • WLAN wireless local area networks
  • Wi-Fi P2P wireless fidelity point to point
  • Bluetooth Bluetooth network
  • zigbee network infrared technology
  • NFC near field communication
  • the above-mentioned local area network can also be a wired communication network.
  • VGA video graphics array
  • DVI digital visual interface
  • HDMI high definition multimedia interface
  • the above-mentioned local area network can also be a remote network.
  • the system composed of devices 1, 2 and 3 can also be called a trust ring.
  • the process of mutual trust between devices can refer to related technologies. For example, it can be judged based on one or more of whether the devices have logged in the same system account, whether they have been authorized, and whether near field communication is used. The details are not repeated here.
  • multiple electronic devices can securely share data, and the shared data includes but is not limited to message reminders, message records, user information of other electronic devices, device type, running status of each application, identification of each application, synchronization of identification of recently used applications, etc.
  • device 1 when device 1 receives a message notification, it can not only display the message notification on device 1, but also transfer the message notification to device 2 and/or device 3 for display. This makes it convenient for users to view messages across devices and improves the efficiency of collaborative work between devices.
  • device 2 can also transfer the received message notification to device 1 and/or device 3
  • device 3 can also transfer the received message notification to device 1 and/or device 2.
  • the screen projection scenario includes a source device and a target device.
  • the source device and the target device use a certain protocol to display the content on the source device on the target device, that is, the source device projects the screen to the target device.
  • Common protocols include DLNA (Digital Living Network Alliance) protocol, Chromecast protocol, Miracast protocol, Airplay protocol, etc.
  • the source device can be a mobile phone, tablet computer, PC, watch, etc.
  • the target device can be a tablet computer, PC, TV, etc.
  • the source device and the target device may also have other names, for example, the source device may be called an active screen projection device, and the target device may be called a passive screen projection device; or the source device may be called a first computer.
  • the sub-device, the target device can be called a second electronic device, etc., and this application does not limit this.
  • the source device as a mobile phone and the target device as a PC as an example.
  • the mobile phone and the PC belong to the same local area network and complete mutual trust recognition, the mobile phone and the PC belong to the same trust ring, that is, a communication channel for message flow can be established between the mobile phone and the PC.
  • multiple types of applications may be installed in a mobile phone.
  • application messages pushed by various applications may be received.
  • the application messages come from a message server corresponding to the application.
  • a social application may be installed in the mobile phone, and the mobile phone may receive application messages pushed by the social application, such as a message “Are you going for a run today?” sent by Xiao Ming, a contact in the social application.
  • the mobile phone After receiving an application message pushed by an application, if the mobile phone is unlocked and the screen is on, the mobile phone can display a prompt message on the current interface. For example, when receiving an application message pushed by a social application, if the current interface of the mobile phone is the main interface 101, a notification bubble 102 is displayed in the main interface 101, and the notification bubble 102 is used to display the application message received by the mobile phone.
  • the mobile phone can transfer the above application message (for example, "Are you going for a run today?" pushed by a social application) to the PC through the communication channel used for transferring messages between the mobile phone and the PC.
  • the above application message for example, "Are you going for a run today?" pushed by a social application
  • a prompt message for the above application message can be displayed.
  • the above prompt message can be displayed on the desktop of the PC in the form of a message notification control.
  • the above message notification control can be a floating window, a message bubble, a message card, etc.
  • the relevant content can be that the application message comes from a mobile phone, or that the application message comes from a social application, or that the main content of the application message, etc. It can be understood that in the embodiment of the present application, there is no specific limitation on the display position of the message card 104.
  • the message card 104 displayed on the desktop 103 can remind the user to pay attention to the application message currently received by the mobile phone. In this way, even if the user is not currently using the mobile phone directly or the mobile phone is not around, the user can still understand the brief content of the application message to avoid missing the message.
  • the message card 104 may have a display time limit.
  • the display time limit is preset to time 1, and the PC starts to display the message card 104 in response to the application message forwarded by the mobile phone.
  • the PC may cancel the display of the message card 104.
  • the PC detects that the user is in contact with the display area 1 in the touch screen, and the PC determines that the user indicates to select the message card 104.
  • the display area 1 is the area on the touch screen where the message card 104 is displayed.
  • the PC is connected to a mouse, it is detected that the cursor corresponding to the mouse is located in the display area 1, and a confirmation instruction is input, and the PC determines that the user indicates to select the message card 104.
  • the user can process the application message by operating the PC, or can choose to process the application message by directly operating the mobile phone.
  • the method of directly operating the mobile phone to process the application message can refer to the relevant technology.
  • the mobile phone in response to the user clicking the notification bubble 102, the mobile phone can run the social application in the foreground and display the application interface containing the above social message. In this way, the user can reply to the application message by operating on the above application interface.
  • the mobile phone responds to the user's instruction to start the social application and displays the application interface of the social application. In this way, the user can By operating on the application interface provided by the social application, the above application messages can be viewed or replied to.
  • choosing to operate the PC to process application messages will trigger screen projection between the mobile phone and the PC.
  • the user can select the message card 104 on the PC.
  • the PC can remotely control the mobile phone to run the social application.
  • the social application enters the running state, the PC can display the projection window on the desktop 103.
  • the user can control the mobile phone through the projection window in the PC, such as operating the mobile phone to process the above-mentioned application messages.
  • the above-mentioned screen projection window can be used to synchronously display the current interface of the mobile phone.
  • the screen projection window also displays the application interface, which is the same-source screen projection.
  • the current interface of the mobile phone can also be displayed asynchronously.
  • the mobile phone displays the main interface
  • the screen projection window displays the application interface, which is a different-source screen projection.
  • heterogeneous screen projection can be realized through data interaction between the PC and the mobile phone. As shown in FIG2 , after the PC determines that the user has selected the message card 104, the heterogeneous screen projection is started, the main interface 101 is still displayed on the display screen of the mobile phone, and the projection window 201 on the display screen of the PC displays the application interface 202 corresponding to the above application message.
  • the content displayed in the projection window will not change synchronously with the content displayed on the mobile phone display.
  • the mobile phone display switches to display information application interface 303, and the projection window 201 on the PC display still displays the application interface 201 corresponding to the above application message.
  • Figure 4 is a schematic diagram of the principle of heterogeneous screen projection in Figures 2 and 3 above.
  • the display information on the target device is not synchronized with the display information on the foreground of the source device.
  • the window of application B is displayed on the source device.
  • the window of application B is displayed in the created virtual display area (VirtualDisplay).
  • the content of the virtual display area (such as the first display area) is invisible to the user and can be used for projection to the target device for display.
  • the source device also includes a user-visible display area (such as the second display area).
  • the user-visible display area displays the window of application A, that is, the source device displays the window of application A in the foreground.
  • the user-visible area on the source device is switched from the window of application A to the window of application C, that is, the source device displays the window of application C in the foreground.
  • the window of application B is still displayed on the target device.
  • the heterogeneous screen projection method has better isolation.
  • the heterogeneous screen projection method can provide users with independent control screens (i.e., the display screen of the source device and the display screen of the target device) to handle different applications.
  • the source device operates application A, or switches the window of application A to the window of application C, and the user can still operate application B on the target device. Therefore, heterogeneous screen projection releases the control of the display information of the source device over the display information of the target device, thereby enhancing the reusability of multiple devices and allowing users to operate different applications on different devices through screen projection.
  • the content displayed in the projection window changes synchronously with the content displayed on the mobile phone display.
  • the heterogeneous screen projection scenario is described as an example.
  • the mobile phone is also in a bright screen and unlocked state.
  • the user interacts with the social application in the mobile phone through the projection window in the PC. That is, the user only needs to operate the PC to use both the PC and the mobile phone at the same time.
  • the mobile phone is in a bright screen and unlocked state for a long time. In this way, it not only consumes power, but also easily causes privacy leakage.
  • the user often chooses to click the lock screen button to instruct the mobile phone to enter the screen-off state.
  • the mobile phone does not detect user operation for a long time, it will automatically enter the lock screen state.
  • the mobile phone if it does not detect user operation for a long time, it will automatically enter the lock screen state.
  • the mobile phone if it does not detect user operation for a long time, it will automatically enter the lock screen state.
  • the mobile phone if it detect user operation for a long time, it will automatically enter the lock screen state.
  • the mobile phone if the mobile phone as the source device enters the screen-off state, it will directly affect the display of the projection window in the PC.
  • the display screen of the mobile phone will be black, and at the same time, the projection window in the PC will also be black, which will directly affect the user's remote control of the mobile phone through the PC.
  • the above-mentioned black screen refers to a state where the display screen is not lit or only displays a screen saver image.
  • the PC displays the projection window while the phone is also in the screen-on and unlocked state. If the phone receives the user's instruction to lock the screen, as shown in Figure 6, the phone enters the lock screen state and displays the lock screen layer. At the same time, the projection window in the PC will also display the lock screen layer, which will directly affect the user's remote control of the phone through the PC.
  • the embodiment of the present application provides a screen projection method and an electronic device to provide users with a convenient and efficient screen projection experience.
  • the PC after receiving the application message transferred from the mobile phone, the PC can display the corresponding message card 104 on the main interface 103.
  • the PC can also receive one or more application messages transferred from any device in the same trust ring.
  • the PC can respond to user operations and display an information list window, which can centrally display received message notifications, or centrally display received and unprocessed message notifications.
  • the above-mentioned message notifications can also be displayed in the form of message cards in the information list window.
  • an information list window may be displayed on the main interface.
  • the information list window may display application messages from different devices, etc. It is understandable that the application messages displayed in the information list window are all information recently received by the PC from other devices in the trust ring.
  • the PC can trigger the device corresponding to the message card to project the screen.
  • the application message displayed on message card a comes from a mobile phone in the trusted ring.
  • the mobile phone can be triggered to project the screen to the PC.
  • the projection may fail.
  • the projection fails.
  • the projection fails.
  • the projection fails.
  • the channel used for projection is occupied by other devices.
  • the projection fails.
  • the projection fails.
  • application lock refers to the protection set by the user of the source device for a certain application in order to protect the security or privacy of the application.
  • the user needs to enter a pre-set password or fingerprint to open the application.
  • the source device has an optional additional layer of protection for application a (for example, the application corresponding to the message card selected by the user on the PC).
  • the source device sets an application lock for application a, it means that the owner of the source device wants to enter a password or fingerprint to unlock application a every time it is accessed. In this case, it is not suitable for such applications with strong privacy protection to directly start screen projection on the target device when the source device is in the locked screen state, which goes against the original intention of the user to set the application lock.
  • the PC after the PC detects that the user has selected a message card, it sends a query message 1 to the device (e.g., a mobile phone) corresponding to the message card.
  • the query message 1 is used to query whether the mobile phone has an application lock configured for the message application. If the mobile phone has an application lock configured for the message application, the mobile phone will send feedback message 1 to the PC.
  • the feedback message 1 indicates that the message application is currently protected by an application lock.
  • the PC can display a reminder message 1, which is used to remind the user that the application corresponding to the selected message card is protected by an application lock.
  • the reminder message 1 can be displayed on the PC in the form of a floating window or a card.
  • the embodiment of the present application does not limit the display form of the reminder message.
  • the source device and the target device can establish a screen projection connection through WLAN or WI-FI P2P.
  • the establishment of the screen projection connection can refer to the above-mentioned method of establishing a communication connection between the source device and the target device, which will not be repeated here.
  • the following embodiment establishes a screen projection connection in a WI-FI P2P manner.
  • the PC after the PC detects that the user has selected a message card, it determines whether the screen projection communication channel (Wi-Fi P2P channel) between the source device (i.e., mobile phone) and the target device (i.e., PC) corresponding to the message card has been occupied by other services. Among them, the source device and the target device establish a screen projection connection through the screen projection channel for data transmission, thereby realizing content mirroring between two or more electronic devices.
  • Wi-Fi P2P channel Wi-Fi P2P channel
  • the screen projection channel corresponding to the source device and the opposite device is occupied, resulting in the source device being unable to project the screen to the target device based on WI-FI P2P.
  • the PC determines that the screen projection channel (Wi-Fi P2P) between the PC and the mobile phone is occupied, and the screen projection fails to start, and the reminder message 2 is displayed.
  • the reminder message 2 exemplarily prompts the user that "screen projection connection conflicts, please disconnect the connection with other devices and try again", thereby prompting the user to go to the source device (mobile phone) or the target device (PC) to disconnect the screen projection connection with other devices.
  • the reminder message 2 can be displayed on the PC in the form of a floating window or a card, and the embodiment of the present application does not limit the display form of the reminder message.
  • the PC determines whether the network status of the source device (ie, the mobile phone) corresponding to the message card meets the screen projection requirements.
  • the network status includes the flow table matching status, the data packet execution port load status, the total number of lost/forwarded/received data packets or the failure of the data packet TTR and other similar situations. Any network status that may occur in the network is included and is not limited here. It can be understood that the network status will affect the service quality of the screen projection, and the PC can optionally have a minimum network status threshold that meets the screen projection conditions. If the PC determines that the current network status is lower than this minimum network status threshold, it is considered that the current network status cannot meet the screen projection requirements, resulting in a screen projection failure.
  • the screen projection fails to start, and a reminder message 3 is displayed.
  • the reminder message 3 can remind the user to try to trigger the source device to cast the screen after the network is stable.
  • the user is prompted with an exemplary message "The network is unstable, please try again" to prompt the user to adjust and optimize the network status.
  • the judgment order of the judgment mechanism can be adjusted, and after a judgment node triggers a failure to start the screen projection, the judgment process can stay at the judgment node and continue to detect whether the conditions for entering the next judgment node are met.
  • the user adjusts according to the prompts, and when the conditions for entering the next judgment node are met, the judgment process automatically enters the next judgment node.
  • Figure 7 illustrates that after the PC detects that the user has selected a message card, the PC can first check whether the corresponding application is protected by an application lock (which can be called application lock detection), then check whether the communication channel used for screen projection is occupied (which can be called channel detection), and finally check whether the network conditions meet the requirements (referred to as network detection).
  • application lock detection which can be called application lock detection
  • channel detection which can be called channel detection
  • network detection the order of the above-mentioned multiple tests is not limited, that is, in some embodiments, the PC can also first check whether the communication channel used for screen projection is occupied, then check whether the corresponding application is protected by an application lock, and finally detect the network conditions.
  • the PC can also perform application lock detection, channel detection, and network detection simultaneously, and display corresponding reminder information when the obtained detection results indicate that at least one detection item fails to trigger screen projection. For example, if the obtained test results indicate that the application is protected by the application lock and the communication channel is occupied, the PC will simultaneously display reminder information 1 and reminder information 2.
  • the PC determines that the mobile phone has the conditions for screen projection, so that, as shown in Figure 8, the PC can display a screen projection window 201, which is used to display the application interface of the social application in the mobile phone. In the case of screen projection failure, the PC can display a corresponding reminder message.
  • the source device can enable a distributed authentication function. That is, when the source device (such as a mobile phone) enables the distributed authentication function and is configured with an unlock password, if the target device (PC) passes the aforementioned tests (such as application lock detection, communication channel detection, and network detection) and determines that the source device has the conditions for screen projection, then the target device can display the screen projection window, and the screen projection window at this time is used to display the unlock verification interface.
  • the above unlock password can be an authentication password used to unlock the mobile phone screen.
  • the PC can display the screen projection window 201.
  • the above-mentioned screen projection window 201 can first display the unlock verification interface.
  • the screen projection window 201 is controlled to switch to display the "prompt waiting information".
  • the mobile phone responds to the instructions of the PC, starts the social application, and draws the application interface of the social application in the virtual interface, and obtains the drawing parameters of each display object in the virtual interface.
  • the above-mentioned virtual interface is a service for the PC, which is not visible on the display screen of the mobile phone.
  • the above-mentioned drawing parameters are related parameters for rendering the interface, which can also be called display information in some examples.
  • the mobile phone After the mobile phone completes drawing the above application interface, it sends the corresponding display information to the PC, instructing the PC to render the above application interface in the projection window 201.
  • the projection window 201 switches to display the application interface of the social application (e.g., the application interface containing the application message corresponding to the message card 104). That is, the mobile phone displays the virtual world.
  • the screen is projected to the projection window 201 of the PC. That is, after the PC receives the mobile phone unlocking password entered by the user and the unlocking password is verified, the mobile phone starts to project the screen to the PC from a different source.
  • the target device when the source device projects the screen to the target device for the first time, if the source device has not set an unlock password, the target device does not need to display the unlock verification interface in the projection window, but instructs the source device to directly project the screen from another source. In this scenario, the target device can directly display the "prompt waiting information". After the source device completes drawing the application interface to be projected, the target device controls the projection window to display the application interface to be projected.
  • the source device projects the screen to the target device for the first time
  • the source device has an unlock password set, that is, the user needs to perform password authentication before directly using the source device.
  • the target device determines that the source device is qualified for screen projection, it can trigger distributed authentication.
  • distributed authentication please refer to the above text and will not be repeated here.
  • the distributed authentication mechanism will be triggered when the target device instructs the source device to perform heterogeneous screen projection.
  • the first time the source device starts screen projection on the target device may refer to the first time the source device starts heterogeneous screen projection on the same target device, or it may refer to the first time the heterogeneous screen projection is started during the period from when the target device is turned on to when it is shut down.
  • the target device is turned on for the first time, and the target device is shut down for the first time after being used for a period of time
  • the first startup to the first shutdown is the first usage period.
  • the target device is turned on for the second time, and the target device is shut down for the second time after being used for a period of time, and the second startup to the second shutdown is the second usage period.
  • the first time the source device starts heterogeneous screen projection on the target device belongs to the "first" startup referred to in the second explanation; during the second usage period, the first time the source device starts heterogeneous screen projection on the target device also belongs to the "first" startup referred to in the second explanation.
  • the source device in a scenario where it is not the first time that the source device is projecting the screen to the target device, that is, the source device has previously initiated heterogeneous screen projection on the same target device, or the source device has initiated heterogeneous screen projection on the target device during the period from when the target device was turned on to when it was shut down.
  • the second explanation is that during the first use period, the source device starts heterogeneous screen projection on the target device for the first time, and when the source device starts heterogeneous screen projection on the target device for the second time, the target device will determine that the source device has started heterogeneous screen projection on the target device; the same applies to the third, fourth, and other times. During the second use period, the source device starts heterogeneous screen projection on the target device for the first time.
  • the target device will determine that the source device has started heterogeneous screen projection on the target device for the first time; when the source device starts heterogeneous screen projection on the target device for the second time, the target device will determine that the source device has started heterogeneous screen projection on the target device; the same applies to the third, fourth, and other times. If the source device has started heterogeneous screen projection on the target device, in order to reduce the number of times the user enters the password while ensuring the privacy of the user, the target device can optionally first determine whether the time since the last distributed authentication exceeds the preset time threshold.
  • the preset time threshold can be a default value or a value set by the user. It indicates that within this time range, the screen projection connection between the source device and the target device still meets the requirements of protecting user privacy and security. Therefore, when the target device determines that the time since the last distributed authentication does not exceed the preset time threshold, it directly starts the screen projection without displaying the unlock verification interface. This can avoid repeated requests.
  • the user inputting passwords makes the operation cumbersome, which improves the convenience of users in using heterogeneous screen projection while ensuring the privacy of users.
  • the time since the last security authentication is passed exceeds the preset time threshold distributed authentication needs to be performed again. It can be understood that when the target device determines that the time since the last distributed authentication is passed exceeds the preset time threshold, it is considered that the screen projection connection between the source device and the target device at this time no longer meets the requirements of protecting user privacy and security.
  • the distributed authentication process in this case is the same as the distributed authentication process of "the source device starts heterogeneous screen projection on the target device for the first time" mentioned above, and will not be repeated here.
  • the source device has set an unlock password and has not previously started heterogeneous screen projection on the target device.
  • the target device determines that the user has instructed the source device to perform screen projection and that the source device has the conditions for screen projection, the distributed authentication mechanism can be triggered.
  • the first heterogeneous screen projection is successfully started; the first heterogeneous screen projection ends at time T1; assuming that the preset time threshold set by the user is 1 hour, at time T2 the target device detects again that the user has instructed the same source device to perform heterogeneous screen projection; the target device determines that the source device has started heterogeneous screen projection on the target device, and then determines whether the time from T2 to T1 is greater than 1 hour; the target device determines that the value of (T2-T1) is less than 1 hour, and directly starts the second heterogeneous screen projection.
  • the mobile phone displays the main interface 101 and the screen projection window 201 on the PC.
  • the screen projection window 201 displays the application interface of the screen-projected application, such as the application interface 202 of the social application.
  • the display screen of the mobile phone enters a pseudo-screen-off state, and the projection window 201 of the PC continues to display the application interface 202.
  • the mobile phone When the mobile phone receives a lock screen operation from the user, or when the mobile phone automatically locks the screen due to lack of user operation for a long time, the mobile phone displays the lock screen interface, and the projection window 201 of the PC continues to display the application interface 202.
  • the layered architecture divides the software into several layers, each with clear roles and division of labor.
  • the layers communicate with each other through software interfaces.
  • the right half of Figure 9 is a software architecture diagram of the source device, such as a mobile phone.
  • the left half of Figure 9 is a software architecture diagram of the target device, such as a tablet (Pad) or PC.
  • the software framework of the source device may include an application layer, a capability service layer, an application framework layer (framework, FWK), an Android runtime (Android runtime) and a system library and a driver layer.
  • the application layer may include a series of application packages, such as social, information, call, camera and other applications (also referred to as applications).
  • exemplary, social applications include WeChat, QQ, etc.
  • information applications include text messages, etc.
  • the application is referred to as the application below.
  • the application on the source device can be a native application (such as an application installed in the source device when the operating system is installed before the source device leaves the factory), or it can be a third-party application (such as an application downloaded and installed by the user through the application store), which is not limited in the embodiments of this application.
  • the capability service layer provides capability support for implementing corresponding services. As shown in Figure 9, it may include a screen projection assistant module, a first virtualization service module, a first screen projection service module, a first device discovery authentication connection module, etc.
  • the screen projection assistant module can be a module used to interact with other electronic devices (such as target devices) to exchange screen projection related information, realize end-to-end logical control and connection management of screen projection, and generate a session key (SessionKey) to ensure the security of the communication session between the source device and the target device.
  • the screen projection assistant module may include functional modules such as a screen projection management module, a heterogeneous switching module, and a virtualization service initialization module.
  • the screen projection management module is responsible for managing screen projection related matters, such as adding and removing related processing logic settings of VirtualDisplay.
  • VirtualDisplay can also be called a virtual display area, a virtual screen, or a virtual Display.
  • the heterogeneous switching module is used to control the source device to switch from a heterogeneous screen projection mode to a homogeneous screen projection mode.
  • the virtualization service initialization module makes initialization settings for the virtualization service that starts heterogeneous screen projection.
  • the first virtualization service module is used to implement end-to-end logical control of screen projection of audio and video streams, and is responsible for the audio and video data streams generated by Speakers, Mics, Cameras, etc. transmitted between the source device and the target device.
  • the first screen projection service module may include a first display virtualization module and a first reverse control module, which respectively provide screen projection capabilities (i.e., the content of the source device is projected to the target device for display) and reverse event (i.e., the event triggered by the target device to control the source device) basic control capabilities.
  • the first screen projection service module receives instructions from the screen projection management module, and provides corresponding screen projection services according to the instructions, thereby realizing the projection of programs running on the local computer to other electronic devices (such as the target device).
  • the first screen projection service module injects reverse control and input method into the display of the target device (hereinafter referred to as the external screen), and adds a virtual display tag (such as Flag1) when creating a virtual display area, so that the external screen only displays its own content, and does not mirror the content of the display of the source device (hereinafter referred to as the main screen), thereby realizing heterogeneous screen projection.
  • a virtual display tag such as Flag1
  • the first device discovery connection authentication module is responsible for discovery, connection, authentication and other tasks, and is used to control the driver layer to implement proximity discovery, authentication, connection and other functions between the source device and the target device, and may include a first communication module and a first security authentication module.
  • the first communication module provides a signaling transmission channel between the source device and the target device, and is used to send instructions to the driver layer to establish a communication connection and a screen projection connection between the source device and the target device. At the same time, the first communication module is also used to start the screen projection assistant module after the transmission channel is established.
  • the first security authentication module is used for security authentication, and is responsible for owner identification and initiating distributed unlocking authentication to obtain a trust ring and ensure the privacy and security of users.
  • the application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer.
  • the application framework layer includes some predefined functions. As shown in Figure 9, the application framework layer may include an activity management service (ActivityManagerService, AMS), a window management service (WindowManagerService, WMS), a multi-screen framework module, a SystemUI module, etc.
  • AMS is responsible for managing Activities, starting, switching, and scheduling components in the system, and managing and scheduling applications. For each Activity, there will be a corresponding application record (ActivityRecord) in AMS, which records the status of the Activity of the application. Each activity will create a corresponding ActivityRecord object in AMS. AMS can use this ActivityRecord as an identifier to schedule the Activity process of the application; these ActivityRecord objects are managed in their respective task stacks, and each task stack can add multiple ActivityRecord objects; all task stacks are uniformly managed by ActivityStack, which is responsible for the order, stacking, and popping of TaskStack. Specifically, AMS defines data classes for saving processes (Process), activities (Activity), and tasks (Task).
  • the data class corresponding to the process can include process file information, process memory status information, and the information contained in the process.
  • Activity, Service, etc. Activity information can be saved in ActivityStack.
  • ActivityStack is used to uniformly schedule application activities. ActivityStack can specifically save all running Activity (i.e. final ArrayList mHistory) information, such as interface configuration information. For example, the running Activity can be saved in newArrayList. ActivityStack can also save the information of historically run Activities, such as interface configuration information. It should be noted that Activity does not correspond to an application, but ActivityThread corresponds to an application. Therefore, Android allows multiple applications to run at the same time, which actually allows multiple ActivityThreads to run at the same time.
  • each application process reports to AMS when it wants to start a new Activity or stop the current Activity.
  • AMS internally records all application processes. When AMS receives a start or stop report, it first updates the internal record and then notifies the corresponding client process to run or stop the specified Activity. Since AMS has records of all Activities internally, it can schedule these Activities and automatically close the background Activity according to the status of the Activity and system memory.
  • WMS carries data and attributes related to the "interface” and is used to manage the state related to the "interface", that is, to manage the graphical user interface (GUI) resources used on the mobile phone screen, such as managing window programs and event dispatching.
  • managing window programs means outputting to the physical screen or other display devices in an orderly manner with the assistance of the application server and WMS according to the display request of the application.
  • Event dispatching means dispatching user events from the keyboard, physical buttons, touch screen, mouse, trackball (TraceBoll), etc. to the corresponding controls or windows.
  • the management window program can also obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc., and is responsible for managing the display mode of the application window, including the coordinate size of the window display, the window display level, etc. Specifically, it may include: creating and destroying windows, displaying and hiding windows, layout of windows, management of focus, and management of input methods and wallpapers, etc.
  • WMS can ensure the visibility of the projection window when the mobile phone is locked and the screen is off.
  • the task stack management function of AMS is used to manage the life cycle of Activities and the scheduling between Activities.
  • the life cycle management function of WMS is used to manage the life cycle of windows. The two can cooperate with each other to activate the window through the callback (onResume) interface, and the activated window is displayed in the foreground; or, the window can be paused through the pause (onPause) interface to switch the window from the foreground to the background.
  • the top Window (top Activity) in the task stack as an example, the top Activity calls onResume to activate it, then the top Activity is displayed in the foreground.
  • the original top Activity calls onPause to pause the window so that the original top Activity switches from the foreground to the background. Similar principles can be used to manage the life cycle of new windows.
  • the multi-screen framework module is responsible for implementing the heterogeneous screen projection logic, calling the PendingIntetRecord interface to start the Acvtivity corresponding to PendingIntet (specify the behavior send), and starting the application corresponding to the Activity to the virtual display area by adding a heterogeneous screen projection tag (such as Flag2).
  • PendingIntent is a capability provided by Android for external programs to call up their own programs. Its life cycle is not related to the main program. When certain conditions are met or certain events are triggered, the specified behavior is executed.
  • the multi-screen framework module is also used to monitor sensing events and interact with the screen projection assistant module.
  • the multi-screen framework module can be used to translate, encapsulate, and process the original input events to obtain input events containing more information and send them to WMS.
  • WMS stores the clickable areas (such as controls) of each application, the location information of the focus window, etc. Therefore, WMS can correctly distribute input events to the specified
  • the multi-screen framework module can respond to the received input events, forward the input events to the WMS, and the WMS distributes them to the corresponding applications.
  • the application sends a registration request to the AMS, and the application starts.
  • the multi-screen framework module is also used to switch the application projected to the external screen back to the main screen when the user clicks on the recent task card or desktop icon.
  • the application projected to the external screen adopts a heterogeneous screen projection method
  • it is necessary to call the heterogeneous switching module mentioned above to switch the heterogeneous screen projection method to the homogeneous screen projection method otherwise the application cannot be displayed on the main screen.
  • the multi-screen framework module is also used to remove the security layer when the source device is in the locked screen state.
  • the multi-screen framework module can also be used to make relevant modifications to the power button of the mobile phone in the screen projection scenario.
  • the SystemUI module is a set of UI components that provide users with system-level information display and interaction, mainly used to implement status bar information display (such as battery, wifi signal, 3G/4G and other icon display), notification panel (such as system messages, third-party message notifications), recent task bar display panel (such as displaying recently used applications), etc.
  • the SystemUI module is responsible for the notification service, which is used to transfer the source device message notification flow to the target device.
  • the application framework layer may also include a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. (not shown in FIG. 9 ). The specific meanings thereof may be referred to in relevant technical documents and will not be described here.
  • the Android runtime is responsible for the scheduling and management of the system.
  • the Android runtime includes a core library and a virtual machine.
  • the core library consists of two parts: one is the function that the programming language (for example, the Java language) needs to call, and the other is the core library of the system.
  • the application layer and the application framework layer run in the virtual machine.
  • the virtual machine executes the programming files (for example, Java files) of the application layer and the application framework layer as binary files.
  • the virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.
  • the system library can include multiple functional modules. For example: Surface Manager, Media Libraries, 3D graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc.
  • the surface manager is used to manage the display subsystem and provides the fusion of 2D (2-Dimensional, 2D) and 3D (3-Dimensional, 3D) layers for multiple applications.
  • the media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc.
  • the media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.
  • the 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.
  • the 2D graphics engine is a drawing engine for 2D drawing.
  • the driver layer is the foundation of the Android operating system, and the final functions of the Android operating system are completed through the driver layer.
  • the driver layer includes the underlying driver, which is responsible for discovery, authentication, connection, etc.
  • the driver layer receives commands issued by the communication module in the application framework layer and performs actions such as connection and disconnection.
  • the driver layer includes WI-FI P2P chips, Bluetooth chips, etc., which can establish transmission channels such as WI-FI P2P and Bluetooth.
  • the driver layer can also include hardware drivers such as display drivers, camera drivers, audio drivers, sensor drivers, virtual card drivers, etc.
  • the software framework of the target device may include a PC manager, an operating system, and a driver layer.
  • Some functional modules of the target device and their functions can refer to the above-mentioned source device and will not be described here.
  • the PC manager of the target device includes a message center module, a second virtualization service module, a second screen projection service module, and a second device discovery connection authentication module.
  • the message center module is used to interact with the messages and signaling of the source device, receive message notifications from the source device, and initiate a session command (openSession) to the source device. At the same time, the message center module is also used to issue a command to start the screen projection service after establishing a session.
  • openSession a session command
  • the functions of the second virtualization service module, the second screen projection service module, and the second device discovery authentication connection module can respectively refer to the first virtualization service module, the first screen projection service module, and the first device discovery connection authentication module in the above-mentioned source device.
  • the second screen projection service module is used to receive and parse video streams and related information sent by other devices, so as to display applications running on other devices on the local device.
  • the operating system of the target device can be Windows OS (such as a personal computer) or Android OS (such as a tablet).
  • driver layer of the target device please refer to the above source device.
  • the mobile phone end includes a first communication module, a screen projection assistant module, a first virtualization service module, a first screen projection service module, a first security authentication module, FWK (including SystemUI module), and an application;
  • the PC end includes a message center module, a second screen projection module, a second communication module, and a second security authentication module.
  • the other functional modules shown in Figure 9 also participate in the specific implementation process of a screen projection method provided in an embodiment of the present application, which is not shown in this timing diagram.
  • the screen projection method may include the following steps:
  • the message center module of the PC receives an operation of selecting a mobile phone by a user.
  • S101 is only an example of establishing a first communication connection between a PC and a mobile phone.
  • the PC detects a device in the same local area network as the PC, displays all connectable devices for the user to select, and the user selects the mobile phone on the PC to start the subsequent connection behavior.
  • the mobile phone detects a device in the same local area network as the PC, displays all connectable devices for the user to select, and the user selects the PC on the mobile phone to start the subsequent connection behavior.
  • the mobile phone and the PC can also establish a near-field connection through the "touch and pay" function, the NFC function, etc., which is not limited here.
  • the message center module sends a message to the second communication module instructing to establish a communication connection with the mobile phone, and the message carries a device identification code of the mobile phone.
  • the user selects the mobile phone as the collaborative notification device from the nearby available collaborative notification devices found by the PC.
  • the message center module of the PC can send an inter-process communication (IPC) message to the second communication module.
  • IPC inter-process communication
  • the IPC message can The device name, device identification code, MAC address, etc. of the mobile phone are carried, and are used to instruct the second communication module to establish a communication connection with the mobile phone.
  • the second communication module establishes a first communication connection with the first communication module of the mobile phone.
  • the first communication connection may be a Bluetooth connection (BR connection), which is used to implement collaborative notifications between the PC and the mobile phone, that is, messages on the mobile phone may be transferred to the PC via a Bluetooth channel.
  • BR connection Bluetooth connection
  • the second communication module returns a message to the message center module indicating that the first communication connection is successfully established.
  • S105 The application of the mobile phone receives an application message.
  • the mobile phone After the PC and the mobile phone establish a first communication connection, the mobile phone receives an application message pushed by the application installed on the mobile phone.
  • the application message can have different forms in the application on the mobile phone, the SystemUI module, and the message center module on the PC, collectively referred to as application messages here.
  • the application sends an application message to the SystemUI module.
  • the application message carries the message content and the application package name.
  • S107 The SystemUI module instructs the first communication module to forward the application message to the PC.
  • the SystemUI module will transfer all messages received by the mobile phone to the PC based on the first communication connection; it may also be transferred selectively, for example, messages with a special tag will be transferred to the PC, or messages from specific applications will be transferred to the PC, or messages judged to be of higher importance will be transferred to the PC, etc., which are not limited here.
  • the first communication module sends an application message to the second communication module based on the first communication connection, where the application message carries message content and an application package name.
  • the second communication module sends an application message to the message center module, where the application message carries message content and an application package name.
  • the message center module receives the application message and controls the display screen to display the corresponding message notification control.
  • the application message received on the mobile phone will flow to the PC. It is understandable that at this time, whether the mobile phone is in an unlocked screen state or a locked screen or a screen off state does not affect the flow of message notifications between the PC and the mobile phone.
  • the PC After the PC receives the application message, it displays the message notification control associated with the application message.
  • the target device receives a user operation on the message notification control, the target device establishes a third communication connection with the source device, the target device sends a request to start heterogeneous screen projection to the source device, and the source device responds to the request and displays the application screen projection window on the target device, that is, as shown in Figures 11 and 12, the above method may also include:
  • the message center module receives a first operation of a user clicking a message notification control.
  • the message notification control can be called a first control
  • the first control displays an application message of a first application, such as a first message.
  • the message center module sends a message to the second communication module instructing the creation of a second session.
  • the message carries the service channel name (WI-FI P2P channel) and the device identification code of the mobile phone.
  • the service channel name can be a WI-FI P2P channel
  • the device identification code is the identification code of the mobile phone
  • the above-mentioned instruction to create a second session is used to instruct the second communication module to establish a WI-FI P2P channel between the PC and the mobile phone corresponding to the device identification code.
  • the second communication module establishes a third communication connection with the first communication module.
  • the third communication connection can be a WI-FI P2P connection, providing a data transmission channel for the screen projection service.
  • the types of the first communication connection, the second communication connection, and the third communication connection established between the PC and the mobile phone are not limited to Bluetooth connection and Wi-Fi P2P connection, but may also be other types of connection methods as long as the requirements for data transmission between the two can be met.
  • the first communication module sends a message instructing the screen projection assistant module to start.
  • the first communication module is also responsible for interacting with the screen projection assistant.
  • the screen projection assistant module is in a closed state when the mobile phone has no screen projection demand.
  • the first communication module triggers the screen projection assistant module to start when there is a screen projection demand.
  • the second communication module returns a message to the message center module indicating that the third communication connection is successfully established.
  • the message center module Based on the third communication connection, the message center module negotiates parameters with the screen projection assistant to determine the session key and the IP address of the other party.
  • SessionKey also known as data encryption key or working key
  • SessionKey is a randomly generated encryption and decryption key to ensure a secure communication session between a user and other computers or between two computers. It can be obtained through negotiation between communication users. It is generally generated dynamically and only when session data encryption is required.
  • the message center module sends a message indicating startup to the second screen projection service module, and the message carries the session key and the IP address of the mobile phone.
  • S209 The second screen projection service module is started and initialized according to the session key and the IP address of the mobile phone.
  • the second screen projection service module returns a message indicating that initialization is completed to the message center module.
  • the screen projection service module provides screen projection capabilities and basic reverse event control capabilities. After the screen projection service on the PC is successfully initialized, it provides capability support for receiving and parsing relevant information sent by the mobile phone, so as to display the application running on the mobile phone on the PC. At the same time, the basic reverse event control capability provides support for heterogeneous screen projection. Therefore, before performing heterogeneous screen projection services, the screen projection service module needs to be initialized to ensure the normal provision of screen projection capabilities and basic reverse event control capabilities.
  • the message center module sends a message to the second screen projection service module to instruct the start of the screen projection service.
  • the second screen projection service module Based on the third communication connection, the second screen projection service module sends a message to the screen projection assistant module to start the screen projection service.
  • step S207 includes the second screen projection service module sending a message to the second communication module, the second communication module then transmits the message to the first communication module through the third communication connection, and the first communication module then sends the message to the screen projection assistant module.
  • the following "based on the third communication connection" includes the data transmission process between the first communication module and the second communication module, which will not be repeated below.
  • the above-mentioned message for starting the screen projection service can also be referred to as the first request.
  • the first request can carry the first identifier.
  • the screen projection management module in the screen projection assistant module is bound to the relevant screen projection services to manage screen projection affairs.
  • the screen projection assistant module sends a message indicating initialization to the first virtualization service module.
  • the virtualization service initialization module in the screen projection assistant module is used to instruct the virtualization service module to initialize.
  • S215 The first virtualization service module completes initialization.
  • the first virtualization service module is responsible for the audio and video virtualization in the screen projection service.
  • the interaction between the first virtualization service module on the mobile phone and the second virtualization service module on the PC can enable the mobile phone application projected to the PC to use the PC's Speaker, Mic, and Camera during the screen projection process, and transmit the generated audio and video data streams between the mobile phone and the PC, thereby realizing functions such as making and receiving audio and video calls during screen projection.
  • the first virtualization service module sends a message to the first screen projection service module to instruct the start of the screen projection service.
  • the first screen projection service module creates a virtual display and adds a virtual display tag to it.
  • the screen projection service module is an important functional module for implementing heterogeneous screen projection.
  • the specific implementation method is that the first screen projection service module on the mobile phone adds Flag1: VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY (virtual display tag, also known as the second tag) when creating a virtual display area, which is used to realize that the screen projection window on the PC only displays its own content, and does not mirror the content displayed on the mobile phone display, so as to ensure that the display content of the mobile phone will not affect the display content of the screen projection window of the PC.
  • VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY virtual display tag, also known as the second tag
  • the mobile phone displays other application interfaces or desktops, which will not affect the display of the PC's projection window.
  • the PC's projection window can maintain its own content without synchronizing with the display content of the mobile phone.
  • the basic reverse event control capabilities provided by the projection service module can achieve reverse control of the mobile phone on the computer side.
  • the implementation of heterogeneous projection also requires the interaction of the second projection service module on the PC side and the first projection service module on the mobile phone side.
  • the first screen projection service module sends a message to the screen projection assistant module indicating that the screen projection service has been started.
  • the first screen projection assistant Based on the third communication connection, the first screen projection assistant sends a message to the message center module indicating that the screen projection service has been started.
  • the message center module Based on the third communication connection, the message center module sends a message to FWK to start the application for screen projection, carrying the application package name.
  • the application package name carried is the package name of the application (e.g., social application) corresponding to the message notification control (e.g., message card 104) selected by the user in S201.
  • the message card 104 is used to display application messages from the social application in the mobile phone, and the message card 104 can be seen to correspond to the social application of the mobile phone.
  • the corresponding relationship between other message notification controls and applications is similar.
  • the process of FWK starting the application projection window can refer to the relevant technology, and the application corresponding to the application package name will be started, and the application interface of the application will be drawn in the virtual display area (first display area) to generate display data, which will not be described in detail.
  • FWK can mark the projection label (first label) for all activities corresponding to the above application package name (that is, the application package name carried in the message of starting the application for projection), and all activities corresponding to the above application package name can be all activities corresponding to the application indicated by the application package name.
  • the first screen projection service module Based on the third communication connection, the first screen projection service module sends the display data of the application projection to the second screen projection service module.
  • the second screen projection service module controls the display screen to render the above display data to the screen projection window.
  • the screen projection window may also be referred to as the first window, and the display data rendered to the first window may be image information.
  • the display data rendered in the first window may be the first image information corresponding to the first application.
  • the screen projection of the first application is taken as an example for description.
  • the second screen projection service module sends a message to the message center indicating that the rendering of the application screen projection interface of the screen projection window is complete.
  • the message center module controls the display screen to display the application projection window.
  • the PC displays the application screen projection window corresponding to the application message, that is, the mobile phone successfully projects the screen to the PC when the screen is on and unlocked.
  • the PC when the PC receives the user's click on the message card 104, the PC will display the projection window 201 corresponding to the social application, which is used to display the application interface 202 corresponding to the application message.
  • the application projection window can be a projection window 201 for displaying the application interface 202.
  • the mobile phone may enter the screen-off or screen-locked state.
  • the user instructs the mobile phone to enter the screen-off or screen-locked state by using the lock screen key of the mobile phone.
  • the mobile phone automatically enters the screen-off or screen-locked state when it does not receive any user operation for a long time.
  • Figure 13 shows the process of interaction between the various functional modules inside the mobile phone when the mobile phone goes from the screen-on, unlocked state to the screen-off or locked state during screen projection.
  • the mobile phone may also include PowerManagerService, KeyguardServiceDelegate (lock screen management service), MultiDisplayPowerManager (multi-screen framework module), ActivityTaskManagerService (activity stack management service), KeyguardController (lock screen controller), ActivityRecord (activity recorder), Task (task stack), RootWindowsContainer (root window container), WindowsCastManager (window appearance manager), and SurfaceFlinger (layer synthesizer).
  • PowerManagerService KeyguardServiceDelegate
  • MultiDisplayPowerManager multi-screen framework module
  • ActivityTaskManagerService activity stack management service
  • KeyguardController lock screen controller
  • ActivityRecord activity recorder
  • Task task stack
  • RootWindowsContainer root window container
  • WindowsCastManager window appearance manager
  • SurfaceFlinger layer synthesizer
  • the above method may include:
  • PowerManagerService receives an operation instructing to lock the screen.
  • PowerManagerService is responsible for the power management of the mobile phone system. Common functions include: turning on the screen, turning off the screen, entering the screen saver, etc. PowerManagerService inherits from SystemService, is started by SystemServer, registered in the system service, and interacts with other components through Binder.
  • PowerManagerService After PowerManagerService receives a press operation from the user via the lock screen key (ie, the Power key) of the mobile phone, it can determine that an operation instructing to lock the screen has been received.
  • PowerManagerService sends a screen lock instruction to KeyguardServiceDelegate.
  • the KeyguardServiceDelegate is a class for proxying the KeyguardService.
  • the PowerManagerService sends a lock screen instruction to the KeyguardServiceDelegate to instruct the start of the lock screen loading process.
  • KeyguardServiceDelegate sends power management information to MultiDisplayPowerManager.
  • KeyguardServiceDelegate when KeyguardServiceDelegate determines that a screen projection service is in progress between the mobile phone and the PC, it instructs MultiDisplayPowerManager to take over PowerManagerService to manage the power of the mobile phone system through power management information.
  • the mobile phone in addition to receiving the user's instruction to lock the screen, when the mobile phone automatically turns off the screen or receives the user's instruction to turn off the screen, it can also trigger the instruction MultiDisplayPowerManager to take over PowerManagerService to manage the power of the mobile phone system. That is, in some possible scenarios, the above method may not include S301 to S302, but include the following S304 to S306.
  • PowerManagerService determines that the display screen is automatically turned off.
  • the process of determining whether the display screen automatically turns off can refer to related technologies and will not be described in detail here.
  • PowerManagerService detects the projection tag of the virtual display screen and interrupts the screen off processing for the virtual display screen.
  • PowerManagerService detects the projection label of the virtual display screen, indicating that the current mobile phone is projecting the screen to other devices and the projection is in progress. In this scenario, in order to avoid the automatic screen off affecting the projection effect, PowerManagerService can interrupt the screen off processing for the virtual display screen.
  • PowerManagerService sends power management information to MultiDisplayPowerManager.
  • MultiDisplayPowerManager can be instructed to take over PowerManagerService to manage the power of the mobile phone system.
  • MultiDisplayPowerManager controls the display screen to enter a fake screen-off state.
  • the above-mentioned fake screen-off state, screen-off state, and screen-on state are state types of the display screen.
  • the screen-on state not only has different LCD online and offline states of the display screen (e.g., the LCD is in a powered-on state in the screen-on state, and the LCD is in an offline state in the screen-off state), but also has different configuration parameters corresponding to the internal related functional services (e.g., AOD service, POWER service).
  • the fake screen-off state only has different LCD online and offline states of the display screen (e.g., the LCD is in a powered-on state in the screen-on state, and the LCD is in an offline state in the fake screen-off state), but the configuration parameters corresponding to the internal related functional services (e.g., AOD service, POWER service) are the same.
  • the LCD online and offline states of the fake screen-off state are the same, but the configuration parameters corresponding to the internal related functional services (e.g., AOD service, POWER service) are also different.
  • the screen-off state and the screen-on state are managed by PowerManagerService.
  • the MultiDisplayPowerManager can control the display screen to enter a fake screen-off state.
  • MultiDisplayPowerManager can respond to the above power management information and switch the display screen of the mobile phone from the on-screen state to the fake off-screen state. During the switching process, MultiDisplayPowerManager only needs to control the LCD power off through the TP service without changing the status and configuration of related services.
  • the above method may further include:
  • PowerManagerService instructs ActivityTaskManagerService to configure the application screen projection window Visible at the front desk.
  • ActivityTaskManagerService sends a message to KeyguardController instructing to set the Activity with the projection tag to the OnResume state.
  • ActivityTaskSupervisor listens to whether the Activity is in the OnResume state by calling KeyguardController. If so, it means that the application projection window corresponding to the Activity is visible to the user and can interact with the user after it is launched on the PC (OnResume state); if the Activity is in the stopped state (OnPause state), it means that the application projection window corresponding to the Activity is in the background after it is launched on the PC, and the user cannot operate it.
  • the OnResume state and the OnPause state are two states in the life cycle of the Activity.
  • the Activity When the Activity is in the OnResume state, the Activity is in an activated state that is visible and can interact with the user, and the user can get the focus of the activity; when the Activity is in the OnPause state, the Activity is covered by another transparent or Dialog-style Activity. At this time, it still maintains a connection with the window manager, and the system continues to maintain its internal state. It is still visible, but it has lost focus, so it cannot interact with the user.
  • the AMS component When the phone is in the locked or off screen state, the AMS component will switch the Activity lifecycle to the onPause state during the process of starting the Activity, causing the Activity to be unable to interact with the user, resulting in screen projection exceptions.
  • KeyguardController identifies the Activity with the projection tag and sets it to the OnResume state.
  • KeyguardController is a lock screen controller. It identifies the Activity with the projection tag and returns "true” to indicate that the Activity is set to the OnResume state, ensuring that when the source device is in the lock screen state, the application projection window corresponding to the Activity with the projection tag is visible on the target device.
  • KeyguardController sends a message to ActivityRecord instructing to keep the Activity with the projection tag in the OnResume state.
  • ActivityRecord controls the Activity with the projection tag to maintain the OnResume state.
  • ActivityRecord sends a verification message to Task, instructing Task to verify the state of the Activity at the top of the task stack.
  • Task determines that the Activity at the top of the task stack is in the OnResume state.
  • an application may correspond to one or more Activities, and one or more Activities will be stored in one or more task stacks.
  • the task stack follows the last-in-first-out rule, and the Activity at the top of the task stack is executed first. In order to ensure that the application projection window corresponding to the Activity to be executed is visible in the foreground, it is necessary to ensure that the Activity at the top of the stack is in the OnResume state.
  • steps S313 and S314 are equivalent to a fault-tolerant mechanism, which checks whether the Activity at the top of the task stack is in the OnResume state, so these two steps are optional, that is, in other embodiments, S313 and S314 may not be included.
  • Task instructs RootWindowsContainer to control all tasks to enter a dormant state.
  • RootWindowsContainer traverses the Activities of all tasks, skips the Activities with the projection tag, and controls other Activities to enter the dormant state.
  • RootWindowsContainer needs to traverse all the activities corresponding to the tasks, filter out the Activities with the screen projection tag and return "false” to indicate that the Activity does not enter sleep; other Activities return "true” to enter the sleep process.
  • Activity1 and Activity2 in the same task stack are both added with heterogeneous screen projection tags, and RootWindowsContainer makes Activity1 and Activity2 skip the sleep process; while Activity3 and Activity4 that are not added with heterogeneous screen projection tags are put into sleep state.
  • Task checks the status of Activity1 to ensure that it is in the OnResume state. Then, after Activity1 is projected from a different source, it is visible to the user on the PC and can be operated by the user in the foreground.
  • Activity1 After executing Activity1, Activity2, which is not dormant at this time, progresses to the top of the task stack. Task checks the status of Activity2 to ensure that it is in the OnResume state. Then, Activity2 is visible to the user on the PC after being projected from another source and can be operated by the user in the foreground. The execution process of other Activities that have been marked with projection from another source is similar and will not be repeated here. Activities that have not been marked with projection from another source will be put into dormant state by RootWindowsContainer and will not be started to display on the external screen.
  • the application screen projection window on the PC is not affected by the state of the mobile phone and can still maintain foreground visibility and operability.
  • the above method may further include:
  • MultiDisplayPowerManager sends a cancel message to WindowsCastManager, where the cancel message indicates to cancel drawing of the lock screen layer in the virtual display area.
  • WindowsCastManager determines not to draw the lock screen layer in the virtual display area.
  • MultiDisplayPowerManager instructs SurfaceFlinger not to switch to the same-source projection.
  • the role of SurfaceFlinger is to receive graphics display data from multiple sources, synthesize them, and then send them to the display device. For example, when opening an application, there are three common layers of display, the status bar at the top, the navigation bar at the bottom or side, and the application interface. Each layer is updated and rendered separately. These interfaces are synthesized by SurfaceFlinger into an application layer display data refreshed to the hardware display.
  • SurfaceFlinger determines that the virtual display area has a virtual display tag, then when synthesizing the application layer, only the layer on the virtual display area is obtained, and the layers of other display areas will not be obtained to synthesize with it, so that the aforementioned heterogeneous screen projection method can be realized, that is, the synthesized application layer is only related to the layer of the virtual display area.
  • SurfaceFlinger will not obtain the layer of the visible display area, so even if the visible display area includes the lock screen interface or other application interfaces, it will not affect the interface displayed by the PC's screen projection window. Because SurfaceFlinger only obtains the layer of the virtual display area and passes it to the PC's screen projection window for display in some way.
  • the application interface of the first application can be drawn with the virtual display area, and then the corresponding display data is synthesized by SurfaceFlinger, such as the first image information.
  • the application interface of the first application can still be drawn with the virtual display area, and the corresponding display data is synthesized by SurfaceFlinger, such as the second image information. Since the above-mentioned first image information and second image information both correspond to the application interface of the first application, in some scenarios, the corresponding display content of the two may be the same.
  • the above application layer may include the application interface corresponding to the currently executed Activity.
  • the above display data may include information such as different display objects in the application layer, display positions and sizes of the display objects in the virtual display screen.
  • the virtual display area is the virtual display area of the flag Flag1 created in step S218.
  • the content of the virtual display area is invisible to the user, and the mobile phone can use the virtual display area as a canvas that occupies a certain amount of storage space.
  • Figure 16 is a process diagram of starting the first application projection window.
  • the PendingIntentRecord will mark all Activities of application 1 with a projection label, as shown in the task stack (created by Task) in Figure 16.
  • Activity1 of application 1 Activity1 corresponds to the first message
  • the mobile phone draws the first application interface corresponding to Activity1 in the virtual display area.
  • SurfaceFlinger determines that the virtual display area has a virtual display label, then only obtains the first application interface layer in the virtual display area, and generates the corresponding first display data and sends it to the target device.
  • the target device After the target device receives the first display data, it draws and renders the application projection interface corresponding to the first message of the first application according to the first display data, and displays it in the projection window of the target device. Please refer to steps S225-S227 in Figure 12 and Figure 7 for details. At this time, the projection window of the target device displays the application projection interface corresponding to the first display data.
  • the mobile phone When executing Activity2 of Application 1 (Activity2 is pushed to the top of the task stack), the mobile phone draws the second application interface corresponding to Activity2 in the virtual display area.
  • SurfaceFlinger determines that the virtual display area has a virtual display tag, then only obtains the second application interface layer in the virtual display area, and generates the corresponding second display data and sends it to the target device. Then the projection window of the target device displays the application projection interface corresponding to the second display data, that is, the application projection interface corresponding to the first display data is switched to the application projection interface corresponding to the second display data, completing the switching of different application interfaces of Application 1 in the projection window.
  • FIG. 17 is a process diagram for starting the second application screen projection window.
  • the source device receives the second message (also known as the second data) from the second application. After the second message flows to the target device, the target device displays the message notification control corresponding to the second message. After the target device receives the operation of the source device clicking the message notification control, it sends a second request to the source device to start the screen projection.
  • the first screen projection service module receives the second request, creates a second virtual display area (VirtualDispaly2) and adds a virtual display tag.
  • Activity3 of Application 2 Activity3 corresponds to the second message
  • the mobile phone draws the third application interface corresponding to Activity3 in the virtual display area, and SurfaceFlinger determines that the virtual display area has a virtual display tag, then only obtains the third application interface layer in the virtual display area, and generates the corresponding third display data and sends it to the target device.
  • the target device After receiving the third display data, the target device draws and renders the application projection interface corresponding to the second message of the second application according to the third display data, and displays it in the projection window of the target device.
  • the projection window of the target device displays the application projection interface corresponding to the second display data and switches to the application projection interface corresponding to the third display data, completing the switching of application interfaces of different applications (such as the first application and the second application) in the projection window.
  • the source device may optionally enable a security authentication function to protect the privacy of the source device that has locked or turned off the screen.
  • a security authentication interface may also be displayed, and a transition loading interface may also be displayed.
  • the above step S201 i.e., the message center module receives the message.
  • the security authentication mechanism After receiving the first operation of the user clicking the message notification control, the security authentication mechanism will be triggered first.
  • the specific implementation process please refer to Figures 14, 15 and the following description.
  • the above method may further include:
  • the message center module receives a second operation of the user clicking a message notification control.
  • step S201 in Figure 11 are both operations in which the user clicks on the message notification control.
  • steps shown in Figures 14 and 15 are added before triggering the establishment of the third communication connection in Figure 11.
  • the security authentication mechanism will be triggered before the screen projection is started from the PC to ensure the user's privacy security.
  • the security authentication mechanism Since the embodiment of the present application directly starts the screen projection on the PC side, the security authentication mechanism will be triggered on the PC side, and the password authentication is distributed on the mobile phone side, so the second security authentication module on the PC side needs to send the authentication information (password) received on the PC side to the first security authentication module on the mobile phone side for password authentication, and the mobile phone side will return the authentication result to the PC side. Therefore, in order to transmit the signaling required for security authentication between the mobile phone and the PC, a transmission channel needs to be established first.
  • the message center module sends a message instructing to create a first session to the second communication module.
  • the message carries a service channel name (Bluetooth channel) and a device identification code of the mobile phone.
  • the service channel name may be a Bluetooth channel
  • the device identification code may be an identification code of a mobile phone
  • the instruction for creating the first session is used to instruct the second communication module to establish a Bluetooth channel between the PC and the mobile phone corresponding to the device identification code.
  • the second communication module establishes a second communication connection (Bluetooth connection) with the first communication module.
  • the second communication connection may be a Bluetooth connection (BR connection), which provides a securely authenticated signaling transmission channel.
  • BR connection Bluetooth connection
  • the first communication module sends a message instructing the screen projection assistant to start.
  • the second communication module returns a message to the message center module indicating that the second communication connection is successfully established.
  • the message center module sends a message instructing the second screen projection service module to start.
  • the second screen projection service module returns a message of successful startup to the message center module.
  • the screen projection assistant on the mobile phone and the second screen projection service module on the PC have been started, providing the necessary functions for realizing the subsequent screen projection service.
  • the method further includes:
  • the message center module sends a security authentication request to the second security authentication module of the PC.
  • the second security authentication module controls the display screen of the PC to display a security authentication window.
  • a security authentication interface is displayed in the projection window 201, and the user can enter a password in the security authentication window.
  • the security authentication method is not limited to entering a password for security authentication, such as fingerprint unlocking scenario
  • fingerprints are collected for verification; in the case of voice unlocking, voice is collected for verification.
  • password unlocking is only used as an example.
  • the second security authentication module receives the password input by the user through the security authentication window.
  • the second security authentication module sends a password to the second communication module.
  • the second communication module sends a password to the first communication module based on the second communication connection.
  • the first communication module sends a password to the first security authentication module of the mobile phone.
  • the security authentication method can also be fingerprint authentication by the user inputting a fingerprint, voice authentication by the user inputting a voice, etc., and this is not limited.
  • the first security authentication module receives the password collected by the PC and compares it with the unlock password set by the user on the mobile phone. If the password collected by the PC matches the unlock password set on the mobile phone, the authentication is successful, otherwise the authentication fails.
  • S507 The first security authentication module successfully performs password authentication.
  • the PC can prompt the user that the password is wrong in the security authentication window.
  • a security authentication request will be sent to the mobile phone again for password verification.
  • it can be set to suspend authentication if the number of authentication failures exceeds the threshold of the number of incorrect passwords.
  • the threshold of the number of incorrect passwords can be set by the user or the default value.
  • the first security authentication module sends a message of successful authentication to the first communication module.
  • the first communication module sends an authentication success message to the second communication module based on the second communication connection.
  • the second communication module sends a message of successful authentication to the second security authentication module.
  • the second security authentication module sends a message of successful authentication to the message center module.
  • the message center module controls the display screen to display a transition loading window.
  • the application screen projection steps of Figures 11 to 12 will be started. It takes a certain amount of time to successfully start the application screen projection, so there is a certain time span between displaying the security authentication window and displaying the application screen projection window.
  • the transition loading window can be started and displayed on the target device, which can prompt the user that the security authentication has passed and the application screen projection is being started.
  • the projection window 201 displays the transition loading interface and switches to the application interface.
  • the target device after the target device receives the user's instruction to project the application screen on the corresponding source device, it first determines whether security authentication is required. For example, whether the source device is projecting the screen on the target device for the first time. For another example, whether the source device has set an unlock password, please refer to the description in the above embodiments for details.
  • the electronic device includes hardware and/or software modules corresponding to the execution of each function.
  • the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to exceed the scope of the present application.
  • the electronic device can be divided into functional modules according to the above method example.
  • each function can be divided into various functional modules, such as a detection unit, a processing unit, a display unit, etc.
  • the above integrated module can be implemented in the form of hardware or in the form of software function modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.
  • the electronic device provided in this embodiment is used to execute the above-mentioned screen projection method, and thus can achieve the same effect as the above-mentioned implementation method.
  • the electronic device may further include a processing module, a storage module and a communication module.
  • the processing module may be used to control and manage the actions of the electronic device.
  • the storage module may be used to support the electronic device to execute stored program codes and data, etc.
  • the communication module may be used to support the electronic device to communicate with other devices.
  • the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application.
  • the processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc.
  • the storage module can be a memory.
  • the communication module can specifically be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, or other devices that interact with other electronic devices.
  • the electronic device involved in this embodiment may be a device having the structure shown in FIG. 9 .
  • An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored.
  • the computer program is executed by a processor, the processor executes the screen projection method of any of the above embodiments.
  • An embodiment of the present application also provides a computer program product.
  • the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement the screen projection method in the above-mentioned embodiment.
  • the embodiments of the present application also provide a device, which can be a chip, a component or a module, and the device may include a connected processor and a memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory, so that the chip executes the caller ID method in the above-mentioned method embodiments.
  • the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding method provided above, so the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.
  • the disclosed devices and methods can be implemented in other ways.
  • the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
  • Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium.
  • the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Telephone Function (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请提供一种投屏方法和电子设备，涉及通信技术领域和终端技术领域。具体方案为：在第一设备向第二设备投屏之后，第二设备显示第一窗口；在第一设备处于亮屏状态时，第二设备在第一窗口中显示来自于第一设备的第一图像信息，第一图像信息对应第一设备中第一应用的应用界面；在第一设备切换至锁屏或息屏状态之后，第二设备在第一窗口中显示来自第一设备的第二图像信息，第二图像信息也对应第一设备中第一应用的应用界面。这样，在第一设备向第二设备投屏之后，即使第一设备由亮屏状态切换至息屏状态或锁屏状态，第二设备依然能够正常显示第一设备提供的应用投屏窗口。

Description

一种投屏方法和电子设备
本申请要求于2022年10月31日提交国家知识产权局、申请号为202211348614.X、发明名称为“一种投屏方法和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及通信技术领域和终端技术领域，尤其涉及一种投屏方法和电子设备。
背景技术
多台电子设备协同工作给用户的生活和工作带来便利。以协同通知为例，手机接收到一条消息，不仅可以由手机显示消息通知，还可以将该消息通知流转到PC，由PC进行显示，这样，方便用户跨设备查看消息，提高设备间协同工作的效率。当然，设备间仅仅进行消息通知的流转是不够的，用户在PC上查看了来自手机的消息通知之后，往往需要通过PC处理该消息通知，这就需要将手机投屏至PC。
然而，在PC和手机投屏期间，如果手机进入息屏和锁屏模式，会造成PC出现投屏异常。
发明内容
本申请实施例提供一种投屏方法和电子设备，在第一设备向第二设备投屏期间，第一设备切换至锁屏或息屏状态之后，第二设备依然能够正常显示第一设备提供的应用投屏窗口。
为达到上述目的，本申请的实施例采用如下技术方案：
第一方面，本申请实施例提供了一种投屏方法，应用于第一设备和第二设备，方法包括：在第一设备向第二设备投屏之后，第二设备显示第一窗口；在第一设备处于亮屏状态时，第二设备在第一窗口中显示来自于第一设备的第一图像信息，第一图像信息对应第一设备中第一应用的应用界面；在第一设备切换至锁屏或息屏状态之后，第二设备在第一窗口中显示来自第一设备的第二图像信息，第二图像信息也对应第一设备中第一应用的应用界面。
在上述实施例中，第一设备在亮屏状态下，向第二设备投屏。比如，进行异源投屏。在异源投屏成功之后，第一设备可以将第一应用的应用界面，绘制成第一图像信息，并发送给第二设备，由第二设备显示于第一窗口。这样，用户可以通过操作第二设备，使用安装有第一设备中的第一应用。之后，第一设备可以响应用户的操作进入息屏或锁屏状态，第一设备可以根据第一应用的应用界面，绘制成第二图像信息，并发送给第二设备，使第二设备可以继续在第一窗口显示与第一应用相关的应用界面。可以理解的，上述第一图像信息和第二图像信息均对应第一应用的应用界面，可以是不同时刻下根据第一应用的应用界面所绘制的图像信息，也就是，第一图像信息和第 二图像信息所显示的内容可以相同，也可以不同。
这样，在第一设备切换至息屏或锁屏状态之后，第一设备可以继续正常的向第二设备进行投屏。同时，也避免用户不使用第一设备期间，长期亮屏导致的能耗过大，以及避免他人未经用户允许使用未锁屏的第一设备。
在一些实施例中，在第一设备切换至锁屏或息屏状态之后，方法还包括：第一设备配置第一应用在生命周期中不休眠，以及配置第一应用的应用界面在生命周期中允许绘制于第一显示区域，第一显示区域是第一设备中不可见的显示区域；第一设备向第二设备发送第一应用的第二图像信息，指示第二设备在第一窗口中显示第二图像信息，第二图像信息是绘制于第一显示区域的数据。
在上述实施例中，通过改变第一应用的不进入休眠，保持第一应用的应用界面前台可见，确保第一设备息屏或锁屏之后，用户可以通过第二设备中的第一窗口，继续使用第一设备中的第一应用。
在一些实施例中，在所述第一设备向所述第二设备投屏之前，所述方法还包括：第一设备接收第一应用的第一消息；第一设备向第二设备转发所述第一消息；第二设备显示第一控件，第一控件用于显示第一消息；第二设备响应于用户对第一控件的操作，向第一设备发送第一请求，第一请求包括所述第一应用的第一标识；第一设备响应于所述第一请求，启动向所述第二设备进行投屏。
在一些实施例中，在启动向所述第二设备进行投屏之后，所述方法还包括：第一设备创建第一显示区域，并启动第一应用；在第一应用的应用界面绘制于第一显示区域之后，第一设备根据第一显示区域中的图像数据，生成第一图像信息；第一设备向所述第二设备发送第一图像信息。
在一些实施例中，在启动第一应用之前，所述方法还包括：第一设备给第一应用标记第一标签；在第一设备配置第一应用在生命周期中不休眠，以及配置第一应用的应用界面在生命周期中允许绘制于第一显示区域之前，所述方法还包括：第一设备确定第一应用具有第一标签。
在一些实施例中，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：第一设备响应于用户指示锁屏或息屏的操作，仅控制所述第一设备的显示屏下电或仅在显示屏上显示锁屏图层，显示屏对应的服务运行状态保持不变。
在一些实施例中，第一显示区域被标记有第二标签，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：在所述第一设备检测到满足自动息屏的条件时，确定所述第一显示区域具有所述第二标签；第一设备仅控制所述第一设备的显示屏下电或仅在所述显示屏上显示锁屏图层，所述显示屏对应的服务运行状态保持不变。
在一些实施例中，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：所述第一设备确定不在所述第一显示区域中绘制锁屏图层。
在一些实施例中，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：所述第一设备确定所述第一显示区域不镜像第二显示区域中的内容，所述第二显示区域是所述第一设备中可见的显示区域。
第二方面，本申请实施例提供了一种投屏方法，应用于第一设备，所述方法包括：第一设备向第二设备进行投屏；在所述第一设备处于亮屏状态时，所述第一设备向所 述第二设备发送第一应用对应的第一图像信息，指示所述第二设备在第一窗口上显示所述第一图像信息；在所述第一设备切换至锁屏或息屏状态之后，所述第一设备向所述第二设备发送所述第一应用对应的第二图像信息，指示所述第二设备在所述第一窗口上显示所述第二图像信息，所述第一图像信息和第二图像信息均为所述第一应用的应用界面。
在一些实施例中，在所述第一设备切换至锁屏或息屏状态之后，所述方法还包括：第一设备配置所述第一应用在生命周期中不休眠，以及配置所述第一应用的应用界面在生命周期中允许绘制于第一显示区域，所述第一显示区域是所述第一设备中不可见的显示区域；所述第一设备根据所述第一显示区域中绘制的内容，生成所述第二图像信息。
在一些实施例中，在所述第一设备向所述第二设备进行投屏之前，所述方法还包括：第一设备接收第一应用的第一消息；第一设备向所述第二设备转发所述第一消息；在接收到所述第二设备发送的第一请求之后，启动向所述第二设备进行投屏，所述第一请求是针对所述第一应用进行投屏的请求信息，所述第一请求携带所述第一应用的第一标识。
在一些实施例中，在所述第一设备向所述第二设备发送第一应用对应的第一图像信息之前，所述方法还包括：所述第一设备创建第一显示区域，并启动所述第一应用；
在所述第一应用的应用界面绘制于所述第一显示区域之后，所述第一设备根据所述第一显示区域中的图像数据，生成所述第一图像信息。
在一些实施例中，在启动所述第一应用之前，所述方法还包括：第一设备给所述第一应用标记第一标签；在所述第一设备配置所述第一应用在生命周期中不休眠，以及配置所述第一应用的应用界面在生命周期中允许绘制于第一显示区域之前，所述方法还包括：所述第一设备确定所述第一应用具有所述第一标签。
在一些实施例中，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：第一设备响应于用户指示锁屏或息屏的操作，仅控制所述第一设备的显示屏下电或仅在所述显示屏上显示锁屏图层，所述显示屏对应的服务运行状态保持不变。
在一些实施例中，所述第一显示区域被标记有第二标签，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：在所述第一设备检测到满足自动息屏的条件时，确定所述第一显示区域具有所述第二标签；所述第一设备仅控制所述第一设备的显示屏下电或仅在所述显示屏上显示锁屏图层，所述显示屏对应的服务运行状态保持不变。
在一些实施例中，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：第一设备确定不在所述第一显示区域中绘制锁屏图层。
在一些实施例中，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：第一设备确定所述第一显示区域不镜像第二显示区域中的内容，所述第二显示区域是所述第一设备中可见的显示区域。
第三方面，本申请实施例提供了一种电子设备，电子设备包括一个或多个处理器和存储器；所述存储器与处理器耦合，存储器用于存储计算机程序代码，计算机程序代码包括计算机指令，当一个或多个处理器执行计算机指令时，所述一个或多个处理 器，用于执行第一方面及其可能的实施例所述的方法；或者，当一个或多个处理器执行计算机指令时，所述一个或多个处理器，用于执行第二方面及其可能的实施例所述的方法。
第四方面，本申请实施例提供了一种计算机存储介质，包括计算机指令，当所述计算机指令在电子设备上运行时，使得所述电子设备执行第一方面及其可能的实施例所述的方法；或者，当所述计算机指令在电子设备上运行时，使得所述电子设备执行第二方面及其可能的实施例所述的方法。
可以理解地，上述各个方面所提供的电子设备、计算机存储介质以及计算机程序产品均应用于上文所提供的对应方法，因此，其所能达到的有益效果可参考上文所提供的对应方法中的有益效果，此处不再赘述。
附图说明
图1是本申请实施例提供的一种消息流转的场景示意图。
图2是本申请实施例提供的一种异源投屏的界面显示示意图。
图3是本申请实施例提供的一种异源投屏的界面显示示意图。
图4是本申请实施例提供的一种异源投屏的原理示意图。
图5是本申请实施例提供的一种异源投屏的界面显示示意图。
图6是本申请实施例提供的一种异源投屏的界面显示示意图。
图7是本申请实施例提供的判断机制的流程图。
图8是本申请实施例提供的一种异源投屏的界面显示示意图。
图9是本申请实施例提供的一种电子设备的软件架构图。
图10是本申请实施例提供的一种投屏方法的具体实现过程图。
图11是本申请实施例提供的又一种投屏方法的具体实现过程图。
图12是本申请实施例提供的又一种投屏方法的具体实现过程图。
图13是本申请实施例提供的又一种投屏方法的具体实现过程图。
图14是本申请实施例提供的又一种投屏方法的具体实现过程图。
图15是本申请实施例提供的又一种投屏方法的具体实现过程图。
图16是本申请实施例提供的一种原理示意图。
图17是本申请实施例提供的又一种原理示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述。其中，在本申请实施例的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；文本中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况，另外，在本申请实施例的描述中，“多个”是指两个或多于两个。
应当理解，本申请的说明书和权利要求书及附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤 或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
在本申请中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本申请所描述的实施例可以与其它实施例相结合。
示例性地，本申请实施例提供了一种系统，包括多台电子设备，如设备1、设备2及设备3。上述系统中的电子设备(也即，设备1、设备2、设备3)可以属于同一局域网络。
该局域网络可以是无线通信网络，如，无线局域网(wireless local areanetworks，WLAN)、无线保真点对点(wireless fidelity point to point，Wi-Fi P2P)、蓝牙网络、zigbee网络、红外技术(infrared，IR)或近场通信(near field communication，NFC)网络等。又示例性地，上述局域网络也可以是有线通信网络。例如，视频图像配接器(video graphics array，VGA)、数字视频接口(digital visual interface，DVI)、高清多媒体接口(high definition multimedia interface，HDMI)或数据传输线等建立的通信网络。再示例性地，上述局域网络还可以是远程网络。
在设备1、设备2和设备3彼此经过信任互认之后，由设备1、设备2和设备3组成的系统又可称为信任环。另外，设备间进行信任互认的过程可参考相关技术，例如，可以根据设备间是否登录有相同的系统账号、是否已进行授权以及是否采用近场通信方式等一项或多项进行判断，具体在此不再赘述。
可以理解地，在同一信任环中，多台电子设备之间可以安全地进行数据共享，共享的数据包括但不限于消息提醒、消息记录以及其他电子设备的用户信息、设备类型、各个应用的运行状态、各个应用的标识、最近使用应用的标识的同步等等。
这样，在信任环内，可以实现多样的协同服务，如，协同通话、协同通知、投屏等服务。
以协同通知为例，在信任环中，设备1接收到消息通知，不仅可以由设备1显示该消息通知，还可以将该消息通知流转到设备2和/或设备3，由设备2和/或设备3进行显示，这样，方便用户跨设备查看消息，提高设备间协同工作的效率。同样，设备2也可以将接收到的消息通知流转到设备1和/或设备3，设备3也可以将接收到的消息通知流转到设备1和/或设备2。
当然，设备间仅仅进行消息的流转是不够的，如果用户在设备2上查看了来自设备1的消息通知之后，往往需要通过设备2处理该消息通知，这就需要将设备1投屏至设备2。
一般来说，投屏场景中包括源端设备和目标设备，源端设备与目标设备之间通过某种协议，实现源端设备上的内容在目标设备上显示，即源端设备向目标设备投屏。常见的协议如DLNA(Digital Living Network Alliance，数字生活网络联盟)协议和Chromecast协议、Miracast协议、Airplay协议等。其中，源端设备可以是手机、平板电脑、PC、手表等。目标设备可以是平板电脑、PC、电视机等。
可以理解的是，源端设备和目标设备还可以有其它称呼，比如，源端设备可以称为主动投屏设备，目标设备可以称为被动投屏设备；或者，源端设备可以称为第一电 子设备，目标设备可以称为第二电子设备等等，本申请对此不作限定。
如图1所示，以源端设备为手机、目标设备为PC为例。在手机和PC属于同一局域网络且完成信任互认之后，手机与PC属于同一信任环，也就是，手机和PC之间可以建立用于流转消息的通信通道。
在一些实施例中，手机中可以安装有多类应用程序，手机运行期间，可以接收各应用程序推送的应用消息，该应用消息来自于应用程序对应的消息服务器。
示例性地，手机中可以安装有社交应用，手机可以接收社交应用推送的应用消息，如，社交应用中的联系人小明所发送的消息“你今天去跑步吗？”。
在接收到应用程序推送的应用消息之后，如果手机处于解锁且亮屏的状态，手机可以在当前界面显示提示信息。比如，在接收到社交应用推送的应用消息时，如果手机的当前界面为主界面101，那么在主界面101中显示通知气泡102，该通知气泡102用于显示手机所接收到的应用消息。
然后，手机可以通过与PC之间用于流转消息的通信通道，将上述应用消息(比如，社交应用推送的“你今天去跑步吗？”)流转到PC。
在一些实施例中，在PC接收到来自手机的应用消息之后，可以显示对上述应用消息的提示信息。其中，上述提示信息可以采用消息通知控件的形式显示于PC的桌面，示例性地，上述消息通知控件可以是悬浮窗口、消息气泡、消息卡片等，当然，本申请实施例中，对于其显示形态不作具体限定。比如，图1所示，在桌面103中显示消息卡片104，该消息卡片104用于显示上述应用消息的相关内容。其中，相关内容可以是该应用消息来自手机，也可以是该应用消息来自于社交应用，还可以是该应用消息的主要内容等。可以理解地，本申请实施例中对于消息卡片104的显示位置不做具体限定。
在图1所示的场景下，如果用户正在使用PC，那么桌面103中显示的消息卡片104可以提醒用户关注到手机当前接收到了应用消息。这样，即使用户当前未直接使用手机，或者手机不在身边，也能够了解该应用消息的简要内容，避免消息的遗漏。
在一些实施例中，上述消息卡片104可以具有显示时限。比如，预先设置显示时限为时长1，PC响应于手机转发的上述应用消息，开始显示消息卡片104。在消息卡片104显示的时长超过时长1，且未检测到用户选中该消息卡片104，PC可以取消显示该消息卡片104。
示例性地，在PC的显示屏是触控屏时，PC检测到用户与触控屏中的显示区域1存在接触，PC确定用户指示选中消息卡片104。其中，上述显示区域1是触控屏上显示消息卡片104的区域。又示例性地，在PC已接入鼠标时，检测到鼠标对应的光标位于显示区域1内，并输入确认指令，PC确定用户指示选中消息卡片104。
在一些实施例中，在PC显示消息卡片104之后，用户可以通过操作PC的方式处理该应用消息，也可以选择直接操作手机的方式处理该应用消息。
其中，直接操作手机处理应用消息的方式可以参考相关技术。例如，手机可以响应于用户点击通知气泡102的操作，前台运行社交应用，并显示包含上述社交消息的应用界面。这样，用户可以通过在上述应用界面上的操作，回复该应用消息。再例如，手机响应用户指示启动社交应用的操作，显示社交应用的应用界面，这样，用户可以 通过在社交应用提供的应用界面上操作，查看上述应用消息或回复该应用消息等。
另外，选择操作PC处理应用消息，就会触发手机与PC之间的投屏。示例性地，用户可以在PC上选中消息卡片104。在PC可以检测到用户选中消息卡片104之后，PC可以遥控手机运行社交应用。在社交应用进入运行状态之后，PC可以在桌面103上显示投屏窗口。在PC显示投屏窗口之后，用户可以通过PC中的投屏窗口，操控手机，如，操作手机处理上述应用消息。
其中，上述投屏窗口，可以用于同步显示手机的当前界面。比如，在手机的当前界面是社交应用的应用界面时，该投屏窗口也显示该应用界面，此为同源投屏。也可以不同步显示手机的当前界面，比如，手机显示主界面，该投屏窗口显示应用界面，此为异源投屏。
请参阅图2，图2介绍异源投屏在上述场景中的界面显示。
作为一种实现方式，在PC确定用户选中消息卡片104之后，通过PC和手机之间的数据交互可实现异源投屏。如图2所示，在PC确定用户选中消息卡片104之后启动异源投屏，手机的显示屏上仍显示主界面101，PC的显示屏上的投屏窗口201显示上述应用消息对应的应用界面202。
此后，投屏窗口所显示的内容不会随着手机显示屏中所显示的内容同步变化。比如，请参阅图3，手机的显示屏上切换为显示信息应用界面303，PC的显示屏上的投屏窗口201仍显示上述应用消息对应的应用界面201。
请参阅图4，图4为上述图2和图3中异源投屏的原理示意图。异源投屏场景下，源端设备向目标设备投屏时，目标设备上的显示信息与源端设备前台的显示信息是不同步的。源端设备上显示应用B的窗口，源端设备向目标设备投屏应用B的窗口时，在创建的虚拟显示区域(VirtualDisplay)中显示应用B的窗口。其中，虚拟显示区域(如称为第一显示区域)的内容对用户不可见，可用于投屏到目标设备上显示。除了虚拟显示区域，源端设备上还包括用户可见显示区域(如称为第二显示区域)，源端设备将应用B的窗口投屏到目标设备时，用户可见显示区域显示应用A的窗口，即源端设备前台显示应用A的窗口。假设源端设备上用户可见区域从应用A的窗口切换为应用C的窗口，即源端设备前台显示应用C的窗口，此时，目标设备上仍然显示应用B的窗口。
可以理解，与同源投屏相比，异源投屏方式具有更好的隔离性。例如，异源投屏方式可以为用户提供独立的操控屏(即源端设备的显示屏和目标设备的显示屏)处理不同的应用。这样一来，用户在将应用B的窗口投屏到目标设备上显示之后，源端设备对应用A进行操作，或者由应用A的窗口切换为应用C的窗口，用户仍然可以在目标设备上针对应用B进行操作。所以异源投屏释放了源端设备的显示信息对目标设备的显示信息的控制，从而增强了多个设备的可复用性，允许用户通过投屏在不同设备上对不同应用进行操作。而同源投屏方式中，投屏窗口所显示的内容随着手机显示屏中所显示的内容同步变化。本申请实施例中，以异源投屏场景为例进行描述。
以上述图1和图2为例，描述了手机接收到消息通知之后，将其流转至PC，然后，用户操作PC上选中显示该消息通知的消息卡片，触发手机向PC进行异源投屏的过程。
在异源投屏期间中，如图5所示，PC显示投屏窗口的同时，手机也处于亮屏且解锁的状态。在此场景下，用户通过PC中的投屏窗口，与手机中的社交应用进行交互。也就是，用户只需操作PC，就可以同时使用PC和手机两台设备。在用户操作PC期间，用户与手机之间没有直接交互，而手机却长期处于亮屏和解锁状态。这样，不仅耗电，还容易造成隐私泄露。这时，用户往往会选择点击锁屏键，指示手机进入息屏状态。除此之外，手机长时间未检测到用户操作，也会自动进入锁屏状态。在相关实施例中，手机向PC投屏期间，也即，PC已正常显示投屏窗口(该投屏窗口中包含被投屏的应用界面)之后，作为源端设备的手机如果进入息屏状态，会直接影响PC中投屏窗口的显示，如图5所示，手机进入息屏状态之后，手机的显示屏会黑屏，与此同时，PC中的投屏窗口也会黑屏，这将会直接影响到用户通过PC遥控手机。需要说明的是，上述黑屏是指显示屏未点亮或者仅显示息屏屏保图像的状态。
在异源投屏期间中，如图6所示，PC显示投屏窗口的同时，手机也处于亮屏且解锁的状态。如果手机接收到用户指示锁屏的操作，如图6所示，手机进入锁屏状态，并显示锁屏图层，与此同时，PC中的投屏窗口也会显示锁屏图层，这将会直接影响到用户通过PC遥控手机。
为了解决上述问题，实现在源端设备锁屏或者息屏状态下，用户能够在目标设备继续启用应用投屏窗口，本申请实施例提供了一种投屏方法及电子设备，给用户提供便捷高效的投屏体验。
以源端设备是手机，目标设备是PC为例，如图1所示，PC在接收到手机流转过来的应用消息之后，可以在主界面103上显示对应的消息卡片104。当然，PC也可以接收同一信任环中任意设备流转过来的一条或多条应用消息。示例性地，PC可以响应用户操作，显示信息列表窗口，该信息列表窗口可以集中显示已接收到的消息通知，或者集中显示已接收且未处理过的消息通知。作为一种实现方式，上述消息通知也可以采用消息卡片的形式显示于信息列表窗口。
在一些实施例中，如果PC检测到指定操作，例如，检测到用户指示显示信息列表窗口的操作，可以在主界面上显示信息列表窗口。该信息列表窗口可以显示来自不同设备的应用消息等。可以理解地，信息列表窗口中所显示的应用消息均为PC近期接收到的、来自信任环中其他设备的信息。
与显示于桌面的消息卡片相同，在PC检测到用户选中信息列表窗口中任意一条消息卡片时，均可触发该消息卡片对应的设备进行投屏，比如，消息卡片a所显示的应用消息来自于信任环中的手机，在消息卡片a被选中的情况下，可以触发手机向PC进行投屏。
当然，PC指示对应的设备进行投屏之后，也可能出现投屏失败的情况。比如，消息卡片对应的应用程序受应用锁保护的情况下，导致投屏失败。再比如，用于投屏的通道被其他设备占用的情况下，导致投屏失败。再例如，网络不稳定的情况下，导致投屏失败等。
首先，应用锁指的是源端设备的用户为了保护某一应用的安全性或隐私性而针对该应用设置了保护，需要输入预先设置好的密码或指纹等才能打开该应用。出于对源端设备安全性和隐私性的考虑，一些可能的实施例中，实际投屏之前，会判断应用程 序a(例如，用户在PC上选中的消息卡片所对应的应用程序)是否被源端设备可选地增加了一层保护。如果源端设备为该应用程序a设置了应用锁，说明该应用程序a是源端设备的机主希望在每次访问时都需要输入密码或指纹等进行解锁的，那么对于这类强隐私保护的应用就不适用在源端设备处于锁屏息屏状态下直接在目标设备一侧启动投屏，这违背了用户设置应用锁的初衷。
示例性地，PC检测到用户选中消息卡片之后，向该消息卡片对应的设备(如，手机)发送查询信息1，该查询信息1用于查询手机是否为信息应用配置有应用锁。如果手机中为信息应用配置有应用锁，那么手机会向PC发送反馈信息1，该反馈信息1指示信息应用当前受应用锁保护。这样，PC在接收到反馈信息1之后，可以显示提醒信息1，该提醒信息1用于提示用户被选中的消息卡片对应的应用程序受应用锁保护。例如，可以是文字“信息应用受应用锁保护，请前往手机解除应用锁”。另外，提醒信息1可以以悬浮窗或者卡片的形式显示在PC上，本申请实施例对提醒信息的显示形式不作限定。
其次，源端设备与目标设备可以通过WLAN或者WI-FI P2P等方式建立投屏连接，投屏连接的建立具体可参考上文源端设备和目标设备建立通信连接的方式，在此不予赘述。为了方便描述，以下实施例以WI-FI P2P方式建立投屏连接。
在本申请的一些实施例中，PC检测到用户选中消息卡片之后，判断该消息卡片对应的源端设备(也即，手机)与目标设备(也即，PC)的投屏通信通道(Wi-Fi P2P通道)是否已经被其他业务占用。其中，源端设备与目标设备通过投屏通道建立投屏连接进行数据传输，从而实现两个或多个电子设备之间实现内容的镜像。当源端设备与目标设备已经建立了基于WI-FI P2P的其他业务，例如PC协同、设备分享等，那么源端设备与对端设备对应的投屏通道就被占用，导致源端设备无法基于WI-FI P2P向目标设备进行投屏。
例如，PC确定与手机之间的投屏通道(Wi-Fi P2P)被占用，启动投屏失败，显示提醒信息2，该提醒信息2示例性的提示用户“投屏连接冲突，请断开与其他设备之间的连接后再试”，以此提示用户前往源端设备(手机)或目标设备(PC)断开与其他设备之间的投屏连接。其中，提醒信息2可以以悬浮窗或者卡片的形式显示在PC上，本申请实施例对提醒信息的显示形式不作限定。
再次，在本申请的一些实施例中，在PC检测到用户选中消息卡片之后，判断该消息卡片对应的源端设备(也即，手机)的网络状态是否满足投屏要求。其中，网络状态包括流表项匹配状态、数据包执行端口负载状态、丢失/转发/收到数据包的总数量或数据包TTR失效等类似情况，只要是网络中可能出现的任何网络状态都包括在内，在此不作限定。可以理解，网络状态会影响投屏的服务质量，PC可选的存在满足投屏条件的最低网络状态阈值。如果PC确定当前网络状态低于这一最低网络状态阈值，则认为当前网络状态无法满足投屏要求，导致投屏失败。
在PC确定当前网络状态不满足投屏要求，启动投屏失败，提醒信息3。该提醒信息3可以提醒用户待网络稳定后再尝试触发源端设备进行投屏。示例性的提示用户“网络不稳定，请重试”，以此提示用户调整和优化网络状态。
可以理解的是，以上三种场景仅作为示例，不作为对本申请实施例的限定。PC判 断对应应用程序是否被应用锁保护、投屏通道是否被占用、网络状态是否满足投屏要求不作为本申请实施例必须具备的操作，同时判断顺序也不作限定。在一些实施例中，请参阅图7，图7为一种可能的判断机制的流程图。PC响应于用户点击消息卡片的操作，在各个判断节点依次判断是否设置了应用锁、投屏通道是否被占用、网络状态是否满足投屏要求，示例性的，只要有一个判断结果触发上述启动投屏失败的场景，此轮判断终止。用户按照提示调整之后，可以再次点击消息卡片，触发新的一轮判断。当然，在一些实施例中，判断机制的判断顺序可以调整，并且可以在某一判断节点触发启动投屏失败之后，判断流程停留在该判断节点，并持续检测是否满足进入下一判断节点的条件。用户按照提示进行调整，当满足进入下一判断节点的条件之后，判断流程自动进入下一判断节点。
可以理解地，图7举例了PC检测到用户选中消息卡片之后，PC可以先检验对应的应用程序是否受应用锁保护(可称为应用锁检测)，再检验用于投屏的通信通道是否被占用(可称为通道检测)，最后检验网络条件是否满足要求(简称网络检测)。实际上，本申请实施例中，不限定上述多项检测的先后顺序，也就是，在一些实施例中，PC还可以先检验用于投屏的通信通道是否被占用，再检验对应的应用程序是否受应用锁保护，最后检测网络条件。
在其他可能的实施例中，PC还可以同步进行应用锁检测、通道检测和网络检测，在得到的检测结果指示至少一项检测项目触发投屏失败时，显示对应的提醒信息。例如，如果得到的测结果指示应用程序受应用锁保护、通信通道被占用，PC会同步显示提醒信息1和提醒信息2。
当然，在检测结果指示上述检测项目均未触发投屏失败的情况下，PC确定手机已具备投屏的条件，这样，如图8所示，PC可以显示投屏窗口201，该投屏窗口201用于显示手机中社交应用的应用界面。在投屏失败的情况下，PC可以显示对应的提醒信息。
在另一些实施例中，源端设备可以启用分布式认证功能。也就是，在源端设备(如，手机)启用分布式认证功能，且配置有解锁密码的情况下，如果目标设备(PC)经过前述几项检测(如，应用锁检测、通信通道检测及网络检测)，确定源端设备已具备投屏的条件，那么目标设备可以显示投屏窗口，此时的投屏窗口用于显示解锁验证界面。其中，上述解锁密码可以是用于解除手机锁屏状态的认证密码。
例如，PC检测到用户选中消息卡片104之后，如果确定手机已具备投屏条件，PC可以显示投屏窗口201。上述投屏窗口201可以先显示解锁验证界面。这样，在PC接收到用户输入的手机解锁密码，且解锁密码验证通过之后，控制投屏窗口201切换显示“提示等待的信息”。在显示“提示等待的信息”期间，手机响应于PC的指示，启动社交应用，并在虚拟界面中绘制社交应用的应用界面，得到虚拟界面中各个显示对象的绘制参数。其中，上述虚拟界面是面向PC的服务，其在手机的显示屏上不可见。上述绘制参数是用于渲染界面的相关参数，在一些示例中也可以称为显示信息。
在手机完成上述应用界面的绘制之后，将对应的显示信息发送给PC，指示PC在投屏窗口201中渲染上述应用界面，这样，上述投屏窗口201切换显示社交应用的应用界面(如，包含消息卡片104对应的应用消息的应用界面)，也即，手机将虚拟界 面投屏到PC的投屏窗口201中。也就是，在PC接收到用户输入的手机解锁密码，且解锁密码验证通过之后，手机开始向PC进行异源投屏。
作为一种实现方式，在源端设备首次向目标设备投屏的场景下，如果源端设备未设置解锁密码，目标设备无需在投屏窗口中显示解锁验证界面，而是指示源端设备直接进行异源投屏。在此场景下，目标设备可以直接显示“提示等待的信息”。待源端设备将需投屏的应用界面绘制完成之后，目标设备再控制上述投屏窗口显示该需投屏的应用界面。
在源端设备首次向目标设备投屏的场景下，如果源端设备设置了解锁密码，也就是说，用户直接使用源端设备之前，也需要进行密码认证。这种情况下，目标设备确定源端设备具备投屏的条件之后，可以触发分布式认证。关于分布式认证的描述可参考上文，在此不再赘述。
另外，需要说明的是，源端设备首次向目标设备进行异源投屏，且设置了解锁密码，那么无论源端设备处于锁屏息屏状态还是解锁状态，当在目标设备指示源端设备进行异源投屏时都会触发分布式认证机制。
需要说明的是，源端设备首次在目标设备上启动投屏，可以指的是源端设备在同一目标设备上第一次启动异源投屏，也可以指的是在目标设备开机至当次关机的使用期间内第一次启动异源投屏。第二种解释具体来说，假设目标设备第一次开机，使用一段时间后目标设备第一次关机，第一次开机至第一次关机为第一使用期间。目标设备第二次开机，使用一段时间后目标设备第二次关机，第二次开机至第二次关机为第二使用期间。在第一使用期间，源端设备第一次在目标设备上启动异源投屏属于第二种解释中所指的“首次”启动；在第二使用期间，源端设备第一次在目标设备上启动异源投屏也属于第二种解释中所指的“首次”启动。
作为一种实现方式，在源端设备非首次向目标设备投屏的场景下，也即，源端设备在同一目标设备上曾经启动过异源投屏的场景下，又或者是，在目标设备开机至当次关机的使用期间内，源端设备在目标设备上启动过异源投屏。
第二种解释具体来说，在第一使用期间，源端设备第一次在目标设备上启动异源投屏，待源端设备第二次在目标设备上启动异源投屏，此时目标设备将判定源端设备在目标设备上启动过异源投屏；此后的第三次、第四次等亦如此。在第二使用期间，源端设备第一次在目标设备上启动异源投屏，由上文可知，此时目标设备将判定源端设备首次在目标设备上启动异源投屏；待源端设备第二次在目标设备上启动异源投屏，此时目标设备将判定源端设备在目标设备上启动过异源投屏；此后的第三次、第四次等亦如此。如果源端设备在目标设备上启动过异源投屏，为了在保障用户隐私安全的前提下，减少用户输入密码的次数，可选的，目标设备可以先判断距离上次通过分布式认证的时间是否超过预设时长阈值。
示例性地，距离上次通过分布式认证的时间没有超过预设时长阈值时，可以不用重新进行安全认证。需要说明的是，预设时长阈值可以是默认值，也可以是用户自行设置的值，它表明在这个时间范围内，源端设备和目标设备之间的投屏连接仍满足保护用户隐私安全的需求。所以当目标设备确定距离上次通过分布式认证的时间没有超过预设时长阈值时则直接启动投屏，无需显示解锁验证界面，这样能够避免反复要求 用户输入密码导致操作繁琐，在保障用户隐私安全的同时提高用户使用异源投屏的便捷性。
又示例性地，距离上次通过安全认证的时间超过预设时长阈值时，需要重新进行分布式认证。可以理解地，当目标设备确定距离上次通过分布式认证的时间超过预设时长阈值时，则认为此时源端设备和目标设备之间的投屏连接已经不满足保护用户隐私安全的需求，这种情况下的分布式认证流程与上文中“源端设备首次在目标设备上启动异源投屏”的分布式认证流程相同，在此不再赘述。
示例性的，假设源端设备设置了解锁密码，此前未在目标设备上启动过异源投屏。在目标设备确定用户指示源端设备进行投屏，且确定源端设备具备投屏条件时，可以触发分布式认证机制，通过分布式认证后，成功启动第一次异源投屏；在T1时间第一次异源投屏结束；假设用户所设的预设时长阈值为1小时，在T2时间目标设备再次检测到用户指示同一源端设备进行异源投屏；目标设备确定源端设备在目标设备上启动过异源投屏，进而判断T2距离T1的时间是否大于1小时；目标设备确定(T2-T1)的值小于1小时，直接启动第二次异源投屏。
另外，在手机完成向PC的异源投屏之后，如图8所示，手机显示主界面101，PC上投屏窗口201。其中，投屏窗口201中显示有被投屏的应用程序的应用界面，如，社交应用的应用界面202。
在手机接收到用户指示息屏的操作的情况下，或者，手机长期未接收到用户操作而自动息屏的情况下，手机的显示屏进入假灭屏状态，PC的投屏窗口201继续显示应用界面202。
在手机接收到用户指示锁屏的操作的情况下，或者，手机长期未接收到用户操作而自动锁屏的情况下，手机显示锁屏界面，PC的投屏窗口201继续显示应用界面202。
下面结合图9介绍本申请实施例提供的一种电子设备的系统结构。
分层架构将软件分成若干个层，每一层都有清晰的角色和分工。层与层之间通过软件接口通信。需要说明的是，图9的右半部分是源端设备的软件架构图，比如可以是一个手机。图9的左半部分是目标设备的软件架构图，比如可以是一个平板电脑(Pad)或者PC。
在一些实施例中，如图9的右半部分所示，源端设备的软件框架可以包括应用程序层、能力服务层、应用程序框架层(framework，FWK)、安卓运行时(Android runtime)和系统库、驱动层。
(1)应用程序层
应用程序层可以包括一系列应用程序包，例如社交，信息，通话，相机等应用程序(也可以称为应用)。示例性的，社交应用包括微信、QQ等，信息应用包括短信等。为方便描述，以下将应用程序简称为应用。源端设备上的应用可以是原生的应用(如在源端设备出厂前，安装操作系统时安装在源端设备中的应用)，也可以是第三方应用(如用户通过应用商店下载安装的应用)，本申请实施例不予限定。
(2)能力服务层
能力服务层提供实现相应服务的能力支撑，如图9所示，可以包括投屏助手模块、第一虚拟化服务模块、第一投屏服务模块、第一设备发现认证连接模块等。
其中，投屏助手模块可以是用于与其他电子设备(如目标设备)交互投屏相关信息的模块，实现投屏端到端的逻辑控制及连接管理等，并用于生成会话密钥(SessionKey)以保证源端设备和目标设备之间通信会话安全。投屏助手模块可以包括投屏管理模块、异源切换模块、虚拟化服务初始化模块等功能模块。投屏管理模块用于负责管理投屏相关事务，比如添加和移除VirtualDisplay的相关处理逻辑设置。其中，VirtualDisplay也可称为虚拟显示区域、虚拟屏或虚拟Display等。异源切换模块用于控制源端设备由异源投屏方式切换为同源投屏方式。虚拟化服务初始化模块为启动异源投屏的虚拟化服务做初始化设置。
第一虚拟化服务模块用于实现对音视频流的投屏端到端逻辑控制，负责源端设备和目标设备之间传输的Speaker、Mic、Camera等产生的音频、视频数据码流。
第一投屏服务模块可以包括第一显示虚拟化模块和第一反向控制模块，分别提供投屏能力(即源端设备的内容投屏到目标设备显示)和反向事件(即通过目标设备触发的对源端设备进行控制的事件)控制基础能力。第一投屏服务模块接收投屏管理模块的指令，按照指令提供相应的投屏服务，从而实现将本机运行的程序投屏至其他电子设备(如目标设备)。第一投屏服务模块将反控和输入法注入到目标设备的显示器(以下称为外屏)，并在创建虚拟显示区域时，添加虚拟显示标签(如，Flag1)，使得外屏只显示其本身的内容，而不镜像源端设备的显示器(以下称为主屏)的内容，从而实现异源投屏。
第一设备发现连接认证模块负责发现、连接、认证等工作，用于控制驱动层实现源端设备和目标设备之间的靠近发现、鉴权、连接等功能，可以包括第一通信模块和第一安全认证模块。第一通信模块，提供源端设备和目标设备之间信令传输通道，用于向驱动层下发建立源端设备和目标设备之间的通信连接和投屏连接的指令。同时，第一通信模块还用于在建立好传输通道后，启动投屏助手模块。第一安全认证模块用于安全认证，负责机主识别和启动分布式解锁认证，获得信任环，保证用户的隐私安全。
(3)应用程序框架层(FWK)
应用程序框架层为应用程序层的应用程序提供应用编程接口(Application Programming Interface，API)和编程框架。应用程序框架层包括一些预先定义的函数。如图9所示，应用框架层可以包括活动管理服务(ActivityManagerService，AMS)、窗口管理服务(WindowManagerService，WMS)、多屏框架模块、SystemUI模块等t。
AMS负责管理Activity，负责系统中各组件的启动、切换、调度及应用程序的管理和调度等工作。对于每一个Activity，在AMS中都会有一个与之对应的应用记录(ActivityRecord)，这个ActivityRecord记录了该应用的Activity的状态。每个活动Activity在AMS中都会创建对应的ActivityRecord对象，AMS可以利用这个ActivityRecord作为标识，调度应用的Activity进程；这些ActivityRecord对象被管理在各自的任务栈中，每个任务栈可以添加多个ActivityRecord对象；所有的任务栈都被ActivityStack统一管理，负责TaskStack的顺序、入栈和出栈。具体的，AMS中定义了分别用来保存进程(Process)、活动(Activity)和任务(Task)的数据类。其中，进程(Process)对应的数据类可以包括进程文件信息、进程的内存状态信息和进程中包含的 Activity、Service等。Activity信息可以保存在ActivityStack中。其中，ActivityStack用于统一调度应用程序Activity。ActivityStack具体可以保存所有正在运行的Activity(即final ArrayList mHistory)信息，如界面配置信息。例如正在运行的Activity可以保存在newArrayList中。ActivityStack还可以保存历史运行过的Activity的信息，如界面配置信息。需要注意，Activity并不对应一个应用程序，ActivityThread才对应一个应用程序。因此Android允许同时运行多个应用程序，实际是允许同时运行多个ActivityThread。在Android中，Activity调度的基本思路是这样的：各应用进程在要启动新的Activity或者停止当前的Activity时，向AMS报告。AMS在内部为所有应用进程都做了记录，当AMS接到启动或停止的报告时，首先更新内部记录，然后再通知相应客户进程运行或者停止指定的Activity。由于AMS内部有所有Activity的记录，因此能够调度这些Activity，并根据Activity和系统内存的状态自动关闭后台的Activity。
WMS承载着和“界面”有关的数据和属性，用于管理和“界面”有关的状态，即用于管理在手机屏幕上使用的图形用户界面(graphical user interface，GUI)资源，例如用于管理窗口程序和事件派发。其中，管理窗口程序是指根据应用程序的显示请求在应用服务端和WMS的协助下有序地输出给物理屏幕或其他显示设备。事件派发是指将来自键盘、物理按键、触摸屏、鼠标、轨迹球(TraceBoll)等的用户事件派发给相应的控件或窗口。管理窗口程序还可以获取显示屏大小，判断是否有状态栏，锁定屏幕，截取屏幕等，并负责管理应用窗口的显示方式，包括窗口显示的坐标大小、窗口显示层级等。具体可包括：窗口的创建和销毁、窗口的显示与隐藏、窗口的布局、焦点的管理以及输入法和壁纸管理等。在本申请的一些实施例中，WMS可以保证在手机锁屏息屏状态下的投屏窗口可见性。
需要说明的是，AMS的任务栈管理功能用于管理Activity的生命周期以及Activity之间的调度。WMS的生命周期管理功能用于管理窗口的生命周期。二者相互配合可以通过回调(onResume)接口激活窗口，激活后的窗口在前台显示；或者，通过暂停(onPause)接口暂停窗口以使窗口从前台切换到后台显示。以任务栈中的最上层的窗口(top Activity)为例，top Activity调用onResume以激活，那么top Activity在前台显示，当新的窗口入栈所述任务栈中，且新的窗口位于原top Activity上层时，原top Activity调用onPause暂停窗口以使原top Activity从前台切换到后台显示，对于新的窗口可以使用类似的原理管理生命周期。
多屏框架模块负责实现异源投屏逻辑，调用PendingIntetRecord接口启动PendingIntet对应的Acvtivity(指定行为send),通过添加异源投屏标签(如，Flag2)将该Activity对应的应用启动至虚拟显示区域。其中，PendingIntent是Android提供的一种用于外部程序调起自身程序的能力，生命周期不与主程序相关，当满足某些条件或触发某些事件后执行指定的行为。多屏框架模块还用于监听感应事件并与投屏助手模块进行交互，比如，调用ActivityRecord监听Activity是否在前台且可见，并将监听结果通知投屏助手模块；调用Task监听虚拟显示区域是否栈空并将监听结果通知投屏助手模块。多屏框架模块可以用于对原始输入事件进行翻译、封装等处理，得到包含更多信息的输入事件，并发送到WMS，WMS中存储有每个应用程序的可点击区域(比如控件)、焦点窗口的位置信息等。因此，WMS可以正确的将输入事件分发到指定 的控件或者焦点窗口。多屏框架模块可以对接收到的输入事件做出响应，将输入事件转发给WMS，WMS分发给相应的应用，应用向AMS发送注册请求，应用启动。
另外，多屏框架模块还用于在用户点击最近任务卡片或者桌面图标时将投至外屏的应用切换回主屏，此时如果投至外屏的应用采用的是异源投屏方式，在将该应用切换回主屏时需调用上文中的异源切换模块，将异源投屏方式切换为同源投屏方式，否则该应用无法在主屏上显示。多屏框架模块还用于在源端设备锁屏状态下，去除安全图层。在一些实施例中，多屏框架模块还可以用于在投屏场景下，对手机的电源键作相关修改。
其中，SystemUI模块是为用户提供系统级别的信息显示与交互的一套UI组件，主要用于实现状态栏信息显示(比如电池，wifi信号，3G/4G等icon显示)、通知面板(比如系统消息、第三方消息通知)、近期任务栏显示面板(比如显示近期使用的应用)等。在本申请实施例中，SystemUI模块负责通知服务，用于将源端设备消息通知流转到目标设备。
可理解，应用程序框架层还可以包括内容提供器，视图系统，电话管理器，资源管理器，通知管理器等(图9中未示出)。其具体含义可以参考相关技术文档，在此不展开说明。
(4)安卓运行时和系统库
安卓运行时负责系统的调度和管理。安卓运行时包括核心库和虚拟机。其中，核心库包含两部分：一部分是编程语言(例如，java语言)需要调用的功能函数，另一部分是系统的核心库。应用程序层和应用程序框架层运行在虚拟机中。虚拟机将应用程序层和应用程序框架层的编程文件(例如，java文件)执行为二进制文件。虚拟机用于执行对象生命周期的管理，堆栈管理，线程管理，安全和异常的管理，以及垃圾回收等功能。
系统库可以包括多个功能模块。例如：表面管理器(Surface Manager)，媒体库(Media Libraries)，三维图形处理库(例如：OpenGL ES)，二维图形引擎(例如：SGL)等。表面管理器用于对显示子系统进行管理，并且为多个应用程序提供了二维(2-Dimensional，2D)和三维(3-Dimensional，3D)图层的融合。媒体库支持多种常用的音频，视频格式回放和录制，以及静态图像文件等。媒体库可以支持多种音视频编码格式，例如:MPEG4，H.264，MP3，AAC，AMR，JPG，PNG等。三维图形处理库用于实现3D图形绘图，图像渲染，合成，和图层处理等。2D图形引擎是2D绘图的绘图引擎。
(5)驱动层
驱动层是Android操作系统的基础，Android操作系统最终的功能都是通过驱动层完成。如图9所示，驱动层包括底层驱动，负责发现、认证、连接等工作，比如，驱动层接收应用框架层中通信模块下发的命令，执行连接、断开等动作。具体而言，驱动层包括WI-FI P2P芯片、蓝牙芯片等，可建立WI-FI P2P、蓝牙等传输通道。当然，驱动层还可以包括硬件驱动比如显示屏驱动、摄像头驱动、音频驱动、传感器驱动、虚拟卡驱动等。
需要说明的是，本申请提供的图9所示的电子设备的软件结构示意图仅作为一种 示例，并不限定Android操作系统不同分层中的具体模块划分，具体可以参考常规技术中对Android操作系统软件结构的介绍。另外，本申请提供的投屏的方法和装置还可以基于其他操作系统实现，本申请不再一一举例。
在一些实施例中，如图9的左半部分所示，目标设备的软件框架可以包括PC管家、操作系统、驱动层。目标设备的部分功能模块以及功能模块的作用可以参照上述源端设备，此处不再赘述。
具体来说，目标设备的PC管家包括消息中心模块、第二虚拟化服务模块、第二投屏服务模块、第二设备发现连接认证模块。
其中，消息中心模块用于与源端设备的消息及信令进行交互接收从源端设备流转过来的消息通知，并负责向源端设备发起会话指令(openSession)。同时，消息中心模块还用于在建立会话(session)后，下发启动投屏服务的指令。
第二虚拟化服务模块、第二投屏服务模块、第二设备发现认证连接模块的功能可分别参考上述源端设备中的第一虚拟化服务模块、第一投屏服务模块、第一设备发现连接认证模块，其中不同的是，第二投屏服务模块用于接收并解析其他设备发送的视频流和相关信息，从而在本机显示在其他设备上运行的应用。
目标设备的操作系统可以是Windows OS(如个人计算机)，也可以是Android OS(如平板电脑)。
目标设备的驱动层请参考上述源端设备。
基于图9所示的电子设备的系统结构，以手机为源端设备(如称为第一设备)、PC为目标设备(如称为第二设备)举例，结合图10至图12说明前述手机在亮屏且解锁的状态下，向PC投屏的过程。
示例性的，手机端包括第一通信模块、投屏助手模块、第一虚拟化服务模块、第一投屏服务模块、第一安全认证模块、FWK(含SystemUI模块)、应用；PC端包括消息中心模块、第二投屏模块、第二通信模块、第二安全认证模块。需要说明的是，图9所示的其他功能模块也参与了本申请实施例提供的一种投屏的方法的具体实现过程，未在本时序图中示出。
如图10所示，上述投屏方法可以包括以下步骤：
S101：PC的消息中心模块接收到用户选中手机的操作。
可以理解的是，S101仅作为PC与手机建立第一通信连接的一种示例，PC检测到与其处于同一局域网的设备，显示所有可连接设备供用户选择，用户在PC上选中手机以启动后续的连接行为。启动PC和手机建立第一通信连接还可以有其他方式，比如手机检测到与其处于同一局域网的设备，显示所有可连接设备供用户选择，用户在手机上选中PC以启动后续的连接行为。又比如，手机与PC也可以通过“碰一碰”功能、NFC功能等建立近场连接，此处不做限定。
S102：消息中心模块向第二通信模块发送指示与手机建立通信连接的消息，该消息中携带手机的设备标识码。
其中，用户在PC搜索发现的附近可用协同通知设备中，选中了手机作为协同通知设备。PC的消息中心模块接收到用户输入的选中操作后，可以向第二通信模块发送一个进程间通信(inter-process communication，IPC)消息，可选地，该IPC消息可以 携带手机的设备名称、设备标识码、MAC地址等，用于指示第二通信模块与手机建立通信连接。
S103：第二通信模块与手机的第一通信模块建立第一通信连接。
其中，示例性的，第一通信连接可以是蓝牙连接(BR连接)，用于实现PC与手机之间的协同通知，即手机上的消息可以通过蓝牙通道流转到PC上。
S104：第二通信模块向消息中心模块返回建立第一通信连接成功的消息。
S105：手机的应用接收到一条应用消息。
在PC和手机建立第一通信连接之后，手机上接收到由安装在手机上的应用推送的一条应用消息，该应用消息在手机上的应用、SystemUI模块以及PC上的消息中心模块中可以有不同的表现形式，此处统称为应用消息。
S106：应用向SystemUI模块发送应用消息，该应用消息携带消息内容和应用包名。
S107：SystemUI模块指示第一通信模块向PC转发应用消息。
示例性的，本申请实施例中，在PC与手机建立第一通信连接之后，SystemUI模块会基于第一通信连接将所有手机接收到的消息流转到PC上；也可以选择性地流转，比如带有某种特殊标签的消息会流转到PC，或者来自特定应用的消息会流转到PC，或者判断重要程度较高的消息会流转到PC，等等，此处不做限定。
S108：第一通信模块基于第一通信连接向第二通信模块发送应用消息，该应用消息携带消息内容和应用包名。
S109：第二通信模块向消息中心模块发送应用消息，该应用消息携带消息内容和应用包名。
S110：消息中心模块接收到应用消息，控制显示屏显示对应的消息通知控件。
其中，PC与手机建立第一通信连接之后，手机上收到的应用消息会流转到PC。可以理解的是，此时手机处于解锁亮屏状态或者锁屏、息屏状态都不影响消息通知在PC和手机之间的流转。PC接收到应用消息后，显示与所述应用消息相关联的消息通知控件。
在一些实施例中，目标设备接收到对消息通知控件的用户操作，目标设备与源端设备建立第三通信连接，目标设备向源端设备发送启动异源投屏请求，源端设备响应于该请求，在目标设备显示应用投屏窗口，也即，如图11和图12所示，上述方法还可以包括：
S201：消息中心模块接收到用户点击消息通知控件的第一操作。
其中，消息通知控件可以称为第一控件，该第一控件显示有第一应用的应用消息，如称为第一消息。
S202：消息中心模块向第二通信模块发送指示创建第二会话的消息，该消息携带业务通道名称(WI-FI P2P通道)和手机的设备标识码。
其中，示例性的，业务通道名称可以为WI-FI P2P通道，设备标识码为手机的标识码，上述创建第二会话的指令用于指示第二通信模块在PC和设备标识码对应的手机之间建立WI-FI P2P通道。
S203：第二通信模块与第一通信模块建立第三通信连接。
示例性的，第三通信连接可以是WI-FI P2P连接，提供投屏服务的数据传输通道。
需要说明的是，上述PC和手机之间所建立第一通信连接、第二通信连接、第三通信连接的类型不限于蓝牙连接和Wi-Fi P2P连接，也可以是其他类型的连接方式，只要能满足两者之间传输数据的要求即可。
S204：第一通信模块向投屏助手模块发送指示启动的消息。
S205：投屏助手模块启动。
第一通信模块还负责与投屏助手的交互，投屏助手模块在手机无投屏需求时处于关闭状态，由第一通信模块在有投屏需求时触发投屏助手模块启动。
S206：第二通信模块向消息中心模块返回建立第三通信连接成功的消息。
S207：基于第三通信连接，消息中心模块与投屏助手进行参数协商，确定会话密钥和对端的IP地址。
参数协商包括业务侧协商SessionKey并获取对端设备的IP地址。其中，SessionKey也称为数据加密密钥或者工作密钥，是保证用户跟其它计算机或者两台计算机之间安全通信会话而随机产生的加密和解密密钥，它可由通信用户之间进行协商得到。它一般是动态地、仅在需要进行会话数据加密时产生。
S208：消息中心模块向第二投屏服务模块发送指示启动的消息，该消息携带会话密钥和手机的IP地址。
S209：第二投屏服务模块启动，根据会话密钥和手机的IP地址进行初始化。
S210：第二投屏服务模块向消息中心模块返回初始化已完成的消息。
由上文可知，投屏服务模块提供投屏能力和反向事件控制基础能力。PC端的投屏服务初始化成功后，为接收并解析手机端发送的相关信息提供能力支撑，以实现在PC端显示在手机端运行的应用。同时反向事件控制基础能力为实现异源投屏提供支撑。所以在进行异源投屏服务之前需要投屏服务模块进行初始化，以保证正常提供投屏能力和反向事件控制基础能力。
S211：消息中心模块向第二投屏服务模块发送指示启动投屏服务的消息。
S212：基于第三通信连接，第二投屏服务模块向投屏助手模块发送启动投屏服务的消息。
需要说明的是，步骤S207包括第二投屏服务模块向第二通信模块发送消息，第二通信模块再通过第三通信连接将该消息传输给第一通信模块，第一通信模块再向投屏助手模块发送该消息。为了方便描述，以下“基于第三通信连接”均包含第一通信模块和第二通信模块之间的数据传输过程，下文不再赘述。另外，上述启动投屏服务的消息又可称为第一请求，在用户选中显示第一消息的第一控件时，该第一请求可以携带第一标识。
S213：投屏助手模块绑定投屏服务。
可理解，投屏助手模块中的投屏管理模块绑定相关的投屏服务以管理投屏事务。
S214：投屏助手模块向第一虚拟化服务模块发送指示初始化的消息。
可理解，投屏助手模块中的虚拟化服务初始化模块用于指示虚拟化服务模块进行初始化。
S215：第一虚拟化服务模块完成初始化。
第一虚拟化服务模块负责投屏服务中的音视频虚拟化，手机端的第一虚拟化服务模块与PC端的第二虚拟化服务模块交互可以实现在投屏过程中，被投屏到PC上的手机应用可以使用PC的Speaker、Mic、Camera，并在手机和PC之间传输所产生的音频、视频数据码流，以此实现投屏时接打音视频电话等功能。
S216：第一虚拟化服务模块向第一投屏服务模块发送指示启动投屏服务的消息。
S217：第一投屏服务模块启动。
S218：第一投屏服务模块创建虚拟显示器，并为其添加虚拟显示标签。
需要说明的是，投屏服务模块是实现异源投屏方式的重要功能模块。具体的实现方式是手机端的第一投屏服务模块在创建虚拟显示区域时添加Flag1：VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY(虚拟显示标签，又可称为第二标签)，用于实现PC上的投屏窗口只显示其本身的内容，而不会镜像手机显示屏所显示的内容，以此保证手机的显示内容不会影响PC的投屏窗口的显示内容。
这样，在手机使用过程中，手机显示其他应用界面或桌面，不会影响PC的投屏窗口的显示，PC的投屏窗口能够维持本身的内容而不与手机的显示内容同步。同时，投屏服务模块提供的反向事件控制基础能力能够实现在电脑端反向控制手机端。当然，异源投屏方式的实现还需要PC端的第二投屏服务模块与手机端的第一投屏服务模块配合交互。
S219：第一投屏服务模块向投屏助手模块发送已启动投屏服务的消息。
S220：基于第三通信连接，第一投屏助手向消息中心模块发送已启动投屏服务的消息。
S221：基于第三通信连接，消息中心模块向FWK发送启动应用进行投屏的消息，携带应用包名。
其中，所携带的应用包名，是S201中被用户选中的消息通知控件(如，消息卡片104)所对应的应用程序(如，社交应用)的包名，可以理解的，如图1所示，消息卡片104用于显示来自手机中社交应用的应用消息，可视该消息卡片104与手机的社交应用对应。其他消息通知控件与应用程序之间的对应关系同理。
S222：FWK启动应用投屏窗口，并为应用包名对应的所有Activity标记投屏标签。
在一些实施例中，FWK启动应用投屏窗口的过程可参考相关技术，会启动应用包名对应的应用程序，并将该应用程序的应用界面绘制于虚拟显示区域(第一显示区域)，生成显示数据，具体不再赘述。与相关技术不同的是，在启动应用投屏窗口过程中，FWK可以为上述应用包名(也即，启动应用进行投屏的消息中携带的应用包名)对应的所有Activity标记投屏标签(第一标签)，上述应用包名对应的所有Activity可以是应用包名指示的应用程序所对应的所有Activity，例如，图1和图2所示，由于用户在PC选中消息卡片104，PC向手机发送的启动应用进行投屏的消息中会携带社交应用的应用包名，这样，手机执行S222时，需要将社交应用的所有Activity标记投屏标签。这样，通过该投屏标签，表征该应用包名指示的应用正在进行异源投屏。
S223：FWK向第一投屏服务模块发送应用投屏的显示数据。
S224：基于第三通信连接，第一投屏服务模块向第二投屏服务模块发送应用投屏的显示数据。
S225：第二投屏服务模块控制显示屏将上述显示数据渲染到投屏窗口。
其中，上述投屏窗口又可称为第一窗口，渲染到第一窗口的显示数据可以是图像信息，比如，手机针对第一应用进行投屏时，上述第一窗口中渲染的显示数据可以是第一应用对应的第一图像信息。后续实施例中，均以第一应用进行投屏为例，进行描述。
S226：第二投屏服务模块向消息中心发送投屏窗口的应用投屏界面渲染完成的消息。
S227：消息中心模块控制显示屏显示应用投屏窗口。
至此，PC显示应用消息所对应的应用投屏窗口，也即，手机在亮屏且解锁的状态下，向PC投屏成功。
示例性的，如图1和图2，当PC接收到用户点击消息卡片104的操作，PC上会显示上述社交应用对应的投屏窗口201，用于显示应用消息所对应的应用界面202。上述应用投屏窗口可以是用于显示应用界面202的投屏窗口201。
在手机向PC投屏成功之后，也即，S227之后，可能会面临手机进入息屏或锁屏状态的场景。例如，用户通过手机的锁屏键，指示手机进入息屏或锁屏状态。再例如，手机长时间未接收到用户操作，自动进入息屏或锁屏状态。
下面结合图13描述，在投屏期间，手机由亮屏、解锁状态进入息屏或锁屏状态时，手机内部各功能模块交互的过程。
示例性地，手机还可以包括PowerManagerService、KeyguardServiceDelegate(锁屏管理服务)、MultiDisplayPowerManager(多屏框架模块)、ActivityTaskManagerService(活动栈管理服务)、KeyguardController(锁屏控制器)、ActivityRecord(活动记录器)、Task(任务栈)、RootWindowsContainer(根窗口容器)、WindowsCastManager(窗口外观管理器)、SurfaceFlinger(图层合成器)。
如图13所示，上述方法可以包括：
S301，PowerManagerService接收到指示锁屏的操作。
其中，PowerManagerService负责手机系统的电源管理，常见的功能有：点亮屏幕、熄灭屏幕、进入屏保等，PowerManagerService继承自SystemService，由SystemServer启动、注册到系统服务中,通过Binder和其他组件进行交互。
在一些实施例中，PowerManagerService通过手机的锁屏键(也即，Power键)接收到用户的按压操作之后，可以确定接收到指示锁屏的操作。
S302，PowerManagerService向KeyguardServiceDelegate发送锁屏指令。
在一些实施例中，上述KeyguardServiceDelegate是用于代理KeyguardService的类。PowerManagerService通过向KeyguardServiceDelegate发送锁屏指令，指示启动锁屏加载流程。
S303，KeyguardServiceDelegate向MultiDisplayPowerManager发送power管理信息。
在一些实施例中，KeyguardServiceDelegate确定手机正与PC之间的投屏业务正在进行时，通过power管理信息，指示MultiDisplayPowerManager接替PowerManagerService，管理手机系统的电源。
当然，除了接收到用户指示锁屏的操作之外，手机自动息屏或者手机接收到用户指示息屏的操作时，也可以触发指示MultiDisplayPowerManager接替PowerManagerService，管理手机系统的电源。也就是，在一些可能的场景中，上述方法可以不含S301～S302，但是包括以下S304～S306。
S304，PowerManagerService确定显示屏自动息屏。
在一些实施例中，确定显示屏自动息屏的过程可参考相关技术，在此不再赘述。
S305，PowerManagerService检测到虚拟显示屏的投屏标签，中断针对虚拟显示屏的息屏处理。
在一些实施例中，PowerManagerService检测到虚拟显示屏的投屏标签，说明当前手机正在向其他设备投屏，且投屏正在进行中。在此场景下，为了避免自动息屏对投屏效果产生影响，PowerManagerService可以中断针对虚拟显示屏的息屏处理。
S306，PowerManagerService向MultiDisplayPowerManager发送power管理信息。
在一些实施例中，通过发送上述power管理信息，可以指示MultiDisplayPowerManager接替PowerManagerService，管理手机系统的电源。
可以理解的，上述S301～S303与上述S304～S306描述两种不同的场景下，触发向MultiDisplayPowerManager发送power管理信息。在手机是进入锁屏状态的场景下，执行S301～S303，在手机接入息屏状态的场景下，执行上述S304～S306。上述S301～S303与上述S304～S306之间并无关联。
S307，MultiDisplayPowerManager控制显示屏进行假灭屏状态。
在此需要说明的是，上述假灭屏状态、灭屏状态、亮屏状态，是显示屏的状态类型。其中，亮屏状态和灭屏状态相比，不仅显示屏的LCD上下线的状态不同(如，亮屏状态下LCD处于上电状态，灭屏状态下LCD处于下线状态)，内部相关的功能服务(如，AOD服务、POWER服务)对应的配置参数也不同。假灭屏状态与亮屏状态相比，仅仅是显示屏的LCD上下线的状态不同(如，亮屏状态下LCD处于上电状态，假灭屏状态下LCD处于下线状态)，但是，内部相关的功能服务(如，AOD服务、POWER服务)对应的配置参数是相同的。假灭屏状态与灭屏状态相比，二者的LCD上下线的状态相同，但是内部相关的功能服务(如，AOD服务、POWER服务)对应的配置参数也不同。
通常灭屏状态、亮屏状态由PowerManagerService进行管理，在本申请实施例中，在MultiDisplayPowerManager接替PowerManagerService进行电源管理之后，该MultiDisplayPowerManager可以控制显示屏进入假灭屏状态。
在一些实施例中，MultiDisplayPowerManager可以响应于上述power管理信息，将手机的显示屏由亮屏状态切换至假灭屏状态，切换过程中，MultiDisplayPowerManager只需通过TP服务，控制LCD下电即可，无需改变相关服务的状态、配置。
在PowerManagerService指示MultiDisplayPowerManager接替电源管理，并且MultiDisplayPowerManager也完成控制显示屏由亮屏状态切入假灭屏状态之后，上述方法还可以包括：
S308，PowerManagerService指示ActivityTaskManagerService配置应用投屏窗口 前台可见。
S309，ActivityTaskManagerService向KeyguardController发送指示将带有投屏标签的Activity设置为OnResume状态的消息。
需要说明的是，ActivityTaskSupervisor通过调用KeyguardController监听Activity是否处于OnResume状态，如果是的话则说明该Activity所对应的应用投屏窗口启动至PC后对用户可见且可与用户交互(OnResume状态)；如果Activity处于停止状态(OnPause状态)，则说明该Activity所对应的应用投屏窗口启动至PC后在后台，用户不能进行操作。其中，OnResume状态和OnPause状态为Activity的生命周期中的两种状态。当Activity处于OnResume状态时，该Activity处于可见且能够与用户交互的激活状态，用户可以获得activity的焦点；当Activity处于OnPause状态时，该Activity被另一个透明或者Dialog样式的Activity覆盖，此时它依然与窗口管理器保持连接，系统继续维护其内部状态，它仍然可见，但它已经失去了焦点，故不可与用户交互。在手机处于锁屏或息屏状态时，AMS组件启动Activity的流程中，会将Activity的生命周期切换至onPause状态，导致Activity不能与用户交互，所以投屏出现异常。
S310，KeyguardController识别出带有投屏标签的Activity，将其设置为OnResume状态。
KeyguardController为锁屏控制器，识别出带有投屏标签的Activity并返回“ture”，以指示将该Activity设置为OnResume状态，保证在源端设备锁屏状态下，带有投屏标签的Activity所对应的应用投屏窗口在目标设备上可见。
S311，KeyguardController向ActivityRecord发送指示将带有投屏标签的Activity保持OnResume状态的消息。
S312，ActivityRecord控制带有投屏标签的Activity保持OnResume状态。
S313，ActivityRecord向Task发送指示校验的消息，指示Task校验位于任务栈栈顶的Activity状态。
S314，Task确定位于任务栈栈顶的Activity处于OnResume状态。
需要说明的是，一个应用可能对应一个或多个Activity，一个或多个Activity会被存放在一个或多个任务栈中。任务栈遵循后进先出的规律，位于任务栈栈顶的Activity最先被执行。为了保证即将被执行的Activity对应的应用投屏窗口前台可见，需要保证栈顶的Activity处于OnResume状态。
可以理解的是，步骤S313和S314相当于容错机制，校验位于任务栈栈顶的Activity是否处于OnResume状态，所以该两个步骤为可选项，即在另一些实施例中，也可以不包括S313和S314。
S315，Task指示RootWindowsContainer控制所有任务进入休眠状态。
S316，RootWindowsContainer遍历所有任务的Activity，跳过带有投屏标签的Activity，控制其他Activity进入休眠状态。
需要说明的是，除了带有投屏标签的Activity跳过休眠流程，其他activity都会被置于休眠状态。因此，RootWindowsContainer需要遍历所有任务对应的Activity，从中过滤出带有投屏标签的Activity并返回“false”，以指示该Activity不进入休眠；其他Activity则返回“true”，使其进入休眠流程。
示例性的，同一个任务栈中的Activity1和Activity2都被添加了异源投屏标记，RootWindowsContainer使Activity1和Activity2跳过休眠流程；而未添加异源投屏标记的Activity3和Activity4则被置于休眠状态。
当Activity1位于任务栈栈顶时，Task校验Activity1的状态，保证其处于OnResume状态，进而Activity1被异源投屏后在PC上对用户可见且在前台可被用户操作。
执行完Activity1，此时未休眠的Activity2递进到了任务栈栈顶，Task校验Activity2的状态，保证其处于OnResume状态，进而Activity2被异源投屏后在PC上对用户可见且在前台可被用户操作。其他被添加了异源投屏标记的Activity的执行流程依此类推，此处不再赘述。而未被添加了异源投屏标记的Activity会被RootWindowsContainer置于休眠状态，不会被启动至外屏显示。
通过上述步骤，PC上的应用投屏窗口不受手机状态的影响，依然可以保持前台可见性和可操作性，之后，继续如图13所示，上述方法还可以包括：
S317，MultiDisplayPowerManager向WindowsCastManager发送取消信息，上述取消信息指示在虚拟显示区域中取消绘制锁屏图层。
S318，WindowsCastManager确定不在虚拟显示区域中绘制锁屏图层。
S319，MultiDisplayPowerManager指示SurfaceFlinger不切换至同源投屏。
S320，SurfaceFlinger配置虚拟显示区不镜像可见显示区域中的显示内容。
其中，SurfaceFlinger的作用是接收多个来源的图形显示数据，将他们合成，然后发送到显示设备。比如打开应用，常见的有三层显示，顶部的statusbar、底部或者侧面的导航栏以及应用的界面，每个图层是单独更新和渲染，这些界面都是由SurfaceFlinger合成为一个应用图层显示数据刷新到硬件显示。
在本申请实施例中，SurfaceFlinger确定虚拟显示区域带有虚拟显示标签，则在合成应用图层时只获取虚拟显示区域上的图层，而不会再获取其他显示区域的图层与它合成，这样就能实现前述的异源投屏方式，即合成的应用图层只与虚拟显示区域的图层相关。请返回参阅图2，SurfaceFlinger不会获取可见显示区域的图层，所以即使可见显示区域里包括锁屏界面或者其他应用界面，都不会影响PC的投屏窗口所显示的界面。因为SurfaceFlinger只获取虚拟显示区域的图层，并将其以某种方式传递到PC的投屏窗口进行显示。
在SurfaceFlinger完成显示数据的合成之后，可以通第一投屏服务模块将其发出，指示PC显示对应的应用投屏窗口，具体可参照S223～S227。另外，在手机亮屏状态下，第一应用的应用界面可以绘制与虚拟显示区域，然后，由SurfaceFlinger合成对应的显示数据，如称为第一图像信息，在手机切入锁屏或息屏状态之后，由于经过前述步骤的处理，第一应用的应用界面依然可以绘制与虚拟显示区域，并由SurfaceFlinger合成对应的显示数据，如称为第二图像信息。由于上述第一图像信息和第二图像信息均对应于第一应用的应用界面，在一些场景下，二者对应的显示内容可以相同。
可理解，上述应用图层可以包括当前执行的Activity对应的应用界面。上述显示数据可以包括应用图层中不同的显示对象、显示对象在虚拟显示屏中的显示位置、大小等信息。
需要说明的是，虚拟显示区域为步骤S218所创建的标记Flag1的虚拟显示区域。 虚拟显示区域的内容对用户不可见，手机可将虚拟显示区域作为占用一定存储空间的画布使用。例如，请参阅图16，图16为启动第一应用投屏窗口的过程图。当启动投屏的第一请求所携带的应用包名对应应用1，那么PendingIntentRecord将应用1的所有Activity都打上投屏标签，如图16中的任务堆栈(由Task创建)所示。在执行应用1的Activity1(Activity1对应第一消息)时，如图16所示，手机在虚拟显示区域中绘制Activity1对应的第一应用界面，SurfaceFlinger确定虚拟显示区域带有虚拟显示标签，则只获取该虚拟显示区域中的第一应用界面图层，并生成对应的第一显示数据发送给目标设备。
目标设备收到第一显示数据后，根据第一显示数据绘制渲染与第一应用的第一消息相对应的应用投屏界面，在目标设备的投屏窗口显示。具体请参阅图12中的步骤S225-S227以及图7。此时目标设备的投屏窗口显示与第一显示数据相对应的应用投屏界面。
在执行应用1的Activity2时(Activity2被推到了任务栈栈顶)，手机在虚拟显示区域中绘制Activity2对应的第二应用界面，SurfaceFlinger确定虚拟显示区域带有虚拟显示标签，则只获取该虚拟显示区域中的第二应用界面图层，并生成对应的第二显示数据发送给目标设备。随后目标设备的投屏窗口显示与第二显示数据相对应的应用投屏界面，即由与第一显示数据相对应的应用投屏界面切换为与第二显示数据相对应的应用投屏界面，完成应用1的不同应用界面在投屏窗口中的切换。
示例性的，在本申请的一些实施例中，请参阅图17，图17为启动第二应用投屏窗口的过程图。在本申请实施例提供的投屏的方法中，当目标设备的投屏窗口显示第一应用对应的应用投屏界面时(如，目标设备的投屏窗口显示与第二显示数据相对应的应用投屏界面)，此时源端设备又收到来自第二应用的第二消息(也称，第二数据)。第二消息流转至目标设备后，目标设备显示第二消息对应的消息通知控件。目标设备接收到源端设备点击该消息通知控件的操作后，向源端设备发送启动投屏的第二请求。在源端设备锁屏或者息屏状态下，第一投屏服务模块接收到第二请求，创建第二虚拟显示区域(VirtualDispaly2)并添加虚拟显示标签。在执行应用2的Activity3(Activity3对应第二消息)时，如图17所示，手机在虚拟显示区域中绘制Activity3对应的第三应用界面，SurfaceFlinger确定虚拟显示区域带有虚拟显示标签，则只获取该虚拟显示区域中的第三应用界面图层，并生成对应的第三显示数据发送给目标设备。
目标设备收到第三显示数据后，根据第三显示数据绘制渲染与第二应用的第二消息相对应的应用投屏界面，在目标设备的投屏窗口显示。示例性的，目标设备的投屏窗口显示与第二显示数据相对应的应用投屏界面切换为与第三显示数据相对应的应用投屏界面，完成不同应用(如，第一应用和第二应用)的应用界面在投屏窗口中的切换。
下面结合图14和图15说明在另一些实施例中，源端设备如果启动了安全认证功能的情况下，前述实施例的具体实现方式。
在另一些实施例中，可选的，源端设备可以启用安全认证功能，保障已锁屏或息屏的源端设备的隐私性。在投屏窗口201显示应用界面202之前，还可以显示安全认证界面，并还可以显示过渡加载界面。这种情况下，上述步骤S201即消息中心模块接 收到用户点击消息通知控件的第一操作后，会先触发安全认证机制，具体实现过程请参阅图14、图15及以下说明。
如图14所示，上述方法还可以包括：
S401，当手机处于锁屏或者息屏状态时，消息中心模块接收到用户点击消息通知控件的第二操作。
需要说明的是，此处的第二操作与图11中步骤S201中的第一操作均为用户点击消息通知控件的操作，在本实施例中，由于增加了安全认证机制，所以在触发图11中建立第三通信连接之前，增加了图14和图15所示步骤。
可以理解的是，当手机锁屏或者息屏时是没有经过用户的安全认证的，这种情况下如果允许PC不通过安全认证直接启动投屏，那么用户的隐私安全就得不到保障。所以，在一些实施例中，从PC端启动投屏之前会触发安全认证机制以保证用户的隐私安全。
由于本申请实施例是在PC端直接启动投屏，所以会在PC端触发安全认证机制，而密码认证分布在手机端，所以PC端的第二安全认证模块需要把在PC端接收到的认证信息(密码)发往手机端的第一安全认证模块进行密码认证，手机端会向PC端返回认证结果。所以为了在手机和PC之间需要进行安全认证的信令传输，需要先建立传输通道。
S402，消息中心模块向第二通信模块发送指示创建第一会话的消息，该消息携带业务通道名称(蓝牙通道)和手机的设备标识码。
其中，示例性的，业务通道名称可以为蓝牙通道，设备标识码为手机的标识码，上述创建第一会话的指令用于指示第二通信模块在PC和设备标识码对应的手机之间建立蓝牙通道。
S403，第二通信模块与第一通信模块建立第二通信连接(蓝牙连接)。
示例性的，第二通信连接可以是蓝牙连接(BR连接)，提供安全认证的信令传输通道。
S404，第一通信模块向投屏助手发送指示启动的消息。
S405，投屏助手启动。
S406，第二通信模块向消息中心模块返回建立第二通信连接成功的消息。
S407，消息中心模块向第二投屏服务模块发送指示启动的消息。
S408，第二投屏服务模块启动。
S409，第二投屏服务模块向消息中心模块返回启动成功的消息。
需要说明的是，此时，手机端的投屏助手和PC端的第二投屏服务模块已启动，为实现接下来的投屏服务提供必要的功能。
在一些实施例中，如图15所示，上述方法还包括：
S501，消息中心模块向PC的第二安全认证模块发送安全认证请求。
S502，第二安全认证模块控制PC的显示屏显示安全认证窗口。
示例性的，在投屏窗口201显示安全认证界面，用户可在上述安全认证窗口输入密码。
可以理解的是，安全认证方式不限于输入密码进行安全认证，比如指纹解锁场景 下，采集指纹进行验证；语音解锁场景下，采集语音进行验证。此处不做限定，仅以密码解锁作为一种示例。
S503，第二安全认证模块接收到用户通过安全认证窗口输入的密码。
S504，第二安全认证模块向第二通信模块发送密码。
S505，第二通信模块基于第二通信连接向第一通信模块发送密码。
S506，第一通信模块向手机的第一安全认证模块发送密码。
可以理解的是，安全认证方式还可以为用户输入指纹进行指纹认证、用户输入语音进行语音认证等，对此不作限定。第一安全认证模块接收到PC端采集的密码，与用户在手机端设置的解锁密码进行比对，如果PC端采集的密码与手机端设置的解锁密码匹配，则认证成功，否则认证失败。
S507，第一安全认证模块密码认证成功。
在手机端密码认证成功则向PC端返回密码认证成功结果；在手机端密码认证失败则向PC端返回密码认证失败结果，可选的，PC端接收到密码认证失败结果，可在安全认证窗口提示用户密码错误，用户重新输入密码后再次向手机端发起安全认证请求进行密码验证。当然，可以设置为认证失败的次数超过密码错误次数阈值则挂起认证，密码错误次数阈值可以为用户自行设置或者为默认值。
S508，第一安全认证模块向第一通信模块发动认证成功的消息。
S509，第一通信模块基于第二通信连接向第二通信模块发送认证成功的消息。
S510，第二通信模块向第二安全认证模块发送认证成功的消息。
S511，第二安全认证模块向消息中心模块发送认证成功的消息。
S512，消息中心模块控制显示屏显示过渡加载窗口。
可以理解的是，认证成功则启动图11至图12的应用投屏步骤，成功启动应用投屏需要一定的时间，所以在显示安全认证窗口至显示应用投屏窗口之间，存在一定的时间跨度。可选的，为了提高用户观感和体验，可以在安全认证成功后，启动过渡加载窗口在目标设备上显示，可提示用户已通过安全认证，正在启动应用投屏。当经过图11至图12的应用投屏步骤成功启动应用投屏后，投屏窗口201显示过渡加载界面切换为应用界面。
在另一些实施例中，目标设备接收到用户指示对应的源端设备进行应用投屏之后，先判断是否需要进行安全认证。比如，源端设备是否首次在目标设备投屏。再比如，源端设备是否设置解锁密码，具体可参考前述实施例中的描述。
上文详细介绍了本申请实施例提供的投屏的方法及电子设备的示例。可以理解的是，电子设备为了实现上述功能，其包含了执行各个功能相应的硬件和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本申请能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。本领域技术人员可以结合实施例对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本申请实施例可以根据上述方法示例对电子设备进行功能模块的划分，例如，可以对应各个功能划分为各个功能模块，例如检测单元、处理单元、显示单元等，也可 以将两个或两个以上的功能集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。需要说明的是，本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
需要说明的是，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
本实施例提供的电子设备，用于执行上述投屏方法，因此可以达到与上述实现方法相同的效果。
在采用集成的单元的情况下，电子设备还可以包括处理模块、存储模块和通信模块。其中，处理模块可以用于对电子设备的动作进行控制管理。存储模块可以用于支持电子设备执行存储程序代码和数据等。通信模块，可以用于支持电子设备与其他设备的通信。
其中，处理模块可以是处理器或控制器。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，数字信号处理(digital signal processing，DSP)和微处理器的组合等等。存储模块可以是存储器。通信模块具体可以为射频电路、蓝牙芯片、Wi-Fi芯片等与其他电子设备交互的设备。
在一个实施例中，当处理模块为处理器，存储模块为存储器时，本实施例所涉及的电子设备可以为具有图9所示结构的设备。
本申请实施例还提供了一种计算机可读存储介质，计算机可读存储介质中存储了计算机程序，当计算机程序被处理器执行时，使得处理器执行上述任一实施例的投屏方法。
本申请实施例还提供了一种计算机程序产品，当该计算机程序产品在计算机上运行时，使得计算机执行上述相关步骤，以实现上述实施例中的投屏方法。
另外，本申请的实施例还提供一种装置，这个装置具体可以是芯片，组件或模块，该装置可包括相连的处理器和存储器；其中，存储器用于存储计算机执行指令，当装置运行时，处理器可执行存储器存储的计算机执行指令，以使芯片执行上述各方法实施例中的来电显示方法。其中，本实施例提供的电子设备、计算机可读存储介质、计算机程序产品或芯片均用于执行上文所提供的对应的方法，因此，其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果，此处不再赘述。
通过以上实施方式的描述，所属领域的技术人员可以了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个装置，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是一个物理单元或多个物理单元，即可以位于一个地方，或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该软件产品存储在一个存储介质中，包括若干指令用以使得一个设备(可以是单片机，芯片等)或处理器(processor)执行本申请各个实施例方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。以上实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (20)

  1. 一种投屏方法，其特征在于，应用于第一设备和第二设备，所述方法包括：
    在所述第一设备向所述第二设备投屏之后，所述第二设备显示第一窗口；
    在所述第一设备处于亮屏状态时，所述第二设备在所述第一窗口中显示来自于所述第一设备的第一图像信息，所述第一图像信息对应所述第一设备中第一应用的应用界面；
    在所述第一设备切换至锁屏或息屏状态之后，所述第二设备在所述第一窗口中显示来自所述第一设备的第二图像信息，所述第二图像信息也对应所述第一设备中第一应用的应用界面。
  2. 根据权利要求1所述的方法，其特征在于，在所述第一设备切换至锁屏或息屏状态之后，所述方法还包括：
    所述第一设备配置所述第一应用在生命周期中不休眠，以及配置所述第一应用的应用界面在生命周期中允许绘制于第一显示区域，所述第一显示区域是所述第一设备中不可见的显示区域；
    所述第一设备向所述第二设备发送所述第一应用的第二图像信息，指示所述第二设备在所述第一窗口中显示所述第二图像信息，所述第二图像信息是绘制于第一显示区域的数据。
  3. 根据权利要求1或2所述的方法，其特征在于，在所述第一设备向所述第二设备投屏之前，所述方法还包括：
    所述第一设备接收第一应用的第一消息；
    所述第一设备向所述第二设备转发所述第一消息；
    所述第二设备显示第一控件，所述第一控件用于显示所述第一消息；
    所述第二设备响应于用户对所述第一控件的操作，向所述第一设备发送第一请求，所述第一请求包括所述第一应用的第一标识；
    所述第一设备响应于所述第一请求，启动向所述第二设备进行投屏。
  4. 根据权利要求3所述的方法，其特征在于，在所述启动向所述第二设备进行投屏之后，所述方法还包括：
    所述第一设备创建第一显示区域，并启动所述第一应用；
    在所述第一应用的应用界面绘制于所述第一显示区域之后，所述第一设备根据所述第一显示区域中的图像数据，生成所述第一图像信息；
    所述第一设备向所述第二设备发送所述第一图像信息。
  5. 根据权利要求4所述的方法，其特征在于，在启动所述第一应用之前，所述方法还包括：
    所述第一设备给所述第一应用标记第一标签；
    在所述第一设备配置所述第一应用在生命周期中不休眠，以及配置所述第一应用的应用界面在生命周期中允许绘制于第一显示区域之前，所述方法还包括：
    所述第一设备确定所述第一应用具有所述第一标签。
  6. 根据权利要求2所述的方法，其特征在于，在所述第一设备切换至锁屏或息屏 状态之前，所述方法还包括：
    所述第一设备响应于用户指示锁屏或息屏的操作，仅控制所述第一设备的显示屏下电或仅在所述显示屏上显示锁屏图层，所述显示屏对应的服务运行状态保持不变。
  7. 根据权利要求2所述的方法，其特征在于，所述第一显示区域被标记有第二标签，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：
    在所述第一设备检测到满足自动息屏的条件时，确定所述第一显示区域具有所述第二标签；
    所述第一设备仅控制所述第一设备的显示屏下电或仅在所述显示屏上显示锁屏图层，所述显示屏对应的服务运行状态保持不变。
  8. 根据权利要求2所述的方法，其特征在于，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：
    所述第一设备确定不在所述第一显示区域中绘制锁屏图层。
  9. 根据权利要求2所述的方法，其特征在于，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：
    所述第一设备确定所述第一显示区域不镜像第二显示区域中的内容，所述第二显示区域是所述第一设备中可见的显示区域。
  10. 一种投屏方法，其特征在于，应用于第一设备，所述方法包括：
    所述第一设备向第二设备进行投屏；
    在所述第一设备处于亮屏状态时，所述第一设备向所述第二设备发送第一应用对应的第一图像信息，指示所述第二设备在第一窗口上显示所述第一图像信息；
    在所述第一设备切换至锁屏或息屏状态之后，所述第一设备向所述第二设备发送所述第一应用对应的第二图像信息，指示所述第二设备在所述第一窗口上显示所述第二图像信息，所述第一图像信息和第二图像信息均为所述第一应用的应用界面。
  11. 根据权利要求10所述的方法，其特征在于，在所述第一设备切换至锁屏或息屏状态之后，所述方法还包括：
    所述第一设备配置所述第一应用在生命周期中不休眠，以及配置所述第一应用的应用界面在生命周期中允许绘制于第一显示区域，所述第一显示区域是所述第一设备中不可见的显示区域；
    所述第一设备根据所述第一显示区域中绘制的内容，生成所述第二图像信息。
  12. 根据权利要求10或11所述的方法，其特征在于，在所述第一设备向所述第二设备进行投屏之前，所述方法还包括：
    所述第一设备接收第一应用的第一消息；
    所述第一设备向所述第二设备转发所述第一消息；
    在接收到所述第二设备发送的第一请求之后，启动向所述第二设备进行投屏，所述第一请求是针对所述第一应用进行投屏的请求信息，所述第一请求携带所述第一应用的第一标识。
  13. 根据权利要求12所述的方法，其特征在于，在所述第一设备向所述第二设备发送第一应用对应的第一图像信息之前，所述方法还包括：
    所述第一设备创建第一显示区域，并启动所述第一应用；
    在所述第一应用的应用界面绘制于所述第一显示区域之后，所述第一设备根据所述第一显示区域中的图像数据，生成所述第一图像信息。
  14. 根据权利要求13所述的方法，其特征在于，在启动所述第一应用之前，所述方法还包括：
    所述第一设备给所述第一应用标记第一标签；
    在所述第一设备配置所述第一应用在生命周期中不休眠，以及配置所述第一应用的应用界面在生命周期中允许绘制于第一显示区域之前，所述方法还包括：
    所述第一设备确定所述第一应用具有所述第一标签。
  15. 根据权利要求11所述的方法，其特征在于，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：
    所述第一设备响应于用户指示锁屏或息屏的操作，仅控制所述第一设备的显示屏下电或仅在所述显示屏上显示锁屏图层，所述显示屏对应的服务运行状态保持不变。
  16. 根据权利要求11所述的方法，其特征在于，所述第一显示区域被标记有第二标签，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：
    在所述第一设备检测到满足自动息屏的条件时，确定所述第一显示区域具有所述第二标签；
    所述第一设备仅控制所述第一设备的显示屏下电或仅在所述显示屏上显示锁屏图层，所述显示屏对应的服务运行状态保持不变。
  17. 根据权利要求11所述的方法，其特征在于，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：
    所述第一设备确定不在所述第一显示区域中绘制锁屏图层。
  18. 根据权利要求11所述的方法，其特征在于，在所述第一设备切换至锁屏或息屏状态之前，所述方法还包括：
    所述第一设备确定所述第一显示区域不镜像第二显示区域中的内容，所述第二显示区域是所述第一设备中可见的显示区域。
  19. 一种电子设备，其特征在于，电子设备包括一个或多个处理器和存储器；所述存储器与处理器耦合，存储器用于存储计算机程序代码，计算机程序代码包括计算机指令，当一个或多个处理器执行计算机指令时，所述一个或多个处理器，用于执行如权利要求10-18中任一项所述的方法。
  20. 一种计算机存储介质，其特征在于，包括计算机指令，当所述计算机指令在电子设备上运行时，使得所述电子设备执行如权利要求10-18中任一项所述的方法。
PCT/CN2023/117850 2022-10-31 2023-09-08 一种投屏方法和电子设备 WO2024093530A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211348614.X 2022-10-31
CN202211348614.XA CN117950612A (zh) 2022-10-31 2022-10-31 一种投屏方法和电子设备

Publications (1)

Publication Number Publication Date
WO2024093530A1 true WO2024093530A1 (zh) 2024-05-10

Family

ID=90798627

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/117850 WO2024093530A1 (zh) 2022-10-31 2023-09-08 一种投屏方法和电子设备

Country Status (2)

Country Link
CN (1) CN117950612A (zh)
WO (1) WO2024093530A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113114840A (zh) * 2021-03-28 2021-07-13 武汉卡比特信息有限公司 一种基于Android手机互联的省电锁屏方法
CN113542825A (zh) * 2020-04-20 2021-10-22 华为技术有限公司 投屏显示方法、系统、终端设备和存储介质
WO2022105403A1 (zh) * 2020-11-20 2022-05-27 Oppo广东移动通信有限公司 跨设备的内容投屏方法、装置、设备及存储介质
CN115048067A (zh) * 2021-02-26 2022-09-13 华为技术有限公司 一种投屏显示方法及电子设备
CN115097926A (zh) * 2022-06-16 2022-09-23 Oppo广东移动通信有限公司 投屏方法、装置、设备、系统及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113542825A (zh) * 2020-04-20 2021-10-22 华为技术有限公司 投屏显示方法、系统、终端设备和存储介质
WO2022105403A1 (zh) * 2020-11-20 2022-05-27 Oppo广东移动通信有限公司 跨设备的内容投屏方法、装置、设备及存储介质
CN115048067A (zh) * 2021-02-26 2022-09-13 华为技术有限公司 一种投屏显示方法及电子设备
CN113114840A (zh) * 2021-03-28 2021-07-13 武汉卡比特信息有限公司 一种基于Android手机互联的省电锁屏方法
CN115097926A (zh) * 2022-06-16 2022-09-23 Oppo广东移动通信有限公司 投屏方法、装置、设备、系统及存储介质

Also Published As

Publication number Publication date
CN117950612A (zh) 2024-04-30

Similar Documents

Publication Publication Date Title
US10853053B1 (en) System, method, and computer program product for coordination among multiple devices
CN112269527B (zh) 应用界面的生成方法及相关装置
EP1589723B1 (en) A device capable of modifying its behaviour based on surrounding devices
US20240111473A1 (en) Distributed display method and terminal for application interface
EP4013003A1 (en) Communication protocol switching method, apparatus and system
CN114286165B (zh) 一种显示设备、移动终端、投屏数据传输方法及系统
KR101343587B1 (ko) 방향 정보를 이용한 데이터 전송 방법 및 상기 방법을 채용한 휴대 전자 기기
US11314894B2 (en) Multi-android system implementation method, non-transitory computer-readable storage medium, and terminal
CN114237779A (zh) 一种显示窗口的方法、切换窗口的方法、电子设备和系统
US20240086231A1 (en) Task migration system and method
WO2024093530A1 (zh) 一种投屏方法和电子设备
WO2024082871A9 (zh) 一种投屏系统和电子设备
WO2023005705A1 (zh) 一种支付方法、装置及终端设备
EP4315017A1 (en) Multi-screen management
CN116743971B (zh) 一种同源投屏方法及电子设备
CN114286320A (zh) 一种显示设备、移动终端及蓝牙连接方法
CN114339591A (zh) 基于超宽带芯片进行定位的方法和相关装置
CN115150644A (zh) 显示设备的应用唤醒方法、显示设备、移动终端及服务器
CN113542882A (zh) 一种唤醒待机显示设备的方法、显示设备及终端
CN112114883A (zh) 一种终端唤醒方法、终端及计算机存储介质
CN111163220A (zh) 一种显示方法及通信终端
CN111159734A (zh) 通信终端及多应用数据互访处理方法
WO2023207218A1 (zh) 一种来电提醒系统、方法及电子设备
WO2024066992A1 (zh) 一种多设备组网系统、方法及终端设备
WO2024027203A1 (zh) 通知消息的提示方法、电子设备及可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23884463

Country of ref document: EP

Kind code of ref document: A1