CN117135396A

CN117135396A - Screen projection method and related equipment thereof

Info

Publication number: CN117135396A
Application number: CN202310163718.1A
Authority: CN
Inventors: 李坤明
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2023-02-14
Filing date: 2023-02-14
Publication date: 2023-11-28

Abstract

The application provides a screen projection method and related equipment, and relates to the technical field of terminals, wherein the method comprises the following steps: responding to a first operation, and initiating screen throwing by a target application in first terminal equipment for the first time; the method comprises the steps that a first terminal device and a second terminal device establish screen-throwing connection, and screen throwing is carried out; starting from the screen-throwing ending time point, closing the target application after a first preset time length; responding to a second operation, and enabling the target application to initiate screen projection for the second time; in response to the third operation, cancelling the screen projection; responding to a fourth operation, and enabling the target application to initiate screen projection for the third time; and the first terminal equipment and the second terminal equipment establish screen-throwing connection and perform screen-throwing. The application can synchronize information of the delivery end and the receiving end when the screen is launched for a plurality of times through decision and management, and the screen is successfully launched.

Description

Screen projection method and related equipment thereof

Technical Field

The application relates to the technical field of terminals, in particular to a screen projection method and related equipment thereof.

Background

With the development of computer and communication technologies, the number of terminal devices owned by users is increasing. Screen sharing between terminal devices has become a daily requirement for users.

The screen-throwing system generally comprises a delivery end (source) and a receiving end (sink), and a common screen-throwing mode between the delivery end and the receiving end is mirror image screen throwing (Miracast). Miracast refers to that a delivery end delivers the display content of the whole screen to a corresponding receiving end for display, and no connecting line is needed. The data transmitted between the delivery end and the receiving end for screen casting in the Miracast mode needs to carry a wireless fidelity direct connection information element (wireless fidelity direct information element, WFD IE) to indicate that the data is used for Miracast screen casting.

However, in the actual application process, the data sent by the delivery end, which needs to carry the WFD IE, often does not carry the WFD IE for some reasons, so that information on two sides of the delivery end and the receiving end are not synchronous, and thus the screen dropping failure is caused. In this regard, a solution is needed.

Disclosure of Invention

The application provides a screen projection method and related equipment, which can enable information of a delivery end and a receiving end to be synchronous when screen projection is initiated for a plurality of times through decision and management, and the screen projection is successful.

In a first aspect, a screen projection method is provided, and is applied to a first terminal device, where the method includes:

responding to a first operation, and initiating screen throwing by a target application in the first terminal equipment for the first time;

The first terminal equipment and the second terminal equipment establish screen-throwing connection and perform screen-throwing;

starting from the screen-throwing ending time point, closing the target application after a first preset time period;

responding to a second operation, and initiating screen throwing by the target application for the second time;

in response to the third operation, cancelling the screen projection;

responding to a fourth operation, and enabling the target application to initiate screen projection for the third time;

and the first terminal equipment and the second terminal equipment establish screen-throwing connection and perform screen-throwing.

The embodiment of the application provides a screen projection method, which can enable the information of a delivery end and a receiving end to be synchronous when the terminal equipment of the screen projection end initiates the screen projection for the non-first time through decision and management, and the screen projection is successful. It should be understood that when the terminal device at the screen-throwing end initiates screen throwing for the first time, the information of the sending end and the receiving end are synchronous, and the screen throwing is successful.

With reference to the first aspect, in some implementations of the first aspect, a duration from a cancelling screen-throwing time point to a third initiation of the target application is greater than a second preset duration;

the second preset duration is determined by a detection mechanism set by the first terminal device to release the occupied P2P interface.

In the implementation mode, a detection mechanism is usually arranged on the terminal equipment, and synchronization of information at two ends of the screen can be influenced when the occupied P2P interface is released according to the detection mechanism, and after the P2P interface is released, screen connection can still be successfully established and screen can be carried out when screen is initiated again through decision and management.

With reference to the first aspect, in some implementations of the first aspect, a duration from a cancelling screen-throwing time point to a third initiation of the target application is less than the first preset duration;

the first preset duration is determined by a mechanism which is set by the first terminal equipment and used for ending the application background process.

In this implementation, from the point of canceling the screen-throwing time, the duration from the third time of initiating the screen-throwing by the target application is longer than the second preset duration but shorter than the first preset duration, so that it is known that the second preset duration is shorter than the first preset duration, that is, through decision and management, in the present application, it is possible to successfully re-initiate the screen-throwing in this case before closing the application after releasing the P2P interface.

With reference to the first aspect, in some implementations of the first aspect, the target application that initiates the screen casting for the second time is the same as the target application that initiates the screen casting for the third time, and the target application that initiates the screen casting for the second time is the same as or different from the target application that initiates the screen casting for the first time.

With reference to the first aspect, in some implementations of the first aspect, the first terminal device and the second terminal device both support Miracast screen-casting technology.

In a second aspect, a screen projection method is provided and applied to a first terminal device, wherein the first terminal device comprises an intelligent IE module, and the intelligent IE module comprises a control sub-module and a configuration management sub-module;

the method comprises the following steps:

after receiving the screen throwing instruction, the control submodule issues a first query instruction to the configuration management submodule, wherein the first query instruction is used for querying whether a target application issuing the screen throwing instruction belongs to an application allowing screen throwing and included in a white list;

when the query result returned by the configuration management sub-module to the control sub-module is yes, the control sub-module issues a second query instruction to the configuration management sub-module, wherein the second query instruction is used for querying a screen-throwing strategy corresponding to a target application issuing the screen-throwing instruction, and the screen-throwing strategy comprises intervention IE management and non-intervention IE management;

and the control submodule indicates to throw a screen based on a screen throwing strategy returned by the configuration management submodule to the control submodule.

In the implementation manner, the control sub-module can accurately identify the application behavior corresponding to the target application, and automatically judge whether the IE management needs to be intervened or not. When the intervention IE management is needed, the method can be combined with a subsequent method, and when the intervention IE management is not needed, the method can be processed according to a screen-throwing method provided by the related technology.

With reference to the second aspect, in certain implementations of the second aspect, the intelligent IE module further includes a data management sub-module; the control submodule indicates to throw the screen based on the screen throwing strategy returned by the configuration management submodule to the control submodule, and the method comprises the following steps:

when the screen projection strategy returned by the configuration management sub-module is intervention IE management, the control sub-module issues a management instruction to the data management sub-module, wherein the management instruction is used for indicating the intervention IE management;

after receiving the management instruction, the data management submodule acquires and records initial data corresponding to the target application;

the control submodule indicates screen projection based on the initial data.

In this implementation, the initial data is used to generate the WFD IE. The application can eliminate the dependence of WFD IE on upper layer application through decision and management, and actively maintain WFD IE.

With reference to the second aspect, in certain implementations of the second aspect, the smart IE module further includes a retry sub-module; the method further comprises the steps of:

after receiving the screen throwing instruction again, the control submodule issues a third query instruction to the data submodule, wherein the third query instruction is used for querying historical data corresponding to the target application;

The data management sub-module returns a query result to the control sub-module;

the control submodule determines whether to add WFD IE based on the query result;

when determining to add the WFD IE, the control submodule indicates to throw a screen and indicates to add the WFD IE;

when the control submodule receives a failure error code for failing to add the WFD IE, a retry instruction is issued to the retry submodule, wherein the retry instruction is used for indicating to regenerate a new WFD IE;

the control submodule instructs screen dropping and instructs addition of a new WFD IE.

It should be appreciated that the control sub-module may receive different failure error codes based on different failure reasons.

In the implementation mode, the method and the device can update the WFD IE information based on the initial data by judging the interaction failure reason of the delivery end and the receiving end, and realize the self-adaptive addition of the WFD IE.

In a third aspect, there is provided a terminal device comprising: one or more processors, memory; the memory is coupled with the one or more processors, the memory is used for storing computer program codes, the computer program codes comprise computer instructions, and the one or more processors call the computer instructions to cause the terminal device to execute any one of the screen projection methods of the first aspect or any one of the screen projection methods of the second aspect.

It should be appreciated that the extensions, definitions, explanations and illustrations of the relevant content in the first and second aspects described above also apply to the same content in the third aspect.

In a fourth aspect, an upgrade apparatus is provided, comprising means for performing any of the screen projection methods of the first aspect.

In a possible implementation, when the upgrade apparatus is a terminal device, the processing unit may be a processor, and the input unit may be a communication interface; the terminal device may further comprise a memory for storing computer program code which, when executed by the processor, causes the terminal device to perform any of the methods of the first aspect.

In a sixth aspect, there is provided a chip system comprising: and a processor for calling and running a computer program from the memory, so that the device mounted with the chip system executes either the screen projection method of the first aspect or the second aspect.

In a seventh aspect, there is provided a computer readable storage medium storing computer program code which, when run by a terminal device, causes the terminal device to perform any one of the method of projecting screens of the first aspect or any one of the method of projecting screens of the second aspect.

In an eighth aspect, there is provided a computer program product comprising: computer program code which, when run by a terminal device, causes the terminal device to perform any one of the screen projection methods of the first aspect or any one of the screen projection methods of the second aspect.

The embodiment of the application provides a screen throwing method, which is characterized in that an intelligent IE module is additionally arranged in a software system of a screen throwing end to make decisions and manage, and whether the bottom layer state needs to be switched can be determined according to the counted number of times of initiating screen throwing, so that after judgment, the bottom layer state can be kept under the counted condition of non-first screen throwing, the purpose of continuously adding WFD IE can be realized by the delivery end, the problem of screen throwing failure caused by inconsistent information of the delivery end and a receiving end is solved, and the screen throwing experience of a user is improved.

In addition, the screen projection method provided by the embodiment of the application does not need to be changed or influence the detection mechanism of the delivery end, and the screen projection method provided by the embodiment of the application has small influence on the original function of the delivery end.

Drawings

FIG. 1 is a schematic illustration of a scenario in which an embodiment of the present application may be used;

Fig. 2 is a schematic diagram of a WiFi-P2P active set;

fig. 3 is a software system diagram of a first terminal device provided by the present application;

FIG. 4 is a flow chart of an ideal screen projection method provided by the related art;

FIG. 5 is a timing diagram of an ideal projection screen corresponding to FIG. 4;

FIG. 6 is a timing diagram of an actual screen shot corresponding to FIG. 4;

FIG. 7 is a schematic diagram of a related display interface according to an embodiment of the present application;

FIG. 8 is a schematic diagram of another related display interface provided by an embodiment of the present application;

fig. 9 is a software system diagram of a first terminal device according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of the WiFi P2P service in fig. 9;

FIG. 11 is a timing diagram of a screen projection method according to an embodiment of the present application;

FIG. 12 is a flowchart of a screen projection method provided by an embodiment of the present application;

fig. 13 is an interaction diagram of software modules in the first terminal device provided by the embodiment of the present application;

FIG. 14 is a schematic diagram of a hardware system suitable for use in a terminal device of the present application;

fig. 15 is a schematic structural diagram of a terminal device provided by the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

1. The image data to be delivered refers to image data required to be projected. In the embodiment of the application, the image data to be delivered can comprise data such as audio, video, pictures, display interfaces, documents and the like.

2. The system comprises a delivery end and a receiving end, wherein the delivery end refers to terminal equipment for delivering image data to be delivered, and the receiving end refers to terminal equipment for receiving the image data to be delivered and playing or displaying the image data.

3. A third party Application (APP), which refers to an APP provided or developed by a software application developer other than the manufacturer of the electronic device. Such as you ku, ai qi yi, etc. The third party APP may provide the user with corresponding functionality by invoking one or more system functions in the electronic device. Illustratively, the cool, loving art may provide the user with the ability to view movie works, etc.

4. An application interface (UI), which refers to a media interface that interacts and exchanges information between an application or an operating system and a user, may enable conversion between an internal form of information and a form that the user may receive. The application interface is source code written in a specific computer language such as java, extensible markup language (extensible markup language, XML) and the like, and the interface source code is analyzed and rendered on the electronic equipment to finally be presented as content which can be identified by a user. A common presentation form of an application interface is a graphical application interface (graphic user interface, GUI), which refers to an application interface related to computer operations that is displayed in a graphical manner. It may be a visual interface element of text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc., displayed in a display of the electronic device.

5. Foreground operation, background operation and closing, and APP corresponds to three operation states: foreground operation, background operation and closing. When the APP runs in the foreground of the electronic equipment, the electronic equipment can display an application interface of the APP on a display screen, and a user can interact with the APP through a control in the application interface of the APP. The APP operates in the electronic device, but the case that the APP operates in the foreground is not the case that the APP operates in the background of the electronic device. The case where the APP is running in the background of the electronic device may include a process in which the APP exists in the electronic device, but an application interface of the APP is not displayed on the display screen. Because the application interface of the APP running in the background is not displayed on the display screen, the user cannot generally interact directly with the APP running in the background. Closing means that the APP is not running on the electronic device.

It should be understood that there may be one or more APP running in the foreground of the electronic device or one or more APP running in the background of the electronic device. Wherein, while one or more APP is operated in the foreground of the electronic equipment, one or more APP can also be operated in the background of the electronic equipment.

6. A process (process), which is a running activity of an application program on a certain data set, is a basic unit of resource allocation and scheduling by an operating system (e.g., an Android system). Each process occupies a memory space, and the application program runs on the operating system in the form of one or more processes to realize corresponding functions.

The foregoing is a simplified description of the terminology involved in the embodiments of the present application, and is not described in detail below.

Because the application relates to a screen projection scene, the screen projection scene applicable to the application is described first. Referring to fig. 1, fig. 1 shows a screen projection scene provided by an embodiment of the present application.

The screen-throwing scene generally comprises a first terminal device serving as a delivery end and a second terminal device serving as a receiving end, wherein the first terminal device displays display contents (including pictures, videos and audios and waiting for delivering image data) of the first terminal device on the second terminal device through a wireless screen-throwing technology. For example, when the wireless screen-throwing technology is mirror image screen-throwing Miracast, the first terminal device can perform screen-throwing recording on display content of the first terminal device by utilizing the mirror image screen-throwing technology, synchronously send recorded screen-throwing data to the second terminal device, and play the screen-throwing data by the second terminal device so as to finish screen-throwing.

The first terminal device and the second terminal device may be terminal devices such as a mobile phone, a television, a personal computer, a tablet computer, or a wearable device, and may be specifically determined according to actual needs, which is not limited in any way in the embodiment of the present application. For example, as shown in fig. 1, in the scenario, the first terminal device is a mobile phone, the second terminal device is a television, and when the mobile phone initiates a screen projection to the television in response to an operation of a user, the mobile phone is a delivery end, the television is a receiving end, and in the process, the mobile phone can display a video screen projection on the television through a mirror image screen projection Miracast technology.

It should be understood that the mirror image screen-cast Miracast technology referred to in the above application scenario refers to a wireless display standard developed by WiFi alliance based on WiFi direct (WFD) protocol. The delivery end (e.g., the first terminal device) supporting the wireless display standard directly plays the desktop, video, audio, etc. data on the receiving end (e.g., the second terminal device) through Miracast technology without any connection or transmission through a hotspot (AP).

WFD is based on the basic technology of a point-to-point connection (WiFi-P2P), that is, WFD (or WiFi-P2P) can support interconnection between terminal devices and high-speed transmission of data without a local area network or the Internet (Internet). Based on the description, before mirror image projection is performed between the first terminal device and the second terminal device, WFD IE needs to be added in a frame message transmitted between the first terminal device and the second terminal device according to the IEEE802.11 protocol before image data to be delivered is delivered, so as to indicate that the frame message is used for mirror image projection.

Here, the frame message generally refers to a search frame, which may include a Beacon frame (Beacon), a Request frame (Probe Request), a Response frame (Probe Response), and the like. That is, WFD IEs need to be added to frame messages such as Beacon, probe Request, probe Response, etc. transmitted between the first terminal device and the second terminal device.

Specifically, for example, in response to a user operation, when the first terminal device turns on the WiFi function to scan, a Probe Request may be sent to the peripheral device; after receiving the Probe Request, the second terminal device replies a Probe Response to the first terminal device. For the first terminal device and the second terminal device, the Probe Request and the Probe Response exist before delivering the image data to be delivered. In addition, the beacon can periodically send the first terminal device or the second terminal device to the other party for informing the opposite terminal device that the local terminal device is online. For example, the first terminal device periodically sends beacon to the second terminal device, so as to inform the second terminal device that the second terminal device is online; the second terminal device also periodically sends beacon to the first terminal device to notify the first terminal device that the first terminal device is online.

For example, in a WiFi hotspot, the SAP represents a hotspot sender, the STA represents a hotspot access party, and when the terminal device represents the STA, WFD IEs need to be added to the Probe Request and the Probe Response sent by the terminal device.

Illustratively, when a P2P group (group) is established after the search is completed, a WFD IE is appended to the Beacon it issues when the terminal devices within the P2P group act as the owner (GO) nodes.

It should be appreciated that according to the WiFi-P2P definition, a WiFi-P2P enabled terminal device needs to act as a P2P group owner node or a P2P Group Client (GC) node to form a P2P group. Referring to fig. 2, fig. 2 shows a schematic diagram of a WiFi-P2P active set. As shown in fig. 2, the 2 terminal devices supporting WiFi-P2P belong to the same group, and data can be transmitted between the 2 terminal devices, so as to realize a screen throwing function. One of the 2 terminal devices acts as a P2P group owner, functions as a conventional router, such as controlling the P2P group, enabling communication of the terminal devices, etc.; the other terminal device acts as a P2P group client, and the connection communicates with the terminal device acting as a P2P group owner. In this P2P group, the terminal devices acting as P2P group owners need to append WFD IEs in issuing beacons.

Based on the above scenario and flow, a software system related to the terminal device is briefly described below, and then a process of adding the WFD IE during wireless screen-on is described in connection with the software system.

Referring to fig. 3, fig. 3 illustrates a software system diagram of a terminal device provided in the related art. The terminal device may be, for example, the first terminal device shown in fig. 1 and 2.

As shown in fig. 3, the software system may be divided into four layers, from top to bottom, an application layer, an application framework layer (framework), a hardware abstraction layer (hardware abstract layer, HAL), and a kernel layer, respectively.

The application layer includes a drop application.

The application framework layer provides an application programming interface (application programming interface, API) and programming framework for application programs of the application layer. The application framework layer may include some predefined functions. For example, the application framework layer includes a WiFi P2P management module (WiFi P2P manager) and a WiFi P2P application module (WiFi P2P service), where the WiFi P2P manager is used to manage, and the WiFi P2P service is used to implement some specific operations in the screen-throwing process. Communication is performed between the WiFi P2P manager and the WiFi P2P service through an android interface definition language (android interface definition language, AIDL), i.e., an interface.

The hardware abstraction layer is used to abstract hardware, and the hardware abstraction layer may include wpa_supplicant. wpa_supplicant is a WPA requester, supporting WPA and WPA2.

The kernel layer is a layer between hardware and software. The kernel layer includes a network device driver (network device driver).

A workflow of screen projection from a first terminal device to a second terminal device provided by the related art is exemplarily described with reference to fig. 3.

Responding to the operation of a user, the first terminal equipment can initiate wireless screen throwing; after the initiation, the screen throwing application issues the identification of the screen throwing command, and the identification of the screen throwing command is transmitted through the WiFi P2P management module and the WiFi P2P application module and issued to the wpa_supplicant; after wpa_supplicant receives the identification of the command, the wpa_supplicant instructs network device driver to drive hardware to start scanning, and feeds back a scanning result upwards, wherein the scanning result comprises names of one or more terminal devices which can serve as a receiving end, for example, names of second terminal devices. In this process, wpa_supplicant will also establish a P2P interface.

In addition, when the wireless screen throwing is initiated, a value corresponding to the WiFi_DISPLAY_ON function included in the first terminal equipment is changed from false to true so as to indicate that the bottom layer state is switched to the state of initiating the wireless screen throwing. When the value corresponding to the wifi_display_on function is changed from false to true, the wpa_supper can call the bottom layer driving firmware through the established P2P interface to send a frame message, and when the frame message is sent, the wpa_supper can further append a WFD IE in the frame message to indicate that the frame message is used for wireless screen casting.

However, when the underlying state of the first terminal device changes, for example, due to the existence of the detection mechanism in the first terminal device, when no service is detected to be performed (such as sending or receiving data) ON the P2P interface for a long time, in order to avoid affecting the call of other services to the P2P interface, the underlying state is actively reset, so that the value corresponding to the wifi_display_on function is modified from true to false. When the value corresponding to the wifi_display_on function is modified from true to false, the wpa_supplicant releases the P2P interface, and the WFD IE is cancelled. If the subsequent application framework layer needs to call the P2P interface to send the frame message, the bottom layer will construct a new P2P interface for the application framework layer to use, so that the situation when the first terminal device sends the frame message after resetting the bottom layer state is different from the situation before resetting, and the WFD IE cannot be carried any more; in this way, the information on two sides of the first terminal device serving as the delivery end and the second terminal device serving as the receiving end are not synchronous, so that the basic functions of the delivery end are affected, for example, the problems that the screen is interrupted, the screen is not searched for the receiving end and the like can occur.

In view of this, the application provides a screen projection method, in which an intelligent IE module is added in a WiFi P2P service of a software system of a delivery end to make decision and manage, so as to determine whether to switch the bottom layer state according to the counted number of times of initiating screen projection; therefore, after judgment, the bottom layer state can be kept under the condition of non-first screen throwing of statistics, so that the purpose of continuously adding the WFD IE can be realized by the delivery end, the problem of failed screen throwing caused by inconsistent information of the delivery end and the receiving end is solved, and the screen throwing experience of a user is improved.

Next, in order to further explain the related art, an ideal screen projection and an actual screen projection process involved in the related art are described below with reference to fig. 4 to 8.

Referring to fig. 4, fig. 4 illustrates a flowchart of an ideal screen projection provided by the related art, and fig. 5 is a timing chart of the ideal screen projection corresponding to fig. 4; fig. 6 is a timing chart of actual screen projection corresponding to fig. 4.

As shown in fig. 4, the method 10 may include the following S101 to S105.

S101, referring to fig. 1 and fig. 5, at a first time point a1, in response to a first operation b1 of a user, the first terminal device initiates screen projection for the first time, and enters a scanning state.

The first terminal device is a terminal device supporting the Miracast screen-throwing technology, and the first terminal device is used as a delivery end to initiate screen throwing at a first time point a 1. Specifically, after receiving a first operation b1 of opening the screen throwing function by a user, the first terminal device operates or starts the screen throwing function.

The screen projection function can be a function built in a delivery end software system, or can be a function of an application program installed in the delivery end, and can be specifically determined according to an actual scene. The starting mode of the screen throwing function can be determined according to the actual scene. For example, the user may start the screen throwing function by operating the delivery end; the screen throwing function of the delivery end can be remotely started in a mode that other equipment sends a starting instruction to the delivery end.

For example, when the screen-casting function is a function built in the delivery-side software system, the screen-casting function may be set up among the system-setting functions of the software system. When the system is used, a user can operate and start the screen throwing function in system setting, or a shortcut starting mode can be provided for the screen throwing function through modes such as a desktop icon, a floating window or a pull-down notification bar. When the screen projection system is used, a user can quickly start the screen projection function by clicking a corresponding icon or area.

FIG. 7 is a schematic diagram of an associated display interface. As shown in fig. 7, a control 3 of "wireless screen throwing" is displayed in a drop-down menu bar of the first terminal device, and the first terminal device opens a screen throwing function in response to a click operation (i.e., a first operation b 1) of the control 3 by a user.

For example, when the screen throwing function is a function in the application program, the application program developer can set the starting mode of the screen throwing function according to actual requirements. For example, for video applications, a drop-in icon may be provided on the video playback interface; and the user clicks the screen throwing icon to realize the start of the screen throwing function.

After the first terminal equipment starts the screen throwing function, the first terminal equipment enters a scanning state and starts scanning nearby terminal equipment capable of throwing a screen. In combination with the software system architecture shown in fig. 3, after the first terminal device opens the screen-throwing function, the screen-throwing application will issue the identifier of the screen-throwing command, and after the identifier of the screen-throwing command is transmitted through the WiFi P2P manager and the WiFi P2P service, the identifier of the screen-throwing command is issued to the wpa_suppicant; wpa_supplicant instructs network device driver drive hardware to start scanning again and feeds back the scanning result upwards. Wherein the scanning result comprises the names of one or more terminal devices that can be receiving ends, for example the names of the second terminal devices.

In the scanning process, the first terminal device can call the bottom driving firmware to send a search frame to the nearby terminal device through the P2P interface created by the wpa_suppuration; meanwhile, WFD IE may be added to the transmitted search frame by wpa_supplicant. In addition, after the first terminal device initiates the wireless screen throwing, the bottom layer also updates the value corresponding to the wifi_display_on function from the flag to true, so as to indicate that the bottom layer state starts to be the state of initiating the wireless screen throwing. It should be understood that the value corresponding to the wifi_display_on function is used to indicate the bottom state, and is the non-screen-throwing state when the value is switched to the flag, and is the screen-throwing state when the value is switched to the true. It should also be appreciated that the value corresponding to the wifi_display_on function remains unchanged or remains true while scanning.

To illustrate that the first terminal device enters the scanning state, fig. 8 is an exemplary schematic diagram of a related display interface. As shown in fig. 8 (a), when the first terminal device initiates the wireless screen projection, a pop-up window corresponding to the wireless screen projection may be displayed on a display interface (such as a desktop) to prompt the user that the first terminal device is scanning at this time. When the first terminal device only searches one device capable of projecting a screen through the mirror image protocol, the first terminal device can directly project the image data to be projected to the device. When the first terminal device searches for a plurality of devices capable of being thrown on a screen through a mirror image protocol, the search result may be displayed on a display interface (such as a desktop) in a list form, such as a display interface shown in (b) in fig. 8, where the list includes names of one or more terminal devices capable of being thrown on the screen, such as names of a second terminal device: mono222. Then, after receiving the selection operation of the user, the first terminal device may screen the image data to be delivered onto the device selected by the user from the list.

S102, as shown in fig. 5, at a second time point a2, in response to a second operation b2 of the user, the first terminal device exits the scanning state. The second point in time a2 follows the first point in time a 1.

For example, in response to the user performing the operation of sliding up from the bottom side bar, the operation of sliding right from the left side bar, or the operation of sliding left from the right side bar (i.e., the second operation) with respect to the screen of the first terminal device, the first terminal device may switch the screen throwing process initiated in step S101 from the foreground operation to the background operation, at which time the first terminal device may be considered to exit the scanning state.

The first terminal device exits the scanning state but is still in the background running condition, and the screen-throwing application is not closed, so that if the user performs other operations, the first terminal device can also switch the process corresponding to screen throwing from the background running to the foreground running in response to other operations of the user, so that the process of process creation is omitted, and the purpose of continuously using the screen-throwing device is realized. Thus, in an ideal case, the underlying state should remain unchanged after the first terminal device exits the scanning state at the second point in time a 2. That is, in an ideal case, wpa_supplicant should continue to be able to invoke the P2P interface and should continue to have the ability to append WFD IEs in the search frame; meanwhile, the value corresponding to the WiFi_DISPLAY_ON function should be kept unchanged and kept true, so that the bottom layer state can be indicated to continue to throw screens.

S103, as shown in fig. 5, at a third time point a3, in response to a third operation b3 by the user, the first terminal device enters the scanning state for the second time. The third time point a3 follows the second time point a 2.

For example, in response to an operation (i.e., the third operation b 3) that the user switches the screen projection process from the background operation to the foreground operation, the first terminal device enters the scan state for the second time.

After the first terminal device exits the scanning state at the second time point a2, in an ideal case, the bottom layer state remains unchanged, at this time, when the scanning state is entered again at the third time point a3, since the process created in step S101 is still in, wpa_supplicant should continue to call the P2P interface, and WFD IE should be added in the search frame. In addition, when the scanning state is entered for the second time, the value corresponding to the wifi_display_on function should also remain unchanged and continue to true.

It should be noted that, in general, a process checking and killing mechanism also exists in the first terminal device, for example, for a screen-throwing application running in the background, after a first preset period of time (for example, 300 seconds) passes, a process corresponding to the screen-throwing application is killed, and the screen-throwing application is closed. Here, the duration between the second time point a2 and the third time point a3 should be less than the first preset duration, that is, before the screen throwing process is controlled to be closed by the process checking and killing mechanism, the user performs a third operation on the first terminal device, and the first terminal device responds to the third operation and enters the scanning state for the second time. The duration between the second time point a2 and the third time point a3 is, for example, 200 seconds.

It should be understood that the time of the first preset duration depends on the policy set by the manufacturer of the first terminal device to end the background process of the application, which is not limited in any way by the present application.

S104, as shown in fig. 5, at a fourth time point a4, in response to a fourth operation b4 of the user, the first terminal device exits the scanning state for the second time. The fourth time point a4 follows the third time point a 3.

The fourth operation b4 may be the same as the second operation b2, and may be, for example, an operation of sliding up from the bottom side rail, an operation of sliding right from the left side rail, or an operation of sliding left from the right side rail for the screen of the first terminal device; of course, the fourth operation b4 may also be different from the second operation b2, and will not be described herein.

Here, the procedure after the first terminal device exits the scan state for the second time is the same as the description in S102, and reference may be made to the description above, which is not repeated here.

S105, as shown in fig. 5, at a fifth time point a5, the screen-casting application is turned off. The fifth time point a5 follows the fourth time point a 4.

The duration between the fourth time point a4 and the fifth time point a5 is equal to a first preset duration, that is, after the first terminal device exits the scanning state for the second time and passes through the first preset duration (for example, 300 seconds), the corresponding process of screen throwing is killed, and the screen throwing application is closed.

Then, in an ideal case, the first terminal device should keep the bottom layer state unchanged after exiting the scanning state for the second time, and after a first preset period of time passes and no other operation is received, the screen-throwing application is closed in response to the searching and killing of the process searching and killing mechanism. After the screen-throwing application is closed, the bottom layer state is changed, namely the wpa_supper does not call the P2P interface any more, and the WFD IE is canceled; the value corresponding to the wifi_display_on function is also switched from true to false, so as to indicate the end of screen projection.

Referring to fig. 4 and fig. 5, steps S101 to S105 are an ideal screen-throwing process, and after the first terminal device starts the screen-throwing application, even if the first terminal device exits the scanning state and enters the scanning state for the second time in the middle, before the process corresponding to the screen-throwing is killed without meeting the killing condition, the first terminal device should maintain the bottom layer state unchanged, keep the WFD IE, and keep the value corresponding to the wifi_display_on function unchanged. The underlying state cannot be switched until after exiting the wireless application APP.

However, in actual situations, as shown in fig. 6, after S102, that is, after the first terminal device exits the scan state for the first time, due to the existence of the detection mechanism in the first terminal device, after a second preset time period passes from the second time point a2, for example, when the first terminal device detects that no service is performed (such as sending or receiving data) ON the P2P interface within the time period of the second preset time period, in order to avoid that the screen dropping initiated by S101 affects the call of other services ON the P2P interface, the first terminal device actively resets the bottom state, and modifies the value corresponding to the wifi_display_on function from true to false; meanwhile, wpa_supplicant releases the P2P interface, and WFD IE is canceled.

It will be appreciated that resetting the underlying state should occur before the first terminal device exits the application (e.g. the seventh time point a7 shown in fig. 6) after a first preset time period has elapsed after exiting the scan, and thus the second preset time period is less than the first preset time period, e.g. 150 seconds; it should be understood that the foregoing is merely an example, and in fact, it may be possible that the first terminal device changes the underlying state after exiting the scan state for the first time due to other reasons.

Subsequently, in S103, i.e. at a third point in time a3, the first terminal device enters the scanning state a second time in response to a third operation b3 by the user. The sixth time point a6 is between the second time point a2 and the third time point a 3.

At this time, since the state of the bottom layer has been changed at the sixth time point a6, wpa_supplicant does not call the P2P interface any more, and WFD IE is cancelled; the value corresponding to the wifi_display_on function is also switched from true to false, so that when the first terminal device enters the scanning state for the second time, the frame message sent by the first terminal device to the second terminal device does not carry the WFD IE any more by the third time point a 3. Then, when the second terminal device receives the frame message without the WFD IE, it is considered that other non-screen-throwing services are executed, so in the actual screen-throwing process, after the third time point a3, a problem of screen-throwing failure may occur between the first terminal device and the second terminal device.

In view of the above problems, referring to fig. 9 and 10, fig. 9 shows a software system diagram of a terminal device according to an embodiment of the present application. Fig. 10 is a schematic diagram of an internal structure of a WiFi P2P service included in the terminal device in fig. 9. The terminal device may be, for example, the first terminal device shown in fig. 1 and 2.

As shown in fig. 9, the level of the software system provided by the embodiment of the present application is the same as the software level architecture provided by the related art shown in fig. 3, and the software level architecture includes an application layer, an application framework layer, a hardware abstraction layer and a kernel layer from top to bottom.

The same as the software system shown in fig. 3, in the application framework layer, the WiFi P2P service provided by the embodiment of the present application may include a discovery module, a connection module, and a message interaction module; the discovery module is used for searching peripheral terminal equipment supporting screen casting based on the command of the WiFi P2P manager; a connection module for performing communication connection with the searched terminal device, such as a second terminal device; and the message interaction module is used for carrying out interactive transmission of messages with the terminal equipment which is in communication connection. It should be noted that the discovery module, the connection module, and the message interaction module are not illustrated in fig. 3.

Unlike fig. 3, the WiFi P2P service provided by the embodiment of the present application includes an intelligent IE module in addition to the discovery module, the connection module, and the message interaction module mentioned above.

The intelligent IE module is respectively in communication connection with the discovery module, the connection module and the message interaction module; the intelligent IE module is used for deciding whether the intelligent IE module needs to be operated or not based on the application behavior of the upper-layer screen-throwing application; and the system is also used for managing interaction with the discovery module, the connection module and the message interaction module under the condition of starting the intelligent IE module.

As shown in fig. 10, an intelligent IE module provided in an embodiment of the present application may include at least: the control submodule and the data management submodule can also comprise other submodules such as a configuration management submodule, a test tube management submodule and the like, and the intelligent IE module can be specifically designed and changed according to the requirements, so that the embodiment of the application is not limited in any way.

Wherein the control sub-module may implement at least one of the following functions:

1) The control submodule can be used for interacting with the upper and lower layers in the software system. For example, up may dock a projection application, down may dock a data management sub-module, a configuration management sub-module, a rearrangement management sub-module, and so on.

2) The control sub-module may make the determination based on the application behavior. For example, whether the intelligent IE module needs to be started for screen-throwing decision selection can be judged according to the application behaviors. Here, the application behavior may include whether the application program is to be screen-cast, the application program is switched to a foreground operation, a background operation, or a shutdown behavior.

3) The control sub-module may make overall decisions. For example, matching of the screen-casting device can be performed according to the history of the application user identity (user identification, UID); the decision can be made as to whether the data management sub-module, the configuration management sub-module, the reconfiguration management sub-module and the like contained in the system are operated or not; a command to add WFD IE information or a command to cancel the addition of WFD IE information may also be issued.

4) The control submodule may also perform whitelist filtering. For example, the whitelist refers to a list of a plurality of applications preset on the terminal device and allowed to be screened, based on which the control submodule can filter according to the whitelist, and only the plurality of applications included in the whitelist are allowed to be screened, and the WFD IE is added.

The data management sub-module can be used for acquiring, storing and recreating various original data (namely WFD IE); the method can also perform data operations such as data recording, data aging, historical application data reading and the like of the application UID and the application behavior.

It should be appreciated that WFD IEs typically include a WiFi alliance-specific organization unique identifier (organizationally unique identifier, OUI), an OUI version number, and other device information, which may include, for example, a device name, device ID, etc. to which the delivery or receiving end corresponds.

The configuration management sub-module may be used to configure the decision strategy of the control sub-module, and may also be configured for the whitelist.

The retry management sub-module can trigger retry according to error codes fed back by the control sub-module, the data management sub-module, the configuration management sub-module, the discovery module, the connection module, the message interaction module and other modules, and update and append the WFD IE.

In addition to the above structure, other structures in the software system provided by the embodiments of the present application may refer to the description of fig. 3, which is not repeated here.

In order to further explain the screen projection method provided by the embodiment of the present application, the screen projection method according to the embodiment of the present application is described below with reference to fig. 9, fig. 10, and fig. 11 and fig. 12.

Referring to fig. 11 and fig. 12, fig. 11 illustrates a timing chart of a screen projection method according to an embodiment of the present application, and fig. 12 is a flowchart of a screen projection method corresponding to fig. 11. It should be understood that the first terminal device serving as the delivery end is an execution body of the screen projection method provided by the embodiment of the present application.

As shown in fig. 12, the method 20 may include the following S201 to S208.

S201, referring to fig. 11 and fig. 12, at a first time point T1, in response to a first operation b11 of a user, the first terminal device initiates screen projection for the first time, and enters a scanning state.

The software system of the first terminal device may refer to the descriptions of fig. 9 and fig. 10, and the starting manner and the first operation b11 of the screen throwing function may refer to the descriptions of the starting manner and the first operation b1 of S101, which are not described herein.

The description of the first terminal device entering the scanning state and the related display interface may refer to the description in S101, which is not repeated herein.

Optionally, before the first time point T1, the first terminal device may be restored to the factory setting, or the screen-throwing function may be turned off in response to a closing operation of the user.

It should be understood that the first terminal device is restored to the factory setting, or in response to a closing operation of the user, a screen throwing function is closed, so as to ensure that when the subsequent first terminal device initiates screen throwing for the first time, there is no screen throwing application running in the foreground or the background on the first terminal device.

S202, the first terminal equipment and the second terminal equipment are connected in a screen projection mode and screen projection is conducted.

In the embodiment of the application, the second terminal device can always monitor the Wi-Fi channel after being started for discovering the first terminal device or being discovered. When the first terminal device scans successfully, that is, surrounding available devices are found, the scanned device identification of the second terminal device, such as a device name, can be displayed on the first frame or the first interface.

And after the second terminal equipment monitors the scanning request of the first terminal equipment, sending a scanning response to the first terminal equipment, wherein the response message can carry identification information of the second terminal equipment, such as information of manufacturer equipment, equipment name, equipment icon, equipment ID, MAC address and the like. Then, based on the identification information of the second terminal device, the first terminal device receives the operation of the user for the identification information, and the first terminal device and the second terminal device can establish a P2P connection. After connection, the first terminal device may send the image data to be delivered, etc., to the second terminal device.

S203, at a second time point T2, the screen projection is finished. The first time point T2 is after the first time point T1.

The end of the screen projection refers to the state that the first terminal device sends the image data to be delivered to the second terminal device.

It should be understood that, although the screen projection is finished at this time, the first terminal device and the second terminal device still establish a P2P connection relationship, and only the first terminal device does not send new image data to be delivered to the second terminal device any more.

In addition, after the screen projection is finished, the second terminal device may also display an interface of the first terminal device, for example, display a desktop of the first terminal device, etc.

S204, as shown in fig. 11, at a third time point T3, the screen-throwing application is turned off.

The third time point T3 is after the second time point T2, and a duration between the third time point T3 and the second time point T2 is equal to a third preset duration.

It should be understood that after the screen-throwing is finished, before the screen-throwing application is closed, that is, between the second time point T2 and the third time point T3, the first terminal device still maintains the previous bottom layer state, retains the WFD IE, and keeps the value corresponding to the wifi_display_on function unchanged as true.

After a third time point T3, namely after the screen-throwing application is closed, the first terminal equipment switches the bottom layer state, and the value corresponding to the WiFi_DISPLAY_ON function is modified from true to false; meanwhile, wpa_supplicant releases the P2P interface, and WFD IE is canceled.

It should be understood that the time of the third preset duration depends on the policy set by the manufacturer of the first terminal device to exit the application after the screen is finished, which is not limited in any way by the present application. For example, the mobile phone may close the screen-casting application after 150 seconds elapse after the screen-casting is finished. For another example, the third preset duration may be the same as the first preset duration, and is 300 seconds, that is, the mobile phone closes the screen-throwing application after 300 seconds elapses after the screen-throwing is finished.

It will be appreciated that S201 to S204 are a normal, complete screen-casting process, and S201 to S204 may be referred to as the first scenario.

S205, at a fourth time point T4, responding to a second operation b12 of the user, initiating screen projection for the second time by the first terminal equipment, and entering a scanning state again. The fourth time point T4 follows the third time point T3.

The second operation b12 may be the same as or different from the first operation b11, and the embodiment of the present application is not limited in any way.

The description of the first terminal device entering the scanning state again and the related display interface may refer to the description in S101, which is not repeated here.

Optionally, before the fourth time point T4, the first terminal device may be restored to the factory setting, or the screen-throwing function may be turned off in response to the operation of the user.

S206, at a fifth time point T5, in response to the third operation b13 of the user, the first terminal device exits the scanning state. The fifth time point T5 follows the fourth time point T4.

The third operation b13 may refer to the description of the fourth operation b4 in S104, which is not described herein.

S207, at a sixth time point T6, responding to a fourth operation b14 of the user, and initiating screen projection for the third time by the first terminal equipment to enter a scanning state.

The sixth time point T6 is after the fifth time point T5, and a time period between the sixth time point T6 and the fifth time point T5 is longer than the second preset time period.

S208, at a seventh time point T7, the first terminal device and the second terminal device are connected in a screen-casting mode, and screen casting is conducted until screen casting is finished.

The seventh time point T7 is after the sixth time point T6.

In S207, after the first terminal device exits the scanning state at the fifth time point T5, due to the existence of the detection mechanism in the first terminal device, after a second preset duration passes from the fifth time point T5, for example, when no service is detected to be performed ON the P2P interface within a time period of the second preset duration at the eighth time point T8, in order to avoid that the screen dropping initiated by S205 affects the call of other services ON the P2P interface, the first terminal device actively resets the bottom state, and modifies the value corresponding to the wifi_display_on function from true to false; meanwhile, wpa_supplicant releases the P2P interface, and WFD IE is canceled. Before the eighth time point T8, that is, between the fourth time point T5 and the eighth time point T8, the bottom layer state of the first terminal device remains unchanged, and the value corresponding to the wifi_display_on function is true; meanwhile, wpa_supplicant calls the P2P interface and adds WFD IE.

Subsequently, at a sixth time point T6, the first terminal equipment initiates screen projection for the third time, and when entering a scanning state, the embodiment of the application can make a decision and manage by utilizing the intelligent IE module, and determines that the bottom layer state can be switched at the moment according to the number of times of initiating screen projection by counting the number of times of initiating screen projection, and the wpa_supplicant recalls the P2P interface and adds the WFD IE; the value corresponding to the wifi_display_on function is also switched from false to true. Therefore, after the sixth time point T6, the decision and management provided by the embodiment of the present application between the first terminal device and the second terminal device can be normally performed, and the bottom layer state can not be switched again until the application is exited.

It should be noted that, the counted number of times of initiating the screen-throwing should be the second time, wherein the second time of starting scanning is recorded as the first time of initiating the screen-throwing, the third time of starting scanning is recorded as the second time of initiating the screen-throwing, and the first time of starting scanning before the second time is a completed screen-throwing process and the screen-throwing process is finished, so that the first time of starting scanning is not counted in the number of times of initiating the screen-throwing this time.

It should be understood that, after the screen-throwing application is started, the screen-throwing method provided by the application has the condition that the screen-throwing application exits and enters the scanning state for the second time in the middle, and in this case, the screen-throwing process can be further realized by using the screen-throwing method provided by the application, and the screen-throwing method S205 to S208 can be called as a second scene.

According to the screen projection method provided by the embodiment of the application, the intelligent IE module is additionally arranged in the WiFi P2Pservice of the software system of the screen projection end to make decisions and management, and whether the bottom layer state needs to be switched can be determined according to the counted number of times of initiating the screen projection, so that after judgment, the bottom layer state can be maintained under the counted condition of non-first screen projection, the purpose of continuously adding WFD IE can be realized by the projection end, the problem of screen projection failure caused by inconsistent information of the projection end and the receiving end is solved, and the screen projection experience of a user is improved.

The process of software module interaction within the first terminal device when performing the steps of S205 to S208 described above is further described below with reference to fig. 13.

Optionally, as shown in fig. 13, the screen projection method provided by the present application may include the following S301 to S316.

S301, responding to operation of a user, and issuing a screen projection instruction to a control submodule by the screen projection application.

The screen throwing instruction is used for indicating to initiate screen throwing. The drop application may also be referred to as a target application.

S302, after receiving the screen throwing instruction, the control submodule issues a first query instruction to the configuration management submodule. The first query instruction is used for querying whether the screen-throwing application initiating screen throwing belongs to an application allowing screen throwing, which is included in the white list.

It should be understood that the number and names of applications included in the preset whitelist in the configuration management sub-module may be set and modified as needed, which the present application does not limit.

S303, the configuration management submodule returns a query result to the control submodule.

It should be understood that, after the configuration management sub-module receives the first query instruction, it may perform traversal or query, in a preset whitelist, whether the screen-throwing application initiating screen-throwing exists in the whitelist or not through other preset query modes. When the query is in the white list, an identification that the query result is yes can be returned; when there is no white list, an indication that the query result is "no" may be returned.

It should be understood that after the control submodule receives the identifier of "no" of the query result, the flow may be ended, or after the control submodule receives the identifier of "no" of the query result, the query result may be returned to the screen-throwing application, and the query result is displayed on the interface of the first terminal device in a popup window form, so as to prompt the user that the screen-throwing application is not in the white list, and the screen-throwing is not allowed currently, and if the screen-throwing is desired, further operations may be performed. For example, the user may grant the screen-casting application the right to initiate screen casting by operating, allowing the screen-casting application to cast a screen. After the subsequent screen-throwing application receives the operation of the user, a first adding instruction can be issued to the control submodule so as to instruct the configuration management submodule to add the screen-throwing application to the white list. In other words, the configuration management sub-module may update the whitelist in response to a user operation.

S304, after the control submodule receives the query result of the screen-throwing application existing in the white list, a second query instruction can be issued to the configuration management submodule.

The second query instruction is used for querying a screen projection strategy corresponding to the screen projection application initiating screen projection.

It should be understood that a preset screen-projection strategy library exists in the configuration management sub-module, and a preset different screen-projection strategy exists in the screen-projection strategy library, for example, the screen-projection strategy can include intervention IE management and non-intervention IE management. The intervening IE management refers to: providing a screen throwing method for continuously adding WFD IE to the screen throwing application; non-invasive IE management refers to: the screen-casting method of continuously adding WFD IE is not provided for the screen-casting application.

Other screen projection strategies can be also included in the screen projection strategy library, and preset screen projection strategies can be set and modified according to requirements, so that the application is not limited in any way.

S305, the configuration management sub-module returns a strategy result to the control sub-module.

It should be understood that after the configuration management sub-module receives the second query instruction, the configuration management sub-module may perform traversal or query, in a screen projection strategy library, a screen projection strategy corresponding to a screen projection application initiating screen projection, and if the corresponding screen projection strategy is found, return the screen projection strategy as a strategy result; if the corresponding screen projection strategy is not searched, an identification that the query result is 'none' can be returned.

It should be understood that after the control submodule receives the identifier that the query result is "none", the flow may be ended, or after the control submodule receives the identifier that the query result is "none", the query result may be returned to the screen-throwing application, and the query result is displayed on the interface of the first terminal device in a popup window form, so as to prompt the user that the screen-throwing application does not have a corresponding screen-throwing strategy, and if the user wants to throw a screen, further operations may be performed, for example, the screen-throwing application may be granted with a certain strategy as a corresponding screen-throwing strategy, and the screen-throwing is allowed with the strategy. After the subsequent screen-throwing application receives the operation of the user, a second adding instruction can be issued to the control submodule so as to instruct the configuration pipe module to add the screen-throwing application and the corresponding screen-throwing strategy into the screen-throwing strategy library. In other words, the configuration pipe module may update the drop strategy library in response to a user operation.

For example, after the control submodule receives the query result that a certain screen-throwing application exists in the white list, a second query instruction may be issued to the configuration management submodule, and after the configuration management submodule queries, if the query result indicates that the corresponding screen-throwing strategy is "intervention IE management", the identifier of "intervention IE management" may be returned to the control submodule.

S306, if the control submodule receives the identifier of the policy result of 'intervention IE management', a management instruction of intervention IE management is issued to the data management submodule.

S307, after the data management sub-module receives the management instruction, an acquisition instruction for acquiring initial data is sent to the screen-throwing application.

The acquisition instruction is used for indicating to acquire initial data such as application behavior, application UID, IE equipment information and the like.

It should be appreciated that different IE device information may be collected and generated according to different application behaviors.

S308, the screen-throwing application sends the obtained initial data to a data management submodule for recording.

It should be understood that the initial data sent by the data management submodule may be recorded in the data management submodule or the memory according to the corresponding relationship.

It should be appreciated that the data management sub-module may also periodically maintain data, e.g., may be periodically deleted for aged data, and may also be periodically recorded for newly added data. The schedule of deletion and recording may be set as desired, and the present application is not limited in this regard.

S309, responding to the operation of the user, and issuing a screen throwing instruction to the control submodule by the screen throwing application again.

And S310, after receiving the screen throwing instruction, the control submodule issues a third query instruction to the data management submodule. And the third query instruction is used for querying the historical data record corresponding to the screen-throwing application.

S311, the data management sub-module returns a query result to the control sub-module.

It should be understood that after the data management submodule receives the third query instruction, the data management submodule may perform traversal or query, in a database, a historical data record corresponding to the screen-throwing application initiating screen throwing, in other preset query modes. For example, when a screen is launched before the screen-casting application is queried, the screen-casting times can be returned to the control submodule as a query result.

S312, the control submodule determines whether to add the WFD IE according to the query result, and if the WFD IE is determined to be added, an adding instruction is sent to the wpa_supplicant.

After the wpa_supplicant receives the append instruction, the WFD IE can be appended to the frame message sent by the terminal device based on the created P2P interface.

S313, if the wpa_supplicant fails to be added, a failure error code is returned to the control submodule.

It should be understood that if the wpa_supprant receives the append instruction to append the WFD IE, and the wfd_supprant fails for some reasons (e.g., the wpa_supprant cannot acquire the WFD IE information, etc.), the wpa_supprant may return a failure error code to the control submodule to inform the control submodule that the append fails.

And S314, the control submodule issues a retry instruction to the retry submodule according to the failure error code.

The retry instruction is used to instruct the retry sub-module to resume the generation of the WFD IE.

S315, the retry sub-module returns the regenerated WFD IE to the control sub-module.

And S316, the control submodule sends an additional instruction to the wpa_suppicant again according to the regenerated WFD IE.

It should be appreciated that after receiving the append instruction again, wpa_supplicant may continue to call the P2P interface to append the regenerated WFD IE.

S313 to S316 described above provide a retry opportunity when the WFD IE addition fails after the intervention IE management, so as to increase the probability of success of the addition. Here, in the event of a secondary append failure, the loop execution S313 to S316 may also be continued, with no limitation in the embodiment of the present application, in particular, in number of loops.

The software system, the screen projection method and the application scenario in several embodiments provided by the present application are described above with reference to fig. 1 to 13. The hardware system and chip system of the terminal device to which the present application is applied will be described in detail with reference to fig. 14 to 15. It should be understood that the hardware system and the chip system in the embodiments of the present application may perform the methods of the foregoing embodiments of the present application, that is, specific working processes of the following various products may refer to corresponding processes in the foregoing method embodiments.

Fig. 14 shows a hardware system of a terminal device suitable for use in the present application. The first terminal device may be configured to implement a processing step corresponding to the terminal device in the screen projection method described in the foregoing method embodiment.

In the embodiment of the present application, the first terminal device may be a mobile phone shown in fig. 1 or fig. 2, a tablet computer, a smart screen, a wearable electronic device, a vehicle-mounted electronic device, an augmented reality device, a Virtual Reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (personal digital assistant, PDA), or a device with a mirror image screen projection function, and the embodiment of the present application does not limit a specific type of the first terminal device. When the first terminal equipment initiates screen projection by using the mirror image screen projection function, even if screen projection is canceled in the process and screen projection is initiated for the second time, the screen projection can be successful, and the experience of the user is improved.

The first terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, keys 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identity module (subscriber identification module, SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

Note that the configuration shown in fig. 14 does not constitute a specific limitation on the first terminal apparatus. In other embodiments of the application, the first terminal device may comprise more or less components than those shown in fig. 14, or the first terminal device may comprise a combination of some of the components shown in fig. 14, or the first terminal device may comprise a sub-component of some of the components shown in fig. 14. The components shown in fig. 14 may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: application processors (application processor, AP), modem processors, graphics processors (graphics processing unit, GPU), image signal processors (image signal processor, ISP), controllers, video codecs, digital signal processors (digital signal processor, DSP), baseband processors, neural-Network Processors (NPU). The different processing units may be separate devices or integrated devices. The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby improving the efficiency of the system.

The connection relationship between the modules shown in fig. 14 is only schematically illustrated, and does not constitute a limitation on the connection relationship between the modules of the first terminal device. Alternatively, each module of the first terminal device may also use a combination of multiple connection manners in the foregoing embodiments.

The charge management module 140 is used to receive power from a charger. The charging management module 140 may also supply power to the first terminal device through the power management module 141 while charging the battery 142. The power management module 141 is used for connecting the battery 142, and the charge management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140 to power the processor 110, the internal memory 121, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 may also be used to monitor parameters such as battery capacity, battery cycle times, and battery state of health (e.g., leakage, impedance). Alternatively, the power management module 141 may be provided in the processor 110, or the power management module 141 and the charge management module 140 may be provided in the same device.

The wireless communication function of the first terminal device may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, the baseband processor, and so on. The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the first terminal device may be adapted to cover a single or multiple communication bands. Different antennas may also be multiplexed to improve the utilization of the antennas.

The mobile communication module 150 may provide a solution for wireless communication applied on the first terminal device, such as at least one of the following: second generation (2) ^th generation, 2G) mobile communication solutions, third generation (3 ^th generation, 3G) mobile communication solution, fourth generation (4 ^th generation, 5G) mobile communication solution, fifth generation (5 ^th generation, 5G) mobile communication solution.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through audio devices (e.g., speaker 170A, receiver 170B) or displays images or video through display screen 194. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional module, independent of the processor 110.

Similar to the mobile communication module 150, the wireless communication module 160 may also provide a wireless communication solution applied on the first terminal device, such as at least one of the following: wireless local area networks (wireless local area networks, WLAN), bluetooth (BT), bluetooth low energy (bluetooth low energy, BLE), ultra Wide Band (UWB), global navigation satellite system (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication (near field communication, NFC), infrared (IR) technologies.

In some embodiments, the antenna 1 of the first terminal device is coupled to the mobile communication module 150 and the antenna 2 of the first terminal device is coupled to the wireless communication module 160 so that the first terminal device can communicate with the network and other terminal devices through wireless communication techniques.

The external memory interface 120 may be used to connect an external memory card, such as a Secure Digital (SD) card, to enable expanding the memory capabilities of the first terminal device. The external memory card communicates with the processor 110 through an external memory interface 120 to implement data storage functions. For example, files such as music, video, etc. are stored in an external memory card.

The internal memory 121 may be used to store computer executable program code including instructions. The internal memory 121 may include a storage program area and a storage data area.

In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. The pressure sensor 180A may be of various types, such as a resistive pressure sensor, an inductive pressure sensor, or a capacitive pressure sensor. The capacitive pressure sensor may be a capacitive sensor comprising at least two parallel plates with conductive material, the first terminal device determining the strength of the pressure based on the change in capacitance when the force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. When a touch operation acts on the display screen 194, the first terminal device detects the touch operation according to the pressure sensor 180A. The first terminal device may also calculate the position of the touch from the detection signal of the pressure sensor 180A. In some embodiments, touch operations that act on the same touch location, but at different touch operation strengths, may correspond to different operation instructions. For example: executing an instruction for checking the short message when the touch operation with the touch operation intensity smaller than the first pressure threshold acts on the short message application icon; and executing the instruction of newly creating the short message when the touch operation with the touch operation intensity being larger than or equal to the first pressure threshold acts on the short message application icon.

The fingerprint sensor 180H is used to collect a fingerprint. The first terminal device can utilize the collected fingerprint characteristics to realize the functions of unlocking, accessing the application lock, photographing, answering the incoming call and the like.

The touch sensor 180K, also referred to as a touch device. The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a touch screen. The touch sensor 180K is for detecting a touch operation acting thereon or thereabout. The touch sensor 180K may communicate the detected touch operation to the application processor to determine the touch event type. Visual output related to touch operations may be provided through the display 194. In other embodiments, the touch sensor 180K may also be disposed on the surface of the first terminal device and at a different location than the display 194.

The keys 190 include a power-on key and an volume key. The keys 190 may be mechanical keys or touch keys. The first terminal device may receive the key input signal, and implement a function related to the case input signal.

Fig. 15 shows a schematic structural diagram of still another terminal device provided by the present application. The dashed line in fig. 15 indicates that the unit or the module is optional, and the first terminal device 200 may be used to implement the screen projection method described in the above method embodiment.

The first terminal device 200 comprises one or more processors 201, which one or more processors 201 may support the first terminal device 200 to implement the method in the method embodiments. The processor 201 may be a general purpose processor or a special purpose processor. For example, the processor 201 may be a central processing unit (central processing unit, CPU), digital signal processor (digital signal processor, DSP), application specific integrated circuit (application specific integrated circuit, ASIC), field programmable gate array (field programmable gate array, FPGA), or other programmable logic device such as discrete gates, transistor logic, or discrete hardware components.

The processor 201 may be configured to control the first terminal device 200, execute a software program, and process data of the software program. The first terminal device 200 may further comprise a communication unit 205 for enabling input (reception) and output (transmission) of signals.

For example, the first terminal device 200 may be a chip, the communication unit 205 may be an input and/or output circuit of the chip, or the communication unit 205 may be a communication interface of the chip, which may be an integral part of the terminal device or other terminal devices.

For another example, the first terminal device 200 may be a terminal device, the communication unit 205 may be a transceiver of the terminal device, or the communication unit 205 may be a transceiver circuit of the terminal device.

The first terminal device 200 may include one or more memories 202, on which a program 204 is stored, where the program 204 may be executed by the processor 201 to generate instructions 203, so that the processor 201 executes the screen-projection method described in the above method embodiment according to the instructions 203.

Optionally, the memory 202 may also have data stored therein. Alternatively, processor 201 may also read data stored in memory 202, which may be stored at the same memory address as program 204, or which may be stored at a different memory address than program 204.

The processor 201 and the memory 202 may be provided separately or may be integrated together; for example, integrated on a System On Chip (SOC) of the terminal device.

Illustratively, the memory 202 may be used to store the relevant program 204 of the screen-casting method provided in the embodiment of the present application, and the processor 201 may be used to call the relevant program 204 of the screen-casting method stored in the memory 202 at the time of upgrading, to execute the screen-casting method of the embodiment of the present application. For example:

Responding to a first operation, and initiating screen throwing by a target application in first terminal equipment for the first time; the method comprises the steps that a first terminal device and a second terminal device establish screen-throwing connection, and screen throwing is carried out; starting from the screen-throwing ending time point, closing the target application after a first preset time length; responding to a second operation, and enabling the target application to initiate screen projection for the second time; in response to the third operation, cancelling the screen projection; responding to a fourth operation, and enabling the target application to initiate screen projection for the third time; and the first terminal equipment and the second terminal equipment establish screen-throwing connection and perform screen-throwing.

According to the screen projection method provided by the application, the information of the delivery end and the receiving end can be synchronized when the screen projection is initiated for a plurality of times through decision and management, and the screen projection is successful.

The present application also provides a computer program product which, when executed by the processor 201, implements the method of projecting a screen according to any one of the method embodiments of the present application.

The computer program product may be stored in the memory 202, for example, the program 204, and the program 204 is finally converted into an executable object file capable of being executed by the processor 201 through preprocessing, compiling, assembling, and linking processes.

The application also provides a computer readable storage medium, on which a computer program is stored, which when executed by a computer implements the screen projection method according to any one of the method embodiments of the application. The computer program may be a high-level language program or an executable object program.

Optionally, the computer readable storage medium is, for example, memory 202. The memory 202 may be volatile memory or nonvolatile memory, or the memory 202 may include both volatile and nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. The volatile memory may be random access memory (random access memory, RAM) which acts as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and direct memory bus RAM (DR RAM).

It will be clearly understood by those skilled in the art that, for convenience and brevity of description, specific working processes and technical effects of the apparatus and device described above may refer to corresponding processes and technical effects in the foregoing method embodiments, which are not described in detail herein.

In the several embodiments provided by the present application, the disclosed systems, devices, and methods may be implemented in other manners. For example, some features of the method embodiments described above may be omitted, or not performed. The above-described apparatus embodiments are merely illustrative, the division of units is merely a logical function division, and there may be additional divisions in actual implementation, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the elements or the coupling between the elements may be direct or indirect, including electrical, mechanical, or other forms of connection.

It should be understood that, in various embodiments of the present application, the size of the sequence number of each process does not mean that the execution sequence of each process should be determined by its functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

In addition, the terms "system" and "network" are often used interchangeably herein. The term "and/or" herein is merely one association relationship describing the associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.

In summary, the foregoing description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A screen projection method, characterized in that it is applied to a first terminal device, the method comprising:

In response to the third operation, cancelling the screen projection;

2. The screen projection method according to claim 1, wherein a time period from canceling the screen projection time point until the target application initiates the screen projection for the third time is longer than a second preset time period;

3. The screen projection method according to claim 1 or 2, wherein a time period from the cancellation of the screen projection time point to the third initiation of the screen projection by the target application is less than the first preset time period;

4. A screen projection method according to any one of claims 1 to 3, wherein the target application for the second time of initiating the screen projection is the same as the target application for the third time of initiating the screen projection, and the target application for the second time of initiating the screen projection is the same as or different from the target application for the first time of initiating the screen projection.

5. The method of claim 1, wherein the first terminal device and the second terminal device each support a Miracast screen-casting technique.

6. The screen projection method is characterized by being applied to first terminal equipment, wherein the first terminal equipment comprises an intelligent IE module, and the intelligent IE module comprises a control sub-module and a configuration management sub-module;

the method comprises the following steps:

7. The screen projection method of claim 6 wherein the intelligent IE module further comprises a data management sub-module; the control submodule indicates to throw the screen based on the screen throwing strategy returned by the configuration management submodule to the control submodule, and the method comprises the following steps:

the control submodule indicates screen projection based on the initial data.

8. The screen projection method of claim 7 wherein the intelligent IE module further comprises a retry sub-module; the method further comprises the steps of:

the control submodule determines whether to add WFDIE or not based on the query result;

when determining to add the WFDIE, the control submodule indicates to throw a screen and indicates to add the WFDIE;

when the control submodule receives a failure error code that the WFD IE is failed to be added, a retry instruction is issued to the retry submodule, and the retry instruction is used for indicating to regenerate a new WFDIE;

The control submodule instructs a screen to be thrown and instructs a new WFDIE to be added.

9. A terminal device comprising a processor and a memory;

the memory is used for storing a computer program capable of running on the processor;

the processor is configured to perform the screen projection method according to any one of claims 1 to 5 and/or the screen projection method according to any one of claims 6 to 8.

10. A chip system, comprising: processor for calling and running a computer program from a memory, causing a device on which the chip is mounted to perform the method of screening according to any one of claims 1 to 5 and/or the method of screening according to any one of claims 6 to 8.

11. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, cause the processor to perform the method of screening according to any one of claims 1 to 5 and/or the method of screening according to any one of claims 6 to 8.