WO2018126941A1

WO2018126941A1 - Screen projection method, device and terminal

Info

Publication number: WO2018126941A1
Application number: PCT/CN2017/118515
Authority: WO
Inventors: 宋晓霞
Original assignee: 中兴通讯股份有限公司
Priority date: 2017-01-04
Filing date: 2017-12-26
Publication date: 2018-07-12
Also published as: CN108268228A

Abstract

Disclosed are screen projection method, device and terminal, wherein the method comprises determining that a screen projection passage is established between a terminal and a display device; and triggering, according to a pre-set trigger condition, the sending of a control instruction to the display device, wherein the control instruction is used to control the screen content of the terminal to be projected onto the display device, and the control instruction is sent by means of the established screen projection passage.

Description

Screen projection method, device and terminal

Technical field

The present disclosure relates to the field of communications, and in particular to a screen projection method, apparatus, and terminal.

Background technique

Miracast forms a connection via Wi-Fi CERTIFIED Wi-Fi Direct, without the need to access any Wi-Fi network, enabling Miracast wireless display sharing to project the screen on the mobile device directly onto the display device. During the test, after the mobile device and the display device are successfully connected, the user operates on the mobile device, and the display device synchronously displays the screen content on the mobile device, and the display device only mirrors the current screen of the mobile device.

The screen projection technology based on Miracast technology is that after the terminal turns on the screen projection function, the content on the terminal is mirrored and displayed on the display terminal, and the screen projection function starts to transmit the image of the terminal screen at the same time that the terminal and the display device are successfully connected. To the display device.

Summary of the invention

The following is an overview of the topics detailed in this document. This Summary is not intended to limit the scope of the claims.

A screen projection method, device and terminal are provided in the embodiments of the present disclosure.

According to an embodiment of the present disclosure, a screen projection method is provided, including: determining that a screen projection channel is established between a terminal and a display device; and triggering to send a control instruction to the display device according to a preset trigger condition, where The control instruction is configured to control the screen content of the terminal to be projected onto the display device, and the control instruction is sent through the established screen projection channel.

In an exemplary embodiment, triggering the sending the control instruction to the display device according to the preset triggering condition includes: monitoring an operation action generated by the terminal; determining whether the monitored operation action and the preset action are Matching; if the result of the determination is a match, sending a control instruction corresponding to the operation action to the display device, wherein the control instruction is used to indicate that the screen content of the terminal is started to be projected or stopped to be projected On the display device.

In an exemplary embodiment, monitoring an operation action generated by the terminal includes: monitoring an operation action generated by at least one of the following: the component, an application control, and an operation key.

In an exemplary embodiment, triggering the sending of the control instruction to the display device according to the preset triggering condition includes: determining whether the current time meets a preset time condition for controlling a screen projection time; and determining that the preset is satisfied In the case of a time condition, the control command is sent to the display device, wherein the control command is used to indicate that the screen content of the terminal is started or stopped being projected onto the display device.

In an exemplary embodiment, in a case where the control instruction is used to indicate that screen content of the terminal is started to be projected onto the display device, triggering to send to the display device according to the preset trigger condition After the control instruction, the method further includes: sending, by the data transmission instruction, a mirror image of the screen content of the terminal to the display device.

In an exemplary embodiment, in a case where the control instruction is used to indicate that screen content of the terminal is started to be projected onto the display device, triggering to send to the display device according to the preset trigger condition After the control instruction, the method further includes: determining display content of the preset area for projecting to the display device on the screen of the terminal; and transmitting the determined display content of the preset area to the display The device is casting.

According to another embodiment of the present disclosure, a screen projection apparatus includes: a determining module configured to determine that a screen projection channel is established between the terminal and the display device; and a triggering module configured to trigger the trigger according to the preset trigger condition The display device sends a control instruction, wherein the control instruction is used to control screen content of the terminal to be projected to the display device, and the control instruction is sent through the established screen projection channel.

In an exemplary embodiment, the triggering module includes: a monitoring unit configured to monitor an operation action generated by the terminal; and a first determining unit configured to determine whether the monitored operation action matches a preset action; a sending unit configured to send, to the display device, a control instruction corresponding to the operation action, where the determination result of the first determining unit is a match, wherein the control instruction is used to indicate a start or Stop projecting the screen content of the terminal onto the display device.

In an exemplary embodiment, the monitoring unit is further configured to monitor an operation action generated by at least one of the following: the component, the application control, and the operation key.

In an exemplary embodiment, the triggering module includes: a second determining unit configured to determine whether a current time meets a preset time condition for controlling a screen projection time; and a second sending unit configured to be in the second determining When the judgment result of the unit is a match, the control instruction is sent to the display device, wherein the control instruction is used to indicate that the screen content of the terminal is started or stopped being projected onto the display device.

In an exemplary embodiment, the apparatus further includes: a first transmitting module configured to transmit data by using the control instruction to instruct to start projecting screen content of the terminal onto the display device An instruction to send a mirror image of the screen content of the terminal to the display device.

In an exemplary embodiment, the apparatus further includes: a second sending module, wherein the determining module is further configured to: at the control instruction, to indicate that screen content of the terminal is started to be projected to the display device In the above case, determining display content of a preset area for projecting to the display device on a screen of the terminal; the second sending module configured to determine the display of the preset area Content is sent to the display device for projection.

According to still another embodiment of the present disclosure, a storage medium is also provided. The storage medium is configured to store program code for performing the steps of: determining that a screen projection channel is established between the terminal and the display device; and triggering to send a control instruction to the display device according to a preset trigger condition, wherein the control instruction The screen content for controlling the terminal is projected onto the display device, and the control instruction is sent through the established screen projection channel.

In an exemplary embodiment, the storage medium is further configured to store program code for performing the step of: triggering the transmitting the control instruction to the display device according to the preset trigger condition comprises: monitoring an operation generated by the terminal An operation of determining whether the monitored operation action matches the preset action; and if the determination result is a match, transmitting a control instruction corresponding to the operation action to the display device, where the control instruction is used Instructing to start projecting or stop projecting the screen content of the terminal onto the display device.

In an exemplary embodiment, the storage medium is further configured to store program code for performing the following steps: monitoring an operation action generated by the terminal includes: monitoring an operation action generated by at least one of the following: the component, the application control , operation keys.

In an exemplary embodiment, the storage medium is further configured to store program code for performing the step of: triggering the sending of the control instruction to the display device according to the preset trigger condition comprises: determining whether the current time is satisfied for controlling the screen projection time a preset time condition; the control instruction is sent to the display device if the result of the determination is that the preset time condition is met, wherein the control instruction is used to indicate that the screen content of the terminal is started or Stop casting onto the display device.

In an exemplary embodiment, the storage medium is further configured to store program code for performing the step of: in the case where the control instruction is used to indicate that screen content of the terminal is initially projected onto the display device, After the triggering the sending of the control instruction to the display device, the method further includes: sending, by using a data transmission instruction, a mirror image of the screen content of the terminal to the display device.

In an exemplary embodiment, the storage medium is further configured to store program code for performing the step of: in the case where the control instruction is used to indicate that screen content of the terminal is initially projected onto the display device, After the triggering the sending of the control instruction to the display device, the method further includes: determining a display content of the preset area for projecting to the display device on the screen of the terminal; The display content of the preset area is sent to the display device for projection.

According to the disclosure, after the screen projection channel is established between the terminal and the display device, the control command is sent to the display device according to the preset trigger condition, and the screen trigger control can be flexibly set by setting the preset trigger condition, so that the solution can be solved. The existing screen projection method has the problem of poor flexibility, and the effect of improving the projection flexibility of the screen is achieved.

Other aspects will be apparent upon reading and understanding the drawings and detailed description.

DRAWINGS

1 is a block diagram showing the hardware structure of a mobile terminal of a screen projection method according to an embodiment of the present disclosure;

2 is a flow chart of a screen projection method in accordance with an embodiment of the present disclosure;

FIG. 3 is a flowchart 1 of a screen projection method according to an exemplary embodiment of the present disclosure;

FIG. 4 is a flowchart 2 of a screen projection method according to an exemplary embodiment of the present disclosure;

FIG. 5 is a structural block diagram 1 of a screen projection apparatus according to an embodiment of the present disclosure; FIG.

FIG. 6 is a structural block diagram 1 of a trigger module 54 of a screen projection apparatus according to an embodiment of the present disclosure;

FIG. 7 is a structural block diagram 2 of a trigger module 54 of a screen projection apparatus according to an embodiment of the present disclosure;

FIG. 8 is a structural block diagram 2 of a screen projection apparatus according to an embodiment of the present disclosure; FIG.

9 is a block diagram 3 of a structure of a screen projection apparatus according to an embodiment of the present disclosure.

detailed description

For the screen projection mode, if the user wants to display the content of a certain time period on the display device, then the screen projection function is turned on at the start time of the time period and the screen projection connection between the terminal and the display device is established, The end time of the time period turns off the screen projection function and disconnects the screen projection connection between the terminal and the display device, and then connects when the user wants to perform the screen projection function.

The above screen projection method has the following problems: 1) When the display is not displayed and is not controlled by the user, as long as the screen projection of the terminal is turned on and the connection with the display device is successful, the content displayed on the screen on the terminal is displayed on the display device at all times, and is turned off. When the screen projection function is used, the connection between the terminal and the display device is also disconnected. If the screen projection is to be performed, it is necessary to reconnect; 2) which content is displayed and which is not displayed or controlled by the user, as long as the user opens the privacy part on the terminal. In the interface, the display terminal will display, unless the screen projection function is turned off, the user's privacy is not well protected during the screen projection process. Therefore, the existing screen projection method has a problem of poor flexibility.

The present disclosure will be described in detail below with reference to the drawings in conjunction with the embodiments.

The terms "first", "second" and the like in the specification and claims of the present disclosure and the above-mentioned figures are used to distinguish similar objects, and are not necessarily used to describe a particular order or order.

Example 1

The method embodiments provided by the present disclosure may be implemented in a mobile terminal, a computer terminal, or the like. Taking a mobile terminal as an example, FIG. 1 is a hardware structural block diagram of a mobile terminal of a screen projection method according to an embodiment of the present disclosure. As shown in FIG. 1, the mobile terminal 10 may include one or more (only one shown) processor 12 (the processor 12 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA). A memory 14 for storing data, and a transmission device 16 for communication functions. It will be understood by those skilled in the art that the structure shown in FIG. 1 is merely illustrative and does not limit the structure of the above electronic device. For example, the mobile terminal 10 may also include more or fewer components than those shown in FIG. 1, or have a different configuration than that shown in FIG.

The memory 14 can be used to store software programs and modules of application software, such as program instructions/modules corresponding to the screen projection method in the embodiment of the present disclosure, and the processor 12 executes various kinds by executing software programs and modules stored in the memory 14. Functional application and data processing, that is, the above method is implemented. Memory 14 may include high speed random access memory, and may also include non-volatile memory such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, memory 14 may further include memory remotely located relative to processor 12, which may be connected to mobile terminal 10 over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Transmission device 16 is for receiving or transmitting data via a network. The above-described network specific example may include a wireless network provided by a communication provider of the mobile terminal 10. In one example, transmission device 16 includes a Network Interface Controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 16 can be a Radio Frequency (RF) module for communicating wirelessly with the Internet.

In this embodiment, a screen projection method running on the mobile terminal is provided. FIG. 2 is a flowchart of a screen projection method according to an embodiment of the present disclosure. As shown in FIG. 2, the flow includes the following steps:

Step S202, determining that a screen projection channel is established between the terminal and the display device;

Step S204, triggering a control command to be sent to the display device according to the preset trigger condition, wherein the control command is used to control the screen content of the terminal to be projected onto the display device, and the control command is sent through the established screen projection channel.

After the screen projection channel is established between the terminal and the display device, the triggering is sent to the display device according to the preset trigger condition, which solves the problem that the existing screen projection mode has poor flexibility and improves the screen projection. flexibility.

In an embodiment, step S204 can trigger the sending of a control command to the display device in a plurality of manners. For example, whether the monitored operation of the terminal screen matches the preset operation action to control the projection of the screen content of the terminal to the display device: monitoring the operation action generated by the terminal; determining whether the monitored operation action matches the preset action; When the result of the determination is a match, a control command corresponding to the operation action is sent to the display device, wherein the control command is used to instruct to start or stop projecting the screen content of the terminal onto the display device. The operation action generated by the monitoring terminal may include: monitoring an operation action generated by at least one of the following: a component, an application control, and an operation key. The operation action here may be that the preset button in the terminal display screen is pressed, or the user is detected to perform the preset action, where the detection may be performed through the screen of the terminal, or may be performed by sensing the motion of the human body. Parts are tested.

For another example, a preset time condition for controlling the screen projection time may be set in advance, and when the current time meets the preset time condition, a control command is transmitted to the display device. The preset time condition here may be: a time at which the screen projection is started, a time at which the screen projection is stopped, and a time period during which the screen is projected. The time or time period referred to here may be an absolute time (for example, 13:15:00), or may be a relative time determined by a timer or the like. For the relative time, for example, the projection may be started for 5 minutes, 5 minutes after the reference time, or the projection may be stopped after 5 minutes after the reference time, and the reference time may be an absolute time or a specified time (for example, a certain time is detected). The moment of an operation action). The preset time condition can be set as needed, and is not specifically limited.

For another example, triggering a control command to the display device may also be triggered by other trigger conditions (eg, a certain video is played to a predetermined time, a presentation is played or played to a certain page, a certain picture is opened, etc.) .

The preset trigger condition can be set as needed, and is not specifically limited.

Through the above solution of the embodiment of the present disclosure, the screen projection control is controlled by whether the monitored operation action matches the preset action, the implementation scheme is simple, the occupation of the terminal memory is reduced, and the practicability of the screen projection scheme is improved; Screen projection control can improve the flexibility of screen projection, meet the different needs of users, and improve user experience.

In an embodiment, in the case that the control instruction is used to indicate that the screen content of the terminal is started to be projected onto the display device, after the control instruction is sent to the display device, the image of the screen content of the terminal may also be sent through the data transmission instruction. Give the display device.

In an embodiment, the entire screen content of the terminal may be projected onto the display device, or a portion of the screen content of the terminal may be projected onto the display device. For example, the display content of the preset area for projecting to the display device on the screen of the terminal may be determined, and the determined display content of the preset area is sent to the display device for projection. The preset area may be any shape. In any position of the terminal screen, the setting of the preset area may be before the screen projection channel is established between the terminal and the display device, or after the screen projection channel is established between the terminal and the display device. , or in the process of screen projection, this is not specifically limited.

Based on the above embodiments and exemplary embodiments, in order to explain the entire process interaction of the solution, in the present exemplary embodiment, a screen projection method is provided, which may be applied to a mobile terminal, a computer terminal or other display device The device that interacts. The screen projection method will be described below.

The terminal opens the Miracast-based screen projection function, and after the screen projection channel is established between the terminal and the display device, the content of the terminal screen is projected onto the display device when the start projection condition is satisfied; likewise, the terminal is stopped when the end projection condition is satisfied. The content on the projected device is projected. These conditions may be triggered by a screen shot when certain preset conditions are encountered during terminal operation, or by a user clicking on an icon.

For the above, the following key methods are implemented:

Method 1: The terminal intelligently turns on and off the screen projection function, presets the conditions for turning on and off the screen projection in the terminal, and automatically turns on or off the screen projection function when the condition is met.

While establishing a screen projection function channel between the terminal and the display device, preset conditions for turning on and off the screen projection function, for example, the condition for turning on the screen projection function may be: the property of the control clicked by the user is a play control attribute such as “play” For example, the condition for turning off the screen projection function may be: the end of the video being played.

For example, after the screen projection channel between the terminal and the display device is established, the terminal monitoring module starts monitoring the action of each operation of the user on the terminal interface, if the action and the preset action of turning on and off the screen are monitored. Consistently, the instruction to turn on or off the screen projection is sent to the display device through the channel projected by the screen, and the display device analyzes whether the received content is a control instruction or a data transmission instruction after receiving, and if it is a control instruction, the control instruction Responding to the response, sending the response result to the terminal, and after the terminal receives the response, performing an action of opening (transferring the terminal interface content to the display device) or closing (stopping the transmission of the terminal interface content to the display device) screen; if displayed The content received by the device is the content of the interface image transmission, and the display terminal directly displays the content; if the action of the user monitored by the terminal is inconsistent with the preset action of turning on or off the screen projection, the terminal monitoring module does not perform any operation.

Method 2: The terminal turns on the screen projection function to establish a screen projection connection channel between the terminal and the display device. On the terminal interface, there is an icon projected by the screen. When the user clicks the icon, the content starting from the current interface can be projected onto the display device, and the content starting from the current interface cannot be projected onto the display device.

After the screen projection function channel is established between the terminal and the display device, there is a small icon of the screen projection function on the screen of the terminal, and the terminal monitors whether the user clicks the screen projection icon, and if the user clicks the action in the terminal, the screen projection operation is turned on. The terminal sends a control instruction through a screen projection channel established between the terminal and the display device, and also displays a type of content sent by the device monitoring terminal, and after receiving the instruction, analyzes whether the control instruction or the content of the transmission data, if it is a control instruction, The control command responds, and the result of the response is sent to the terminal, and the terminal starts to mirror the transmission interface content after receiving the message; if the "click" action is to turn off the screen projection operation, the display device responds after receiving, and stops the terminal interface from being displayed. Displayed on the device.

The above method 1 and method 2 will be described below in conjunction with specific implementation schemes.

method 1:

The modules involved in the method 1 mainly include the following modules: a monitoring module on the terminal, a screen layout extraction module, a screen projection on-and-off condition preset module on the terminal (a conditional preset module), and a comparison module on the terminal, which are combined below. The module describes Method 1.

1. The monitoring module on the terminal: set to monitor the operation actions and operation objects of the interface where the current user is located, and the monitoring format may be (action: operation object), and the monitoring module further includes an action monitoring sub-module and an operation object generating sub-module. The action monitoring is responsible for monitoring the current operation actions of the user, including clicking, double-clicking, sliding, etc.; the operation object generation sub-module determines the operation object according to the attributes of the control operated by the user.

The operation object refers to the user-operated control, which is unique on the current interface. It is obtained by using the control property in the screen layout file. Why not use the "action: control property" directly because the control property is not single. On a screen of the terminal, a control is unique, but the properties of the control in the screen layout file are not necessarily unique. Sometimes, multiple attributes are used to comprehensively determine the control object, as in the following example 1. .

Example 1: The terminal monitoring module monitors the user's operation on the interface as: "Click: com.android.gallery3d: id/gl_root_view (the resource ID attribute value of the video play control)", the attribute value of the control may be the resource ID, Description and text attribute, according to the property value of the control (com.android.gallery3d: id/gl_root_view) to get the object of the control (video play control), the corresponding action (action: operation object) is: "Click: Video Play control".

Example 2 below illustrates how to get the object of the control based on the property value of the control.

Example 2: The user's operation is "Click: slideshow control property", where the action is click, the slideshow control property is: (type, value), type is the above slideshow control property type, value Is the value of the control type, where type=text, value=Slideshow, action=click. Then use the API function provided by android uiautomator to get the control object according to the text attribute of the control, as follows:

{UiObject mItem=new UiObject(new UiSelector().text(Slideshow));}

//mItem is the control text for the slideshow corresponding object

2. Screen layout extraction module: The layout file corresponding to the screen describes some information of the control in the screen, including the class name, package name, coordinates, resource ID, text attribute, description and other information corresponding to the control. The layout file extraction module completes the extraction of the layout file of the cloth screen. The extraction of layout files can be obtained using uiautomatorviewer.bat in the android SDK.

3. Screen projection on the terminal On and off condition Preset module: After the channel is successfully created by the screen, the terminal prompts the user to preset the screen projection on and off conditions. The format of the condition is “action: operation object”.

The establishment of the preset condition may display some scenes to the user after the user establishes the screen projection channel, and the user operates in the scene, and the “action: operation object” of the operation will be recorded as a preset condition. For example, after the screen projection channel is successfully established, the displayed scene can be started for playing in the video player, prompting the user to set the click play button as a preset condition.

The conditional preset module can also be dynamically added during the projection of the screen, and the user can mark the action as a preset condition after operating on a control object.

A set of preset conditions are grouped, including open and close conditions, meaning that if the user's current operation meets the open screen projection condition in a certain set of preset conditions, then the condition of the end screen is also in the set of conditions. in. A new set of screen projection presets can only be opened after the current group's presets are turned on and off. For example, if the screen projection is turned on by the open screen projection condition in a certain group (for example, group A) preset conditions, the end screen projection can be turned off only by the group A screen closing condition of group A.

As a solution, it is also possible to start and close the screen projection by turning on the screen projection condition and turning off the screen projection condition in different preset conditions according to the scene. For example, by turning on the screen projection by turning on the screen projection condition in the A group preset condition, the screen projection can be turned off by the off screen projection condition in the B group preset condition. In this scenario, although the control of the screen projection is more flexible, the screen projection control has an unstable problem. The specific method of screen projection control can be set according to actual needs. The preset conditions for the aforementioned screen projections can be as shown in Table 1. The following opening and closing refers to the opening and closing of the screen projection.

Table 1: Preset condition table for screen projection

4. Comparison module on the terminal: Each step of the user on the terminal interface, the monitoring module 2 on the terminal compares the monitored user's operation (action: operation object) with the content in the preset module. For example, the user clicks a control to extract the control attribute in the layout file, and compares the action and the control attribute in the preset condition. If they are consistent, the corresponding open screen projection operation in the preset condition is executed, and the group preset is extracted at this time. The condition corresponding to the condition of closing the screen projection, if the value of the monitored control attribute is consistent with the value of the preset closed screen projection space, the screen off function is performed.

The module involved in the method 2 mainly has the following modules: a monitoring module on the terminal, an instruction sending module of the terminal, a mirror data sending module of the terminal, and a monitoring module on the display device, and the method 1 is described below in combination with each module.

1. The monitoring module on the terminal: used to monitor whether the user clicks the screen projection icon to turn on or off the screen projection function, and if the user clicks the icon, the terminal instruction sending module is called.

2. The terminal's instruction sending module: monitors the monitoring module to send the content to the display device. An identification bit is added to each frame of data to indicate that the content of the frame transmission is a control instruction.

For example, Miracast-based transmission uses the RTP protocol to transmit data in the form of "frames", set the padding position to "1", fill an octet at the end of the message, and use this 8 group to represent the transmission. The type of control instruction and the content of the control instruction, the first bit is set to "0" to indicate that it is a control instruction, and the next 7 bits represent the content of the control instruction. The contents of the instruction include turning on the screen projection and turning off the screen projection, and also comparing Easy to use with 7-bit binary numbers.

3. The mirror data sending module of the terminal: sends the mirrored content of the terminal device interface to the display device.

For example, the Miracast-based transmission uses the RTP protocol to transmit data in the form of "frames", setting the padding position to "0" or the content of the padding bit to "1" but the first bit of the 8-bit header at the end of the message is "1". ", indicating that the content of the frame is mirrored data content.

4. The monitoring module on the display device: monitors the content sent by the terminal, extracts the content of the padding bit, and if it is "1", extracts the 8-bit content at the end of the message. If the first bit is "0", it indicates that it is a control command. The content of the instruction, parsing the content of the instruction, and executing the parsed content; if the content of the padding bit is "0" or the content of the padding bit is "1" but the first bit of the 8 bits at the end of the message is "1", the mirrored content will be The mirrored content is displayed on the display device using existing Miracast technology.

The screen projection method of the present exemplary embodiment will be described below.

FIG. 3 is a flowchart 1 of a screen projection method according to an exemplary embodiment of the present disclosure, and the flowchart illustrated in FIG. 3 may be summarized as the following steps:

Step 1: The terminal and the display device establish a Miracast connection through the WiFi Direct. After the connection is established, the screen projection is started, the terminal starts the terminal monitoring module, and the display device starts the monitoring module of the display device.

Step 2: Establish a preset database; the preset database may be established before the channel is established, or concurrently with the channel establishment. If established before the channel is established, the preset database can be sent to the display device through the setup channel after the channel is established. While the screen projection channel is established, the user displays some scenes in which the user operates, and the "action: operation object" of the operation will be recorded as a preset condition.

Step 3: The user operates on the terminal interface, and the terminal monitoring device monitors the control attribute of the current user operation, and uses the operation object generation module to generate the control object by using the control attribute, and compares with the control object in the preset condition, and if the control object is consistent, check again. Whether the operation actions are consistent, if they are consistent, the corresponding open screen operation in the preset condition table is started; at the same time, the condition for closing the screen projection in the set of preset conditions is extracted, and the attribute and attribute values of the control object are extracted, and the monitoring module is extracted. Once the properties and attribute values of the controls in the current screen are monitored and the preset values are always on, the screen operation is closed, and step 4 is performed;

Step 4: After the monitoring module on the terminal monitors, send a control command to the display device, and after the display device receives the response, send the response to the terminal; if the user's operation is to turn on the screen projection, step 5 is performed; if the user operation is closed Screen projection, then perform step 6;

Step 5: The terminal transmits the screen interface image to the display device, and the display device receives and displays;

In step 6, the terminal stops the screen interface image transmission to the display device, and the display device stops receiving.

FIG. 4 is a flowchart 2 of a screen projection method according to an exemplary embodiment of the present disclosure, and the flowchart illustrated in FIG. 4 may be summarized as the following steps:

Step 1: The terminal and the display device establish a Miracast connection through the WiFi Direct, the terminal starts the monitoring module, and the display device starts the monitoring module;

Step 2: The user operates on the terminal interface. When the user wants to display the content after the interface starts on the display device or does not display on the display device, then click the open screen projection icon on the current interface or close. Screen projection icon, go to step 3;

Step 3: After the monitoring module on the terminal monitors, send a control command to the display device, and after the display device receives the response, send the response to the terminal; if the user's operation is to turn on the screen projection, step 4 is performed; if the user operation is closed Screen projection, then perform step 5;

Step 4: The terminal transmits the screen interface image to the display device, and the display device receives and displays;

In step 5, the terminal stops the screen interface image transmission to the display device, and the display device stops receiving.

Through the above solution of the exemplary embodiment of the present disclosure, the terminal screen projection can be flexibly controlled to meet different operation requirements of the user, and the user experience is improved.

Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases, the former is The often used implementation. Based on this understanding, the solution of the present disclosure may be embodied in the form of a software product stored in a storage medium (such as a ROM/RAM, a magnetic disk, an optical disk), and includes a plurality of instructions for making one The terminal device (which may be a cell phone, computer, server, or network device, etc.) performs the methods described in various embodiments of the present disclosure.

Example 2

A screen projection device is also provided in the embodiment of the present disclosure, which is used to implement the above-described embodiments and exemplary embodiments, and details are not described herein. As used below, the term "module" may implement a combination of software and/or hardware of a predetermined function. Although the devices described in the following embodiments are typically implemented in software, hardware, or a combination of software and hardware, is also possible and contemplated.

FIG. 5 is a structural block diagram 1 of a screen projection apparatus according to an embodiment of the present disclosure. As shown in FIG. 5, the apparatus includes:

The determining module 52 is configured to determine that a screen projection channel is established between the terminal and the display device;

The triggering module 54 is connected to the determining module 52, and configured to send a control command to the display device according to the preset triggering condition, wherein the control command is used to control the screen content of the terminal to be projected onto the display device, and the control command is established by The screen cast channel is sent.

FIG. 6 is a structural block diagram of a triggering module 54 of a screen projection apparatus according to an embodiment of the present disclosure. As shown in FIG. 6, the triggering module 54 includes:

The monitoring unit 62 is configured to monitor an operation action generated by the terminal;

The first determining unit 64 is connected to the monitoring unit 62, and configured to determine whether the monitored operation action matches the preset action;

The first sending unit 66 is connected to the first determining unit 64, and configured to send a control command corresponding to the operation action to the display device when the determination result of the first determining unit 64 is matched, wherein the control command is used by the control device. Instructing to start or stop projecting the screen content of the terminal onto the display device.

In an embodiment, the monitoring unit 62 is further configured to monitor an operation action generated by at least one of the following components: a component, an application control, and an operation key.

FIG. 7 is a structural block diagram 2 of a triggering module 54 of a screen projection apparatus according to an embodiment of the present disclosure. As shown in FIG. 7, the triggering module 54 includes:

The second determining unit 72 is configured to determine whether the current time meets a preset time condition for controlling the screen projection time;

The second sending unit 74 is connected to the second determining unit 72, and configured to send a control command to the display device when the determination result of the second determining unit 72 is matched, wherein the control command is used to indicate the screen of the terminal. The content starts or stops being projected onto the display device.

FIG. 8 is a structural block diagram 2 of a screen projection apparatus according to an embodiment of the present disclosure. As shown in FIG. 8 , the apparatus includes: in addition to all the modules shown in FIG. 5 , the apparatus includes:

The first sending module 82 is configured to, when the control command is used to indicate that the screen content of the terminal is started to be projected onto the display device, send a mirror image of the screen content of the terminal to the display device by using a data transmission instruction.

9 is a structural block diagram 3 of a screen projection apparatus according to an embodiment of the present disclosure. As shown in FIG. 9, the apparatus includes: a second sending module 92, in addition to all the modules shown in FIG.

The determining module 52 is further configured to: when the control instruction is used to indicate that the screen content of the terminal is started to be projected onto the display device, determine the display content of the preset area on the screen of the terminal for projecting to the display device;

The second sending module 92 is configured to send the determined display content of the preset area to the display device for projection.

A terminal is also provided in an embodiment of the present disclosure, the terminal comprising the screen projection device of any of the above embodiments.

The above modules may be implemented by software or hardware. For the latter, the foregoing may be implemented by, but not limited to, the above modules are all located in the same processor; or, the above modules are respectively located in different combinations. In the processor.

Example 3

A storage medium is provided in an embodiment of the present disclosure. In this embodiment, the above storage medium may be configured to store program code for performing the following steps:

S1, determining that a screen projection channel is established between the terminal and the display device;

S2, according to the preset trigger condition, triggering to send a control instruction to the display device, wherein the control instruction is used to control the screen content of the terminal to be projected onto the display device, and the control command is sent through the established screen projection channel.

In this embodiment, the storage medium may also be arranged to store program code for performing the following steps:

According to the preset trigger condition, triggering the sending of the control instruction to the display device includes:

S1, monitoring an operation action generated by the terminal;

S2, determining whether the monitored operation action matches the preset action;

S3. If the determination result is a match, send a control instruction corresponding to the operation action to the display device, where the control instruction is used to indicate that the screen content of the terminal is started to be projected or stopped to be projected onto the display device.

In this embodiment, the storage medium may be further configured to store program code for performing the following steps: monitoring the operation action generated by the terminal comprises: monitoring an operation action generated by at least one of the following: the component, the application control, the operation key .

S1, determining whether the current time meets a preset time condition for controlling a screen projection time;

S2. If the result of the determination is that the preset time condition is met, the control instruction is sent to the display device, where the control instruction is used to indicate that the screen content of the terminal is started or stopped being projected onto the display device.

In this embodiment, the storage medium may be further configured to store program code for performing the following steps: in the case where the control instruction is used to indicate that the screen content of the terminal is started to be projected onto the display device, in accordance with the preset trigger condition After triggering the sending of the control instruction to the display device, the method further includes: sending, by using the data transmission instruction, the image of the screen content of the terminal to the display device.

In the case that the control instruction is used to indicate that the screen content of the terminal is started to be projected on the display device, after triggering the sending of the control instruction to the display device according to the preset trigger condition, the method further includes:

S1. Determine display content of a preset area on the screen of the terminal for projecting to the display device;

S2: Send the determined display content of the preset area to the display device for projection.

In this embodiment, the foregoing storage medium may include, but not limited to, a USB flash drive, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk. A variety of media that can store program code.

In this embodiment, the processor may be configured to: according to the stored program code in the storage medium, determine that a screen projection channel is established between the terminal and the display device; and trigger a control instruction to send to the display device according to the preset trigger condition, where the control The instruction is used to control the screen content of the terminal to be projected onto the display device, and the control command is sent through the established screen projection channel.

In this embodiment, the processor may be executed according to the stored program code in the storage medium: triggering the sending of the control instruction to the display device according to the preset trigger condition includes: monitoring an operation action generated by the terminal; determining the monitored operation action and the pre-operation If the result of the determination is a match, the control command corresponding to the operation action is sent to the display device, wherein the control command is used to indicate that the screen content of the terminal is started to be projected or stopped to be projected onto the display device.

In this embodiment, the processor may be executed according to the stored program code in the storage medium: the operation action generated by the monitoring terminal includes: monitoring an operation action generated by at least one of the following: the component, the application control, and the operation key.

In this embodiment, the processor may execute according to the stored program code in the storage medium: triggering the sending of the control instruction to the display device according to the preset trigger condition includes: determining whether the current time meets a preset time for controlling the screen projection time Condition; if the result of the determination is that the preset time condition is met, the control instruction is sent to the display device, wherein the control instruction is used to indicate that the screen content of the terminal is started or stopped being projected onto the display device.

In this embodiment, the processor may execute according to the stored program code in the storage medium: in the case that the control instruction is used to indicate that the screen content of the terminal is started to be projected onto the display device, triggering according to the preset trigger condition After the display device sends the control instruction, the method further includes: sending, by using the data transmission instruction, the image of the screen content of the terminal to the display device.

In this embodiment, the processor may execute according to the stored program code in the storage medium: in the case that the control instruction is used to indicate that the screen content of the terminal is started to be projected onto the display device, triggering according to the preset trigger condition After the display device sends the control instruction, the method further includes: determining display content of the preset area for projecting to the display device on the screen of the terminal; and transmitting the determined display content of the preset area to the display device for projection.

For specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiments and exemplary embodiments, and details are not described herein again.

Those of ordinary skill in the art will appreciate that all or some of the steps, systems, and functional blocks/units of the methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be composed of several physical The components work together. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer readable medium, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and nonvolatile, implemented in any method or technology for storing information, such as computer readable instructions, data structures, program modules or other data. Sex, removable and non-removable media. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridge, magnetic tape, magnetic disk storage or other magnetic storage device, or may Any other medium used to store the desired information and that can be accessed by the computer. Moreover, it is well known to those skilled in the art that communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media. .

The above description is only exemplary embodiments of the present disclosure, and is not intended to limit the disclosure, and various changes and modifications may be made to the present disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and scope of the present disclosure are intended to be included within the scope of the present disclosure.

Industrial applicability

Claims

A screen projection method includes:

Determining that a screen projection channel is established between the terminal and the display device (S202);

Trimming to send a control instruction to the display device according to a preset trigger condition (S204), wherein the control instruction is used to control screen content of the terminal to be projected onto the display device, and the control instruction is established by The screen projection channel is transmitted.
The method according to claim 1, wherein triggering the sending of the control instruction to the display device according to the preset trigger condition (S204) comprises:

Monitoring operation actions generated by the terminal;

Determining whether the monitored operation action matches the preset action;

When the result of the determination is a match, a control instruction corresponding to the operation action is sent to the display device, wherein the control instruction is used to indicate that the screen content of the terminal is started to be projected or stopped to be projected to the Display on the device.
The method of claim 2, wherein monitoring the operational actions generated by the terminal comprises:

Monitoring operational actions generated by at least one of the following: the component, the application control, and the operation key.
The method according to claim 1, wherein triggering the sending of the control instruction to the display device according to the preset trigger condition (S204) comprises:

Determining whether the current time meets a preset time condition for controlling a screen projection time;

Sending, to the display device, the control instruction, where the result of the determination is that the preset time condition is met, wherein the control instruction is used to indicate that the screen content of the terminal is started or stopped to be projected to the display On the device.
The method according to any one of claims 1 to 4, wherein, in the case where the control instruction is used to instruct to start projecting screen content of the terminal onto the display device, according to the preset After the triggering condition is triggered to send the control instruction to the display device (S204), the method further includes:

A mirror image of the screen content of the terminal is transmitted to the display device by a data transfer instruction.
The method according to any one of claims 1 to 4, wherein, in the case where the control instruction is used to instruct to start projecting screen content of the terminal onto the display device, according to the preset After the triggering condition is triggered to send a control instruction to the display device (S204), the method further includes:

Determining display content of a preset area on the screen of the terminal for projecting to the display device;

And transmitting the determined display content of the preset area to the display device for projection.
A screen projection device comprising:

a determining module (52) configured to determine that a screen projection channel is established between the terminal and the display device;

The triggering module (54) is configured to trigger sending a control instruction to the display device according to the preset triggering condition, where the control command is used to control screen content of the terminal to be projected to the display device, where the control The instruction is sent through the established screen projection channel.
The apparatus of claim 7, wherein the triggering module (54) comprises:

a monitoring unit (62) configured to monitor an operation action generated by the terminal;

The first determining unit (64) is configured to determine whether the monitored operation action matches the preset action;

The first sending unit (66) is configured to, when the determination result of the first determining unit (64) is matched, send a control instruction corresponding to the operating action to the display device, where The control instruction is for instructing to start or stop projecting the screen content of the terminal onto the display device.
The apparatus of claim 8, wherein the monitoring unit (62) is further configured to monitor an operational action generated by at least one of the following: a component, an application control, an operation key.
The apparatus of claim 7, wherein the triggering module (54) comprises:

a second determining unit (72) configured to determine whether the current time meets a preset time condition for controlling a screen projection time;

a second sending unit (74) configured to send the control instruction to the display device if the determination result of the second determining unit (72) is a match, wherein the control instruction is used to indicate that The screen content of the terminal begins or stops being projected onto the display device.
The apparatus according to any one of claims 7 to 10, further comprising:

a first sending module (82) configured to: when the control instruction is used to indicate that the screen content of the terminal is started to be projected onto the display device, by using a data transfer instruction, the screen content of the terminal is The image is sent to the display device.
The apparatus according to any one of claims 7 to 10, further comprising: a second transmitting module (92), wherein

The determining module (52) is further configured to: when the control instruction is used to indicate that the screen content of the terminal is started to be projected onto the display device, determine that the screen of the terminal is used to Displaying the display content of the preset area projected by the device;

The second sending module (92) is configured to send the determined display content of the preset area to the display device for projection.
A terminal comprising the screen projection device of any one of claims 7 to 12.
A computer readable storage medium storing computer executable instructions that, when executed by a processor, implement the method of any of claims 1-6.