WO2017088429A1 - Method and system for controlling touchscreen application program in display terminal by mobile terminal - Google Patents

Abstract

A method for controlling a touchscreen application program in a display terminal by a mobile terminal comprises the following steps: when detecting a click operation of a user on a preset universal button, the mobile terminal acquires coordinate information corresponding to the preset universal button clicked by the user, a mapping relationship between different universal buttons and different coordinate information being preset in the mobile terminal (S10); the mobile terminal sends the coordinate information to a display terminal (S20); the display terminal identifies a click instruction in a second coordinate position of the display terminal according to the coordinate information (S30); and the display terminal controls a currently-running touchscreen application program according to the click instruction (S40). A system for controlling a touchscreen application program in a display terminal by a mobile terminal. A touchscreen application program running in a display terminal is controlled by means of a mobile terminal.

Description

 Method and system for controlling touch screen application on display terminal by mobile terminal

Technical field

The present invention relates to the field of intelligent control technologies, and in particular, to a method and system for a mobile terminal to control a touch screen application on a display terminal.

Background technique

With the development of mobile terminals, touch operation functions have become the basic functions of most mobile terminals, and more and more touch screen applications can be used by mobile terminals. Due to the versatility of the mobile terminal and the display terminal platform, many mobile terminal-based touch screen applications can be directly installed and run on the display terminal. However, the touch screen application is operated and controlled based on the touch screen, and The display terminal does not have a touch screen function or is inconvenient to touch the display terminal due to the distance being too far. The user is inconvenient to directly perform touch control on the display terminal when using the touch screen application on the display terminal. Since the mobile terminal can access and control the display terminal very conveniently through the home wireless local area network, there is an urgent need for a method for controlling the touch screen application running on the display terminal through the mobile terminal.

Summary of the invention

A main object of the present invention is to provide a method and system for controlling a touch screen application on a display terminal by a mobile terminal, and aims to control a touch screen application running on the display terminal through the mobile terminal.

To achieve the above objective, the present invention provides a method for a mobile terminal to control a touch screen application on a display terminal, the method comprising the following steps:

When the mobile terminal receives the calibration instruction of the preset universal button on the mobile terminal, the mobile terminal acquires a screenshot of the control button on the display terminal that includes the currently running touch screen application;

The mobile terminal displays the screenshot and acquires a coordinate point of the user performing a click operation on the area where the control button is located in the screenshot displayed by the mobile terminal;

The mobile terminal acquires a coordinate percentage of the coordinate point in the screenshot, and sets a correspondence between the coordinate percentage and the preset universal button;

When the mobile terminal detects that the first coordinate position clicked by the user falls within the range of the preset universal button, acquiring the coordinate information preset corresponding to the preset universal button;

Transmitting, by the mobile terminal, the coordinate information to a display terminal;

The display terminal identifies a click instruction on the second coordinate position of the display terminal according to the coordinate information;

The display terminal controls the currently running touch screen application according to the click instruction.

Preferably, when the mobile terminal detects the click operation of the preset universal button, the step of acquiring the coordinate information corresponding to the preset universal button that the user clicks includes:

When the mobile terminal detects that the first coordinate position clicked by the user falls within the range of the preset universal button, the preset coordinate information corresponding to the preset universal button is acquired.

Preferably, the step of the display terminal identifying the click instruction on the second coordinate position of the display terminal according to the coordinate information comprises:

The display terminal acquires a second coordinate position on the display terminal according to the coordinate information, and identifies a click instruction on a control button on the display terminal, where the second coordinate position is within a range of the control button.

Preferably, the step of the mobile terminal acquiring the coordinate percentage of the coordinate point in the screenshot, and setting the correspondence between the coordinate percentage and the preset universal button comprises:

If the resolution of the mobile terminal is W1*H1, the resolution of the display terminal is W2*H2, and the coordinate point is (X1, Y1), calculate the coordinate percentage of the coordinate point in the screenshot. For P (X1) /W1, Y1 / H1), and set a correspondence relationship between the coordinate percentage P and the preset universal button;

The step of the display terminal identifying the click command on the second coordinate position of the display terminal according to the coordinate information further includes:

The display terminal calculates a second coordinate position on the display terminal according to the coordinate percentage P, and simulates a click action on the second coordinate position, injects into the system, and identifies the second coordinate position. Clicking on the instruction, wherein the coordinates of the second coordinate position are (X1) /W1*W2, Y1 /H1* H2).

The present invention also provides a method for a mobile terminal to control a touch screen application on a display terminal, the method comprising the following steps:

When the mobile terminal detects the click operation of the preset universal button, the mobile terminal acquires coordinate information corresponding to the preset universal button that the user clicks, wherein the mobile terminal is pre-set with a mapping relationship between different common buttons and different coordinate information. ;

Transmitting, by the mobile terminal, the coordinate information to a display terminal;

The display terminal identifies a click instruction on the second coordinate position of the display terminal according to the coordinate information;

The display terminal controls the currently running touch screen application according to the click instruction.

Preferably, when the mobile terminal detects the click operation of the preset universal button, the step of acquiring the coordinate information corresponding to the preset universal button that the user clicks includes:

When the mobile terminal detects that the first coordinate position clicked by the user falls within the range of the preset universal button, the preset coordinate information corresponding to the preset universal button is acquired.

Preferably, the step of the display terminal identifying the click instruction on the second coordinate position of the display terminal according to the coordinate information comprises:

The display terminal acquires a second coordinate position on the display terminal according to the coordinate information, and identifies a click instruction on a control button on the display terminal, where the second coordinate position is within a range of the control button.

Preferably, the step of obtaining the coordinate information corresponding to the preset universal button clicked by the user when the mobile terminal detects the click operation of the preset universal button further includes:

When the mobile terminal receives the calibration instruction of the preset universal button on the mobile terminal, the mobile terminal acquires a screenshot of the control button on the display terminal that includes the currently running touch screen application;

The mobile terminal displays the screenshot and acquires a coordinate point of the user performing a click operation on the area where the control button is located in the screenshot displayed by the mobile terminal;

The mobile terminal acquires a coordinate percentage of the coordinate point in the screenshot, and sets a correspondence between the coordinate percentage and the preset universal button.

Preferably, the step of the mobile terminal acquiring the coordinate percentage of the coordinate point in the screenshot, and setting the correspondence between the coordinate percentage and the preset universal button comprises:

If the resolution of the mobile terminal is W1*H1, the resolution of the display terminal is W2*H2, and the coordinate point is (X1, Y1), calculate the coordinate percentage of the coordinate point in the screenshot. For P (X1) /W1, Y1 / H1), and set a correspondence relationship between the coordinate percentage P and the preset universal button;

The step of the display terminal identifying the click command on the second coordinate position of the display terminal according to the coordinate information further includes:

The display terminal calculates a second coordinate position on the display terminal according to the coordinate percentage P, and simulates a click action on the second coordinate position, injects into the system, and identifies the second coordinate position. Clicking on the instruction, wherein the coordinates of the second coordinate position are (X1) /W1*W2, Y1 /H1* H2).

In addition, in order to achieve the above object, the present invention further provides a system for a mobile terminal to control a touch screen application on a display terminal, the system comprising: a mobile terminal and a display terminal, wherein

When the mobile terminal detects a click operation of the preset universal button, the mobile terminal acquires coordinate information corresponding to the preset universal button that the user clicks, wherein the mobile terminal is pre-set with different common buttons and different coordinate information. Mapping relationship; sending the coordinate information to the display terminal;

The display terminal identifies a click instruction on the second coordinate position of the display terminal according to the coordinate information; and controls the currently running touch screen application according to the click instruction.

Preferably, the mobile terminal is further configured to:

When it is detected that the first coordinate position clicked by the user falls within the range of the preset universal button, the coordinate information preset corresponding to the preset universal button is acquired.

Preferably, the display terminal is further configured to:

Obtaining a second coordinate position on the display terminal according to the coordinate information, and identifying a click instruction on a control button on the display terminal, where the second coordinate position is within a range of the control button.

Preferably, the mobile terminal is further configured to:

Receiving, by the user, a calibration instruction of the preset universal button on the mobile terminal, acquiring a screenshot of the control button on the display terminal that includes the currently running touch screen application; displaying the screenshot and acquiring the user's display on the mobile terminal The coordinate point of the click control operation in the area where the control button is located in the screenshot; the coordinate percentage of the coordinate point in the screenshot is obtained, and the correspondence between the coordinate percentage and the preset universal button is set.

Preferably, the mobile terminal is further configured to:

If the resolution of the mobile terminal is W1*H1, the resolution of the display terminal is W2*H2, and the coordinate point is (X1, Y1), calculate the coordinate percentage of the coordinate point in the screenshot. For P (X1) /W1, Y1 / H1), and set a correspondence relationship between the coordinate percentage P and the preset universal button;

The display terminal is further configured to:

Calculating a second coordinate position on the display terminal according to the coordinate percentage P, and simulating a click action on the second coordinate position, injecting into the system, and identifying a click instruction on the second coordinate position, wherein The coordinates of the second coordinate position are (X1) /W1*W2, Y1 /H1* H2).

The invention provides a method and a system for controlling a touch screen application program on a display terminal by using a mobile terminal, and using the mobile terminal to acquire a preset universal button clicked by the user on the mobile terminal, and setting coordinate information corresponding to the preset universal button Sending to the display terminal, the display terminal can acquire the second coordinate position on the display terminal according to the coordinate information, and identify a click instruction on the second coordinate position to touch the touch screen application currently running on the display terminal Take control. When the user clicks the preset universal button on the mobile terminal, the display terminal can convert and recognize the click instruction on the second coordinate position of the display terminal according to the coordinate information corresponding to the preset universal button, and therefore, the user A corresponding click command may be generated in the display terminal by performing a click operation on the mobile terminal, so that the display terminal controls the touch screen application according to the click instruction, thereby implementing the touch screen application running on the display terminal by the mobile terminal.

DRAWINGS

1 is a schematic flowchart of a first embodiment of a method for a mobile terminal to control a touch screen application on a display terminal according to the present invention;

2 is a schematic flowchart of a second embodiment of a method for a mobile terminal to control a touch screen application on a display terminal according to the present invention;

FIG. 3 is a schematic structural diagram of a system for controlling a touch screen application program on a display terminal of a mobile terminal according to the present invention.

The implementation, functional features, and advantages of the present invention will be further described in conjunction with the embodiments.

detailed description

It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The invention provides a method for a mobile terminal to control a touch screen application on a display terminal.

Referring to FIG. 1, FIG. 1 is a schematic flowchart diagram of a first embodiment of a method for a mobile terminal to control a touch screen application on a display terminal according to the present invention.

In the first embodiment, the method for the mobile terminal to control the touch screen application on the display terminal includes:

In step S10, when the mobile terminal detects the click operation of the preset universal button, the mobile terminal acquires coordinate information corresponding to the preset universal button that the user clicks, wherein the mobile terminal is pre-set with different common buttons and different coordinate information. Mapping relationship;

In this embodiment, the mobile terminal is provided with a plurality of common buttons commonly used in virtual handles, such as above, lower, left, right, hit, jump, etc., to meet button support in the touch screen application running on the display terminal. . Real-time monitoring of the user's touch screen operation on the mobile terminal, such as a click operation on the preset universal button, and when detecting the user's click operation on the preset universal button on the mobile terminal, acquiring the preset universal button corresponding to the user click Coordinate information. The mapping relationship between different common buttons and different coordinate information is pre-set on the mobile terminal. The coordinate information may be information such as a specific coordinate point position, a coordinate percentage, and the like, such as an abscissa of the coordinate point, a width of the ordinate relative to the width of the touch screen area, a percentage of the length, and the like. It should be noted that, in this embodiment, the preset coordinate information corresponding to the universal button is only related to the position of the control button of the currently running touch screen application on the display terminal, and is independent of the position of the universal button on the mobile terminal. . For example, the coordinate information is taken as a coordinate point position. If the coordinate point position of the control target A of the touch screen application currently running on the display terminal is a, the user needs to use the preset universal button B on the mobile terminal to control When the target A is controlled, the mapping relationship between the universal button B and the coordinate point position a may be preset on the mobile terminal. In this way, when the mobile terminal detects the click operation of the universal button B by the user, the coordinate information corresponding to the common button B clicked by the user, that is, the coordinate point position a can be acquired.

Step S20, the mobile terminal sends the coordinate information to the display terminal;

The mobile terminal sends the coordinate information corresponding to the preset universal button obtained by the user when the user clicks the preset universal button operation to the display terminal by using the home wireless local area network, the UPnP protocol, or the like. For example, the coordinate point position a corresponding to the acquired universal button B clicked by the user can be sent to the display terminal.

Step S30, the display terminal identifies a click instruction on the second coordinate position of the display terminal according to the coordinate information;

After the display terminal obtains the coordinate information sent by the mobile terminal, the second coordinate position on the display terminal may be calculated and obtained according to the coordinate information, such as the coordinate point position sent by the display terminal according to the mobile terminal. The control target A in the touch screen application currently running in the display terminal corresponding to the coordinate point position a can be identified, the control target A is also the second coordinate position, and a mouse click is simulated according to the second coordinate position. The action is input to the display terminal system as a touch screen event, and the display terminal can recognize the click command at the second coordinate position of the display terminal, that is, the control target A.

The display terminal is configured to recognize the second coordinate position on the display terminal according to the coordinate information corresponding to the preset universal button sent by the mobile terminal, so the user clicks on the mobile terminal It is assumed that the relative position of the universal button in the mobile terminal can be completely different from the second coordinate position. In this way, the user can perform a click operation on different common buttons on the mobile terminal according to the target point that needs to be controlled on the display terminal, so that the display terminal can obtain coordinate information corresponding to different common buttons of the click. To identify click commands for different control targets on the display terminal.

Step S40: The display terminal controls the currently running touch screen application according to the click instruction.

After the display terminal recognizes the click command on the second coordinate position of the display terminal, the display terminal may respond to the recognized click command, and directly respond to the click operation effect on the display terminal, thereby responding to the click command. To control the touch screen application currently running on the display terminal.

In this embodiment, the mobile terminal is used to acquire a preset universal button that is clicked by the user on the mobile terminal, and the coordinate information corresponding to the preset universal button is sent to the display terminal, and the display terminal can acquire the display terminal according to the coordinate information. And a second coordinate position on the second coordinate position and identifying a click command at the second coordinate position to control a touch screen application currently running on the display terminal. When the user clicks the preset universal button on the mobile terminal, the display terminal can convert and recognize the click instruction on the second coordinate position of the display terminal according to the coordinate information corresponding to the preset universal button, and therefore, the user A corresponding click command may be generated in the display terminal by performing a click operation on the mobile terminal, so that the display terminal controls the touch screen application according to the click instruction, thereby implementing the touch screen application running on the display terminal by the mobile terminal.

Further, in other embodiments, the foregoing step S10 may include:

When the mobile terminal detects that the first coordinate position clicked by the user falls within the range of the preset universal button, the preset coordinate information corresponding to the preset universal button is acquired.

In this embodiment, when the user clicks on the mobile terminal and obtains the first coordinate position of the user click, if the first coordinate position is determined to fall within the range of the preset universal button on the mobile terminal, Corresponding coordinate information corresponding to the preset universal button. The preset universal button is preset with corresponding coordinate information, so that the same universal and the preset can be obtained as long as the user clicks on any coordinate point within the range of the preset universal button on the mobile terminal. The coordinate information corresponding to the button, the display terminal may identify a click instruction at the second coordinate position of the display terminal according to the coordinate information, and control the currently running touch screen application. In this embodiment, the user only needs to click on any coordinate point within the range of the preset universal button on the mobile terminal to perform the same control on the touch screen application currently running on the display terminal, which is more convenient for the user to pass the mobile terminal. Controls the touch screen application running on the display terminal.

Further, in other embodiments, the foregoing step S30 may include:

The display terminal acquires a second coordinate position on the display terminal according to the coordinate information, and identifies a click instruction on a control button on the display terminal, where the second coordinate position is within a range of the control button.

In this embodiment, after the display terminal acquires the second coordinate position on the display terminal according to the coordinate information, further, if the second coordinate position falls within the range of the control button on the display terminal Then, it is recognized as a click command to the control button on the display terminal. In this way, not only the user can recognize the click command of a target point in the touch screen application running on the display terminal through the mobile terminal, but also recognize the click command of a button in the touch screen application running on the display terminal, which is applicable. More broad and more flexible.

As shown in FIG. 2, a second embodiment of the present invention provides a method for a mobile terminal to control a touch screen application on a display terminal. On the basis of the foregoing embodiment, before the step S10, the method further includes:

Step S50: When the mobile terminal receives the calibration instruction of the preset universal button on the mobile terminal, the mobile terminal acquires a screenshot of the control button on the display terminal that includes the currently running touch screen application.

Step S60, the mobile terminal displays the screenshot and acquires a coordinate point of the user performing a click operation on the area where the control button is located in the screenshot displayed by the mobile terminal;

Step S70: The mobile terminal acquires a coordinate percentage of the coordinate point in the screenshot, and sets a correspondence between the coordinate percentage and the preset universal button.

The embodiment further includes a calibration operation for presetting a universal button on the mobile terminal, and a virtual handle may be preset on the mobile terminal, where the virtual handle includes a plurality of common preset buttons, as above, below, and left. Right button, menu, selection and other function buttons, some common preset buttons in the virtual handle can meet the needs of buttons in most touch screen applications. When using the preset universal button in the mobile terminal to control the touch screen application in the display terminal, the preset universal button in the mobile terminal is first calibrated to be in the touch screen application in the display terminal. The corresponding control buttons are controlled accurately and efficiently.

When receiving the calibration instruction of the preset universal button on the mobile terminal, the mobile terminal acquires a full-screen screenshot of the control button on the display terminal that includes the currently running touch screen application, and the screenshot includes the current running touch screen application. All the control buttons used. The mobile terminal also displays the screenshot in full screen, and the user may perform a click operation on the control button in the screenshot according to the screenshot displayed on the mobile terminal, and acquire a control button in the screenshot displayed by the user on the mobile terminal. Performing a coordinate point of the click operation, acquiring a coordinate percentage of the coordinate point in the screenshot, and setting a correspondence relationship between the coordinate percentage and the preset universal button, and saving the coordinate percentage and the preset The correspondence of the universal buttons can complete the calibration of the preset universal buttons on the mobile terminal.

After the preset universal button is calibrated on the mobile terminal, in the subsequent operation, if the mobile terminal detects that the first coordinate position clicked by the user falls within the range of the preset universal button, the preset may be acquired. The universal button corresponds to the preset coordinate information, that is, the coordinate percentage, in the calibration operation, and the display terminal can obtain the control button that the user selects the click in the screenshot in the calibration operation according to the coordinate information, thereby realizing the user clicking on the mobile terminal. When the universal button is preset, a click command for the control button selected by the user in the screenshot is recognized on the display terminal, thereby controlling the current operation of the touch screen application on the display terminal according to the preset universal button on the mobile terminal The corresponding control button in .

Specifically, since the resolution of the display terminal and the resolution of the mobile terminal may be inconsistent, the coordinate point of the click is converted into coordinate information such as a coordinate percentage, and the coordinate percentage is associated with the universal button on the virtual handle, and in subsequent operations, the user Click the preset universal button on the virtual handle to send the coordinate percentage corresponding to the preset universal button to the display terminal. After receiving the coordinate percentage, the display terminal restores the coordinate percentage to the coordinate point according to the resolution of the screen, and then according to the The coordinate point simulates the action of a mouse click and is injected into the system to recognize the click command for the coordinate point. Specifically, when data conversion is performed between coordinate points and coordinate percentages, if the resolution of the mobile terminal is assumed to be W1*H1, the display terminal resolution is W2*H2. When the universal button B on the mobile terminal is calibrated, if the user clicks on the screenshot on the mobile terminal, the coordinate point P1 obtained by the control button A is (X1, Y1), the calculated coordinate percentage is P (X1) /W1,Y1 /H1), and set the correspondence between the coordinate percentage P and the universal button B. When the user controls the touch screen application on the display terminal through the mobile terminal, if the user clicks the click operation on the universal button B, the preset coordinate percentage P corresponding to the universal button B can be acquired, and the coordinate percentage P is obtained. Sending to the display terminal; after obtaining the coordinate percentage P, the display terminal can obtain the second coordinate position on the display terminal, that is, the coordinate point P2 (X1) according to the coordinate percentage P. /W1*W2, Y1 /H1* H2), and identifying a click command on the second coordinate position of the display terminal, that is, the coordinate point P2, that is, identifying a click command of the control button A on the display terminal, thereby controlling the currently running touch screen application .

Further, in other embodiments, the touch screen application is a game application that is operated by a touch button. Since the game application is mostly operated based on the touch screen button, since the display terminal generally has a larger screen, better stereo sound and the like than the mobile terminal, playing the game on the display terminal can bring the player with respect to the mobile terminal. For more sense of presence and realism, in this embodiment, the game application can be run on the display terminal and the buttons in the game application can be controlled and operated by the mobile terminal, which can bring better game effects to the user.

The present invention further provides a system in which a mobile terminal controls a touch screen application on a display terminal.

It should be noted that, in the embodiment of the present invention, the mobile terminal may be a terminal having a touch function such as a mobile phone or a pad, and the display terminal may be a television, a computer, or the like, which is not limited herein.

Referring to FIG. 3, FIG. 3 is a schematic structural diagram of a system for controlling a touch screen application program on a display terminal of a mobile terminal according to the present invention.

In an embodiment, the system for controlling the touch screen application on the display terminal by the mobile terminal comprises:

a mobile terminal 1 and a display terminal 2, wherein

When the mobile terminal 1 detects the click operation of the preset universal button, the mobile terminal 1 acquires coordinate information corresponding to the preset universal button that the user clicks, wherein the mobile terminal 1 is pre-set with a mapping relationship between different common buttons and different coordinate information. Sending the coordinate information to the display terminal 2;

The display terminal 2 identifies a click command at the second coordinate position of the display terminal 2 based on the coordinate information; and controls the currently running touch screen application according to the click command.

In this embodiment, the mobile terminal 1 is provided with a plurality of common buttons commonly used in virtual handles, such as above, below, left, right, hit, jump, etc., to meet the touch screen application running on the display terminal 2. Button support. Real-time monitoring of the user's touch screen operation on the mobile terminal 1 such as a click operation on the preset universal button, and when detecting the user's click operation on the preset universal button on the mobile terminal 1, acquiring the preset universal clicked by the user The coordinate information corresponding to the button. The mapping relationship between different common buttons and different coordinate information is preset on the mobile terminal 1. The coordinate information may be information such as a specific coordinate point position, a coordinate percentage, and the like, such as an abscissa of the coordinate point, a width of the ordinate relative to the width of the touch screen area, a percentage of the length, and the like. It should be noted that, in this embodiment, the preset coordinate information corresponding to the universal button is only related to the position of the control button of the touch screen application currently running on the display terminal 2, and the universal button is on the mobile terminal 1. Position is irrelevant. For example, the coordinate information is taken as a coordinate point position. If the coordinate point position of the control target A of the touch screen application currently running on the display terminal 2 is a, the user needs to use the preset universal button B on the mobile terminal 1. To control the control target A, the mapping relationship between the universal button B and the coordinate point position a can be preset on the mobile terminal 1. In this way, when the mobile terminal 1 detects the click operation of the universal button B by the user, the coordinate point position a corresponding to the common button B clicked by the user can be acquired.

The mobile terminal 1 transmits the coordinate information corresponding to the preset universal button acquired by the user when the user clicks the preset universal button operation to the display terminal 2 through the home wireless local area network, the UPnP protocol, or the like. For example, the coordinate point position a corresponding to the acquired universal button B clicked by the user can be sent to the display terminal 2.

After the display terminal 2 acquires the coordinate information sent by the mobile terminal 1, the second coordinate position on the display terminal 2 can be calculated according to the coordinate information, and the display terminal 2 can recognize the coordinate position a according to the coordinate terminal sent by the mobile terminal 1. The coordinate point position a corresponds to the control target A in the touch screen application currently running on the display terminal 2, the control target A is also the second coordinate position, and simulates a mouse click action according to the second coordinate position as a time The touch screen event is input to the display terminal 2 system, and the display terminal 2 can recognize the click command at the second coordinate position of the display terminal 2, that is, the control target A.

Since the display terminal 2 calculates the second coordinate position recognized on the display terminal 2 according to the coordinate information corresponding to the preset universal button transmitted by the mobile terminal 1, the preset universal button clicked by the user on the mobile terminal 1 is The relative position in the mobile terminal 1 and the second coordinate position may be completely different. In this way, the user can click on the different common buttons on the mobile terminal 1 according to the target point that is controlled on the display terminal 2, so that the display terminal 2 can recognize the coordinate information corresponding to the different common buttons that are clicked. Click commands for different control targets on the display terminal 2.

After the display terminal 2 recognizes the click command on the second coordinate position of the display terminal 2, it can respond to the recognized click command, and directly responds to the click operation effect on the display terminal 2, thereby displaying the response in response to the click command. The touch screen application currently running on terminal 2 is controlled.

In this embodiment, the mobile terminal 1 is used to acquire a preset universal button that is clicked by the user on the mobile terminal 1, and the coordinate information corresponding to the preset universal button is sent to the display terminal 2. The display terminal 2 can obtain the coordinate information according to the coordinate information. A second coordinate position on the terminal 2 is displayed, and a click command at the second coordinate position is recognized to control the touch screen application currently running on the display terminal 2. When the user clicks the preset universal button on the mobile terminal 1, the display terminal 2 can convert and recognize the click command on the second coordinate position of the display terminal 2 according to the coordinate information corresponding to the preset universal button. The user can generate a corresponding click instruction in the display terminal 2 by performing a click operation on the mobile terminal 1 for the display terminal 2 to control the touch screen application according to the click instruction, thereby implementing control of the display terminal 2 by the mobile terminal 1. Touchscreen app.

Further, in other embodiments, the mobile terminal 1 can be used to:

When it is detected that the first coordinate position clicked by the user falls within the range of the preset universal button, the coordinate information preset corresponding to the preset universal button is acquired.

In this embodiment, when the user clicks on the mobile terminal 1 and obtains the first coordinate position clicked by the user, if it is determined that the first coordinate position falls within the range of the preset universal button on the mobile terminal 1, And acquiring coordinate information preset corresponding to the preset universal button. The preset universal button is preset with corresponding coordinate information, so that the same and the preset can be obtained as long as the user clicks on any coordinate point within the range of the preset universal button on the mobile terminal 1. The coordinate information corresponding to the universal button, the display terminal 2 can recognize the click command at the second coordinate position of the display terminal 2 based on the coordinate information, and control the currently operating touch screen application. In this embodiment, the user only needs to click on any coordinate point within the range of the preset universal button on the mobile terminal 1 to perform the same control on the touch screen application currently running on the display terminal 2, which is more convenient for the user to pass the mobile terminal. 1 controls the display of the touch screen application running on the terminal 2.

Further, in other embodiments, the display terminal 2 can be used to:

Acquiring a second coordinate position on the display terminal 2 according to the coordinate information, and identifying a click command to the control button on the display terminal 2, the second coordinate position being within the range of the control button.

In this embodiment, after the display terminal 2 acquires the second coordinate position on the display terminal 2 according to the coordinate information, further, if the second coordinate position falls within the range of the control button on the display terminal 2, It is recognized as a click command to the control button on the display terminal 2. In this way, not only the user can recognize the click command of a target point in the touch screen application running on the display terminal 2 through the mobile terminal 1, but also recognize the click of a button in the touch screen application running on the display terminal 2. The instructions are more applicable and more flexible.

Further, in other embodiments, the mobile terminal 1 can also be used to:

Receiving a calibration instruction of the preset universal button on the mobile terminal 1 by the user, acquiring a screenshot of the control button on the display terminal 2 including the currently running touch screen application; displaying the screenshot and acquiring the user's display to the mobile terminal 1 In the screenshot, the coordinate point of the area where the control button is located is clicked; the coordinate percentage of the coordinate point in the screenshot is obtained, and the correspondence between the coordinate percentage and the preset universal button is set.

In this embodiment, the calibration operation of the preset universal button on the mobile terminal 1 is further included, and the virtual handle can be set in advance on the mobile terminal 1, and the virtual handle includes a plurality of common preset buttons, as above, below, left, and right. Direction buttons, menus, selections, and other function buttons. Several common preset buttons in the virtual handle can meet the needs of buttons in most touch screen applications. When the touch screen application in the display terminal 2 is controlled by using the preset universal button in the mobile terminal 1, the preset universal button in the mobile terminal 1 is first calibrated to touch the touch screen in the display terminal 2. The corresponding control buttons in the application are controlled accurately and efficiently.

When receiving the calibration instruction of the preset universal button on the mobile terminal 1 , the mobile terminal 1 acquires a full-screen screenshot of the control button on the display terminal 2 including the currently running touch screen application, and the screenshot includes the currently running touch screen application. All the control buttons you need. The mobile terminal 1 also displays the screenshot in full screen, and the user can perform a click operation on the control button in the screenshot according to the screenshot displayed on the mobile terminal 1, and obtain a click operation on the control button in the screenshot displayed by the mobile terminal 1 by the user. And a coordinate percentage of the coordinate point in the screenshot, and setting a correspondence between the coordinate percentage and the preset universal button, and saving the coordinate percentage and the preset universal button Corresponding relationship, the calibration of the preset universal button on the mobile terminal 1 can be completed.

After the preset universal button is calibrated on the mobile terminal 1, in the subsequent operation, if the mobile terminal 1 detects that the first coordinate position clicked by the user falls within the range of the preset universal button, the preset may be acquired. The universal button corresponds to the preset coordinate information, that is, the coordinate percentage, in the calibration operation, and the display terminal 2 can obtain the control button that the user selects the click in the screenshot in the calibration operation according to the coordinate information, thereby realizing the user in the mobile terminal 1 When the preset universal button is clicked, a click command for the control button selected by the user in the screenshot is recognized on the display terminal 2, thereby controlling the current operation on the display terminal 2 according to the preset universal button on the mobile terminal 1. Run the appropriate control button in the touchscreen application.

Specifically, since the resolution of the display terminal 2 and the resolution of the mobile terminal 1 may be inconsistent, the coordinate points of the click are converted into coordinate information such as a coordinate percentage, and the coordinate percentage is associated with the universal button on the virtual handle, in subsequent operations. The user clicks the preset universal button on the virtual handle, that is, sends the coordinate percentage corresponding to the preset universal button to the display terminal 2. After receiving the coordinate percentage, the display terminal 2 restores the coordinate percentage to the coordinate point according to the resolution of the screen. Then, according to the coordinate point to simulate a mouse click action, injected into the system, the click command for the coordinate point can be recognized. Specifically, when data conversion is performed between the coordinate point and the coordinate percentage, if the resolution of the mobile terminal 1 is assumed to be W1*H1, the display terminal 2 has a resolution of W2*H2. When the universal button B on the mobile terminal 1 is calibrated, if the user clicks on the screenshot on the mobile terminal 1 the coordinate point P1 obtained by the control button A is (X1, Y1), the calculated coordinate percentage is P (X1) /W1,Y1 /H1), and set the correspondence between the coordinate percentage P and the universal button B. When the user controls the touch screen application on the display terminal 2 through the mobile terminal 1, if the user clicks the click operation on the universal button B, the preset coordinate percentage P corresponding to the common button B can be acquired, and the coordinates are obtained. The percentage P is sent to the display terminal 2; after obtaining the coordinate percentage P, the display terminal 2 can obtain the second coordinate position on the display terminal 2, that is, the coordinate point P2 (X1) according to the coordinate percentage P. /W1*W2, Y1 /H1* H2), and a click command on the second coordinate position of the display terminal 2, that is, the coordinate point P2, that is, a click command for controlling the button A on the display terminal 2 is recognized, thereby controlling the currently operating touch screen application.

Further, in other embodiments, the touch screen application is a game application that is operated by a touch button. Since the game application is mostly operated based on the touch screen button, since the display terminal 2 generally has a larger screen, better stereo sound and the like than the mobile terminal 1, the game is played on the display terminal 2 with respect to the mobile terminal 1 The player can bring more sense of presence and realism to the player. In this embodiment, the game application can be run on the display terminal 2, and the button in the game application can be controlled and operated by the mobile terminal 1, which can bring the user Better game effects.

The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments. Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better. Implementation. Based on such understanding, the technical solution of the present invention, which is essential or contributes to the prior art, may be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, The optical disc includes a number of instructions for causing a terminal device (which may be a cell phone, a computer, a server, or a network device, etc.) to perform the methods described in various embodiments of the present invention.

The above are only the preferred embodiments of the present invention, and are not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the present invention and the drawings are directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Claims (16)

1.  A method for a mobile terminal to control a touch screen application on a display terminal, the method comprising the steps of:

   When the mobile terminal receives the calibration instruction of the preset universal button on the mobile terminal, the mobile terminal acquires a screenshot of the control button on the display terminal that includes the currently running touch screen application;

   The mobile terminal displays the screenshot and acquires a coordinate point of the user performing a click operation on the area where the control button is located in the screenshot displayed by the mobile terminal;

   The mobile terminal acquires a coordinate percentage of the coordinate point in the screenshot, and sets a correspondence between the coordinate percentage and the preset universal button;

   When the mobile terminal detects that the first coordinate position clicked by the user falls within the range of the preset universal button, acquiring the coordinate information preset corresponding to the preset universal button;

   Transmitting, by the mobile terminal, the coordinate information to a display terminal;

   The display terminal identifies a click instruction on the second coordinate position of the display terminal according to the coordinate information;

   The display terminal controls the currently running touch screen application according to the click instruction.
2. The method for controlling a touch screen application on a display terminal of the mobile terminal according to claim 1, wherein when the mobile terminal detects a click operation of the preset universal button, the mobile terminal acquires the preset universal click clicked by the user The steps of the coordinate information corresponding to the button include:

   When the mobile terminal detects that the first coordinate position clicked by the user falls within the range of the preset universal button, the preset coordinate information corresponding to the preset universal button is acquired.
3. The method for controlling a touch screen application on a display terminal of the mobile terminal according to claim 2, wherein the step of the display terminal identifying the click command at the second coordinate position of the display terminal according to the coordinate information comprises: :

   The display terminal acquires a second coordinate position on the display terminal according to the coordinate information, and identifies a click instruction on a control button on the display terminal, where the second coordinate position is within a range of the control button.
4. The method for controlling a touch screen application on a display terminal of the mobile terminal according to claim 1, wherein the mobile terminal acquires a coordinate percentage of the coordinate point in the screenshot, and sets the coordinate percentage and The step of the corresponding relationship of the preset universal buttons includes:

   If the resolution of the mobile terminal is W1*H1, the resolution of the display terminal is W2*H2, and the coordinate point is (X1, Y1), calculate the coordinate percentage of the coordinate point in the screenshot. For P (X1) /W1, Y1 / H1), and set a correspondence relationship between the coordinate percentage P and the preset universal button;

   The step of the display terminal identifying the click command on the second coordinate position of the display terminal according to the coordinate information further includes:

   The display terminal calculates a second coordinate position on the display terminal according to the coordinate percentage P, and simulates a click action on the second coordinate position, injects into the system, and identifies the second coordinate position. Clicking on the instruction, wherein the coordinates of the second coordinate position are (X1) /W1*W2, Y1 /H1* H2).
5. A method for a mobile terminal to control a touch screen application on a display terminal, the method comprising the steps of:

   When the mobile terminal detects the click operation of the preset universal button, the mobile terminal acquires coordinate information corresponding to the preset universal button that the user clicks, wherein the mobile terminal is pre-set with a mapping relationship between different common buttons and different coordinate information. ;

   Transmitting, by the mobile terminal, the coordinate information to a display terminal;

   The display terminal identifies a click instruction on the second coordinate position of the display terminal according to the coordinate information;

   The display terminal controls the currently running touch screen application according to the click instruction.
6. The method for controlling a touch screen application on a display terminal of the mobile terminal according to claim 5, wherein when the mobile terminal detects a click operation of the preset universal button, the mobile terminal acquires the preset universal click clicked by the user The steps of the coordinate information corresponding to the button include:

   When the mobile terminal detects that the first coordinate position clicked by the user falls within the range of the preset universal button, the preset coordinate information corresponding to the preset universal button is acquired.
7. The method for controlling a touch screen application on a display terminal of the mobile terminal according to claim 6, wherein the step of the display terminal identifying the click command at the second coordinate position of the display terminal according to the coordinate information comprises: :

   The display terminal acquires a second coordinate position on the display terminal according to the coordinate information, and identifies a click instruction on a control button on the display terminal, where the second coordinate position is within a range of the control button.
8. The method for controlling a touch screen application on a display terminal of the mobile terminal according to claim 7, wherein when the mobile terminal detects a click operation of the preset universal button, the mobile terminal acquires the preset universal click clicked by the user The steps of the coordinate information corresponding to the button also include:

   When the mobile terminal receives the calibration instruction of the preset universal button on the mobile terminal, the mobile terminal acquires a screenshot of the control button on the display terminal that includes the currently running touch screen application;

   The mobile terminal displays the screenshot and acquires a coordinate point of the user performing a click operation on the area where the control button is located in the screenshot displayed by the mobile terminal;

   The mobile terminal acquires a coordinate percentage of the coordinate point in the screenshot, and sets a correspondence between the coordinate percentage and the preset universal button.
9. The method for controlling a touch screen application on a display terminal of a mobile terminal according to claim 8, wherein the mobile terminal acquires a coordinate percentage of the coordinate point in the screenshot, and sets the coordinate percentage and The step of the corresponding relationship of the preset universal buttons includes:

   If the resolution of the mobile terminal is W1*H1, the resolution of the display terminal is W2*H2, and the coordinate point is (X1, Y1), calculate the coordinate percentage of the coordinate point in the screenshot. For P (X1) /W1, Y1 / H1), and set a correspondence relationship between the coordinate percentage P and the preset universal button;

   The step of the display terminal identifying the click command on the second coordinate position of the display terminal according to the coordinate information further includes:

   The display terminal calculates a second coordinate position on the display terminal according to the coordinate percentage P, and simulates a click action on the second coordinate position, injects into the system, and identifies the second coordinate position. Clicking on the instruction, wherein the coordinates of the second coordinate position are (X1) /W1*W2, Y1 /H1* H2).
10. A system for controlling a touch screen application on a display terminal of a mobile terminal, wherein the system comprises: a mobile terminal and a display terminal, wherein

    When the mobile terminal detects a click operation of the preset universal button, the mobile terminal acquires coordinate information corresponding to the preset universal button that the user clicks, wherein the mobile terminal is pre-set with different common buttons and different coordinate information. Mapping relationship; sending the coordinate information to the display terminal;

    The display terminal identifies a click instruction on the second coordinate position of the display terminal according to the coordinate information; and controls the currently running touch screen application according to the click instruction.
11. The system for controlling a touch screen application on a display terminal of the mobile terminal according to claim 10, wherein the mobile terminal is further configured to:

    When it is detected that the first coordinate position clicked by the user falls within the range of the preset universal button, the coordinate information preset corresponding to the preset universal button is acquired.
12. The system for controlling a touch screen application on a display terminal of the mobile terminal according to claim 11, wherein the display terminal is further configured to:

    Obtaining a second coordinate position on the display terminal according to the coordinate information, and identifying a click instruction on a control button on the display terminal, where the second coordinate position is within a range of the control button.
13. The system for controlling a touch screen application on a display terminal of the mobile terminal according to claim 11, wherein the mobile terminal is further configured to:

    Receiving, by the user, a calibration instruction of the preset universal button on the mobile terminal, acquiring a screenshot of the control button on the display terminal that includes the currently running touch screen application; displaying the screenshot and acquiring the user's display on the mobile terminal The coordinate point of the click control operation in the area where the control button is located in the screenshot; the coordinate percentage of the coordinate point in the screenshot is obtained, and the correspondence between the coordinate percentage and the preset universal button is set.
14. The system for controlling a touch screen application on a display terminal of the mobile terminal according to claim 12, wherein the mobile terminal is further configured to:

    Receiving, by the user, a calibration instruction of the preset universal button on the mobile terminal, acquiring a screenshot of the control button on the display terminal that includes the currently running touch screen application; displaying the screenshot and acquiring the user's display on the mobile terminal The coordinate point of the click control operation in the area where the control button is located in the screenshot; the coordinate percentage of the coordinate point in the screenshot is obtained, and the correspondence between the coordinate percentage and the preset universal button is set.
15. The system for controlling a touch screen application on a display terminal of the mobile terminal according to claim 13, wherein the mobile terminal is further configured to:

    If the resolution of the mobile terminal is W1*H1, the resolution of the display terminal is W2*H2, and the coordinate point is (X1, Y1), calculate the coordinate percentage of the coordinate point in the screenshot. For P (X1) /W1, Y1 / H1), and set a correspondence relationship between the coordinate percentage P and the preset universal button;

    The display terminal is further configured to:

    Calculating a second coordinate position on the display terminal according to the coordinate percentage P, and simulating a click action on the second coordinate position, injecting into the system, and identifying a click instruction on the second coordinate position, wherein The coordinates of the second coordinate position are (X1) /W1*W2, Y1 /H1* H2).
16. The system for controlling a touch screen application on a display terminal of the mobile terminal according to claim 14, wherein the mobile terminal is further configured to:

    If the resolution of the mobile terminal is W1*H1, the resolution of the display terminal is W2*H2, and the coordinate point is (X1, Y1), calculate the coordinate percentage of the coordinate point in the screenshot. For P (X1) /W1, Y1 / H1), and set a correspondence relationship between the coordinate percentage P and the preset universal button;

    The display terminal is further configured to:

    Calculating a second coordinate position on the display terminal according to the coordinate percentage P, and simulating a click action on the second coordinate position, injecting into the system, and identifying a click instruction on the second coordinate position, wherein The coordinates of the second coordinate position are (X1) /W1*W2, Y1 /H1* H2).