WO2018120768A1 - Remote control method and terminal - Google Patents

Abstract

Embodiments of the invention provide a remote control method and a terminal. The method comprises: a first terminal displaying a first control interface, the first control interface being consistent with a first display interface currently output by a second terminal, and the second terminal being remotely controlled by the first terminal; the first display interface displaying at least two icons; the first terminal receiving a first touch operation input in the first control interface, the first touch operation comprising a selection operation of a first icon in the first control interface; the first terminal generating a first remote control instruction according to the first touch operation, and sending the first remote control instruction to the second terminal, the first remote instruction being used to instruct the second terminal to select an icon corresponding to the first icon in the first display interface; and after receiving the first touch operation, the first terminal displaying a second control interface, the second control interface being consistent with a second display interface output after the second terminal executes the first remote control instruction. The embodiments of the present invention can improve efficiency and convenience of remote control operations.

Description

Remote control method and terminal

The present application claims priority to Chinese Patent Application No. 201611265792.0, filed on Dec. 30, 2016, entitled "A Method and Apparatus for Remote Control Television for Intelligent Terminals", the entire contents of which are incorporated by reference. In this application.

Technical field

Embodiments of the present invention relate to communication technologies, and in particular, to a remote control method and a terminal.

Background technique

With the development of network technology, users can view rich video content through smart TV or Internet Protocol Television (IPTV) or set-top box, and can view, review or manage video content.

When on-demand, review, or management of video content, it is often necessary to use a remote control to operate the TV or set-top box. In the prior art, the remote controller generally includes some function buttons and direction buttons. When the user wishes to switch from one television interface to another, or when the user wishes to select one of a plurality of program options, it is often necessary to perform multiple operations on the keys of the remote control. As a result, the operation of the remote controller is cumbersome, causing inconvenience to the user.

Summary of the invention

The embodiment of the invention provides a remote control method and a terminal, which can improve the efficiency and convenience of the remote control operation.

In a first aspect, an embodiment of the present invention provides a remote control method, where the method is applied to a first terminal having a touch screen. The first terminal remotely controls the second terminal by using the method. The method includes: the first terminal displays a first control interface, where the first control interface is consistent with a first display interface currently output by the second terminal; the first display interface displays at least two icons, The number and arrangement manner of the icons displayed by the first control interface are the same as the number and arrangement manner of the icons displayed by the first display interface; and the icons displayed by the first control interface and the first display interface are displayed. One-to-one correspondence of the icons; the first terminal receives a first touch operation input on the first control interface, and the first touch operation includes at least a selection operation of the first icon in the first control interface The first terminal generates a first remote control instruction according to the first touch operation; the first terminal sends the first remote control instruction to the second terminal, and the first remote control instruction is used to indicate the The second terminal selects an icon corresponding to the first icon in the first display interface; after receiving the first touch operation, the first terminal displays a second control interface, The second control interface display screen of the two second output terminal coincide, the second display interface output display interface after performing the second terminal of the first remote-control instruction. The control interface displayed by the first terminal may change according to the change of the display interface output by the second terminal, and the layout of the control interface is consistent with the current display interface, so that the user can pass a small amount of touch on the control interface. The operation can control the display interface of the second terminal, thereby making the use of the remote controller more convenient.

Optionally, the generating, by the first terminal, the first remote control instruction according to the first touch operation comprises: acquiring, by the first terminal, an initial position of the cursor before the first icon is selected, and then according to the initial position of the cursor And generating, by the location where the first icon is located, the first remote control instruction.

Optionally, the first touch operation further includes an operation of selecting a second icon corresponding to the initial position of the cursor before the first icon is selected, and the second icon is included in the first control interface. The cursor initial position is an initial position of the cursor on the first display interface. There are many ways to get the initial position of the cursor.

Optionally, the second display interface is an editable input interface, and the second control interface is an input interface including a soft keyboard. The interface layout of the soft keyboard in the second control interface is the same as or different from the interface layout of the soft keyboard in the second display interface. The second control interface may include the same soft keyboard as the second display interface, and may also include different soft keyboards. For example, the second control interface may include a soft keyboard that is more convenient to operate.

Optionally, the method further includes: the first terminal receives a second touch operation input on a soft keyboard of the second control interface, and the second touch operation includes soft to the second control interface a first character selection operation on the keyboard; the first terminal generates a second remote control instruction according to the second touch operation; the first terminal sends the second remote control instruction to the second terminal, The second remote control instruction is used to instruct the second terminal to input the first character in the second display interface. Characters can be conveniently entered in the second terminal by a touch operation on the soft keyboard of the first terminal.

Optionally, after the receiving the first touch operation, the first terminal displays the second control interface, specifically: the first terminal acquires the first one from a memory of the first terminal Touching the second control interface corresponding to the operation; the first terminal displays the second control interface. The second control interface may be pre-stored in the first terminal.

Optionally, after the first terminal sends the first remote command to the second terminal, the method further includes: the first terminal sending an interface information request to the second terminal; The first terminal receives interface information of the second display interface sent by the second terminal; the first terminal displays the second control interface according to the interface information. When the first terminal does not always store the second control interface in advance, the second control interface may be requested from the second terminal.

Optionally, after the first terminal sends the first remote control command to the second terminal, the method further includes: the first terminal receiving an operation for starting a photographing; the first terminal Obtaining a picture having the second display interface by photographing; the first terminal identifying content in the picture of the second display interface, and displaying the identified content as the second control interface. When the first terminal does not store the second control interface in advance, the first terminal may obtain the second control interface by taking a photo.

Optionally, the second control interface is consistent with the second display interface output by the second terminal, and specifically includes any one of the following. The first type: when the second display interface includes at least two icons, the number of icons displayed by the second control interface is the same as the number of icons displayed by the second display interface, and the second control interface displays The arrangement of the icons is the same as the arrangement of the icons displayed by the second display interface, and the icons displayed by the second control interface are in one-to-one correspondence with the icons displayed by the second display interface. Second: when the second display interface is an editable input interface, the second control interface is an input interface including a soft keyboard. The third type: when the second display interface is an audio and video playing interface, the second control interface includes any one or more of the following: a volume control item, a play progress control item, and a play/stop control item.

Optionally, the method further includes: before the first terminal displays the first control interface, the first terminal displays a thumbnail or an icon of at least two control interfaces; the first terminal receives a third Touch operation; the first terminal selects the first control interface from the at least two control interfaces to perform display according to the third touch operation. The first terminal may be triggered to display the first control interface by the third touch operation.

Optionally, when the second terminal is a television or a similar display device, the first display interface and the second display interface are displayed by the second terminal. When the second terminal is a set top box or the like, the first display interface and the second display interface are output by the second terminal to an external device for display, and the external device may be a television, a computer, or the like. device.

Optionally, the first terminal sends the remote control command to the second terminal by using any one of the following communication modes: infrared, Bluetooth, Wi-Fi, and NFC.

Optionally, the first terminal is a mobile phone or a tablet, and the second terminal is a television or a set top box.

Optionally, the method further includes: the first terminal identifies a gesture of the user, the first terminal converts the gesture into a third remote control instruction, and sends the third remote control instruction to the second terminal. That is, the process in which the first terminal is remotely controlling the second terminal The corresponding remote operation can be performed by recognizing the user's gesture.

Optionally, the method further includes: the first terminal identifies a voice of the user, the first terminal converts the voice into a fourth remote command, and sends the fourth remote command to the second terminal. The first terminal may also control the second terminal by using a voice of the user.

In a second aspect, an embodiment of the present invention provides a first terminal, where the first terminal is used to remotely control a second terminal, where the first terminal includes: a touch screen; one or more processors; one or more memories, Storing one or more computer programs in the one or more memories, the one or more computer programs including instructions, when the instructions are executed by the first terminal, causing the first terminal to perform the first Any of the methods described in the aspects.

In a third aspect, an embodiment of the present invention provides a graphical user interface (GUI), where the GUI is stored in the first terminal. The first terminal includes a touch screen, one or more memories, one or more processors, and the one or more processors are for executing one or more computer programs stored in the one or more memories. The GUI includes: a first GUI displayed on the touch screen, the first GUI is consistent with a first display interface currently output by the second terminal, and the second terminal is a terminal remotely controlled by the first terminal; The first display interface displays at least two icons, the number of icons displayed by the first GUI is the same as the number of icons displayed by the first display interface, and the arrangement manner of icons displayed by the first GUI The icon displayed on the first display interface is arranged in the same manner, and the icon displayed by the first GUI is in one-to-one correspondence with the icon displayed on the first display interface; in response to the first gesture of the user on the touch screen, according to The first gesture generates a first remote control instruction, and sends the first remote control instruction to the second terminal; the first gesture includes a selection operation of a first icon in the first GUI, The first remote control instruction is used to instruct the second terminal to select an icon corresponding to the first icon in the first display interface; in response to the first gesture or the third gesture of the user on the touch screen, The touch The second GUI is displayed, the second GUI is consistent with the second display interface output by the second terminal, and the second display interface is a display interface that is output after the second terminal executes the first remote control instruction .

Optionally, the second display interface is an editable input interface, and the second GUI is an input interface including a soft keyboard. The interface layout of the soft keyboard in the second GUI is the same as or different from the interface layout of the soft keyboard in the second display interface.

Optionally, the GUI further includes: generating a second remote control instruction according to the second gesture, and generating the second remote control, in response to a second gesture input by the user on the soft keyboard of the second GUI Sending an instruction to the second terminal; the second gesture includes a selection operation of a first character on a soft keyboard of the second GUI, and the second remote control instruction is used to indicate that the second terminal is in the The first character is input in the second display interface.

Optionally, the second GUI is consistent with the second display interface output by the second terminal. For details, refer to the foregoing description about the second control interface being consistent with the second display interface output by the second terminal.

In a fourth aspect, an embodiment of the present invention provides a first terminal, where the first terminal is used to remotely control a second terminal, where the first terminal includes: a display unit, an input unit, a control unit, and a sending unit. The display unit is configured to display a first control interface, where the first control interface is consistent with a first display interface currently output by the second terminal, and the second terminal is a terminal remotely controlled by the first terminal; The first display interface displays at least two icons, the number of icons displayed by the first control interface is the same as the number of icons displayed by the first display interface, and the icons displayed by the first control interface are arranged The icons displayed by the first display interface are arranged in the same manner, and the icons displayed by the first control interface are in one-to-one correspondence with the icons displayed by the first display interface. The input unit is configured to receive a first touch operation input on the first control interface, where the first touch operation includes a selection operation on a first icon in the first control interface. The control unit is configured to generate a first remote control instruction according to the first touch operation received by the input unit. The sending unit is configured to send the first remote control instruction to the second terminal, where the first remote control instruction is used to instruct the second terminal to select the first icon and the first icon Corresponding map Standard. The display unit is further configured to: after receiving the first touch operation, display a second control interface, where the second control interface is consistent with a second display interface output by the second terminal, the second The display interface is a display interface that is output after the second terminal executes the first remote control instruction. Each unit included in the first terminal is used to implement each method in the foregoing first aspect, and the respective units may be implemented by hardware or software.

In a fifth aspect, an embodiment of the present invention provides a remote control apparatus, which is included in a first terminal, and has a function of implementing the behavior of the first terminal in each method described in the foregoing first aspect. The functions may be implemented by hardware or by corresponding software implemented by hardware. The hardware or software includes one or more modules or units corresponding to the functions described above.

In a sixth aspect, an embodiment of the present invention provides a computer program product comprising instructions, when the computer program product is run on an electronic device, causing the electronic device to perform the method described in the first aspect above.

In a seventh aspect, an embodiment of the present invention provides a computer readable storage medium, including instructions, when the instruction is run on an electronic device, causing the electronic device to perform the method described in the first aspect above.

In an eighth aspect, an embodiment of the present invention further provides a data processing system, including a module for performing the methods provided by the foregoing first aspect.

DRAWINGS

1 is a block diagram showing a part of a structure of a terminal according to an embodiment of the present invention;

2A is a schematic diagram of a scenario of a remote control method according to an embodiment of the present invention;

2B is a flowchart of a remote control method according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of an interface of a first terminal according to an embodiment of the present disclosure;

4A-4E are schematic diagrams of a remote control interface according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of an interface of a remote control method according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of another remote control interface according to an embodiment of the present invention; FIG.

FIG. 7 is a schematic diagram of another remote control interface according to an embodiment of the present invention; FIG.

8A and 8B are schematic diagrams of another remote control interface according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of another remote control interface according to an embodiment of the present invention; FIG.

FIG. 10 is a schematic diagram of another remote control interface according to an embodiment of the present invention; FIG.

FIG. 11 is a flowchart of another remote control method according to an embodiment of the present invention;

FIG. 12 is a flowchart of a method for remote control by using a gesture according to an embodiment of the present invention.

detailed description

When viewing audio and video programs using televisions, set-top boxes, or other similar terminals, televisions, set-top boxes, or the like are typically controlled by conventional remote controls. Through the remote control method provided by the embodiments of the present invention, a terminal having a touch screen can be used to control a television, a set top box or the like in place of the ordinary remote controller. The terminal can be, for example, a mobile phone, a tablet personal computer, a laptop computer, a digital camera, a personal digital assistant (PDA), a navigation device, and a mobile internet device (Mobile Internet). Device, MID) or Wearable Device.

FIG. 1 is a block diagram showing a partial structure of a terminal according to an embodiment of the present invention. The terminal uses the mobile phone 100 as an example for description. Referring to FIG. 1, the mobile phone 100 includes: a radio frequency (RF) circuit 110, a power source 120, a processor 130, a memory 140, an input unit 150, a display unit 160, a camera 165, a sensor 170, an audio circuit 180, and wireless fidelity ( Wireless fidelity, Wi-Fi) Module 190, infrared emitter 192, and Bluetooth module 194 and the like. It will be understood by those skilled in the art that the structure of the handset shown in FIG. 1 does not constitute a limitation to the handset, and may include more or less components than those illustrated, or some components may be combined, or different components may be arranged.

The components of the mobile phone 100 will be specifically described below with reference to FIG. 1 :

The RF circuit 110 can be used to send and receive information or to receive and transmit signals during a call. For example, the RF circuit 110 may send downlink data received from the base station to the processor 130 for processing, and send the uplink data to the base station. Generally, RF circuits include, but are not limited to, RF chips, antennas, at least one amplifier, transceiver, coupler, Low Noise Amplifier (LNA), duplexer, RF switch, and the like. In addition, RF circuitry 110 can also communicate wirelessly with networks and other devices. The wireless communication may use any communication standard or protocol, including but not limited to Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (Code). Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), E-mail, Short Messaging Service (SMS), etc.

The memory 140 can be used to store software programs and modules, and the processor 130 executes various functional applications and data processing of the mobile phone 100 by running software programs and modules stored in the memory 140. The memory 140 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may be stored. Data created according to the use of the mobile phone 100 (such as audio data, phone book, etc.). Moreover, memory 140 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 150 can be configured to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the handset 100. Specifically, the input unit 150 may include a touch panel 151 and other input devices 152. The touch panel 151, also referred to as a touch screen, can collect touch operations on or near the user (such as the user using a finger, a stylus, or the like on the touch panel 151 or near the touch panel 151. Operation), and drive the corresponding connecting device according to a preset program. Optionally, the touch panel 151 may include two parts: a touch detection device and a touch controller. Wherein, the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information. The processor 130 is provided and can receive commands from the processor 130 and execute them. In addition, the touch panel 151 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 151, the input unit 150 may also include other input devices 152. Specifically, other input devices 152 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and the like.

The display unit 160 can be used to display information input by the user or information provided to the user as well as various menus and interfaces of the mobile phone 100. The display unit 160 may include a display panel 161. Alternatively, the display panel 161 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED). Further, the touch panel 151 can cover the display panel 161. When the touch panel 151 detects a touch operation on or near the touch panel 151, the touch panel 151 transmits to the processor 130 to determine the type of the touch event, and then the processor 130 according to the touch event. The type provides a corresponding visual output on display panel 161. Although the touch panel 151 and the display panel 161 are two independent components to implement the input and input functions of the mobile phone 100 in FIG. 1, in some embodiments, the touch panel 151 may be integrated with the display panel 161. The input and output functions of the mobile phone 100 are implemented.

The handset 100 can also include one or more cameras 165. Camera 165 can be used to capture and capture images. In mobile phone 100 To include one or more front cameras and one or more rear cameras.

The handset 100 can also include at least one type of sensor 170, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 161 according to the brightness of the ambient light, and the proximity sensor may close the display panel 161 when the mobile phone 100 moves to the ear. / or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity. It can be used to identify the gesture of the mobile phone (such as horizontal and vertical screen switching, related Game, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping). The mobile phone 100 can also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, and will not be described herein.

The audio circuit 180, the speaker 181, and the microphone 182 can provide an audio interface between the user and the handset 100. The audio circuit 180 can transmit the converted electrical data of the received audio data to the speaker 181 for conversion to the sound signal output by the speaker 181; on the other hand, the microphone 182 converts the collected sound signal into an electrical signal by the audio circuit 180. After receiving, it is converted into audio data, and then the audio data is output to the RF circuit 110 for transmission to, for example, another mobile phone, or the audio data is output to the memory 140 for further processing.

Wi-Fi, infrared and Bluetooth are all short-range wireless transmission technologies. The mobile phone 100 can help the user to send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 190, which provides users with wireless broadband Internet access. The mobile phone 100 can emit infrared light to a certain extent through the infrared emitter 192. The handset can communicate with external devices via the Bluetooth module 194.

The processor 130 is the control center of the handset 100, which connects various portions of the entire handset using various interfaces and lines, by running or executing software programs and/or modules stored in the memory 140, and recalling data stored in the memory 140, The various functions and processing data of the mobile phone 100 are executed, thereby realizing various services based on the mobile phone. Optionally, the processor 130 may include one or more processing units; preferably, the processor 130 may integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, an application, and the like. The modem processor primarily handles wireless communications. It can be understood that the above modem processor may not be integrated into the processor 130.

In the embodiment of the present invention, the processor 130 may execute the program instructions stored in the memory 140, so that the mobile phone as the first terminal performs the method shown in any of the following embodiments.

The mobile phone 100 also includes a power source 120 (such as a battery) that supplies power to various components. Preferably, the power source can be logically coupled to the processor 130 through a power management system to manage functions such as charging, discharging, and power consumption through the power management system.

The remote control method provided by the embodiment of the present invention will be described in detail below with reference to the accompanying drawings. The terminal that can replace the remote controller is referred to as a first terminal, and the remotely controlled television, set-top box, or the like is referred to as a second terminal. 2A is a schematic diagram of a scenario of a remote control method according to an embodiment of the present invention. FIG. 2B is a flowchart of a remote control method according to an embodiment of the present invention.

As shown in FIG. 2A, the first terminal 100 can control the second terminal 200 by using a remote control method provided by an embodiment of the present invention. It should be noted that the remote control method provided by the embodiment of the present invention does not conflict with the existing remote control method, that is, the second terminal 200 can also be controlled by the ordinary remote controller 300 at the same time.

FIG. 3 is a schematic diagram of an interface of a first terminal according to an embodiment of the present invention. 4A-4E are schematic diagrams of a remote control interface according to an embodiment of the present invention. As shown in FIG. 3, a plurality of applications may be installed on the first terminal, which may include a remote control application 400. After the remote control application 400 is opened, the first terminal can use the remote control application 400 to control the second terminal. As shown in FIG. 2B, when the second terminal is remotely controlled using the first terminal, the remote control method includes the following steps.

Step 201: The first terminal displays a first control interface. As shown in FIG. 4D, the second terminal currently outputs the first display The interface 401 displays the at least two icons 402. Six icons 402 are exemplarily shown in FIG. 4D. The first control interface 410 is consistent with the first display interface 401, and may be specifically: the number of icons 412 displayed by the first control interface 410 and the icon 402 displayed by the first display interface 401. The number of the icons displayed by the first control interface 410 is the same as the arrangement of the icons displayed by the first display interface 401; and the icons displayed by the first control interface and the first The icons displayed on the display interface correspond one-to-one. Wherein, since the size and shape of the first control interface and the first display interface may be different, the content displayed in the first control interface may be scaled relative to the content displayed in the first display interface. Or size adaptation. The identifier of the icon in the first control interface 410 may be the same as the identifier of the icon in the first display interface 401 (as shown in FIG. 4D), or may be different (as shown in FIG. 4E). The first display interface can take many forms, and FIG. 4D is merely an exemplary illustration. For example, as shown in FIG. 9 , the first display interface 901 may be in another form, and the corresponding first control interface 910 is in a form corresponding to the first display interface 901 . Through the interface shown in FIG. 9, it is possible to review the programs played during the seven days from January 1st to January 7th.

It should be noted that, according to the first display interface, the first control interface is displayed on the first terminal. That is to say, the embodiment of the present invention does not screen the interface displayed on the first terminal to the second terminal. Moreover, when the second terminal is controlled by using the first terminal, the second terminal can also be controlled by an ordinary remote controller at the same time, as shown in FIG. 2A.

Step 202: The first terminal receives a first touch operation input on the first control interface. The first touch operation may include a selection operation of the first icon in the first control interface. The first touch operation may be one or more touch operations. That is, the user can perform one or more touch operations on the first control interface.

Step 203: The first terminal generates a first remote control instruction according to the first touch operation, and sends the first remote control instruction to the second terminal. The first remote control instruction is used to instruct the second terminal to select an icon corresponding to the first icon in the first display interface.

Specifically, the process of the first terminal generating the first remote control instruction may include: the first terminal acquiring an initial position of a cursor before the first icon is selected; and then the first terminal is initially initialized according to the cursor The location and the location of the first icon generate the first remote control command. As shown in the first display interface 401 shown in FIG. 5(a), the cursor initial position is at the icon 4021. The second terminal may send the relative position of the initial position of the cursor in the first display interface 401 to the first terminal, so that the first terminal acquires the initial position of the cursor at the first Corresponding to the second icon 4121 on the control interface; or the first terminal may receive a touch operation of the user on the second icon 4121 to identify that the initial position of the cursor corresponds to the second icon 4121; or In the first control interface 410, the initial position of the cursor is defaulted to be at the second icon 4121, and the first terminal can obtain the default position when entering the first control interface 410. After the second terminal acquires the cursor initial position corresponding to the second icon 4121, the cursor initial position may be displayed on the first control interface 410, as shown in FIG. 5(b). When the user wishes to select the icon 4023 in the first display interface, the user may select the first icon 4123 corresponding to the icon 4023 on the first control interface 410, as shown in FIG. 5(c). After the first terminal acquires that the first icon 4123 is selected, the first remote control instruction is generated according to the initial position and the location where the first icon 4123 is located, and the first remote control command is sent. To the second terminal. The second terminal selects the icon 4023 according to the first remote control instruction, as shown in FIG. 5(d). The second icon is included in the first control interface. The initial position of the cursor refers to the position where the cursor is located when entering the first display interface. Alternatively, the initial position of the cursor refers to a position where the cursor is located before the second terminal receives the first remote control instruction. Since the first control interface is the same as the icon distribution manner of the first display interface, the first control interface includes an icon corresponding to the initial position of the cursor.

The first touch operation may be in different forms.

When the first terminal identifies the initial position of the cursor according to a touch operation of the user, the first touch operation may have The following two operations are sequentially performed: first, a touch operation on the second icon corresponding to the initial position of the cursor, and then a selection operation on the first icon; for example, the touch operation on the second icon may be a click The operation of selecting the first icon may be clicking, double-clicking, long-pressing or re-pressing, or after clicking, double-clicking, long-pressing or re-pressing, further including a touch operation on the determining button. That is, the selection of the first icon may be implemented by clicking, double-clicking, long-pressing, or re-pressing the first icon, or the determining button may be further included in the first control interface 410 by clicking, After double-clicking, long-pressing, or re-pressing the first icon, clicking or touching the OK button implements selection of the first icon.

When the second terminal sends the cursor initial position to the first terminal, or when the first terminal can obtain the default cursor initial position when entering the first control interface 410, The first touch operation may include only a selection operation of the first icon.

The first remote control command can take a variety of forms.

One is that the first terminal can convert the first touch operation into a common one or more remote control commands. For example, the first terminal acquires that the cursor needs to be moved to the right twice according to the coordinates of the second icon 4121 corresponding to the initial position of the cursor and the coordinate information of the first icon 4123, whereby the first terminal will The first touch operation is converted into two remote control commands that move to the right once, and each remote command indicates moving to the right once. In this manner, the second terminal can use the existing television or set-top box, that is, the second terminal can be controlled by using the remote control method provided by the embodiment of the present invention, without modifying the existing second terminal. .

Alternatively, the first terminal may convert the first touch operation into a remote control command indicating a movement displacement of the cursor. For example, the first terminal acquires that the cursor needs to be moved to the right twice according to the coordinate information of the second icon 4121 and the coordinate information of the first icon 4123 corresponding to the initial position of the cursor, thereby the first terminal The first touch operation is converted into a remote control command that indicates moving to the right twice. The second terminal receives and parses the first remote control command, and then controls the cursor to move to the right twice. In this manner, the same database may be preset in the first terminal and the second terminal, so that the second terminal can correctly parse the remote command sent by the first terminal.

Third: the first terminal can convert the first touch operation into a remote control command indicating a change in touch screen coordinates. For example, the first terminal sends the coordinate information of the second icon 4121 corresponding to the initial position of the cursor and the coordinate information of the first icon 4123 to the second terminal as the first remote control instruction; The coordinate information of the first icon 4123 relative to the second icon 4121 is sent to the second terminal as the first remote control command. The second terminal receives and parses the first remote control command, and then controls the cursor to move to the right twice. In this manner, the same database may be preset in the first terminal and the second terminal, so that the second terminal can correctly parse the remote command sent by the first terminal.

Step 204: After receiving the first touch operation, the first terminal displays a second control interface. The second control interface is consistent with the second display interface output by the second terminal, and the second display interface is a display interface that is output after the second terminal executes the first remote control instruction.

It should be noted that the embodiment does not limit the execution order of step 203 and step 204. Step 204 may be performed after step 203, and step 204 may also be executed in parallel with step 203.

After the second terminal executes the first remote control instruction, the second display interface is output, so the first terminal also switches the first control interface to the second control that is consistent with the second display interface. The interface is such that the first terminal can continue to control the second terminal.

The second control interface corresponding to the execution of the first touch operation may be pre-stored in the first terminal. After the first terminal receives the first touch operation, the first terminal may acquire information of the second control interface from a memory, and then display the second control interface. For example, after the second terminal receives the first remote control command, The interface of 5(d) is switched to the second display interface 601 shown in FIG. A second control interface 610 corresponding to the second display interface 601 is pre-stored in the first terminal, and the first terminal automatically switches to display the second control interface 610, as shown in FIG. 6.

Further, if the second control interface is not stored in the first terminal, the method may further include the following steps: the first terminal sends an interface information request to the second terminal; The first terminal receives interface information of the second display interface sent by the second terminal. Then in step 204, the first terminal may acquire and display the second control interface according to the interface information. Specifically, as shown in FIG. 5(c), after the first icon 4123 is selected, the user can request the interface information of the second control interface by clicking the button 420. After detecting that the button 420 is touched, the first terminal sends an interface information request to the second terminal; after receiving the interface information request, the second terminal acquires a current output of the second terminal. For example, the interface information of the second display interface 601 may be screen shot or the display data corresponding to the second display interface 601 may be directly obtained; then the second terminal will be the second The interface information of the display interface is sent to the first terminal; after the first terminal processes the interface information, the second control interface 610 is output.

In another optional manner, if the second control interface is not pre-stored in the first terminal, the method may further include the following steps after the step 203: the first terminal receives a photo for starting a photo taking. The first terminal acquires a picture having the second display interface by photographing; the first terminal identifies content in the picture of the second display interface, and uses the identified content as the first The second control interface is displayed. Specifically, as shown in FIG. 5(c), the button 420 therein may be replaced with a photo button. After the first icon 4123 is selected, the user may take a photo of the second display interface by clicking the photo button. . After acquiring the picture of the second display interface, the first terminal performs icon recognition and cropping on the picture, and then outputs the second control interface.

It should be noted that, after acquiring the second control interface by taking a photo or making a request to the second terminal, the first terminal may save the second control interface, so that the next operation is performed. The second control interface can be automatically displayed.

The second control interface and the second display interface may specifically include any one or more of the following situations.

The first type: when the second display interface is an editable input interface, the second control interface is an input interface including a soft keyboard. Input and editing on the second display interface can be implemented by a soft keyboard in the second control interface. The interface layout of the soft keyboard in the second control interface and the interface layout of the soft keyboard in the second display interface may be the same, as shown in FIG. 8A. Alternatively, the interface layout of the soft keyboard in the second control interface may be different from the interface layout of the soft keyboard in the second display interface, as shown in FIG. 8B.

Second type: when the second display interface includes at least two icons, for example, when the second display interface is the first display interface or the second display shown in FIG. 4D, FIG. 4E, FIG. 5 or FIG. When the interfaces are the same or similar, the number of icons displayed by the second control interface is the same as the number of icons displayed by the second display interface, and the arrangement manner of the icons displayed by the second control interface and the second display The icons displayed on the interface are arranged in the same manner, and the icons displayed by the second control interface are in one-to-one correspondence with the icons displayed on the second display interface. That is, the second control interface is the same as or similar to the first control interface or the second control interface shown in FIG. 4D, FIG. 4E, FIG. 5 or FIG.

The third type: when the second display interface is an audio and video playing interface, the second control interface includes any one or more of the following: a volume control item, a play progress control item, and a play/stop control item. As shown in FIG. 10, when the second terminal is playing an audio and video program, the first terminal presents a second control interface 1010, including a volume control item 1012, a play progress control item 1013, and a play/stop control item 1015. . Controlling the volume, playing progress, or controlling the program broadcast through the control interface of the first terminal Put or pause. It should be noted that the play progress control item 1013 and the play/stop control item 1015 are generally used to control non-live broadcast audio and video programs. For the live audio and video program, the second progress control interface may not include the play progress control item and the play/stop control item, or the play progress control item and the play/stop control item may be invalid icons.

Further, after the step 204, when the second display interface is an editable input interface, and the second control interface is an input interface including a soft keyboard, the method may further include the following steps 205 and 206. For example, as shown in FIG. 7 and FIG. 8A and FIG. 8B, after the search button 480 on the first control interface 410 is clicked, the first terminal presents a second control interface 810A or 810B.

Step 205: The first terminal receives a second touch operation input on a soft keyboard of the second control interface. The second touch operation includes a selection operation of a first character on a soft keyboard of the second control interface.

The second control interface displayed by the first terminal includes a soft keyboard. When the user needs to input information in the second display interface, the touch input can be directly performed in the soft keyboard on the second control interface, and the touch input is the second touch operation. The second touch operation may be a touch operation on a first character on the soft keyboard, such as clicking, double-clicking, long-pressing or re-pressing the first character.

Step 206: The first terminal generates a second remote control instruction according to the second touch operation, and sends the second remote control instruction to the second terminal, where the second remote control instruction is used to indicate the second The terminal inputs the first character in the second display interface.

The first terminal may generate multiple second remote control commands according to the second touch operation, specifically:

The first type: the first terminal receives a plurality of the second touch operations, each second touch operation corresponds to one character; and the first terminal can obtain the The relative positions of the characters corresponding to the two second touch operations are respectively converted, and then the relative positions are converted into ordinary one or more remote control commands, and the one or more remote control commands are used to indicate the displacement of the cursor movement. For example, as shown in FIG. 8A, the first terminal first receives a second touch operation 811 for the character H, and then receives a second touch operation 814 for the character A, the first terminal according to the second touch operation 811. And the second touch operation 814, the relative position of the two characters can be obtained, that is, when the cursor is at the character H, the cursor A is moved to the left 7 times to obtain the character A; then the relative position is converted into seven leftward movements. One remote command, each remote command means moving to the right once. It should be noted that, in the process of performing the input operation, the second touch operation is used as a reference for each second touch operation. The first second touch operation is based on the character corresponding to the initial position of the cursor. The cursor initial position refers to a position where the cursor is located when the second terminal first enters the second display interface. The initial position of the cursor may be sent by the second terminal to the first terminal, or the first terminal may obtain a default value in the memory, or may be further determined by the user according to the first The initial position of the cursor presented in the second terminal, and the touch operation is performed on the character corresponding to the initial position of the cursor in the second control interface.

The second type: the first terminal receives a plurality of the second touch operations, each second touch operation corresponds to one character; and the first terminal can obtain the second two touch operations according to the second The relative positions of the characters corresponding to the two second touch operations are respectively described, and then the relative position is converted into a remote command for indicating the movement displacement of the cursor. For example, as shown in FIG. 8A, the second touch operation 811 received by the first terminal and the second touch operation 814 received later may be obtained by moving the cursor to the left 7 times to obtain the character A; Convert the relative position to a remote command that moves seven times to the left. The second terminal receives and parses the second remote control command, and then controls the cursor to move to the left seven times. In this manner, the same database may be preset in the first terminal and the second terminal, so that the second terminal can correctly parse the remote command sent by the first terminal. In the same manner as the first mode, in the process of performing the input operation, the second touch operation is used as a reference for each second touch operation, and the second touch operation of the first time is the character corresponding to the initial position of the cursor. Benchmark.

Third: the first terminal may convert the second touch operation of two adjacent times into a remote control instruction indicating a change in the coordinates of the touch screen. For example, as shown in FIG. 8A, the first terminal sends coordinate information of the character H and coordinate information of the character A as the second remote control command to the second terminal, or coordinates of the character H with respect to the character A. Information is sent to the second terminal as the second remote control command. The second terminal receives and parses the second remote control command, and then controls the cursor to move to the left seven times. In this manner, the same database may be preset in the first terminal and the second terminal, so that the second terminal can correctly parse the remote command sent by the first terminal. In the same manner as the first mode, in the process of performing the input operation, the second touch operation is used as a reference for each second touch operation, and the second touch operation of the first time is the character corresponding to the initial position of the cursor. Benchmark.

The fourth type: the first terminal directly sends the character corresponding to the second touch operation to the second terminal as a second remote control instruction, and the second terminal directly recognizes and displays the character. In this manner, the second terminal can obtain the corresponding character directly according to the current second touch operation without depending on the initial position of the cursor and the last second touch operation.

Further, after the step 204, when the second display interface includes at least two icons, the layout (ie, the number and arrangement manner) of the icons in the second control interface and the icons in the second display interface The method may also include the following step 207 when the layouts are the same. For example, in FIG. 5(c), after the first icon 4123 is selected, as shown in FIG. 6, the second terminal displays the second display interface 601, and the first terminal displays the second control interface 610.

Step 207: The first terminal receives a touch operation input on the second control interface, generates a remote control instruction according to the touch operation, and then sends the remote control command to the second terminal to control the The second terminal. The second display interface is the same as or similar to the first display interface, and the second control interface is the same as or similar to the first control interface. Therefore, the specific steps of the step 207 can be referred to the above steps. The description in 201-step 203 will not be repeated here. For example, as shown in FIG. 6, the initial position of the cursor corresponds to the icon movie 1. After the icon movie 8 is selected, the first terminal can send a remote command to the second terminal, so that the second terminal turns the cursor to the right. Move three times and move down once.

Further, after the step 204, when the second display interface is an audio and video playing interface, the second control interface includes any one or more of the following: a volume control item, a play progress control item, and a play/stop control. The method may further include the following step 208 and the step 209.

Step 208: The first terminal receives a touch operation input on the second control interface. The touch operation can be a click, a double tap, a long press, a re-press, or a swipe operation. For example, as shown in FIG. 10, when the second terminal is playing an audio and video program, the user can adjust the volume by sliding left and right on the slide bar 912, or can adjust the play by sliding left and right on the slide bar 913. The progress can also be controlled by the click, double click, long press or re-press of the button 915 to control the play and pause of the program.

Step 209: The first terminal generates a remote control instruction according to the touch operation in step 208, and sends the remote control instruction to the second terminal, where the remote control instruction is used to control the volume, progress, and/or of the currently played program. Or play status. Specifically, the first terminal may generate a remote command of the volume or the play progress according to the direction and distance of the touch slide, or may generate a remote control command for controlling the play state according to the click state of the play button (for example, the button 915).

It should be noted that the first control interface of step 201 can be triggered to display in multiple manners. For example, before step 201, the embodiment of the present invention may further include the following step 220, step 230, and step 240A, step 240B or step 240C.

Step 220: After entering the remote control interface, the first terminal displays thumbnails or icons of at least two control interfaces. As shown in FIG. 3 and FIG. 4A, after clicking the remote control icon 300 in FIG. 3, the remote control interface 430 shown in FIG. 4A can be accessed. The remote control interface 430 can include thumbnails or icons of a plurality of control interfaces, such as menu one, menu two, and menu three. At this time, the second terminal displays the first display interface, such as the first display interface 401 in FIG. 4D.

Step 230: The first terminal receives a third touch operation. The third touch operation may be a selection of a thumbnail or an icon shown by the remote control interface 430, or the third touch operation may trigger the first terminal to the second terminal. Sending an interface information request, or the third touch operation may trigger the first terminal to take a photo of the first display interface currently output by the second terminal. When the first terminal pre-stores a control interface corresponding to the first display interface currently output by the second terminal, step 240A is performed after step 230. When the control interface corresponding to the first display interface currently output by the second terminal is not stored in the first terminal, step 240B or step 240C is performed after step 230.

Step 240A: The first terminal selects the first control interface from the at least two control interfaces according to the third touch operation, and then the first terminal displays the first control interface in step 201. . The first control interface is pre-stored in the first terminal, and the user selects a thumbnail or an icon corresponding to the first control interface from the remote control interface 430 to enter the first control interface. . For example, if the menu 1 pre-stored in the first terminal is the first control interface, the menu 1 can be selected to enter the first control interface 410, as shown in FIG. 4A and FIG. 4D.

Step 240B: The first terminal sends an interface information request to the second terminal according to the third touch operation; the first terminal receives interface information of the first display interface sent by the second terminal. Then in step 201, the first terminal may acquire and display the first control interface according to the interface information. For example, as shown in FIG. 4A, the remote control interface 430 includes a request button 440. When the first control interface is not in the first terminal, that is, the menu 1, the menu 2, and the menu 3 are not the first control interface, the second terminal may be requested by clicking the request button 440. The interface information of the first display interface is described. The first terminal generates the first control interface according to the interface information of the first display interface, and displays a menu 460 corresponding to the first control interface on the remote control interface 430, as shown in FIG. 4C. . The first control interface can then be accessed by selection of menu four 460.

Step 240C: The first terminal starts photographing according to the third touch operation; the first terminal acquires a picture having the first display interface by photographing; the first terminal identifies a picture of the first display interface The content in the content is then displayed in step 201 as the first control interface. For example, as shown in FIG. 4B, the remote control interface 430 includes a photo button 450. When the first control interface is not in the first terminal, the photographing can be started by clicking the photographing button 450. After the first terminal acquires the information of the first control interface, the menu 4 corresponding to the first control interface may be displayed on the remote control interface 430, as shown in FIG. 4C. The first control interface can then be accessed by selecting menu four.

The process of acquiring the first control interface by the first terminal in steps 240B and 240C is similar to the process of acquiring the second control interface.

It should be noted that the first control interface in this embodiment may be the first control interface after the first terminal enters the remote control program, or may be any control after the first terminal enters the remote control program. interface.

The second terminal according to various embodiments of the present invention may be a smart TV or a network television, that is, a television that can perform on-demand audio and video programs. In this case, the first display interface and the second display interface are output and displayed by the second terminal.

In other embodiments, the second terminal in the embodiments of the present invention may be a set top box. The set top box cooperates with a television or other display device to play an audio and video interface, and the set top box can perform on-demand broadcasting of audio and video programs. In this case, the first display interface and the second display interface are output by the second terminal to an external device such as a television, and then displayed by the external device.

According to the remote control method provided by the embodiment of the present invention, the control interface displayed by the first terminal may change according to the change of the display interface output by the second terminal, and the layout of the control interface is consistent with the current display interface, thereby the user The display interface of the second terminal can be controlled by a small amount of touch operation on the control interface, thereby making the use of the remote controller more convenient. In the prior art, when the television is remotely controlled by the ordinary remote controller, more operation of the direction keys or other similar buttons is required to complete the same remote operation as the embodiment of the present invention. Moreover, in the prior art, in which the mobile phone is used as a remote controller, the mobile phone displays a virtual button that is identical to the button of the ordinary remote controller, and then uses the same operation mode as the ordinary remote controller to remotely control the television. Compared with the prior art, the remote control method provided by the embodiment of the present invention is simpler and more convenient.

The remote control method provided by the embodiment of the invention is mainly applied to a remote control television or a similar display device. The first terminal can be used as a remote controller for a variety of appliances. That is, after the first terminal opens the remote control application, a remote control interface corresponding to a plurality of electrical appliances may be displayed on the interface, and the user may select a remote control interface that matches the electrical appliance to be remotely controlled. For example, the first terminal can display electrical appliances for remote control televisions, set top boxes, DVDs, air conditioners, electric fans, and the like.

FIG. 11 is a flowchart of another remote control method according to an embodiment of the present invention. The first terminal in the method takes a mobile phone as an example, and the second terminal takes a set top box as an example. The interface output by the second terminal can be displayed by using a television, a computer or other display device. The method takes a television as an example. The method is based on the embodiments described above with reference to Figures 1 - 10. As shown in FIG. 11, the method includes:

Step 1101: The mobile phone opens a preset remote control application.

Step 1102: Select a corresponding control interface according to a scene that the TV screen needs to operate. The user may manually select a corresponding control interface in the mobile phone according to the current display content of the television screen, or may send the content currently displayed by the television to the mobile phone by the set top box, and the corresponding control interface is automatically selected by the mobile phone. If the current TV interface displays the menu interface, step 1103 is performed; if the current television interface displays an editable input interface, step 1108 is performed; if the current television interface is playing the audio and video interface, step 1110 is performed. .

Step 1103: The mobile phone enters a menu control interface of the remote control application. According to whether a control interface corresponding to the display interface of the current television is stored in the mobile phone, after step 1103, step 1104, step 1105, and step 1106 may be performed respectively.

Step 1104: The mobile phone selects a control interface corresponding to the current television display interface from the pre-stored control interface according to the user's selection operation. When the mobile phone stores a control interface corresponding to the display interface of the current television, the user can directly select the control interface on the mobile phone.

Step 1105: The mobile phone takes a photo on the TV screen according to the operation triggered by the user, and then automatically cuts and analyzes the taken photo, recognizes the menu option in the photo, and generates a corresponding control interface.

Step 1106: The mobile phone sends an interface information request to the set top box according to the trigger of the user, and then the interface information of the current television sent by the receiver set top box, and generates a corresponding control interface according to the interface information. After step 1105 or step 1106, the mobile phone can save the generated control interface, so that the user can directly select the control interface in the next use without having to take a photo or request to the set top box.

After step 1104, step 1105, and step 1106, the method performs step 1107.

Step 1107: The mobile phone acquires a touch operation of the user on the screen of the mobile phone, and then converts the touch operation into a remote control instruction. For example, the mobile phone remote control application converts the touch operation F1 into the remote control command O1=A1 (F1) through the instruction library A1. After the handset generates a remote command, the method proceeds to step 1112.

Step 1108: The mobile phone enters an input interface of the remote control application.

Step 1109: The mobile phone receives characters such as English or pinyin letters input by the user, and then converts the character F2 into a remote control command O2=A2 (F2) through the instruction library A2, and then executes step 1112.

Step 1110: The mobile phone enters a volume/play adjustment interface of the remote control application.

Step 1111: The mobile phone receives a sliding or clicking operation of the user on the volume/playback adjustment interface, and converts the sliding or clicking operation F3 into a remote control command O3=A3 (F3) through the instruction library A3, and then performs step 1112.

Step 1112: The mobile phone sends the generated remote command O1, O2 or O3 to the set top box.

Step 1113: After receiving the remote control command, the set top box parses the O1, O2, or O3 instruction into a corresponding operation through a command library matched with the instruction library A1, and then performs a corresponding operation to implement menu jump, volume adjustment, and play. Control or enter characters.

After step 1112, the method may further comprise the following steps:

Step 1114: The mobile phone performs a jump of the control interface according to a menu jump of the television. Wherein, the user may select a corresponding control interface on the mobile phone according to the interface change of the television. It is also possible to automatically implement a jump of the control interface by the mobile phone. It is also possible to request the jumped interface from the mobile phone to the set top box or the mobile phone to take a picture of the interface after the jump. Moreover, the mobile phone can record the interface before the jump, and record the correspondence between the touch operation F1 and the interface after the jump, so that the next time the same touch operation F1 is performed, the mobile phone can realize the automatic jump of the interface, reducing the again Take a photo operation or reduce the number of requests to the set-top box.

After step 1107 and step 1112, the television may jump to an editable input interface. At this time, correspondingly, the mobile phone may jump to the input interface described in step 1108.

The process of converting the touch operation into the remote control command in steps 1107, 1109, and 1111 can be referred to the detailed description in the foregoing embodiments.

Based on the foregoing embodiments, the embodiment of the present invention may further provide a method for remote control by using a gesture. The mobile phone supports a variety of gesture operations, and the advantages of the gesture operation can improve the remote operation efficiency and enhance the user experience. The embodiments of the present invention exemplify several gesture operations, but the operation is not limited to these types of gesture operations. Any gesture operation supported by the mobile phone can be applied to the remote operation, and the user can configure various gestures to achieve remote operation according to his or her preference. . In the remote control method described in each of the above embodiments, the remote control by gesture can be further implemented by the method shown in FIG. FIG. 12 is a flowchart of a method for remote control by using a gesture according to an embodiment of the present invention. As shown in FIG. 12, the method includes:

Step 1201: The mobile phone opens the remote control application.

Step 1202: The remote control application of the mobile phone recognizes the gesture operation of the user, and converts the gesture into a corresponding remote control instruction. The correspondence between the gesture operation and the remote control command may be preset in advance. The correspondence may be the default setting when the phone is shipped from the factory, or it may be set and edited by the user.

Step 1203: The mobile phone sends a remote control command to the set top box.

Step 1204: The set top box receives the remote control command and converts the remote control command into an executable operation instruction, and performs an operation corresponding to the operation instruction. For example, the set-top box can perform operations such as returning to the previous layer, shutting down, audio and video fast forward or rewind, returning to the main menu according to the user's gesture operation. The gesture operation of the user may be various operations that are easy to recognize and are not easy to be misjudged, such as a circle operation, a fold line operation, or other specific graphic operations. It should be noted that the gesture of the user may be a gesture of touching the touch screen of the mobile phone, or may be a gesture of touching the touch screen of the mobile phone.

The first terminal provided by each embodiment of the present invention may further have a voice recognition function. In the process of remote control, the voice recognition function can be activated to convert the voice sent by the user into a remote command to remotely control the second terminal.

In various embodiments of the present invention, the communication manner between the first terminal and the second terminal may be infrared, Bluetooth, Wi-Fi or NFC. That is, the first terminal may send the remote control command to the second terminal by using infrared, Bluetooth, Wi-Fi or NFC technology, and the second terminal may also use any one of the communication methods when transmitting the interface information to the first terminal.

The embodiment of the invention can implement the operation of the touch screen remote control set top box or the television by using a wide range of handheld terminals, reducing the dependence on the ordinary remote controller and reducing the hardware cost.

The embodiment of the invention provides a simple and fast touch screen remote operation method for different usage scenarios, and improves the user remote operation efficiency and experience.

The embodiment of the invention further provides a graphical user interface (GUI), the GUI is stored in the first terminal, and the first terminal package A touch screen, one or more memories, one or more processors for executing one or more computer programs stored in the one or more memories. The GUI includes:

In the first GUI displayed on the touch screen, the first GUI is consistent with a first display interface currently output by the second terminal, and the second terminal is a terminal remotely controlled by the first terminal; the first display The interface displays at least two icons, the number of icons displayed by the first GUI is the same as the number of icons displayed by the first display interface, and the arrangement manner of the icons displayed by the first GUI and the first display interface The displayed icons are arranged in the same manner, and the icons displayed by the first GUI are in one-to-one correspondence with the icons displayed by the first display interface;

Responding to the first gesture of the user on the touch screen, generating a first remote control instruction according to the first gesture, and transmitting the first remote control instruction to the second terminal; the first gesture includes a first operation of the first icon in the first GUI, the first remote control instruction is used to instruct the second terminal to select an icon corresponding to the first icon in the first display interface;

Displaying a second GUI in the touch screen in response to the first gesture or the third gesture of the user on the touch screen, the second GUI being consistent with the second display interface output by the second terminal, The second display interface is a display interface that is output after the second terminal executes the first remote control instruction.

The second display interface is an editable input interface, and the second GUI is an input interface including a soft keyboard. The interface layout of the soft keyboard in the second GUI is the same as or different from the interface layout of the soft keyboard in the second display interface.

Further, the GUI further includes: generating a second remote control instruction according to the second gesture, and generating the second remote control instruction, in response to a second gesture input by the user on the soft keyboard of the second GUI Sending to the second terminal; the second gesture includes a selection operation of a first character on a soft keyboard of the second GUI, and the second remote control instruction is used to indicate that the second terminal is in the The first character is input in the second display interface.

It should be noted that the first GUI described in this embodiment corresponds to the first control interface in the foregoing embodiments, and the second GUI corresponds to the second control interface in each of the foregoing embodiments. The first gesture and the second gesture in this embodiment respectively correspond to the first touch operation and the second touch operation in the foregoing embodiments. The third gesture in this embodiment corresponds to the interface information request operation or the operation of starting the photographing in the foregoing embodiments. For details of the GUI, refer to the description in the foregoing embodiments, and details are not described herein again.

The embodiment of the present invention further provides a remote control device, which is included in a first terminal, and has a function of implementing the behavior of the first terminal in the foregoing method embodiments. The functions may be implemented by hardware or by corresponding software implemented by hardware. The hardware or software includes one or more modules or units corresponding to the functions described above.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented in hardware, firmware implementation, or a combination thereof. When implemented in software, the functions described above may be stored in or transmitted as one or more instructions or code on a computer readable medium. Computer readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one location to another. A storage medium may be any available media that can be accessed by a computer. By way of example and not limitation, computer readable media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage media or other magnetic storage device, or can be used for carrying or storing in the form of an instruction or data structure. The desired program code and any other medium that can be accessed by the computer. Moreover, any connection can suitably be a computer readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable , fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave The procedure is included in the fixing of the medium to which it belongs. As used in the present invention, a disk and a disc include a compact disc (CD), a laser disc, a compact disc, a digital versatile disc (DVD), a floppy disk, and a Blu-ray disc, wherein the disc is usually magnetically copied, and the disc is The laser is used to optically replicate the data. Combinations of the above should also be included within the scope of the computer readable media.

The above is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or substitutions within the technical scope of the present invention. It should be covered by the scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims (22)

1. A remote control method, the method being applied to a first terminal having a touch screen, the method comprising:

   The first terminal displays a first control interface, where the first control interface is consistent with a first display interface currently output by the second terminal, and the second terminal is a terminal remotely controlled by the first terminal; a display interface displays at least two icons, the number of icons displayed by the first control interface is the same as the number of icons displayed by the first display interface, and the arrangement manner of the icons displayed by the first control interface is The icons displayed on the first display interface are arranged in the same manner, and the icons displayed on the first control interface are in one-to-one correspondence with the icons displayed on the first display interface;

   The first terminal receives a first touch operation input on the first control interface, where the first touch operation includes a selection operation of a first icon in the first control interface;

   The first terminal generates a first remote control instruction according to the first touch operation;

   The first terminal sends the first remote control instruction to the second terminal, where the first remote control instruction is used to instruct the second terminal to select the first display interface corresponding to the first icon. icon;

   After receiving the first touch operation, the first terminal displays a second control interface, where the second control interface is consistent with the second display interface output by the second terminal, and the second display interface is The display interface output by the second terminal after executing the first remote control instruction.
2. The method according to claim 1, wherein the generating, by the first terminal, the first remote control instruction according to the first touch operation comprises:

   Obtaining, by the first terminal, an initial position of the cursor before the first icon is selected;

   The first terminal generates the first remote control instruction according to the initial position of the cursor and the position where the first icon is located.
3. The method according to claim 2, wherein the first touch operation further comprises an operation of selecting a second icon corresponding to the initial position of the cursor before the first icon is selected, the second The icon is included in the first control interface; the initial position of the cursor is an initial position of the cursor on the first display interface.
4. The method according to any one of claims 1-3, wherein the second display interface is an editable input interface, and the second control interface is an input interface including a soft keyboard.
5. The method according to claim 4, wherein an interface layout of the soft keyboard in the second control interface is the same as an interface layout of a soft keyboard in the second display interface.
6. The method according to claim 4, wherein an interface layout of the soft keyboard in the second control interface is different from an interface layout of a soft keyboard in the second display interface.
7. The method according to any one of claims 4-6, further comprising:

   The first terminal receives a second touch operation input on a soft keyboard of the second control interface, and the second touch operation includes a selection operation of a first character on a soft keyboard of the second control interface;

   The first terminal generates a second remote control instruction according to the second touch operation;

   The first terminal sends the second remote control instruction to the second terminal, and the second remote control instruction is used to instruct the second terminal to input the first character in the second display interface.
8. The method according to any one of claims 1-7, wherein the displaying, by the first terminal, the second control interface after the receiving the first touch operation comprises:

   Obtaining, by the first terminal, the second control interface corresponding to the first touch operation from a memory of the first terminal;

   The first terminal displays the second control interface.
9. The method according to any one of claims 1-7, wherein the first remote command is sent by the first terminal After being sent to the second terminal, the method further includes:

   Sending, by the first terminal, an interface information request to the second terminal;

   Receiving, by the first terminal, interface information of the second display interface sent by the second terminal;

   The first terminal displays the second control interface according to the interface information.
10. The method according to any one of claims 1 to 7, wherein after the first terminal sends the first remote control command to the second terminal, the method further includes:

    The first terminal receives an operation for starting a photographing;

    The first terminal acquires a picture having the second display interface by taking a photo;

    The first terminal identifies content in the picture of the second display interface, and displays the identified content as the second control interface.
11. The method according to any one of claims 1-10, wherein the second control interface is consistent with the second display interface output by the second terminal, and includes:

    When the second display interface includes at least two icons, the number of icons displayed by the second control interface is the same as the number of icons displayed by the second display interface, and the row of icons displayed by the second control interface The layout of the icons displayed by the second display interface is the same as the icon displayed by the second display interface, and the icons displayed by the second control interface are in one-to-one correspondence with the icons displayed by the second display interface; or

    When the second display interface is an editable input interface, the second control interface is an input interface including a soft keyboard; or

    When the second display interface is an audio and video playing interface, the second control interface includes any one or more of the following: a volume control item, a play progress control item, and a play/stop control item.
12. The method of any of claims 1-11, further comprising:

    Before the first terminal displays the first control interface, the first terminal displays thumbnails or icons of at least two control interfaces;

    The first terminal receives a third touch operation;

    The first terminal selects the first control interface from the at least two control interfaces to perform display according to the third touch operation.
13. A method according to any one of claims 1-12, characterized in that:

    The first display interface and the second display interface are displayed by the second terminal; or

    The first display interface and the second display interface are output by the second terminal to an external device for display.
14. A method according to any one of claims 1-13, characterized in that:

    Transmitting, by the first terminal, the remote control instruction to the second terminal by using any one of the following communication modes: infrared, Bluetooth, Wi-Fi, and NFC; or

    The first terminal is a mobile phone or a tablet, and the second terminal is a television or a set top box; or

    The method further includes: the first terminal identifying a gesture of the user, the first terminal converting the gesture into a third remote control instruction, and transmitting the third remote control instruction to the second terminal; or

    The method further includes: the first terminal identifying a voice of the user, the first terminal converting the voice into a fourth remote control command, and transmitting the fourth remote control command to the second terminal.
15. A first terminal is configured to remotely control a second terminal, where the first terminal includes:

    touch screen;

    One or more processors;

    One or more memories, the one or more memories having one or more computer programs, the one or more computer programs including instructions that, when executed by the first terminal, cause the The first terminal performs the following steps:

    Displaying, by the touch screen, a first control interface, where the first control interface is consistent with a first display interface currently output by the second terminal; the first display interface displays at least two icons, the first control interface The number of icons displayed is the same as the number of icons displayed by the first display interface, and the icons displayed by the first control interface are arranged in the same manner as the icons displayed by the first display interface. The icons displayed by the first control interface are in one-to-one correspondence with the icons displayed by the first display interface;

    Receiving, by the touch screen, a first touch operation input on the first control interface, the first touch operation including a selection operation of a first icon in the first control interface;

    Generating a first remote control instruction according to the first touch operation;

    Transmitting the first remote control instruction to the second terminal, where the first remote control instruction is used to instruct the second terminal to select an icon corresponding to the first icon in the first display interface;

    After receiving the first touch operation, displaying, by the touch screen, a second control interface, where the second control interface is consistent with a second display interface output by the second terminal, where the second display interface is a display interface that is output after the second terminal executes the first remote control instruction.
16. The first terminal according to claim 15, wherein in the step of generating a first remote control instruction according to the first touch operation, the first terminal executing the instruction causes the first terminal to perform the following steps :

    Obtaining an initial position of the cursor before the first icon is selected;

    The first remote control instruction is generated according to the initial position of the cursor and the position where the first icon is located.
17. The first terminal according to claim 16, wherein the first touch operation further comprises an operation of selecting a second icon corresponding to the initial position of the cursor before the first icon is selected, The second icon is included in the first control interface; the initial position of the cursor is an initial position of the cursor on the first display interface.
18. The first terminal according to any one of claims 15-17, wherein the second display interface is an editable input interface, and the second control interface is an input interface including a soft keyboard.
19. The first terminal according to claim 18, wherein an interface layout of the soft keyboard in the second control interface is the same as an interface layout of a soft keyboard in the second display interface.
20. The first terminal according to claim 18, wherein an interface layout of the soft keyboard in the second control interface is different from an interface layout of a soft keyboard in the second display interface.
21. The first terminal according to any one of claims 18 to 20, wherein the first terminal executes the instruction such that the first terminal further performs the following steps:

    Receiving, by the touch screen, a second touch operation input on a soft keyboard of the second control interface, the second touch operation comprising a selection operation of a first character on a soft keyboard of the second control interface;

    Generating a second remote control instruction according to the second touch operation;

    And sending the second remote control instruction to the second terminal, where the second remote control instruction is used to instruct the second terminal to input the first character in the second display interface.
22. The first terminal according to any one of claims 15 to 21, wherein in the step of displaying the second control interface through the touch screen after receiving the first touch operation, the first terminal executes the The instructions cause the first terminal to perform the following steps:

    Acquiring, by the memory of the first terminal, the second control interface corresponding to the first touch operation;

Images