CN108476339B

CN108476339B - Remote control method and terminal

Info

Publication number: CN108476339B
Application number: CN201780004723.0A
Authority: CN
Inventors: 李创军; 黄薇
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2016-12-30
Filing date: 2017-07-14
Publication date: 2020-12-01
Anticipated expiration: 2037-07-14
Also published as: CN108476339A; WO2018120768A1

Abstract

The embodiment of the invention provides a remote control method and a terminal. The method comprises the following steps: the first terminal displays a first control interface, the first control interface is consistent with a first display interface currently output by a second terminal, and the second terminal is a terminal remotely controlled by the first terminal; the first display interface displays at least two icons; the method comprises the steps that a first terminal receives a first touch operation input on a first control interface, wherein the first touch operation comprises a selection operation of a first icon in the first control interface; the first terminal generates a first remote control instruction according to the first touch operation and sends the first remote control instruction to the second terminal, and the first remote control instruction is used for indicating the second terminal to select an icon corresponding to the first icon in the first display interface; and after receiving the first touch operation, the first terminal displays a second control interface, and the second control interface is consistent with a second display interface output after the second terminal executes the first remote control instruction. The embodiment can improve the efficiency and convenience of remote control operation.

Description

Remote control method and terminal

The present application claims priority of chinese patent application entitled "method and apparatus for remotely controlling a television by an intelligent terminal" filed by chinese patent office on 30/12/2016 under the reference of application number 201611265792.0, which is incorporated herein by reference in its entirety.

Technical Field

The embodiment of the invention relates to a communication technology, in particular to a remote control method and a terminal.

Background

With the development of network technology, users can watch rich video contents through an intelligent Television or an Internet Protocol Television (IPTV) or a set-top box, and can play, review or manage the video contents.

When video content is requested, viewed or managed, a television or a set-top box is usually operated by using a remote controller. In the prior art, the remote controller generally comprises a plurality of function keys and direction keys. When a user wishes to switch from one television interface to another, or when the user wishes to select a program from a plurality of program options, multiple operations of the keys of the remote control are typically required. Therefore, the remote controller is complicated to operate, and operation inconvenience is brought to a user.

Disclosure of Invention

The embodiment of the invention provides a remote control method and a terminal, which can improve the efficiency and convenience of remote control operation.

In a first aspect, an embodiment of the present invention provides a remote control method, where the method is applied to a first terminal having a touch screen. And the first terminal remotely controls the second terminal through the method. The method comprises the following steps: the first terminal displays a first control interface, and the first control interface is consistent with a first display interface currently output by the second terminal; the first display interface displays at least two icons, and the number and the arrangement mode of the icons displayed by the first control interface are respectively the same as those of the icons displayed by the first display interface; the icons displayed on the first control interface correspond to the icons displayed on the first display interface one by one; the first terminal receives a first touch operation input on the first control interface, wherein the first touch operation at least comprises a selection operation of a first icon in the first control interface; the first terminal generates a first remote control instruction according to the first touch operation; the first terminal sends the first remote control instruction to the second terminal, wherein the first remote control instruction is used for indicating the second terminal to select the icon corresponding to the first icon in the first display interface; after the first touch operation is received, the first terminal displays a second control interface, the second control interface is consistent with a second display interface output by the second terminal, and the second display interface is a display interface output by the second terminal after the first remote control instruction is executed. The control interface displayed by the first terminal can change along with the change of the display interface output by the second terminal, and the layout of the control interface is consistent with the current display interface, so that a user can control the display interface of the second terminal through a small amount of touch operation on the control interface, and the remote controller is more convenient to use.

Optionally, the generating, by the first terminal, the first remote control instruction according to the first touch operation specifically includes: the first terminal obtains an initial cursor position before the first icon is selected, and then generates the first remote control instruction according to the initial cursor position and the position of the first icon.

Optionally, the first touch operation further includes an operation of selecting a second icon corresponding to the initial position of the cursor before the first icon is selected, where the second icon is included in the first control interface; the initial cursor position is an initial cursor position on the first display interface. The initial position of the cursor can be obtained in various ways.

Optionally, the second display interface is an editable input interface, and the second control interface is an input interface including a soft keyboard. The interface layout of the soft keyboard in the second control interface is the same as or different from the interface layout of the soft keyboard in the second display interface. The second control interface may include the same soft keyboard as the second display interface, or may include a different soft keyboard, for example, the second control interface may include a soft keyboard for easier operation.

Optionally, the method further includes: the first terminal receives a second touch operation input on the soft keyboard of the second control interface, wherein the second touch operation comprises a selection operation on a first character on the soft keyboard of the second control interface; the first terminal generates a second remote control instruction according to the second touch operation; and the first terminal sends the second remote control instruction to the second terminal, and the second remote control instruction is used for indicating the second terminal to input the first character in the second display interface. Characters can be conveniently input in the second terminal through touch operation on the soft keyboard of the first terminal.

Optionally, after receiving the first touch operation, the first terminal displays a second control interface, which specifically includes: the first terminal acquires the second control interface corresponding to the first touch operation from a memory of the first terminal; and the first terminal displays the second control interface. The second control interface may be pre-stored in the first terminal.

Optionally, after the first terminal sends the first remote control instruction to the second terminal, the method further includes: the first terminal sends an interface information request to the second terminal; the first terminal receives interface information of the second display interface sent by the second terminal; and the first terminal displays the second control interface according to the interface information. When the first terminal does not always store the second control interface in advance, the second control interface can be requested from the second terminal.

Optionally, after the first terminal sends the first remote control instruction to the second terminal, the method further includes: the first terminal receives an operation for starting photographing; the first terminal obtains a picture with the second display interface by photographing; and the first terminal identifies the content in the picture of the second display interface and displays the identified content as the second control interface. When the first terminal does not store the second control interface in advance, the first terminal can acquire the second control interface in a photographing mode.

Optionally, the second control interface is consistent with the second display interface output by the second terminal, and specifically includes any one of the following. The first method comprises the following steps: when the second display interface comprises at least two icons, the number of the icons displayed by the second control interface is the same as the number of the icons displayed by the second display interface, the arrangement mode of the icons displayed by the second control interface is the same as the arrangement mode of the icons displayed by the second display interface, and the icons displayed by the second control interface are in one-to-one correspondence with the icons displayed by the second display interface. And the second method comprises the following steps: and when the second display interface is an editable input interface, the second control interface is an input interface comprising a soft keyboard. And the third is that: when the second display interface is an audio and video playing interface, the second control interface comprises any one or more of the following: volume control item, play progress control item, play/stop control item.

Optionally, the method further includes: before the first terminal displays the first control interface, the first terminal displays thumbnails or icons of at least two control interfaces; the first terminal receives a third touch operation; and the first terminal selects the first control interface from the at least two control interfaces to display according to the third touch operation. The first terminal can be triggered to display the first control interface through the third touch operation.

Optionally, when the second terminal is a television or similar display device, the first display interface and the second display interface are displayed by the second terminal. When the second terminal is a set-top box or similar equipment, the first display interface and the second display interface are output to external equipment by the second terminal for display, and the external equipment can be a television, a computer or similar display equipment.

Optionally, the first terminal sends the remote control instruction to the second terminal through any one of the following communication modes: infrared, bluetooth, Wi-Fi, and NFC.

Optionally, the first terminal is a mobile phone or a tablet computer, and the second terminal is a television or a set-top box.

Optionally, the method further includes: the first terminal identifies the gesture of the user, converts the gesture into a third remote control instruction and sends the third remote control instruction to the second terminal. Namely, in the process of remotely controlling the second terminal by the first terminal, the corresponding remote control operation can be executed by recognizing the gesture of the user.

Optionally, the method further includes: and the first terminal identifies the voice of the user, converts the voice into a fourth remote control instruction and sends the fourth remote control instruction to the second terminal. The first terminal may also control the second terminal by the user's voice.

In a second aspect, an embodiment of the present invention provides a first terminal, where the first terminal is configured to remotely control a second terminal, and the first terminal includes: a touch screen; one or more processors; one or more memories in which one or more computer programs are stored, the one or more computer programs comprising instructions which, when executed by the first terminal, cause the first terminal to perform any of the methods of the first aspect described above.

In a third aspect, embodiments of the present invention provide a Graphical User Interface (GUI) stored in a first terminal. The first terminal includes a touchscreen, one or more memories, one or more processors to execute one or more computer programs stored in the one or more memories. The GUI includes: the first GUI is displayed on the touch screen, the first GUI is consistent with a first display interface currently output by a second terminal, and the second terminal is a terminal remotely controlled by the first terminal; the first display interface displays at least two icons, the number of the icons displayed by the first GUI is the same as that of the icons displayed by the first display interface, the arrangement mode of the icons displayed by the first GUI is the same as that of the icons displayed by the first display interface, and the icons displayed by the first GUI correspond to the icons displayed by the first display interface one to one; responding to a first gesture of a user on the touch screen, generating a first remote control instruction according to the first gesture, and sending the first remote control instruction to the second terminal; the first gesture comprises a selection operation of a first icon in the first GUI, and the first remote control instruction is used for instructing the second terminal to select an icon corresponding to the first icon in the first display interface; and responding to the first gesture or the third gesture of the user on the touch screen, and displaying a second GUI in the touch screen, wherein the second GUI is consistent with a second display interface output by the second terminal, and the second display interface is output by the second terminal after the first remote control instruction is executed.

Optionally, the second display interface is an editable input interface, and the second GUI is an input interface including a soft keyboard. The interface layout of the soft keyboard in the second GUI is the same as or different from the interface layout of the soft keyboard in the second display interface.

Optionally, the GUI further comprises: responding to a second gesture input by a user on a soft keyboard of the second GUI, generating a second remote control instruction according to the second gesture, and sending the second remote control instruction to the second terminal; the second gesture comprises a selection operation of a first character on a soft keyboard of the second GUI, and the second remote control instruction is used for instructing the second terminal to input the first character in the second display interface.

Optionally, the second GUI is consistent with a second display interface output by the second terminal. Specifically, reference may be made to the foregoing description regarding the correspondence between the second control interface and the second display interface output by the second terminal.

In a fourth aspect, an embodiment of the present invention provides a first terminal, where the first terminal is configured to remotely control a second terminal, and the first terminal includes: the device comprises a display unit, an input unit, a control unit and a sending unit. The display unit is used for displaying a first control interface, the first control interface is consistent with a first display interface currently output by a second terminal, and the second terminal is a terminal remotely controlled by the first terminal; the first display interface displays at least two icons, the number of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, the arrangement mode of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, and the icons displayed by the first control interface are in one-to-one correspondence with the icons displayed by the first display interface. The input unit is used for receiving a first touch operation input on the first control interface, and the first touch operation comprises a selection operation of a first icon in the first control interface. The control unit is used for generating a first remote control instruction according to the first touch operation received by the input unit. The sending unit is configured to send the first remote control instruction to the second terminal, where the first remote control instruction is used to instruct the second terminal to select an icon corresponding to the first icon in the first display interface. The display unit is further configured to display a second control interface after receiving the first touch operation, where the second control interface is consistent with a second display interface output by the second terminal, and the second display interface is output by the second terminal after executing the first remote control instruction. The first terminal includes various units for implementing the methods in the first aspect, and the various units may be implemented by hardware or software.

In a fifth aspect, an embodiment of the present invention provides a remote control device, which is included in a first terminal, and has a function of implementing behaviors of the first terminal in the methods described in the first aspect. The functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules or units corresponding to the above functions.

In a sixth aspect, an embodiment of the present invention provides a computer program product including instructions, which, when run on an electronic device, causes the electronic device to perform the method according to the first aspect.

In a seventh aspect, an embodiment of the present invention provides a computer-readable storage medium, which includes instructions, when the instructions are executed on an electronic device, cause the electronic device to perform the method described in the first aspect.

In an eighth aspect, an embodiment of the present invention further provides a data processing system, which includes modules for executing the methods provided in the first aspect.

Drawings

Fig. 1 is a block diagram of a partial structure of a terminal according to an embodiment of the present invention;

fig. 2A is a schematic view of a scene of a remote control method according to an embodiment of the present invention;

fig. 2B is a flowchart of a remote control method according to an embodiment of the present invention;

fig. 3 is a schematic interface diagram of a first terminal according to an embodiment of the present invention;

FIGS. 4A-4E are schematic diagrams of a remote control interface provided by an embodiment of the present invention;

fig. 5 is a schematic interface diagram of a remote control method according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of another remote control interface provided by embodiments of the present invention;

FIG. 7 is a schematic diagram of another remote control interface provided by embodiments of the present invention;

FIGS. 8A and 8B are schematic diagrams of another remote control interface provided by an embodiment of the present invention;

FIG. 9 is a schematic diagram of another remote control interface provided by embodiments of the present invention;

FIG. 10 is a schematic diagram of another remote control interface provided by embodiments of the present invention;

FIG. 11 is a flow chart of another remote control method provided by an embodiment of the present invention;

fig. 12 is a flowchart of a method for performing remote control through gestures according to an embodiment of the present invention.

Detailed Description

When watching audio and video programs using a television, a set-top box or other similar terminals, the television, the set-top box or other similar terminals are generally controlled by a common remote controller. Through the remote control method provided by the embodiments of the invention, a terminal with a touch screen can be used for replacing a common remote controller to control a television, a set-top box or a similar terminal. The terminal may be, for example: mobile phones, Tablet Personal computers (Tablet Personal computers), Laptop computers (Laptop computers), digital cameras, Personal Digital Assistants (PDAs), navigation devices, Mobile Internet Devices (MIDs), Wearable devices (Wearable devices), and the like.

Fig. 1 is a block diagram of a partial structure of a terminal according to an embodiment of the present invention. The terminal is illustrated with a mobile phone 100 as an example. Referring to fig. 1, a cellular phone 100 includes: radio Frequency (RF) circuit 110, power supply 120, processor 130, memory 140, input unit 150, display unit 160, camera 165, sensor 170, audio circuit 180, wireless fidelity (Wi-Fi) module 190, infrared transmitter 192, and bluetooth module 194. Those skilled in the art will appreciate that the handset configuration shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

The following describes the components of the mobile phone 100 in detail with reference to fig. 1:

the RF circuit 110 may be used for transmitting and receiving information or for receiving and transmitting signals during a call. For example: RF circuitry 110 may send downlink data received from the base station to processor 130 for processing and may send uplink data to the base station. In general, RF circuits include, but are not limited to, an RF chip, an antenna, at least one Amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, a radio frequency switch, and the like. In addition, the RF circuitry 110 may also communicate wirelessly with networks and other devices. The wireless communication may use any communication standard or protocol, including but not limited to Global System for Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), and the like.

The memory 140 may be used to store software programs and modules, and the processor 130 executes various functional applications and data processing of the mobile phone 100 by operating the software programs and modules stored in the memory 140. The memory 140 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone 100, and the like. Further, the memory 140 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 150 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the cellular phone 100. Specifically, the input unit 150 may include a touch panel 151 and other input devices 152. The touch panel 151, also referred to as a touch screen, may collect a touch operation performed by a user on or near the touch panel 151 (e.g., an operation performed by the user on or near the touch panel 151 using any suitable object or accessory such as a finger or a stylus), and drive a corresponding connection device according to a preset program. Alternatively, the touch panel 151 may include two parts, a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 130, and can receive and execute commands sent by the processor 130. In addition, the touch panel 151 may be implemented in various types, such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. The input unit 150 may include other input devices 152 in addition to the touch panel 151. In particular, other input devices 152 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit 160 may be used to display information input by or provided to the user, as well as various menus and interfaces of the handset 100. The display unit 160 may include a display panel 161, and optionally, the display panel 161 may be configured in the form of a Liquid Crystal Display (LCD), an electromechanical laser display (OLED), or the like. Further, the touch panel 151 may cover the display panel 161, and when the touch panel 151 detects a touch operation thereon or nearby, the touch panel transmits the touch operation to the processor 130 to determine the type of the touch event, and then the processor 130 provides a corresponding visual output on the display panel 161 according to the type of the touch event. Although the touch panel 151 and the display panel 161 are shown in fig. 1 as two separate components to implement the input and output functions of the mobile phone 100, in some embodiments, the touch panel 151 and the display panel 161 may be integrated to implement the input and output functions of the mobile phone 100.

The handset 100 may also include one or more cameras 165. The camera 165 may be used to capture and shoot images. One or more front-facing cameras and one or more rear-facing cameras may be included in the handset 100.

The handset 100 may also include at least one sensor 170, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that adjusts the brightness of the display panel 161 according to the brightness of ambient light, and a proximity sensor that turns off the display panel 161 and/or the backlight when the mobile phone 100 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used for applications of recognizing the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), and the like. The mobile phone 100 may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which are not described herein again.

The audio circuitry 180, speaker 181, microphone 182 may provide an audio interface between a user and the handset 100. The audio circuit 180 may transmit the electrical signal converted from the received audio data to the speaker 181, and the electrical signal is converted into a sound signal by the speaker 181 and output; on the other hand, the microphone 182 converts the collected sound signals into electrical signals, which are received by the audio circuit 180 and converted into audio data, which are then output to the RF circuit 110 for transmission to, for example, another cell phone, or to the memory 140 for further processing.

Wi-Fi, infrared, and Bluetooth are short-range wireless transmission technologies. The handset 100, through the Wi-Fi module 190, may assist the user in sending and receiving e-mail, browsing web pages, accessing streaming media, etc., which provides wireless broadband internet access for the user. The cellular phone 100 may emit infrared light to the outside in a certain range through the infrared emitter 192. The mobile phone can communicate with an external device through the bluetooth module 194.

The processor 130 is a control center of the mobile phone 100, connects various parts of the entire mobile phone by using various interfaces and lines, and performs various functions of the mobile phone 100 and processes data by operating or executing software programs and/or modules stored in the memory 140 and calling data stored in the memory 140, thereby implementing various services based on the mobile phone. Optionally, processor 130 may include one or more processing units; preferably, the processor 130 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 130.

In this embodiment of the present invention, the processor 130 may execute the program instructions stored in the memory 140, so that the mobile phone as the first terminal performs the method shown in any one of the following embodiments.

The handset 100 also includes a power supply 120 (e.g., a battery) for powering the various components, which may preferably be logically connected to the processor 130 via a power management system, such that the power management system may manage charging, discharging, and power consumption functions.

The remote control method provided by the embodiment of the invention is explained in detail below with reference to the accompanying drawings. The terminal capable of replacing the remote controller is referred to as a first terminal, and the remote-controlled television, set-top box or the like is referred to as a second terminal. Fig. 2A is a scene schematic diagram of a remote control method according to an embodiment of the present invention. Fig. 2B is a flowchart of a remote control method according to an embodiment of the present invention.

As shown in fig. 2A, the first terminal 100 may control the second terminal 200 through a remote control method provided by an embodiment of the present invention. It should be noted that the remote control method provided by the embodiment of the present invention does not conflict with the existing remote control method, that is, the second terminal 200 may also be controlled by the common remote controller 300.

Fig. 3 is a schematic interface diagram of a first terminal according to an embodiment of the present invention. Fig. 4A-4E are schematic diagrams of a remote control interface provided by an embodiment of the invention. As shown in fig. 3, a plurality of applications, which may include a remote control application 400, may be installed on the first terminal. After the remote control application 400 is opened, the first terminal can control the second terminal using the remote control application 400. As shown in fig. 2B, the remote control method includes the following steps when a second terminal is remotely controlled using a first terminal.

Step 201, the first terminal displays a first control interface. As shown in fig. 4D, the second terminal currently outputs the first display interface 401, and the first display interface displays at least two icons 402. Fig. 4D shows an exemplary display of 6 icons 402. The first control interface 410 is consistent with the first display interface 401, and specifically may be: the number of icons 412 displayed by the first control interface 410 is the same as the number of icons 402 displayed by the first display interface 401; the arrangement mode of the icons displayed on the first control interface 410 is the same as that of the icons displayed on the first display interface 401; and the icons displayed on the first control interface correspond to the icons displayed on the first display interface one to one. Wherein, because the size and shape of the first control interface and the first display interface can be very different, the content displayed in the first control interface can be zoomed or adapted to the size of the content displayed in the first display interface. And the identification of the icon in the first control interface 410 and the identification of the icon in the first display interface 401 may be the same (as shown in fig. 4D) or different (as shown in fig. 4E). The first display interface may take many forms, and FIG. 4D is merely exemplary. For example: as shown in fig. 9, the first display interface 901 may be in another form, and the corresponding first control interface 910 is in a form corresponding to the first display interface 901. Through the interface shown in fig. 9, programs played within seven days of 1 month, 1 day to 1 month, 7 days can be reviewed.

It should be noted that, based on the first display interface, the first control interface is displayed on the first terminal. That is, the embodiment of the present invention does not project the interface displayed on the first terminal to the second terminal. In addition, when the first terminal is used to control the second terminal, the second terminal may also be controlled by a common remote controller, as shown in fig. 2A.

Step 202, the first terminal receives a first touch operation input on the first control interface. The first touch operation may include a selection operation of a first icon in the first control interface. The first touch operation may be one or more touch operations. That is, the user may perform one or more touch operations on the first control interface.

And 203, the first terminal generates a first remote control instruction according to the first touch operation and sends the first remote control instruction to the second terminal. The first remote control instruction is used for indicating the second terminal to select the icon corresponding to the first icon in the first display interface.

Specifically, the process of generating the first remote control instruction by the first terminal may include: the first terminal acquires an initial position of a cursor before the first icon is selected; and then the first terminal generates the first remote control instruction according to the initial position of the cursor and the position of the first icon. As with the first display interface 401 shown in fig. 5(a), the initial position of the cursor is at icon 4021. The second terminal may send the relative position of the initial cursor position in the first display interface 401 to the first terminal, so that the first terminal obtains a second icon 4121 corresponding to the initial cursor position on the first control interface; alternatively, the first terminal may receive a touch operation of a user on the second icon 4121 to recognize that the initial position of the cursor corresponds to the second icon 4121; or, when entering the first control interface 410, the initial position of the cursor defaults to be at the second icon 4121, and the first terminal may acquire the default position when entering the first control interface 410. After the second terminal acquires the second icon 4121 corresponding to the initial cursor position, the initial cursor position may be displayed on the first control interface 410, as shown in fig. 5 (b). When the user wishes to select an icon 4023 in the first display interface, the user may select a first icon 4123 corresponding to the icon 4023 on the first control interface 410, as shown in fig. 5 (c). After the first terminal acquires that the first icon 4123 is selected, the first terminal generates the first remote control instruction according to the initial position and the position of the first icon 4123, and sends the first remote control instruction to the second terminal. The second terminal selects the icon 4023 according to the first remote control instruction, as shown in fig. 5 (d). The second icon is included in the first control interface. The initial cursor position refers to a position where a cursor is located when the first display interface is entered. Or, the initial position of the cursor refers to a position where the cursor is located before the second terminal receives the first remote control instruction. The first control interface and the first display interface are distributed in the same icon mode, so that the first control interface comprises an icon corresponding to the initial position of the cursor.

Wherein the first touch operation may be in different forms.

When the first terminal identifies the initial position of the cursor according to the touch operation of the user, the first touch operation may have the following two operations in sequence: firstly, touch operation is carried out on a second icon corresponding to the initial position of the cursor, and then selection operation is carried out on the first icon; for example: the touch operation on the second icon can be click operation; the selection operation on the first icon may be clicking, double-clicking, long-pressing or re-pressing, or after clicking, double-clicking, long-pressing or re-pressing, a touch operation on a determined button is further included. That is, the selection of the first icon may be achieved only by clicking, double-clicking, long-pressing, or re-pressing the first icon, or the first control interface 410 may further include a determination button, and the selection of the first icon may be achieved by clicking or touching the determination button after clicking, double-clicking, long-pressing, or re-pressing the first icon.

When the second terminal sends the initial cursor position to the first terminal, or when the first terminal may obtain a default initial cursor position when entering the first control interface 410, the first touch operation may only include a selection operation on the first icon.

The first remote control command may take a variety of forms.

One is as follows: the first terminal may convert the first touch operation into a common one or more remote control commands. For example: the first terminal acquires the cursor needing to move rightwards twice according to the coordinate of the second icon 4121 corresponding to the initial position of the cursor and the coordinate information of the first icon 4123, so that the first terminal converts the first touch operation into two remote control instructions moving rightwards once, and each remote control instruction represents moving rightwards once. In this way, the second terminal may use an existing television or set-top box, that is, the second terminal may be controlled by using the remote control method provided in the embodiment of the present invention without improving the existing second terminal.

The other one is as follows: the first terminal may convert the first touch operation into a remote control command representing a movement displacement of a cursor. For example: the first terminal acquires that the cursor needs to move rightwards twice according to the coordinate information of the second icon 4121 and the coordinate information of the first icon 4123 corresponding to the initial position of the cursor, and therefore the first terminal converts the first touch operation into a remote control instruction which represents moving rightwards twice. And the second terminal receives and analyzes the first remote control instruction and then controls the cursor to move rightwards twice. In this way, the same database may be preset in the first terminal and the second terminal, so that the second terminal can correctly analyze the remote control instruction sent by the first terminal.

And the third is that: the first terminal may convert the first touch operation into a remote control instruction representing a coordinate change of the touch screen. For example: the first terminal sends the coordinate information of the second icon 4121 and the coordinate information of the first icon 4123 corresponding to the initial position of the cursor to the second terminal as the first remote control instruction; or sending the coordinate information of the first icon 4123 relative to the second icon 4121 as the first remote control instruction to the second terminal. And the second terminal receives and analyzes the first remote control instruction and then controls the cursor to move rightwards twice. In this way, the same database may also be preset in the first terminal and the second terminal, so that the second terminal can correctly analyze the remote control instruction sent by the first terminal.

And 204, after the first touch operation is received, the first terminal displays a second control interface. The second control interface is consistent with a second display interface output by the second terminal, and the second display interface is output by the second terminal after the second terminal executes the first remote control instruction.

It should be noted that the execution order of step 203 and step 204 is not limited in this embodiment, step 204 may be executed after step 203, and step 204 may also be executed in parallel with step 203.

And the second terminal outputs the second display interface after executing the first remote control instruction, so that the first terminal also switches the first control interface into the second control interface consistent with the second display interface, and the first terminal can continuously control the second terminal.

The second control interface corresponding to the first touch operation after the first touch operation is executed can be stored in the first terminal in advance. After the first terminal receives the first touch operation, the first terminal may acquire information of the second control interface from a memory and then display the second control interface. For example: after the second terminal receives the first remote control instruction, the interface in fig. 5(d) is switched to a second display interface 601 shown in fig. 6. If the first terminal stores the second control interface 610 corresponding to the second display interface 601 in advance, the first terminal automatically switches to display the second control interface 610, as shown in fig. 6.

Further, if the second control interface is not pre-stored in the first terminal, the method may further include, after step 203, the following step: the first terminal sends an interface information request to the second terminal; and the first terminal receives the interface information of the second display interface sent by the second terminal. Then, in step 204, the first terminal may obtain and display the second control interface according to the interface information. Specifically, as shown in fig. 5(c), after the first icon 4123 is selected, the user may request interface information of the second control interface by clicking a button 420. The first terminal sends an interface information request to the second terminal after detecting that the button 420 is touched; after receiving the interface information request, the second terminal obtains interface information of the second display interface 601 currently output by the second terminal, for example, the second terminal may capture a screen of the second display interface 601 or directly obtain display data corresponding to the second display interface 601; then the second terminal sends the interface information of the second display interface to the first terminal; and after the first terminal processes the interface information, outputting the second control interface 610.

Alternatively, if the second control interface is not pre-stored in the first terminal, the method may further include, after step 203, the following step: the first terminal receives an operation for starting photographing; the first terminal obtains a picture with the second display interface by photographing; and the first terminal identifies the content in the picture of the second display interface and displays the identified content as the second control interface. Specifically, as shown in fig. 5(c), the button 420 may be replaced by a photographing button, and after the first icon 4123 is selected, the user may photograph the second display interface by clicking the photographing button. And after the first terminal acquires the picture of the second display interface, carrying out icon identification and cutting on the picture, and then outputting the second control interface.

It should be noted that, after the second control interface is acquired by taking a picture or making a request to the second terminal, the first terminal may store the second control interface, so that the second control interface may be automatically displayed in the next operation.

The correspondence between the second control interface and the second display interface may specifically include any one or more of the following situations.

The first method comprises the following steps: and when the second display interface is an editable input interface, the second control interface is an input interface comprising a soft keyboard. The input and editing on the second display interface may be implemented by a soft keyboard in the second control interface. The interface layout of the soft keyboard in the second control interface and the interface layout of the soft keyboard in the second display interface may be the same, as shown in fig. 8A. Or, the interface layout of the soft keyboard in the second control interface and the interface layout of the soft keyboard in the second display interface may also be different, as shown in fig. 8B.

And the second method comprises the following steps: when the second display interface includes at least two icons, for example, when the second display interface is the same as or similar to the first display interface or the second display interface shown in fig. 4D, 4E, 5, or 6, the number of icons displayed by the second control interface is the same as the number of icons displayed by the second display interface, the arrangement manner of the icons displayed by the second control interface is the same as the arrangement manner of the icons displayed by the second display interface, and the icons displayed by the second control interface correspond to the icons displayed by the second display interface one to one. That is, the second control interface is the same as or similar to the first or second control interface shown in fig. 4D, 4E, 5, or 6.

And the third is that: when the second display interface is an audio and video playing interface, the second control interface comprises any one or more of the following items: volume control item, play progress control item, play/stop control item. As shown in fig. 10, when the second terminal is playing an audio-video program, the first terminal presents a second control interface 1010, which includes a volume control item 1012, a play progress control item 1013, and a play/stop control item 1015. The volume, the playing progress or the playing or pausing of the program can be controlled through the control interface of the first terminal. It should be noted that the play progress control item 1013 and the play/stop control item 1015 are generally used to control non-live audiovisual programs. For the live audio and video program, the second control interface may not include the play progress control item and the play/stop control item, or the play progress control item and the play/stop control item are invalid icons.

Further, after step 204, when the second display interface is an editable input interface and the second control interface is an input interface including a soft keyboard, the method may further include the following steps 205 and 206. For example: as shown in fig. 7, 8A and 8B, after the search button 480 on the first control interface 410 is clicked, the first terminal presents the

second control interface

810A or 810B.

And step 205, the first terminal receives a second touch operation input on the soft keyboard of the second control interface. The second touch operation comprises a selection operation of a first character on a soft keyboard of the second control interface.

And the second control interface displayed by the first terminal comprises a soft keyboard. When a user needs to input information in the second display interface, touch input can be directly performed in the soft keyboard on the second control interface, and the touch input is the second touch operation. The second touch operation may be a touch operation on a first character on the soft keyboard, for example, a click, double click, long press, or re-press operation on the first character.

And step 206, the first terminal generates a second remote control instruction according to the second touch operation, and sends the second remote control instruction to the second terminal, wherein the second remote control instruction is used for indicating the second terminal to input the first character in the second display interface.

The first terminal may generate a plurality of second remote control instructions according to the second touch operation, specifically:

the first method comprises the following steps: the first terminal receives a plurality of second touch operations in sequence, and each second touch operation corresponds to one character; the first terminal can obtain the relative positions of the characters corresponding to the two second touch operations according to the two adjacent second touch operations, and then converts the relative positions into one or more common remote control instructions, wherein the one or more remote control instructions are used for expressing the displacement of the cursor movement. For example: as shown in fig. 8A, the first terminal receives a second touch operation 811 for a character H and then receives a second touch operation 814 for the character a, and the first terminal may obtain the relative positions of the two characters according to the second touch operation 811 and the second touch operation 814, that is, when the cursor is at the character H, the cursor is moved left 7 times to obtain the character a; the relative position is then converted into seven remote control commands moving once to the left, each remote control command indicating moving once to the right. In the process of executing the current input operation, more than one second touch operation is taken as a reference in each second touch operation. The first second touch operation takes the character corresponding to the initial position of the cursor as a reference. Wherein the initial position of the cursor refers to a position where the cursor is located when the second terminal just enters the second display interface. The initial position of the cursor may be sent to the first terminal by the second terminal, or a default value may be obtained in a memory by the first terminal, or before a user starts an input operation, a touch operation may be performed on a character corresponding to the initial position of the cursor in the second control interface according to the initial position of the cursor presented in the second terminal.

And the second method comprises the following steps: the first terminal receives a plurality of second touch operations in sequence, and each second touch operation corresponds to one character; the first terminal can obtain the relative positions of the characters corresponding to the two second touch operations according to the two adjacent second touch operations, and then converts the relative positions into a remote control instruction for representing the movement displacement of the cursor. For example: as shown in fig. 8A, the first terminal receives the second touch operation 811 first and receives the second touch operation 814 later, and it can be obtained that the character a can be obtained by moving the cursor left 7 times; the relative position is then converted into a remote control command that moves seven times to the left. And the second terminal receives and analyzes the second remote control instruction, and then controls the cursor to move leftwards seven times. In this way, the same database may be preset in the first terminal and the second terminal, so that the second terminal can correctly analyze the remote control instruction sent by the first terminal. In the same way as in the first mode, in the process of executing the current input operation, more than one second touch operation is taken as a reference in each second touch operation, and the first second touch operation is taken as a reference in a character corresponding to the initial position of the cursor.

And the third is that: the first terminal can convert the second touch operation of two adjacent times into a remote control instruction representing the coordinate change of the touch screen. For example: as shown in fig. 8A, the first terminal sends the coordinate information of the character H and the coordinate information of the character a as the second remote control instruction to the second terminal, or sends the coordinate information of the character H with respect to the character a as the second remote control instruction to the second terminal. And the second terminal receives and analyzes the second remote control instruction, and then controls the cursor to move leftwards seven times. In this way, the same database may also be preset in the first terminal and the second terminal, so that the second terminal can correctly analyze the remote control instruction sent by the first terminal. In the same way as in the first mode, in the process of executing the current input operation, more than one second touch operation is taken as a reference in each second touch operation, and the first second touch operation is taken as a reference in a character corresponding to the initial position of the cursor.

And fourthly: and the first terminal directly sends the character corresponding to the second touch operation as a second remote control instruction to the second terminal, and the second terminal directly identifies and displays the character. In this way, the second terminal can acquire the corresponding character directly according to the second touch operation of this time without depending on the initial position of the cursor and the second touch operation of the last time.

Further, after step 204, when the second display interface includes at least two icons, and the layout (i.e. the number and arrangement) of the icons in the second control interface is the same as the layout of the icons in the second display interface, the method may further include step 207. For example: in fig. 5(c), after the first icon 4123 is selected, the second terminal displays the second display interface 601 and the first terminal displays the second control interface 610, as shown in fig. 6.

And step 207, the first terminal receives the touch operation input on the second control interface, generates a remote control instruction according to the touch operation, and then sends the remote control instruction to the second terminal to control the second terminal. The second display interface is the same as or similar to the first display interface, and the second control interface is the same as or similar to the first control interface, so that the specific steps of step 207 may refer to the descriptions in step 201 to step 203, and are not described herein again. For example, as shown in fig. 6, the initial position of the cursor corresponds to the icon movie 1, and after the icon movie 8 is selected, the first terminal may send a remote control command to the second terminal, so that the second terminal moves the cursor three times to the right and once down.

Further, after step 204, when the second display interface is an audio/video playing interface, the second control interface includes any one or more of the following items: volume control item, play progress control item, play/stop control item, the method may further comprise the following step 208 and said step 209.

And 208, the first terminal receives the touch operation input on the second control interface. The touch operation may be a single click, a double click, a long press, a heavy press, or a slide operation. For example: as shown in fig. 10, when the second terminal plays an audio/video program, the user may adjust the volume by sliding left and right on slider 912, may adjust the playing progress by sliding left and right on slider 913, and may control the playing and pausing of the program by clicking, double clicking, long pressing, or re-pressing button 915.

And 209, the first terminal generates a remote control instruction according to the touch operation in the step 208, and sends the remote control instruction to the second terminal, wherein the remote control instruction is used for controlling the volume, the progress and/or the playing state of the currently played program. Specifically, the first terminal may generate a remote control command for controlling a volume or a playing progress according to the direction and distance of the touch slide, or may generate a remote control command for controlling a playing state according to a click state of a playing button (for example, the button 915).

It should be noted that the first control interface of step 201 may be triggered to be displayed in various ways. For example, before step 201, the embodiment of the present invention may further include the following step 220, step 230, and step 240A, step 240B, or step 240C.

And step 220, after the first terminal enters the remote control interface, displaying thumbnails or icons of at least two control interfaces. As shown in FIGS. 3 and 4A, after clicking on the remote control icon 300 in FIG. 3, the remote control interface 430 shown in FIG. 4A may be entered. The remote control interface 430 may include thumbnails or icons for multiple control interfaces, such as menu one, menu two, and menu three. At this time, the second terminal displays the first display interface, for example, the first display interface 401 in fig. 4D.

And step 230, the first terminal receives a third touch operation. The third touch operation may be selection of a thumbnail or an icon shown in the remote control interface 430, or the third touch operation may trigger the first terminal to send an interface information request to the second terminal, or the third touch operation may trigger the first terminal to photograph a first display interface currently output by the second terminal. When the first terminal stores a control interface corresponding to the first display interface currently output by the second terminal in advance, step 240A is executed after step 230. And when the control interface corresponding to the first display interface currently output by the second terminal is not stored in the first terminal, executing step 240B or step 240C after step 230.

Step 240A, the first terminal selects the first control interface from the at least two control interfaces according to the third touch operation, and then the first terminal displays the first control interface in step 201. Since the first control interface is pre-stored in the first terminal, the user can enter the first control interface by selecting a thumbnail or an icon corresponding to the first control interface from the remote control interface 430. For example: if the menu pre-stored in the first terminal is the first control interface, the first control interface 410 can be accessed by selecting the menu as soon as the menu is selected, as shown in fig. 4A and 4D.

Step 240B, the first terminal sends an interface information request to the second terminal according to the third touch operation; and the first terminal receives the interface information of the first display interface sent by the second terminal. Then, in step 201, the first terminal may obtain and display the first control interface according to the interface information. For example: as shown in fig. 4A, the remote control interface 430 includes a request button 440. When the first terminal does not have the first control interface, that is, the menu is single, the menu is two, and the menu is not three, the interface information of the first display interface can be requested to the second terminal by clicking the request button 440. Then, the first terminal generates the first control interface according to the interface information of the first display interface, and may display a menu four 460 corresponding to the first control interface on a remote control interface 430, as shown in fig. 4C. The first control interface is then accessible by selection of menu four 460.

Step 240C, the first terminal starts to take a picture according to the third touch operation; the first terminal obtains a picture with the first display interface by photographing; the first terminal identifies content in the picture of the first display interface, and then displays the identified content as the first control interface in step 201. For example: as shown in fig. 4B, the remote control interface 430 includes a take button 450. When the first control interface is not available in the first terminal, the photographing can be started by clicking the photographing button 450. After the first terminal acquires the information of the first control interface, a fourth menu corresponding to the first control interface may be displayed on a remote control interface 430, as shown in fig. 4C. And then the first control interface can be accessed by selecting the menu four.

The process of acquiring the first control interface by the first terminal in step 240B and step 240C is similar to the process of acquiring the second control interface.

It should be noted that, the first control interface in this embodiment may be the first control interface after the first terminal enters the remote control program, or may be any one control interface after the first terminal enters the remote control program.

The second terminal in the embodiments of the present invention may be an intelligent television or a network television, that is, a television capable of playing audio and video programs on demand. In this case, the first display interface and the second display interface are output and displayed by the second terminal.

In other embodiments, the second terminal according to each embodiment of the present invention may be a set-top box, and the set-top box is used in cooperation with a television or other display devices to play an audio/video interface, so that the set-top box can play audio/video programs on demand. In this case, the first display interface and the second display interface are output to an external device such as a television by the second terminal and then displayed by the external device.

According to the remote control method provided by the embodiment of the invention, the control interface displayed by the first terminal can change along with the change of the display interface output by the second terminal, and the layout of the control interface is consistent with the current display interface, so that a user can control the display interface of the second terminal on the control interface through a small amount of touch operation, and the remote controller is more convenient to use. In the prior art, when a television is remotely controlled by a common remote controller, the same remote control operation as that of the embodiment of the invention can be completed by operating the direction keys or other similar keys more times. In addition, in the scheme of using the mobile phone as the remote controller in the prior art, the mobile phone displays virtual keys which are completely the same as the keys of the common remote controller, and then the television is remotely controlled by using the same operation mode as the common remote controller. Compared with the prior art, the remote control method provided by the embodiment of the invention is simpler and more convenient.

The remote control method provided by the embodiment of the invention is mainly applied to remote control of a television or similar display equipment. And the first terminal may be a remote controller for various appliances. That is, after the first terminal opens the remote control application, remote control interfaces respectively corresponding to the plurality of electric appliances may be displayed on the interface, and the user may select a remote control interface matched with the electric appliance to be remotely controlled. For example: the first terminal can display the electric appliance for remote control of a television, a set top box, a DVD, an air conditioner, an electric fan and the like.

Fig. 11 is a flowchart of another remote control method according to an embodiment of the present invention. In the method, the first terminal is taken as a mobile phone as an example, the second terminal is taken as a set-top box as an example, and an interface output by the second terminal can be displayed through a television, a computer or other display equipment. The method is based on the embodiment described above with reference to fig. 1-10. As shown in fig. 11, the method includes:

step 1101, the mobile phone opens a preset remote control application.

Step 1102, selecting a corresponding control interface according to a scene needing to be operated on a television screen. The corresponding control interface can be manually selected in the mobile phone by a user according to the current display content of the television screen, or the current display content of the television can be sent to the mobile phone by the set-top box, and the corresponding control interface can be automatically selected by the mobile phone. If the current television interface displays a menu interface, executing step 1103; if the current TV interface displays an editable input interface, execute step 1108; if the current tv interface is playing the audio/video interface, step 1110 is executed.

Step 1103, the mobile phone enters a menu control interface of the remote control application. After step 1103, step 1104, step 1105 and step 1106 may be executed, respectively, according to whether the mobile phone stores the control interface corresponding to the display interface of the current television.

And 1104, selecting a control interface corresponding to the current television display interface from the prestored control interfaces by the mobile phone according to the selection operation of the user. When the control interface corresponding to the display interface of the current television is stored in the mobile phone, the user can directly select the control interface on the mobile phone.

Step 1105, the mobile phone photographs the television screen according to the operation of starting photographing triggered by the user, then automatically cuts and analyzes the photographed photo, identifies the menu options in the photo, and then generates a corresponding control interface.

Step 1106, the mobile phone sends an interface information request to the set-top box according to the trigger of the user, receives the interface information of the current television sent by the set-top box, and generates a corresponding control interface according to the interface information. After step 1105 or step 1106, the handset may save the generated control interface so that the user can select the control interface directly at the next use without taking a picture or requesting from the set-top box.

After

steps

1104, 1105, and 1106, the method proceeds to step 1107.

Step 1107, the mobile phone obtains the touch operation of the user on the mobile phone screen, and then converts the touch operation into a remote control instruction. For example: the mobile phone remote control application converts the touch operation F1 into a remote control command O1 — a1(F1) through the command library a 1. After the handset generates the remote control command, the method performs step 1112.

And step 1108, the mobile phone enters an input interface of the remote control application.

In step 1109, the mobile phone receives characters, such as english or pinyin letters, input by the user, and then converts the characters F2 into remote control commands O2 — a2 through the command library a2(F2), and then executes step 1112.

Step 1110, the mobile phone enters a volume/play adjustment interface of the remote control application.

In step 1111, the mobile phone receives a sliding or clicking operation of the user on the volume/play adjustment interface, converts the sliding or clicking operation F3 into a remote control command O3 — A3(F3) through the command library A3, and then executes step 1112.

And step 1112, the mobile phone sends the generated remote control command O1, O2 or O3 to the set-top box.

And step 1113, after the set-top box receives the remote control command, resolving the O1, O2 or O3 command into corresponding operation through a command library matched with the command library A1, and then executing the corresponding operation to realize menu jump, volume adjustment, play control or character input.

After step 1112, the method may further comprise the steps of:

and 1114, the mobile phone executes the jump of the control interface according to the menu jump of the television. The user can select a corresponding control interface on the mobile phone according to the interface change of the television. And the control interface can be automatically jumped by the mobile phone. The interface after the jump can be requested to the set top box by the mobile phone or can be photographed by the mobile phone. In addition, the mobile phone can record the interface before the jump and record the corresponding relation between the touch operation F1 and the interface after the jump, so that the mobile phone can realize the automatic jump of the interface when the same touch operation F1 is performed next time, thereby reducing the number of times of photographing operations again or requests to the set-top box.

After step 1107, step 1112, the tv may jump to the editable input interface, and in this case, the handset may jump to the input interface as described in step 1108.

The process of converting the touch operation into the remote control instruction in step 1107, step 1109 and step 1111 may be referred to as detailed description in the foregoing embodiments.

On the basis of the foregoing embodiments, the embodiments of the present invention may further provide a method for performing remote control through gestures. The mobile phone supports various gesture operations, and remote control operation efficiency and user operation experience can be improved by utilizing the advantages of the gesture operations. The embodiment of the invention lists several gesture operations, but is not limited to the gesture operations actually, all the gesture operations supported by the mobile phone can be applied to remote control operation, and the user can configure various gestures according to own preference to realize the remote control operation. In the remote control method according to the foregoing embodiments, remote control by gestures may be further implemented by the method shown in fig. 12. Fig. 12 is a flowchart of a method for performing remote control through gestures according to an embodiment of the present invention. As shown in fig. 12, the method includes:

step 1201, the mobile phone opens the remote control application.

Step 1202, the remote control application of the mobile phone recognizes the gesture operation of the user and converts the gesture into a corresponding remote control instruction. The corresponding relation between the gesture operation and the remote control instruction can be preset. The corresponding relation may be default setting of the mobile phone when leaving factory, or may be set and edited by the user.

And step 1203, the mobile phone sends the remote control instruction to the set-top box.

And step 1204, the set-top box receives the remote control instruction, converts the remote control instruction into an executable operation instruction, and executes an operation corresponding to the operation instruction. For example: the set top box can execute operations of returning to the previous layer, shutting down, audio and video fast forward or fast backward, returning to the main menu and the like according to gesture operations of a user. The gesture operation of the user can be various operations which are easy to recognize and not easy to misjudge, such as circle drawing operation, fold line operation or operation of other specific graphs. It should be noted that the gesture of the user may be a gesture of contacting a touch screen of the mobile phone, or may be a gesture of non-contacting the touch screen of the mobile phone.

The first terminal provided by the embodiments of the present invention may further have a voice recognition function. In the process of remote control, the voice recognition function can be started, and voice sent by a user is converted into a remote control instruction to remotely control the second terminal.

In each embodiment of the invention, the communication mode between the first terminal and the second terminal can be infrared, Bluetooth, Wi-Fi or NFC. That is, the first terminal may send the remote control instruction to the second terminal through infrared, bluetooth, Wi-Fi or NFC technology, and the second terminal may use any one of these communication modes when sending the interface information to the first terminal.

The embodiment of the invention can realize the operation of remotely controlling the set top box or the television by the touch screen by adopting a widely-used handheld terminal, reduces the dependence on a common remote controller and reduces the hardware cost.

The embodiment of the invention provides a simple and quick touch screen remote control operation method aiming at different use scenes, and improves the remote control operation efficiency and experience of users.

Embodiments of the present invention also provide a Graphical User Interface (GUI) stored in a first terminal comprising a touch screen, one or more memories, and one or more processors to execute one or more computer programs stored in the one or more memories. Wherein the GUI comprises:

the first GUI is displayed on the touch screen, the first GUI is consistent with a first display interface currently output by a second terminal, and the second terminal is a terminal remotely controlled by the first terminal; the first display interface displays at least two icons, the number of the icons displayed by the first GUI is the same as that of the icons displayed by the first display interface, the arrangement mode of the icons displayed by the first GUI is the same as that of the icons displayed by the first display interface, and the icons displayed by the first GUI correspond to the icons displayed by the first display interface one to one;

responding to a first gesture of a user on the touch screen, generating a first remote control instruction according to the first gesture, and sending the first remote control instruction to the second terminal; the first gesture comprises a selection operation of a first icon in the first GUI, and the first remote control instruction is used for instructing the second terminal to select an icon corresponding to the first icon in the first display interface;

and responding to the first gesture or the third gesture of the user on the touch screen, and displaying a second GUI in the touch screen, wherein the second GUI is consistent with a second display interface output by the second terminal, and the second display interface is output by the second terminal after the first remote control instruction is executed.

The second display interface is an editable input interface, and the second GUI is an input interface comprising a soft keyboard. The interface layout of the soft keyboard in the second GUI is the same as or different from the interface layout of the soft keyboard in the second display interface.

Further, the GUI further comprises: responding to a second gesture input by a user on a soft keyboard of the second GUI, generating a second remote control instruction according to the second gesture, and sending the second remote control instruction to the second terminal; the second gesture comprises a selection operation of a first character on a soft keyboard of the second GUI, and the second remote control instruction is used for instructing the second terminal to input the first character in the second display interface.

The first GUI described in the present embodiment corresponds to the first control interface in the foregoing embodiments, and the second GUI corresponds to the second control interface in the foregoing embodiments. The first gesture and the second gesture in this embodiment correspond to the first touch operation and the second touch operation in the foregoing embodiments, respectively. The third gesture in this embodiment is equivalent to the interface information request operation or the operation of starting photographing in the foregoing embodiments. For details of the GUI, reference may be made to the description in the foregoing embodiments, and details are not repeated here.

The embodiment of the invention also provides a remote control device which is contained in the first terminal and has the function of realizing the behavior of the first terminal in the method embodiments. The functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules or units corresponding to the above functions.

From the above description of the embodiments, it is clear to those skilled in the art that the present invention can be implemented by hardware, firmware, or a combination thereof. When implemented in software, the functions described above may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. Taking this as an example but not limiting: computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Further, any connection is properly termed a computer-readable medium. For example, if software is transmitted from a website, a server, or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a Digital Subscriber Line (DSL), or a wireless technology such as infrared, radio, and microwave, the coaxial cable, the fiber optic cable, the twisted pair, the DSL, or the wireless technology such as infrared, radio, and microwave are included in the fixation of the medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A remote control method applied to a first terminal having a touch screen, the method comprising:

the first terminal displays a first control interface, the first control interface is consistent with a first display interface currently output by a second terminal, and the second terminal is a terminal remotely controlled by the first terminal; the first display interface displays at least two icons, the number of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, the arrangement mode of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, and the icons displayed by the first control interface are in one-to-one correspondence with the icons displayed by the first display interface;

the first terminal receives a first touch operation input on the first control interface, wherein the first touch operation comprises a selection operation of a first icon in the first control interface;

the first terminal generates a first remote control instruction according to the first touch operation;

the first terminal sends the first remote control instruction to the second terminal, wherein the first remote control instruction is used for indicating the second terminal to select the icon corresponding to the first icon in the first display interface;

the first terminal receives an operation for starting photographing;

the first terminal obtains a picture with a second display interface by photographing; the second display interface is output after the second terminal executes the first remote control instruction;

the first terminal identifies content in the picture of the second display interface and displays the identified content as a second control interface, wherein the second control interface is consistent with a second display interface output by the second terminal;

the first terminal saves the second control interface;

when the second terminal is controlled by the first terminal, the second terminal can be simultaneously controlled by other remote controllers matched with the second terminal.

2. The method of claim 1, wherein the first terminal generating a first remote control instruction according to the first touch operation comprises:

the first terminal acquires an initial position of a cursor before the first icon is selected;

and the first terminal generates the first remote control instruction according to the initial position of the cursor and the position of the first icon.

3. The method according to claim 2, wherein the first touch operation further comprises an operation of selecting a second icon corresponding to the initial position of the cursor before the first icon is selected, wherein the second icon is included in the first control interface; the initial cursor position is an initial cursor position on the first display interface.

4. A method according to any of claims 1-3, wherein the second display interface is an editable input interface and the second control interface is an input interface comprising a soft keyboard.

5. The method of claim 4, wherein the interface layout of the soft keyboard in the second control interface is the same as the interface layout of the soft keyboard in the second display interface.

6. The method of claim 4, wherein the interface layout of the soft keyboard in the second control interface is different from the interface layout of the soft keyboard in the second display interface.

7. The method of claim 4, further comprising:

the first terminal receives a second touch operation input on the soft keyboard of the second control interface, wherein the second touch operation comprises a selection operation on a first character on the soft keyboard of the second control interface;

the first terminal generates a second remote control instruction according to the second touch operation;

and the first terminal sends the second remote control instruction to the second terminal, and the second remote control instruction is used for indicating the second terminal to input the first character in the second display interface.

8. The method according to any one of claims 1-3, wherein the second control interface is consistent with a second display interface output by the second terminal, and comprises:

when the second display interface comprises at least two icons, the number of the icons displayed by the second control interface is the same as the number of the icons displayed by the second display interface, the arrangement mode of the icons displayed by the second control interface is the same as the arrangement mode of the icons displayed by the second display interface, and the icons displayed by the second control interface are in one-to-one correspondence with the icons displayed by the second display interface; or

When the second display interface is an editable input interface, the second control interface is an input interface comprising a soft keyboard; or

When the second display interface is an audio and video playing interface, the second control interface comprises any one or more of the following: volume control item, play progress control item, play/stop control item.

9. The method of any of claims 1-3, further comprising:

before the first terminal displays the first control interface, the first terminal displays thumbnails or icons of at least two control interfaces;

the first terminal receives a third touch operation;

and the first terminal selects the first control interface from the at least two control interfaces to display according to the third touch operation.

10. A method according to any one of claims 1 to 3, wherein:

the first display interface and the second display interface are displayed by the second terminal; or

And the first display interface and the second display interface are output to an external device by the second terminal for display.

11. A method according to any one of claims 1 to 3, wherein:

the first terminal sends the remote control instruction to the second terminal through any one of the following communication modes: infrared, Bluetooth, Wi-Fi, and NFC; or

The first terminal is a mobile phone or a tablet computer, and the second terminal is a television or a set-top box; or

The method further comprises the following steps: the first terminal identifies a gesture of a user, converts the gesture into a third remote control instruction and sends the third remote control instruction to the second terminal; or

The method further comprises the following steps: and the first terminal identifies the voice of the user, converts the voice into a fourth remote control instruction and sends the fourth remote control instruction to the second terminal.

12. A first terminal, wherein said first terminal is adapted to remotely control a second terminal, said first terminal comprising:

a touch screen;

one or more processors;

one or more memories having one or more computer programs stored therein, the one or more computer programs comprising instructions, which when executed by the first terminal, cause the first terminal to perform the steps of:

displaying a first control interface through the touch screen, wherein the first control interface is consistent with a first display interface currently output by the second terminal; the first display interface displays at least two icons, the number of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, the arrangement mode of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, and the icons displayed by the first control interface are in one-to-one correspondence with the icons displayed by the first display interface;

receiving a first touch operation input on the first control interface through the touch screen, wherein the first touch operation comprises a selection operation of a first icon in the first control interface;

generating a first remote control instruction according to the first touch operation;

sending the first remote control instruction to the second terminal, wherein the first remote control instruction is used for indicating the second terminal to select an icon corresponding to the first icon in the first display interface;

receiving an operation for starting photographing;

acquiring a picture with a second display interface by photographing; the second display interface is output after the second terminal executes the first remote control instruction;

identifying content in the picture of the second display interface, and displaying the identified content as a second control interface through the touch screen, wherein the second control interface is consistent with a second display interface output by the second terminal;

saving the second control interface;

13. The first terminal according to claim 12, wherein in the step of generating a first remote control instruction according to the first touch operation, the first terminal executes the instruction to cause the first terminal to perform the steps of:

acquiring an initial position of a cursor before the first icon is selected;

and generating the first remote control instruction according to the initial position of the cursor and the position of the first icon.

14. The first terminal according to claim 13, wherein the first touch operation further comprises an operation of selecting a second icon corresponding to the initial position of the cursor before the first icon is selected, the second icon being included in the first control interface; the initial cursor position is an initial cursor position on the first display interface.

15. The first terminal according to any of claims 12-14, wherein the second display interface is an editable input interface and the second control interface is an input interface comprising a soft keyboard.

16. The first terminal of claim 15, wherein the interface layout of the soft keyboard in the second control interface is the same as the interface layout of the soft keyboard in the second display interface.

17. The first terminal of claim 15, wherein an interface layout of the soft keyboard in the second control interface is different from an interface layout of a soft keyboard in the second display interface.

18. The first terminal of claim 15, wherein execution of the instructions by the first terminal causes the first terminal to further perform the steps of:

receiving a second touch operation input on the soft keyboard of the second control interface through the touch screen, wherein the second touch operation comprises a selection operation on a first character on the soft keyboard of the second control interface;

generating a second remote control instruction according to the second touch operation;

and sending the second remote control instruction to the second terminal, wherein the second remote control instruction is used for indicating the second terminal to input the first character in the second display interface.

19. The first terminal of any of claims 12-14, wherein the second control interface is consistent with a second display interface output by the second terminal, comprising:

20. The first terminal of any of claims 12-14, wherein execution of the instructions by the first terminal causes the first terminal to further perform the steps of:

before the first control interface is displayed through the touch screen, displaying thumbnails or icons of at least two control interfaces through the touch screen;

receiving a third touch operation;

and selecting the first control interface from the at least two control interfaces for displaying according to the third touch operation.

21. The first terminal according to any of claims 12-14, characterized by:

22. A first terminal, wherein said first terminal is adapted to remotely control a second terminal, said first terminal comprising:

the display unit is used for displaying a first control interface, the first control interface is consistent with a first display interface currently output by a second terminal, and the second terminal is a terminal remotely controlled by the first terminal; the first display interface displays at least two icons, the number of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, the arrangement mode of the icons displayed by the first control interface is the same as that of the icons displayed by the first display interface, and the icons displayed by the first control interface are in one-to-one correspondence with the icons displayed by the first display interface;

the input unit is used for receiving a first touch operation input on the first control interface, and the first touch operation comprises a selection operation of a first icon in the first control interface;

the control unit is used for generating a first remote control instruction according to the first touch operation received by the input unit;

the sending unit is used for sending the first remote control instruction to the second terminal, wherein the first remote control instruction is used for indicating the second terminal to select the icon corresponding to the first icon in the first display interface;

the input unit is also used for receiving the operation for starting photographing;

the control unit is further used for acquiring a picture with a second display interface by photographing, identifying content in the picture of the second display interface, using the identified content as a second control interface and storing the second control interface; the second display interface is output after the second terminal executes the first remote control instruction;

the display unit is further configured to display the second control interface, where the second control interface is consistent with a second display interface output by the second terminal;

23. The first terminal of claim 22, wherein the second display interface is an editable input interface and the second control interface is an input interface including a soft keyboard.

24. The first terminal of claim 23,

the input unit is further used for receiving a second touch operation input on the soft keyboard of the second control interface, wherein the second touch operation comprises a selection operation on a first character on the soft keyboard of the second control interface;

the control unit is further used for generating a second remote control instruction according to the second touch operation;

the sending unit is further configured to send the second remote control instruction to the second terminal, where the second remote control instruction is used to instruct the second terminal to input the first character in the second display interface.

25. An electronic device, characterized in that the electronic device comprises means for performing the method according to any of claims 1 to 11.

26. A computer-readable storage medium comprising instructions that, when executed on an electronic device, cause the electronic device to perform the method of any of claims 1-11.