WO2023040848A9

WO2023040848A9 - Device control method and apparatus

Info

Publication number: WO2023040848A9
Application number: PCT/CN2022/118543
Authority: WO
Inventors: 刘华军; 杨金华; 杨毅轩
Original assignee: 荣耀终端有限公司
Priority date: 2021-09-16
Filing date: 2022-09-13
Publication date: 2023-05-25
Also published as: CN115814403A; WO2023040848A1

Abstract

The present application relates to the technical field of terminals. Provided are a device control method and apparatus. By means of the device control method provided in the present application, a user only needs to establish a communication connection between a first terminal and a second terminal. The method comprises: a first terminal sending control information to a second terminal; the second terminal enabling a target functional unit in response to the control information; and then, the second terminal executing a first function in cooperation with the first terminal on the basis of the target functional unit. Therefore, the operation steps for a user are simplified, that is, a second terminal can execute a first function in cooperation with a first terminal on the basis of a target functional unit, thereby improving the operation efficiency.

Description

Device control method and device

This application claims the priority of a Chinese patent application with application number 202111089487.1 and application title "equipment control method and device" filed with the State Intellectual Property Office of China on September 16, 2021, the entire contents of which are incorporated herein by reference .

technical field

The present application relates to the technical field of terminals, and in particular to a device control method and device.

Background technique

At present, with the development of terminal technology, terminal equipment has become a part of people's work and life. As the functions of terminal devices become more and more abundant, for more complex functions, a terminal device often needs a third terminal to complete. For example, when the first terminal needs to execute the target function, it needs to use the second terminal to control the first terminal to complete the target function.

Usually, people use the second terminal to control the process of the first terminal to complete the target function, and the operation steps are cumbersome and complicated, and the efficiency is low.

Contents of the invention

The present application provides a device control method and device, which can enable the user to perform simple operations, realize target functions more conveniently by using the second terminal to control the first terminal, and improve the user's operation efficiency.

The present application provides a device control method, which is applied to a communication system. The communication system includes a first terminal and a second terminal. The method includes: the first terminal is in the first state, wherein, when the first terminal is in the first state, it can Execute the first function; the first terminal sends control information to the second terminal based on the established communication connection; the second terminal activates the target function unit in response to the control information; the second terminal cooperates with the first terminal to execute the first function based on the target function unit. Function.

In the device control method provided in this application, the user only needs to establish a communication connection between the first terminal and the second terminal. The first terminal sends control information to the second terminal. The second terminal turns on the target function unit in response to the control information. Further, the second terminal cooperates with the first terminal to execute the first function based on the target function unit. Furthermore, the user's operation steps are simplified, and the second terminal can cooperate with the first terminal to execute the first function based on the target function unit, thereby improving the operation efficiency.

In a possible implementation manner, the second terminal cooperates with the first terminal to execute the first function based on the target functional unit, including: the second terminal sends target data to the first terminal in response to a trigger operation on the target functional unit; A terminal performs a first function associated with target data.

In a possible implementation manner, the target functional unit is a virtual text button in the second interface, and the target data is text data. The first terminal being in the first state includes: the first terminal displays a first interface, where the first interface includes an area where text is to be input. The first terminal executes the first function associated with the target data, including: when the first terminal displays the first interface, inputting text data in an area where text is to be input on the first interface.

In this way, the user needs at most two operations to realize the text data input in the first interface of the first terminal by means of the second terminal. Furthermore, the user's operation steps are simplified, and the operation efficiency is improved.

In a possible implementation manner, the target data is control data. The first terminal being in the first state includes: the first terminal displays a first interface, where the first interface includes game characters. Executing the first function associated with the target data by the first terminal includes: when the first terminal displays the first interface, controlling the game character in the first interface to perform the game behavior corresponding to the control data.

In this way, the user needs at most two operations to control the game character in the first terminal to perform the game behavior associated with the triggering operation by means of the second terminal. Furthermore, the user's operation steps are simplified, and the operation efficiency is improved.

In a possible implementation manner, the target functional unit is a virtual control keyboard in the second interface, where the virtual control keyboard includes virtual keys for controlling the game behavior of the game character. Alternatively, the control data is attitude data, and the target functional unit is an attitude sensor of the second terminal for collecting attitude data.

In a possible implementation manner, the target functional unit is a virtual control keyboard in the second interface, and after the first terminal sends the control information to the second terminal, the method further includes: the second terminal receives the first The first interface data of the interface. The second terminal displays second interface data on the second interface, where the second interface data includes display content corresponding to the first interface data.

In a possible implementation manner, the target functional unit is a video collection module, and the target data is the second video data collected by the video collection module. The first terminal being in the first state includes: the first terminal displays a first interface, where the first interface includes first video data from the third terminal. The first terminal executes the first function associated with the target data, including: the first terminal receives the second video data from the second terminal when the first interface is displayed; the first terminal sends the second video data to the third terminal.

In this way, the user needs at most two operations to implement the function of the first terminal performing video interaction with the help of the second terminal, and the user's hands are freed. Furthermore, the user's operation steps are simplified, and the operation efficiency is improved.

In a possible implementation manner, after the second terminal enables the target functional unit in response to the control information, the method further includes: the second terminal receives the first video data from the first terminal. The second terminal displays a second interface, where the second interface includes the first video data.

In this way, the user can also watch the first video data from the third terminal on the second terminal.

In a possible implementation manner, the method for establishing a communication connection includes: when the first terminal is in the first state, performing device discovery; when the first terminal discovers the second terminal, establishing a communication connection with the second terminal . Alternatively, when the distance between the second terminal and the target electronic device is less than the distance threshold, the device identifier of the first terminal stored in the target electronic device is acquired; the second terminal establishes a communication connection with the first terminal according to the device identifier. Alternatively, the first terminal receives voice information input by the user; wherein, the voice information is used to indicate connection with the second terminal; the first terminal performs device discovery, and the first terminal establishes communication with the second terminal when the second terminal is found connect. Alternatively, the second terminal transmits a UWB signal, the UWB signal carries the device identifier of the second terminal, the first terminal receives the UWB signal from the second terminal, and the first terminal establishes a communication connection with the second terminal according to the device identifier in the UWB signal .

In a possible implementation manner, before the first terminal sends the control information to the second terminal based on the established communication connection, the method further includes: the second terminal displays a fifth interface, the fifth interface includes a First control to exit third preset mode. The second terminal responds to the trigger operation on the first control, and the second terminal is in the third preset mode.

In a possible implementation manner, the control information is information indicating that the first terminal is in the first state, and the second terminal starts the target function unit in response to the control information, including: the second terminal indicates that the first terminal is in the first state. status information, enable the target functional unit. Alternatively, the control information is an instruction for enabling the target functional unit, and the second terminal responds to the control information for enabling the target functional unit, including: the second terminal activates the target functional unit according to the instruction for enabling the target functional unit.

In this way, the target functional unit may be activated by the first terminal decision, or the target functional unit may be activated by the second terminal decision.

In a second aspect, the present application also provides a device control device, which is applied to a first terminal. The device control device provided in the present application includes: a first processing unit, configured to control the first terminal to be in the first state, wherein the first terminal While in the first state, a first function can be performed. The first communication unit is configured to send control information to the second terminal based on the established communication connection. Wherein, the control information is used to instruct the second terminal to activate the target function unit, so as to cooperate with the first terminal to execute the first function.

In a possible implementation manner, the first communication unit is further configured to receive target data from the second terminal. The first processing unit is further configured to execute a first function associated with the target data.

In a possible implementation manner, the target functional unit is a virtual text button in the second interface, and the target data is text data. The first processing unit is specifically configured to control the first terminal to display the first interface, where the first interface includes an area where text is to be input. The first processing unit is further specifically configured to control the first terminal to input text data in an area where text is to be input on the first interface when the first interface is displayed.

In a possible implementation manner, the target data is control data. The first processing unit is specifically configured to control the first terminal to display the first interface, where the first interface includes game characters. The first processing unit is further specifically configured to control the first terminal to control the game character in the first interface to perform the game behavior corresponding to the control data when the first interface is displayed.

In a possible implementation manner, the target functional unit is a video collection module, and the target data is the second video data collected by the video collection module. The first processing unit is specifically configured to control the first terminal to display the first interface, wherein the first interface includes first video data from the third terminal, and the first processing unit is specifically configured to control the first terminal to display the first video data. interface, receiving the second video data from the second terminal; sending the second video data to the third terminal.

In a third aspect, the present application further provides a device control device, which is applied to a second terminal. The device control device provided in the third aspect includes: a second communication unit, configured to receive a communication from the first terminal based on an established communication connection control information. The second processing unit is configured to enable the target function unit in response to the control information. The second processing unit is further configured to cooperate with the first terminal to execute the first function based on the target function unit.

In a possible implementation manner, the second processing unit is specifically configured to, in response to a trigger operation on the target functional unit, control the second communication unit to send target data to the first terminal; where the target data is used to indicate that the first terminal A first function associated with the target data is performed.

In a possible implementation manner, the target functional unit is a virtual text button in the second interface, and the target data is text data, and the text data is used to indicate that when the first terminal displays the first interface, the key to be input in the first interface Text field to enter text data.

In a possible implementation manner, the target data is control data, and the target data is used to instruct the first terminal to control the game character in the first interface to perform the game behavior corresponding to the control data when the first interface is displayed.

In a possible implementation, the target function unit is a virtual control keyboard in the second interface, wherein the virtual control keyboard includes virtual keys for controlling the game behavior of the game character; or, the control data is gesture data, and the target function The unit is an attitude sensor of the second terminal for collecting attitude data.

In a possible implementation manner, the target functional unit is a video collection module, and the target data is second video data collected by the video collection module, and the second video data is used to instruct the first terminal to display the first video data including the first video data. interface, sending the second video data to the third terminal.

In a possible implementation manner, the control information is information indicating that the first terminal is in the first state, and the second processing unit is specifically configured to enable the target functional unit according to the information indicating that the first terminal is in the first state. Alternatively, the control information is an instruction for enabling the target functional unit, and the second processing unit is specifically configured to enable the target functional unit according to the instruction for enabling the target functional unit.

In the fourth aspect, the present application also provides a computer-readable storage medium, the computer-readable storage medium stores instructions, and when the instructions are executed, the computer executes the computer as described in the first aspect or any implementation manner of the first aspect. The method executed by the first terminal or the second terminal.

In the fifth aspect, the present application further provides a computer program product, including a computer program, which, when the computer program is run, causes the computer to execute the first terminal or the first terminal described in any implementation manner of the first aspect or the first aspect. The method executed by the terminal.

It should be understood that the second aspect to the fifth aspect of the present application correspond to the technical solution of the first aspect of the present application, and the advantageous effects obtained by each aspect and the corresponding feasible implementation manners are similar, so details are not repeated here.

Description of drawings

FIG. 1-FIG. 2 are schematic diagrams of the interface for inputting text data in the first interface displayed by the mobile phone 200 to the smart TV 200;

FIG. 3 is a schematic diagram of a hardware system architecture of a terminal device provided in an embodiment of the present application;

FIG. 4 is a schematic diagram of a software system architecture of a terminal device provided in an embodiment of the present application;

5 is a schematic diagram of obtaining the device identity of the smart TV 100 by the mobile phone 200 approaching the remote control 300 of the smart TV 100 provided by the embodiment of the present application;

FIG. 6 is a schematic diagram of the mobile phone 200 receiving the voice information "connect to TV" input by the user provided by the embodiment of the present application;

FIG. 7 is a schematic diagram of establishing a communication connection between the mobile phone 200 and the smart TV 100 provided by the embodiment of the present application;

FIG. 8 is a schematic diagram of an interface in which the smart TV 100 sends the interface identifier of the first interface 101 to the mobile phone 200 during the process of displaying the first interface 101 provided by the embodiment of the present application;

Fig. 9 shows that in the process of displaying the second interface 207 of the mobile phone 200 provided by the embodiment of the present application, in response to the user's trigger operation on the virtual text button in the second interface 207, input in the first interface 101 of the smart TV 100 Schematic diagram of account and password;

FIG. 10 is a schematic diagram of an interface for establishing a communication connection with a mobile phone 100 when the smart TV 100 provided in the embodiment of the present application displays a game interface;

FIG. 11 is a schematic diagram of an interface of the smart TV 100 sending the interface identifier of the game interface to the mobile phone 200 during the process of displaying the game interface provided by the embodiment of the present application;

FIG. 12 shows that the mobile phone 200 provided by the embodiment of the present application responds to the user's trigger operation on the light-hand attack button "A" in the second interface 207, and controls the game character A in the first interface 101 of the smart TV 100 to control the light-hand attack game. Interface diagram of character B;

FIG. 13 is a schematic diagram of an interface for establishing a communication connection with a smart TV 200 when the mobile phone 200 displays the video playback interface provided by the embodiment of the present application;

FIG. 14 is a schematic diagram of the interface of the mobile phone 200 implementing the large-screen video interaction function by means of the software and hardware of the smart TV 100 provided by the embodiment of the present application;

FIG. 15 is a schematic diagram of an interface for setting the mobile phone 200 in the third preset mode provided by the embodiment of the present application;

FIG. 16 is one of the schematic flowcharts of the device control method provided by the embodiment of the present application;

FIG. 17 is a schematic diagram of a functional architecture in which the first terminal executes the function associated with the target data by means of the second terminal provided by the embodiment of the present application;

FIG. 18 is a structural block diagram of a terminal device 1800 provided in an embodiment of the present application;

FIG. 19 is a structural block diagram of a terminal device 1900 provided in an embodiment of the present application;

FIG. 20 is a schematic diagram of a hardware structure of a terminal device provided in an embodiment of the present application;

FIG. 21 is a schematic structural diagram of a chip provided by an embodiment of the present application.

Detailed ways

In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with basically the same function and effect. For example, the first value and the second value are only used to distinguish different values, and their sequence is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

It should be noted that, in this application, words such as "exemplary" or "for example" are used as examples, illustrations or illustrations. Any embodiment or design described herein as "exemplary" or "for example" is not to be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete manner.

In this application, "at least one" means one or more, and "multiple" means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B, which can mean: A exists alone, A and B exist at the same time, and B exists alone, where A, B can be singular or plural. The character "/" generally indicates that the contextual objects are an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one item (piece) of a, b, or c can represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, c can be single or multiple .

As the functions of terminal devices become more and more abundant, for more complicated functions, the first terminal often needs the second terminal to complete them more conveniently. Exemplarily, taking the first terminal as the smart TV 100 and the second terminal as the mobile phone 200 as an example, with reference to FIGS. 1-2 , how the smart TV 100 can more conveniently complete a certain function with the help of the mobile phone 200 is described.

As shown in FIG. 1 , the smart TV 100 displays a first interface 101, which includes an account input area A and a password input area B, and the user needs to input an account in the account input area A and a password in the password input area B. , to complete the login and use the smart TV 100. For example, the user may use the mobile phone 200 to perform at least the following steps to enter the account and password on the first interface 101 .

Step 1: As shown in FIG. 1 , the user can establish a communication connection between the mobile phone 200 and the smart TV 100 .

Step 2: As shown in (a)-(b) in FIG. 2 , the mobile phone 200 displays the system main interface 201 in response to the user's unlocking operation.

Step 3: As shown in (b) in Figure 2, the main interface 201 of the display system includes the icon 202 of the "cooperative" application, and the mobile phone 200 responds to the user's trigger operation on the icon 202 of the "cooperative" application, as shown in Figure 2 As shown in (c), the main interface 203 of the "collaboration" application is displayed.

Step 4: As shown in (c) of FIG. 2 , the main interface 203 of the "cooperation" application includes a control 204 for instructing to realize the coordination function. The mobile phone 200 may display the function list interface 205 in response to the user's trigger operation on the control 204 . The “text input application”, “handle application” and “game application” are displayed in the function list interface 205 , and switch controls 206 are displayed on one side of the “text input application”, “handle application” and “game application”.

Step 5: As shown in (c)-(d) in FIG. 2 , the mobile phone 200 may display a second interface 207 in response to the user's trigger operation on the switch control 206 on the "text input application" side, the second interface 207 includes virtual keys. The mobile phone 200 can control the smart TV 100 to input an account in the account input area A and a password in the password input area B in response to the user's trigger operation on the virtual key, so as to complete the login.

However, the above smart TV 100 uses the mobile phone 200 to complete the function of inputting the account and password, at least requiring the user to operate the above steps 1-5, the operation steps are complicated and inefficient.

In view of this, an embodiment of the present application provides a device control method. When the first terminal displays the first interface and needs to use the second terminal to complete a certain function, a communication connection can be established between the first terminal and the second terminal; and , the first terminal sends the identifier of the first interface to the second terminal; the second terminal recognizes the identifier of the first interface, and displays the second interface corresponding to the identifier of the first interface; during the process of displaying the second interface, the second terminal, The target data is acquired; the second terminal sends the target data to the first terminal; the first terminal executes a function associated with the target data. In this way, the user's operation steps are reduced, and the process in which the first terminal completes the target function by means of the second terminal can be realized, thereby improving efficiency.

It can be understood that both the foregoing first terminal and the second terminal may be referred to as terminal devices. The terminal device can be mobile phone, smart TV, wearable device, tablet computer (Pad), computer with wireless transceiver function, virtual reality (virtual reality, VR) terminal device, augmented reality (augmented reality, AR) terminal Equipment, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grid, transportation Wireless terminals in transportation safety, wireless terminals in smart city, wireless terminals in smart home, etc. The embodiment of the present application does not limit the specific technology and specific device form adopted by the terminal device.

In order to better understand the embodiment of the present application, the structure of the terminal device in the embodiment of the present application is introduced below. Exemplarily, FIG. 3 is a schematic structural diagram of a terminal device provided in an embodiment of the present application.

The terminal device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, an antenna 1, an antenna 2, and a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a sensor module 180, a button 190, an indicator 192, a camera 193, and a display screen 194, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc. Wherein, the combination of the gyro sensor 180B and the acceleration sensor 180E can be understood as an attitude sensor.

It can be understood that, the structure shown in the embodiment of the present application does not constitute a specific limitation on the terminal device. In other embodiments of the present application, the terminal device may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

Processor 110 may include one or more processing units. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. A memory may also be provided in the processor 110 for storing instructions and data.

The USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the terminal device, and can also be used to transmit data between the terminal device and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.

The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger. The power management module 141 is used for connecting the charging management module 140 and the processor 110 .

The wireless communication function of the terminal device can be realized by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Antennas in end devices can be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas.

The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on terminal equipment. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation.

The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system ( global navigation satellite system (GNSS), frequency modulation (frequency modulation, FM) and other wireless communication solutions.

The terminal device realizes the display function through the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. In some embodiments, the terminal device may include 1 or N display screens 194, where N is a positive integer greater than 1.

The terminal device can realize the shooting function through ISP, camera 193 , video codec, GPU, display screen 194 and application processor.

Camera 193 is used to capture still images or video. In some embodiments, the terminal device may include 1 or N cameras 193, where N is a positive integer greater than 1.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. Such as saving music, video and other files in the external memory card.

The internal memory 121 may be used to store computer-executable program codes including instructions. The internal memory 121 may include an area for storing programs and an area for storing data. Exemplarily, the internal memory 121 can be used to store the first object table and the second object table. For the specific content of the first target table, refer to Table 1 in the following embodiments, and for the specific content of the second target table, refer to Table 2 in the following embodiments, which will not be introduced here.

The terminal device can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals. The terminal device can play music or listen to voice through the speaker 170A. The receiver 170B, also called "earpiece", is used to convert audio electrical signals into audio signals. When the terminal device answers a phone call or voice information, the receiver 170B can be placed close to the human ear to listen to the voice. The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals, such as collecting voice.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . The gyroscope sensor 180B can be used to determine the motion posture of the terminal device. The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall sensor. The acceleration sensor 180E can detect the acceleration of the terminal device in various directions (generally three axes). The distance sensor 180F is used to measure the distance. Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The ambient light sensor 180L is used for sensing ambient light brightness. The fingerprint sensor 180H is used to collect fingerprints. The temperature sensor 180J is used to detect temperature. The touch sensor 180K is also called "touch device". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The bone conduction sensor 180M can acquire vibration signals.

The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The terminal device can receive key input and generate key signal input related to user settings and function control of the terminal device. The indicator 192 can be an indicator light, and can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications, and the like.

The software system of the terminal device may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture, etc., which will not be repeated here.

The embodiment of the present application takes the Android system with a layered architecture as an example to illustrate the software structure of the terminal device. FIG. 4 is a software structural block diagram of a terminal device applicable to an embodiment of the present application. The layered architecture divides the software system of the terminal device into several layers, and each layer has a clear role and division of labor. Layers communicate through software interfaces. In some embodiments, the Android system can be divided into five layers, namely applications, application framework, Android runtime, system library, and hardware abstraction layer. layer, HAL) and the kernel layer (kernel).

The application program layer may include a series of application program packages, and the application program layer runs the application program by calling an application program interface (application programming interface, API) provided by the application program framework layer. As shown in Figure 3, the application package can include applications such as camera, text input, game, handle, collaborative control, WeChat, phone, map, navigation, WLAN, and Bluetooth information. Wherein, the display interface when the text input application is opened includes a virtual text key, and the virtual text key is used for inputting text. Exemplarily, the virtual text keys include number keys, letter keys, and symbol keys, etc., which are not limited herein. When the game application is opened, the display interface is the game interface. For example, the game interface may include game character A and game character B in a fighting game. The display interface when the handle application is opened may include a virtual control keyboard. The virtual control keyboard can include forward button "←", back button "→", jump button "↑", squat button "↓", light hand attack button "A", heavy hand attack button "B", light foot attack button "C" and the heavy foot attack button "D". The coordinated control application can be used to call application programs such as text input application, joystick application, and video application. In addition, the coordinated control application can also be used to call hardware modules such as the acceleration sensor 180E, the gyroscope sensor 180B, the microphone 170A, and the speaker 170C.

The application framework layer provides API and programming framework for applications in the application layer. The application framework layer includes some predefined functions. As shown in Figure 4, the application framework layer can include window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. Content providers are used to store and retrieve data and make it accessible to applications. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. The view system can be used to build applications. A display interface can consist of one or more views.

The Android runtime includes core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core library consists of two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android. The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application program layer and the application program framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection. A system library can include multiple function modules.

The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of various commonly used audio and video formats, as well as still image files, etc.

The hardware abstraction layer can contain multiple library modules, such as camera library modules, motor library modules, etc. The Android system can load corresponding library modules for the device hardware, and then realize the purpose of the application framework layer accessing the device hardware. Device hardware may include, for example, speakers, display screens, and cameras in terminal devices.

The kernel layer is the layer between hardware and software. The kernel layer is used to drive the hardware and make the hardware work. The kernel layer includes at least a display driver, a motor driver, a sensor driver, etc., which are not limited in this embodiment of the present application.

The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. The following specific embodiments may be implemented independently, or may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

In the following, taking the first terminal as the smart TV 100 and the second terminal as the mobile phone 200 as an example, the device control method provided by the embodiment of the present application will be described in conjunction with the schematic interface diagrams in Figures 5-15. Example limitations. The following embodiments can be combined with each other, and the same or similar concepts or processes will not be repeated.

As shown in FIG. 5 or FIG. 6, when the user turns on the smart TV 100, the smart TV 100 displays a first interface 101 (that is, the smart TV 100 is in a first state). The first interface 101 includes an account input area A where text is to be input, and a password input area B (ie, the area where text is to be input) (ie, the smart TV 100 can realize the function of text input, ie, the first function). Wherein, the first interface 101 can be understood as a text input interface. In this way, the user needs to use the mobile phone 200 to enter the account and password in the first interface 101 (that is, the smart TV 100 needs to use the mobile phone 200 to input text data), and the user can use the smart TV 100 only after the smart TV 100 successfully verifies the account and password. For example, the user uses the smart TV 100 to watch live channels, order movies, and play games.

Before the user enters the account and password on the first interface 101 with the help of the mobile phone 200 , the smart TV 100 needs to establish a communication connection with the mobile phone 200 . Exemplarily, the ways for the mobile phone 200 to communicate with the smart TV 100 may include but not limited to the following ways:

The first type: the smart TV 100 performs device discovery during the process of displaying the first interface 101 . When the smart TV 100 discovers the mobile phone 200 , it establishes a communication connection with the mobile phone 200 . Wherein, the communication connection may be a Bluetooth connection, a WiFi connection, an NFC connection, UWB, etc., which is not limited herein. In this way, the communication connection between the smart TV 100 and the mobile phone 200 can be realized without user operation, which can reduce user operation steps and improve efficiency.

The second type: as shown in FIG. 5 , the mobile phone 200 is close to the remote control 300 of the smart TV 100 (for example, the user uses the mobile phone 200 to touch the remote control 300 once), and the mobile phone 200 can establish an NFC communication connection with the remote control 300 . In this way, the mobile phone 200 can obtain the device identifier of the smart TV 100 stored in the remote controller 300 through NFC communication (which may include the communication address of the smart TV 100 ). Furthermore, as shown in FIG. 7 , the mobile phone 200 can establish a communication connection with the smart TV 100 according to the device identifier. In this way, the user only needs to bring the mobile phone 200 close to the remote control 300 to establish a communication connection between the mobile phone 200 and the smart TV 100 , which is convenient and fast.

In addition, in order to prevent the mobile phone 200 from establishing a communication connection with the smart TV 100 when the mobile phone 200 contacts the remote controller 300 by mistake. The mobile phone 200 may establish a communication connection with the smart TV 100 only when the mobile phone 200 establishes NFC communication with the remote control 300 twice (that is, the user uses the mobile phone 200 to touch the remote control 300 twice).

Alternatively, the aforementioned remote controller 300 may be replaced with a router, or a smart TV 100 , etc., which is not limited herein. It can be understood that the aforementioned remote control 300 , router, or smart TV 100 are target electronic devices.

The third type: as shown in FIG. 6 , the mobile phone 200 can receive the voice message "connect with TV" input by the user. Furthermore, as still shown in FIG. 7 , the mobile phone 200 discovers the smart TV 100 and establishes a communication connection with the smart TV 100 . Understandably, the above voice information is used to indicate a communication connection with the smart TV 100 . In this way, the above-mentioned voice information may also be replaced with "connect TV", etc., which is not limited here.

The fourth type: the mobile phone 200 can transmit UWB signals. Wherein, the UWB signal carries the device identification of the mobile phone 200 (may include the communication address of the mobile phone 200). When the antenna of the mobile phone 200 is aimed at the smart TV 100 , the smart TV 100 can receive the device identification of the mobile phone 200 . Furthermore, the smart TV 100 establishes a communication connection with the mobile phone 200 according to the device identifier. In this way, the user only needs to align the mobile phone 200 with the smart TV 100 to establish a communication connection between the mobile phone 200 and the smart TV 100 , which is convenient and quick.

After the mobile phone 200 is successfully connected to the smart TV 100 , the smart TV 100 controls the mobile phone 200 to display the second interface during the process of displaying the first interface 101 . Wherein, the second interface includes a virtual text button, and the virtual text button is used for inputting text. Exemplarily, the virtual text keys include number keys, letter keys, and symbol keys, etc., which are not limited herein. Understandably, the above-mentioned second interface is a keyboard interface. In the following, how the smart TV 100 controls the mobile phone 200 to display the second interface will be described in conjunction with Table 1 and Table 2 respectively.

Exemplarily, the mobile phone 200 may store a first target table, and the first target table includes a mapping relationship between the interface identifier of the smart TV 100 and the interface identifier of the mobile phone 200 . Wherein, the contents of the first target table may be shown in Table 1 below:

智能电视100的界面标识Interface logo of smart TV 100	手机200的界面标识The interface identification of the mobile phone 200
文本输入界面的界面标识Interface ID of the text input interface	键盘界面的界面标识Interface ID for the keyboard interface
游戏界面的界面标识Interface ID of the game interface	手柄界面的界面标识Interface ID of the handle interface
视频播放界面的界面标识Interface ID of the video playback interface	视频播放界面的界面标识Interface ID of the video playback interface

Table 1

After the mobile phone 200 is successfully connected with the smart TV 100, as shown in FIG. , that is, control information). The mobile phone 200 searches out the interface identifier of the second interface 207 (ie, the interface identifier of the keyboard interface) from the first target table according to the interface identifier of the first interface 101 . Furthermore, the mobile phone 200 displays the second interface 207 (ie, the keyboard interface) according to the found interface identifier.

In addition, the above mapping relationship can also be replaced by: the mapping relationship between the interface identifier of the smart TV 100 and the program package name of the mobile phone 200 . Exemplarily, the content of the first target table may also be shown in Table 2 below:

智能电视100的界面标识Interface logo of smart TV 100	手机200的程序包名The program package name of the mobile phone 200
文本输入界面的界面标识Interface ID of the text input interface	键盘应用的程序包名The package name of the keyboard application
游戏界面的界面标识Interface ID of the game interface	手柄应用的程序包名The package name of the controller application
视频播放界面的界面标识Interface ID of the video playback interface	视频应用的程序包名The package name of the video application

Table 2

Based on Table 2, the mobile phone 200 may also find out the program package name of the keyboard application from the first target table according to the interface identifier of the first interface 101 . Furthermore, the mobile phone 200 opens the keyboard application to display the second interface 207 according to the found program package name of the keyboard application.

As shown in FIG. 9 , in the process of displaying the second interface 207, the mobile phone 200 may respond to the user's trigger operation on the virtual text button in the second interface 207, and enter the account and password. Among them, the ways for the mobile phone 200 to input the account and password on the first interface 101 include but not limited to the following two:

The first type: the virtual text button is marked with a button identifier. Wherein, the button identification can be a number "0" or "1", etc., an English letter "a" or "b" and so on. The mobile phone 200 may, in response to the user's trigger operation on the virtual text key, send the key identification (ie target data) of the triggered virtual text key to the smart TV 100 . The smart TV 100 inputs the received button identification on the first interface 101 .

The second type: a button identifier is marked on the virtual text button. Wherein, the button identification may be a number "0" or "1", and an English letter "a" or "b". There may be a mapping relationship between key identifiers and key codes. Exemplarily, both the smart TV 100 and the mobile phone 200 store a second target table, and the second target table includes a mapping relationship between key identifiers and key codes. Wherein, the content of the second target table may be shown in Table 3 below:

按键标识button identification	第二键控代码Second key code
按键标识“0”Button ID "0"	4848
按键标识“1”Button ID "1"	4949
按键标识“2”Button ID "2"	5050
按键标识“3”Button identification "3"	5151
按键标识“4”Button identification "4"	5252
按键标识“5”Button identification "5"	5353
按键标识“6”Button identification "6"	5454
按键标识“7”Button identification "7"	5555
按键标识“8”Button identification "8"	5656
按键标识“9”Button identification "9"	5757
按键标识“Enter”Button identification "Enter"	1313
按键标识“A”Button identification "A"	6565
按键标识“B”Button identification "B"	6666
按键标识“C”Button identification "C"	6767
......	......
按键标识“Z”Button identification "Z"	9090

table 3

The mobile phone 200 may, in response to the user's trigger operation on the virtual text key, search out the key code corresponding to the key identifier of the triggered virtual text key from the second target table. The mobile phone 200 sends the found key code to the smart TV 100 . The smart TV 100 finds the keyboard identifier corresponding to the key code from the second target table, and inputs the found key identifier on the first interface 101 .

To sum up, in the above embodiments, the user only needs to establish a communication connection between the smart TV 100 and the mobile phone 200 .

In this way, the smart TV 100 sends the identifier of the first interface to the mobile phone 200 . After the mobile phone 200 recognizes the first interface identifier, it displays the second interface corresponding to the first interface identifier. During the process of displaying the second interface, the mobile phone 200 receives a user's trigger operation; the smart TV 100 inputs the account and password corresponding to the trigger operation on the first interface. In this way, the user needs at most two operations to realize the account and password entered in the first interface of the smart TV 100 by means of the mobile phone 200 . Furthermore, the user's operation steps are simplified, and the operation efficiency is improved.

It should be noted that, in the above embodiment, the first interface 101 of the smart TV 100 is used as the text input interface, and the user enters the account and password on the text input interface of the smart TV 100 by means of the mobile phone 200 as an example. Hereinafter, taking the first interface 101 of the smart TV 100 as the game interface as an example, how the user controls the game character in the game interface of the smart TV 100 to execute the game behavior with the help of the mobile phone 200 .

As shown in FIG. 10 , when the user uses the smart TV 100 to play a fighting game, the smart TV 100 displays a first interface 101 (that is, the smart TV 100 is in a first state). The first interface 101 includes game character A and game character B (that is, the smart TV 100 has a function of controlling the behavior of the game character, that is, the first function). Understandably, the first interface 101 is a game interface. In addition, the mobile phone 200 displays the system main interface 201 at this moment. Of course, the main interface 201 of the system displayed on the mobile phone 200 can also be replaced by the mobile phone 200 being in a locked screen state or displaying other browsing interfaces, etc., and this is just an example.

Still as shown in FIG. 10 , if the user wants to control the game behavior of the game character A through the mobile phone 200 , the smart TV 100 needs to establish a communication connection with the mobile phone 200 . Exemplarily, the manner of the communication connection between the mobile phone 200 and the smart TV 100 may refer to the description in the foregoing embodiments, which will not be repeated here.

After the mobile phone 200 establishes a communication connection with the smart TV 100 , the smart TV 100 controls the mobile phone 200 to display the second interface 207 during the process of displaying the first interface 101 .

Exemplarily, as shown in (a)-(b) in FIG. , that is, control information). The mobile phone 200 displays the second interface 207 according to the interface identifier of the first interface 101 . Wherein, the principle of the mobile phone 200 displaying the second interface 207 according to the interface identifier of the first interface 101 is the same as that in the above-mentioned embodiment, and will not be repeated here.

Exemplarily, as shown in (b) of FIG. 11 , the second interface 207 may include a virtual control keyboard 208 . Wherein, the virtual control keyboard 208 includes virtual keys for controlling the game behavior of the game character A. For example, the virtual control keyboard 208 may include a forward key "←", a backward key "→", a jump key "↑", a squat key "↓", a light hand attack key "A", a heavy hand attack key "B", a light hand Foot attack button "C" and heavy foot attack button "D".

In addition, as shown in (b) in Figure 11, the mobile phone 200 can also receive the interface data displayed on the first interface 101 from the smart TV 100, and display the interface data of the first interface 101 on the second interface 207, and then , the user can view the interface data of the first interface 101 in the smart TV 100 at close range on the mobile phone 200 .

It can be understood that when the mobile phone 200 is displaying the second interface 207 , the mobile phone 200 can control the game behavior of the game character A in the first interface 101 in response to a user's trigger operation.

As shown in (a) in Figure 12, the mobile phone 200 can control the game character A in the first interface 101 of the smart TV 100 to lightly respond to the user's trigger operation on the light hand attack button "A" in the second interface 207. Hand attack game character B. Among them, the principle that the user controls the game character A in the first interface 101 of the smart TV 100 to attack the game character B with the help of the mobile phone 200 is the same as in the above-mentioned embodiment, the user uses the mobile phone 200 to control the game character A in the first interface 101 of the smart TV 100. The principle of entering the account and password is the same, and will not be repeated here. It can be understood that the mobile phone 200 can also control the game character A in the first interface 101 of the smart TV 100 to perform other operations based on the same principle, which will not be limited or repeated here.

In addition, (b) in the above-mentioned FIG. 11 may also be replaced with (c) in the above-mentioned FIG. 11 . As shown in (c) of FIG. 11 , the mobile phone 200 displays a second interface 211 . The second interface 211 includes a virtual control keyboard 212, wherein, when the virtual control keyboard 211 is triggered, control data (i.e. target data) on the game character is generated, wherein the control data is used to control the game character A in the first interface 211 the behavior of. The virtual control keyboard 212 may include a forward key "←", a backward key "→", a jump key "↑", a squat key "↓", a light hand attack key "A", a heavy hand attack key "B", a light foot attack key Button "C" and heavy foot attack button "D". It can be seen that the second interface 211 is the handle interface. The mobile phone 200 can also control the game character A in the first interface 101 of the smart TV 100 to lightly attack the game character B in response to the user's trigger operation on the light-hand attack button "A" in the second interface 211 .

In addition, (b) in the above-mentioned FIG. 11 may also be replaced with (d) in the above-mentioned FIG. 11 . As shown in (d) of FIG. 11 , the mobile phone 200 displays a third interface 213 . The first prompt information is displayed on the third interface 213 . The first prompt information is used to indicate that the mobile phone 200 is in the first preset mode, where the first preset mode can be understood as the mobile phone 200 turning on the game mode. Exemplarily, the first prompt information may be but not limited to text information of "in game mode". In addition, in addition to displaying the third interface 213 , the mobile phone 200 also finds out the attitude sensor according to the logo of the first interface 101 of the smart TV 100 and the preset mapping relationship. In this way, the mobile phone 200 can share the attitude data of the mobile phone 200 collected by the attitude sensor to the smart TV 100 . After receiving the gesture data, the smart TV 100 can control the game character A to perform operations associated with the gesture data.

Exemplarily, when the mobile phone 200 moves to the right, it sends right gesture data (ie target data) to the smart TV 100 . In this way, the smart TV 100 can control the game character A to move to the right according to the gesture data of the movement to the right. For another example, when the mobile phone 200 moves downward, it sends downward posture data to the smart TV 100 . In this way, the smart TV 100 can control the game character A to squat according to the downward posture data. For another example, when the mobile phone 200 is turned forward, it sends the attitude data of the forward turn to the smart TV 100 . In this way, the smart TV 100 can control the game character A to attack the game character B with a light hand according to the posture data flipped forward.

In this way, the smart TV 100 sends the identifier of the first interface to the mobile phone 200 . After recognizing the identifier of the first interface, the mobile phone 200 displays the second interface corresponding to the identifier of the first interface. During the process of displaying the second interface, the mobile phone 200 receives the user's trigger operation, and controls the game character in the smart TV 100 to perform the game behavior associated with the trigger operation. In this way, the user needs at most two operations to control the game character in the smart TV 100 to perform the game behavior associated with the triggering operation by means of the mobile phone 200 . Furthermore, the user's operation steps are simplified, and the operation efficiency is improved.

It should be noted that, in the above two embodiments, the smart TV 100 uses the software and/or hardware of the mobile phone 200 to implement a certain function. For example, the smart TV 100 uses the mobile phone 200 to input the account and password; or, the smart TV 100 uses the mobile phone 200 to control the game character to execute the game behavior. In the following, taking the first terminal as the mobile phone 200 and the second terminal as the smart TV 100 as an example, it is explained that the mobile phone 200 implements the large-screen video interactive function by means of the software and hardware of the smart TV 100 .

When the user is using the smart TV 100 to watch a TV program, the user's mobile phone 200 receives a video call request from the third terminal. As shown in FIG. 13 , the mobile phone 200 receives the first video data from the third terminal and collects the second video data in response to the confirmation operation of the video call request input by the user. The first video data includes the image D collected by the third terminal and first sound data. The second video data includes the first sound data collected by the microphone of the mobile phone 200 and the image C collected by the camera of the mobile phone 200 . In this way, the mobile phone 200 displays the first interface 209 (i.e. the video playback interface, and the mobile phone 200 is in the first state), wherein the first interface 209 includes image C and image D, and the mobile phone 200 plays the first sound data on the speaker of the mobile phone 200 (That is, the mobile phone 200 has a function of video interaction, that is, the first function). The mobile phone 200 sends the second video data to the third terminal, so that the third terminal can play the second video data. In this way, video interaction between the mobile phone 200 and the third terminal can be realized.

If the user does not want to hold the mobile phone 200 to collect the second video data during video interaction (that is, the user wants to free his hands), the mobile phone 200 can use the smart TV 100 to collect the second video data. Optionally, if the user wants to view image C and image D more clearly, the mobile phone 200 may use the smart TV 100 to display image C and image D. 13 and 14, how the mobile phone 200 collects the second video data with the smart TV 100, and how the mobile phone 200 displays the image C and the image D with the smart TV 100.

As shown in FIG. 13 , the mobile phone 200 establishes a communication connection with the smart TV 100 during the process of displaying the first interface 209 . Exemplarily, the manner of the communication connection between the mobile phone 200 and the smart TV 100 may refer to the description in the foregoing embodiments, which will not be repeated here.

After the mobile phone 200 establishes a communication connection with the smart TV 100 , the mobile phone 200 controls the smart TV 100 to display the second interface 103 during the process of displaying the first interface 209 . Wherein, image C and image D are also included in the second interface 103 . Next, the process of how the mobile phone 200 controls the smart TV 100 to display the second interface 103 will be specifically introduced.

As shown in (a)-(b) in Fig. 14, in the process of displaying the first interface 209, the mobile phone 200 sends the interface identification of the first interface 209 to the smart TV 100 (i.e. the interface identification of the video playback interface, i.e. the control information). The smart TV 100 finds the identifier of the second interface 103 and the identifier of the speaker according to the interface identifier of the first interface 209 and the preset mapping relationship. The smart TV 100 receives the first video data from the mobile phone 200, and the first video data includes the image D collected by the third terminal and the first sound data. In this way, the smart TV 100 displays the second interface 103 (namely, the video playback interface), wherein the second interface 103 includes the image D, and the speaker of the smart TV 100 plays the first sound data. Generally, the size of the image D displayed on the smart TV 100 is larger than the size of the image D displayed on the mobile phone 200 . In this way, the user can watch the image D on the smart TV 100 more clearly.

In addition, optionally, as shown in (d) of FIG. 14 , the mobile phone 200 may also display a fourth interface 210 after sending the first video data to the smart TV 100 . Second prompt information is displayed on the fourth interface 210 . The second prompt information is used to indicate that the mobile phone 200 is in a second preset mode (such as video mode). Exemplarily, the first prompt information may be, but not limited to, text information of "in video call".

In addition, the smart TV 100 also finds the identifier of the microphone and the identifier of the camera according to the interface identifier of the first interface 209 and the preset mapping relationship. When the smart TV 100 displays the second interface 103 , the camera of the smart TV 100 collects the image C, and the microphone of the smart TV 100 collects the second sound data. Furthermore, as shown in (c)-(d) in FIG. 14 , the smart TV 100 also sends second video data (ie target data) to the mobile phone 200 , the second video data includes image C and second sound data. Furthermore, the mobile phone 200 sends the second video data to the third terminal, so that the third terminal can play the second video data. In this way, the mobile phone 200 implements video interaction with the third terminal through the smart TV 100 . Since the second video data is collected by the smart TV 100, the user does not need to hold the mobile phone 200 to collect the second video data, which frees the user's hands.

In this way, the mobile phone 200 sends the identifier of the video playback interface of the smart TV 100 to the smart TV 100 . After the smart TV 100 recognizes the first interface identifier, it displays the video playback interface corresponding to the identifier of the video playback interface, and the speaker of the smart TV 100 plays the first sound data from the mobile phone 200 . During the process of displaying the video playing interface, the smart TV 100 collects the second video data. The smart TV 100 sends the second video data to the mobile phone 200 . The mobile phone 200 sends the second video data to the third terminal, so that the third terminal can play the second video data. In this way, the user needs at most two operations to realize the function of the mobile phone 200 controlling the smart TV 100 to perform large-screen video interaction, and frees the user's hands. Furthermore, the user's operation steps are simplified, and the operation efficiency is improved.

It should be noted that, in the above three embodiments, in order to further prevent the mobile phone 200 and the smart TV 100 from establishing a communication connection by mistake. The mobile phone 200 needs to be in the third preset mode before it can establish a communication connection with the smart TV 100 . Exemplarily, the user can trigger the mobile phone 200 to be in the third preset mode. Wherein, the third preset mode can be understood as a sharing mode, a collaborative mode, and the like. As shown in (a)-(b) in FIG. 15 , the mobile phone 200 can display a function list interface 212 (that is, the fifth interface) in response to the user's trigger operation on the "settings" icon 211 in the system main interface 201. The function list interface 212 includes a first control 213 for controlling entering/exiting the third preset mode. The mobile phone 200 may be in the third preset mode in response to the trigger operation on the first control 213 . In the third preset mode, the mobile phone 200 can establish a communication connection with the smart TV 100 .

It can be understood that, in the above introduction in conjunction with FIG. 5 to FIG. 15 , the embodiment of the present application is introduced in the manner of a schematic interface diagram. In the following, the device control method provided by the embodiment of the present application will be described with reference to the schematic flowchart provided in FIG. 16 . It should be noted that the basic principles and technical effects of the equipment control method provided in Figure 16 are the same as those of the above-mentioned embodiments. Corresponding content.

FIG. 16 is one of the schematic flowcharts of the device control method provided by the embodiment of the present application. As shown in Figure 16, the device control method provided in the embodiment of the present application may include:

S1601: The first terminal displays a first interface.

Wherein, the first interface may be a text input interface, a game interface, or a video playback interface, etc. in the above embodiments, which is not limited herein.

S1602: When the first terminal is displaying the first interface, the first terminal establishes a communication connection with the second terminal.

Wherein, for the specific process of S1602, reference may be made to the introduction in the foregoing embodiments, and details are not described here. Wherein, the first terminal may be the smart TV 100 in the above-mentioned embodiment, and the second terminal may be the above-mentioned mobile phone 200 . That is to say, the smart TV 100 can act as a function demander, and realize functions such as text input and game character behavior control with the help of the mobile phone 200 as a function provider. Certainly, the first terminal may also be the aforementioned mobile phone 200 , and the second terminal may also be the aforementioned smart TV 100 . That is, the mobile phone 200 may serve as a function demander, and realize functions such as video interaction with the help of the smart TV 100 as a function provider.

S1603: During the process of displaying the first interface by the first terminal, the first terminal sends control information to the second terminal. Wherein, the control information is used to instruct the second terminal to enable the target function unit, and when the target function unit is triggered, it can realize the trigger-associated function.

Exemplarily, the target function unit may be a second interface including a virtual text key; when the virtual text key is triggered, the function of inputting text data (that is, target data) to the text input interface of the first terminal may be implemented. Wherein, when the second interface includes a virtual text key and is triggered, the function of inputting text data to the text input interface of the first terminal can be realized, and the introduction of FIGS.

The target functional unit can also be the second interface of the virtual control keyboard. When the virtual control keyboard is triggered, it can control the game character on the first interface of the first terminal to perform the game behavior associated with the trigger. The target functional unit can also be a gesture sensor, and when the gesture sensor is triggered, it can control the game character on the first interface of the first terminal to perform the game behavior associated with the trigger. Wherein, when the virtual control keyboard or the gesture sensor is triggered, it can be realized to control the game character on the first interface of the first terminal to perform the game behavior associated with the trigger. You can refer to the introduction of Fig. 11-Fig. 12 in the above-mentioned embodiment. This will not be repeated here.

The target functional unit may be a video acquisition module (such as a camera, a microphone). When the video acquisition module is turned on, it can control the first terminal and the third terminal to realize the video interaction function. Wherein, when the video acquisition module is turned on, the first terminal and the third terminal may be controlled to realize video interaction function, and reference may be made to the above introduction to FIG. 14 , which will not be repeated here.

S1604: The second terminal responds to the trigger operation on the target function unit, so that the first terminal executes a function associated with the trigger operation when the first interface is displayed.

Exemplarily, referring to the above-mentioned illustration of FIG. 9, when the first interface is a text input interface and the target functional unit is a virtual text key in the second interface, the second terminal may respond to the user triggering the virtual text key Operation, get text data. Understandably, the text data is the target data.

Or; Exemplarily, referring to the above-mentioned explanation of (a) in FIG. Based on the trigger operation on the virtual control keyboard in the second interface, the game character on the first interface of the first terminal is controlled to perform the game behavior associated with the trigger. Understandably, the game character on the first interface of the first terminal is controlled to perform a game behavior associated with the trigger, that is, a function associated with the trigger operation.

Or; Exemplarily, when the first interface is a game interface and the target functional unit is a gesture sensor, the second terminal may respond to a user's trigger operation, so that the gesture data of the second terminal collected by the gesture sensor; to control the first terminal The game character on the first interface executes the game behavior associated with the trigger. Understandably, the game character on the first interface of the first terminal is controlled to perform a game behavior associated with the trigger, that is, a function associated with the trigger operation.

Or; Exemplarily, when the first interface is a video playback interface, and the target functional unit includes a video acquisition module (such as a microphone and a camera), the first terminal and the third terminal may be controlled to implement a video interaction function. Understandably, the first terminal and the third terminal are controlled to implement a video interaction function, that is, a function associated with a trigger operation.

Figure 17 shows that when the first terminal is a smart TV 100 and the second terminal is a mobile phone 200 provided by the embodiment of the present application, the smart TV 100 implements the function associated with the target data with the help of the mobile phone 200, or the first terminal is the mobile phone 200 and the second terminal When the terminal is the smart TV 100 , the mobile phone 200 implements the functional architecture associated with the target data by means of the smart TV 100 .

As shown in FIG. 17 , the smart TV 100 and the mobile phone 200 can be connected through a short-distance communication module. Wherein, the short-distance communication module may be a Bluetooth module, a WiFi module or an NFC module, etc., which are not limited herein. When the smart TV 100 received the interface identification from the video playback interface of the mobile phone 200, the coordinated control application of the smart TV 100 invoked and opened the video playback interface (comprising image C and image D) of the video application and called the speaker to play the video from the mobile phone 200. The first sound data; and, calling the camera of the smart TV 100 to collect the image C and the microphone to collect the second sound data. It can be seen that the cooperative control application of the smart TV 100 decides to call the video application, speaker, camera, and microphone based on the interface identifier of the video playback interface. In addition, the collaborative control application of the mobile phone 100 may also generate instructions for invoking video applications, speakers, cameras, and microphones in the video playback interface. Smart TV 200 invokes video application, speaker, camera and microphone after receiving the instruction from mobile phone 100 . In this way, the mobile phone 100 can also decide to call a video application, a speaker, a camera, and a microphone. The mobile phone 200 receives the image C and the second sound data from the smart TV 100, and sends the image C and the second sound data to the third terminal. In this way, the mobile phone 200 can use the smart TV 100 to realize a large-screen video interactive function. It can be understood that the interface identification of the above-mentioned video playback interface and the instructions for invoking the video application, speaker, camera, and microphone are all control information.

In the above process, it can be understood that the smart TV 200 can decide to call the video application, speaker, camera, and microphone according to the functions currently running on the smart TV 200 and the functions currently running on the mobile phone 100; The functions currently running and the functions currently running on the smart TV 200 decide to call a video application, a speaker, a camera, and a microphone.

When the mobile phone 200 received the identification from the text input interface of the smart TV 100, the cooperative control application decision of the mobile phone 200 calls the text input application to open the keyboard interface. In response to the user's trigger operation on the keyboard interface, the mobile phone 200 sends text data associated with the trigger operation to the smart TV 100 . The smart TV 100 displays the received text data. In addition, the collaborative control application of the smart TV 200 can also decide to generate an instruction to call the text input application based on the same principle as above, and control the mobile phone 100 to call the text input application to open the keyboard interface, which will not be repeated here.

When the mobile phone 200 receives the identification from the game interface of the smart TV 100, the cooperative control application of the mobile phone 200 decides to invoke the handle application to open the handle interface. In response to the user's trigger operation on the handle interface, the mobile phone 200 sends control data on the game character associated with the trigger operation to the smart TV 100 . The smart TV 100 controls the game character in the game interface to execute the game behavior associated with the control data.

Or, when the mobile phone 200 receives the identification from the game interface of the smart TV 100 , the cooperative control application of the mobile phone 200 decides to call the posture sensor to collect the posture data (ie control data) of the mobile phone 200 . The mobile phone 200 sends attitude data to the smart TV 100 . The smart TV 100 controls the game character in the game interface to perform the game behavior associated with the gesture data. In addition, the cooperative control application of the smart TV 200 can also decide to generate an instruction to call the handle application or the attitude sensor based on the above-mentioned same principle, and control the mobile phone 100 to call the handle application or the attitude sensor to obtain control data, which will not be repeated here.

It should be noted that, the carrier of the cooperative control application mentioned above may be an operable icon located on a negative screen, an activity of an application program, or an interface of a background service, etc., which is not limited herein.

In addition, in the above-mentioned introduction to the device control method provided by the embodiment of the present application, the trigger operation mentioned may include: click operation, long press operation, and operation of changing the posture of the terminal device, etc., which are not limited here.

In addition, in the above-mentioned embodiments, it is described that the first terminal is in the first state and the first terminal displays the first interface as an example. In addition, the first terminal is in the first state, and may also be in a state of playing music, and further, the first terminal plays music through the speaker of the second terminal. It is not limited here.

In addition, in the above embodiments, the first terminal performs the first function as an example. Here, in the scene where the first terminal plays music through the speaker of the second terminal, it is the second terminal that performs the first function. That is, the second terminal performs the function of playing music.

Referring to FIG. 18 , the embodiment of the present application further provides a device control apparatus 1800, which is applied to a first terminal. The device control apparatus 1800 provided in the embodiment of the present application includes: a first processing unit 1802 configured to control the first terminal to be in the first state, where the first terminal can execute the first function when in the first state. The first communication unit 1801 is configured to send control information to the second terminal based on the established communication connection. Wherein, the control information is used to instruct the second terminal to activate the target function unit, so as to cooperate with the first terminal to execute the first function.

In a possible implementation manner, the first communication unit 1801 is further configured to receive target data from the second terminal. The first processing unit 1802 is further configured to execute a first function associated with the target data.

In a possible implementation manner, the target functional unit is a virtual text button in the second interface, and the target data is text data. The first processing unit 1802 is specifically configured to control the first terminal to display a first interface, where the first interface includes an area where text is to be input. The first processing unit 1802 is further specifically configured to control the first terminal to input text data in an area where text is to be input on the first interface when the first interface is displayed.

In a possible implementation manner, the target data is control data. The first processing unit 1802 is specifically configured to control the first terminal to display a first interface, where the first interface includes game characters. The first processing unit 1802 is also specifically configured to control the first terminal to control the game character in the first interface to perform the game behavior corresponding to the control data when the first interface is displayed.

In a possible implementation manner, the target functional unit is a video collection module, and the target data is the second video data collected by the video collection module. The first processing unit 1802 is specifically configured to control the first terminal to display the first interface, wherein the first interface includes the first video data from the third terminal, and the first processing unit 1802 is specifically configured to control the first terminal to display the first interface. In the first interface, receiving the second video data from the second terminal; sending the second video data to the third terminal.

Referring to FIG. 19 , the embodiment of the present application further provides a device control apparatus 1900, which is applied to a second terminal. The device control apparatus 1900 provided in the embodiment of the present application includes: a second communication unit 1901, configured to receive control information from a first terminal based on an established communication connection. The second processing unit 1902 is configured to enable the target function unit in response to the control information. The second processing unit 1902 is further configured to cooperate with the first terminal to execute the first function based on the target function unit.

In a possible implementation manner, the second processing unit 1902 is specifically configured to control the second communication unit 1901 to send target data to the first terminal in response to a trigger operation on the target functional unit; A terminal performs a first function associated with target data.

In a possible implementation manner, the control information is information indicating that the first terminal is in the first state, and the second processing unit 1902 is specifically configured to enable the target functional unit according to the information indicating that the first terminal is in the first state . Alternatively, the control information is an instruction for enabling the target functional unit, and the second processing unit 1902 is specifically configured to enable the target functional unit according to the instruction for enabling the target functional unit.

Exemplarily, FIG. 20 is a schematic diagram of a hardware structure of a first terminal or a second terminal provided in an embodiment of the present application. As shown in FIG. 20 , the first terminal or the second terminal includes a processor 2001, a communication line 2004 and At least one communication interface (the communication interface 2003 is used as an example in FIG. 20 for illustration).

The processor 2001 can be a general-purpose central processing unit (central processing unit, CPU), a microprocessor, a specific application integrated circuit (application-specific integrated circuit, ASIC), or one or more for controlling the implementation of the application program program integrated circuit.

Communication lines 2004 may include circuitry that communicates information between the components described above.

The communication interface 2003 uses any device such as a transceiver for communicating with other devices or communication networks, such as Ethernet, wireless local area networks (wireless local area networks, WLAN) and so on.

Possibly, the first terminal or the second terminal may further include a memory 2002 .

The memory 2002 may be a read-only memory (read-only memory, ROM) or other types of static storage devices that can store static information and instructions, a random access memory (random access memory, RAM) or other types that can store information and instructions It can also be an electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or can be used to carry or store desired program code in the form of instructions or data structures and can be programmed by a computer Any other medium accessed, but not limited to. The memory may exist independently and be connected to the processor through the communication line 2004 . Memory can also be integrated with the processor.

Wherein, the memory 2002 is used to store computer-executed instructions for implementing the solution of the present application, and the execution is controlled by the processor 2001 . The processor 2001 is configured to execute computer-executed instructions stored in the memory 2002, so as to implement the device control method performed by the first terminal or the second terminal provided in the embodiment of the present application.

Possibly, the computer-executed instructions in the embodiment of the present application may also be referred to as application program code, which is not specifically limited in the embodiment of the present application.

In a specific implementation, as an embodiment, the processor 2001 may include one or more CPUs, for example, CPU0 and CPU1 in FIG. 20 .

In a specific implementation, as an example, the first terminal or the second terminal may include multiple processors, for example, processor 2001 and processor 2005 in FIG. 20 . Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).

Exemplarily, FIG. 21 is a schematic structural diagram of a chip provided by an embodiment of the present application. The chip 210 includes one or more than two (including two) processors 2110 and a communication interface 2130 .

In some embodiments, the memory 2140 stores the following elements: executable modules or data structures, or subsets thereof, or extensions thereof.

In this embodiment of the present application, the memory 2140 may include a read-only memory and a random access memory, and provides instructions and data to the processor 2110 . A part of the memory 2140 may also include a non-volatile random access memory (non-volatile random access memory, NVRAM).

In the embodiment of the present application, the memory 2140 , the communication interface 2130 and the memory 2140 are coupled together through the bus system 2120 . Wherein, the bus system 2120 may include not only a data bus, but also a power bus, a control bus, and a status signal bus. For ease of description, various buses are labeled as bus system 2120 in FIG. 21 .

The methods described in the foregoing embodiments of the present application may be applied to the processor 2110 or implemented by the processor 2110 . The processor 2110 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method may be implemented by an integrated logic circuit of hardware in the processor 2110 or instructions in the form of software. The above-mentioned processor 2110 may be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), an off-the-shelf programmable gate Array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components, the processor 2110 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application .

The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. Wherein, the software module may be located in a mature storage medium in the field such as random access memory, read-only memory, programmable read-only memory, or electrically erasable programmable read only memory (EEPROM). The storage medium is located in the memory 2140, and the processor 2110 reads the information in the memory 2140, and completes the steps of the above method in combination with its hardware.

In the above embodiments, the instructions stored in the memory for execution by the processor may be implemented in the form of computer program products. Wherein, the computer program product may be written in the memory in advance, or may be downloaded and installed in the memory in the form of software.

A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, special purpose computer, computer network, or other programmable device. Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL) or wireless (such as infrared, wireless, microwave, etc.) transmission to another website site, computer, server or data center. Computer readable storage medium can be Any available media capable of being stored by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. For example, available media may include magnetic media (e.g., floppy disks, hard disks, or tapes), optical media (e.g., A digital versatile disc (digital versatile disc, DVD)), or a semiconductor medium (for example, a solid state disk (solid state disk, SSD)), etc.

The embodiment of the present application also provides a computer-readable storage medium. The methods performed by the first terminal or the second terminal described in the foregoing embodiments may be implemented in whole or in part by software, hardware, firmware or any combination thereof. Computer-readable media may include computer storage media and communication media, and may include any medium that can transfer a computer program from one place to another. A storage media may be any target media that can be accessed by a computer.

As a possible design, the computer-readable medium may include compact disc read-only memory (compact disc read-only memory, CD-ROM), RAM, ROM, EEPROM or other optical disc storage; the computer-readable medium may include a magnetic disk memory or other disk storage devices. Also, any connected cord is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, compact disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Reproduce data.

Combinations of the above should also be included within the scope of computer-readable media. The above is only the specific implementation of the application, but the scope of protection of the application is not limited thereto. Anyone familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the application, and should cover Within the protection scope of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

A device control method, characterized in that it is applied to a communication system, the communication system includes a first terminal and a second terminal, and the method includes:

The first terminal is in a first state, wherein when the first terminal is in the first state, it can perform a first function;

The first terminal sends control information to the second terminal based on the established communication connection;

enabling the target functional unit by the second terminal in response to the control information;

The second terminal cooperates with the first terminal to execute the first function based on the target function unit.
The method according to claim 1, wherein the second terminal cooperates with the first terminal to execute the first function based on the target functional unit, comprising:

The second terminal sends target data to the first terminal in response to a trigger operation on the target functional unit;

The first terminal executes the first function associated with the target data.
The method according to claim 2, wherein the target functional unit is a virtual text button in the second interface, and the target data is text data,

The first terminal being in the first state includes: displaying a first interface by the first terminal, where the first interface includes an area where text is to be input;

The first terminal executing the first function associated with the target data includes:

When the first terminal displays the first interface, input the text data in the region where the text is to be input on the first interface.
The method according to claim 2, wherein the target data is control data,

The first terminal being in the first state includes: the first terminal displays a first interface, wherein the first interface includes a game character,

The first terminal executing the first function associated with the target data includes:

When the first terminal displays the first interface, it controls the game character in the first interface to execute the game behavior corresponding to the control data.
The method according to claim 4, characterized in that,

The target function unit is a virtual control keyboard in the second interface, wherein the virtual control keyboard includes virtual keys for controlling the game behavior of the game character;

Alternatively, the control data is attitude data, and the target functional unit is an attitude sensor of the second terminal for collecting the attitude data.
The method according to claim 5, wherein the target functional unit is a virtual control keyboard in the second interface, and after the first terminal sends control information to the second terminal, the method Also includes:

The second terminal receives first interface data from the first interface of the first terminal;

The second terminal displays second interface data on the second interface, where the second interface data includes display content corresponding to the first interface data.
The method according to claim 2, wherein the target functional unit is a video acquisition module, and the target data is the second video data collected by the video acquisition module,

The first terminal being in the first state includes: the first terminal displays a first interface, wherein the first interface includes first video data from a third terminal,

The first terminal executing the first function associated with the target data includes: when the first terminal displays the first interface, receiving the second video data from the second terminal; The first terminal sends the second video data to the third terminal.
The method according to claim 7, wherein after the second terminal enables the target functional unit in response to the control information, the method further comprises:

The second terminal receives first video data from the first terminal;

The second terminal displays a second interface, where the second interface includes the first video data.
The method according to claim 1, wherein the method for establishing the communication connection comprises:

When the first terminal is in the first state, performing device discovery; when the first terminal discovers the second terminal, establishes a communication connection with the second terminal;

Or, when the distance between the second terminal and the target electronic device is less than a distance threshold, acquire the device identifier of the first terminal stored in the target electronic device; The first terminal establishes a communication connection;

Alternatively, the first terminal receives voice information input by the user; wherein, the voice information is used to indicate connection with the second terminal; the first terminal performs device discovery, and the first terminal discovers the second terminal In the case of two terminals, establishing a communication connection with the second terminal;

Alternatively, the second terminal transmits a UWB signal, the UWB signal carries the device identifier of the second terminal, the first terminal receives the UWB signal from the second terminal, and the first terminal according to the The device identifier in the UWB signal establishes a communication connection with the second terminal.
The method according to claim 1, wherein before the first terminal sends control information to the second terminal based on the established communication connection, the method further comprises:

The second terminal displays a fifth interface, and the fifth interface includes a first control for controlling entering/exiting the third preset mode;

The second terminal responds to a trigger operation on the first control, and the second terminal is in the third preset mode.
The method according to any one of claims 1-10, characterized in that,

The control information is information indicating that the first terminal is in the first state, and the second terminal starts the target function unit in response to the control information, including: the second terminal according to the indication information that the first terminal is in the first state, and enable the target functional unit;

Alternatively, the control information is an instruction for enabling the target functional unit, and the second terminal activates the target functional unit in response to the control information, including: the second terminal activates the target functional unit according to the instruction for enabling the target functional unit an instruction of the target functional unit to start the target functional unit.
A device control device, characterized in that it is applied to a first terminal, and the device includes:

A first processing unit, configured to control the first terminal to be in a first state, where the first terminal is capable of performing a first function when in the first state;

The first communication unit is configured to send control information to the second terminal based on the established communication connection;

Wherein, the control information is used to instruct the second terminal to activate the target function unit, so as to cooperate with the first terminal to perform the first function.
The device according to claim 12, wherein the first communication unit is further configured to receive target data from the second terminal;

The first processing unit is further configured to execute the first function associated with the target data.
The device according to claim 13, wherein the target functional unit is a virtual text button in the second interface, and the target data is text data,

The first processing unit is specifically configured to control the first terminal to display a first interface, where the first interface includes an area where text is to be input;

The first processing unit is further specifically configured to control the first terminal to input the text data in the area where text is to be input on the first interface when the first interface is displayed.
The device according to claim 13, wherein the target data is control data,

The first processing unit is specifically configured to control the first terminal to display a first interface, where the first interface includes game characters,

The first processing unit is further specifically configured to control the first terminal to control the game character in the first interface to perform the game behavior corresponding to the control data when the first interface is displayed.
The device according to claim 15, characterized in that,

The target function unit is a virtual control keyboard in the second interface, wherein the virtual control keyboard includes virtual keys for controlling the game behavior of the game character;

Alternatively, the control data is attitude data, and the target functional unit is an attitude sensor of the second terminal for collecting the attitude data.
The device according to claim 13, wherein the target functional unit is a video acquisition module, and the target data is the second video data collected by the video acquisition module,

The first processing unit is specifically configured to control the first terminal to display a first interface, where the first interface includes first video data from a third terminal,

The first processing unit is specifically configured to control the first terminal to receive the second video data from the second terminal when displaying the first interface; to send the second video data to the third terminal. Two video data.
A device control device, characterized in that it is applied to a second terminal, and the device includes:

a second communication unit, configured to receive control information from the first terminal based on the established communication connection;

a second processing unit, configured to enable the target functional unit in response to the control information;

The second processing unit is further configured to cooperate with the first terminal to execute the first function based on the target function unit.
The device according to claim 18, wherein the second processing unit is specifically configured to control the second communication unit to send a target message to the first terminal in response to a trigger operation on the target function unit. data; wherein, the target data is used to instruct the first terminal to execute the first function associated with the target data.
The device according to claim 19, wherein the target functional unit is a virtual text button in the second interface, the target data is text data, and the text data is used to indicate that the first terminal is displaying In the first interface, the text data is input in an area where text is to be input on the first interface.
The device according to claim 19, wherein the target data is control data, and the target data is used to instruct the first terminal to control the game character in the first interface when the first interface is displayed Execute game behaviors corresponding to the control data.
The device according to claim 21, characterized in that,

The target function unit is a virtual control keyboard in the second interface, wherein the virtual control keyboard includes virtual keys for controlling the game behavior of the game character;

Alternatively, the control data is attitude data, and the target functional unit is an attitude sensor of the second terminal for collecting the attitude data.
The device according to claim 19, wherein the target functional unit is a video capture module, the target data is the second video data collected by the video capture module, and the second video data is used to indicate the The first terminal sends the second video data to the third terminal when displaying the first interface including the first video data.
The device according to any one of claims 18-23, characterized in that,

The control information is information indicating that the first terminal is in the first state, and the second processing unit is specifically configured to activate the target according to the information indicating that the first terminal is in the first state. functional unit;

Alternatively, the control information is an instruction for enabling the target functional unit, and the second processing unit is specifically configured to enable the target functional unit according to the instruction for enabling the target functional unit.
A computer-readable storage medium, the computer-readable storage medium stores a computer program, wherein, when the computer program is executed by a processor, the computer executes the first method according to any one of claims 1 to 11. A method performed by a terminal or a second terminal.