CN113014614A

CN113014614A - Equipment control method, control equipment and controlled equipment

Info

Publication number: CN113014614A
Application number: CN201911329850.5A
Authority: CN
Inventors: 耿炳钰; 王龙龙
Original assignee: Hisense Mobile Communications Technology Co Ltd
Current assignee: Hisense Mobile Communications Technology Co Ltd
Priority date: 2019-12-20
Filing date: 2019-12-20
Publication date: 2021-06-22

Abstract

The invention discloses a device control method, control equipment and controlled equipment, which are used for realizing cross-equipment remote operation. The control equipment in the embodiment of the invention displays the display interface of the controlled equipment in the display window; in response to an operation triggered by a user in the display window, the control device sends control information for representing the operation of the user in the display window to the controlled device so that the controlled device executes the operation corresponding to the control information. According to the embodiment of the invention, the display interface of the controlled equipment can be displayed in the display window of the control equipment, and the operation triggered by the user in the display interface of the control equipment can be converted into the operation on the controlled equipment, so that the interaction process between the equipment is completed, the user can operate the controlled equipment on the control equipment without replacing the equipment, the cross-equipment remote operation is conveniently and quickly completed, and the user experience is improved.

Description

Equipment control method, control equipment and controlled equipment

Technical Field

The invention relates to the technical field of terminal display, in particular to a device control method, control equipment and controlled equipment.

Background

With the popularization of intelligent communication terminals, in order to meet the requirements of different occasions, a user may have two communication terminals at the same time, and if the user needs to operate one communication terminal which is not in hand, such as a short reply message or browse a certain file, the user needs to acquire the communication terminal and then operate the communication terminal; or if another communication terminal of the user is not at a position which is conveniently taken by the user at present, but the communication terminal receives information which needs to be processed by the user or the user needs to browse the content on the communication terminal, the user cannot check the content in time.

Therefore, how to perform remote operation across devices becomes a problem to be solved urgently.

Disclosure of Invention

The exemplary embodiment of the invention provides a device control method, control device and controlled device, which are used for realizing cross-device remote operation.

According to a first aspect of the exemplary embodiments, there is provided a control device comprising a touch screen, a display screen and a processor;

the touch screen is used for receiving touch operation from a user;

the display screen is used for displaying a display interface of the controlled equipment in the display window;

the processor is configured to:

and in response to the operation triggered by the user in the display window, sending control information for representing the operation of the user in the display window to the controlled device so as to enable the controlled device to execute the operation corresponding to the control information.

According to the embodiment of the invention, the display interface of the controlled equipment can be displayed in the display window of the control equipment, and the operation triggered by the user in the display interface of the control equipment can be converted into the operation on the controlled equipment, so that the interaction process between the equipment is completed, the user can operate the controlled equipment on the control equipment without replacing the equipment, the cross-equipment remote operation is conveniently and quickly completed, and the user experience is improved.

In one possible embodiment, the processor is further configured to:

before a display interface of the controlled equipment is displayed in a display window, responding to an operation triggered by a user and used for controlling the controlled equipment, sending a remote control instruction to the controlled equipment, so that the controlled equipment executes screen recording operation after receiving the remote control instruction, and sending the recorded display interface to the control equipment; and receiving the display interface sent by the controlled equipment.

According to the embodiment of the invention, the control equipment can receive the display interface recorded by the controlled equipment and display the display interface in the display window, so that the display interface of the controlled equipment can be displayed in the display window of the control equipment, the control equipment can control the controlled equipment, and a user can conveniently operate the control equipment.

In one possible implementation, the processor is specifically configured to:

sending a remote control instruction to the controlled device through the Bluetooth connection with the controlled device; or sending a remote control instruction to the controlled equipment through WiFi Direct connection with the controlled equipment.

In the embodiment of the invention, the control equipment can adopt Bluetooth connection or WiFi Direct connection to send the remote control instruction to the controlled equipment, so that a method for establishing connection between the control equipment and the controlled equipment is provided, and data exchange can be carried out when the control equipment and the controlled equipment are interacted.

In one possible implementation, the processor is specifically configured to:

receiving the display interface sent by the controlled device through a streaming media link in WiFi Direct connection with the controlled device;

and sending the control information to the controlled equipment through a control link in WiFi Direct connection with the controlled equipment.

In the embodiment of the invention, when data transmission is carried out between the control equipment and the controlled equipment, the streaming media link is used for transmitting the display interface, and the control link is used for transmitting the control information, so that the controlled equipment can continuously send the recorded display interface to the control equipment through the streaming media link, and the transmission bandwidth can be dynamically adapted.

In one possible implementation, the processor is further configured to:

after receiving notification information sent by the controlled device through Bluetooth connection, displaying the notification information in the display window; the notification information is information pushed by other communication equipment and received by the controlled equipment; and responding to the operation of replying the notification information by the user, returning reply information input by the user to the controlled equipment, so that the controlled equipment sends the reply information to the other communication equipment.

In the embodiment of the invention, the controlled device can complete a simple message reply function through the Bluetooth with the control device, so that the control device can also receive and reply the notification information received by the controlled device under the condition of not establishing WiFi Direct connection, low power consumption is ensured, simple remote operation of the device is provided, and user experience is improved.

In one possible embodiment, the control information includes some or all of the following information:

operation position, operation state, trigger operation time and operation number;

wherein the position of the operation is an actual position of an operation triggered by a user in a display area of the control device; or the operation position is a position corresponding to the operation position of the user in the display area of the control device mapped to the display interface of the controlled device.

According to the embodiment of the invention, the control device can send the operation triggered by the user in the display area to the controlled device in the form of the control information, and the controlled device can execute the corresponding operation according to the content in the control information after receiving the control information, so that the process of controlling the controlled device by using the control device is completed, the remote operation among the devices is realized, and the user experience is improved.

According to a second aspect of the exemplary embodiments, a controlled device is provided, which can be applied to a controlled device, and comprises a touch screen, a display screen and a processor;

the touch screen is used for receiving touch operation from a user;

the display screen is used for displaying a display interface;

the processor is configured to:

receiving control information which is sent by a control device and used for representing the operation of a user in a display window of the control device; the display window displays a display interface of the controlled device, and the control information is sent by the control device after responding to the operation triggered by the user in the display window;

and executing the operation corresponding to the control information.

In one possible implementation, the processor is further configured to:

after receiving a remote control instruction sent by the control equipment, executing screen recording operation; the remote control instruction is sent by the control equipment after responding to the operation triggered by the user and used for controlling the controlled equipment; and sending the recorded display interface to the control equipment.

In one possible implementation, the processor is specifically configured to:

receiving a remote control instruction sent by the control equipment through Bluetooth connection with the control equipment; or

And receiving a remote control instruction sent by the control equipment through WiFi Direct connection with the control equipment.

In one possible implementation, the processor is specifically configured to:

sending the display interface to the control equipment through a streaming media link in WiFi Direct connection with the control equipment;

and receiving the control information sent by the control equipment through a control link in the WiFi Direct connection with the control equipment.

In one possible implementation, the processor is specifically configured to:

sending notification information to the control device through a Bluetooth connection; the notification information is information pushed by other communication equipment and received by the controlled equipment;

receiving reply information returned by the control equipment, and sending the reply information to the other communication equipment; and the reply information is information input when the user replies the notification message through the control equipment.

According to a third aspect of the exemplary embodiments, there is provided a device control method applied to control a device, the method including:

the control equipment displays a display interface of the controlled equipment in a display window;

in response to an operation triggered by a user in the display window, the control device sends control information for representing the operation of the user in the display window to the controlled device, so that the controlled device executes an operation corresponding to the control information.

In a possible implementation manner, before the control device displays the display interface of the controlled device in the display window, the method further includes:

responding to an operation triggered by a user and used for controlling the controlled equipment, and sending a remote control instruction to the controlled equipment by the control equipment so that the controlled equipment executes screen recording operation after receiving the remote control instruction and sends a recorded display interface to the control equipment;

and the control equipment receives the display interface sent by the controlled equipment.

One possible implementation manner is that the control device sends a remote control instruction to the controlled device, and the method includes:

the control device sends a remote control instruction to the controlled device through the Bluetooth connection with the controlled device; or

And the control equipment sends a remote control instruction to the controlled equipment through WiFi Direct connection with the controlled equipment.

One possible implementation manner is that the control device receives the display interface sent by the controlled device, and includes:

the control equipment receives the display interface sent by the controlled equipment through a streaming media link in WiFi Direct connection with the controlled equipment;

the control device sends control information for representing the operation of the user in the display window to the controlled device, and the control information comprises:

and the control equipment sends the control information to the controlled equipment through a control link in the WiFi Direct connection with the controlled equipment.

In one possible embodiment, the method further comprises:

the control equipment displays the notification information in the display window after receiving the notification information sent by the controlled equipment through Bluetooth connection; the notification information is information pushed by other communication equipment and received by the controlled equipment;

and responding to the operation of replying the notification information by the user, returning reply information input by the user to the controlled device by the control device, so that the controlled device sends the reply information to the other communication devices.

wherein the operation position is an actual position of an operation triggered by a user in a display area of the control device; or the operation position is a position corresponding to the operation position of the user in the display area of the control device mapped to the display interface of the controlled device.

According to a fourth aspect of the exemplary embodiments, there is provided a device control method applied to a controlled device, the method including:

the method comprises the steps that a controlled device receives control information which is sent by a control device and used for representing the operation of a user in a display window of the control device; the display window displays a display interface of the controlled device, and the control information is sent by the control device after responding to the operation triggered by the user in the display window;

the controlled device executes an operation corresponding to the control information.

In a possible implementation manner, before the controlled device receives the control information sent by the control device and used for representing the user operation in the display window, the method further includes:

after the controlled equipment receives the remote control instruction sent by the control equipment, screen recording operation is executed; the remote control instruction is sent by the control equipment after responding to the operation triggered by the user and used for controlling the controlled equipment;

and the controlled equipment sends the recorded display interface to the control equipment.

A possible implementation manner that the controlled device receives a remote control instruction sent by the control device includes:

the controlled device receives a remote control instruction sent by the control device through Bluetooth connection with the control device; or

And the controlled equipment receives the remote control instruction sent by the control equipment through WiFi Direct connection with the control equipment.

A possible implementation manner, where sending the recorded display interface to the control device includes:

the controlled equipment sends the display interface to the control equipment through a streaming media link in WiFi Direct connection with the control equipment;

the controlled device receives control information which is sent by the control device and used for representing the operation of a user in a display window, and the control information comprises:

and the controlled equipment receives the control information sent by the control equipment through a control link in WiFi Direct connection with the control equipment.

In one possible embodiment, the method further comprises:

the controlled device sends notification information to the control device through Bluetooth connection; the notification information is information pushed by other communication equipment and received by the controlled equipment;

the controlled device receives reply information returned by the control device and sends the reply information to the other communication devices; and the reply information is information input when the user replies the notification message through the control equipment.

On the basis of the common knowledge in the field, the above preferred conditions can be combined randomly to obtain the preferred embodiments of the invention.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic structural diagram of a communication terminal according to an embodiment of the present invention;

fig. 2 is a schematic diagram illustrating a software architecture of a communication terminal according to an embodiment of the present invention;

fig. 3 is a schematic diagram illustrating a user interface of a communication terminal provided by an embodiment of the present invention;

FIG. 4 is a schematic diagram illustrating an appliance control system according to an embodiment of the present invention;

FIG. 5 is a schematic interface diagram illustrating a control device and a controlled device establishing a Bluetooth connection according to an embodiment of the present invention;

fig. 6 is an interface schematic diagram illustrating that a control device and a controlled device establish a WiFi Direct connection according to an embodiment of the present invention;

FIG. 7 is a schematic diagram illustrating an interface for a device A to pull a control grant to a device B according to an embodiment of the present invention;

FIG. 8 is a schematic diagram illustrating remote operation of a control device and a controlled device through a Bluetooth connection according to an embodiment of the present invention;

FIG. 9 is a schematic diagram illustrating a control device controlling a controlled device according to an embodiment of the present invention;

fig. 10 is a flowchart illustrating a method for a control device to remotely receive push information on a controlled device according to an embodiment of the present invention;

fig. 11 is a flowchart illustrating a method for a control device to remotely control a controlled device according to an embodiment of the present invention;

fig. 12 is a schematic structural diagram schematically showing a control apparatus according to an embodiment of the present invention;

fig. 13 is a schematic structural diagram illustrating an apparatus control device that controls an apparatus according to an embodiment of the present invention;

FIG. 14 is a schematic structural diagram of a controlled device according to an embodiment of the present invention;

fig. 15 is a schematic structural diagram illustrating an apparatus control device that can be applied to a controlled apparatus according to an embodiment of the present invention;

fig. 16 is a flowchart illustrating an apparatus control method applied to a control apparatus according to an embodiment of the present invention;

fig. 17 is a flowchart illustrating an apparatus control method applied to a controlled apparatus according to an embodiment of the present invention.

Detailed Description

The technical solution in the embodiments of the present application will be described in detail and removed with reference to the accompanying drawings. In the description of the embodiments herein, "/" means "or" unless otherwise specified, for example, a/B may mean a or B; "and/or" in the text is only an association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B may mean: three cases of a alone, a and B both, and B alone exist, and in addition, "a plurality" means two or more than two in the description of the embodiments of the present application.

In the following, the terms "first", "second" are used for descriptive purposes only and are not to be understood as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of embodiments of the application, unless stated otherwise, "plurality" means two or more.

Some terms appearing herein are explained below:

1. the term "and/or" in the embodiments of the present invention describes an association relationship of associated objects, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

2. The term "streaming media" in the embodiments of the present invention refers to a technology for compressing a series of media data and then transmitting the compressed media data in a streaming manner in segments in a network to realize real-time transmission of video and audio on the network for viewing.

3. The term "remote control" in embodiments of the present invention is a capability to operate a remote device using radio or electrical signals.

Fig. 1 shows a schematic configuration of a communication terminal 100.

The following describes an embodiment specifically taking the communication terminal 100 as an example. It should be understood that the communication terminal 100 shown in fig. 1 is only an example, and the communication terminal 100 may have more or less components than those shown in fig. 1, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

A block diagram of a hardware configuration of a communication terminal 100 according to an exemplary embodiment is exemplarily shown in fig. 1. As shown in fig. 1, the communication terminal 100 includes: a Radio Frequency (RF) circuit 110, a memory 120, a display unit 130, a camera 140, a sensor 150, an audio circuit 160, a Wireless Fidelity (Wi-Fi) module 170, a processor 180, a bluetooth module 181, and a power supply 190.

The RF circuit 110 may be used for receiving and transmitting signals during information transmission and reception or during a call, and may receive downlink data of a base station and then send the downlink data to the processor 180 for processing; the uplink data may be transmitted to the base station. Typically, the RF circuitry includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like.

The memory 120 may be used to store software programs and data. The processor 180 executes various functions of the communication terminal 100 and data processing by executing software programs or data stored in the memory 120. The memory 120 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. The memory 120 stores an operating system that enables the communication terminal 100 to operate. The memory 120 may store an operating system and various application programs, and may also store codes for performing the methods described in the embodiments of the present application.

The display unit 130 may be used to receive input numeric or character information and generate signal input related to user settings and function control of the communication terminal 100, and particularly, the display unit 130 may include a touch screen 131 disposed on the front surface of the communication terminal 100 and may collect touch operations of a user thereon or nearby, such as clicking a button, dragging a scroll box, and the like.

The display unit 130 may also be used to display a Graphical User Interface (GUI) of information input by or provided to the user and various menus of the terminal 100. Specifically, the display unit 130 may include a display screen 132 disposed on the front surface of the communication terminal 100. The display screen 132 may be configured in the form of a liquid crystal display, a light emitting diode, or the like. The display unit 130 may be used to display various graphical user interfaces described herein.

The touch screen 131 may cover the display screen 132, or the touch screen 131 and the display screen 132 may be integrated to implement the input and output functions of the communication terminal 100, and after the integration, the touch screen may be referred to as a touch display screen for short. In the present application, the display unit 130 may display the application programs and the corresponding operation steps.

The camera 140 may be used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing elements convert the light signals into electrical signals which are then passed to the processor 180 for conversion into digital image signals.

The communication terminal 100 may further comprise at least one sensor 150, such as an acceleration sensor 151, a distance sensor 152, a fingerprint sensor 153, a temperature sensor 154. The communication terminal 100 may also be configured with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, optical sensor, motion sensor, and the like.

The audio circuitry 160, speaker 161, microphone 162 may provide an audio interface between a user and the communication terminal 100. The audio circuit 160 may transmit the electrical signal converted from the received audio data to the speaker 161, and convert the electrical signal into a sound signal for output by the speaker 161. The communication terminal 100 may also be provided with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 162 converts the collected sound signal into an electrical signal, converts the electrical signal into audio data after being received by the audio circuit 160, and outputs the audio data to the RF circuit 110 to be transmitted to, for example, another terminal or outputs the audio data to the memory 120 for further processing. In this application, the microphone 162 may capture the voice of the user.

Wi-Fi belongs to a short-distance wireless transmission technology, and the communication terminal 100 may help a user to send and receive e-mails, browse webpages, access streaming media, and the like through the Wi-Fi module 170, which provides a wireless broadband internet access for the user.

The processor 180 is a control center of the communication terminal 100, connects various parts of the entire terminal using various interfaces and lines, and performs various functions of the communication terminal 100 and processes data by running or executing software programs stored in the memory 120 and calling data stored in the memory 120. In some embodiments, processor 180 may include one or more processing units; the processor 180 may also integrate an application processor, which mainly handles operating systems, user interfaces, applications, etc., and a baseband processor, which mainly handles wireless communications. It will be appreciated that the baseband processor described above may not be integrated into the processor 180. In the present application, the processor 180 may run an operating system, an application program, a user interface display, and a touch response, and the processing method described in the embodiments of the present application. Further, the processor 180 is coupled with the display unit 130.

And the bluetooth module 181 is configured to perform information interaction with other bluetooth devices having a bluetooth module through a bluetooth protocol. For example, the communication terminal 100 may establish a bluetooth connection with a wearable electronic device (e.g., a smart watch) having a bluetooth module via the bluetooth module 181, so as to perform data interaction.

The communication terminal 100 also includes a power supply 190 (such as a battery) to power the various components. The power supply may be logically connected to the processor 180 through a power management system to manage charging, discharging, power consumption, etc. through the power management system. The communication terminal 100 may also be configured with power buttons for powering the terminal on and off, and for locking the screen.

Fig. 2 is a block diagram of a software configuration of the communication terminal 100 according to the embodiment of the present invention.

The layered architecture divides the software into several layers, each layer having a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the Android system is divided into four layers, an application layer, an application framework layer, an Android runtime (Android runtime) and system library, and a kernel layer from top to bottom.

The application layer may include a series of application packages.

As shown in fig. 2, the application package may include applications such as camera, gallery, calendar, phone call, map, navigation, WLAN, bluetooth, music, video, short message, etc.

The application framework layer provides an Application Programming Interface (API) and a programming framework for the application program of the application layer. The application framework layer includes a number of predefined functions.

As shown in FIG. 2, the application framework layers may include a window manager, content provider, view system, phone manager, resource manager, notification manager, and the like.

The window manager is used for managing window programs. The window manager can obtain the size of the display screen, judge whether a status bar exists, lock the screen, intercept the screen and the like.

The content provider is used to store and retrieve data and make it accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

The view system includes visual controls such as controls to display text, controls to display pictures, and the like. The view system may be used to build applications. The display interface may be composed of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide a communication function of the communication terminal 100. Such as management of call status (including on, off, etc.).

The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and the like.

The notification manager enables the application to display notification information in the status bar, can be used to convey notification-type messages, can disappear automatically after a short dwell, and does not require user interaction. Such as a notification manager used to inform download completion, message alerts, etc. The notification manager may also be a notification that appears in the form of a chart or scroll bar text at the top status bar of the system, such as a notification of a background running application, or a notification that appears on the screen in the form of a dialog window. For example, text information is prompted in the status bar, a prompt tone is given, the communication terminal vibrates, and an indicator light flashes.

The Android Runtime comprises a core library and a virtual machine. The Android runtime is responsible for scheduling and managing an Android system.

The core library comprises two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The application layer and the application framework layer run in a virtual machine. And executing java files of the application program layer and the application program framework layer into a binary file by the virtual machine. The virtual machine is used for performing the functions of object life cycle management, stack management, thread management, safety and exception management, garbage collection and the like.

The system library may include a plurality of functional modules. For example: surface managers (surface managers), Media Libraries (Media Libraries), three-dimensional graphics processing Libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), and the like.

The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

The media library supports a variety of commonly used audio, video format playback and recording, and still image files, among others. The media library may support a variety of audio-video encoding formats, such as MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, and the like.

The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is a layer between hardware and software. The inner core layer at least comprises a display driver, a camera driver, an audio driver and a sensor driver.

The following exemplifies the workflow of the software and hardware of the communication terminal 100 in connection with capturing a photographing scene.

When the touch screen 131 receives a touch operation, a corresponding hardware interrupt is issued to the kernel layer. The kernel layer processes the touch operation into an original input event (including touch coordinates, a time stamp of the touch operation, and other information). The raw input events are stored at the kernel layer. And the application program framework layer acquires the original input event from the kernel layer and identifies the control corresponding to the input event. Taking the touch operation as a touch click operation, and taking a control corresponding to the click operation as a control of a camera application icon as an example, the camera application calls an interface of an application framework layer, starts the camera application, further starts a camera drive by calling a kernel layer, and captures a still image or a video through the camera 140.

The communication terminal 100 in the embodiment of the present application may be a mobile phone, a tablet computer, a wearable device, a notebook computer, a television, and the like.

Fig. 3 is a schematic diagram for illustrating a user interface on a communication terminal (e.g., communication terminal 100 of fig. 1). In some implementations, a user can open a corresponding application by touching an application icon on the user interface, or can open a corresponding folder by touching a folder icon on the user interface.

The invention provides a device control system, which can remotely control one or more controlled devices through one control device. As shown in fig. 4, the system comprises a control device 401 and at least one controlled device 402;

the control device 401 may be a communication terminal commonly used by a user, such as a mobile phone; the controlled device 402 may be other communication terminal of the user, such as a tablet computer, a smart television, a smart watch, and the like.

The control device 401 displays a display interface of the controlled device in a display window, the user operates in the display window of the control device 401, the control device 401 sends control information indicating the user operates in the display window to the controlled device, and the controlled device 402 executes an operation corresponding to the control information after receiving the control information sent by the control device. According to the embodiment of the invention, the display interface of the controlled equipment can be displayed in the display window of the control equipment, and the operation triggered by the user in the display interface of the control equipment can be converted into the operation on the controlled equipment, so that the interaction process between the equipment is completed, the user can operate the controlled equipment on the control equipment without replacing the equipment, the cross-equipment remote operation is conveniently and quickly completed, and the user experience is improved.

In the embodiment of the present invention, the two devices may be connected by bluetooth or WiFi Direct, and the following two connection modes are introduced respectively:

1. bluetooth connection

Specifically, the Bluetooth connection may be a BLE (Bluetooth Low Energy) connection, and when the device a establishes the Bluetooth connection with the device B, pairing information is sent to the device B in a device list in a Bluetooth setting interface of the device a, and after the device B agrees to pair with the device a, the Bluetooth connection between the device a and the device B is successfully established; similarly, the pairing information may also be sent to the device a in the device list in the bluetooth setting interface of the device B, and after the device a agrees to pair with the device B, the bluetooth connection between the device B and the device a is successfully established.

For example, as shown in fig. 5, taking an example that a device a initiates a bluetooth pairing request to a device B, in a bluetooth setting interface of the device a, the device B is selected in a device list, pairing information is displayed in a display interface of the device B and the device a, and after confirmation is clicked, the device a and the device B are successfully paired.

2. WiFi Direct connection

In the specific implementation, the WiFi Direct connection can be established by successfully pairing the device a and the device B after the WiFi P2P function is opened, and the WiFi Direct connection in the embodiment of the present invention includes two TCP (Transmission Control Protocol) links: the control link can transmit control information, and the streaming media link can transmit streaming media data.

Specifically, device B is selected from a device list of a WiFi P2P function setting interface of device a, device a sends pairing request information to device B, and after device B agrees to the pairing request information of device a, device a and device B successfully establish WiFi Direct connection; similarly, device a may also be selected from the device list of the WiFi P2P function setting interface of device B, device B sends pairing request information to device a, and after device a agrees to the pairing request information of device B, device B and device a successfully establish WiFi Direct connection.

For example, as shown in fig. 6, taking the example that the device a sends the pairing request information to the device B, in the WiFi P2P function setting interface of the device a, the device B is selected from the device list, the pairing request information is sent to the device B, the device B selects to approve the pairing request of the device a, and then the device a and the device B successfully establish the WiFi Direct connection.

After the two devices successfully establish the bluetooth connection or the WiFi Direct connection, the user may set the control device and the controlled device, for example, the user selects the device B in the device list through a device control setting interface in the device a, pulls the control authorization to the device B through the bluetooth or WiFi Direct connection, and when the device B agrees that the device a controls the device B, takes the device a initiating the control authorization process as the control device and takes the device B agreeing to control the device B as the controlled device; similarly, the device B may also pull a control authorization to the device a through bluetooth or WiFi Direct connection, and when the device a agrees that the device B controls the device a, the device B is used as a control device and the device a is used as a controlled device.

As shown in fig. 7, taking an example that a user pulls a control authorization from a device a to a device B, the device a selects the device B in the device list, sends request information of the pull control authorization to the device B through bluetooth, and the device B selects to approve the device a to control the device B, so that the device a becomes a control device and the device B becomes a controlled device.

The control device in the embodiment of the invention can remotely receive the push message received by the controlled device, or the control device can also remotely control the controlled device. The control mode of the controlled device by the control device is described in detail below.

Firstly, the control equipment remotely receives push information on the controlled equipment.

After receiving the notification message pushed by other communication equipment, the controlled equipment sends notification information to the control equipment;

after receiving the notification message, the control equipment displays the notification message in a display window; the user can check the notification message through the display window of the control device, and when the user needs to reply to the notification message, the user operates in the display window of the control device; responding to the operation of the user for replying the notification message, and returning reply information input by the user to the controlled equipment by the control equipment;

and the controlled device sends the reply information to other communication devices.

The controlled device can send notification information to the control device through Bluetooth connection or WiFi Direct connection with the control device; and

the control equipment can return the reply information input by the user to the controlled equipment through Bluetooth connection or WiFi Direct connection with the control equipment;

it should be noted that the connection mode used by the controlled device to send the notification message may be the same as or different from the mode used by the control device to return the reply message to the controlled device.

According to the embodiment of the invention, the Bluetooth connection is adopted to realize that the control equipment remotely receives the push information on the controlled equipment, so that simple control operation between the control equipment and the controlled equipment can be realized in a low-power consumption state, and the user experience is improved under the condition that the normal use of the equipment is not influenced.

The display window of the control device is a display window corresponding to a device control application installed in the control device, and in the display window of the device control application, the user can see the notification information sent by the controlled device and can perform a reply operation.

Specifically, the message pushed by the other communication device received by the controlled device may be a message sent by the other communication device through a short message, other instant messaging application program, or the like.

For example, as shown in fig. 8, the control device a receives the notification message sent by the controlled device B, and displays the notification message in the display window of the device control application, and the user may input the reply information in the input box of the display window.

After receiving the reply information returned by the control equipment, the controlled equipment sends the reply information to other communication equipment;

in specific implementation, the controlled device calls a reply interface corresponding to the notification message, and sends reply information returned by the control device to other communication devices through the reply interface.

And secondly, the control equipment remotely controls the controlled equipment.

The user selects to control the controlled equipment, and the control equipment sends a remote control instruction to the controlled equipment;

the user may select to control the controlled device at any time, or the user may select to control the controlled device after the control device receives the notification information sent by the controlled device.

Specifically, the control device may send the remote control instruction to the controlled device in the following two ways:

in the mode 1, the control equipment sends a remote control instruction to the controlled equipment through the Bluetooth connection with the controlled equipment;

and in the mode 2, the control equipment sends a remote control instruction to the controlled equipment through WiFi Direct connection with the controlled equipment.

It should be noted that, if the control device sends the remote control instruction to the controlled device through the bluetooth connection, and the WiFi Direct connection is not established between the current control device and the controlled device, after the controlled device receives the remote control instruction sent by the control device, the WiFi P2P function is turned on, and the WiFi Direct connection is established with the control device, where the WiFi Direct connection includes two TCP links: a control link and a streaming media link;

after receiving the remote control instruction, the controlled device executes screen recording operation and sends a recorded display interface to the control device;

in specific implementation, after receiving a remote control instruction sent by the control device, the controlled device starts a screen recording function, records a current display interface of the controlled device into an H264 video and an AAC audio, and sends the recorded display interface to the control device through a streaming media link in a WiFi Direct connection with the control device.

In one possible implementation manner, after receiving a display interface through a streaming media link in a WiFi Direct connection between controlled devices, a control device displays the display interface in a display window;

the display window of the control device is a display window corresponding to a device control application installed in the control device, and in the display window of the device control application, a user can see and operate the display interface sent by the controlled device.

For example, as shown in fig. 9, the controlled device B records a current display interface, and sends the recorded display interface to the control device a, and the control device a displays the received display interface of the controlled device B in a display window.

In this way, in the display window of the control device, the user can see the display interface of the controlled device B, and the user can trigger the touch operation in the display window of the control device, thereby realizing the remote control of the controlled device B;

in implementation, in response to an operation triggered by a user in a display window, the control device sends control information for indicating the operation of the user in the display window to the controlled device;

the control information includes some or all of the following information:

the operation position is an actual position of an operation triggered by a user in a display area of the control equipment; or the operation position is a position corresponding to the operation position of the user in the display area of the control device mapped to the display interface of the controlled device;

the operating state may include pressing, releasing, sliding;

the triggering operation time is the time when the user triggers the operation;

in specific implementations, the trigger operation time can be represented by a long integer, such as 1576832191665 (2019.12.20.16:56:33.200), wherein 1576832191665 is the time difference between 2019.12.20.16:56:33.200 and 1979.1.1.0:0: 0.000;

the operation numbers are numbers of different positions corresponding to the operations triggered by the user at the same time;

for example, the user has triggered a single click operation in the display area, at point 1, pressed at 2019.12.20.16:56:33.200, at (1, 2); the time point of release at point 1 is 2019.12.20.16:56:33.400, and the position of release is (1,2), then the control information is { point 1, press, 1576832191665, (1,2) }; { Point 1, unclamp, 1576832191865, (1,2) }.

In a specific implementation, in response to an operation of a user in a display window, the control device may record and package the operation of the user as a data structure InputEvent- > { id, status, time, location x, location y }, where id is an operation number, status is an operation state, time is a trigger operation time, and (location x, location y) is an operation position.

The following is described for two different operation positions in the control information:

mode 1, the operation position in the control information is the actual position of the operation triggered by the user in the display area of the control device.

The control device sends the actual position of the operation triggered by the user in the display area to the controlled device as the operation position, and then the controlled device determines the position of the operation position in the control information when the operation position is mapped to the display interface of the controlled device according to the aspect ratio of the display area of the control device to the display interface of the controlled device after receiving the control information sent by the control device, and executes the corresponding operation at the determined position when the operation position is mapped to the display interface of the controlled device.

For example, the operation position in the control information is an actual position (2, 2) of an operation triggered by a user in the display area of the control device, the aspect ratio of the display area of the control device to the display interface of the controlled device is 1:1.5, and after receiving the control information, the controlled device determines that the position when the operation position in the control information is mapped to the display interface of the controlled device is (3, 3).

In the mode 2, the operation position in the control information is a position corresponding to the operation position of the user in the display area of the control device mapped to the display interface of the controlled device.

The control device determines that an operation position of a user in a display area of the control device is mapped to a corresponding position in a display interface of the controlled device according to an aspect ratio of the display area of the control device to the display interface of the controlled device, and then sends the determined position mapped to the corresponding position in the display interface of the controlled device to the controlled device as the operation position, so that the controlled device directly executes corresponding operation at the operation position after receiving control information sent by the control device.

For example, if the operation position of the user in the display area of the control device is (3,6), and the aspect ratio of the display area of the control device to the display interface of the controlled device is 1:2, it is determined that the operation position of the user in the display area of the control device is mapped to the corresponding position in the display interface of the controlled device as (6,12), and the position (6,12) is sent to the controlled device as the operation position in the control information.

The types of the triggering operation of the user in the display window of the control device in the embodiment of the invention include but are not limited to:

single click operation, long press operation, double click operation and sliding operation;

the following description is directed to these several different operations, respectively:

1. a user triggers a single-click operation in a display window of the control equipment;

if the user triggers a single click operation in the display area of the control device, the control information corresponding to the single click operation is as follows: { touch point number, press, time t1, touch position }, { touch point number, release, time t2, touch position };

after the controlled device receives the control information, if the time interval of pressing and releasing in the operation of the same point in the control information is not larger than a preset threshold value, determining that the operation triggered by a user in a display window of the control device is a click operation, and executing the control operation corresponding to the click operation;

for example, if the user triggers a single-click operation at point (1,2) in the display area of the control device, the control device determines that the control information corresponding to the single-click operation is: { point 1, press, 1576832191665, (1,2) }, { point 1, release, 1576832191865, (1,2) }; assuming that the first preset threshold is 350ms, since the operation time interval between the pressing and releasing of the operation on the point 1 in the control message is 200ms, which is less than 350ms, the controlled device determines that the point 1 is in a single-click operation.

And the controlled device determines that the single-click operation is triggered at the (1,2) position, and the controlled device triggers the corresponding function after executing the single-click operation at the (1,2) position of the display interface.

2. A user triggers a long-press operation in a display window of the control equipment;

if the user triggers a long press operation in the display area of the control device, the control information corresponding to the long press operation is: { touch point number, press, time t1, touch position }, { touch point number, release, time t2, touch position };

after the controlled equipment receives the control information, if the time interval between pressing and releasing in the operation of the same point in the control information is greater than a first preset threshold, determining that the operation triggered by a user in a display window of the control equipment is long pressing, and executing the control operation corresponding to the long pressing operation;

for example, if the user triggers a long-press operation at point (1,2) in the display area of the control device, the control information corresponding to the long-press operation is: { point 1, press, 1576832191665, (1,2) }; { point 1, loose, 1576832192365, (1,2) }; assuming that the first preset threshold is 350ms, since the operation time interval between the pressing and releasing of the operation on the point 1 in the control message is 700ms, which is greater than 350ms, the controlled device determines that the point 1 is in the long-press operation.

And the controlled equipment determines that the long-press operation is triggered at the (1,2), and the controlled equipment triggers the corresponding function after executing the long-press operation at the (1,2) position of the display interface.

3. A user triggers double-click operation in a display window of the control equipment;

if the user triggers a double-click operation in the display area of the control device, the control information corresponding to the double-click operation is: { touch point number 1, press, time t1, touch position 1}, { touch point number 1, release, time t2, touch position 1}, { touch point number 2, press, time t3, touch position 2}, and { touch point number 2, release, time t4, touch position 2 }; the touch position 1 may be the same as or different from the touch position 2;

after the controlled device receives the control information, if the time interval of the two click operations is not greater than a second preset threshold, determining that the operation is a double-click operation, and executing a control operation corresponding to the double-click operation;

for example, if the user triggers a double-click operation at the (1,2) point in the display area of the control device, the control information corresponding to the double-click operation is: { point 1, press, 1576832191665, (1,2) }; { point 1, loose, 1576832191865, (1,2) }; { point 1, press, 1576832191965, (1,2) }; { point 1, loose, 1576832192115, (1,2) }; assuming that the second preset threshold is 400ms, the time interval between two single-click operations at point 1 is 300ms, which is less than 400ms, and the controlled device determines that the operation is a double-click operation.

And the controlled device determines that the double-click operation is triggered at the (1,2), and the controlled device triggers the corresponding function after executing the double-click operation at the (1,2) position of the display interface.

4. A user triggers a sliding operation in a display window of the control equipment;

if the user triggers a sliding operation in the display area of the control device, the control information corresponding to the sliding operation is: { touch point number, press, time t1, touch position 1}, and { touch point number, release, time t2, touch position 2 };

after receiving the control information, the controlled equipment determines that the operation triggered by the user in the display window of the control equipment is sliding operation and executes the control operation corresponding to the sliding operation according to the operation state and the operation position in the control information if the operation position corresponding to the pressing state and the releasing state changes;

when a user triggers a sliding operation between points (1,2) to (9,12) in the display area of the control device, the control information corresponding to the sliding operation is { point 1, press, 1576832191665, (1,2) }; { point 1, loose, 1576832192265, (9,12) }; after receiving the control information, the controlled device determines that a sliding operation is triggered between the (1,2) point and the (9,12) point, and then the controlled device triggers a corresponding function after executing the sliding operation at the position between the (1,2) point and the (9,12) point of the display interface.

It should be noted that the first preset threshold and the second preset threshold may be empirical thresholds of those skilled in the art.

The device control method provided by the invention is further described in a complete embodiment as follows:

the control device A and the controlled device B are connected through Bluetooth, and after the controlled device B receives the notification information pushed by other communication devices, the controlled device B sends the notification information to the control device A through BLE.

After receiving the notification message sent by the controlled device B, the control device a starts a device control application program, and selects whether to perform device control on the device B in the device control application program, and if the control device a selects not to perform device control on the controlled device B, the control device a can simply reply to the notification message sent by the controlled device B.

If the control device a selects to perform device control on the controlled device B, the control device a sends a remote control instruction to the controlled device B, the controlled device B starts a WiFi P2P function on the controlled device B after receiving the remote control instruction sent by the control device a, and the control device a and the controlled device B establish a WiFi Direct connection.

Specifically, the control device a and the controlled device B establish two TCP links, which are a control link and a streaming media link, respectively.

After the WiFi connection is established between the control device A and the controlled device B, the controlled device B starts a screen recording function, a display interface is used as a media source to establish streaming media recording, and the display interface is recorded as H264 video and AAC audio.

And the controlled device B sends the recorded display interface to the control device A through a streaming media link in the WiFi Direct connection, and the control device A displays the received display interface of the controlled device B in a display window of a device control application program, so that the current display interface of the controlled device B can be seen and operated in the control device A.

The control device A responds to the operation triggered by the user in the display window of the device control application program, and sends the control information for representing the operation of the user in the display window to the controlled device B through the control link in the WiFi Direct connection.

After receiving the control information sent by the control device a, the controlled device B converts the control information into an input event of the controlled device B, and the controlled device B invokes an interface corresponding to the input event to execute an operation corresponding to the control information.

The control device A responds to the ending control operation triggered by the user in the device control application program, sends a stopping instruction to the controlled device B, the controlled device B closes the WiFi P2P function, stops sending the display interface to the control device A, and accordingly enables the control device A and the controlled device B to be disconnected in WiFi and recover to the BLE communication mode.

As shown in fig. 10, a flowchart of a method for a control device to remotely receive push information on a controlled device according to an embodiment of the present invention includes the following steps:

step S1001, the controlled device receives notification information pushed by other communication devices;

step S1002, the controlled device sends the notification information to the control device through Bluetooth;

step S1003, the control device displays the notification information in the display window;

step S1004, the control device responds to the operation of the user for replying the notification information and sends the reply information input by the user to the controlled device;

step S1005, the controlled device sends the reply information to the other communication device.

As shown in fig. 11, a flowchart of a method for a control device to remotely control a controlled device according to an embodiment of the present invention includes the following steps:

step S1101, the control device responds to the operation triggered by the user and used for controlling the controlled device, and sends a remote control instruction to the controlled device through the Bluetooth connection;

step S1102, WiFi Direct connection is established between the control equipment and the controlled equipment;

step S1103, the controlled device executes screen recording operation;

step S1104, the controlled device sends the recorded display interface to the control device through the streaming media link in the WiFi Direct connection;

step S1105, the control device displays the display interface of the controlled device in the display window;

step S1106, the control device responds to the operation triggered by the user in the display window and sends control information used for expressing the operation of the user in the display window to the controlled device through a control link in WiFi Direct connection;

in step S1107, the controlled device executes an operation corresponding to the control information.

As shown in fig. 12, an embodiment of the present invention provides a control terminal, which may be applied to a control device, including a touch screen 1201, a display screen 1202, and a processor 1203;

the touch screen 1201 is used for receiving touch operation from a user;

the display screen 1202 is used for displaying a display interface of the controlled device in a display window;

the processor 1203 is configured to:

In one possible implementation, the processor 1203 is further configured to:

before a display interface of controlled equipment is displayed in a display window, responding to an operation which is triggered by a user and used for controlling the controlled equipment, sending a remote control instruction to the controlled equipment, so that the controlled equipment executes screen recording operation after receiving the remote control instruction, and sending the recorded display interface to the control equipment;

and receiving the display interface sent by the controlled equipment.

In a possible implementation, the processor 1203 is specifically configured to:

sending a remote control instruction to the controlled device through the Bluetooth connection with the controlled device; or

And sending a remote control instruction to the controlled equipment through WiFi Direct connection with the controlled equipment.

In a possible implementation, the processor 1203 is specifically configured to:

In one possible implementation, the processor 1203 is further configured to:

after receiving notification information sent by the controlled device through Bluetooth connection, displaying the notification information in the display window; the notification information is information pushed by other communication equipment and received by the controlled equipment;

and responding to the operation of replying the notification information by the user, returning reply information input by the user to the controlled equipment, so that the controlled equipment sends the reply information to the other communication equipment.

As shown in fig. 13, an apparatus control device according to an embodiment of the present invention may be applied to a control apparatus, and includes:

the display module 1301 is configured to display a display interface of the controlled device in a display window;

a sending module 1302, configured to send, in response to an operation triggered by a user in the display window, control information indicating that the user operates in the display window to the controlled device, so that the controlled device executes an operation corresponding to the control information.

In one possible embodiment, the display module 1301 is further configured to:

before a display interface of controlled equipment is displayed in a display window, responding to an operation which is triggered by a user and used for controlling the controlled equipment, sending a remote control instruction to the controlled equipment, so that the controlled equipment executes screen recording operation after receiving the remote control instruction, and sending the recorded display interface to the control equipment; and receiving the display interface sent by the controlled equipment.

In one possible implementation, the sending module 1302 is further configured to:

In a possible implementation manner, the sending module 1302 is specifically configured to:

As shown in fig. 14, the embodiment of the present invention provides a communication terminal, which can be applied to a controlled device, including a touch screen 1401, a display screen 1402 and a processor 1403;

the touch screen 1401 is used for receiving touch operations from a user;

the display screen 1402 is used for displaying a display interface;

the processor 1403 is configured to:

and executing the operation corresponding to the control information.

In a possible implementation, the processor 1403 is further configured to:

after receiving a remote control instruction sent by the control equipment, executing screen recording operation; the remote control instruction is sent by the control equipment after responding to the operation triggered by the user and used for controlling the controlled equipment;

and sending the recorded display interface to the control equipment.

In a possible implementation, the processor 1403 is specifically configured to:

As shown in fig. 15, an apparatus control device provided in an embodiment of the present invention may be applied to a controlled apparatus, and includes:

a receiving module 1501, configured to receive control information sent by a control device and used for indicating a user to operate in a display window;

an executing module 1502 is configured to execute an operation corresponding to the control information.

In a possible implementation, the receiving module 1501 is further configured to:

In a possible implementation, the receiving module 1501 is specifically configured to:

Based on the same inventive concept, an embodiment of the present invention provides an apparatus control method, applied to control an apparatus, as shown in fig. 16, including the following steps:

step S1601, the control device displays a display interface of the controlled device in a display window;

step S1602, in response to an operation triggered by the user in the display window, the control device sends control information indicating the operation of the user in the display window to the controlled device, so that the controlled device performs an operation corresponding to the control information.

responding to an operation triggered by a user and used for controlling a controlled device, wherein the control device sends a remote control instruction to the controlled device, so that the controlled device executes screen recording operation after receiving the remote control instruction, and sends a recorded display interface to the control device;

In one possible embodiment, the method further comprises:

Based on the same inventive concept, an embodiment of the present invention provides an apparatus control method, applied to a controlled apparatus, as shown in fig. 17, including the following steps:

step 1701, the controlled device receives control information which is sent by the control device and used for representing the operation of a user in a display window of the control device; the control information is sent by the control equipment after responding to the operation triggered by the user in the display window;

in step S1702, the controlled device performs an operation corresponding to the control information.

In one possible embodiment, the method further comprises:

Since the communication terminal and the computer storage medium in the embodiment of the present invention may be applied to the processing method, reference may also be made to the above method embodiment for obtaining technical effects, and details of the embodiment of the present invention are not described herein again.

Those of ordinary skill in the art will understand that: all or a portion of the steps of implementing the above-described method embodiments may be performed by hardware associated with program instructions. The program may be stored in a computer-readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that these are by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.

Claims

1. A control device is characterized by comprising a touch screen, a display screen and a processor;

the touch screen is used for receiving touch operation from a user;

the processor is configured to:

2. The control device of claim 1, wherein the processor is further configured to:

before a display interface of the controlled equipment is displayed in a display window, responding to an operation triggered by a user and used for controlling the controlled equipment, sending a remote control instruction to the controlled equipment, so that the controlled equipment executes screen recording operation after receiving the remote control instruction, and sending the recorded display interface to the control equipment;

and receiving the display interface sent by the controlled equipment.

3. The control device of claim 2, wherein the processor, when sending the remote control instruction to the controlled device, is specifically configured to: sending a remote control instruction to the controlled device through the Bluetooth connection with the controlled device; or, a remote control instruction is sent to the controlled device through the WiFi Direct connection with the controlled device;

the processor, when receiving the display interface sent by the controlled device, is specifically configured to: receiving the display interface sent by the controlled device through a streaming media link in WiFi Direct connection with the controlled device;

when sending control information to the controlled device, the processor is specifically configured to: and sending the control information to the controlled equipment through a control link in WiFi Direct connection with the controlled equipment.

4. The control device of claim 1, wherein the processor is further configured to:

5. The control device according to any one of claims 1 to 4, wherein the control information includes part or all of the following information:

6. A controlled device is characterized by comprising a touch screen, a display screen and a processor;

the touch screen is used for receiving touch operation from a user;

the display screen is used for displaying a display interface;

the processor is configured to:

and executing the operation corresponding to the control information.

7. The controlled device of claim 6, wherein the processor is further configured to:

and sending the recorded display interface to the control equipment.

8. The controlled device of claim 7, wherein the processor, upon receiving a remote control instruction sent by the control device, is specifically configured to: receiving a remote control instruction sent by the control equipment through Bluetooth connection with the control equipment; or, receiving a remote control instruction sent by the control equipment through WiFi Direct connection with the control equipment;

when the recorded display interface is sent to the control device, the processor is specifically configured to: sending the display interface to the control equipment through a streaming media link in WiFi Direct connection with the control equipment;

when receiving control information sent by the control device, the processor is specifically configured to: and receiving the control information sent by the control equipment through a control link in the WiFi Direct connection with the control equipment.

9. An apparatus control method, characterized in that the method comprises:

10. An apparatus control method, characterized in that the method comprises: