CN110517470B

CN110517470B - Remote control method, equipment and system

Info

Publication number: CN110517470B
Application number: CN201910631841.5A
Authority: CN
Inventors: 刘野; 张科; 刘鹏举
Original assignee: Huawei Technologies Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2019-07-12
Filing date: 2019-07-12
Publication date: 2020-11-17
Anticipated expiration: 2039-07-12
Also published as: CN110517470A

Abstract

The embodiment of the application discloses a remote control method, a device and a system, relates to the field of electronic equipment, and solves the problems that the remote control precision is low, and a cursor in a display screen of display equipment cannot be quickly and accurately controlled to move to a corresponding position of an operation content desired by a user. The specific scheme is as follows: the control device receives the operation of a user and transmits a high-frequency microwave signal to the display device in response to the operation; the display device receives the high-frequency microwave signal through the antenna array arranged on the display screen, and determines the position of the control device pointing to the display screen according to the irradiation area of the high-frequency microwave signal in the antenna array, so that the corresponding event can be executed on the control at the position of the control device pointing to the display screen on the first interface displayed on the display screen in response to the operation, and the second interface is displayed.

Description

Remote control method, equipment and system

Technical Field

The present application relates to the field of electronic devices, and in particular, to a remote control method, device and system.

Background

Large-screen display devices have been commonly used to display scenes, such as large-screen televisions, commercial large-screen displays, and display projectors. And people have higher and higher requirements on the operation and control of display equipment. For example, the content displayed on the display screen of the display device can be directly manipulated by using the touch screen technology. Specifically, a resistor-capacitor circuit (RC circuit) may be formed by a transparent line having a vertical row and a vertical column in a display screen of the display device, and a reference waveform may be output by inputting a signal to the RC circuit formed by the transparent line. When a user touches the display screen, the RC circuit formed by the human body and the transparent circuit forms a parallel capacitor, which can cause the change of the reference waveform. The display device can calculate the position touched by the user and the operation content according to the change of the reference waveform, and respond according to the position touched by the user and the operation content, so that the user can control the display device. Although the control method has higher precision, the remote control cannot be realized.

In order to realize the remote control of the display device, the remote control method can be used for realizing the remote control. For example, remote control of the display device can be realized by an air mouse (air mouse for short). Specifically, the air mouse can collect the track and the direction of the movement of the air mouse controlled by the user by using a built-in gravity sensor, an acceleration sensor or a touch panel according to the operation of the user. Then, the mouse can calculate the direction and distance of the cursor on the display device according to the track and the direction. The air mouse can also collect the operation of the user on the keys on the air mouse. The air mouse transmits the calculated direction and distance of the cursor on the display device, which need to move, and the operation of the user on the air mouse keys to the display device through Bluetooth or infrared rays. The display device can respond correspondingly according to the received information. However, such remote control has low precision, and cannot quickly and accurately control the cursor in the display screen of the display device to move to the corresponding position of the content that the user wants to operate.

Disclosure of Invention

The embodiment of the application provides a remote control method, a device and a system, and solves the problems that the remote control precision is low, and a cursor in a display screen of a display device cannot be quickly and accurately controlled to move to a corresponding position of a content which a user wants to operate.

In a first aspect of embodiments of the present application, a remote control method is provided, which may be applied to a remote control system including a control device and a display device. The display device comprises a display screen provided with an antenna array, and a first interface is displayed on the display screen. The method can comprise the following steps: the control equipment receives the operation of a user; the control device responds to the operation and transmits a high-frequency microwave signal to the display device; the display equipment receives the high-frequency microwave signal through the antenna array; the display equipment determines the position of the control equipment pointing to the display screen according to the irradiation area of the high-frequency microwave signal in the antenna array; and the display equipment responds to the operation, executes a corresponding event to a control at the position of the control equipment pointing to the display screen on the first interface, and displays a second interface.

In this way, when the user wants to remotely control the display device, the control device transmits a high-frequency microwave signal to the display device, the display device receives the high-frequency microwave signal through the antenna array, and determines the position the user wants to control and the operation the user wants to perform according to the high-frequency microwave signal, so that the display device responds. The high-frequency microwave signal in the method has the characteristic of rapidness and accuracy, and the display device can know the pointed position of the user and the operation to be realized in real time through the high-frequency microwave signal, so that the method realizes the remote control of the user on the display device and simultaneously improves the accuracy of the control.

With reference to the first aspect, in one possible implementation manner, an antenna array includes a plurality of array units; before the display device determines the position of the control device pointing to the display screen according to the irradiation area of the high-frequency microwave signal in the antenna array, the method further comprises the following steps: the display equipment determines a plurality of first array units from the antenna array according to the high-frequency microwave signals, wherein the first array units are array units with changed electrical parameters in the antenna array; the display device determines the illumination area according to the positions of the first array units in the antenna array. Therefore, the display device can accurately determine the irradiation area of the display screen pointed by the user according to the received high-frequency microwave signal.

With reference to the first aspect and the foregoing possible implementation manners, in another possible implementation manner, the determining, by the display device, a position where the control device points at the display screen according to an irradiation area of the high-frequency microwave signal in the antenna array includes: the display equipment determines the geometric center of the irradiation area, and determines the position of the control equipment pointing to the display screen according to the geometric center of the irradiation area; or the display device divides the plurality of first array units into at least two array unit groups according to the magnitude of the change of the electrical parameter of each first array unit in the plurality of first array units; determining the geometric center of an area formed by all first array units in the array unit groups aiming at each array unit group in the at least two array unit groups, and obtaining the geometric centers which are in one-to-one correspondence with the at least two array unit groups; and determining the position of the control equipment pointing to the display screen according to the geometric centers which are in one-to-one correspondence with the at least two array unit groups. In this way, after determining the illuminated area, the display device can determine the specific location at which the user is pointing at the display screen.

With reference to the first aspect and the foregoing possible implementation manners, in another possible implementation manner, after the control device receives an operation of a user, the method further includes: the control equipment determines the operation type of the operation, and the operation type is as follows: single click, double click, drag or long press; the control equipment generates a control signal according to the operation type of the operation; the control device is responsive to operation to emit a high frequency microwave signal, including; the control equipment transmits the high-frequency microwave signals corresponding to the control signals according to the control signals, the control signals corresponding to different operation types are different, and the high-frequency microwave signals corresponding to different control signals are different. Therefore, the control device can transmit different high-frequency microwave signals to the display device according to different types of user input operation, so that the user can remotely control the display device in different types.

With reference to the first aspect and the possible implementations described above, in another possible implementation, the display device determines an operation type according to the high-frequency microwave signal; the display device responds to the operation, executes a corresponding event to a control at a position, pointing to the display screen, of the control device on the first interface, and displays a second interface, wherein the method comprises the following steps: and the display equipment responds to the operation according to the operation type, executes a corresponding event to the control at the position where the control equipment points to the display screen, and displays a second interface. Therefore, the display device can determine the operation which the user wants to perform according to different high-frequency microwave signals, execute the corresponding event on the control at the pointed position of the user and display the interface after the execution. Thereby realizing the response of the display device to the user operation.

With reference to the first aspect and the foregoing possible implementation manners, in another possible implementation manner, the display device displays a cursor on the first interface according to a position where the control device points to the display screen. In this way, the display device can visually show the user where the user is pointing at the display screen via the remote control device.

With reference to the first aspect and the possible implementations described above, in another possible implementation, the operating frequency of the high-frequency microwave signal is 30GHz to 3000 GHz. Therefore, the high-frequency microwave signal has the characteristics of small delay, narrow beam and low power consumption, and is more favorable for accurate remote control of a user on the display equipment.

In a second aspect of the embodiments of the present application, a control device is provided, where the control device may be applied to a remote control system, and the remote control system may further include a display device, where the display device includes a display screen provided with an antenna array; the control apparatus includes: at least one input module, a processor, an oscillator, and a transmit antenna; the processor is used for determining a control identifier according to the operation of a user on at least one input module and transmitting the control identifier to the oscillator; the oscillator is used for generating a control signal according to the control identifier and transmitting the control signal to the transmitting antenna; the transmitting antenna is used for transmitting the high-frequency microwave signal according to the control signal, so that the display equipment receives the high-frequency microwave signal through the antenna array.

With reference to the second aspect and the foregoing possible implementation manners, in one possible implementation manner, the determining, by a processor, a control identifier according to an operation of a user on at least one input module includes: the processor is used for determining the operation type of the operation according to the operation of the user on at least one input module, and determining the control identifier according to the operation type of the operation, wherein the operation type is as follows: single click, double click, drag or long press; the transmitting antenna is used for transmitting a high-frequency microwave signal according to a control signal, and comprises: the transmitting antenna is used for transmitting the high-frequency microwave signals corresponding to the control signals according to the control signals, the control marks corresponding to different operation types are different, the control signals corresponding to different control marks are different, and the high-frequency microwave signals corresponding to different control signals are different.

With reference to the second aspect and the foregoing possible implementation manners, in another possible implementation manner, the control device further includes: a filter and a power amplifier disposed between the oscillator and the transmitting antenna; the filter is used for filtering the control signal so as to enable the control signal to work in a preset working frequency band; the amplifier is used for amplifying the control signal filtered by the filter and transmitting the amplified control signal to the transmitting antenna.

With reference to the second aspect and the possible implementations described above, in another possible implementation, the operating frequency of the high-frequency microwave signal is 30GHz to 3000 GHz.

In a third aspect of the embodiments of the present application, there is provided a display device, which may be applied to a remote control system, the remote control system further including a control device, the control device being capable of transmitting a high-frequency microwave signal to the display device according to an operation of a user; the display device includes: the antenna array comprises a display screen and a processor, wherein the display screen is provided with an antenna array, and a first interface is displayed on the display screen; the display screen is used for receiving the high-frequency microwave signals transmitted by the control equipment through the antenna array; the processor is used for determining the position of the control equipment pointing to the display screen according to the irradiation area of the high-frequency microwave signal in the antenna array; responding to the operation, and executing a corresponding event on a control at a position corresponding to the display screen pointed by the control equipment on the first interface; the display screen is further used for displaying a second interface, and the second interface is an interface after responding to the operation and executing the corresponding event on the control at the position where the control equipment points to the display screen on the first interface.

With reference to the third aspect and the foregoing possible implementation manners, in one possible implementation manner, the antenna array includes a plurality of array units; the processor is further used for determining a plurality of first array units from the antenna array according to the high-frequency microwave signals, wherein the first array units are array units with changed electrical parameters in the antenna array; and determining an irradiation area according to the positions of the plurality of first array units in the antenna array.

With reference to the third aspect and the foregoing possible implementation manners, in another possible implementation manner, the processor is configured to determine, according to an irradiation area of the high-frequency microwave signal in the antenna array, a position where the control device is directed to the display screen, and includes: the processor is used for determining the geometric center of the irradiation area and determining the position of the control equipment pointing to the display screen according to the geometric center of the irradiation area; or the processor is used for dividing the plurality of first array units into at least two array unit groups according to the magnitude of the change of the electrical parameter of each first array unit in the plurality of first array units; determining the geometric center of an area formed by all first array units in the array unit groups aiming at each array unit group in the at least two array unit groups, and obtaining the geometric centers which are in one-to-one correspondence with the at least two array unit groups; and determining the position of the control equipment pointing to the display screen according to the geometric centers which are in one-to-one correspondence with the at least two array unit groups.

With reference to the third aspect and the foregoing possible implementation manners, in another possible implementation manner, the processor is further configured to determine an operation type of the operation according to the high-frequency microwave signal; the operation types are as follows: single click, double click, drag or long press; the processor responds to the operation and executes corresponding events to the control at the position corresponding to the position of the display screen pointed by the control equipment on the first interface, and the corresponding events comprise: and the processor responds to the operation according to the operation type and executes a corresponding event to the control at the position corresponding to the position of the control equipment pointing to the display screen on the first interface.

With reference to the third aspect and the foregoing possible implementation manners, in another possible implementation manner, the display screen is further configured to display a cursor on the first interface according to the position, determined by the processor, of the control device pointing to the display screen.

With reference to the third aspect and the foregoing possible implementation manners, in another possible implementation manner, the antenna array is made of an optically transparent material or opaque thin metal wires.

In a fourth aspect of embodiments of the present application, there is provided a remote control system, which may include: the antenna array comprises a control device and a display device comprising a display screen provided with an antenna array; the control equipment is used for receiving the operation of a user, responding to the operation and transmitting a high-frequency microwave signal; the display device is used for receiving the high-frequency microwave signal through the antenna array, determining the position of the control device pointing to the display screen according to the irradiation area of the high-frequency microwave signal in the antenna array, responding to operation, executing corresponding events on a control at the position of the control device pointing to the display screen on a first interface displayed on the display screen at present, and displaying a second interface.

In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, including: computer software instructions; when the computer software instructions are run in the control device, the control device is caused to perform the control device side method of the remote control method as described in the first aspect or any one of the possible implementations of the first aspect.

In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, including: computer software instructions; when the computer software instructions are run in the display device, the control device is caused to perform the display device side method of the remote control method as described in the first aspect or any one of the possible implementations of the first aspect.

In a seventh aspect, an embodiment of the present application provides a computer program product, which when run on a computer, causes the computer to execute the remote control method according to the first aspect or any one of the possible implementation manners of the first aspect, so as to implement a behavior function of a control device.

In an eighth aspect, an embodiment of the present application provides a computer program product, which when run on a computer, causes the computer to execute the remote control method according to the first aspect or any one of the possible implementation manners of the first aspect, so as to implement a behavior function of a display device.

It is to be understood that the control device according to the second aspect provided above, the display device according to the third aspect provided above, the remote control system according to the fourth aspect provided above, the computer-readable storage medium according to the fifth aspect and the sixth aspect provided above, and the computer program product according to the seventh aspect and the eighth aspect are all configured to perform the corresponding method provided above, and therefore, the beneficial effects achieved by the control device according to the second aspect, the computer-readable storage medium according to the fifth aspect and the computer program product according to the seventh aspect and the eighth aspect may refer to the beneficial effects of the corresponding method provided above, and will not be described herein again.

Drawings

Fig. 1 is a schematic diagram illustrating a remote control system according to an embodiment of the present application;

fig. 2 is a schematic structural diagram of an antenna array according to an embodiment of the present application;

fig. 3 is a schematic structural diagram illustrating an antenna array etched on a touch screen according to an embodiment of the present disclosure;

fig. 4 is a schematic flowchart of a remote control method according to an embodiment of the present application;

fig. 5 is a schematic diagram illustrating a correspondence relationship between an operation type and a control signal according to an embodiment of the present application;

fig. 6 is a schematic diagram illustrating a method for determining a position desired to be operated by a user according to an embodiment of the present application;

FIG. 7 is a schematic diagram of a method for determining an operation type according to a high-frequency microwave signal according to an embodiment of the present application;

fig. 8 is a schematic application diagram of a scenario provided in an embodiment of the present application;

fig. 9 is an application diagram of another scenario provided in the embodiment of the present application;

fig. 10 is an application diagram of another scenario provided in the embodiment of the present application;

fig. 11 is an application diagram of another scenario provided in the embodiment of the present application;

FIG. 12 is a schematic diagram of a control apparatus according to an embodiment of the present disclosure;

fig. 13 is a schematic composition diagram of a display device according to an embodiment of the present application.

Detailed Description

The embodiment of the application provides a remote control method which is applied to a remote control system. The remote control system may include a control device and a display device. Wherein the display device may comprise a display screen provided with an antenna array. By the method provided by the embodiment of the application, a user can operate the control equipment to trigger the control equipment to emit the high-frequency microwave signal. The display device can receive the high-frequency microwave signal transmitted by the control device through the antenna array arranged in the display screen, and the position of the control device pointing to the display screen is determined according to the high-frequency microwave signal, so that which part of the content displayed on the display device is to be controlled by a user through the control device can be determined. And then the display device can correspondingly respond to the operation of the user according to the determined position of the control device pointing to the display screen, so that the effect of controlling the display device by using the control device is realized. According to the method, the display equipment can rapidly and accurately determine the specific position of the display screen which is required to be controlled by a user according to the high-frequency microwave signal received by the antenna array arranged in the display screen, and the control precision of remote control is improved.

For example, the display device of the electrical parameter in the embodiment of the present application may be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a television, a commercial display, a projection, or other devices including a display screen, and the display screen has an antenna array disposed therein. The embodiment of the present application does not specifically limit the specific form of the apparatus.

Embodiments of the present application will be described in detail below with reference to the accompanying drawings.

Please refer to fig. 1, which is a schematic diagram illustrating a remote control system according to an embodiment of the present disclosure. As shown in fig. 1, the remote control system may include a control device 100 and a display device 110. The display device 110 may comprise a display screen 111 provided with an antenna array 114.

For example, in the present embodiment, when a user wants to remotely manipulate the display device 110 by using the control device 100, the control device 100 is operated by holding the control device 100 to a desired manipulation position on the display screen 111 of the display device 110. The control device 100 may emit a high-frequency microwave signal upon receiving the operation. In this way, the display device 110 may receive the high-frequency microwave signal through the antenna array 114 disposed on the display screen 111, determine a specific position of the display screen 111 that the user wants to manipulate according to the received high-frequency microwave signal, and accordingly respond to the user's operation, that is, implement remote manipulation.

Specific structures of the control device 100 and the display device 110 are exemplified below.

Referring to fig. 1, the control device 100 may include: at least one input module 101, a processor 102 connected to the at least one input module 101, an oscillator 103 connected to the processor 102, and a transmitting antenna 106 connected to an output of the oscillator 103. The control device 100 further includes a communication line 108 for connecting the modules in the above-described connection manner to realize communication between the modules.

The at least one input module 101 may be at least one physical key, may also be at least one touch pad (touch panel), and of course, may also include at least one physical key and at least one touch pad at the same time. It should be noted that fig. 1 illustrates one input module, and in an actual product, one or more input modules 101 may be used. The embodiments of the present application are not limited thereto.

Illustratively, the control device 100 may receive an operation of a user through the at least one input module 101 and transmit the operation to the processor 102, so that the processor 102 controls the oscillator 103 to generate a control signal according to the operation. The control signal may then be transmitted by the transmitting antenna 106 in the form of a high frequency microwave signal to enable remote control of the display device 110.

In other embodiments of the present application, an antenna switch 107 may also be provided in the control device 100. As shown in fig. 1, the antenna switch 107 may be disposed between the transmitting antenna 106 and the oscillator 103 while being connected to the input module 101. In this scenario, the control device 100 may not include the processor 102.

For example, the oscillator 103 may continuously generate a basic control signal, which may be transmitted to the transmitting antenna 106 through the antenna switch 107. The control device 100 may receive an operation of a user through the at least one input module 101, and send on-off information and/or amplitude information included in the operation to the antenna switch 107, so that the antenna switch 107 adjusts the basic control signal according to the on-off information and/or amplitude information included in the operation, and the transmitting antenna 106 may transmit the adjusted control signal in the form of a high-frequency microwave signal, so as to implement remote control on the display device 110.

It should be noted that, for the transmitting antenna, all fed signals can be transmitted in the form of electromagnetic waves, but only when the strength and the signal-to-noise ratio of the fed signals meet certain requirements and the signals and the receiving end have the same operating frequency band, the electromagnetic waves transmitted by the signals through the transmitting antenna can be effectively received by the receiving end. Therefore, in the control device provided in this embodiment of the present application, in order to ensure that the control signal corresponding to the operation type of the user operation can be transmitted through the transmitting antenna and can be effectively received by the receiving end (e.g., the display device), the filter 104 and the power amplifier 105 may be disposed in the control device 100, so as to ensure that the signal strength and the signal-to-noise ratio of the control signal meet the requirement of being transmitted, and at the same time, the control signal has the same frequency band as that of the receiving end.

Illustratively, as shown in fig. 1, further, the control device 100 may further include a filter 104 and a power amplifier 105 disposed between the oscillator 103 and the transmitting antenna 106.

In some embodiments of the present application, the antenna switch 107 is not included for the case where the control device 100 includes the processor 102. The oscillator 103 may transmit a control signal corresponding to an operation input by a user to the filter 104 for filtering, so that the control signal operates in a preset operating frequency band, and the control signal is adjusted to remove interference of an external signal on the control signal (i.e., to improve a signal-to-noise ratio of the control signal). The filtered control signal may be sent to the power amplifier 105, amplified by the power amplifier 105, and transmitted to the transmitting antenna 106, so that the signal strength of the control signal is increased.

In other embodiments of the present application, the antenna switch 107 is included for the case where the control device 100 does not include the processor 102. The oscillator 103 generates a basic control signal continuously, and before transmitting the basic control signal to the transmitting antenna 106 through the antenna switch 107, the basic control signal may be transmitted to the filter 104 for filtering, so that the basic control signal operates in a preset operating frequency band, and the basic control signal is adjusted to remove interference of an external signal to the basic control signal (i.e., to improve a signal-to-noise ratio of the basic control signal). The filtered base control signal may be sent to the power amplifier 105 and amplified by the power amplifier 105 such that the signal strength of the control signal is increased. And then, the filtered and amplified basic control signal is sent to the antenna switch 107, so that the antenna switch 107 adjusts the basic control signal according to the on-off information and/or the amplitude information included in the operation, and the basic control signal is obtained and transmitted to the transmitting antenna 106 for transmission.

Of course, in some scenarios, if the control signal generated by the oscillator 103 or the basic control signal can be transmitted by the transmitting antenna and effectively received by the receiving end (e.g., the display device 110), the filter 104 and the power amplifier 105 may not be included in the control device 100, or only one of the filter 104 and the power amplifier 105 may be included as required.

In the embodiment of the application, the high-frequency microwave signal can work in the range of 30GHz-3000 GHz. Generally, the control device 100 needs to have a feature convenient for a user to use, and thus, cannot be excessively large in volume. The high-frequency microwave signal with better directivity needs larger antenna aperture and more antenna array units. Therefore, a balance between the selection and design of the transmitting antenna 106 of the control device 100 needs to be taken into account.

Illustratively, assuming that the transmitting antenna 106 is in the form of a microstrip antenna array to realize a high frequency microwave signal of 60GHz, an illuminated area of less than 30cm can be produced on the display device if required at a working distance of 3.5 m. Then, the following characteristics are required for the transmitting antenna 106 in the control device as calculated by simulation: the antenna aperture needs to be more than 85mm, and at least 512 antenna array units are arranged on the antenna array. Therefore, in the manufacturing process of the transmitting antenna 106, the following points need to be noted: because at least 512 antenna array units are needed, the connection of feeding needs to be considered, and more routing leads to higher loss; under the condition of limiting the irradiation area, the farther the used distance is, the narrower the lobe width of the required microwave signal is, and the larger the antenna array unit and the whole aperture are.

That is, the design of the transmitting antenna 106 capable of transmitting the high-frequency microwave signal to the display device 110 by the control device 100 proposed in the embodiment of the present application should be within the protection scope of the present application. The manufacturing process of the transmitting antenna 106 needs to refer to the above points, and is flexibly selected in the practical use process.

With continued reference to fig. 1, in some embodiments, the display device 110 described above may also include a processor 112.

In some embodiments, the processor 112 may include a controller 116 and a coordinate calculation circuit 117.

The controller 116 may also be referred to as a main processing chip. Which may be the neural center and the command center of the display device 110. For example, the controller 116 may determine an operation type corresponding to the high-frequency microwave signal from the high-frequency microwave signal received by the antenna array 114. The coordinate calculation circuit 117 may be configured to determine a position of the display screen 111, etc., at which the control device 100 is directed, based on the high-frequency microwave signal received by the display screen 111 through the antenna array 114 included therein.

It will be appreciated that the associated functions of the coordinate calculation circuit 117 may also be performed by other components within the processor 112, such as the controller 116 shown in FIG. 1. When the function of the coordinate calculation circuit 117 is performed by other components in the processor 112, the coordinate calculation circuit 117 may not be included in the processor 112. In addition, the processor 112 may further be provided with a filter and a power amplifier, which are used for filtering and power amplifying the signal transmitted to the processor 112 by the display screen 111, and may be flexibly selected according to the actual use scenario.

As shown in fig. 1, the display screen 111 may further include a display module 115. The display device 110 can control the display module 115 to display information to be displayed, such as a display interface, through the processor 112. For example, the Display module 115 may be a Liquid Crystal Display (LCD) module, a Light Emitting Diode (LED) array, a projection screen, a wall, or the like.

In addition, a transparent cover 118 for protecting the antenna array 114 may be further disposed in the display 111, and a substrate 119 for placing the antenna array 114 is disposed therein. The transparent cover 118, the antenna array 114, the substrate 119 and the display module 115 are sequentially disposed. The display device 110 may further include a communication line 117 for connecting the modules in the connection manner to realize communication between the modules in the display device 110. It should be noted that, the display 111 provided in the embodiment of the present application may not include the transparent cover 118 or the substrate 119, but may be replaced by another component that can protect the antenna array 114 or a component that can be used to place the antenna array 114.

For example, the display device 110 may receive a high-frequency microwave signal through the antenna array 114, so that the processor 112 determines the position of the control device pointing to the display screen 111 according to the irradiation area of the high-frequency microwave signal in the antenna array 114. The display device 110 can thus accurately acquire the position in the display screen that the user wants to operate. In response to the user operation, the processor 112 may execute a corresponding event to a control at a position where the control device points to the display screen 111, and control the display module 115 to display a corresponding interface.

It should be noted that the antenna array 114 may be composed of a plurality of array units 113. The array unit 113 constituting the antenna array 114 may be a light transparent array unit manufactured according to a transparent thin film antenna technology, or an array unit made of opaque thin metal wires. For example, the array unit may be made of nano-silver or copper wire to reduce the manufacturing cost and improve the brightness of the image displayed by the display module 115. In other embodiments of the present disclosure, a shielding layer may be disposed between the antenna array 114 and the display module 115 (when the display screen 111 is disposed with the substrate 119, the shielding layer may be disposed between the substrate 119 and the display module 115). The shielding layer can reduce interference generated by the display module 115 to the antenna array 114 during operation, and prevent the antenna array 114 from being unable to accurately position the position pointed by the control device, thereby improving the positioning accuracy.

Fig. 2 in the embodiment of the present application provides a schematic structural diagram of an antenna array. As shown in fig. 2, the antenna array may be composed of a plurality of array elements, and the example of the antenna array is illustrated as 9 array elements arranged in 3 × 3. Since the antenna array needs to receive the high-frequency microwave signal, the operating frequency band of the antenna array needs to be similar to or the same as the operating frequency of the high-frequency microwave signal. According to the working frequency of the high-frequency microwave signal, the working wavelength corresponding to the working frequency can be determined, and then the size of the array unit in the antenna array can be determined according to the working wavelength. For example, assuming an operating wavelength λ, the array elements may be microstrip antennas with a side length of λ/2, and the distance between two adjacent array elements may be λ/4. Each array unit of the plurality of array units may transmit a change in a signal received by the array unit to a processor including a coordinate calculation circuit or a controller in the display device through the microstrip transmission line.

It should be noted that the antenna array for receiving the high-frequency microwave signal provided in the embodiment of the present application may be separately provided. For touch-enabled display screens (e.g., referred to as touch screens), the antenna array may also be integrated on an existing touch screen to enable remote control of a display device that includes the touch screen. Illustratively, taking a touch screen as a projected capacitive screen, the composition of the touch screen is shown in fig. 3 (a). For example, the touch panel may be formed by arranging two conductive films (e.g., a conductive film a and a conductive film B) made of Indium Tin Oxide (ITO) side by side, and the two conductive films are not connected to each other. The conductive film a is provided with a plurality of longitudinally connected conductive units (e.g., unfilled conductive units shown in fig. 3 (a)), and the conductive film B is provided with a plurality of transversely connected conductive units (e.g., grid-filled conductive units shown in fig. 3 (a)). The detection signal input terminal may input a detection signal to the conductive film a, and the conductive film B may be coupled to the conductive film a to form a stable capacitance between each conductive unit of the two films. When a user touches the touch screen, because a human body is charged, the capacitance between the conductive units of the two layers of conductive films at the corresponding positions is changed when the user touches the touch screen. By detecting the signal at the signal output end, the position of the conductive unit with the changed capacitance can be determined, and therefore the coordinate of the position where the user touches the touch screen can be determined, and touch control is achieved.

In the embodiment of the present application, the antenna array may be etched on each conductive element disposed on a conductive film (e.g., including a conductive film a and a conductive film B) on the touch screen, so that the touch screen can also receive the high-frequency microwave signal emitted by the control device. For example, as shown in fig. 3 (b), a schematic diagram of the conductive elements after etching the antenna array is shown. Array elements of the antenna array may be etched on the conductive elements, the size of the array elements being adjusted according to the operating wavelength of the high frequency microwave signal. If the operating wavelength is λ, the array elements may be microstrip antennas with a length of λ/2, and the distance between two adjacent array elements may be λ/4. In the case of ensuring the size and pitch of the array elements, the array elements of the antenna array may be etched as many as possible on the conductive elements, and as shown in fig. 3 (b), 4 array elements in total may be etched on one conductive element by 2 × 2. Therefore, the control precision is improved while the remote control on the touch screen is realized.

It should be understood that the above description is only one design of the antenna array provided in the embodiment of the present application, and other antenna array designs capable of receiving the high-frequency microwave signal and transmitting the variation of the electrical parameter generated by the high-frequency microwave signal to the processor as proposed in the embodiment of the present application should be within the protection scope of the present application.

In addition, the structure of the display device 110 illustrated in the present embodiment does not constitute a specific limitation of the device. In other embodiments, display device 110 may include more or fewer components than shown, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. For example, display device 110 may also include one or more of the following modules (not shown in fig. 1): a memory for storing computer executable program code, the executable program code comprising instructions. The processor 112 may execute various functional applications of the display device 110 and data processing by executing instructions stored in the memory. The charge management module is used to receive charging input from the charger and also to power the components of the display device 110. The mobile communication device comprises an antenna, a mobile communication module, a wireless communication module, a modulation and demodulation processor, a baseband processor and the like, and is used for realizing a wireless communication function. A GPU (Graphics Processing Unit), an application processor, and the like, for implementing a display function in cooperation with the display screen 111. The audio module, the loudspeaker, the telephone receiver, the microphone, the earphone interface, the application processor and the like are used for realizing the audio function. Such as music playing, recording, etc.

The remote control method provided by the embodiment of the application can be realized in the remote control system. The remote control method provided by the embodiment of the present application is explained in detail below.

Fig. 4 is a schematic flowchart of a remote control method according to an embodiment of the present application. The method may be applied to the remote control system described above. As shown in fig. 4, the method may include S401-S406.

S401, the control device receives the operation of the user.

When a user wants to remotely control the display apparatus, the user can hold the control apparatus by hand and operate the control apparatus.

Illustratively, in conjunction with fig. 1, the control device 100 may be provided with at least one input module 101, and the operation may be an operation of the at least one input module 101 by a user. For example, the input module 101 may be a physical key. The operation may be an operation of the physical key by the user. For another example, the input module 101 may be a touch pad. The operation may be a user operation on a touch pad.

In some embodiments, it may be only necessary to effect one control of the display device by the control device, such as a single click or a double click. In this embodiment, only one input module 101, such as a physical key, may be disposed in the control device 100, and the user may control the display device 110 by performing a single-click or double-click operation on the physical key.

In some other embodiments, the control device may be used to implement multiple controls on the display device, such as two or more of clicking, double-clicking, dragging (dragging may refer to long-pressing the input module and moving the control device), turning pages and switching screens. The type of operation performed by the user is different for different controls. As in the present embodiment, the operation type may include one of a single click, a double click, a drag, or a long press. The control of page turning and screen cutting can be realized by single click or double click or dragging. In this embodiment, one or more input modules 101 may be included in the control device 100. For example, one or more physical keys may be included in the control device 100, or a touch pad may be included in the control device 100, or one or more physical keys and a touch pad may be included in the control device 100. The user may achieve different controls for the display device 110 by performing different operations on the one or more input modules 101. For example, the control device 100 includes a physical key, and the user can perform a single-click operation on the physical key to realize single-click control on the display device, and perform a double-click operation on the physical key to realize double-click control on the display device. For another example, the control device 100 includes two physical keys, namely a physical key 1 and a physical key 2, so that the user can perform a single-click operation on the physical key 1 to realize single-click control on the display device, and perform a double-click control on the display device through a single-click operation on the physical key 2.

S402, the control device responds to the operation of the user and transmits the high-frequency microwave signal to the display device.

Corresponding to the description in S401, in some embodiments, for a scenario where only one control of the display device needs to be implemented by the control device. Referring to fig. 1, the control device 100 may control the oscillator 103 to generate a control signal and transmit the control signal to the transmitting antenna 106 for transmission after receiving an operation (e.g., a one-click operation) of a user through a physical key.

In other embodiments, the control device may be configured to control the display device in a plurality of different ways. In order to enable the display device to recognize different operations of the user, the control device may determine an operation type corresponding to the operation of the user after receiving the operation of the user. And generating and transmitting a high-frequency microwave signal corresponding to the operation type according to the operation type. For example, the control device may generate a control signal corresponding to the operation type according to the operation type, and transmit the control signal in the form of a high-frequency microwave signal.

For example, in some embodiments of the present application, in conjunction with fig. 1, the input module 101 may send the received operation of the user to the processor 102, and the processor 102 may determine a control identifier corresponding to the operation according to the operation of the user and control the oscillator 103 to generate a control signal corresponding to the control identifier. The oscillator 103 may then transmit the control signal to the transmitting antenna 106, and transmit a high-frequency microwave signal corresponding to the control signal to the display device 110. In this embodiment, the control identifiers corresponding to different operation types are different, different control identifiers may correspond to different control signals, and the high-frequency microwave signals corresponding to different control signals are also different. Different types of control signals can be distinguished through different on-off intervals or signal strengths, and the control signals are used for corresponding to user operations of different operation types.

For example, please refer to fig. 5, which is a schematic diagram illustrating a corresponding relationship between different operation types and control signals according to an embodiment of the present application. As shown in fig. 5, the control flag 1 may correspond to the control signal 1, or the control flag 1 is a flag of the control signal 1. The control signal 1 may be a clock signal comprising an amplitude a during a period T. Assuming that the operation type corresponding to the control identifier 1 is single-click, after the control device receives the single-click operation of the user, the control device may generate the control signal 1 corresponding to the control identifier 1.

Similarly, assume that the operation type corresponding to control flag 2 is a double click. The control identifier 2 may correspond to the control signal 2, or the control identifier 2 is an identifier of the control signal 2. The control signal 2 may be a clock signal comprising two amplitude a clock signals within a period T, wherein the period of occurrence of the amplitude a clock signal is T/2. Then, when the control device receives the double-click operation of the user, the control device may generate a control signal corresponding to the control identifier 2.

Assume that the operation type corresponding to the control identifier 3 is drag. The control flag 3 may correspond to the control signal 3, or the control flag 3 is a flag of the control signal 3. The control signal 3 may be a clock signal comprising an amplitude a/2 during a period T. Then, when the control device receives the drag operation of the user, the control device may generate the control signal corresponding to the control signal 3.

Assume that the operation type corresponding to the control flag 4 is long press. The control mark 4 may correspond to the control signal 4, or the control mark 4 is a mark of the control signal 4. The control signal 4 may be a signal comprising two clock signals of amplitude a/2 during a period T, wherein each clock signal of amplitude a/2 occurs with a period T/2. Then, when the control device receives a long press operation from the user, the control device may generate a control signal corresponding to the control identifier 4.

According to the above-described method example, the control device can generate a corresponding control signal according to the operation type of the operation, from which a corresponding high-frequency microwave signal can be emitted to the display device.

In the implementation method, the control device can generate control signals with different on-off intervals or signal strengths according to different types of operations input by a user, and then transmit different high-frequency microwave signals to the display device.

In other embodiments of the present application, the control device may further implement the correspondence between the control signal and different operations input by the user by changing the on-off interval or the signal strength of the control signal. The method can be realized by arranging an antenna switch on the control equipment.

Illustratively, in conjunction with fig. 1, the oscillator 103 in the control device 100 may continuously generate the same basic control signal, which is filtered by the filter 104 and amplified by the power amplifier 105, and then transmitted by the transmitting antenna 106 in the form of a high-frequency microwave signal. The antenna switch 107 can control the on/off state of the high-frequency microwave signal transmitted by the transmitting antenna 106 by opening and closing. When a user inputs an operation, the antenna switch 107 may receive the operation, performing corresponding on/off control of the transmitting antenna 106. Thus, the high-frequency microwave signal emitted by the transmitting antenna 106 becomes discontinuous, and the high-frequency microwave signal in different on-off states can correspond to different operation types of operations. The display device 110 can receive high frequency microwave signals having different on/off states corresponding to user operations.

In the embodiment of the application, the operating frequency of the high-frequency microwave signal can be between 30GHz and 3000 GHz. The high frequency microwave signal needs to have a good directivity so that the signal emitted by the control device can better reflect the operation performed by the user. For the design requirement of the transmitting antenna, please refer to the above description about the remote control system with reference to fig. 1, which is not repeated herein.

In addition, the embodiment of the application adopts the high-frequency microwave signal to realize remote control, and the external interference on the microwave signal working at higher frequency is far less than that on the signal working at an infrared, Bluetooth or Wi-Fi frequency band. In addition, the high-frequency microwave signal also has the characteristics of small delay, narrow beam and small power consumption, is more favorable for the miniaturization of the transmitting antenna, and can not cause additional radiation damage to human bodies in the using process. Therefore, remote control is achieved through the high-frequency microwave signals, external interference can be effectively avoided, accuracy of remote control is achieved, and control efficiency can be improved.

And S403, the display equipment receives the high-frequency microwave signal through the antenna array.

The display device is provided with an antenna array, and the working frequency band of the antenna array can cover the frequency band of the high-frequency microwave signal transmitted by the control device, so that the high-frequency microwave signal can be received by the antenna array on the display device.

It is understood that after the antenna array receives the high-frequency microwave signal, the array elements on the antenna array can convert the high-frequency microwave signal into electrical signals. For example, the conversion process may be embodied such that, after receiving the high-frequency microwave signal, the electrical parameters of the array elements on the antenna array may be changed accordingly.

S404, the display device determines the position of the control device pointing to the electric parameter display screen according to the irradiation area of the high-frequency microwave signal in the antenna array.

It is understood that the high-frequency microwave signal transmitted by the control device can be projected to a part of the area in the antenna array on the display device in a concentrated manner, and therefore, the display device can determine the irradiation area of the high-frequency microwave signal according to the high-frequency microwave signal received by the antenna array and projected to the antenna array in a concentrated manner. For example, the array elements in the antenna array receiving the high-frequency microwave signal will generate corresponding changes in the electrical parameters, and the display device can determine that the high-frequency microwave signal is projected onto the antenna array to the irradiation area. Wherein the electrical parameter may be current, voltage or other electrical parameter. That is, the irradiation area of the high-frequency microwave signal in the antenna array is determined by whether the electrical parameters of the array elements in the antenna array are changed. As in this embodiment, the array unit having the electrical parameter changed may be referred to as a first array unit.

For example, referring to fig. 1, when the control device 100 transmits a high-frequency microwave signal, in the antenna array 114 of the display screen 111 of the display device 110, the array unit 113 receiving the high-frequency microwave signal converts the received high-frequency microwave signal into an electrical parameter. The antenna array 114 may transmit the electrical parameters of all antenna array elements 113 to the coordinate calculation circuit 117. The coordinate calculation circuit 117 determines the irradiation area according to the variation of the electrical parameters of all the antenna array units 113. For example, the coordinate calculation circuit 117 may determine, as the irradiation region, all array elements (e.g., referred to as first array elements) with the changed electrical parameters in the region where the antenna array is located.

For example, referring to fig. 6, before the antenna array does not receive the high-frequency microwave signal, the array elements do not have the electrical parameter change, as shown in fig. 6 a. After the antenna array receives the high-frequency microwave signal, the electrical parameters of some array elements on the antenna array change, as shown in fig. 6b, and the array elements indicated by black filling may be the array elements with the changed electrical parameters, that is, the first array elements, and these first array elements constitute the irradiation area (e.g., the circular irradiation area shown in fig. 6 c).

After the display device determines the irradiation area, the position of the control device pointing to the electrical parameter display screen can be determined according to the irradiation area.

In some embodiments of the present application, the display device may determine a geometric center of the illuminated area as the position at which the control device is directed to the display screen. For example, referring to fig. 6, after the irradiation region shown as 6c in fig. 6 is determined, the geometric center of the irradiation region, the origin of the circular irradiation region shown as 6d in fig. 6, may be determined as the position where the control device is directed to the display screen.

It can be understood that when the high-frequency microwave signal is irradiated onto the array antenna of the display device, because the power density of the high-frequency microwave signal at different positions in space is different, the signal strength of the high-frequency microwave signal received by the array units at different positions on the array antenna is also different, and the corresponding magnitude of the electrical parameter change is also different. In other embodiments of the present application, the display device may divide the plurality of first array units into at least two array unit groups according to a magnitude of the change in the electrical parameter of each of the plurality of first array units. For each array unit group of the at least two array unit groups, the display device can obtain the geometric centers corresponding to the at least two array unit groups one by determining the geometric centers of the areas formed by all the first array units in the array unit groups, and determine the position of the control device pointing to the electrical parameter display screen according to the plurality of geometric centers.

As shown in fig. 6, after the display device determines the irradiation region according to the high frequency microwave signal, the plurality of first array units in the irradiation region may be grouped. For example, the variation values of the electrical parameters of the array units before and after receiving the high-frequency microwave signal may be compared, and when the variation of the electrical parameter of the first array unit is smaller than a first threshold, the corresponding first array unit is determined as a first array unit group a; when the variation of the electrical parameter of the first array unit is greater than the first threshold value, the corresponding first array unit is determined as a first array unit group B. It should be noted that the smaller the change of the electrical parameter, the stronger the signal intensity of the received high-frequency microwave signal at the array unit is reflected. Conversely, the smaller the change of the electrical parameter is, the weaker the intensity of the high-frequency microwave signal received by the array unit at the corresponding position is. As shown in fig. 6e, array elements numbered 1, 2, 3, and 4 may constitute a first array element group a, and array elements numbered 5, 6, 7, and 8 may constitute a first array element group B. Thus, the display device can determine the geometric center P1 of the first array cell group a based on the positions of the array elements numbered 1, 2, 3, and 4, and similarly, determine the geometric center P2 of the first array cell group B based on the positions of the array elements numbered 5, 6, 7, and 8. Then, in some embodiments, the midpoint of the line connecting P1 and P2 may be determined as the location where the control device is pointed at the electrical parameter display screen as shown at 6f in FIG. 6.

Of course, in other embodiments, since the intensity of the high-frequency microwave signals received by the array elements of the first array element group B is stronger, there is a greater probability that the geometric center of the first array element group B is closer to the position that the user wants to operate. Therefore, a weight corresponding to the change of the electrical parameter of each array unit group after receiving the high-frequency microwave signal can be introduced, for example, the weight of the first array unit group A is set as a₁The weight of the first array unit group B is a₂. A may be set as the intensity of the high frequency microwave signal received by the first array unit group B is stronger₂Greater than a₁Then, the display device may be according to the above-mentioned line connecting P1 and P2 and a in the process of calculating the position of the pointing control device to the display screen₁And a₂Determines the position of the control device pointing at the display of the electrical parameter, for example referred to as P3. For example, assume that the link length from P1 to P2 is in units of 1, a₂＝2*a₁Then the distance P3 from the position P1 where the control device is pointing to the display screen of the electrical parameter and the distance P3 from P2 may be 2:1, that is, the position P3 where the control device is pointing to the display screen of the electrical parameter may be the point on the line connecting P1 and P2 where the distance P1 is 2/3. Therefore, the position of the display device, which is determined according to the high-frequency microwave signal, of the control device pointing to the electric parameter display screen is closer to the point with higher strength of the high-frequency microwave signal, and is also closer to the position which a user wants to operate.

It is understood that the above method for determining the position of the control device pointing to the electrical parameter display screen is only exemplary, and other methods for determining the position of the control device pointing to the electrical parameter display screen based on the embodiments of the present application are within the scope of the present application. And will not be described in detail herein.

S405, the display device displays a cursor on the first interface according to the position of the control device pointing to the electrical parameter display screen.

The display device may have a first interface displayed on its display screen before receiving no high frequency microwave signal transmitted by the control device. Accordingly, the user's operation may be performed based on the content presented on the first interface. That is, the position that the user who is confirmed by the display device wants to operate may be a position on the first interface. And the display equipment can display a cursor at the position corresponding to the first interface according to the determined position of the control equipment pointing to the display screen.

And S406, the display device responds to the operation to execute a corresponding event on the control at the position corresponding to the position of the control device pointing to the electrical parameter display screen on the first interface, and displays a second interface.

Corresponding to the description in S401, in some embodiments, for a scenario where only one control of the display device needs to be implemented by the control device. The display device receives only one kind of high-frequency microwave signal, and after receiving the high-frequency microwave signal and determining the position of the display screen executed by the control device, the display device may execute a corresponding event on the control at the position of the display screen pointed by the control device and display the second interface.

In other embodiments, the control device may be configured to control the display device in a plurality of different ways. The display device receives a plurality of high-frequency microwave signals transmitted by the control device, and different high-frequency microwave signals transmitted by the control device correspond to different operation types one by one. Therefore, the display device may determine the operation type corresponding to the high frequency microwave signal after receiving the high frequency microwave signal, and further determine which control the user wants to implement. For example, the display device may determine on-off information and/or amplitude information of the high-frequency microwave signal according to a change of an electrical parameter generated after the antenna array receives the high-frequency microwave signal, so as to determine an operation type corresponding to the high-frequency microwave signal. For example, referring to fig. 7, the control device may obtain a control signal corresponding to the change of the electrical parameter according to the change of the electrical parameter, and further determine an operation type corresponding to the control signal.

Referring to fig. 7 (a), a method for determining the control signal according to the variation of the electrical parameter is illustrated by taking the electrical parameter as the voltage.

As shown in (a) of fig. 7, when the voltage changes to 0, it can be considered that the corresponding clock signal is at the first level during the period of time; when the voltage changes to be between 0-A, the second level of the corresponding clock signal in the period of time can be considered; when the voltage variation is greater than a, it can be considered that the period corresponds to the third level of the clock signal. And A is a voltage value larger than 0, and the signal amplitude corresponding to the first level is smaller than the signal amplitude corresponding to the second level and smaller than the signal amplitude corresponding to the third level. Thus, the display device can determine, based on the variation of the electrical parameter, a clock signal consisting of the different levels mentioned above, which may be control signals corresponding to different control types.

That is, after the display device receives the high frequency microwave signal, the clock signal corresponding thereto can be determined. Assume that the amplitude of the clock signal corresponding to the first level is 0, the amplitude of the clock signal corresponding to the second level is 1/2a, and the amplitude of the clock signal corresponding to the third level is a. As shown in (b) of fig. 7, when the display device receives a clock signal with amplitude a during time T, it is determined that the user has performed a "click" operation to the display device at this time. When the display device receives two signals of the clock signal with the amplitude A within the time T, wherein the period of the clock signal with the amplitude A is T/2, then the user can be determined to send the operation of double-click to the display device at the moment. When the display device receives a clock signal with the amplitude of A/2 within the time T, the fact that the user sends a 'drag' operation to the display device at the moment can be determined. When the display device receives two signals of the clock signals with the amplitude of A/2 within the time T, wherein the period of the occurrence of each clock signal with the amplitude of A/2 is T/2, then the fact that the user sends the long-press operation to the display device at the moment can be determined.

In response to the user's operation, the display device may execute a corresponding event to a control on the first interface where the control device points to the location corresponding to the electrical parameter display screen. And displaying a second interface corresponding to the event on the display screen.

The embodiment of the application can realize accurate and quick remote control on the display equipment under a plurality of scenes. The following description exemplarily illustrates specific implementations of embodiments of the present application in different scenarios.

In the scenario shown in fig. 8, taking the display device as a computer and the control device including a physical key as an example, the user may open the folder by performing a double-click operation using the control device. As shown in fig. 8 (a), a first interface is displayed on the display screen of the computer, and the first interface may include folders named "camera", "WeChat", "screen recording and screenshot", "magazine lock", and "bluetooth sharing". Assuming that the user wants to open the "camera" folder to view the contents therein, the user can hold the control device by hand, point to the corresponding location in the display screen of the "camera" folder, and double-click a physical key on the control device. The control equipment receives double-click operation input by a user through a physical key, controls the oscillator to generate a control signal, and transmits the control signal in a high-frequency microwave signal mode through the transmitting antenna. The display screen of the computer can receive the high-frequency microwave signal through the antenna array arranged on the display screen, and determines the irradiation area according to the high-frequency microwave signal so as to determine the position which the user wants to operate as a 'camera' folder. Then, as shown in fig. 8 (b), the computer can display a cursor at the position, so that the computer provides the user with an intuitive position of the operated control. In addition, the computer may perform an event corresponding to the "double-click" operation on the "camera" folder, that is, open the folder, and display an interface after performing the operation, such as an interface after opening the "camera" folder shown in (c) of fig. 8.

In the game scenario shown in fig. 9, assuming that the display device includes a display screen and the control device includes a physical key, the user can play a "cut fruit" game using the control device. As shown in fig. 9 (a), in this scenario, a plurality of moving fruit images appear on the display screen rapidly, and the user needs to control the cursor to slide across the fruit images in the display device as much as possible when playing the "cut fruit" game, and the score is higher the more fruit images are slid. It can be understood that in the game scene, the display device is required to respond to the operation of the user quickly and accurately so as to sweep over the fruit image in the quick movement as much as possible. When the method provided by the embodiment of the application is applied to the scene, when the user starts to cut fruit, the user can hold the control device (not shown in fig. 9) by hand and input the operation of "dragging". For example, a physical key is pressed for a long time, and the control device is moved along a line capable of crossing most of the fruit image (as a route indicated by a dotted arrow shown in (a) of fig. 9) until the action of cutting the fruit is finished, and the physical key is released. Thus, the control device can generate a control signal according to the "drag" operation, and continuously transmit the high-frequency microwave signal to the display device along the route of the dotted arrow included in (a) of fig. 9. The display device receives the high-frequency microwave signal through the antenna array arranged on the display screen, obtains a cursor moving line which a user wants to operate through analysis and calculation of a processor in the display device, and executes an event corresponding to the high-frequency microwave signal, namely, cuts off fruits on the line. For example, as shown in fig. 9 (b), the fruit on the moving line is cut off and the juice splashing effect after the fruit is cut off is displayed on the display screen, and the change of the total number of the cut fruits can be displayed on the upper left corner of the display screen.

In the instant communication scenario shown in fig. 10, it is assumed that the user wants to perform handwriting input within the chat frame shown in (a) of fig. 10. When the method provided by the embodiment of the present application is applied to this scenario, a user may hold a control device (not shown in fig. 10) with his hand to input a "drag" operation. For example, the input module on the control device is pressed long and moved against an input area (a simulated handwriting area as shown in fig. 10 (a)) in the chat frame displayed in the display device, so as to input the intended text. For example, as shown in fig. 10 (a), assuming that the user wants to input "Hi" in the chat box, the user can hold the control device with his hand, press the input module long and move in the stroke directions of "H" and "i" in the simulated handwriting area. The control device can generate a corresponding control signal according to the operation input by the user and transmit the control signal in the form of a high-frequency microwave signal. And the display device can receive the high-frequency microwave signal through the antenna array and acquire the operation type input by the user and the line on which the user wants to move the cursor according to the high-frequency microwave signal. So that the display device analyzes the information, determines the contents of the user input, and displays the information that the user wants to input in the input box, as shown in (b) of fig. 10. According to the method provided by the embodiment of the application, the display equipment can accurately receive the user operation and respond, so that the chat frame of the chat software can accurately acquire the information input by the user, and further, the efficient and smooth chat experience is provided for the user.

Please refer to fig. 11, which is a video playing scene provided in an embodiment of the present application. In this scenario, the user may view the video content through the display screen of the display device, or may skip the video frame by moving the progress marker. The user may hold the control device (not shown) with his hand and move the preview mark frame to a desired position of the progress bar, and different positions of the progress bar may correspond to video frames of different moments in the video content, so that the user can directly watch the video at the desired moment. For example, assume that the currently viewed video content is a video frame at the 01:01 th time in the video information (as shown in (a) of fig. 11). And the user wants to jump to the time of 26:01 to continue watching the video, the user can hold the control device by hand to align with the position of the time of 26:01 on the progress bar and input the operation of clicking. The control device receives the operation, and can generate a control signal corresponding to the operation type of the single click and transmit the control signal in the form of a high-frequency microwave signal after confirming that the operation type of the operation is the single click. As shown in (b) of fig. 11, the display screen of the display device receives the high frequency microwave signal through the antenna array provided in the display screen and determines that the operation type is "one click" based on this, it is possible to jump the video frame to a position (e.g., time 26: 01) to which the user wants to point in response to the user's operation and display the video frame at that time.

In this way, the control device generates a high-frequency microwave signal corresponding to the operation according to the operation of the user and transmits the high-frequency microwave signal to the display device, so that the information of the operation of the user can be accurately transmitted to the display device. The display equipment receives the high-frequency microwave signals through a plurality of array units in the antenna array, determines an irradiation area according to the change condition of the electrical parameters caused by the high-frequency microwave signals, and further determines the position of the control equipment pointing to the display screen. Meanwhile, the display device can also determine the operation type of the operation performed by the user according to the high-frequency microwave signal received by the antenna array. The display device is thus able to know the specific location at which the user wants to control the display device and what operation to perform. Then, in response to an operation of the user input control device, the display device may execute a corresponding event when the control device points to a control at a corresponding position of the display screen and display an interface after the corresponding event is executed. By the method, the purposes of improving the control precision, quickly and accurately controlling the cursor in the display screen to move to the corresponding position of the content which the user wants to operate and responding according to the operation of the user in the remote control of the display equipment are achieved.

The above description mainly introduces the solutions provided in the embodiments of the present application from the perspective of the control device and the display device. It will be appreciated that in order to implement the remote control function described above, it includes corresponding hardware structures and/or software modules for performing the respective functions. Those of skill in the art will readily appreciate that the present application is capable of hardware or a combination of hardware and computer software implementing the various illustrative algorithm steps described in connection with the embodiments disclosed herein. Whether a function is performed as hardware or computer software drives hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiment of the present application, the control device and the display device may be divided into functional modules according to the above method examples, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation.

In the case of dividing each functional unit with corresponding functions, fig. 12 shows a possible composition diagram of the control apparatus involved in the above-described embodiment, as shown in fig. 12, the control apparatus includes: at least one input unit 1201 (input units 1-n as shown in fig. 12), a processing unit 1202, and a transmitting unit 1203.

The input unit 1201 is used for receiving an operation input by a user. Illustratively, the at least one input unit 1201 may be used to perform S401 of the remote control method illustrated in fig. 4 described above.

The processing unit 1202 is configured to generate a control signal according to the control identifier, and transmit the control signal to the transmitting unit 1203.

A transmitting unit 1203 is configured to transmit the high-frequency microwave signal according to the control signal, so that the display device receives the high-frequency microwave signal through the antenna array. Illustratively, the processing unit 1202 and the transmitting unit 1203 may be configured to execute S402 of the remote control method shown in fig. 4.

Further, the processing unit 1202 may be further configured to determine an operation type of the operation according to an operation of the user on the at least one input unit, and determine the control identifier according to the operation type of the operation, where the operation type is: single click, double click, drag or long press.

The transmitting unit 1203 may also be configured to transmit the high-frequency microwave signal corresponding to the control signal according to the control signal, where the control identifiers corresponding to different operation types are different, the control signals corresponding to different control identifiers are different, and the high-frequency microwave signals corresponding to different control signals are different. Wherein the working frequency of the high-frequency microwave signal is 30GHz-3000 GHz.

The processing unit 1202 may be further configured to filter the control signal so that the control signal operates in a preset operating frequency band.

The processing unit 1202 may also be configured to amplify the control signal and transmit the amplified control signal to the transmitting unit 1203.

It should be noted that all relevant contents of each step related to the above method embodiment may be referred to the functional description of the corresponding functional unit, and are not described herein again. The control device provided by the embodiment of the application is used for executing the remote control method, so that the same effect as the remote control method can be achieved.

In the case of dividing each functional unit with corresponding functions, fig. 13 shows a possible composition diagram of the display device involved in the above-described embodiment, as shown in fig. 13, the display device includes: a display unit 1301 provided with an antenna array and a processing unit 1302. For example, the display unit 1301 may be the display screen 111 shown in fig. 1, and the antenna array may be made of an optically transparent material or opaque thin metal wires.

The display unit 1301 is configured to receive a high-frequency microwave signal transmitted by the control device through the antenna array. Illustratively, the display unit 1301 may be configured to perform S403 of the remote control method illustrated in fig. 4 described above.

The processing unit 1302 is configured to determine, according to an irradiation area of the high-frequency microwave signal in the antenna array, a position where the control device points to the display screen; and responding to the operation of the user, and executing corresponding events on the control at the position corresponding to the display screen pointed by the control equipment on the first interface.

The display unit 1301 is further configured to display a second interface, where the second interface is an interface after responding to the operation and executing a corresponding event on a control at a position on the first interface where the control device points to the display screen. Illustratively, the display unit 1301 and the processing unit 1302 may be configured to perform S404-S406 of the remote control method illustrated in fig. 4 described above.

Further, the antenna array in the display unit 1301 may include a plurality of array units.

The processing unit 1302 is further configured to determine a plurality of first array units from the antenna array according to the high-frequency microwave signal, where the first array units are array units in which electrical parameters in the antenna array are changed; and determining an irradiation area according to the positions of the plurality of first array units in the antenna array.

The processing unit 1302 is further configured to determine a geometric center of the irradiation area, and determine a position where the control device points to the display screen according to the geometric center of the irradiation area; or dividing the plurality of first array units into at least two array unit groups according to the magnitude of the change of the electrical parameter of each first array unit in the plurality of first array units; determining the geometric center of an area formed by all first array units in the array unit groups aiming at each array unit group in the at least two array unit groups, and obtaining the geometric centers which are in one-to-one correspondence with the at least two array unit groups; and determining the position of the control equipment pointing to the display screen according to the geometric centers which are in one-to-one correspondence with the at least two array unit groups.

The display unit 1301 is further configured to display a cursor on the first interface according to the position, determined by the processing unit 1302, of the control device pointing to the display screen.

The processing unit 1302 is further configured to determine an operation type of the operation according to the high-frequency microwave signal; the operation types are as follows: and clicking, double clicking, dragging or long pressing, responding to the operation according to the operation type, and executing a corresponding event on a control at a position corresponding to the position of the control equipment pointing to the display screen on the first interface.

It should be noted that all relevant contents of each step related to the above method embodiment may be referred to the functional description of the corresponding functional unit, and are not described herein again. The display device provided by the embodiment of the application is used for executing the remote control method, so that the same effect as the remote control method can be achieved.

An embodiment of the present application further provides a remote control system, where the remote control system may include: a control device and a display device comprising a display screen provided with an antenna array.

And the control equipment is used for receiving the operation of a user and transmitting the high-frequency microwave signal in response to the operation. Illustratively, the control device may be configured to perform S401-S402 of the remote control method shown in fig. 4 described above.

The display device is used for receiving the high-frequency microwave signals through the antenna array, determining the position of the control device pointing to the display screen according to the irradiation area of the high-frequency microwave signals in the antenna array, responding to operation, executing corresponding events on a control at the position of the control device pointing to the display screen on a first interface displayed on the display screen at present, and displaying a second interface. Illustratively, the display device may be used to perform S403-S406 of the remote control method shown in fig. 4 described above.

It should be noted that all relevant contents of each step related to the above method embodiment may be referred to the functional description of the corresponding functional module, and are not described herein again. The remote control system provided by the embodiment of the application is used for executing the remote control method, so that the same effect as the remote control method can be achieved.

Through the above description of the embodiments, it is clear to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, a module or a unit may be divided into only one logic function, and may be implemented in other ways, for example, a plurality of units or components may be combined or integrated into another device, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed to a plurality of different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially or partially contributed to by the prior art, or all or part of the technical solutions may be embodied in the form of a software product, where the software product is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, or the like) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above is only an embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A remote control method is characterized by being applied to a remote control system, wherein the remote control system comprises a control device and a display device, the display device comprises a display screen provided with an antenna array, and a first interface is displayed on the display screen; the method comprises the following steps:

the control equipment receives the operation of a user;

the control equipment responds to the operation and transmits a high-frequency microwave signal to the display equipment, and the working frequency of the high-frequency microwave signal is 30GHz-3000 GHz;

the display device receives the high-frequency microwave signal through the antenna array;

the display device determines the position of the control device pointing to the display screen according to the irradiation area of the high-frequency microwave signal in the antenna array;

and the display equipment responds to the operation, executes a corresponding event to a control at the position, pointing to the display screen, of the control equipment on the first interface, and displays a second interface.

2. The method of claim 1, wherein the antenna array comprises a plurality of array elements; before the display device determines the position of the control device pointing to the display screen according to the irradiation area of the high-frequency microwave signal in the antenna array, the method further includes:

the display equipment determines a plurality of first array units from the antenna array according to the high-frequency microwave signals, wherein the first array units are array units with changed electrical parameters in the antenna array;

and the display equipment determines the irradiation area according to the positions of the plurality of first array units in the antenna array.

3. The method of claim 2, wherein the determining, by the display device, the position at which the control device is directed to the display screen according to the irradiation area of the high-frequency microwave signal in the antenna array comprises:

the display equipment determines the geometric center of the irradiation area, and determines the position of the control equipment pointing to the display screen according to the geometric center of the irradiation area; or the like, or, alternatively,

the display device divides the plurality of first array units into at least two array unit groups according to the magnitude of the change of the electrical parameter of each first array unit in the plurality of first array units; determining the geometric center of a region formed by all first array units in the array unit groups aiming at each array unit group in the at least two array unit groups, and obtaining the geometric centers which are in one-to-one correspondence with the at least two array unit groups; and determining the position of the control equipment pointing to the display screen according to the geometric centers in one-to-one correspondence with the at least two array unit groups.

4. The method according to any one of claims 1-3, wherein after the control device receives an operation by a user, the method further comprises:

the control device determines an operation type of the operation, wherein the operation type is as follows: single click, double click, drag or long press;

the control equipment generates a control signal according to the operation type of the operation;

said control device transmitting a high frequency microwave signal in response to said operation, including;

and the control equipment transmits the high-frequency microwave signals corresponding to the control signals according to the control signals, the control signals corresponding to different operation types are different, and the high-frequency microwave signals corresponding to different control signals are different.

5. The method of claim 4, further comprising:

the display equipment determines the operation type according to the high-frequency microwave signal;

the display device responds to the operation, executes a corresponding event to a control at a position, pointing to the display screen, of the control device on the first interface, and displays a second interface, wherein the corresponding event comprises:

and the display equipment responds to the operation according to the operation type, executes a corresponding event to the control at the position where the control equipment points to the display screen, and displays the second interface.

6. The method of claim 5, further comprising:

and the display equipment displays a cursor on the first interface according to the position of the control equipment pointing to the display screen.

7. The control equipment is characterized by being applied to a remote control system, and further comprising display equipment, wherein the display equipment comprises a display screen provided with an antenna array; the control apparatus includes: at least one input module, a processor, an oscillator, and a transmit antenna;

the processor is used for determining a control identifier according to the operation of a user on the at least one input module and transmitting the control identifier to the oscillator;

the oscillator is used for generating a control signal according to the control identifier and transmitting the control signal to the transmitting antenna;

the transmitting antenna is used for transmitting a high-frequency microwave signal according to the control signal so that the display equipment can receive the high-frequency microwave signal through the antenna array, and the working frequency of the high-frequency microwave signal is 30GHz-3000 GHz.

8. The control device of claim 7, wherein the processor is configured to determine a control identifier according to a user operation of the at least one input module, and comprises:

the processor is configured to determine an operation type of the operation according to an operation of a user on the at least one input module, and determine the control identifier according to the operation type of the operation, where the operation type is: single click, double click, drag or long press;

the transmitting antenna is used for transmitting a high-frequency microwave signal according to the control signal, and comprises:

the transmitting antenna is used for transmitting the high-frequency microwave signals corresponding to the control signals according to the control signals, the control identifications corresponding to different operation types are different, the control signals corresponding to different control identifications are different, and the high-frequency microwave signals corresponding to different control signals are different.

9. A control device according to any of claims 7-8, characterized in that the control device further comprises: a filter and a power amplifier disposed between the oscillator and the transmit antenna;

the filter is used for filtering the control signal so as to enable the control signal to work in a preset working frequency band;

the amplifier is used for amplifying the control signal filtered by the filter and transmitting the amplified control signal to the transmitting antenna.

10. A display device is applied to a remote control system, and the remote control system further comprises a control device which can transmit a high-frequency microwave signal to the display device according to the operation of a user; the display device includes: the antenna array is arranged on the display screen, a first interface is displayed on the display screen, and the working frequency of the high-frequency microwave signal is 30GHz-3000 GHz;

the display screen is used for receiving the high-frequency microwave signal transmitted by the control equipment through the antenna array;

the processor is used for determining the position of the control equipment pointing to the display screen according to the irradiation area of the high-frequency microwave signal in the antenna array; responding to the operation, and executing corresponding events on controls, pointing to the display screen, of the control equipment on the first interface;

the display screen is further configured to display a second interface, where the second interface is an interface after executing a corresponding event on a control at a position on the first interface, where the control device points to the display screen, in response to the operation.

11. The display device according to claim 10, wherein the antenna array comprises a plurality of array elements;

the processor is further used for determining a plurality of first array units from the antenna array according to the high-frequency microwave signal, wherein the first array units are array units with changed electrical parameters in the antenna array; and determining the irradiation area according to the positions of the plurality of first array units in the antenna array.

12. The display device of claim 11, wherein the processor is configured to determine a position of the control device pointing to the display screen according to an irradiation area of the high-frequency microwave signal in the antenna array, and the position comprises:

the processor is used for determining the geometric center of the irradiation area and determining the position of the control equipment pointing to the display screen according to the geometric center of the irradiation area; or the like, or, alternatively,

the processor is used for dividing the plurality of first array units into at least two array unit groups according to the magnitude of the change of the electrical parameter of each first array unit in the plurality of first array units; determining the geometric center of a region formed by all first array units in the array unit groups aiming at each array unit group in the at least two array unit groups, and obtaining the geometric centers which are in one-to-one correspondence with the at least two array unit groups; and determining the position of the control equipment pointing to the display screen according to the geometric centers in one-to-one correspondence with the at least two array unit groups.

13. A display device as claimed in any one of claims 10 to 12,

the processor is further used for determining the operation type of the operation according to the high-frequency microwave signal; the operation types are as follows: single click, double click, drag or long press;

the processor responds to the operation and executes corresponding events on the control at the position corresponding to the position of the display screen pointed by the control equipment on the first interface, and the corresponding events comprise:

and the processor responds to the operation according to the operation type and executes a corresponding event to a control at a position corresponding to the position of the display screen pointed by the control equipment on the first interface.

14. A display device as claimed in claim 13,

the display screen is further used for displaying a cursor on the first interface according to the position, determined by the processor, of the control device pointing to the display screen.

15. A display device as claimed in claim 10 or 14,

the antenna array is made of light transparent materials or opaque thin metal wires.

16. A remote control system, comprising: the antenna array comprises a control device and a display device comprising a display screen provided with an antenna array; wherein the content of the first and second substances,

the control equipment is used for receiving the operation of a user and transmitting a high-frequency microwave signal in response to the operation;

the display device is used for receiving the high-frequency microwave signal through the antenna array, determining the position of the control device pointing to the display screen according to the irradiation area of the high-frequency microwave signal in the antenna array, responding to the operation, executing a corresponding event on a control at the position of the control device pointing to the display screen on a first interface currently displayed on the display screen, and displaying a second interface.