CN113656131A - Remote control method, device, electronic equipment and storage medium - Google Patents
Remote control method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113656131A CN113656131A CN202110939100.0A CN202110939100A CN113656131A CN 113656131 A CN113656131 A CN 113656131A CN 202110939100 A CN202110939100 A CN 202110939100A CN 113656131 A CN113656131 A CN 113656131A
- Authority
- CN
- China
- Prior art keywords
- control instruction
- terminal
- coordinate
- control
- gray scale
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000004891 communication Methods 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- QGZKDVFQNNGYKY-UHFFFAOYSA-O Ammonium Chemical compound [NH4+] QGZKDVFQNNGYKY-UHFFFAOYSA-O 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000007599 discharging Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
- G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/0485—Scrolling or panning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
- G06F3/1462—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay with means for detecting differences between the image stored in the host and the images displayed on the remote displays
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
- G06F9/453—Help systems
Abstract
The application discloses a remote control method, a remote control device, electronic equipment and a storage medium, and belongs to the technical field of communication. The method comprises the following steps: under the condition that a first terminal is connected with a second terminal, acquiring a first control instruction data set generated by the first terminal, wherein the first control instruction data set comprises a first control instruction, a first control instruction coordinate and a first gray area, and the first gray area is determined based on the first control instruction coordinate in a first page displayed by the first terminal; determining a second gray scale area in a second interface displayed by the second terminal based on the first control instruction coordinate in the first control instruction data set; and processing the first control instruction based on the correlation coefficient of the first gray scale region and the second gray scale region.
Description
Technical Field
The application belongs to the technical field of communication, and particularly relates to a remote control method and device, electronic equipment and a storage medium.
Background
With the continuous development of communication technology, the internet has gradually merged into the daily life and work and study of people.
Remote assistance is a technology for assisting and controlling another terminal by remotely leaving one terminal of the internet, and after the assisting party and the assisted party establish connection, a terminal interface of the assisted party is transmitted to the assisting party. Then, when the assisting party operates the assisted party, the instruction needs to be transmitted from the assisting party to the assisted party, the instruction is executed on the mobile phone of the assisted party after the instruction arrives, and the situation that the instruction cannot be normally executed is likely to occur in the remote assistance assisting process.
Therefore, how to better implement the remote control method has become an urgent problem to be solved in the industry.
Disclosure of Invention
An embodiment of the present application provides a remote control method, an apparatus, an electronic device, and a storage medium, which can solve the problem of better implementing remote control.
In a first aspect, an embodiment of the present application provides a remote control method, where the method includes:
under the condition that a first terminal is connected with a second terminal, acquiring a first control instruction data set generated by the first terminal, wherein the first control instruction data set comprises a first control instruction, a first control instruction coordinate and a first gray area, and the first gray area is determined based on the first control instruction coordinate in a first page displayed by the first terminal;
determining a second gray scale area in a second interface displayed by the second terminal based on the first control instruction coordinate in the first control instruction data set;
and processing the first control instruction based on the correlation coefficient of the first gray scale region and the second gray scale region.
In a second aspect, an embodiment of the present application provides a remote control apparatus, including:
the terminal comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a first control instruction data set generated by a first terminal under the condition that the first terminal is connected with a second terminal, the first control instruction data set comprises a first control instruction, a first control instruction coordinate and a first gray area, and the first gray area is determined based on the first control instruction coordinate in a first page displayed by the first terminal;
the determining module is used for determining a second gray scale area in a second interface displayed by the second terminal based on the first control instruction coordinate in the first control instruction data set;
and the processing module is used for processing the first control instruction based on the correlation coefficient of the first gray scale region and the second gray scale region.
In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program or instructions stored on the memory and executable on the processor, and when executed by the processor, the program or instructions implement the steps of the method according to the first aspect.
In a fourth aspect, embodiments of the present application provide a readable storage medium, on which a program or instructions are stored, which when executed by a processor implement the steps of the method according to the first aspect.
In a fifth aspect, an embodiment of the present application provides a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to execute a program or instructions to implement the method according to the first aspect.
In this embodiment of the application, when the first terminal of the assisting party sends the control instruction, information of the first gray scale region is written in the control instruction data set, the first gray scale region is a gray scale subset around the first control instruction coordinate in the first page, and when the controlled second terminal receives the first control instruction data set, the first control instruction is not immediately executed, but the second gray scale region around the first control instruction coordinate in the second page is determined, then the first gray scale region and the second gray scale region are compared in a correlation manner, and then whether to execute the first control instruction is determined, so that a situation that the second page of the second terminal is changed after the first control instruction reaches the second terminal, which causes an error in the first instruction can be effectively avoided, and reliability of remote control is effectively improved.
Drawings
Fig. 1 is a schematic flowchart of a remote control method according to an embodiment of the present disclosure;
FIG. 2 is a flow chart of a remote control provided by an embodiment of the present application;
FIG. 3 is a second flowchart of a remote control process provided in the embodiments of the present application;
FIG. 4 is a schematic diagram of a remote control structure provided in an embodiment of the present application;
FIG. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;
fig. 6 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present disclosure.
The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/" generally means that a preceding and succeeding related objects are in an "or" relationship.
The remote control method, the remote control device, the electronic device, and the storage medium according to the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 1 is a schematic flow chart of a remote control method provided in an embodiment of the present application, as shown in fig. 1, including:
specifically, the first terminal described in the embodiment of the present application may specifically refer to a helper in remote assistance, which may be a mobile terminal such as a mobile phone and a tablet computer, or a non-mobile terminal such as a computer.
The second terminal described in the present application may specifically be a assisted party in remote assistance, a mobile terminal such as a mobile phone or a tablet computer thereof, or a non-mobile terminal such as a computer, and the first terminal and the second terminal may be the same type of terminal device or different types of terminal devices.
In the embodiment of the application, the first terminal and the second terminal are connected, that is, it is stated that the first terminal and the second terminal have achieved remote control through a network, the second interface of the second terminal is transmitted to the first terminal through a video stream, at this time, the first terminal can see the interface of the second terminal through the first interface, at this time, a user can remotely control the second terminal through the first terminal, and therefore the purpose of remote assistance is achieved.
And if the first terminal wants to control the second terminal, the first terminal needs to send a control instruction to the second terminal, and at this time, the second terminal can acquire the first control data set generated by the first terminal.
The first control instruction described in this application may refer to a remote control instruction sent by the first terminal to the second terminal, and specifically may refer to a click instruction, a slide instruction, or a long press instruction, and for the slide instruction, in the scheme of the embodiment of the present application, one slide instruction may be decomposed into a plurality of sub-slide instructions, and a specific decomposition manner is determined according to a frequency at which the first terminal and the second terminal are synchronized.
The first control instruction coordinate described in this application is a position where the first control instruction falls on the first page, and may be specifically expressed as a pixel coordinate in the first page, and for the slide instruction, the first control instruction coordinate may be a pixel coordinate set composed of a series of pixel coordinates.
The first gray scale region described in the embodiment of the present application is a gray scale subset of pixel points within a preset region range, that is, an array of pixel values within the region, obtained in a first page with a first pixel coordinate as a center; for example, a square pixel region with 20 pixels at each upper and lower position as a side length is obtained by taking the first pixel coordinate as a center, and a gray subset of the pixel region is obtained to obtain a first gray region.
In the embodiment of the application, part of the interface display information related to the first instruction can be determined through the first gray scale region and is sent to the second terminal, so that the second terminal can verify whether the second interface is changed greatly.
specifically, the second interface displayed by the second terminal described in the present application should be identical to the first interface in a theoretical case, but may be different from the first interface because the second terminal may receive a control command from a user or other terminals in addition to a remote control command of the first terminal.
In the embodiment of the present application, the second gray scale region is determined, specifically, in the second interface, the gray scale subset of the pixel points in the preset region range is obtained by taking the first pixel coordinate as a center, and the preset region range is the same as the preset region range for generating the first gray scale region.
If the first interface and the second interface are the same, the first gray scale region and the second gray scale region are theoretically the same, and if the first interface and the second interface are different, the difference between the first gray scale region and the second gray scale region may be larger, and in order to better hook the interface comparison and control command, the comparison region is still selected by the first control command coordinate.
The correlation coefficient described in the embodiment of the present application may be determined by comparing the repetition ratio between the array of pixel values in the first gray scale region and the array of pixel values in the second gray scale region, and may be specifically implemented by a stereo matching algorithm.
However, there is a certain time delay when the first control instruction is sent to the second terminal, and at this time, the first control instruction is received only after the second interface of the second terminal is changed, and at this time, executing the first control execution instruction may cause an instruction execution error.
Therefore, in the embodiment of the present application, whether the first control command is normally executed or not can be determined according to the correlation coefficient, and the situation that the command is executed incorrectly can be effectively avoided.
In this embodiment of the application, when the first terminal of the assisting party sends the control instruction, information of the first gray scale region is written in the control instruction data set, the first gray scale region is a gray scale subset around the first control instruction coordinate in the first page, and when the controlled second terminal receives the first control instruction data set, the first control instruction is not immediately executed, but the second gray scale region around the first control instruction coordinate in the second page is determined, then the first gray scale region and the second gray scale region are compared in a correlation manner, and then whether to execute the first control instruction is determined, so that a situation that the second page of the second terminal is changed after the first control instruction reaches the second terminal, which causes an error in the first instruction can be effectively avoided, and reliability of remote control is effectively improved.
Optionally, the processing the first control instruction based on the correlation coefficient of the first gray scale region and the second gray scale region includes:
executing the first control instruction under the condition that the correlation coefficient is larger than a first preset threshold value;
or, in case the correlation coefficient is less than or equal to the first preset threshold, discarding the first control instruction.
Specifically, the first preset threshold described in the embodiment of the present application may be a preset value of the correlation coefficient, such as 0.6 or 0.7.
And under the condition that the correlation coefficient is larger than a first preset threshold, the similarity between the first gray scale region and the second gray scale region is higher, at the moment, the second interface is not changed, the first control instruction can normally realize control, and at the moment, the first control instruction is considered to be normally executed.
And under the condition that the correlation coefficient is smaller than or equal to a first preset threshold, indicating that the similarity between the first gray scale region and the second gray scale region is low, at this moment, indicating that the second interface is likely to change, if the first control instruction is continuously executed, the situation of an error instruction may occur, and at this moment, the second terminal discards the first control instruction.
More specifically, after the first control command is discarded, the second terminal sends a packet instructing the discarding to the first terminal, so that the first terminal can know that the command is discarded.
The implementation of the invention judges whether the first control instruction is the wrong instruction or not by acquiring the first gray scale area of the first terminal and the second gray scale area of the second terminal and carrying out correlation comparison, thereby avoiding the execution of the wrong instruction.
Optionally, in a case that the first control instruction data set further includes first control instruction time information, the processing the first control instruction includes:
acquiring second control instruction record information of a second terminal, wherein the second control instruction record information comprises a second control instruction, a second control instruction coordinate and second control instruction time information;
under the condition that the second control instruction and the first control instruction belong to the same type of control instruction and the difference value between the second control instruction coordinate and the first control instruction coordinate is smaller than a coordinate threshold value, acquiring the time difference value between the second control instruction time information and the first control instruction time information;
and determining an execution scheme of the first control instruction based on the time difference value, and executing the first control instruction according to the execution scheme.
Specifically, since the assisted and controlled second terminal also receives a second control instruction of the user for the second terminal in the remote assistance process, in order to ensure user experience, the second terminal immediately executes the second control instruction and records the executed second control instruction, specifically records the category of the second control instruction, the coordinate of the second control instruction and the execution time of the second control instruction, and generates second control instruction recording information.
The second control instruction described in the present application may specifically refer to a click instruction, a slide instruction, a long press instruction, or the like.
The second control instruction coordinate described in this application is a position where the second control instruction falls in the second page, and may be specifically expressed as a pixel coordinate in the second page.
The first control instruction time information described in the present application refers to a time when the first control instruction reaches the second terminal.
The second control instruction time information described in this application is a time when the second terminal receives the second control instruction.
In the embodiment of the present application, the second control instruction and the first control instruction belong to the same type of control instruction, for example, the second control instruction and the first control instruction both belong to a click instruction, or both belong to a slide instruction.
When the second control command and the first control command belong to the same type of control command, and the distance between the coordinate of the second control command and the coordinate of the first control command is short, there may be a risk of command collision, so it is necessary to further determine the temporal relationship between the two commands.
And obtaining a time difference value between the second control instruction time information and the first control instruction time information, where the time difference value may be obtained by subtracting the second control instruction time information from the first control instruction time information.
The time difference in the embodiment of the present application indicates a time interval between execution of the two control instructions, and if the time interval between execution of the two control instructions is longer, it indicates that there is no risk of instruction collision, and if the two control instructions are the same and the time interval between execution is shorter, it indicates that there is a risk of instruction collision between the two control instructions.
In the embodiment of the present application, a difference between a second control instruction coordinate and a first control instruction coordinate indicates a position difference between the two instructions, if a distance difference between the two control instructions is long, it indicates that there is no risk of instruction collision, and if the distance between the two control instructions is short, it indicates that there is a risk of instruction collision between the two control instructions.
Therefore, in the embodiment of the present application, an execution scheme of the first control instruction may be determined according to the time difference, and the first control instruction is executed according to the execution scheme.
In the embodiment of the application, the situation that the second control instruction received by the second terminal finger may conflict with the first control instruction is fully considered, and the execution scheme of the first control instruction is determined by comparing the types and the positions of the first control instruction and the second control instruction and the difference between the time information of the second control instruction and the time information of the first control instruction, so that the situation of instruction conflict is effectively avoided.
Optionally, the determining, based on the time difference, an execution scheme of the first control instruction, and executing the first control instruction according to the execution scheme includes:
under the condition that the time difference value is smaller than a second preset threshold value, determining a discarding scheme of the first control instruction, and discarding the first control instruction;
and under the condition that the time difference is greater than or equal to a second preset threshold value, determining an operation scheme of the first control instruction, and operating the first control instruction.
Specifically, the second preset threshold described in the embodiment of the present application is a set threshold regarding a time difference, and specifically may be 1 second or 0.5 second.
In this embodiment of the application, if the difference is greater than or equal to the second preset threshold, it indicates that the second control instruction received by the second terminal finger may not conflict with the first control instruction, and therefore the second terminal normally executes the first control instruction at this time.
Under the condition that the difference value is smaller than the second preset threshold value, it is indicated that the second control instruction received by the second terminal possibly conflicts with the first control instruction, at this time, the first control instruction should be discarded, and the second terminal sends a prompt message to the first terminal to prompt that the first control instruction is discarded due to the conflict with the second control instruction, at this time, the user of the first terminal can inform the user of the second terminal of not needing to operate temporarily through voice prompt or other manners.
In the embodiment of the application, whether the first control instruction is operated or the control instruction is discarded is judged by comparing the time difference value with the second preset threshold, so that the execution of repeated wrong instructions can be effectively avoided, and the effectiveness of remote operation is ensured.
Optionally, fig. 2 is one of remote control flowcharts provided in the embodiment of the present application, and as shown in fig. 2, includes: step 210, the assisting party terminal establishes a remote assisting connection with the assisted party terminal, and the assisting party terminal establishes a connection with the assisted party terminal after receiving the request in a manner that the assisted party requests the assisting party. The assisted terminal interface is transmitted to the assisting party through video streaming. The assisting party can see the interface of the assisted party on the terminal of the assisting party. The assisting party can operate the mobile phone of the assisted party to achieve the aim of remote assistance.
this is exemplified by clicking a certain button when the assisting party is operating. The coordinates of the operation point are acquired, and a grayscale subset of a square region with 20 pixels each, i.e., 40 pixels on a side, is acquired with the coordinate point as the center.
And step 240, packaging the instruction, the instruction coordinate point gray scale subset, the operation time stamp and the like into a data packet, and transmitting the data packet to the assisted terminal.
The data packet structure may be json, { "instruction": "touch", "instruction coordinates": "x: 100, y: 200", "grayscale subset" [ [1,0,1], [0,0,1] ] "," timestamp ": "1626676220562"}. And transmitting the data packet to the assisted party through the websocket.
And 250, the assisted terminal analyzes the data packet and acquires the gray subset of the current interface according to the instruction coordinate.
After receiving the data packet, the assisted party analyzes the data packet according to the sending format to obtain json data. The instruction coordinates are acquired and then a grayscale subset of the region having a side length of 40 centered on the coordinate point is acquired in the manner of step 230.
When the correlation coefficient of the gray subset is detected to be lower than the set value, the command is judged to be an error command, and the command is discarded. At this time, the terminal of the assisted party sends a data packet indicating that the instruction is discarded to the assisted party, so that the assisted party can know that the instruction is discarded.
And step 280, executing the instruction, and finishing the operation.
And when the gray subset correlation system is detected to be greater than or equal to the set value, the instruction is a valid instruction, and the instruction is executed on the assisted terminal. And finishing the execution of the remote assistance instruction.
Fig. 3 is a second flowchart of a remote control provided in the embodiment of the present application, as shown in fig. 3, including:
in step 320, the assisting party terminal initiates an operation on the assisted party terminal, and after the connection is established, the assisting party terminal can operate the assisted party terminal, such as sliding left and right to switch desktops, sliding up and down to scroll pages, clicking a button once, or pressing for a long time.
In step 340, when the command packet arrives at the assisted party, the commands in 1s are sequenced and de-duplicated, which is because the command packet may not be transmitted according to the time sequence during the communication transmission process, so the command of the assisted party does not immediately execute after arriving at the assisted party. The instructions arriving within 1s are sorted by the timestamp of the instruction operation and duplicate instructions are deduplicated to obtain an instruction set s 1.
for the instruction set s1 in step 340 and step 350, s2 is classified according to instruction type and coordinates to form a new set s3, the instruction of s2 is marked as executed, the mark of s1 is marked as to be executed, and the intersection of s1 and s2 is obtained.
The same instruction as the assisted party is discarded, step 370.
For the classified instructions in s3, it is determined whether the instructions are already included in s2, and for the included instructions, a set of instructions s4 that do not conflict with the assisted party is discarded.
The set s4 obtained in step 370 is executed sequentially on the assisted handset.
It should be noted that, in the remote control method provided in the embodiments of the present application, the execution main body may be a remote control device, or a control module in the remote control device for executing the remote control method. In the embodiment of the present application, a method for executing remote control by a remote control device is taken as an example, and a remote control device provided in the embodiment of the present application is described.
Fig. 4 is a schematic structural diagram of a remote control provided in an embodiment of the present application, as shown in fig. 4, including:
the obtaining module 410 is configured to obtain a first control instruction data set generated by a first terminal under a condition that the first terminal establishes a connection with a second terminal, where the first control instruction data set includes a first control instruction, a first control instruction coordinate, and a first grayscale region, and the first grayscale region is determined based on the first control instruction coordinate in a first page displayed by the first terminal;
the determining module 420 is configured to determine a second gray scale region in a second interface displayed by the second terminal based on the first control instruction coordinate in the first control instruction data set;
the processing module 430 is configured to process the first control instruction based on the correlation coefficient of the first gray scale region and the second gray scale region.
Optionally, the processing module is specifically configured to:
executing the first control instruction under the condition that the correlation coefficient is larger than a first preset threshold value;
or, in case the correlation coefficient is less than or equal to the first preset threshold, discarding the first control instruction.
Optionally, in a case that the first control instruction data set further includes first control instruction time information, the processing module is specifically configured to:
acquiring second control instruction record information of a second terminal, wherein the second control instruction record information comprises a second control instruction, a second control instruction coordinate and second control instruction time information;
under the condition that the second control instruction and the first control instruction belong to the same type of control instruction and the difference value between the second control instruction coordinate and the first control instruction coordinate is smaller than a coordinate threshold value, acquiring the time difference value between the second control instruction time information and the first control instruction time information;
and determining an execution scheme of the first control instruction based on the time difference value, and executing the first control instruction according to the execution scheme.
Optionally, the processing module is specifically configured to:
under the condition that the time difference value is smaller than a second preset threshold value, determining a discarding scheme of the first control instruction, and discarding the first control instruction;
and under the condition that the time difference is greater than or equal to a second preset threshold value, determining an operation scheme of the first control instruction, and operating the first control instruction.
The remote control device in the embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The device can be mobile electronic equipment or non-mobile electronic equipment. By way of example, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palm top computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), and the like, and the non-mobile electronic device may be a server, a Network Attached Storage (NAS), a Personal Computer (PC), a Television (TV), a teller machine or a self-service machine, and the like, and the embodiments of the present application are not particularly limited.
The remote control device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, and embodiments of the present application are not limited specifically.
The remote control device provided in the embodiment of the present application can implement each process implemented by the method embodiments of fig. 1 to fig. 3, and is not described herein again to avoid repetition.
Optionally, fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application, and as shown in fig. 5, an embodiment of the present application further provides an electronic device 500, which includes a processor 501, a memory 502, and a program or an instruction stored in the memory 502 and capable of running on the processor 501, and when the program or the instruction is executed by the processor 501, the process of the embodiment of the remote control method is implemented, and the same technical effect can be achieved, and details are not repeated here to avoid repetition.
It should be noted that the electronic device in the embodiment of the present application includes the mobile electronic device and the non-mobile electronic device described above.
Fig. 6 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
The electronic device 600 includes, but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, a processor 610, and the like.
Those skilled in the art will appreciate that the electronic device 600 may further comprise a power source (e.g., a battery) for supplying power to the various components, and the power source may be logically connected to the processor 610 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system. The electronic device structure shown in fig. 6 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown, or combine some components, or arrange different components, and thus, the description is omitted here.
The interface unit 608 is configured to obtain a first control instruction data set generated by a first terminal when the first terminal and a second terminal are connected, where the first control instruction data set includes a first control instruction, a first control instruction coordinate, and a first grayscale region, and the first grayscale region is determined based on the first control instruction coordinate in a first page displayed by the first terminal;
the processor 610 is configured to determine a second gray scale region in a second interface displayed by the second terminal based on the first control instruction coordinate in the first control instruction data set;
the processor 610 is configured to process the first control instruction based on the correlation coefficient of the first gray scale region and the second gray scale region.
The processor 610 is configured to execute the first control instruction if the correlation coefficient is greater than a first preset threshold;
or, in case the correlation coefficient is less than or equal to the first preset threshold, discarding the first control instruction.
The processor 610 is configured to obtain second control instruction record information of the second terminal, where the second control instruction record information includes a second control instruction, a second control instruction coordinate, and second control instruction time information;
the processor 610 is configured to obtain a time difference between the second control instruction time information and the first control instruction time information when the second control instruction and the first control instruction belong to the same type of control instruction and a difference between a second control instruction coordinate and a first control instruction coordinate is smaller than a coordinate threshold;
the processor 610 is configured to determine an execution scheme of the first control instruction based on the time difference, and execute the first control instruction according to the execution scheme.
The processor 610 is configured to determine a discarding scheme of the first control instruction and discard the first control instruction if the time difference is smaller than a second preset threshold;
and under the condition that the time difference is greater than or equal to a second preset threshold value, determining an operation scheme of the first control instruction, and operating the first control instruction.
It is to be understood that, in the embodiment of the present application, the input Unit 604 may include a Graphics Processing Unit (GPU) 6041 and a microphone 6042, and the Graphics Processing Unit 6041 processes image data of a still picture or a video obtained by an image capturing apparatus (such as a camera) in a video capturing mode or an image capturing mode. The display unit 606 may include a display panel 6061, and the display panel 6061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 607 includes a touch panel 6071 and other input devices 6072. A touch panel 6071, also referred to as a touch screen. The touch panel 6071 may include two parts of a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein. The memory 609 may be used to store software programs as well as various data including, but not limited to, application programs and an operating system. The processor 610 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 610.
The embodiment of the present application further provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or the instruction is executed by a processor, the program or the instruction implements each process of the remote control method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.
The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and so on.
The embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to execute a program or an instruction to implement each process of the above remote control method embodiment, and can achieve the same technical effect, and the details are not repeated here to avoid repetition.
It should be understood that the chips mentioned in the embodiments of the present application may also be referred to as system-on-chip, system-on-chip or system-on-chip, etc.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Further, it should be noted that the scope of the methods and apparatus of the embodiments of the present application is not limited to performing the functions in the order illustrated or discussed, but may include performing the functions in a substantially simultaneous manner or in a reverse order based on the functions involved, e.g., the methods described may be performed in an order different than that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present application may be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.
While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the invention is not limited to the precise embodiments described above, which are meant to be illustrative and not restrictive, and that various changes may be made therein by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A remote control method, comprising:
under the condition that a first terminal is connected with a second terminal, acquiring a first control instruction data set generated by the first terminal, wherein the first control instruction data set comprises a first control instruction, a first control instruction coordinate and a first gray area, and the first gray area is determined based on the first control instruction coordinate in a first page displayed by the first terminal;
determining a second gray scale area in a second interface displayed by the second terminal based on the first control instruction coordinate in the first control instruction data set;
and processing the first control instruction based on the correlation coefficient of the first gray scale region and the second gray scale region.
2. The remote control method according to claim 1, wherein the processing the first control instruction based on the correlation coefficient of the first gray scale region and the second gray scale region includes:
executing the first control instruction under the condition that the correlation coefficient is larger than a first preset threshold value;
or, in case the correlation coefficient is less than or equal to the first preset threshold, discarding the first control instruction.
3. The remote control method according to claim 1, wherein in a case where the first control instruction data set further includes first control instruction time information, the processing the first control instruction includes:
acquiring second control instruction record information of a second terminal, wherein the second control instruction record information comprises a second control instruction, a second control instruction coordinate and second control instruction time information;
under the condition that the second control instruction and the first control instruction belong to the same type of control instruction and the difference value between the second control instruction coordinate and the first control instruction coordinate is smaller than a coordinate threshold value, acquiring the time difference value between the second control instruction time information and the first control instruction time information;
and determining an execution scheme of the first control instruction based on the time difference value, and executing the first control instruction according to the execution scheme.
4. The remote control method according to claim 3, wherein the determining an execution scheme of the first control instruction based on the time difference value, the executing the first control instruction according to the execution scheme, comprises:
under the condition that the time difference value is smaller than a second preset threshold value, determining a discarding scheme of the first control instruction, and discarding the first control instruction;
and under the condition that the time difference is greater than or equal to a second preset threshold value, determining an operation scheme of the first control instruction, and operating the first control instruction.
5. A remote control apparatus, comprising:
the terminal comprises an acquisition module, a display module and a display module, wherein the acquisition module is used for acquiring a first control instruction data set generated by a first terminal under the condition that the first terminal is connected with a second terminal, the first control instruction data set comprises a first control instruction, a first control instruction coordinate and a first gray area, and the first gray area is determined based on the first control instruction coordinate in a first page displayed by the first terminal;
the determining module is used for determining a second gray scale area in a second interface displayed by the second terminal based on the first control instruction coordinate in the first control instruction data set;
and the processing module is used for processing the first control instruction based on the correlation coefficient of the first gray scale region and the second gray scale region.
6. The remote control device according to claim 5, wherein the processing module is specifically configured to:
executing the first control instruction under the condition that the correlation coefficient is larger than a first preset threshold value;
or, in case the correlation coefficient is less than or equal to the first preset threshold, discarding the first control instruction.
7. The remote control device according to claim 5, wherein in a case where the first control instruction data set further includes first control instruction time information, the processing module is specifically configured to:
acquiring second control instruction record information of a second terminal, wherein the second control instruction record information comprises a second control instruction, a second control instruction coordinate and second control instruction time information;
under the condition that the second control instruction and the first control instruction belong to the same type of control instruction and the difference value between the second control instruction coordinate and the first control instruction coordinate is smaller than a coordinate threshold value, acquiring the time difference value between the second control instruction time information and the first control instruction time information;
and determining an execution scheme of the first control instruction based on the time difference value, and executing the first control instruction according to the execution scheme.
8. The remote control device according to claim 7, wherein the processing module is specifically configured to:
under the condition that the time difference value is smaller than a second preset threshold value, determining a discarding scheme of the first control instruction, and discarding the first control instruction;
and under the condition that the time difference is greater than or equal to a second preset threshold value, determining an operation scheme of the first control instruction, and operating the first control instruction.
9. An electronic device comprising a processor, a memory and a program or instructions stored on the memory and executable on the processor, the program or instructions when executed by the processor implementing the steps of the remote control method according to any one of claims 1 to 4.
10. A readable storage medium, characterized in that it stores thereon a program or instructions which, when executed by a processor, implement the steps of the remote control method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110939100.0A CN113656131A (en) | 2021-08-16 | 2021-08-16 | Remote control method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110939100.0A CN113656131A (en) | 2021-08-16 | 2021-08-16 | Remote control method, device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113656131A true CN113656131A (en) | 2021-11-16 |
Family
ID=78491182
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110939100.0A Pending CN113656131A (en) | 2021-08-16 | 2021-08-16 | Remote control method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113656131A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115868933A (en) * | 2022-12-12 | 2023-03-31 | 四川互慧软件有限公司 | Method and system for collecting waveform of monitor |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015409A (en) * | 2020-07-16 | 2020-12-01 | 深圳市大富网络技术有限公司 | Webpage editing data synchronization method, device and system and computer storage medium |
| CN112632918A (en) * | 2019-10-14 | 2021-04-09 | 腾讯科技（深圳）有限公司 | Document editing method, device, terminal and storage medium |
| CN113014615A (en) * | 2019-12-20 | 2021-06-22 | 西安诺瓦星云科技股份有限公司 | Data synchronization method, device and system |
| CN113238942A (en) * | 2021-05-13 | 2021-08-10 | 京东数字科技控股股份有限公司 | User interface testing method and device, electronic equipment and storage medium |
-
2021
- 2021-08-16 CN CN202110939100.0A patent/CN113656131A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112632918A (en) * | 2019-10-14 | 2021-04-09 | 腾讯科技（深圳）有限公司 | Document editing method, device, terminal and storage medium |
| CN113014615A (en) * | 2019-12-20 | 2021-06-22 | 西安诺瓦星云科技股份有限公司 | Data synchronization method, device and system |
| CN112015409A (en) * | 2020-07-16 | 2020-12-01 | 深圳市大富网络技术有限公司 | Webpage editing data synchronization method, device and system and computer storage medium |
| CN113238942A (en) * | 2021-05-13 | 2021-08-10 | 京东数字科技控股股份有限公司 | User interface testing method and device, electronic equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115868933A (en) * | 2022-12-12 | 2023-03-31 | 四川互慧软件有限公司 | Method and system for collecting waveform of monitor |
| CN115868933B (en) * | 2022-12-12 | 2024-01-05 | 四川互慧软件有限公司 | Method and system for collecting waveform of monitor |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111884908B (en) | Contact person identification display method and device and electronic equipment | |
| WO2022156606A1 (en) | Information processing method and apparatus, and electronic device | |
| CN109753202B (en) | Screen capturing method and mobile terminal | |
| CN112486444A (en) | Screen projection method, device, equipment and readable storage medium | |
| CN112083854A (en) | Application program running method and device | |
| CN112462990A (en) | Image sending method and device and electronic equipment | |
| CN113127432A (en) | Operation execution method, device, electronic equipment and medium | |
| CN109669710B (en) | Note processing method and terminal | |
| CN113676395B (en) | Information processing method, related device and readable storage medium | |
| CN113656131A (en) | Remote control method, device, electronic equipment and storage medium | |
| US20170168582A1 (en) | Click response processing method, electronic device and system for motion sensing control | |
| CN113721876A (en) | Screen projection processing method and related equipment | |
| CN111897474A (en) | File processing method and electronic equipment | |
| CN107885887B (en) | file storage method and mobile terminal | |
| CN113852540B (en) | Information transmission method, information transmission device and electronic equipment | |
| CN113268179B (en) | Session message processing method, device, equipment and storage medium | |
| CN113965640A (en) | Message processing method and device | |
| CN113360038A (en) | Application function searching method and device and electronic equipment | |
| CN112866475A (en) | Image sending method and device and electronic equipment | |
| CN111813741A (en) | File sharing method and electronic equipment | |
| CN112637407A (en) | Voice input method and device and electronic equipment | |
| CN112269511A (en) | Page display method and device and electronic equipment | |
| CN113114559B (en) | Information display method, device and equipment | |
| CN111447353B (en) | Image acquisition method and device, electronic equipment and storage medium | |
| CN107992347B (en) | Function demonstration method and mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |