CN115061829A - Method, device, system, equipment and storage medium for remotely controlling browser - Google Patents

Method, device, system, equipment and storage medium for remotely controlling browser Download PDF

Info

Publication number
CN115061829A
CN115061829A CN202210652022.0A CN202210652022A CN115061829A CN 115061829 A CN115061829 A CN 115061829A CN 202210652022 A CN202210652022 A CN 202210652022A CN 115061829 A CN115061829 A CN 115061829A
Authority
CN
China
Prior art keywords
browser
execution script
target function
interface
current interface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210652022.0A
Other languages
Chinese (zh)
Inventor
肖坤宇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Sankuai Online Technology Co Ltd
Original Assignee
Beijing Sankuai Online Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Sankuai Online Technology Co Ltd filed Critical Beijing Sankuai Online Technology Co Ltd
Priority to CN202210652022.0A priority Critical patent/CN115061829A/en
Publication of CN115061829A publication Critical patent/CN115061829A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/547—Remote procedure calls [RPC]; Web services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
    • G06F9/4482—Procedural
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation

Abstract

The application discloses a method, a device, a system, equipment and a storage medium for remotely controlling a browser, and belongs to the technical field of computers. The method for remotely controlling the browser comprises the following steps: receiving an interface image of a current interface of a browser, which is sent by a controlled terminal, and sending the interface image to the control terminal; receiving equipment operation information corresponding to the interface image and sent by a control terminal; determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser; and sending the execution script of the target function to the controlled terminal. In the remote control process, the operating system of the controlled terminal does not need to process the device operation information of the control terminal as local device operation information, and the controlled terminal can normally process the device operation information input by the user locally. By adopting the scheme, when the remote control is carried out, the browser of the controlled terminal can execute the operation according to the instruction of the control terminal, and can normally execute other processing under the operation of a local user.

Description

Method, device, system, equipment and storage medium for remotely controlling browser
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method, an apparatus, a system, a device, and a storage medium for remotely controlling a browser.
Background
The research and development personnel at the front end of the browser often encounter various problems of user feedback, but due to the fact that the research and development personnel are different in browser version used by the user, the research and development personnel are different in operation mode with the user and the like, the research and development personnel cannot reproduce on the equipment, and the problems cannot be solved in time.
In order to reproduce the problem as much as possible, research personnel can reproduce and troubleshoot the problem in a mode of remotely controlling a user computer. Remote control generally means that a controlled terminal (i.e., a terminal of a user) extracts device operation information, such as a left button or a right button clicked by a mouse, a position clicked by the mouse, and the like, at a control terminal (i.e., a terminal of a developer), then determines corresponding processing according to the device operation information, and finally executes the processing.
In general, in the remote control process, the operating system of the controlled terminal needs to process the device operation information sent by the control terminal as local device operation information. At this time, if the operating system of the controlled terminal simultaneously processes the device operation information locally input by the user, processing confusion occurs. This makes it impossible for the controlled terminal to normally perform other processing under the operation of the local user, only under the control of the control terminal, during the remote manipulation period.
Disclosure of Invention
The embodiment of the application provides a method, a device, a system, equipment and a storage medium for remotely controlling a browser, which can solve the problem that an operator cannot normally use a terminal in a remote control period in the related art. The technical scheme is as follows:
in a first aspect, a method for remotely controlling a browser is provided, the method comprising:
receiving an interface image of a current interface of a browser, which is sent by a controlled terminal, and sending the interface image to a control terminal;
receiving equipment operation information corresponding to the interface image and sent by a control terminal;
determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser;
and sending the execution script of the target function to the controlled terminal.
In one possible implementation, the method further includes:
receiving a website of a webpage loaded on a current interface of the browser and sent by the controlled terminal;
acquiring webpage data corresponding to the website;
the determining of the execution script of the target function corresponding to the device operation information in the current interface of the browser includes:
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data.
In one possible implementation, the method further includes:
receiving webpage position information sent by the controlled terminal, wherein the webpage position information is used for indicating the position of a part of the webpage in the current interface of the browser in the webpage;
the determining, based on the web page data, an execution script of a target function corresponding to the device operation information in a current interface of the browser includes:
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data and the webpage position information.
In one possible implementation, the device operation information includes an operation type and the operation location;
the determining, based on the web page data and the web page location information, an execution script of a target function corresponding to the device operation information in a current interface of the browser includes:
and when the operation type is a first operation type, determining a control corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the control.
In one possible implementation, the device operation information includes an operation type and the operation location;
the determining, based on the web page data and the web page location information, an execution script of a target function corresponding to the device operation information in a current interface of the browser includes:
and when the operation type is a second operation type, determining whether the operation position corresponds to a control in the current interface or not based on the webpage data and the webpage position information, if the operation position corresponds to the control, determining an execution script of a corresponding target function based on the type of the control, and if the operation position does not correspond to the control, acquiring the execution script of the target function corresponding to the area without the control.
In one possible implementation, the device operation information includes an operation type and the operation location;
the determining, based on the web page data and the web page location information, an execution script of a target function corresponding to the device operation information in a current interface of the browser includes:
and when the operation type is a third operation type, determining an interface area corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the interface area.
In a possible implementation manner, the determining an execution script of a target function corresponding to the device operation information in a current interface of the browser includes:
and determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser based on the browser application data.
In a second aspect, a method for remotely controlling a browser is provided, the method comprising:
acquiring an interface image of a current interface of a browser;
sending an interface image of the current interface of the browser to a server;
receiving an execution script of a target function of the browser, which is sent by the server;
and executing the execution script of the target function through the browser.
In a third aspect, an apparatus for remotely controlling a browser is provided, where the apparatus includes:
a receiving module to: receiving an interface image of a current interface of a browser, which is sent by a controlled terminal, and sending the interface image to a control terminal; receiving equipment operation information corresponding to the interface image and sent by a control terminal;
the determining module is used for determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser;
and the sending module is used for sending the execution script of the target function to the controlled terminal.
In a possible implementation manner, the receiving module is further configured to:
receiving a website of a webpage loaded on a current interface of the browser and sent by the controlled terminal;
acquiring webpage data corresponding to the website;
the determining module is configured to:
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data.
In a possible implementation manner, the receiving module is further configured to:
receiving webpage position information sent by the controlled terminal, wherein the webpage position information is used for indicating the position of a part of the webpage in the current interface of the browser in the webpage;
the determining module is configured to:
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data and the webpage position information.
In one possible implementation, the device operation information includes an operation type and the operation location;
the determining module is configured to:
and when the operation type is a first operation type, determining a control corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the control.
In one possible implementation, the device operation information includes an operation type and the operation location;
the determining module is configured to:
and when the operation type is a second operation type, determining whether the operation position corresponds to a control in the current interface or not based on the webpage data and the webpage position information, if the operation position corresponds to the control, determining an execution script of a corresponding target function based on the type of the control, and if the operation position does not correspond to the control, acquiring the execution script of the target function corresponding to the area without the control.
In one possible implementation, the device operation information includes an operation type and the operation location;
the determining module is configured to:
and when the operation type is a third operation type, determining an interface area corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the module.
In one possible implementation manner, the determining module is configured to:
and determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser based on the browser application data.
In a fourth aspect, an apparatus for remotely controlling a browser is provided, where the apparatus includes:
the acquisition module is used for acquiring an interface image of a current interface of the browser;
the sending module is used for sending the interface image of the current interface of the browser to a server;
the receiving module is used for receiving the target browser execution script sent by the server;
and the execution module is used for executing the target browser execution script through the browser.
In a fifth aspect, a system for remotely operating a browser, the system comprising: the system comprises a control terminal, a server and a controlled terminal;
the controlled terminal is used for acquiring an interface image of a current interface of a browser and sending the interface image of the current interface of the browser to the server;
the server is used for sending the interface image to the control terminal;
the control terminal is used for displaying the interface image, acquiring equipment operation information corresponding to the interface image and sending the equipment operation information to the server;
the server is used for determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser and sending the execution script of the target function to the controlled terminal;
and the controlled terminal is used for executing the execution script of the target function through a browser.
In one possible implementation, the server is further configured to:
receiving a website of a webpage loaded on a current interface of the browser and sent by the controlled terminal;
acquiring webpage data corresponding to the website;
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data.
In one possible implementation, the server is further configured to:
receiving webpage position information sent by the controlled terminal, wherein the webpage position information is used for indicating the position of a part of the webpage in the current interface of the browser in the webpage;
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data and the webpage position information.
In one possible implementation, the device operation information includes an operation type and the operation location;
the server is configured to:
and when the operation type is a first operation type, determining a control corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the control.
In one possible implementation, the device operation information includes an operation type and the operation location;
the server is configured to:
and when the operation type is a second operation type, determining whether the operation position corresponds to a control in the current interface or not based on the webpage data and the webpage position information, if the operation position corresponds to the control, determining an execution script of a corresponding target function based on the type of the control, and if the operation position does not correspond to the control, acquiring the execution script of the target function corresponding to the area without the control.
In one possible implementation, the device operation information includes an operation type and the operation location;
the server is configured to:
and when the operation type is a third operation type, determining an interface area corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the interface area.
In one possible implementation, the server is configured to:
and determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser based on the browser application data.
In a sixth aspect, a computer device is provided, where the computer device includes a processor and a memory, where the memory stores at least one instruction, and the instruction is loaded by the processor and executed to implement the operations performed by the method for remotely controlling a browser in the first aspect and the possible implementation manners thereof or the second aspect.
In a seventh aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the operations performed by the method for remotely controlling a browser in the first aspect and the possible implementation manners thereof or the second aspect.
In an eighth aspect, a computer program product is provided, the computer program product comprising computer program code, which when executed by a computer device, causes the computer device to perform the method for remotely controlling a browser provided in the first aspect and its possible implementation manners or the second aspect.
The technical scheme provided by the embodiment of the application has the following beneficial effects:
according to the scheme provided by the embodiment of the application, after the server receives the interface image of the current interface of the browser sent by the controlled terminal, the interface image is sent to the control terminal. Then, the server receives the device operation information of the corresponding interface image at the control terminal, and determines the execution script of the corresponding target function according to the device operation information. And finally, the server sends the execution script to the controlled terminal, and the controlled terminal executes the execution script. In the remote control process, the operating system of the controlled terminal does not need to process the equipment operation information of the control terminal as local equipment operation information. Therefore, the controlled terminal can normally process the device operation information input locally by the user. Therefore, by adopting the scheme, during the remote control, the controlled terminal can normally execute other processing under the operation of the local user while the browser executes the operation according to the instruction of the control terminal.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a system for remotely controlling a browser according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a server provided in an embodiment of the present application;
fig. 3 is a schematic structural diagram of a controlled terminal according to an embodiment of the present application;
fig. 4 is a flowchart of a method for remotely controlling a browser according to an embodiment of the present application;
fig. 5 is a schematic diagram illustrating establishment of a remote operation connection according to an embodiment of the present application;
FIG. 6 is a schematic diagram of an interface image provided by an embodiment of the present application;
fig. 7 is a schematic structural diagram of an apparatus for remotely controlling a browser according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of an apparatus for remotely controlling a browser according to an embodiment of the present application;
fig. 9 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
The embodiment of the application provides a method for remotely controlling a browser, which can be applied to the process of remotely controlling the browser in terminal equipment. The method is applied to a remote control system, and as shown in fig. 1, the remote control system includes a server, a controlled terminal and a control terminal. In the following, each device in the remote control system is described:
server
The server may be a physical server or a cloud server, which is not limited herein. From a hardware composition perspective, the server may be structured as shown in fig. 2, and include a processor 210, a memory 220, and a communication unit 230.
The processor 210 may be a Central Processing Unit (CPU), a system on chip (SoC), or the like. The processor 210 may be configured to obtain web page data corresponding to the web address, may be configured to determine an execution script of a target function corresponding to the device operation information in a current interface of the browser, and the like.
The memory 220 may be various volatile memories or nonvolatile memories, such as a SSD (solid state disk), a DRAM (dynamic random access memory) memory, and the like. The memory 220 may be used to store pre-stored data, intermediate data and result data during remote manipulation. Such as interface images, device operating information, execution scripts, web sites for web pages, web page data, browser application data, and the like.
The communication part 230 may be a wired network connector, a WiFi (wireless fidelity) module, a bluetooth module, a cellular network communication module, etc. The communication component 230 may be configured to perform data transmission with other devices, where the other devices are a controlled terminal, a control terminal, and other servers, for example, receive an interface image of a current interface of a browser sent by the controlled terminal, send the interface image to the control terminal, send an execution script of a target function to the controlled terminal, and so on.
Controlled terminal
The controlled terminal can be a desktop computer, a notebook computer, a tablet computer and other devices. From the viewpoint of hardware composition, the controlled terminal may be configured as shown in fig. 3, and include a processor 310, a memory 320, a communication section 330, and a display section 340.
The processor 310 may be a Central Processing Unit (CPU), a system on chip (SoC), or the like. The processor 210 may be used to obtain an interface image of the current interface of the browser, and so on.
The storage 320 may be various volatile memories or nonvolatile memories, such as a SSD (solid state disk), a DRAM (dynamic random access memory) memory, and the like. The memory 220 may be used to store pre-stored data, intermediate data and result data during remote manipulation. Such as an interface image of the current interface, an execution script of the target function, and so on.
The communication part 330 may be a wired network connector, a WiFi (wireless fidelity) module, a bluetooth module, a cellular network communication module, etc. The communication component 230 can be used for data transmission with other devices, such as a server, for example, can be used for sending an interface image of a current interface of a browser to the server, can be used for executing an execution script of a target function through the browser, and the like.
The display part 340 may be a display panel integrated with the user terminal or may be a display device that establishes a communication connection separate from the user terminal. When the display unit 340 is a display panel, it may be a TN (twisted nematic) panel, a VA (vertical alignment) panel, an in-plane switching (IPS) panel, or the like. The display part may be used to display an interface image of a current interface of the browser, may be used to display an execution script of a target function, and the like.
Control terminal
From the perspective of hardware composition, the control terminal and the controlled terminal have the same or similar structures, and the functions of the components are also the same or similar, so that the detailed description is omitted here.
Next, a flow of a method for remotely controlling a browser provided in an embodiment of the present application is described, where a processing flow of the method may be as shown in fig. 4, and includes the following processing steps:
s401, remote control connection is established between the control terminal and the controlled terminal.
Before the remote control begins, the operator may install an application for remote control at the control terminal, through which the operator may establish a communication connection between the control terminal and the server. The controlled terminal may then install an application for remote control at the controlled terminal, through which the controlled terminal may establish a communication connection between the controlled terminal and the server. Communication connection can be established between the control terminal and the server and between the controlled terminal and the server through Hyper Text Transfer Protocol (HTTP).
The control terminal and the controlled terminal are provided with unique corresponding identification information for identification of identity. The identification information may be an IP (internet protocol) address of the terminal device, a character string allocated by the application program to each terminal, a field set by the user when using the application program, or the like.
Firstly, an operator may send a control request message for establishing remote control to a server through a control terminal, where the control request message may carry first identification information of the control terminal, second identification information of a certain controlled terminal, and instruction information indicating whether to approve control.
Then, the server may send the instruction information indicating whether the controlled terminal agrees to be controlled and the first identification information of the control terminal to the controlled terminal corresponding to the second identification information.
And then, the controlled terminal can display the instruction information and the first identification information after receiving the instruction information and the first identification information. For example, as shown in fig. 5, the controlled terminal displays a window of "whether to allow a remote operation (i.e., the first identification information)", if so, please click yes, if not, please click no ", and so on.
And finally, the controlled person clicks 'yes', which means that the controller is allowed to control the controlled terminal through the control terminal, namely, the control terminal and the controlled terminal are connected in a remote control mode.
Optionally, the controlled terminal may send an invitation control request message for establishing remote control to the server, where the invitation control request message may carry third identifier information of the controlled terminal, fourth identifier information of a certain control terminal, and instruction information indicating whether to approve remote control. The server can forward the related information to the control terminal corresponding to the fourth identification information, and a controller at the control terminal can select whether to perform remote control. The specific process is similar to the above, and is not repeated here.
S402, the server receives the interface image of the current interface of the browser sent by the controlled terminal and sends the interface image to the control terminal.
After the control terminal and the controlled terminal establish remote control connection through the server, the controlled terminal can acquire an interface image of a current interface of the browser and send the interface image of the current interface to the server. Meanwhile, the first identification information of the control terminal, the website of the webpage loaded by the current interface of the browser and the webpage position information can be sent to the server. The webpage position information is used for indicating the position of the part of the webpage in the current interface of the browser in the webpage.
After receiving the interface image of the current interface of the browser sent by the controlled terminal and the website of the webpage loaded by the current interface, the server sends the interface image of the current interface to the control terminal corresponding to the first identification information. Meanwhile, the server can obtain the webpage data corresponding to the website. The web page data may include web page static resources, the server analyzes the web page static resources to generate an execution script corresponding to an operable element in a web page, specifically, the execution script of all controls in the current interface of the browser under different operation types (such as mouse click, keyboard input, and the like) may be obtained through analysis, and the execution script may be a JavaScript script. For example, the widget clicks a corresponding JavaScript script on the left mouse button, and the widget presses the corresponding JavaScript script when the enter keyboard button is pressed, and so on.
Optionally, after the control terminal establishes the remote control connection with the controlled terminal, at the controlled terminal, the browser may be in a maximized state to perform full-screen display, may also be in a specified display area to perform small-screen display, and may also be in a minimized state, which is not limited herein.
And S403, the server receives the equipment operation information corresponding to the interface image sent by the control terminal.
After receiving the interface image sent by the server, the control terminal displays the interface image to the operator. The operator can perform various operations on the interface image through the input part at the control terminal. For example, the input component is a mouse, and the operator can click a certain position in the interface image with a left key, or the input component is a keyboard, and the controller can input letters at a certain position in the interface image, and so on.
The control terminal may obtain device operation information that an operator operates the interface image, where the device operation information may include an operation position and an operation type, and for example, the device operation information includes a left mouse click (i.e., an operation type) and a position of a mouse pointer in the interface image (i.e., an operation position) when the left mouse click is performed, and so on.
Alternatively, the device operation information may include only the operation position and not the operation type, for example, the device operation information includes a position (i.e., operation position) where a mouse pointer is located when the mouse is moved. The device operation information may also include only the operation type and not the operation position, for example, the device operation information includes a keyboard input (i.e., operation type), and the like.
The control terminal can send the acquired device operation information corresponding to the interface image to the server through an ajax (asynchronous javascript and xml) technology, and the server performs subsequent processing based on the received device operation information.
S404, the server determines the execution script of the target function corresponding to the device operation information in the current interface of the browser.
The server stores, in addition to the web page data acquired in step S402, browser application data corresponding to the browser, where the browser application data may be regarded as an execution script corresponding to a control carried by the browser under different operation types.
After receiving the device operation information, the server may determine, according to the device operation information and the web page data (or the browser application data), an execution script of a target function corresponding to the device operation information in the current interface of the browser, and specifically determine a process, which will be described in detail below.
S405, sending the execution script of the target function to the controlled terminal.
The server can establish communication with the browser of the controlled terminal through a WebSocket technology, and sends the execution script of the target function to the browser of the controlled terminal through a WebSocket protocol port. And the browser of the controlled terminal receives the execution script, executes the execution script and dynamically updates the page image of the browser.
The foregoing is a general description of a method for remotely controlling a browser according to an embodiment of the present application, and generally, an interface of the browser mentioned above may include two portions, namely, a web page display area and a browser function area, as shown in fig. 6. After receiving the interface image of the current interface of the browser, the server can also acquire webpage data of a webpage loaded by the current interface of the browser and browser application data.
When the operation position corresponding to the device operation information is located in the browser function area of the current interface, the server may determine, according to the browser application data, an execution script of a target function corresponding to the device operation information in the current interface of the browser. For example, when the operation type is left mouse click and the operation position is the region of the browser's "home page" control, the server determines that the corresponding control is the "home page" according to the current operation position, thereby determining a JavaScript script (i.e., an execution script) corresponding to the left mouse click and the "home page" control in the browser application data, and sending the JavaScript script to the controlled terminal.
When the operation position corresponding to the device operation information is located in the webpage display area of the current interface, the server may further receive webpage position information corresponding to the current interface of the browser sent by the controlled terminal, where the webpage position information is used to indicate a position of a part of a webpage in the current interface of the browser in the webpage. The server can determine the execution script of the target function corresponding to the equipment operation information in the current interface of the browser according to the webpage data and the webpage position information corresponding to the current interface.
As an example, the following four different operation types of operations are performed in the web page display area, and the method of determining the execution script is described below:
case one, first operation type
When the operation type is a first operation type (such as left mouse button clicking), the server may first determine the web page position information corresponding to the interface image of the current interface; then, determining a control corresponding to the operation position in a webpage corresponding to the current interface according to the operation position, the webpage position information and the webpage data of the current interface; finally, the server may determine an execution script of a target function corresponding to the control in the first operation type.
For example, the server receives device operation information, wherein the first operation type is 'left mouse click' and the operation position is a position corresponding to a 'search' button in a webpage, at this time, the server already determines a plurality of webpage data corresponding to the 'left mouse click', and then the server determines webpage position information corresponding to an interface image according to the interface image corresponding to the operation position; then, according to the webpage position information and the position corresponding to the search button, determining the corresponding control as the search button; and finally, the server determines a JavaScript script of the search function corresponding to the search in the webpage data corresponding to the left mouse click, and the like.
Optionally, after receiving the device operation information, the server may first determine a corresponding execution script to be supplemented according to the operation type, then determine an execution code of the control corresponding to the operation position, and finally add the execution code corresponding to the control to the specified position of the execution script to be supplemented, thereby obtaining the execution script of the target function corresponding to the control.
Case two, second type of operation
When the operation type is a second operation type (such as right mouse click), the server may determine the web page position information corresponding to the interface image first; then, determining whether the operation position corresponds to a control in the current interface or not according to the webpage data and the webpage position information; and if the operation position corresponds to a control, the server determines the execution script of the corresponding target function according to the type of the control, and if the operation position does not correspond to the control, the execution script of the target function corresponding to the area without the control is obtained. The server stores in advance a corresponding relationship between the control type and the execution script in the second operation type, for example, the first control type corresponds to a first type of execution script, the second control type corresponds to a second type of execution script, and so on. The specific determination process for executing the script is similar to that in case one, and is not described herein again.
Case three, third operation type
When the operation type is a third operation type (such as rolling a mouse wheel and the like), the server can determine webpage position information corresponding to the interface image firstly; and then, according to the webpage data and the webpage position information, determining an interface area corresponding to the operation position in the current interface, and acquiring an execution script of a target function corresponding to the interface area. When the third operation type is to scroll a mouse wheel, the target function corresponding to the interface area may be to scroll an interface image, that is, to display other parts of the web page corresponding to the interface image; the target function corresponding to the interface area may also be an image of a local area in the interface image of the current interface. The third operation type and the target function corresponding to the interface area are only exemplary and are not limited.
Case four, fourth operation type
When the operation type is the fourth operation type, the process of determining to execute the script is described by taking the case of clicking a key of the keyboard as an example. The server can determine the position of an insertion point under the current interface; then, judging the type of the key; if the button is a character input type button (such as A, B, 1, 2, etc.), the server may input a corresponding character at the insertion point without determining a corresponding execution control, and if the button is a function type button (such as enter key, etc.), the server may determine a control corresponding to the insertion point, and further determine an execution script corresponding to the control according to the web page data. The specific determination process for executing the script is similar to the above three cases, and is not described herein again.
According to the scheme provided by the embodiment of the application, after the server receives the interface image of the current interface of the browser sent by the controlled terminal, the interface image is sent to the control terminal. Then, the server receives the device operation information of the corresponding interface image at the control terminal, and determines the execution script of the corresponding target function according to the device operation information. And finally, the server sends the execution script to the controlled terminal, and the controlled terminal executes the execution script. When the controlled terminal executes the execution script sent by the server, it usually only depends on the browser itself at the controlled terminal and does not depend on the operating system at the controlled terminal, that is, only occupying the browser does not occupy the entire operating system of the controlled terminal. Therefore, by adopting the scheme, during the remote control, the operator controls the browser of the controlled terminal, and the controlled terminal can be normally used by the controlled terminal.
An embodiment of the present application provides an apparatus for remotely controlling a browser, where the apparatus may be a server in the foregoing embodiment, and as shown in fig. 7, the apparatus includes:
a receiving module 710 configured to: receiving an interface image of a current interface of a browser, which is sent by a controlled terminal, and sending the interface image to the control terminal; receiving equipment operation information corresponding to the interface image and sent by a control terminal;
a determining module 720, configured to determine an execution script of a target function corresponding to the device operation information in a current interface of the browser;
and a sending module 730, configured to send the execution script of the target function to the controlled terminal.
In a possible implementation manner, the receiving module 710 is further configured to:
receiving a website of a webpage loaded on a current interface of a browser and sent by a controlled terminal;
acquiring webpage data corresponding to the website;
a determining module 720 for:
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data.
In a possible implementation manner, the receiving module 710 is further configured to:
receiving webpage position information sent by a controlled terminal, wherein the webpage position information is used for indicating the position of a part of a webpage in a current interface of a browser in the webpage;
a determining module 720 for:
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data and the webpage position information.
In one possible implementation, the device operation information includes an operation type and an operation location;
a determining module 720, configured to:
and when the operation type is the first operation type, determining a control corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the control.
In one possible implementation, the device operation information includes an operation type and an operation location;
a determining module 720 for:
and when the operation type is a second operation type, determining whether the operation position corresponds to a control in the current interface or not based on the webpage data and the webpage position information, if the operation position corresponds to the control, determining an execution script of a corresponding target function based on the type of the control, and if the operation position does not correspond to the control, acquiring the execution script of the target function corresponding to the area without the control.
In one possible implementation, the device operation information includes an operation type and an operation location;
a determining module 720 for:
and when the operation type is a third operation type, determining an interface area corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the interface area.
In one possible implementation, the determining module 720 is configured to:
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the browser application data.
And after receiving the interface image of the current interface of the browser sent by the controlled terminal, the server sends the interface image to the control terminal. Then, the server receives the device operation information of the corresponding interface image at the control terminal, and determines the execution script of the corresponding target function according to the device operation information. And finally, the server sends the execution script to the controlled terminal, and the controlled terminal executes the execution script. When the controlled terminal executes the execution script sent by the server, it usually only depends on the browser itself at the controlled terminal and does not depend on the operating system at the controlled terminal, that is, only occupying the browser does not occupy the entire operating system of the controlled terminal. Therefore, by adopting the scheme, during the remote control, the operator controls the browser of the controlled terminal, and the controlled terminal can be normally used by the controlled terminal.
It should be noted that: the device for remotely controlling a browser provided in the above embodiment is only exemplified by the division of the above devices when executing the method for remotely controlling a browser. In addition, the device for remotely controlling a browser provided in the above embodiment and the method embodiment for remotely controlling a browser belong to the same concept, and specific implementation processes thereof are detailed in the method embodiment and are not described herein again.
An embodiment of the present application provides an apparatus for remotely controlling a browser, where the apparatus may be a controlled terminal in the foregoing embodiment, and as shown in fig. 8, the apparatus includes:
an obtaining module 810, configured to obtain an interface image of a current interface of a browser;
a sending module 820, configured to send an interface image of a current interface of the browser to the server;
a receiving module 830, configured to receive a target browser execution script sent by a server;
an executing module 840, configured to execute the target browser execution script through the browser.
And after receiving the interface image of the current interface of the browser sent by the controlled terminal, the server sends the interface image to the control terminal. Then, the server receives the device operation information of the corresponding interface image at the control terminal, and determines the execution script of the corresponding target function according to the device operation information. And finally, the server sends the execution script to the controlled terminal, and the controlled terminal executes the execution script. When the controlled terminal executes the execution script sent by the server, it usually only depends on the browser itself at the controlled terminal and does not depend on the operating system at the controlled terminal, that is, only occupying the browser does not occupy the entire operating system of the controlled terminal. Therefore, by adopting the scheme, during the remote control, the operator controls the browser of the controlled terminal, and the controlled terminal can be normally used by the controlled terminal.
It should be noted that: the device for remotely controlling a browser provided in the above embodiment is only exemplified by the division of the above devices when executing the method for remotely controlling a browser. In addition, the device for remotely controlling a browser provided in the above embodiment and the method embodiment for remotely controlling a browser belong to the same concept, and specific implementation processes thereof are detailed in the method embodiment and are not described herein again.
An embodiment of the present application provides a system for remotely controlling a browser, and a corresponding structure is shown in fig. 1, where the system includes: the system comprises a control terminal, a server and a controlled terminal;
the controlled terminal is used for acquiring an interface image of a current interface of a browser and sending the interface image of the current interface of the browser to the server;
the server is used for sending the interface image to the control terminal;
the control terminal is used for displaying the interface image, acquiring equipment operation information corresponding to the interface image and sending the equipment operation information to the server;
the server is used for determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser and sending the execution script of the target function to the controlled terminal;
and the controlled terminal is used for executing the execution script of the target function through a browser.
In one possible implementation, the server is further configured to:
receiving a website of a webpage loaded on a current interface of a browser and sent by a controlled terminal;
acquiring webpage data corresponding to the website;
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data.
In one possible implementation, the server is further configured to:
receiving webpage position information sent by a controlled terminal, wherein the webpage position information is used for indicating the position of a part of a webpage in a current interface of a browser in the webpage;
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data and the webpage position information.
In one possible implementation, the device operation information includes an operation type and an operation location;
the server is used for:
and when the operation type is the first operation type, determining a control corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the control.
In one possible implementation, the device operation information includes an operation type and an operation location;
the server is used for:
and when the operation type is a second operation type, determining whether the operation position corresponds to a control in the current interface or not based on the webpage data and the webpage position information, if the operation position corresponds to the control, determining an execution script of a corresponding target function based on the type of the control, and if the operation position does not correspond to the control, acquiring the execution script of the target function corresponding to the area without the control.
In one possible implementation, the device operation information includes an operation type and an operation location;
the server is used for:
and when the operation type is a third operation type, determining an interface area corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the interface area.
In one possible implementation, the server is configured to:
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the browser application data.
It should be noted that: the above embodiments of the system for remotely controlling a browser provided in the above embodiments are only illustrated by dividing the above devices when executing the method for remotely controlling a browser. In addition, the system for remotely controlling a browser provided in the above embodiment and the method embodiment for remotely controlling a browser belong to the same concept, and specific implementation processes thereof are detailed in the method embodiment and are not described herein again.
The computer device provided by the embodiment of the present application may be the server in the above embodiment. Fig. 9 is a schematic structural diagram of the computer device, and the computer device 900 may have a relatively large difference due to different configurations or performances, and may include one or more processors 910 and one or more memories 920, where the memory 920 stores at least one instruction, and the at least one instruction is loaded and executed by the processors 910 to implement the methods provided by the above-mentioned method embodiments. Certainly, the computer device may further have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input and output, and the computer device may further include other components for implementing the functions of the device, which is not described herein again.
In an exemplary embodiment, a computer-readable storage medium, such as a memory, including instructions executable by a processor in a terminal to perform the method of remotely manipulating a browser in the above-described embodiments is also provided. The computer readable storage medium may be non-transitory. For example, the computer-readable storage medium may be a ROM (read-only memory), a RAM (random access memory), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
It should be noted that information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals (including but not limited to signals transmitted between a user terminal and other equipment, etc.) referred to in the present application are authorized by a user or are sufficiently authorized by various parties, and the collection, use, and processing of the relevant data need to comply with relevant laws and regulations and standards in relevant countries and regions. For example, the "first identification information", "second identification information", and "interface image of the current interface of the browser" referred to in the present application are all acquired with sufficient authorization.
The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims (14)

1. A method for remotely operating a browser, the method comprising:
receiving an interface image of a current interface of a browser, which is sent by a controlled terminal, and sending the interface image to a control terminal;
receiving equipment operation information corresponding to the interface image and sent by a control terminal;
determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser;
and sending the execution script of the target function to the controlled terminal.
2. The method of claim 1, further comprising:
receiving a website of a webpage loaded on a current interface of the browser and sent by the controlled terminal;
acquiring webpage data corresponding to the website;
the determining of the execution script of the target function corresponding to the device operation information in the current interface of the browser includes:
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data.
3. The method of claim 2, further comprising:
receiving webpage position information sent by the controlled terminal, wherein the webpage position information is used for indicating the position of a part of the webpage in the current interface of the browser in the webpage;
the determining, based on the web page data, an execution script of a target function corresponding to the device operation information in a current interface of the browser includes:
and determining an execution script of a target function corresponding to the equipment operation information in the current interface of the browser based on the webpage data and the webpage position information.
4. The method of claim 3, wherein the device operation information comprises an operation type and the operation location;
the determining, based on the web page data and the web page location information, an execution script of a target function corresponding to the device operation information in a current interface of the browser includes:
and when the operation type is a first operation type, determining a control corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the control.
5. The method of claim 3, wherein the device operation information comprises an operation type and the operation location;
the determining, based on the web page data and the web page location information, an execution script of a target function corresponding to the device operation information in a current interface of the browser includes:
and when the operation type is a second operation type, determining whether the operation position corresponds to a control in the current interface or not based on the webpage data and the webpage position information, if the operation position corresponds to the control, determining an execution script of a corresponding target function based on the type of the control, and if the operation position does not correspond to the control, acquiring the execution script of the target function corresponding to the area without the control.
6. The method of claim 3, wherein the device operation information comprises an operation type and the operation location;
the determining, based on the web page data and the web page location information, an execution script of a target function corresponding to the device operation information in a current interface of the browser includes:
and when the operation type is a third operation type, determining an interface area corresponding to the operation position in the current interface based on the webpage data and the webpage position information, and acquiring an execution script of a target function corresponding to the interface area.
7. The method of claim 1, wherein the determining an execution script of a target function corresponding to the device operation information in a current interface of the browser comprises:
and determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser based on the browser application data.
8. A method for remotely operating a browser, the method comprising:
acquiring an interface image of a current interface of a browser;
sending an interface image of the current interface of the browser to a server;
receiving an execution script of a target function of the browser, which is sent by the server;
and executing the execution script of the target function through the browser.
9. An apparatus for remotely operating a browser, the apparatus comprising:
a receiving module to: receiving an interface image of a current interface of a browser, which is sent by a controlled terminal, and sending the interface image to a control terminal; receiving equipment operation information corresponding to the interface image and sent by a control terminal;
the determining module is used for determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser;
and the sending module is used for sending the execution script of the target function to the controlled terminal.
10. An apparatus for remotely operating a browser, the apparatus comprising:
the acquisition module is used for acquiring an interface image of a current interface of the browser;
the sending module is used for sending the interface image of the current interface of the browser to a server;
the receiving module is used for receiving the target browser execution script sent by the server;
and the execution module is used for executing the target browser execution script through the browser.
11. A system for remotely operating a browser, the system comprising: the system comprises a control terminal, a server and a controlled terminal;
the controlled terminal is used for acquiring an interface image of a current interface of a browser and sending the interface image of the current interface of the browser to the server;
the server is used for sending the interface image to the control terminal;
the control terminal is used for displaying the interface image, acquiring equipment operation information corresponding to the interface image and sending the equipment operation information to the server;
the server is used for determining an execution script of a target function corresponding to the equipment operation information in a current interface of the browser and sending the execution script of the target function to the controlled terminal;
and the controlled terminal is used for executing the execution script of the target function through a browser.
12. A computer device comprising a processor and a memory, wherein at least one instruction is stored in the memory, and wherein the at least one instruction is loaded and executed by the processor to perform operations performed by the method for remotely manipulating a browser of any of claims 1 to 8.
13. A computer-readable storage medium having stored therein at least one instruction which is loaded and executed by a processor to perform operations performed by a method of remotely manipulating a browser as claimed in any one of claims 1 to 8.
14. A computer program product, characterized in that the computer program product comprises computer program code which, when executed by a computer device, performs the method of remotely manipulating a browser as claimed in any one of claims 1 to 8.
CN202210652022.0A 2022-06-09 2022-06-09 Method, device, system, equipment and storage medium for remotely controlling browser Pending CN115061829A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210652022.0A CN115061829A (en) 2022-06-09 2022-06-09 Method, device, system, equipment and storage medium for remotely controlling browser

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210652022.0A CN115061829A (en) 2022-06-09 2022-06-09 Method, device, system, equipment and storage medium for remotely controlling browser

Publications (1)

Publication Number Publication Date
CN115061829A true CN115061829A (en) 2022-09-16

Family

ID=83200478

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210652022.0A Pending CN115061829A (en) 2022-06-09 2022-06-09 Method, device, system, equipment and storage medium for remotely controlling browser

Country Status (1)

Country Link
CN (1) CN115061829A (en)

Similar Documents

Publication Publication Date Title
EP3623942B1 (en) Message processing method and apparatus, storage medium, and computer device
US10353718B2 (en) Providing access to a remote application via a web client
CN106874519B (en) Page display method and device
CN111368235B (en) Page rendering and displaying method and device, server and storage medium
US20150046562A1 (en) Web browser orchestration
CN107045546B (en) Webpage processing method and device and intelligent terminal
US20140082471A1 (en) Displaying a Syntactic Entity
WO2024011899A1 (en) Widget loading method and apparatus, device, and storage medium
CN111880858A (en) Document processing and instant messaging method, device, electronic equipment and storage medium
CN102833238A (en) Method, device, equipment and system for assisting network equipment to carry out user authentication
CN111431767A (en) Multi-browser resource synchronization method and device, computer equipment and storage medium
CN114254222A (en) Method and device for counting interactive time of single-page application and electronic equipment
JP5612796B1 (en) Information processing apparatus, information processing method, and information processing program
CN115061829A (en) Method, device, system, equipment and storage medium for remotely controlling browser
CN112822089B (en) Method and device for adding friends
CN108459890B (en) Interface display method and device for application
CN112988310A (en) Online experiment method based on multi-split-screen browser
CN112269741A (en) Application program compatibility testing method and related equipment
US10776002B2 (en) Method and apparatus for inputting a text
CN107193670B (en) Remote management method, device and system for cluster workstations
CN111522599A (en) Method and apparatus for transmitting information
US9942361B2 (en) Reporting page composition data
US10460005B2 (en) Data consumption between heterogeneous web applications
CN104423973B (en) Processing method, device and system for floating frame page loading information
WO2022113315A1 (en) User interface extension system, user interface extension method, and user interface extension program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination