CN110569195A

CN110569195A - Page clicking method and device and clicking equipment

Info

Publication number: CN110569195A
Application number: CN201910858821.1A
Authority: CN
Inventors: 杨凯
Original assignee: Individual
Current assignee: Individual
Priority date: 2019-09-11
Filing date: 2019-09-11
Publication date: 2019-12-13
Anticipated expiration: 2039-09-11
Also published as: CN110569195B; WO2021047135A1

Abstract

according to one embodiment of the method, in the page clicking process, operating equipment generates an operation request according to a position needing to be clicked and a clicking mode, the operation request is sent to clicking equipment, the clicking equipment converts the operation request into an operation event and feeds the operation event back, and then the operation event is responded to complete clicking on a page to be clicked. The operation request is sent to the clicking equipment and the operation event is received from the clicking equipment through different communication channels, and the communication channel for sending the operation event to the running equipment by the clicking equipment is realized through an input special interface of the running equipment. Therefore, the operation event is converted into the operation event of the system process, and the target program can respond to the operation event. The above process automatically completes the page click in the operating device by the aid of one click device, and the effectiveness of the page click can be improved.

Description

Page clicking method and device and clicking equipment

Technical Field

One or more embodiments of the present disclosure relate to the field of computer technologies, and in particular, to a method and an apparatus for automatically clicking a page on a computer through a pointing device.

Background

With the development of automation technology, more and more scenes can reduce human participation. There may also be automation requirements for scenarios involving page clicks. For example, in the field of software testing, it is desirable that the testing process be automated. In the case where there are interactions of a plurality of pages, jumping between the pages is possible by a click operation of a hyperlink, a button, or the like. In the conventional technology, the automatic click jump between the mobile terminal pages can be realized by the following methods:

First, the software issues a click command without the need for hardware assistance. Functionally, this can satisfy all clicks, however, such clicks usually require root privileges, or are controlled from the computer side by connecting computer devices, so that this approach may be limited in the manner of deployment. Without root privileges, such a click cannot act on a dialog box that the system pops up, and the automated process may be interrupted when the system pops up a dialog box.

Second, hardware coordination enables fixed-location clicks, such as click heads for swipe cards. When the click positions on the pages are not consistent, the correct click may not be automatically achieved.

disclosure of Invention

One or more embodiments of the present specification describe a method, an apparatus, and a pointing device for page clicking, which can implement automatic clicking operation at any position on a page through cooperation between an operating device and the pointing device, thereby improving effectiveness of automatic page clicking.

According to a first aspect, a method for clicking a page is provided, and is applied to an operating device which is connected with a clicking device in advance, wherein the operating device is used for operating a target program, the clicking device is respectively provided with a first communication channel and a second communication channel with the operating device through a first communication module and a second communication module, and the second communication channel is established based on an input special interface of the operating device; the method comprises the following steps: under the condition that a page to be clicked appears in the running process of a target program, sending an operation request to the clicking equipment through the first communication channel so that the clicking equipment converts the operation request into an operation event, and feeding back the operation event through the second communication channel, wherein the operation event is used for describing offset operation and/or clicking operation indicated in the operation request; and responding to the operation event to finish clicking on the page to be clicked.

In one embodiment, the sending the operation request to the pointing device through the first communication channel includes: generating a positioning page and a first operation request, wherein the positioning page is used for covering the page to be clicked and preferentially responding to an operation event sent by the clicking equipment, and the page to be clicked at least corresponds to a first target position to be clicked and a first clicking mode aiming at the first target position; sending the first operation request to the clicking device so that the clicking device generates a first clicking event according to the first operation request, wherein the first clicking event is used for describing clicking operation of the clicking device according to a preset clicking mode; the responding to the operation event to complete the click on the page to be clicked comprises: responding to the first click event through the positioning page, and determining a first click position of a current click; and finishing clicking on the page to be clicked based on the comparison between the first clicking position and the first target position.

In one embodiment, in a case that the first click position and the first target position are consistent, the responding to the operation event to complete the click on the page to be clicked further includes: canceling the positioning page; sending a second operation request to the pointing device through the first communication channel so that the pointing device converts the second operation request into a second operation event to perform feedback through the second communication channel, wherein the second operation event is used for describing a pointing event corresponding to the first pointing mode; and responding to the second operation event through the page to be clicked, so that clicking on the page to be clicked is completed.

in one embodiment, in a case that the first click position and the first target position are not consistent, the responding to the operation event to complete the click on the page to be clicked further includes: respectively acquiring coordinates of the first click position and the first target position in the same page coordinate system; determining the coordinate offset from the first click position to the first target position according to the coordinates of the first click position and the first target position; generating a third operation request at least based on the coordinate offset, and sending the third operation request to the click equipment so that the click equipment can generate a third operation event according to the third operation request; and responding to the third operation event, and finishing clicking the page to be clicked.

In one embodiment, the generating a third operation request based at least on the coordinate offset comprises: determining an offset corresponding to the pointing device according to the coordinate offset and a preset conversion rule, wherein the conversion rule is used for describing a mapping relation between a moving distance corresponding to the pointing device and an offset distance of a pointing position corresponding to the pointing device on a screen; generating the third operation request according to the offset corresponding to the pointing device and the first pointing mode, so that the pointing device generates the offset corresponding to the movement of the pointing device according to the third operation request and performs a third operation event of pointing according to the first pointing mode; the completing the click on the page to be clicked in response to the third operation event further comprises: and canceling the positioning page, and responding to the third operation event through the page to be clicked.

In one embodiment, the generating a third operation request based at least on the coordinate offset comprises: taking the coordinate offset as an offset corresponding to the click equipment to generate a third operation request so that the click equipment can generate a third operation event offset according to the offset; responding to the three operation events through the positioning page, and determining a second click position generated by the third operation event; and detecting whether the second click position is consistent with the first target position, generating a fourth operation request according to a detection result, and sending the fourth operation request to the click device, and so on until the current click position is consistent with the first target position.

According to a second aspect, a click method is provided, which is applied to a click device that is connected with an operating device in advance, wherein the operating device is used for operating a target program, the click device is respectively established with the operating device through a first communication module and a second communication module to form a first communication channel and a second communication channel, and the second communication channel is established based on an input dedicated interface of the operating device; the method comprises the following steps: receiving an operation request sent by the running equipment through the first communication channel, wherein the operation request is generated by the running equipment under the condition that a page to be clicked appears in the running process of a target program; converting the operation request into an operation event of the clicking equipment, wherein the operation event is used for describing the offset operation and/or the clicking operation indicated in the operation request; and sending the operation event to the running equipment.

In one embodiment, the operational event comprises one of: the system comprises a display device, a display device and a display device, wherein the display device is used for describing a click event of the click device according to a preset click mode; the mobile event is used for describing the mobile event of the pointing device moving according to the preset offset and the pointing event of the pointing device moving according to the preset pointing mode.

According to a third aspect, there is provided a page clicking apparatus, provided in an operating device for operating a target program, the apparatus including: the sending unit is configured to send an operation request to the clicking device through a first communication channel under the condition that a page to be clicked appears in the running process of a target program, so that the clicking device converts the operation request into an operation event to be fed back through a second communication channel, wherein the operation event is used for describing offset operation and/or clicking operation indicated in the operation request, the clicking device is respectively provided with a first communication channel and a second communication channel with the running device through a first communication module and a second communication module in advance, and the second communication channel is established based on an input special interface of the running device; and the response unit is configured to respond to the operation event so as to finish clicking on the page to be clicked.

according to a fourth aspect, a pointing device is provided for pointing a page on an operating device to which a connection is pre-established; the device comprises: the system comprises a first communication module, a second communication module and a third communication module, wherein the first communication module is configured to establish a first communication channel with running equipment and receive an operation request sent by the running equipment through the first communication channel, and the operation request is generated by the running equipment under the condition that a page to be clicked appears in the running process of a target program; the conversion module is configured to convert the operation request into an operation event of the pointing device, wherein the operation event is used for describing an offset operation and/or a pointing operation indicated in the operation request; and the second communication module is configured to establish a second communication channel with the running equipment through an input special interface of the running equipment, and send the operation event to the running equipment through the second communication channel.

According to a fifth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method of the first or second aspect.

according to a sixth aspect, there is provided a computing device comprising a memory and a processor, wherein the memory has stored therein executable code, and wherein the processor, when executing the executable code, implements the method of the first or second aspect.

According to the method and the device provided by the embodiment of the specification, in the page clicking process, the operation equipment generates the operation request according to the position needing to be clicked and the clicking mode, sends the operation request to the clicking equipment, converts the operation request into the operation event by the clicking equipment and feeds the operation event back, and then responds to the operation event to finish clicking on the page to be clicked. The operation request is sent to the clicking equipment and the operation event is received from the clicking equipment through different communication channels, and the communication channel for sending the operation event to the running equipment by the clicking equipment is realized through an input special interface of the running equipment, so that the operation event is converted into the operation event of the system process and can be responded by the target program. The above process automatically completes the page click in the operating device by the aid of one click device, and the effectiveness of the page click can be improved.

Drawings

in order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 illustrates a schematic diagram of an implementation scenario of an embodiment disclosed herein;

FIG. 2 illustrates a flowchart of a method of page clicking according to one embodiment;

FIG. 3 illustrates a flow diagram of a clicking method according to one embodiment;

FIG. 4 is a diagram illustrating an interaction process of a running device and a clicking device completing a page click according to a specific example;

FIG. 5 shows a schematic block diagram of a device for page clicking according to one embodiment;

FIG. 6 illustrates a schematic block diagram of a pointing device, according to one embodiment.

Detailed Description

The scheme provided by the specification is described below with reference to the accompanying drawings.

For convenience of explanation, a specific application scenario of the embodiment of the present specification shown in fig. 1 is described. Fig. 1 is a schematic view of an implementation scenario of an embodiment disclosed in this specification. In the implementation scenario, a running device and a clicking device are included. The running device may be a mobile terminal such as a mobile phone and a tablet computer, or may be a notebook computer, a personal computer, or the like.

The operating device may include a system program, a test module, and a click control module. The system program is generally a basic program of the computer, and the system program may serve various applications or modules loaded in the system. The test module and the click control module are application modules loaded in the computer system. The test module may be used to run the target program. The click control module is used for sending an operation request to the click device so as to control the click device to generate a corresponding operation event. The click control module may be integrated with the test module, or may be a data processing module independent from the test module, which is not limited herein. Under the condition that the click control module and the test module are integrated together, the click control module can be realized in an SDK mode, and at the moment, the test module can trigger the operation logic of the click control module in a calling mode. Under the condition that the click control module and the test module are mutually independent, the test module can trigger the operation logic of the click control module in a mode of sending a notification message.

A pointing device may be used to generate a click event. The pointing device may be, for example, a mouse, or a mouse-like function device. The structure of the pointing device may be different from that of a mouse, for example, the pointing device may be simplified in hardware composition with respect to a mouse, which may not include a structure of a button, a wheel, or the like. The pointing device in the embodiments of the present description includes at least two communication modules, and the two communication modules communicate with the execution device through different connection channels. Assuming that the two communication modules are a first communication module and a second communication module respectively, and the two corresponding connection channels are a first communication channel and a second communication channel respectively, the first communication channel may be used by the operating device to send information to the pointing device, and the second communication channel may be used by the pointing device to transmit an operation event to the operating device. It is worth mentioning that the second communication channel is established based on the input-specific interface of the operating device and can be identified as a system-level communication channel. Here, the first and second numbers merely indicate correspondence, and do not limit the order or name.

In the process of testing the target program, the target program may jump between pages by clicking, display different contents within the range of the current page by clicking and positioning, and the like. These pages may all be referred to as pages to click on. When a page to be clicked appears in the running process of a tested target program, the test module can trigger the click control module to send a click request to the click device by sending a notice or calling so as to complete the automatic click process of the page to be clicked. The page to be clicked may correspond to at least a first target position to be clicked and a first click pattern for the first target position. The click control module may obtain information of the first target location and the first click pattern.

And the click control module responds to the situation that the page to be clicked appears in the target program, and sends an operation request to the click device through the first communication channel to tell the click device what operation is needed. The pointing device may generate a corresponding operation event according to the operation request, for example, an offset event, a pointing event, and the like corresponding to the pointing device. Taking a mouse as an example, a mouse pointer corresponding to the mouse may be arranged on the operating device connected with the mouse. When the mouse moves, a relative movement distance is generated, the relative displacement can be packaged into an offset event and sent to the electronic equipment, and the running equipment converts the relative displacement of the clicking equipment into a corresponding offset distance of a mouse pointer on a screen by analyzing the offset event. When the operation event further includes a click event, the execution device may respond to the relevant click event at a position to which the mouse pointer is finally moved.

In an alternative implementation, the running device may also generate a positioning page first. The positioning page can be covered on the page to be clicked and responds to the click event received by the system program preferentially. After or at the same time as the positioning page is generated, the click control module may send an operation request to the pointing device. The operation request may include a preset click mode, such as a left click. The pointing device may generate a pointing event according to a pointing mode included in the first pointing request, and send the pointing event to the operating device. And after receiving the click event, the system program of the running equipment responds to the click event through the positioning page and determines the current first click position. Further, the click control module may compare the first click position with a first target position to be clicked on the page to be clicked, and determine whether the click position of the click device needs to be adjusted on the basis of the currently clicked first click position according to a comparison result. Further, an operation request is generated according to the operation request, so that the clicking device adjusts the offset distance and/or the clicking mode according to the operation request, and a new clicking event is generated, so that the clicking operation of the page to be clicked is completed.

In this way, in the implementation scenario shown in fig. 1, the click control module transmits the required operation information to the click device, and the click device directly generates the corresponding operation event according to the operation information without generating a real and physical operation, so that the effectiveness of automatic page click can be improved.

The technical idea of the present specification is described below from the viewpoint of an operating device and a pointing device, respectively.

FIG. 2 illustrates a method of page clicking of one embodiment. The execution subject of the method shown in fig. 2 may be a running device (such as the running device shown in fig. 1), i.e., a device that runs the target program. Further, the method shown in FIG. 2 may be performed by a module in the operating device, such as the click control module in FIG. 1. The module may be implemented by software or hardware, and is not limited herein. This step may be performed by a module in the operating device, such as the click control module shown in FIG. 1.

the running device can establish a connection with the clicking device in advance. The running device and the pointing device can establish connection through two mutually independent communication channels. Specifically, the running device may establish a first communication channel with a first communication module in the pointing device and a second communication channel with a second communication module in the pointing device. It is understood that the communication channel is a relatively visual description method. In actual implementation, the communication may be wired or wireless, and is not limited herein. The wireless means may be, for example, bluetooth, infrared, etc. In one embodiment, the first communication module is a bluetooth module and the second communication module is an infrared connection module, and accordingly, the first communication channel may be in a bluetooth connection mode and the second communication channel may be in an infrared connection mode. In case the first communication module and the second communication module are of the same type of communication module, the first communication channel and the second communication channel may also have different channel identifications. For example, the channel of the first communication channel is identified as "4089", the channel of the second communication channel is identified as "1078", and so on.

it is worth noting that computer devices, such as desktop computers, mobile terminals, etc., typically reserve a dedicated input device interface for the device. Devices connected through a dedicated interface are identified as system-level input devices. In an implementation architecture of the present specification, the second communication channel may be established based on an input-specific interface of the operating device. In this way, information sent by the pointing device to the runtime device may be received and recognized by the system process.

The method shown in fig. 2 comprises the following steps: step 201, when a page to be clicked appears in the running process of a target program, sending an operation request to a clicking device through a first communication channel so that the clicking device converts the operation request into an operation event, and feeding back the operation event through a second communication channel, wherein the operation event is used for describing offset operation and/or clicking operation indicated in the operation request; step 202, responding to the above operation event to complete the click on the page to be clicked.

First, in step 201, in a case that a page to be clicked appears in the running process of a target program, an operation request is sent to a click device through a first communication channel.

Here, the object program is determined depending on the scene. For example, in an automatic test scenario, the target program is the program being tested. In the scenario of automatic authentication or the like, the target program may be various scenarios including authentication information. The running device can run the target program. During the running process of the target program, page jump may occur, the page jump may be realized automatically through the h5 page, or may be realized through clicking elements such as hyperlinks, buttons and the like in the current page. It is understood that the page to be clicked appears during the running process of the target program, that is, the current page displayed by the target program contains clickable elements, such as hyperlinks, buttons and the like.

It can be understood that the element to be clicked in the page to be clicked, that is, the object to be clicked. Each object to be clicked typically corresponds to a position on the current page and a click pattern. Wherein the position in the current page can be represented by coordinates in the screen coordinate system. The screen coordinate system may be the system default coordinate system. Alternatively, the coordinates in the screen coordinate system may be represented by pixels, for example (1, 2) may represent one pixel laterally from the origin of coordinates and 2 pixels longitudinally from the origin of coordinates. The click pattern may be, for example, a left click, a left double click, a right click, etc.

When there are multiple click objects in the page to be clicked, the current process may be directed to one of the positions to be clicked and the click mode corresponding thereto. In this specification, for convenience of description, it may be assumed that the currently targeted object to be clicked is the first object to be clicked, and a position and a click mode corresponding to the first object to be clicked may be written as a first target position and a first click mode targeted at the first target position, respectively.

under the condition that a page to be clicked appears in the running process of the target program, a corresponding operation request can be generated and sent to the clicking device through the first communication channel. The corresponding offset and/or click requirement may be indicated in the operation request. It can be understood that after the connection between the pointing device and the running device is established, any operation of the pointing device will generate a corresponding operation event to be fed back to the running device. Taking a mouse as an example, the relative position deviation of the mouse can be represented by a mouse pointer of a device connected with the mouse. The relative movement of the mouse and the distance of movement of the mouse pointer on the screen may have a certain mapping relationship. The various click modes of the mouse also produce a response at the current position of the mouse pointer. Under the technical idea of the present specification, the pointing device may not physically move and click, but needs to generate an operation event similar to a mouse, i.e., an event corresponding to an offset operation and/or a click operation. The specific contents of these events (e.g. how to offset and how to click) can be indicated by an operation request sent from the running device to the clicking device.

The clicking equipment can convert the operation request sent by the running equipment into a corresponding operation event by analyzing the operation request, and the operation event is fed back to the running equipment through the second communication channel by using the second communication module. It is understood that the operation events sent by the pointing device to the execution device may be at a system level, and the system may respond to the corresponding operation events according to the priority level of each application currently running. Therefore, in step 202, the clicking operation on the page to be clicked can be completed on the basis of responding to the operation event.

it will be appreciated that in practice, where the operational events generated by the pointing device include offset events, the offset in the offset events will typically be the relative offset of the pointing device. In the technical idea of the present specification, the relative offset corresponding to the pointing device is a virtual relative offset, that is, the pointing device does not generate an offset on the physical entity, but has the same meaning as the pointing device generates the relative offset. The relative offset of the pointing device and the unit of offset on the screen of the pointing device corresponding to the pointing device in the running device may be the same or different. For example, in a conventional pointing device mouse, the relative offset may be calculated in units of millimeters, centimeters, or the like, while the cursor may be calculated in units of pixels. In the event of an operation, if the data being transferred does not have the same metric, a conversion rule is required between the two. This conversion rule is negligible in the manual process because it is visible, however, for the auto-click scenario, the running equipment is required to indicate the corresponding offset. For various situations, when generating an operation request, different embodiments may be available, and there are different corresponding ways to operate an event.

According to one embodiment, the position of the pointing device's corresponding pointing pointer (e.g., mouse pointer) in the operating device cannot be determined, and needs to be first pointed. The operation request sent to the pointing device in step 201 may be a click request. The purpose of this click is to determine the current click position, i.e. the position at which the pointing pointer is currently pointing. Therefore, the current operation may only include a click operation of an arbitrary click pattern (without an offset). For convenience of description, the operation request generated in this case may be referred to as a first operation request herein. The running device may generate the first operation request according to a predetermined click mode, or randomly designate a click mode to generate the first operation request, which is not limited herein. And the clicking equipment generates a first clicking event according to the first operation request and feeds back the first clicking event.

it can be understood that, if the first click event is responded through the current page to be clicked, since the click position corresponding to the first click event cannot be determined in advance, if the current page to be clicked is the first target position on the page to be clicked, and the click mode is the first click mode, the operation event is responded directly through the page to be clicked, and a possible click position may be a second target position to be clicked other than the first target position, and the like, which easily causes a click error. To prevent a click error, a positioning page may be generated before or at the same time as the first operation request is generated. As the name implies, a positioning page, i.e., an auxiliary page for positioning the click location. The positioning page is used for determining the current click position, so that the positioning page can cover the page to be clicked and preferentially responds to the operation event sent by the click device. Thus, the current click position can be obtained by positioning the page. The click location generated by a click event may be referred to herein as the first click location.

Then, the click on the page to be clicked can be completed based on the comparison of the first click position and the first target position. The comparison between the first click position and the first target position can be realized by their respective coordinates in the screen coordinate system, which is not described herein again.

Further, in one embodiment, if the first click position is consistent with the first target position, the positioning page may be cancelled, and the second operation request may be sent to the click device through the first communication channel. The second operation request here indicates the first click mode. Thus, the click event generated by the pointing device is a click operation performed in situ according to the first click mode, and is referred to as a second operation event. At this time, the click of the first target position according to the first click mode can be completed by responding to the second operation event through the page to be clicked.

In another embodiment, if the first click position and the first target position do not coincide, the click position may need to be adjusted from the first click position to the first target position. At this time, coordinates of the first click position and the first target position in the screen coordinate system may be acquired, respectively, a coordinate offset amount from the first click position to the first target position may be determined in accordance with the coordinates of the first click position and the first target position, and the third operation request may be generated based on at least the coordinate offset amount. And sending the third operation request to the clicking equipment so that the clicking equipment generates a third operation event according to the third operation request, responding to the third operation event and finishing clicking the page to be clicked.

it will be appreciated that, in practice, it may be pre-stored that the offset corresponding to the pointing device and the offset of the pointing position (i.e. the offset of the pointing component in the operating device) may have a predetermined conversion rule. For example, a translation ratio of 1:2, i.e., the amount of offset for a pointing device is 1 unit (e.g., millimeters), and the amount of offset for a pointing location is 2 units (e.g., pixels). In the present embodiment, the amount of shift of the click position is known, that is, the amount of coordinate shift from the first click position to the first target position. Thus, in an alternative implementation, the offset corresponding to the pointing device may be determined according to the coordinate offset and a predetermined conversion rule. It should be noted that, in the conventional technology, the offset corresponding to the pointing device is the relative movement amount of the pointing device, and in the present embodiment, the offset corresponding to the pointing device converted by the operation request is a virtual offset, and an offset event consistent with the offset corresponding to the actual movement of the pointing device is generated.

As a simple example, assuming that the coordinate offset between the first click position and the first target position is (200, 400), the conversion rule is: if the translation ratio between the offset corresponding to the pointing device and the offset of the pointing position (i.e., the coordinate offset) is 1:2, the offset corresponding to the pointing device is (100, 200).

and generating a third operation request according to the coordinate offset and the first click mode, and after the third operation request is sent to the click equipment, converting the third operation request into a corresponding operation event by the click equipment for feedback. That is, in the foregoing example, the operation request includes a first click pattern corresponding to a first target position, which is shifted by 100 pixels in the horizontal direction, and is shifted by 200 pixels in the vertical direction. The operation information generated by the pointing device includes an offset of 100 units horizontally, an offset of 200 units vertically, and a click event through the first click pattern. At this time, the positioning page can be cancelled, and the click of the first target position on the page to be clicked is completed by responding to the corresponding operation event through the page to be clicked.

In practice, if a conversion rule of the offset corresponding to the pointing device and the offset of the pointing position (i.e., the offset of the positioning pointer in the operating device) is not stored in advance, an accurate operation cannot be directly indicated. Under the inventive concept of the present specification, the pointing device does not actually shift, but receives an operation request sent by the operating device through the first communication module, and converts the operation request into a corresponding pointing event, wherein the shift of the pointing device is virtualized in the pointing event. Thus, a click test is required for the actual click effect of the click event.

In an alternative implementation, the click position may be iteratively transformed by using the idea of manual operation until the click position is consistent with the first target position. Specifically, the coordinate offset is used as the offset corresponding to the pointing device, and a third operation request is generated. The third operation request may also indicate any of the aforementioned click events or a click event of a predetermined click pattern. In this way, the pointing device can generate a third operation event offset according to the third operation request. Responding to the three operation events through the positioning page, determining a second click position generated by the third operation event, detecting whether the second click position is consistent with the first target position, generating a fourth operation request according to a detection result, sending the fourth operation request to the click equipment, and so on until the click position corresponding to the operation event sent by the click equipment is consistent with the first target position.

In another alternative implementation manner, the conversion rule of the offset corresponding to the pointing device and the offset of the pointing position (i.e., the offset of the positioning pointer in the operating device) may be determined by clicking the obtained pointing position again, and the pointing is performed according to the obtained conversion rule. Specifically, the third operation request is generated by taking any offset or coordinate offset of the first click position and the first target position as the offset corresponding to the pointing device. The third operation request may further indicate a click event of a predetermined click pattern. In this way, the pointing device can generate a third operation event offset according to the third operation request. And responding to a third operation event through the positioning page, determining a second click position generated by the third operation event, and detecting the coordinate offset of the second click position and the first click position. According to the relationship between the offset in the third operation request and the coordinate offset of the second click position relative to the first click position, a conversion rule between the offset corresponding to the click device and the offset of the click position can be determined. And then, generating a corresponding operation event according to the conversion rule, which is not described herein again.

In each of the above embodiments, the process of sending the operation request to the pointing device and responding to the operation event sent by the pointing device each time the device is operated may be regarded as a specific implementation process of the flow of fig. 2, and details are not described here. In each of the above embodiments, the flow shown in fig. 2 is executed at least twice.

according to another embodiment, the coordinates of the final click position in the previous operation event may be recorded in advance. At this time, when the operation event is generated currently, the page is not positioned as required, but the operation event is generated according to a conversion rule between the coordinate offset from the first click position to the first target position, the offset corresponding to the click device and the offset of the click position, and the first click mode, and the click on the first target position according to the first click mode is completed directly according to the operation event fed back by the click device.

next, referring to fig. 3, fig. 3 illustrates a pointing method applied to a pointing device that establishes a connection with an operating device in advance. The operating equipment is used for operating the target program, and the clicking equipment establishes a first communication channel and a second communication channel with the operating equipment through the first communication module and the second communication module respectively. The second communication channel is established based on an input special interface of the running equipment, so that the running equipment responds to the operation event of the pointing equipment through the system process.

the method shown in fig. 3 comprises the following steps: step 301, receiving an operation request sent by an operating device through a first communication channel, wherein the operation request is generated by the operating device when a page to be clicked appears in the operating process of a target program; step 302, converting the operation request into an operation event of the pointing device, wherein the operation event is used for describing the offset operation and/or the pointing operation indicated in the operation request; step 303, sending the operation event to the running device.

First, in step 301, an operation request sent by a running device is received through a first communication channel. Here, the first communication channel is a channel for communicating with the operating device through the first communication module. The communication channel is a relatively-visual speech, and the essence of the communication channel is a communication mode predetermined between the first communication module and the operating device. Taking bluetooth as an example, the first communication module and the running device may have a dedicated connection channel. The connection channel may be represented by a channel identifier such as a channel number or may be distinguished from other communication channels by a specific format agreed upon. The operation request sent by the running device may have a default channel identifier or a specific format. In this way, the pointing device can recognize that it needs to receive the information by the appointed channel identifier or the specific format. And the second communication module does not recognize and process according to the channel identification agreed with the first communication module or the information with a specific format. This corresponds to establishing a dedicated first communication channel between the first communication module and the operating device.

In an automatic click scenario, the operation request may be generated by the running device when a page to be clicked appears in the running process of the target program, and indicates an offset operation and/or a click operation corresponding to the click device.

Then, in step 302, the operation request is converted into an operation event of the pointing device, wherein the operation event is used for describing the offset operation and/or the pointing operation indicated in the operation request. In this step, the information in the operation request may be parsed and information communicated over the second communication channel may be generated. Similarly to the first communication module, the second communication module and the operating device may have a dedicated connection channel, that is, a second communication channel. The second communication channel may also be represented by a channel identification, such as a channel number, or distinguished from other communication channels by a specific format agreed upon. The operation event sent by the pointing device may have an agreed channel identifier or a specific format. In this way, only the running device can receive and identify the operation event.

As a specific example, in the case where the first communication channel and the second communication channel are both wireless communication channels, assuming that the operating device sends broadcast information "4089 #0, 0# $1 $", which indicates a request to be sent to the communication module with a channel identification of 4089, #0, 0# indicates horizontal movement of 0 pixels and vertical movement of 0 pixels, and in $1$, 1 indicates left click, # and $ are both in a special format, the first communication module in the pointing device recognizes information sent to its own channel according to the header "4089" of the broadcast information, and can receive and parse the information. Then, the information is encapsulated according to the information sending format of the second communication module as follows: "1078% 0, 0, 1", where 1078 is the channel id of the second communication module,% is the specific format of the agreed operation event, and the encapsulated information "1078% 0, 0, 1" is the information expressing the operation event. It can be seen that, in the conversion process, the offset and/or click information contained in the operation event is not changed relative to the operation request, and only the packaging form is changed.

step 303, sending the operation event to the running device. For example, the second communication module broadcasts the above encapsulation information "1078% 0, 0, 1". At this point, the running device may identify the information as a click event by 1078 identification.

through the above process, in the flow of the clicking method shown in fig. 3, the clicking device is used to transparently transmit data. That is, data received through the first communication module is propagated out through the second communication module. The reason for this is that, in the conventional technology, some terminals have a sandbox mechanism, and non-system programs do not have authority to control the position of a positioning component (such as a mouse pointer) and a clicking operation, so that a corresponding operation requirement needs to be converted into an operation event through a clicking device by means of a clicking device and forwarded to a running device. It will be appreciated that the connection of the pointing device to the running device via the second communication module is similar to the connection of a mouse to a computer in the conventional art. In this way, the second communication module and the running device can perform system-level communication, that is, the running device can receive information sent to the running device through the second communication module through the system process, so that various applications running on the running device can be oriented.

It is worth noting that the embodiments described in fig. 2 and 3 describe the interaction period from the run device and the click device, respectively. In each interaction period, the running equipment is a master side and can generate different operation information according to requirements, and the clicking equipment is a slave side and is converted into corresponding operation events according to the received operation requests. In each particular embodiment, the descriptions with respect to fig. 2 and 3 are coordinated and adapted to each other.

To more clearly describe the cooperation and adaptation between the process executed by the execution device and the process executed by the pointing device, please refer to fig. 4, which shows an interaction diagram of an embodiment.

as shown in fig. 4, first, in S401, a target program running on the running device appears as a page to be clicked. The page to be clicked may include at least a first target position to be clicked and a first click pattern for the first target position, such as a single click, a double click, and the like.

it should be noted that, in the case that the execution subject of the page clicking method is a module in the running device, the module may be embedded in the target program, or may be a module independent of the target program, and is not limited herein. When the module is embedded in the target program, the module can be packaged in a mode of SDK, for example, when a page to be clicked appears in the target program, the first target position and the first click mode are used as transfer parameters, and a method for executing page click by the module is called. When the module is independent of the target program, the target program can inform the module in an event form when a page to be clicked appears, so that the module can execute a page clicking method through event triggering.

Next, in S402, a positioning page is generated. The positioning page may be overlaid on top of the current page, preferably in response to a click event. For positioning convenience, the positioning page and each position on the current page can be represented by a screen coordinate system. And converting the position relation through the coordinate system under the condition that the positioning page and the current page are positioned in different coordinate systems.

Simultaneously with or after S402, in S403, an operation request is generated and sent to the pointing device. The operation request is used to instruct the pointing device to generate a first click on the current page to be clicked (the current page to be clicked is not necessarily on the current page to be clicked). The operation request is also an operation request sent for the first time for the current page to be clicked, and may indicate a simple click event.

The purpose of this click is to determine the current click location. Thus, the current click may be a click of any click pattern. The operating device may send the click request to the pointing device according to a predetermined click mode, or may randomly designate a click mode to send the click request to the pointing device, which is not limited herein.

Further, in S404, the pointing device receives the operation request sent by the execution device through the first communication module, and converts the operation request into a first pointing event. It will be appreciated that conventional pointing devices, such as a mouse, generate a click event by physically relative displacement and/or keystroke actions. Under the inventive concept of the present specification, the pointing device may not perform the relative offset and/or the key action, but directly convert into the operation event according to the operation request of the operating device instead of the physical relative offset and/or the key action of the pointing device.

In this step, the relative offset and/or the key pressing action included in the first click request may be converted into the format of the click event, and then the corresponding channel identifier of the second communication module is added, or the specific format is agreed.

Next, in S405, the pointing device may feed back the first pointing event to the running device through the second communication module. The communication mode between the second communication module and the operating device has been described above, and is not described herein again.

Then, in S406, the execution device may locate a current first click location in response to the first click event through the location page. Since the execution device generates the positioning page in S402 and preferentially responds to the click event, the first click event received by the execution device system process herein may respond through the positioning page. Here, the direct response of the page to be clicked is not performed, so that the click position generated by the current click event can be avoided, and the error click is caused under the condition that other buttons or hyperlinks and other elements capable of responding to the click exist, so that the page generates uncontrollable jump and unnecessary confusion is caused.

next, in S407, it may be compared whether the first click position coincides with the first target position, and a new operation request may be generated according to the comparison result. In this step, it may be compared whether the coordinates of the first click position and the first target position in the screen coordinate system coincide. If the first click position and the first target position are represented by coordinates in different coordinate systems, the coordinates of the first click position and the first target position can be converted into coordinates in the same coordinate system for representation and then compared.

It is understood that, in the case where the two are identical, the new operation request may indicate only one click operation of the current click position. In the case where the two are not in agreement, the new operation request may indicate both the offset operation and the click operation. Here, the shift operation may be a shift of the first click position to the first target position, and the click operation may be represented by a shift amount and may be represented by a first click pattern.

in S408, the running device sends a new operation request to the pointing device through the first communication channel.

Then, in S409, the pointing device receives a new operation request, generates a new operation event accordingly, and sends new operation information to the running device through S410. When the new operation request only indicates sequential click operation, the new operation event only includes a click event, and when the new operation request indicates two parts, namely offset operation and click operation, the new operation event may include an offset event and a click event corresponding to the click device.

further, in S411, the positioning page is undone, and the click on the first target position is completed according to the new operation request.

Fig. 4 shows an interaction flow illustrating two interaction processes. It can be seen that in each interactive flow, the running device can generate different operation requests according to different requirements. And performing other auxiliary operations according to the operation request, such as generating a positioning page, canceling the positioning page and the like. The clicking equipment performs an information conversion operation in each process, and converts what information when receiving the information, which is equivalent to the assistance of the clicking process in the running equipment.

Reviewing the above process, in the process of clicking the page, the operating device generates an operation request according to the position and the clicking mode which need to be clicked, sends the operation request to the clicking device, converts the operation request into an operation event by the clicking device and feeds the operation event back, and then responds to the operation event to finish clicking on the page to be clicked. The operation request is sent to the clicking equipment and the operation event is received from the clicking equipment through different communication channels, and the communication channel for sending the operation event to the running equipment by the clicking equipment is realized through an input special interface of the running equipment, so that the operation event is converted into the operation event of the system process and can be responded by the target program. The above process automatically completes the page click in the operating device by the aid of one click device, and the effectiveness of the page click can be improved.

According to an embodiment of another aspect, a device for page clicking is also provided. FIG. 5 shows a schematic block diagram of a device for page clicking according to one embodiment. The apparatus 500 may be provided in an execution device that executes an object program. The apparatus 500 comprises: the sending unit 51 is configured to send, when a page to be clicked appears in an operating process of a target program, an operation request to the pointing device through the first communication channel so that the pointing device converts the operation request into an operation event to be fed back through the second communication channel, where the operation event is used for describing an offset operation and/or a pointing operation indicated in the operation request, where the pointing device establishes a first communication channel and a second communication channel with the operating device through the first communication module and the second communication module in advance, and the second communication channel is established based on an input dedicated interface of the operating device; the response unit 52 is configured to respond to the operation event to complete the click on the page to be clicked.

it should be noted that the apparatus 500 shown in fig. 5 is an apparatus embodiment corresponding to the method embodiment shown in fig. 2, and the corresponding description in the method embodiment shown in fig. 2 is also applicable to the apparatus 500, and is not repeated herein.

According to an embodiment of another aspect, a pointing device is also provided. FIG. 6 illustrates a schematic block diagram of a pointing device, according to one embodiment. The pointing device 600 is used to click on a page on a running device to which a connection is previously established. The pointing device 600 includes: the first communication module 61 is configured to establish a first communication channel with the running device, and receive an operation request sent by the running device through the first communication channel, where the operation request is generated by the running device when a page to be clicked appears in the running process of the target program; the conversion module 62 is configured to convert the operation request into an operation event of the pointing device, where the operation event is used to describe an offset operation and/or a pointing operation indicated in the operation request; and the second communication module 63 is configured to establish a second communication channel with the running device through the input dedicated interface of the running device, and send the operation event to the running device through the second communication channel.

in one possible design, the pointing device may be implemented by a mouse.

It should be noted that the apparatus 600 shown in fig. 6 is an apparatus embodiment corresponding to the method embodiment shown in fig. 3, and the corresponding description in the method embodiment shown in fig. 3 is also applicable to the apparatus 600, and is not repeated herein.

According to an embodiment of another aspect, there is also provided a computer-readable storage medium having stored thereon a computer program which, when executed in a computer, causes the computer to perform the method described in connection with fig. 2 or fig. 3.

According to an embodiment of yet another aspect, there is also provided a computing device comprising a memory and a processor, the memory having stored therein executable code, the processor, when executing the executable code, implementing the method described in connection with fig. 2 or fig. 3.

Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in the embodiments of this specification may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium.

The above-mentioned embodiments are intended to explain the technical idea, technical solutions and advantages of the present specification in further detail, and it should be understood that the above-mentioned embodiments are merely specific embodiments of the technical idea of the present specification, and are not intended to limit the scope of the technical idea of the present specification, and any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the embodiments of the present specification should be included in the scope of the technical idea of the present specification.

Claims

1. A method for clicking a page is applied to operating equipment which is connected with clicking equipment in advance, the operating equipment is used for operating a target program, the clicking equipment is respectively provided with a first communication channel and a second communication channel with the operating equipment through a first communication module and a second communication module, and the second communication channel is established on the basis of an input special interface of the operating equipment; the method comprises the following steps:

Under the condition that a page to be clicked appears in the running process of a target program, sending an operation request to the clicking equipment through the first communication channel so that the clicking equipment converts the operation request into an operation event, and feeding back the operation event through the second communication channel, wherein the operation event is used for describing offset operation and/or clicking operation indicated in the operation request;

And responding to the operation event to finish clicking on the page to be clicked.

2. the method of claim 1, wherein the sending the operation request to the pointing device through the first communication channel comprises:

Generating a positioning page and a first operation request, wherein the positioning page is used for covering the page to be clicked and preferentially responding to an operation event sent by the clicking equipment, and the page to be clicked at least corresponds to a first target position to be clicked and a first clicking mode aiming at the first target position;

sending the first operation request to the clicking device so that the clicking device generates a first clicking event according to the first operation request, wherein the first clicking event is used for describing clicking operation of the clicking device according to a preset clicking mode;

The responding to the operation event to complete the click on the page to be clicked comprises:

Responding to the first click event through the positioning page, and determining a first click position of a current click;

And finishing clicking on the page to be clicked based on the comparison between the first clicking position and the first target position.

3. The method of claim 2, wherein, in the event that the first click location and the first target location coincide, the responding to the operational event to complete the click on the page to click further comprises:

Canceling the positioning page;

Sending a second operation request to the pointing device through the first communication channel so that the pointing device converts the second operation request into a second operation event to perform feedback through the second communication channel, wherein the second operation event is used for describing a pointing event corresponding to the first pointing mode;

And responding to the second operation event through the page to be clicked, so that clicking on the page to be clicked is completed.

4. The method of claim 2, wherein in the event that the first click location and the first target location do not coincide, the responding to the operational event to complete the click on the page to click further comprises:

respectively acquiring coordinates of the first click position and the first target position in a screen coordinate system;

determining the coordinate offset from the first click position to the first target position according to the coordinates of the first click position and the first target position;

generating a third operation request at least based on the coordinate offset, and sending the third operation request to the click equipment so that the click equipment can generate a third operation event according to the third operation request;

And responding to the third operation event, and finishing clicking the page to be clicked.

5. the method of claim 4, wherein the generating a third operation request based at least on the coordinate offset comprises:

Determining an offset corresponding to the pointing device according to the coordinate offset and a preset conversion rule, wherein the conversion rule is used for describing a mapping relation between a moving distance corresponding to the pointing device and an offset distance of a pointing position corresponding to the pointing device on a screen;

Generating the third operation request according to the offset corresponding to the pointing device and the first pointing mode, so that the pointing device generates the offset corresponding to the movement of the pointing device according to the third operation request and performs a third operation event of pointing according to the first pointing mode;

The completing the click on the page to be clicked in response to the third operation event further comprises:

And canceling the positioning page, and responding to the third operation event through the page to be clicked.

6. the method of claim 4, wherein the generating a third operation request based at least on the coordinate offset comprises:

Taking the coordinate offset as an offset corresponding to the click equipment to generate a third operation request so that the click equipment can generate a third operation event offset according to the offset;

Responding to the third operation event through the positioning page, and determining a second click position generated by the third operation event;

And detecting whether the second click position is consistent with the first target position, generating a fourth operation request according to a detection result, and sending the fourth operation request to the click equipment, and so on until the click position generated by the operation event fed back by the click equipment is consistent with the first target position.

7. a clicking method is applied to clicking equipment which is connected with operating equipment in advance, the operating equipment is used for operating a target program, the clicking equipment is respectively provided with a first communication channel and a second communication channel with the operating equipment through a first communication module and a second communication module, and the second communication channel is established on the basis of an input special interface of the operating equipment; the method comprises the following steps:

Receiving an operation request sent by the running equipment through the first communication channel, wherein the operation request is generated by the running equipment under the condition that a page to be clicked appears in the running process of a target program;

Converting the operation request into an operation event of the clicking equipment, wherein the operation event is used for describing the offset operation and/or the clicking operation indicated in the operation request;

and sending the operation event to the running equipment.

8. The method of claim 7, wherein the operational event comprises one of:

The system comprises a display device, a display device and a display device, wherein the display device is used for describing a click event of the click device according to a preset click mode;

The mobile event is used for describing the mobile event of the pointing device moving according to the preset offset and the pointing event of the pointing device moving according to the preset pointing mode.

9. A page clicking device is arranged on running equipment running a target program, and comprises:

the sending unit is configured to send an operation request to the clicking device through a first communication channel under the condition that a page to be clicked appears in the running process of a target program, so that the clicking device converts the operation request into an operation event to be fed back through a second communication channel, wherein the operation event is used for describing offset operation and/or clicking operation indicated in the operation request, the clicking device is respectively provided with a first communication channel and a second communication channel with the running device through a first communication module and a second communication module in advance, and the second communication channel is established based on an input special interface of the running device;

And the response unit is configured to respond to the operation event so as to finish clicking on the page to be clicked.

10. A clicking device is used for clicking a page on running equipment which is connected in advance; the pointing device includes:

the system comprises a first communication module, a second communication module and a third communication module, wherein the first communication module is configured to establish a first communication channel with running equipment and receive an operation request sent by the running equipment through the first communication channel, and the operation request is generated by the running equipment under the condition that a page to be clicked appears in the running process of a target program;

The conversion module is configured to convert the operation request into an operation event of the pointing device, wherein the operation event is used for describing an offset operation and/or a pointing operation indicated in the operation request;

And the second communication module is configured to establish a second communication channel with the running equipment through an input special interface of the running equipment, and send the operation event to the running equipment through the second communication channel.