CN115794593A

CN115794593A - Method, device, equipment, storage medium and program product for determining operation position

Info

Publication number: CN115794593A
Application number: CN202111053426.XA
Authority: CN
Inventors: 许简; 邵珊珊
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Beijing ByteDance Network Technology Co Ltd
Priority date: 2021-09-09
Filing date: 2021-09-09
Publication date: 2023-03-14

Abstract

The invention provides a method, a device, equipment, a storage medium and a program product for determining an operation position, and relates to a program testing technology.

Description

Method, device, equipment, storage medium and program product for determining operation position

Technical Field

The disclosed embodiments relate to program testing technologies, and in particular, to a method, an apparatus, a device, a storage medium, and a program product for determining an operation location.

Background

Generally, the developed software needs to be tested, for example, whether the software can normally display the content or normally log in can be tested. And the login test is basically a necessary step in the software testing process.

In order to improve the efficiency of software testing, software is usually tested in an automated testing manner. When software is tested, controls which need to be operated in sequence when an account is logged in need to be observed in advance, and a set of logging steps and logic are maintained, so that the accounts can be logged in only by operating the controls in sequence when automatic testing is performed.

However, different software or different versions of the same software have different positions or shapes of the login control in the login interface, which results in different interaction logics when different software is automatically logged in, so that in the prior art, when an automatic login test is performed, a set of interaction logics needs to be maintained in advance for each program to be tested, which reduces the test efficiency and consumes human resources during the test.

Disclosure of Invention

The embodiment of the disclosure provides a method, a device, equipment, a storage medium and a program product for determining an operation position, so as to improve the efficiency of login testing of a program.

In a first aspect, an embodiment of the present disclosure provides a method for determining an operation location for logging in an account, including:

under the condition that the program to be tested is not logged in, acquiring an image of a current page of the program to be tested;

determining a page type corresponding to the current page according to the image of the current page;

and determining a control trigger chain for realizing login operation according to the page category, and sequentially determining and feeding back information of a control to be operated according to the control trigger chain, wherein the information of the control to be operated represents an operation control which needs to be operated when the control is used for logging in an account.

In a second aspect, an embodiment of the present disclosure provides an apparatus for determining an operation location for logging in an account, including:

the acquisition unit is used for acquiring an image of a current page of the program to be tested under the condition that the program to be tested is not logged in;

the category determining unit is used for determining a page category corresponding to the current page according to the image of the current page;

and the control information feedback unit is used for determining a control trigger chain for realizing login operation according to the page type, and sequentially determining and feeding back information of a control to be operated according to the control trigger chain, wherein the information of the control to be operated represents an operation control which needs to be operated when the account is logged in.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;

the memory stores computer execution instructions;

the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the method of determining an operating position for a login account as set forth in the first aspect and various possible designs of the first aspect.

In a fourth aspect, the present disclosure provides a computer-readable storage medium, where computer-executable instructions are stored, and when a processor executes the computer-executable instructions, a method for determining an operation location for logging in an account according to the first aspect and various possible designs of the first aspect is implemented.

In a fifth aspect, embodiments of the present disclosure provide a computer program product comprising a computer program that, when executed by a processor, implements a method of determining an operating position for a login account as set forth in the first aspect above and in various possible designs of the first aspect.

The method, apparatus, device, storage medium and program product for determining an operation location provided in this embodiment include: under the condition that a program to be tested is not logged in, acquiring an image of a current page of the program to be tested; determining a page type corresponding to the current page according to the image of the current page; determining a control trigger chain for realizing login operation according to the page type, and sequentially determining and feeding back information of a control to be operated according to the control trigger chain, wherein the information of the control to be operated represents an operation control which needs to be operated when the control to be operated is used for logging in an account. According to the scheme, when the program to be tested does not log in the account, the category of the program page can be identified, the control trigger chain is determined according to the category, the information of the control to be operated is determined in the current page of the program to be tested according to the control trigger chain, operation can be carried out based on the information of the control to be operated, a set of interaction logic does not need to be generated for each program to be tested, the program to be tested is controlled to automatically log in the account through the method, and testing efficiency can be improved.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to the drawings without inventive exercise.

Fig. 1 is a flowchart illustrating a method for determining an operation position for logging in an account according to a first exemplary embodiment of the disclosure;

FIG. 2 is a schematic diagram illustrating an interaction between a server and test equipment according to an exemplary embodiment of the present disclosure;

FIG. 3 is a diagram illustrating a login page in accordance with an illustrative embodiment;

FIG. 4 is a schematic illustration of a popup window shown in an exemplary disclosed embodiment;

FIG. 5 is a diagram illustrating a content page, disclosing an example embodiment;

fig. 6 is a flowchart illustrating a method for determining an operation position for logging in an account according to a second exemplary embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a landing page and its lower pages shown in an exemplary embodiment of the present disclosure;

fig. 8 is a schematic diagram of a login page and a lower page thereof shown in another exemplary embodiment of the present disclosure;

FIG. 9 is a schematic diagram of a page image shown in an exemplary embodiment of the present disclosure;

fig. 10 is a flowchart illustrating a method for determining an operation position for logging in an account according to a third exemplary embodiment of the present disclosure;

FIG. 11 is a schematic diagram of a page image shown in an exemplary embodiment of the present disclosure;

fig. 12 is a schematic structural diagram illustrating an apparatus for determining an operation position for logging in an account according to an exemplary embodiment of the present disclosure;

fig. 13 is a schematic structural diagram of an apparatus for determining an operation position for logging in an account according to another exemplary embodiment of the present disclosure;

fig. 14 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure.

Detailed Description

To make the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions in the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

When the functions of the software are tested, a login test is set in a common test project, so that whether the software can normally log in an account is verified. Meanwhile, in order to improve the testing efficiency, the software is generally tested in an automated manner, which also includes testing the login function of the software.

However, each time the software is automatically tested by using the scheme in the prior art, a set of interaction logic of the program to be tested needs to be maintained in advance, so that the login function of the program to be tested is tested according to the interaction logic maintained in advance. However, this process requires the user to maintain a set of interaction logic in advance, which consumes additional human resources and reduces the testing efficiency of the software.

In order to solve the technical problem, the scheme provided by the disclosure can identify the image of the page displayed by the program to be tested so as to determine the category of the page, and then identify the operation position for logging in the account in the page according to the control trigger chain corresponding to the category of the page, so as to improve the testing efficiency. The method only needs to maintain the control trigger chains corresponding to various pages in advance, and the control trigger chains can be reused by a plurality of programs to be tested, so that human resources can be saved.

Fig. 1 is a flowchart illustrating a method for determining an operation position for logging in an account according to a first exemplary embodiment of the present disclosure.

As shown in fig. 1, the method for determining an operation location for logging in an account provided by the present disclosure includes:

step 101, under the condition that a program to be tested is not logged in, acquiring an image of a current page of the program to be tested.

The scheme provided by the disclosure can be applied to a server side, and when a program to be tested is tested, an image of a current display page of the program to be tested can be acquired under the condition that the program to be tested is not logged in.

Optionally, the server may actively acquire an image of a currently displayed page of the program to be tested, or may be sent to it by other devices.

Optionally, a program to be tested may be set in the testing device, the testing device sends a request to the server, so that the server determines information of the control to be operated in the image according to the received request, and then feeds back the information of the control to be operated to the testing device, and the testing device may operate according to the received information of the control to be operated, thereby logging in the account.

Fig. 2 is a schematic diagram illustrating an interaction between a server and a testing device according to an exemplary embodiment of the disclosure.

As shown in fig. 2, the testing device 21 may send a request to the server 22 to request the server 22 to feed back information of the control to be operated.

The server 22 may process the received request based on the scheme provided by the present disclosure, determine information of the control to be operated, and send the information of the control to be operated to the testing device 21.

Specifically, the request sent by the testing device to the server may include an image of a current page of the program to be tested.

When the program is tested, the program to be tested needs to be operated, and because no personnel participates in the automatic testing process, if the program to be tested needs to be logged in, the server needs to determine a specific operation position, so that the server can obtain a page image of the program to be tested, and determine the information of a control to be operated based on the page image, and the information of the control to be operated can represent the control to be operated and the position of the control.

Optionally, the testing device may send a request to the server before each operation, where the request may include a screenshot of a currently displayed page of the program to be tested, so that the server feeds back information of the control to be operated. Therefore, the server can record the test condition of each program to be tested in each test device, and when the test device sends the request for determining the operation position of the program to be tested again, the server can also determine whether the request is a step in the existing test task according to the recorded information.

Optionally, the request sent by the testing device to the server may further include an identifier of the testing device, and may further include an identifier of the program to be tested. For example, if the program to be tested by the testing device a is app1, the request sent by the testing device a to the server may include the identifier of the testing device a and the identifier of the app 1.

The server may determine the state of the program to be tested according to the identification information in the request, for example, the state may include: no login, initial login attempt, no response to a jam. Specifically, the server may further determine whether the received request is a new task according to the identification information, for example, whether information that the testing device indicated by the device identifier tests the program to be tested indicated by the program identifier exists may be queried, if the information exists, it may be determined that the request is not a new task, and if the information does not exist, it may be determined that the request is a new task.

If the task is a new task, the server can directly determine that the program to be tested is in an unregistered state.

If the task is not a new task, the server can inquire the state of the program to be tested in the testing equipment and determine the state of the program to be tested according to the inquiry result.

If the server determines that the program to be tested in the testing device is stuck, the server may not process the request. If the server determines that the program to be tested in the testing device is in the unregistered state, the server can feed back the operation position information to the testing device based on the scheme disclosed by the invention, and then the testing device operates and logs in the account based on the received operation position information.

Furthermore, a plurality of test devices can be connected with the server, and different test devices can test the same program to be tested and can also test different programs to be tested. And the test progress of each test device can be the same or different.

And 102, determining the page type corresponding to the current page according to the image of the current page.

Further, the server may identify content included in the image of the current page to determine the page category. For example, key information corresponding to different page categories may be set, and if the key information corresponding to the page category a is identified in the image, the category of the page may be determined to be a.

Optionally, the image of the current page may also be input into a preset image classification model, so as to obtain a page category corresponding to the current page.

In practical application, a preset image classification model may be set in the server, and the model may be obtained by pre-training. The server may input the acquired current page image into the preset image classification model, so that the preset image classification model outputs the category of the image of the current page.

The images of the program pages can be collected in advance and labeled, and then the image with the labeling information can be used for training the model to obtain the preset image classification model.

Specifically, the classification result of the page may include a login page, a popup page, a content page, and the like. The login page refers to a page including a login key, such as a page for selecting a login mode, and further such as a page for inputting a login account. The pop-up window page refers to a page including a pop-up window to be closed, such as displaying an advertisement window to be closed on an upper layer of the page. Pages that are neither login pages nor popup pages may be considered content pages.

FIG. 3 is a diagram illustrating a landing page in accordance with an illustrative embodiment of the present disclosure.

FIG. 4 is a schematic illustration of a pop-up window page shown in an exemplary embodiment of the disclosure.

FIG. 5 is a diagram illustrating a content page, disclosing an exemplary embodiment.

Fig. 3-5 are schematic diagrams of a login page, a popup page, and a content page, respectively, according to an embodiment of the disclosure.

And 103, determining a control trigger chain for realizing login operation according to the page category, and sequentially determining and feeding back information of a control to be operated according to the control trigger chain, wherein the information of the control to be operated represents an operation control which needs to be operated when the account is logged in.

Optionally, a control trigger chain corresponding to each page category may be preset. For example, for a login page, a control trigger chain may be set, where the control trigger chain includes control information that needs to be sequentially operated, and the server may determine information of a control to be operated in the page according to the control trigger chain, and may further log in an account according to the information of the control to be operated.

Furthermore, when login operation needs to be executed, the operation flows in the pages of different types are different, so that the scheme provided by the disclosure is preset with control trigger chains for the pages of different types, a corresponding control trigger chain can be determined according to the type of the current page of the program to be tested, and the information of the control to be operated is determined in the image of the current page based on the determined control trigger chain.

In this embodiment, when the program to be tested does not log in the account, the information of which control to be operated is identified in the image can be determined according to the control trigger chain, and then the information of the control to be operated is determined in the image based on the image identification mode, so that the account can be logged in by using the information of the control to be operated, for example, the test equipment can operate based on the information of the control to be operated to log in the account.

In practical application, if it is determined that the current page is a login page, the server may determine information of a control to be operated in the login page according to the control trigger chain, may perform operation according to the information of the control to be operated, and trigger the program to be tested to display a lower page, for example, a login page for inputting account information, and the server may continue to acquire an image of the login page for inputting account information and may further feed back the information of the control to be operated.

If the current page is determined to be the popup page, the server may determine, according to the control trigger chain, buttons for representing positive options such as "allow, agree" and the like in the popup page, and may also determine, in the popup page, operation buttons for closing the popup, where the buttons are controls to be operated, and may close the popup in the page according to information of the controls to be operated, and the server may obtain an image of the page of the program to be tested again.

Specifically, if the current page is determined to be the content page, the server may determine information of the control to be operated in the content page according to the control trigger chain. The program to be tested can be tested according to the information of the control to be operated, and the login page of the program to be tested is entered, or after the operation, the first-level page of the program to be tested can be entered, wherein the first-level page comprises a login entry.

Furthermore, the information of the control to be operated represents the operation control which needs to be operated when the information is used for logging in the account, and after the server determines the information of the control to be operated in the image of the current page, the information of the control to be operated can be sent to the testing equipment, so that the testing equipment can operate on the page currently displayed by the program to be tested according to the received information of the control to be operated, and the account can be logged in.

The method for determining the operation position for logging in the account comprises the following steps: under the condition that a program to be tested is not logged in, acquiring an image of a current page of the program to be tested; determining a page type corresponding to the current page according to the image of the current page; determining a control trigger chain for realizing login operation according to the page type, and sequentially determining and feeding back information of a control to be operated according to the control trigger chain, wherein the information of the control to be operated represents an operation control which needs to be operated when the control to be operated is used for logging in an account. According to the method for determining the operation position for logging in the account, when the program to be tested does not log in the account, the type of the page of the program can be identified, the control trigger chain is determined according to the type, the information of the control to be operated is determined in the current page of the program to be tested according to the control trigger chain, operation can be performed based on the information of the control to be operated, a set of interaction logic does not need to be generated for each program to be tested, the program to be tested is controlled to automatically log in the account through the method, and testing efficiency can be improved.

Fig. 6 is a flowchart illustrating a method for determining an operation position for logging in an account according to a second exemplary embodiment of the disclosure.

As shown in fig. 6, the method for determining an operation location for logging in an account provided by the present disclosure includes:

step 601, acquiring an image of a current page of the program to be tested under the condition that the program to be tested is not logged in.

Step 601 is similar to the implementation of step 101, and is not described again.

Step 602, determining a page category corresponding to the current page according to the image of the current page.

Step 602 is similar to the implementation of step 102, and is not described again.

Step 603, if the page type is a login page, determining a login mode according to the attribute information of the program to be tested.

If the server determines that the current page of the program to be tested is the login page, the attribute information of the program to be tested can be determined. The attribute information of the program to be tested may be determined specifically according to the identification information in the request, for example, the attribute information of the program to be tested may be determined according to the identification of the program to be tested in the identification information.

Specifically, the server may determine a login mode when logging in the account according to the attribute information of the program to be tested.

Further, if the program to be tested is a third-party program, the server may determine a login mode authorized by the communication software. And if the program to be tested is an internal test program, the server determines the login mode of short message authentication.

Step 604, determining and feeding back a login entry position when the account is logged in through the login mode in the image of the current page according to the login mode; the login entry position is an operation position when entering a lower page in which an account is logged in a login manner.

In actual application, the server determines the login entry position when the account is logged in through the login mode in the image of the current page according to the determined login mode.

Optionally, the server may feed back the login entry position to the testing device, and when the testing device operates the login entry position, the server may enter a lower page of the program to be tested when the account is logged in the login manner.

Specifically, if the login mode is a login mode authorized by the communication software, the server can acquire the icon features of the communication software and extract the page features of the image of the current page when the login entry position of the account login in the login mode is determined in the image of the current page according to the login mode; the server can also match the icon characteristics and the page characteristics, and determines a login entry position for selecting communication software to authorize login in the icon of the current page according to a matching result.

Further, the server may obtain an icon of the communication software and extract a feature of the icon.

In practice, the server may extract the Features of the icon of the communication software and the page Features of the image of the current page based on Speeded Up Robust Features (SURF algorithms).

The icon features and the page features can be matched, so that the position matched with the icon features is determined in the image of the current page, and the position is used as the login entry position determined in the image of the current page and authorized to be logged in by selecting the communication software.

Specifically, the server may determine a matching point matching the icon feature in the image of the current page according to the icon feature and the page feature. In order to avoid determining the position inconsistent with the icon as the matching point matched with the icon characteristic, the server can also determine the matching points belonging to the same range according to the distance between the matching points, and eliminate the matching points with wrong matching according to the matching points in each range.

For example, the size of the icon may be determined according to the icon of the communication software, and the size of the preset range may be determined. If the distance between the matching points does not exceed the size of the preset range, the matching points can be considered to belong to the same range.

Further, the server may eliminate matching points with matching errors according to the matching points in each range. For example, if the number of matching points in a range is less than a preset value, the matching points in the range may be removed as the wrong matching points.

In actual application, the matching points can be aggregated, so that abnormal matching points are identified and determined as matching points with wrong matching.

The server can remove matching points with wrong matching, and the login entry position for selecting the authorized login of the communication software is determined in the image of the current page according to the remaining matching points. The server can directly use the positions of the rest matching points as login entrance positions for authorized login of the communication software.

Specifically, if the login mode is a short message verification login mode, determining a login entry position when the account is logged in the login mode in the image of the current page according to the login mode, including:

and acquiring the text information in the image of the current page, and determining a login entry position for selecting the short message to verify login according to the text information in the image of the current page.

Further, if the server determines that the login mode of the account is a short message verification login mode, the server can acquire the text information in the image of the current page, and then determines the login entry position of the short message verification login in the page according to the text information. For example, the position of the text short message in the image of the current page can be identified, and the position is determined as the login entry position for selecting the short message to verify login.

605, acquiring an image of a lower page of the program to be tested, which enters based on the login entry position, and determining and feeding back an operation position for logging in the account according to the image of the lower page; the operation position is used for representing a position needing to be operated when the account is logged in.

Specifically, after receiving the login entry position fed back by the server, the testing device may operate the login entry position to enter a lower page of the program to be tested when logging in the account in the login manner, where the lower page includes a position for inputting account information.

The lower page is a page displayed after the operation of the components in the current page. For example, at the click position a in the current page, the test device can display a new page, which is a lower page of the current page.

Further, the server may acquire an image of the lower page and determine an operation position for logging in the account in the lower page. For example, the screenshot of the lower page may be sent to the server by the testing device.

In practical application, the server may determine an operation position for logging in the account in the image of the lower page, and feed back the determined operation position, for example, the operation position for logging in the account may be fed back to the testing device, so that the testing device operates at the corresponding operation position to log in the account.

The information of the control to be operated can be the login entry position and the operation position for logging in the account.

If the login mode is a login mode authorized by the communication software, the lower page received by the server comprises an authorization key, so that the server can determine the key position agreeing to authorize the login of the program to be tested in the image of the lower page and can feed back the key position, and the key position is used for logging in the program to be tested by using the login mode authorized by the communication software.

Specifically, the server may obtain a pixel value of each point in the image of the lower page, and then may determine, according to the pixel value, a key position where the user agrees to authorize to log in the program to be tested, for example, a highlight region may be determined in the image of the lower page, and a region where the highlight region conforms to a preset shape is used as the key position where the user agrees to authorize to log in the program to be tested.

In another embodiment, the server may identify the text content in the lower page image, and use the area including the preset text content such as "approve", "allow", etc. as the key position for approving the authorized login to the program to be tested.

The server can feed back the determined key position to the testing equipment, so that the testing equipment can operate the key position to agree to authorize to log in the program to be tested, and further log in the program to be tested in a communication software authorization mode.

Fig. 7 is a schematic diagram of a login page and a lower page thereof according to an exemplary embodiment of the present disclosure.

As shown in fig. 7, the login page 71 is provided with a first login entry 711 and a second login entry 712 corresponding to a login method authorized by the communication software and a login method verified by the short message. Assuming that the server determines to log in the account in the login manner authorized by the communication software according to the attribute information of the program to be tested, the server may determine the location of the login entry corresponding to the login manner authorized by the communication software, such as the location of the first login entry 711, in the image of the login page.

When the testing device operates the position of the first login entry 711, the testing device can enter the lower page 72 of the program to be tested when logging in the account in the login mode authorized by the communication software.

The testing device may send an image of the lower page 72 to the server, where the server may determine the key position 721 and feed back the key position 721 to the testing device.

If the login mode is a short message verification login mode, the lower-level page is a page for inputting the mobile phone number, and the image of the lower-level page received by the server can comprise the position of inputting the mobile phone number and can also comprise an operation position for acquiring the verification code. In this embodiment, the server may determine a mobile phone number input position and an operation position for acquiring the verification code in the acquired lower page image.

For example, the server may identify, according to a pixel value of each pixel point in the lower page image, an area in which a mobile phone number is input and an operation position for acquiring the verification code, where the operation position may be a key, for example.

The server can also send the determined mobile phone number input position and the operation position for acquiring the verification code to the testing equipment so that the testing equipment can operate at the corresponding position.

When the server feeds back the mobile phone number input position, the server can also send information for representing that the position is the position of inputting the mobile phone number, so that the testing equipment obtains the mobile phone number of the local mobile phone and inputs the mobile phone number to the received mobile phone number input position.

Specifically, the test equipment can also operate according to the operation position for acquiring the verification code to acquire the verification code. For example, the test device may simulate clicking an operation position for acquiring the verification code, and further trigger the test device to send a request to a server for sending the verification code, so that the server for sending the verification code issues a short message including the verification code to the test device.

Further, the server may further acquire an image of a page for inputting the passcode, and determine a passcode input position, an operation position for confirming login, in the image of the page for inputting the passcode. For example, after the test device operates according to the operation position for acquiring the verification code, the test device may display an interface for inputting the verification code. At this time, the test device may obtain a screenshot of a currently displayed page for inputting the verification code, and send the screenshot to the server. The server may determine a location of the input validation code in the received image, confirm the location of the login, and feed back the determined location to the testing device.

In practical application, the server may identify, according to a pixel value of each pixel point in the page image for inputting the verification code, an area for inputting the verification code therein and an operation position for confirming login, where the operation position may be a key, for example.

The server can also send the determined position of the input verification code and the operation position for confirming login to the test equipment so that the test equipment operates at the corresponding position.

The operation position for logging in the account comprises the following steps: the mobile phone number input method comprises a mobile phone number input position, an operation position used for obtaining a verification code, a verification code input position and an operation position used for confirming login. The server may in turn feed this information back to the test equipment.

When the server feeds back the position of the input verification code, the server can also send information for representing that the position is the position of the input verification code, so that the testing equipment reads the short message of the local machine to obtain the verification code in the short message, and inputs the verification code to the position of the input verification code.

Specifically, the test device may further perform an operation according to the operation position for confirming the login, so as to log in according to the input verification code. For example, the test device may simulate clicking an operation position for confirming login, and further trigger the test device to log in a program to be tested.

Fig. 8 is a schematic diagram of a login page and a lower page thereof according to another exemplary embodiment of the present disclosure.

As shown in fig. 8, the login page 81 is provided with a first login entry 811 and a second login entry 812 corresponding to a login method authorized by communication software and a login method verified by short message. If the server determines to log in the account in the login manner verified by the short message according to the attribute information of the program to be tested, the server may determine a location of a login entry corresponding to the login manner verified by the short message, such as a location of the first login entry 812, in the image of the login page.

When the testing device operates the position of the first login entry 812, the page 82 for inputting the mobile phone number in the program to be tested can be entered.

The test device may send the image of the page 82 for inputting the mobile phone number to the server, where the server may determine the mobile phone number input position 821, the operation position 822 for acquiring the verification code, and feed back corresponding position information to the test device.

The test equipment may operate in the page 82 for inputting the phone number to obtain the verification code based on the received location information. Thereafter, the test equipment may display a page 83 for entering the verification code.

The test device may transmit the image of the page 83 for inputting the verification code to the server, where the server may determine the verification code input position 831, the operation position 832 for confirming the login, and feed back the corresponding position information to the test device, so that the test device operates based on the corresponding operation position, logs in the account.

In an optional implementation manner, the server may further determine the authorized operation position in the image of the current page, and/or the image of the page for inputting the mobile phone number, and/or the image of the page for inputting the verification code, and send the authorized operation position.

The server can identify the acquired image of the current page, the image of the page for inputting the mobile phone number or the image of the page for inputting the verification code to determine whether the authorized operation position exists or not, and if so, the authorized operation position is fed back to the test equipment.

Specifically, in the identification process, the server may determine the size of the selection box for signing the authorization protocol according to the size of characters in the image of the current page, and/or in the image of the page for inputting the mobile phone number, and/or in the image of the page for inputting the verification code. For example, the text information for representing the content of the signing authorization agreement can be identified in the page, the size of the text is identified, and the size of the check box is predicted according to the size of the text.

Further, the server may determine a selection frame template according to the predicted size, and determine a position of the selection frame for signing the authorization agreement in the image of the current page, and/or the image of the page for inputting the mobile phone number, and/or the image of the page for inputting the verification code according to the selection frame template.

Fig. 9 is a schematic diagram of a page image according to an exemplary embodiment of the present disclosure.

As shown in fig. 9, the server may acquire a text 91 for characterizing agreement with the authorization agreement in the page image, and predict the size of the selection box according to the size of the text 91, thereby determining a selection box template 92 according to the determined size of the selection box, and the server may perform matching in the page image according to the selection box 92 to determine a position 93 of the selection box for agreement with the authorization agreement in the page image.

In an alternative embodiment, the server may cut out the interference portion in the image, leaving only a small area, and then use the selection frame template for matching. For example, the region where the text is located may be expanded to determine the reserved small region.

In practical application, the preset selection frame template may be square or circular, and when the circular selection frame template is used for matching in the reserved small area, a hough circle mode may be adopted.

Referring to fig. 9, as shown in fig. 9, in the scheme provided by the present disclosure, the region where the text is located may be expanded, so as to reserve a small region 94 in the page image, and further match a selection box in the small region 94.

Fig. 10 is a flowchart illustrating a method for determining an operation position for logging in an account according to a third exemplary embodiment of the present disclosure.

As shown in fig. 10, a method for determining an operation location for logging in an account provided by the present disclosure includes:

step 1001, acquiring an image of a current page of a program to be tested under the condition that the program to be tested is not logged in.

Step 1002, determining a page type corresponding to the current page according to the image of the current page.

Steps

1001 and 1002 are similar to the implementation of

steps

101 and 102, and are not described again.

Step 1003, if the page type is the content page, acquiring text information in the image of the current page, and determining the alternative operation position according to the preset keywords and the text information.

If the category of the current page is the content page, it indicates that there is no login entry in the current page, and the location of the login entry can be found only by performing an operation in the current content page.

Specifically, the content page generally includes a large number of text contents, and in the scheme provided by the disclosure, text information in the content page image can be acquired, and then the text information is filtered according to preset keywords, so that text information which is possibly represented for entering the login page is screened out, and the position of the corresponding text information is determined as the standby operation position.

The preset keywords can be set according to requirements, for example, the preset keywords can be text contents such as "me", "user", and the like.

And 1004, extracting local features of the region where each alternative operation position is located, and determining and feeding back the target operation position in the alternative operation positions according to the local features.

Further, when the text information in the content page is relatively large, the operation position entering the login page may not be accurately identified based on the content of the text itself. Therefore, local features of the region where each alternative operation position is located can be extracted, so that the target operation position is determined in the alternative operation positions according to the local features, and the target operation position is fed back to the test equipment.

In practical application, in a page, the position of the page, which can be entered after being operated, is generally different from the display mode of ordinary characters, for example, the position is highlighted, and the background color of the position which can be operated is different from that of other areas.

Therefore, the server can acquire the color information of the region where each alternative operation position is located, so as to determine whether the region where the alternative operation position is located is a highlight region according to the color information.

For example, when "my" is set as a preset keyword, the positions of the two words of "my" may be determined as alternative operation positions, and the regions of the two words of "my" may be determined as the regions of the alternative operation positions.

Specifically, if it is determined that the region in which the candidate operation position is located is a highlight region, the candidate operation position is determined as the target operation position.

In one embodiment, a server may obtain a first pixel value of each pixel point in a region where an alternative operation position is located, and determine a background color standard deviation of the region where the alternative operation position is located according to the first pixel value; and determining whether the area where the alternative operation position is located is a highlight area or not according to the standard deviation of the background color.

Specifically, the standard deviation of the background color of the region where each alternative operation position is located may be determined, if the standard deviation of the background color is 0, the region may be considered as gray, and there are no other colors, and if the standard deviation of the background color is greater than 0, it may be considered that there are other colors in the region, and the region may be a highlight region.

In an optional implementation manner, the server may obtain a first pixel value of each pixel point in the region where the alternative operation position is located, and determine a background color standard deviation of the region where the alternative operation position is located according to the first pixel value.

And acquiring a second pixel value of each pixel point in the page, and determining a page color standard deviation according to the second pixel value.

And determining whether the area where the alternative operation position is located is a highlight area or not according to the background color standard deviation and the page color standard deviation of the area where the alternative operation position is located.

If the area where the alternative operation position is located is highlighted, the difference between the background color of the area and the background color of the whole page may be large, so that the standard deviation of the background color of the area where the alternative operation position is located and the standard deviation of the page color may be compared, and if the difference between the background color of the area where the alternative operation position is located and the standard deviation of the page color is large, it may be determined that the area where the alternative operation position is located is the highlighted area.

Specifically, a threshold may be further set, and if a difference between a standard deviation of a background color of a region where the alternative operation position is located and a standard deviation of a page color is greater than the threshold, the region where the alternative operation position is located may be considered as a highlight region.

In an optional implementation manner, the text area in the alternative operation position may be further expanded to obtain the area where the alternative operation position is located. For example, the position of the text matched with the preset keyword may be expanded, so as to obtain the area where the alternative operation position is located.

The server can obtain a pixel value of each pixel point in the area where the alternative operation position is located, and determines the pixel value with the highest occurrence frequency in the area as the background color of the area where the alternative operation position is located; and determining whether the area where the alternative operation position is located is a highlight area or not according to the background color.

The pixel information of the highlight color may be preset, and the background color of the candidate operation position is compared with the preset pixel information, and if the background color of the candidate operation position is consistent with the preset pixel information, the region where the candidate operation position is located may be considered as the highlight region.

In practical application, in a page, the page can be moved down after being operated, and the position of the page is generally different from the display mode of common characters, for example, the color of the characters is different. Therefore, the server can acquire the color information of the characters at each alternative operation position and determine the target operation position in the alternative operation positions according to the color information of each character.

For example, the server may further obtain color information of all characters in the entire page, and if the color of the character at the alternative operation position is not consistent with the color of most characters in the page, the server may determine the alternative operation position as the target operation position.

In order to identify the character information with difference display in the page, the server can also obtain the character size information of each alternative operation position and obtain the average size of the characters in the page; and determining a target operation position in the alternative operation positions according to the size information of each character and the average size of the characters.

If the size of the characters displayed at the alternative operation position is larger than the average size of the characters in the page, the server can determine that the characters at the alternative operation position are highlighted, and further can determine the alternative operation position with the highlighted characters as the target operation position.

Specifically, in an optional implementation manner, if the characters at the alternative operation positions include graphics, extracting local features of a region where each alternative operation position is located, and when the target operation position is determined in the alternative operation positions according to the local features, the server may remove the character content included in the graphics, and compare the preset graphic template with the graphics to obtain a comparison result; and determining a target operation position in the alternative operation positions according to the comparison result.

Further, graphic templates, such as rectangles or rectangles with arc-shaped four corners, may be preset, and the server may obtain the graphic templates, compare the graphic templates with the graphics outside the characters at the candidate operation position, and if the comparison is consistent, the candidate operation position may be considered as the target operation position.

By the method, the server can clear the interference content at the alternative operation position, and then can more accurately compare the graphic template with the graphics outside the characters, so that a more accurate target operation position is obtained.

Fig. 11 is a schematic diagram of a page image according to an exemplary embodiment of the present disclosure.

As shown in fig. 11, the server may determine the alternative operation position in the page image according to the text content in the page, and further determine whether the alternative operation position is actually the target operation position according to the local feature at the alternative operation position.

As in fig. 11, the server may determine a target operating location 111 therein.

Step 1005, acquiring the image of the lower page.

After receiving the target operation position, the test equipment can operate in the currently displayed page according to the position, so as to enter a lower-level page of the page.

Specifically, the lower page may be a login portal page or may be provided with a key for entering the login portal page. Accordingly, the test device may acquire an image of the lower page and transmit the lower page image to the server.

Further, the server may continue to perform the step of determining the image category from the image of the lower page such as may perform step 602, or step 1002.

Optionally, on the basis of any one of the above embodiments, the scheme provided by the present disclosure may further include:

determining whether the program to be tested is a new task or not according to the identification information of the program to be tested;

if the task is determined not to be a new task, acquiring the equipment state corresponding to the identification information;

and if the equipment state is the stuck state, not processing the acquired image of the current page.

After receiving the request, the server may determine whether the request is a new task according to the identification information included in the request. If the task is a new task, the server may obtain a device state corresponding to the identification information.

Specifically, the server may record a state of each testing device when running the program to be tested, so that the server may obtain the device state according to the device identifier.

Further, if the device status acquired by the server is a stuck status, the server may not process the page image in the request. After the server sends the information of the control to be operated to the test equipment, the test equipment needs to operate based on the received information of the control to be operated, but if the test equipment is in a stuck state, the operation cannot be executed, so that the server does not need to process the request, and resources of the server are saved.

In actual application, preset information of a lower-level page corresponding to the page type is preset. The preset information refers to information which should be included in a lower page displayed by the testing equipment after the testing equipment operates in the current page according to the received position information. If the server receives the image of the lower page sent by the testing equipment and if the image does not contain the preset information, the server sends the alternative operation information to the testing equipment; the alternative operation information includes any one of the following information: and clicking skipping, clicking an icon which accords with the preset property, and performing sliding operation.

In the embodiment, a bottom-holding strategy is set to prevent the login process from being blocked and incapable of continuing.

In an alternative embodiment, if the page category is a popup page, the server may determine, in the image of the current page, an operation position for closing the popup page, and specifically may be an active option such as "allow", "agree", and the like.

If the image of the current page does not include the options, the server may determine an operation position for closing the pop-up window in the current page, and then determine the position as the operation position.

The server may transmit the operation position information to the test device according to the determined operation position.

Fig. 12 is a schematic structural diagram of an apparatus for determining an operation position for logging in an account according to an exemplary embodiment of the present disclosure.

As shown in fig. 12, the apparatus 1200 for determining an operation location for logging in an account provided by the present disclosure includes:

an obtaining unit 1210 configured to obtain an image of a current page of the program to be tested when the program to be tested is in an unregistered state;

a category determining unit 1220, configured to determine a page category corresponding to the current page according to the image of the current page;

and a control information feedback unit 1230, configured to determine a control trigger chain for implementing login operation according to the page category, and sequentially determine and feed back information of a control to be operated according to the control trigger chain, where the information of the control to be operated represents an operation control that needs to be operated when the account is logged in.

The apparatus provided by the present disclosure is similar to the embodiment shown in fig. 1, and is not described again.

Fig. 13 is a schematic structural diagram of an apparatus for determining an operation position for logging in an account according to another exemplary embodiment of the present disclosure.

As shown in fig. 13, in an apparatus 1300 for determining an operation position for logging in an account provided by the present disclosure, the page category includes any one of:

login page, content page, popup page.

In an optional implementation manner, if the page type is a login page, the control information feedback unit 1230 includes:

a login mode determining module 1231, configured to determine a login mode according to the attribute information of the program to be tested;

a login entry feedback module 1232, configured to determine and feed back, according to the login manner, a login entry position when the account is logged in the current page in the image of the current page; the login entry position is an operation position when entering a lower page when the account is logged in the login mode;

a lower page processing module 1233, configured to obtain an image of a lower page of the program to be tested, which enters based on the login entry position, and determine and feed back an operation position for a login account according to the image of the lower page, where the operation position is used to represent a position that needs to be operated when the login account is logged in;

and the information of the control to be operated comprises the login entry position and the operation position for logging in the account.

In an optional implementation manner, if the login manner is a login manner authorized by communication software, the login entry feedback module 1232 is specifically configured to:

acquiring the icon features of the communication software, and extracting the page features of the image of the current page;

and matching the icon features and the page features, and determining a login entry position for selecting authorized login of communication software in the image of the current page according to a matching result.

In an optional implementation manner, the login portal feedback module 1232 is specifically configured to:

determining a matching point matched with the icon feature in the image of the current page according to the icon feature and the page feature;

determining matching points belonging to the same range according to the distance between the matching points, and eliminating matching points with wrong matching according to the matching points in each range;

and determining a login entry position for selecting authorized login of the communication software in the image of the current page according to the remaining matching points.

In an optional embodiment, the lower page processing module 1233 is specifically configured to:

and determining a key position for agreeing to authorize to log in the program to be tested in the image of the lower page, wherein the key position is used for logging in the program to be tested by using a login mode authorized by the communication software.

In an optional implementation manner, if the login manner is a short message verification login manner, the login portal feedback module 1232 is specifically configured to:

and acquiring the character information in the image of the current page, and determining a login entry position for selecting short message authentication login according to the character information in the image of the current page.

In an optional implementation manner, the lower page is a page for inputting a mobile phone number; the lower page processing module 1233 is specifically configured to:

determining a mobile phone number input position and an operation position for acquiring a verification code in an image of a lower page for inputting the mobile phone number;

acquiring an image of a page for inputting a verification code, and determining a verification code input position and an operation position for confirming login in the image of the page for inputting the verification code;

wherein the operation position for logging in the account includes: the mobile phone number input method comprises a mobile phone number input position, an operation position used for obtaining a verification code, a verification code input position and an operation position used for confirming login.

In an optional implementation, the lower page processing module 1233 is further configured to:

and determining an authorized operation position in the image of the current page, and/or the image of the page for inputting the mobile phone number, and/or the image of the page for inputting the verification code, and feeding back the authorized operation position.

determining the size of a selection frame for signing an authorization protocol according to the size of characters in the image of the current page, and/or the image of the page for inputting the mobile phone number, and/or the image of the page for inputting the verification code;

and determining a selection frame template according to the size, and determining the position of a selection frame for signing an authorization protocol in the image of the current page, the image of the page for inputting the mobile phone number and/or the image of the page for inputting the verification code according to the selection frame template.

In an optional implementation manner, if the page category is a content page, the control information feedback unit 1230 includes:

a text obtaining module 1234, configured to obtain text information in the image of the current page, and determine an alternative operation position according to a preset keyword and the text information;

the control information determining module 1235 is configured to extract a local feature of a region where each candidate operation position is located, and determine and feed back a target operation position in the candidate operation position according to the local feature;

an obtaining module 1236, configured to obtain an image of a lower page, and continue to perform, according to the image of the lower page, a step of determining a page category corresponding to the current page according to the image of the current page.

In an optional implementation manner, the control information determining module 1235 is specifically configured to:

acquiring color information of a region where the alternative operation position is located, and determining whether the region where the alternative operation position is located is a highlight region according to the color information;

the alternative operation position in the highlighted area is determined as the target operation position.

In an optional embodiment, the control information determining module 1235 is specifically configured to:

acquiring a first pixel value of each pixel point in the area where the alternative operation position is located, and determining the background color standard deviation of the area where the alternative operation position is located according to the first pixel value;

determining whether the area where the alternative operation position is located is a highlight area or not according to the standard deviation of the background color;

alternatively, the first and second electrodes may be,

acquiring a second pixel value of each pixel point in the page, and determining a page color standard deviation according to the second pixel value;

determining whether the area where the alternative operation position is located is a highlight area or not according to the standard deviation of the background color of the area where the alternative operation position is located and the standard deviation of the page color;

alternatively, the first and second liquid crystal display panels may be,

expanding the character area in the alternative operation position to obtain the area where the alternative operation position is located;

acquiring a pixel value of each pixel point in a region where the alternative operation position is located, and determining the pixel value with the highest occurrence frequency in the region as the background color of the region where the alternative operation position is located;

and determining whether the area where the alternative operation position is located is a highlight area according to the background color.

acquiring color information of characters at each alternative operation position;

and determining a target operation position in the alternative operation positions according to the color information of each character.

acquiring character size information of each alternative operation position, and acquiring average size of characters in the page;

and determining a target operation position in the alternative operation positions according to the size information of each character and the average size of the characters.

In an optional implementation manner, if the text at the alternative operation position includes a graphic, the control information determination module 1235 is specifically configured to:

removing the text content included in the graph, and comparing the graph with a preset graph template to obtain a comparison result;

and determining a target operation position in the alternative operation positions according to the comparison result.

In an optional embodiment, the apparatus further includes a determining unit 1240, configured to:

and if the equipment state is a stuck state, not processing the acquired image of the current page.

In an optional implementation manner, preset information of a lower page corresponding to a page category is preset;

the lower page processing module 1233 is further configured to:

if the received image of the lower page does not comprise preset information, sending alternative operation information;

the alternative operation information includes any one of the following information:

clicking to skip, clicking the icon which accords with the preset property, and performing sliding operation.

The apparatus provided in this embodiment may be used to implement the technical solutions of the above method embodiments, and the implementation principles and technical effects are similar, which are not described herein again.

Referring to fig. 14, a schematic structural diagram of an electronic device 1400 suitable for implementing an embodiment of the present disclosure is shown, where the electronic device 1400 may be a terminal device or a server. Among them, the terminal Device may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a Digital broadcast receiver, a Personal Digital Assistant (PDA), a tablet computer (PAD), a Portable Multimedia Player (PMP), a car terminal (e.g., car navigation terminal), etc., and a fixed terminal such as a Digital TV, a desktop computer, etc. The electronic device shown in fig. 14 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 14, the electronic device 1400 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 1401, which can perform various suitable actions and processes according to a program stored in a Read Only Memory (ROM) 1402 or a program loaded from a storage device 1408 into a Random Access Memory (RAM) 1403. In the RAM 1403, various programs and data necessary for the operation of the electronic device 1400 are also stored. The processing device 1401, the ROM1402, and the RAM 1403 are connected to each other by a bus 1404. An input/output (I/O) interface 1405 is also connected to bus 1404.

Generally, the following devices may be connected to the I/O interface 1405: input devices 1406 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 1407 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage devices 1408 including, for example, magnetic tape, hard disk, etc.; and a communication device 1409. The communication means 1409 may allow the electronic device 1400 to communicate wirelessly or by wire with other devices to exchange data. While fig. 14 illustrates an electronic device 1400 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, the processes described above with reference to the flow diagrams may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication device 1409, or installed from the storage device 1408, or installed from the ROM 1402. The computer program, when executed by the processing apparatus 1401, performs the functions defined in the methods of the embodiments of the present disclosure.

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of Network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first retrieving unit may also be described as a "unit for retrieving at least two internet protocol addresses".

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems on a chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In a first aspect, according to one or more embodiments of the present disclosure, there is provided a method for determining an operation location for logging in an account, including:

According to one or more embodiments of the present disclosure, the page category includes any one of:

login page, content page, popup page.

According to one or more embodiments of the present disclosure, if the page type is a login page, the sequentially determining and feeding back information of the control to be operated according to the control trigger chain includes:

determining a login mode according to the attribute information of the program to be tested;

determining and feeding back a login entry position when the account is logged in through the login mode in the image of the current page according to the login mode; the login entry position is an operation position when entering a lower page when the account is logged in the login mode;

acquiring an image of a lower page of the program to be tested, which enters based on the login entry position, and determining and feeding back an operation position for a login account according to the image of the lower page, wherein the operation position is used for representing a position needing to be operated when the login account is logged in;

According to one or more embodiments of the present disclosure, if the login manner is a login manner authorized by communication software, determining, according to the login manner, a login entry position when an account is logged in the current page image by the login manner, includes:

and matching the icon features and the page features, and determining a login entry position for selecting authorized login of the communication software in the image of the current page according to a matching result.

According to one or more embodiments of the present disclosure, the matching the icon features and the page features, and determining a login entry position for selecting authorized login of communication software in the image of the current page according to a matching result includes:

and determining the login entry position for selecting the authorized login of the communication software in the image of the current page according to the remaining matching points.

According to one or more embodiments of the present disclosure, determining an operation position for a login account from an image of the lower page includes:

According to one or more embodiments of the present disclosure, if the login manner is a login manner verified by a short message, determining a login entry position when an account is logged in the current page in the image of the current page according to the login manner, includes:

and acquiring the character information in the image of the current page, and determining a login entry position for selecting short message verification login according to the character information in the image of the current page.

According to one or more embodiments of the present disclosure, the lower page is a page for inputting a mobile phone number; determining an operation position for logging in an account according to the image of the lower page, and sending the operation position, wherein the method comprises the following steps:

determining a mobile phone number input position and an operation position for acquiring an identifying code in an image of a lower page for inputting the mobile phone number;

wherein the operation position for logging in the account includes: the mobile phone number input method comprises a mobile phone number input position, an operation position used for obtaining the verification code, a verification code input position and an operation position used for confirming login.

According to one or more embodiments of the present disclosure, further comprising:

and determining an authorized operation position in the image of the current page, and/or the image of the page for inputting the mobile phone number, and/or the image of the page for inputting the verification code, and sending the authorized operation position.

According to one or more embodiments of the present disclosure, the determining an authorized operation position in the image of the current page, and/or the image of the page for inputting the mobile phone number, and/or the image of the page for inputting the verification code includes:

According to one or more embodiments of the present disclosure, if the page type is a content page, sequentially determining and feeding back information of a control to be operated according to the control trigger chain, including:

acquiring character information in the image of the current page, and determining an alternative operation position according to a preset keyword and the character information;

extracting local features of a region where each alternative operation position is located, and determining and feeding back a target operation position in the alternative operation positions according to the local features;

and acquiring an image of a lower page, and continuing to execute the step of determining the page type corresponding to the current page according to the image of the lower page.

According to one or more embodiments of the present disclosure, the extracting a local feature of a region in which each candidate operation position is located, and determining a target operation position in the candidate operation positions according to the local feature includes:

and determining the alternative operation position in the highlight area as the target operation position.

According to one or more embodiments of the present disclosure, the obtaining color information of a region where the alternative operation position is located, and determining whether the region where the alternative operation position is located is a highlight region according to the color information includes:

alternatively, the first and second liquid crystal display panels may be,

and determining whether the region where the alternative operation position is located is a highlight region according to the background color.

According to one or more embodiments of the present disclosure, if the text at the alternative operation position includes a graphic, the extracting a local feature of an area where each alternative operation position is located, and determining a target operation position in the alternative operation position according to the local feature includes:

determining whether the request for testing the program to be tested is a new task according to the identification information of the program to be tested;

According to one or more embodiments of the present disclosure, preset information of a lower page corresponding to a page category is preset;

if the received image of the lower page does not include preset information, sending alternative operation information;

and clicking skipping, clicking an icon which accords with the preset property, and performing sliding operation.

In a second aspect, according to one or more embodiments of the present disclosure, there is provided an apparatus for determining an operation location for logging in an account, including:

and the control information feedback unit is used for determining a control trigger chain for realizing login operation according to the page type, and sequentially determining and feeding back information of a control to be operated according to the control trigger chain, wherein the information of the control to be operated represents an operation control which needs to be operated when the account is logged in. According to one or more embodiments of the present disclosure, the page category includes any one of:

login page, content page, popup page.

In an optional implementation manner, if the page type is a login page, the control information feedback unit includes:

the login mode determining module is used for determining a login mode according to the attribute information of the program to be tested;

the login entry feedback module is used for determining and feeding back the position of a login entry when the account is logged in the login mode in the image of the current page according to the login mode; the login entry position is an operation position when entering a lower page when the account is logged in the login mode;

the lower page processing module is used for acquiring an image of a lower page of the program to be tested, which enters based on the login entry position, and determining and feeding back an operation position for a login account according to the image of the lower page, wherein the operation position is used for representing a position needing to be operated when the login account is logged in;

In an optional implementation manner, if the login manner is a login manner authorized by communication software, the login entry feedback module is specifically configured to:

In an optional implementation manner, the lower page processing module is specifically configured to:

In an optional implementation manner, if the login manner is a short message verification login manner, the login entry feedback module is specifically configured to:

In an optional embodiment, the lower page is a page for inputting a mobile phone number; the lower page processing module 1233 is specifically configured to:

wherein the operation position for logging in the account comprises: the mobile phone number input method comprises a mobile phone number input position, an operation position used for obtaining a verification code, a verification code input position and an operation position used for confirming login.

In an optional embodiment, the lower page processing module is further configured to:

In an optional implementation manner, if the page category is a content page, the control information feedback unit includes:

the character acquisition module is used for acquiring character information in the image of the current page and determining alternative operation positions according to preset keywords and the character information;

the control information determining module is used for extracting local characteristics of an area where each alternative operation position is located, and determining and feeding back a target operation position in the alternative operation positions according to the local characteristics;

and the acquisition module is used for acquiring the image of the lower page and continuously executing the step of determining the page category corresponding to the current page according to the image of the lower page.

In an optional implementation manner, the control information determination module is specifically configured to:

alternatively, the first and second liquid crystal display panels may be,

In an optional implementation manner, if the text at the alternative operation position includes a graphic, the control information determination module is specifically configured to:

In an optional implementation manner, the apparatus further includes a determining unit, configured to:

the lower level page processing module is further to:

In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device including: at least one processor and memory;

the memory stores computer-executable instructions;

In a fourth aspect, according to one or more embodiments of the present disclosure, there is provided a computer-readable storage medium having stored therein computer-executable instructions that, when executed by a processor, implement the method for determining an operating position for a login account as set forth in the first aspect and various possible designs of the first aspect.

In a fifth aspect, embodiments of the present disclosure provide a computer program product comprising a computer program which, when executed by a processor, implements a method of determining an operating position for a login account as described above in the first aspect and in various possible designs of the first aspect.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other combinations of features described above or equivalents thereof without departing from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method for determining an operation position for logging in an account is characterized by comprising the following steps:

under the condition that a program to be tested is not logged in, acquiring an image of a current page of the program to be tested;

2. The method according to claim 1, wherein if the page type is a login page, the sequentially determining and feeding back information of the controls to be operated according to the control trigger chain comprises:

determining and feeding back a login entry position when an account is logged in through the login mode in the image of the current page according to the login mode; the login entry position is an operation position when entering a lower page when the account is logged in the login mode;

3. The method of claim 2, wherein if the login mode is a login mode authorized by communication software, determining a login entry position when an account is logged in by the login mode in the image of the current page according to the login mode comprises:

4. The method of claim 3, wherein the matching the icon features and the page features and determining a login entry position for selecting authorized login of the communication software in the image of the current page according to the matching result comprises:

5. The method according to claim 3, wherein if the login mode is a short message authentication login mode, the lower page is a page for inputting a mobile phone number; determining an operation position for a login account according to the image of the lower page, comprising:

6. The method of claim 5, further comprising:

7. The method according to claim 2, wherein if the page type is a content page, sequentially determining and feeding back information of the control to be operated according to the control trigger chain, includes:

and acquiring an image of a lower page, and continuously executing the step of determining the page type corresponding to the current page according to the image of the lower page.

8. The method according to claim 7, wherein the extracting a local feature of a region in which each candidate operation position is located, and determining a target operation position among the candidate operation positions according to the local feature comprises:

9. The method according to claim 8, wherein the obtaining color information of the region where the alternative operation position is located, and determining whether the region where the alternative operation position is located is a highlight region according to the color information comprises:

alternatively, the first and second liquid crystal display panels may be,

alternatively, the first and second electrodes may be,

expanding the text area in the alternative operation position to obtain the area where the alternative operation position is located;

acquiring a pixel value of each pixel point in a region where the alternative operation position is located, and determining the pixel value with the highest occurrence frequency in the region as a background color of the region where the alternative operation position is located;

10. The method according to claim 7, wherein the extracting local features of the region where each candidate operation position is located, and determining the target operation position in the candidate operation positions according to the local features comprises:

acquiring color information of characters at each alternative operation position; determining a target operation position in the alternative operation positions according to the color information of each character;

alternatively, the first and second liquid crystal display panels may be,

11. The method according to claim 7, wherein if the text at the alternative operation positions includes graphics, the extracting local features of the area where each alternative operation position is located, and determining the target operation position in the alternative operation positions according to the local features comprises:

12. The method according to any one of claims 1-11, further comprising:

determining whether the test of the program to be tested is a new task according to the identification information of the program to be tested;

13. An apparatus for determining an operating location for logging into an account, comprising:

the device comprises an acquisition unit, a display unit and a control unit, wherein the acquisition unit is used for acquiring an image of a current page of a program to be tested under the condition that the program to be tested is not logged in;

and the control information feedback unit is used for determining a control trigger chain for realizing login operation according to the page type, and sequentially determining and feeding back information of a control to be operated according to the control trigger chain, wherein the information of the control to be operated represents an operation control which needs to be operated when the control to be operated is used for logging in an account.

14. An electronic device, comprising: at least one processor and memory;

the memory stores computer-executable instructions;

the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the method of determining an operating position for a login account according to any one of claims 1 to 11.

15. A computer-readable storage medium having stored thereon computer-executable instructions which, when executed by a processor, implement a method of determining an operating position for a login account as claimed in any one of claims 1 to 11.

16. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-11.