CN116431468A - Test method, test device, test processing equipment and storage medium - Google Patents
Test method, test device, test processing equipment and storage medium Download PDFInfo
- Publication number
- CN116431468A CN116431468A CN202310135419.7A CN202310135419A CN116431468A CN 116431468 A CN116431468 A CN 116431468A CN 202310135419 A CN202310135419 A CN 202310135419A CN 116431468 A CN116431468 A CN 116431468A
- Authority
- CN
- China
- Prior art keywords
- characters
- interface
- character
- test
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 246
- 238000012545 processing Methods 0.000 title claims abstract description 35
- 238000010998 test method Methods 0.000 title claims description 12
- 238000013515 script Methods 0.000 claims abstract description 43
- 238000000034 method Methods 0.000 claims description 56
- 230000008569 process Effects 0.000 claims description 18
- 238000002372 labelling Methods 0.000 claims description 8
- 238000012549 training Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012015 optical character recognition Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000008676 import Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/19—Recognition using electronic means
- G06V30/19007—Matching; Proximity measures
- G06V30/19013—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Character Discrimination (AREA)
Abstract
The invention provides a testing method, a testing device, processing equipment and a storage medium, and relates to the technical field of automatic testing. Comprising the following steps: acquiring a test script containing a first character and a screenshot of a first interface of an application to be tested; character recognition is carried out on the screenshot of the first interface, so that a plurality of groups of characters and position information of each group of characters in the first interface are obtained; determining a target group character matched with the first character from the plurality of groups of characters; and performing test operation on the first interface according to the test script and the position information corresponding to the target group characters. And carrying out character recognition on the screenshot of the first interface, determining a target group character matched with the first character from the plurality of groups of recognized characters, carrying out test operation on the first interface according to the position information corresponding to the target group character, accurately determining the operation position of the first interface by the screenshot character recognition of the first interface and the way of matching the plurality of groups of recognized characters with the first character, and further improving the test accuracy.
Description
Technical Field
The present invention relates to the field of automated testing technologies, and in particular, to a testing method, a testing device, a processing apparatus, and a storage medium.
Background
Automated testing has been a difficult task, especially for gaming scenarios where interfaces are complex and make automated testing difficult to implement successfully. In recent years, some testing tools make automated testing simpler and more feasible, and automated testing with testing tools is also popular.
In the related art, a screen shot is obtained, matching is carried out according to a target image in a test script and the screen shot, an operation position in the screen shot is found, and corresponding test operation is carried out according to the operation position by adopting the test script.
However, in the related art, matching is performed on the target image and the screenshot, so that the problem that the determined operation position is inaccurate easily occurs, and the accuracy of the test is reduced.
Disclosure of Invention
The present invention is directed to a testing method, a testing device, a testing apparatus, a testing device, and a storage medium, which address the above-mentioned problems of the related art.
In order to achieve the above purpose, the technical scheme adopted by the embodiment of the invention is as follows:
in a first aspect, an embodiment of the present invention provides an application testing method, including:
Acquiring a test script containing a first character and a screenshot of a first interface of an application to be tested;
character recognition is carried out on the screenshot of the first interface, so that a plurality of groups of characters and the position information of each group of characters in the first interface are obtained;
determining a target set of characters from the plurality of sets of characters that match the first character;
and performing test operation on the first interface according to the test script and the position information corresponding to the target group character.
In a second aspect, an embodiment of the present invention further provides an application testing apparatus, including:
the acquisition module is used for acquiring a test script containing a first character and a screenshot of a first interface of an application to be tested;
the recognition module is used for carrying out character recognition on the screenshot of the first interface to obtain a plurality of groups of characters and the position information of each group of characters in the first interface;
a determining module, configured to determine a target set of characters that matches the first character from the multiple sets of characters;
and the operation module is used for carrying out test operation on the first interface according to the test script and the position information corresponding to the target group character.
In a third aspect, an embodiment of the present invention further provides a processing apparatus, including: a memory storing a computer program executable by the processor, and a processor implementing the test method of any one of the first aspects above when the processor executes the computer program.
In a fourth aspect, embodiments of the present invention further provide a computer readable storage medium having a computer program stored thereon, which when read and executed, implements the test method according to any one of the first aspects.
The beneficial effects of the invention are as follows: the embodiment of the application provides a testing method, which comprises the following steps: acquiring a test script containing a first character and a screenshot of a first interface of an application to be tested; character recognition is carried out on the screenshot of the first interface, so that a plurality of groups of characters and position information of each group of characters in the first interface are obtained; determining a target group character matched with the first character from the plurality of groups of characters; and performing test operation on the first interface according to the test script and the position information corresponding to the target group characters. The screenshot of the first interface is subjected to character recognition, a target group character matched with the first character is determined from the recognized multiple groups of characters, the first interface is subjected to test operation according to the position information corresponding to the recognized target group character, and the operation position of the first interface can be accurately determined by the screenshot character recognition of the first interface and the manner of the recognized multiple groups of characters and the first character matching, so that the test accuracy can be improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a testing method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a testing method according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of a testing method according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of a testing method according to an embodiment of the present invention;
FIG. 5 is a schematic flow chart of a testing method according to an embodiment of the present invention;
FIG. 6 is a schematic flow chart of a testing method according to an embodiment of the present invention;
FIG. 7 is a schematic flow chart of a testing method according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a testing device according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a processing apparatus according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention.
Thus, the following detailed description of the embodiments of the present application, as provided in the accompanying drawings, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
In the description of the present application, it should be noted that, if the terms "upper", "lower", and the like indicate an azimuth or a positional relationship based on the azimuth or the positional relationship shown in the drawings, or an azimuth or the positional relationship that is commonly put when the product of the application is used, it is merely for convenience of description and simplification of the description, and does not indicate or imply that the apparatus or element to be referred to must have a specific azimuth, be configured and operated in a specific azimuth, and therefore should not be construed as limiting the present application.
Furthermore, the terms first, second and the like in the description and in the claims and in the above-described figures, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that, without conflict, features in embodiments of the present application may be combined with each other.
Aiming at the problems that in the related art, the target image and the screenshot are matched, the determined operation position is inaccurate easily, and the accuracy of the test is reduced, the embodiment of the application test method is provided, character recognition is carried out on the screenshot of the first interface, the target group character matched with the first character is determined from the plurality of groups of recognized characters, the test operation is carried out on the first interface according to the position information corresponding to the recognized target group character, the operation position of the first interface can be accurately determined by the screenshot character recognition of the first interface and the manner that the plurality of groups of recognized characters are matched with the first character, and then the accuracy of the test can be improved.
The embodiment of the application provides a test method, which is applied to processing equipment, wherein the processing equipment can be a test terminal or a test server, and the test terminal can be any one of the following: notebook computers, desktop computers, tablet computers, smart phones, and the like.
A test method provided in the embodiments of the present application is explained below.
Fig. 1 is a flow chart of a test method according to an embodiment of the present invention, as shown in fig. 1, the method may include:
s101, acquiring a test script containing a first character and a screenshot of a first interface of an application to be tested.
The screenshot of the first interface of the application to be tested may be a screenshot of the entire first interface, the application to be tested may be a game application, a financial application, or another type of application, which is not specifically limited in this embodiment of the present application.
In addition, the test script may include a plurality of preset sentences, where the plurality of preset sentences includes a target sentence, and the target sentence includes a first character. Optionally, the first character is used for representing a character corresponding to the control to be tested in the first interface.
S102, carrying out character recognition on the screenshot of the first interface to obtain a plurality of groups of characters and position information of each group of characters in the first interface.
The position information of each group of characters in the first interface may be coordinate information of each group of characters in the first interface.
In some embodiments, the screenshot of the first interface is subjected to character recognition by adopting a preset character recognition model, a preset character recognition algorithm or a preset character recognition rule, so that a plurality of groups of characters and position information of each group of characters in the first interface are obtained.
It should be noted that, the plurality of groups of characters may include characters corresponding to the controls in the first interface, where one control may correspond to a group of characters, or may include characters corresponding to the description information in the first interface, which is not specifically limited in the embodiment of the present application, and in practical application, a user may operate on the controls and may not operate on the description information.
S103, determining target group characters matched with the first character from the plurality of groups of characters.
Wherein the target group character may be the same character as the first character.
In the embodiment of the present application, each set of characters in the plurality of sets of characters may be matched with the first character at the same time, each set of characters in the plurality of sets of characters may be matched with the first character in sequence, and each set of characters in the plurality of sets of characters may be matched with the first character in other sequences.
S104, testing the first interface according to the test script and the position information corresponding to the target group characters.
The position information corresponding to the target group character may be coordinate information of the target group character in the first interface.
In some embodiments, according to an operation instruction corresponding to a first character in a target sentence in the test script, test operation is performed on position information corresponding to a target group character in the first interface, so that accurate automatic test for the first interface can be realized.
The test operation is any one of the following: single click operation, double click operation, sliding operation, etc.
In summary, the embodiment of the present application provides a testing method, including: acquiring a test script containing a first character and a screenshot of a first interface of an application to be tested; character recognition is carried out on the screenshot of the first interface, so that a plurality of groups of characters and position information of each group of characters in the first interface are obtained; determining a target group character matched with the first character from the plurality of groups of characters; and performing test operation on the first interface according to the test script and the position information corresponding to the target group characters. The screenshot of the first interface is subjected to character recognition, a target group character matched with the first character is determined from the recognized multiple groups of characters, the first interface is subjected to test operation according to the position information corresponding to the recognized target group character, and the operation position of the first interface can be accurately determined by the screenshot character recognition of the first interface and the manner of the recognized multiple groups of characters and the first character matching, so that the test accuracy can be improved.
In the embodiment of the present application, the foregoing processes of S101 to S104 may be performed using a preset test tool.
Optionally, fig. 2 is a schematic flow chart of a testing method according to an embodiment of the present invention, as shown in fig. 2, a process of performing character recognition on a screenshot of a first interface in the above step S102 to obtain a plurality of groups of characters and position information of each group of characters in the first interface may include:
s201, determining a target sentence in the test script according to the first character, the preset prefix name and the operation instruction.
S202, executing a target sentence, and processing the screenshot of the first interface to obtain a plurality of groups of characters and the position information of each group of characters in the first interface.
Wherein the preset prefix name may be related to an algorithm employed for character recognition of the screenshot of the first interface. For example, an OCR (Optical Character Recognition ) algorithm may be used to recognize the screenshot of the first interface, and the corresponding preset prefix name may be OCR.
In some implementations, the target statement can include, in order: preset prefix name + underline + operation instruction + first character. Alternatively, the operation instructions may include any one of the following: touch, double click, swipe, find all.
Wherein, the operation instruction corresponding interface may include: ocr _touch represents a click character; ocr _double_click represents a double click character; ocr _wipe indicates that two sets of characters are accepted, sliding from the position where the first character is located to the position where the second character is located; ocr _exists indicates whether or not a character exists; ocr _find_all indicates that there are sometimes multiple target characters in the screenshot, and this interface returns all target character positions.
By way of example, the target statement may be: ocr _touch ("immediate application"), wherein ocr is a preset prefix name, touch is an operation instruction, and "immediate application" is a target group character.
Optionally, fig. 3 is a flow chart of a test method provided by the embodiment of the present invention, as shown in fig. 3, a process of determining a target sentence in a test script according to a first character, a preset prefix name and an operation instruction in S201 may include:
s301, replacing an initial picture in the initial sentence with a first character to obtain a replaced initial sentence, wherein the replaced initial sentence further comprises an operation instruction.
S302, adding a preset prefix name to the preset position of the replaced initial sentence to obtain a target sentence.
It should be noted that, in the related art, the matching is performed on the target image and the screenshot, and the corresponding initial sentence may be an operation instruction+an initial picture, which may be, for example, touch (initial picture), and it should be understood that, here, the initial picture refers to a specific picture.
In some embodiments, the initial picture in the initial sentence may be replaced with the first character, and the replaced initial sentence may include: and (3) operating the instruction and the first character, and adding a preset prefix name in front of the replaced initial sentence to obtain a target sentence. For example, ocr _touch ("immediate application"), where ocr is a preset prefix name, touch is an operation instruction, and "immediate application" is the first character.
In the embodiment of the application, the interface designs are consistent with the test framework of the preset test tool, and the initial statement of the image processing in the preset test tool can be changed into the target statement without almost any cost for the test script of the preset test tool.
It should be noted that, the internal execution logic of each interface is consistent with the preset test tool, and the initial sentence is replaced by the target sentence to improve the execution success rate of the text scene test script. The benefit of doing so is mainly to facilitate maintenance of the plug-in, and the plug-in does not need to be updated separately when the framework of the preset test tool is updated and iterated. Alternatively, the predetermined test tool may be airtest.
In practical application, the plug-in unit relies on a frame of a preset test tool to obtain a screenshot of the first interface. Therefore, the plug-in can be directly used in multiple platforms such as ios (an operating system), android (an operating system), windows (an operating system) windows and the like. After the screenshot of the first interface and the first character are obtained, the coordinates of the target group character are obtained through calculation through a character recognition technology, and then corresponding operation is executed through a frame of a preset test tool.
Optionally, fig. 4 is a flowchart of a testing method according to an embodiment of the present invention, as shown in fig. 4, a process of determining, in S103, a target group character matched with a first character from multiple groups of characters may include:
s401, matching each group of characters with the first character respectively to obtain the matching degree corresponding to each group of characters.
The character group recognized by the character is not all correct, and phenomena such as character adhesion can also occur, so in order to improve the execution success rate of the test script, that is, the position of the character of the target group can still be found in the screenshot of the first interface under the condition of misidentification, the one-way fuzzy matching links of the characters are added in the modes of the S401 and the S402.
S402, determining target group characters matched with the first character from a plurality of groups of characters according to the matching degree corresponding to each group of characters.
In some embodiments, character level detection is performed, according to the matching degree corresponding to each group of characters, a group of characters with matching degree larger than a preset matching degree threshold value is determined as a target group of characters from multiple groups of characters, and if multiple groups of characters exist larger than the preset matching degree threshold value, a group of characters with highest matching degree is selected as the target group of characters.
Optionally, fig. 5 is a schematic flow chart of a testing method according to an embodiment of the present invention, as shown in fig. 5, where the method may further include:
s501, acquiring attribute information of a plurality of controls in a second interface of the application to be tested by using a target test tool.
The target test tool and the preset test tool may be two different test tools. Alternatively, the target test tool may be a poco (a test tool).
In some embodiments, a target testing tool is adopted to obtain the number of a plurality of controls in a second interface of the application to be tested, and text attribute information, namely text attribute information, of each control is obtained. The test process of the target test tool does not rely on the screenshot of the second interface.
S502, executing test operation on the target control in the second interface by adopting the target test tool according to the attribute information of the plurality of controls and the attribute information of the target control.
It should be noted that, a target testing tool may be used to compare attribute information of a plurality of controls and attribute information of a target control, determine attribute information of a control from attribute information of a plurality of controls, where a control corresponding to the attribute information of the control is a target control, and execute a testing operation on the target control in the second interface.
By way of example, the execution statements in the target test tool may include: poco ("go"). Click (). Wherein go is attribute information of the target control, and click indicates that clicking operation is executed for the target control.
Optionally, fig. 6 is a schematic flow chart of a testing method according to an embodiment of the present invention, as shown in fig. 6, the method may further include:
s601, if the target test tool fails to test the second interface, acquiring a second character and a screenshot of the second interface.
Wherein the attribute information of the target control includes a second character. And acquiring a screenshot of the second character and the second interface by adopting a preset testing tool.
S602, processing the screenshot of the second interface to obtain a plurality of character sets and position information of each character set in the second interface.
S603, determining a target character set matched with the second character from the plurality of character sets.
Note that the processes of S602 and S603 are similar to the processes of S102 and S103, and are not repeated here.
S604, testing the target control in the second interface by using a target testing tool according to the position information of the target character set.
In practical application, in a test script of a preset test tool, a target test tool is often used to directly acquire interface control information and operate, and the invasive tool expands some functions, and the target test tool sometimes faces a scene where control text cannot be acquired.
In the embodiment of the present application, the above-mentioned processes S601 to S604 are adopted, and the method of performing supplementary recognition on the script of the target test tool directly without modifying the interface code is not required. When the attribute information of the control in the second interface is not acquired by the target test tool, for example, some custom controls cannot be identified due to the fact that the attribute information of the control is not available. At this time, a row of execution statement poco_patch () call can be added at the beginning of the script of the target test tool, which has the function of attempting to use the character recognition interface (ocr interface) of the framework to recognize when the target test tool fails to recognize, and if the corresponding control can be found by adopting the character recognition technology, continuing to run the script of the target test tool.
Alternatively, the implementation code of the above procedure may be as follows:
from airtest.core.api import*
from airtest_ocr.api import poco_patch from poco.drivers.android.uiautomation import Androiduiautomationpoco auto_setup(__file_)
this line of code will OCR enhance all the poco (text= "xx") statements below, e.g. the control of text= "xx" cannot be found, will automatically try again to identify poco_patch with 0 CR(s)
connect device("android:///")
poco＝AndroidUiautomationPoco(use_airtest_input＝True，screenshot_each_action＝False)
# an ordinary poco statement, click text attribute is control of automobile
Poco (text= "car"). Click ()
Optionally, the process of performing character recognition on the first interface screenshot in S102 to obtain multiple sets of characters and position information of each set of characters in the first interface may include:
And carrying out character recognition on the first interface screenshot by the test terminal through a preset character recognition model to obtain a plurality of groups of characters and position information of each group of characters in the first interface.
Optionally, fig. 7 is a schematic flow chart of a test method according to an embodiment of the present invention, as shown in fig. 7, a process of performing character recognition on the first interface screenshot in S102 to obtain a plurality of groups of characters and position information of each group of characters in the first interface may include:
and S701, if the test terminal starts the character recognition service, the test terminal sends a first interface screenshot to the test server so that the test server adopts a preset character recognition model to perform character recognition on the first screen screenshot to obtain a plurality of groups of characters and the position information of each group of characters in the first interface.
The user can manually control whether to call the character recognition service, and when the user selects to enable the character recognition service, the server side is used for carrying out character recognition on the first screenshot to obtain a plurality of groups of characters and position information of each group of characters in the first interface.
S702, receiving the position information of a plurality of groups of characters in the first interface, returned by the test server.
In the embodiment of the application, the test server interface is preferentially called to perform character recognition, and in most cases, only necessary links such as matching, screen capturing, test operation execution and the like are needed to be executed locally. Therefore, the framework supports the execution of test scripts in a local large scale without bringing too much computational pressure to the test terminal equipment. Only when the pressure of the test server is too high, the test terminal equipment is switched to, and the CPU (central processing unit ) equipment of the test terminal equipment is used for complex calculation to perform character recognition.
It is worth to say that, at the test server, a request queue is maintained, whether the current request is overtime is judged before the request is executed, if the current request is overtime, the execution is abandoned, the request priority is determined according to the residual time and waiting time of the request, and the execution is performed from high to low. Wherein the request includes an interface screenshot to be character recognized.
Optionally, the method further comprises:
if the test server does not return the plurality of groups of characters and the position information of each group of characters in the first interface within the preset time length, the test terminal adopts a preset character recognition model to process the screenshot of the first interface, and the position information of the plurality of groups of characters and each group of characters in the first interface is obtained.
When the user manually controls whether to call the text service, the user can select a preset duration, namely a timeout time. After the character recognition service is started, the test server is used for character recognition processing each time preferentially, and the local computing power is used for character recognition calculation only when the time is overtime or the test server returns abnormal information.
Optionally, the method may further include:
training the initial character recognition model according to the plurality of sample images and the labeling information of the sample images to obtain a preset character recognition model;
the labeling information is used for representing sample characters in the sample image and positions of the sample characters in the sample image. The position of the sample character in the sample image may be coordinates of the sample character in the sample image.
In some embodiments, the initial character recognition model may be trained according to the plurality of sample images in the training set and the labeling information of the sample images, and the trained initial character recognition model is verified according to the plurality of sample images in the test set and the labeling information of the sample images, and if verification is successful, the preset character recognition model is obtained.
In the algorithm level, a character recognition technology (comprising two parts of character position detection and character recognition) can be introduced, so that in order to measure the performance of a large number of open-source character recognition related models under the test scene under the application program, a test set is constructed, and finally, according to the test result, two functions of character position detection and character recognition are realized by using dbnet (a detection algorithm) and crnn (a recognition algorithm).
In summary, the embodiment of the invention provides a testing method, which performs character recognition on a screenshot of a first interface, determines a target group character matched with the first character from a plurality of groups of recognized characters, performs testing operation on the first interface according to position information corresponding to the recognized target group character, and can accurately determine an operation position of the first interface by means of the screenshot character recognition of the first interface and the matching of the plurality of groups of recognized characters and the first character, thereby improving testing accuracy.
And if the test of the target test tool for the second interface fails, character recognition can be called so as to adopt the target test tool to carry out continuous test, thereby improving the reliability of the test by adopting the target test tool. The calculation force of the test server is preferentially used for character recognition, so that the processing pressure of the test terminal in the test process is reduced.
The following describes a testing device, a processing device, a storage medium, etc. for executing the testing method provided in the present application, and specific implementation processes and technical effects of the testing device and the processing device are referred to in the relevant content of the testing method, which are not described in detail below.
Fig. 8 is a schematic structural diagram of a testing device according to an embodiment of the present invention, as shown in fig. 8, the testing device may include:
An obtaining module 801, configured to obtain a test script including a first character and a screenshot of a first interface of an application to be tested;
the recognition module 802 is configured to perform character recognition on the screenshot of the first interface to obtain a plurality of groups of characters and position information of each group of characters in the first interface;
a determining module 803, configured to determine a target set of characters matching the first character from the multiple sets of characters;
and an operation module 804, configured to perform a test operation on the first interface according to the test script and the location information corresponding to the target group character.
Optionally, the identifying module 802 is specifically configured to determine a target sentence in the test script according to the first character, a preset prefix name and an operation instruction; and executing the target sentence, and processing the screenshot of the first interface to obtain the plurality of groups of characters and the position information of each group of characters in the first interface.
Optionally, the identifying module 802 is specifically configured to replace an initial picture in an initial sentence with the first character to obtain a replaced initial sentence, where the replaced initial sentence further includes the operation instruction; and adding the preset prefix name to the preset position of the replaced initial sentence to obtain the target sentence.
Optionally, the determining module 803 is specifically configured to match each set of characters with the first character, so as to obtain a matching degree corresponding to each set of characters; and determining target group characters matched with the first character from the plurality of groups of characters according to the matching degree corresponding to each group of characters.
Optionally, the apparatus further includes:
the first acquisition module is used for acquiring attribute information of a plurality of controls in a second interface of the application to be tested by adopting a target test tool;
and the first operation module is used for executing test operation on the target control in the second interface by adopting the target test tool according to the attribute information of the plurality of controls and the attribute information of the target control.
Optionally, the apparatus further includes:
the second acquisition module is used for acquiring the second character and the screenshot of the second interface if the test of the target test tool on the second interface fails, and the attribute information of the target control comprises the second character;
the first processing module is used for processing the screenshot of the second interface to obtain a plurality of character sets and position information of each character set in the second interface;
A first determining module, configured to determine a target character set matched with the second character from the plurality of character sets;
and the second operation module is used for carrying out test operation on the target control in the second interface by adopting the target test tool according to the position information of the target character set.
Optionally, the identifying module 802 is specifically configured to perform character identification on the first screenshot by using a preset character identification model by using a test terminal, so as to obtain a plurality of groups of characters and position information of each group of characters in the first interface.
Optionally, the identifying module 802 is specifically configured to send, by the test terminal, the first screenshot to the test server if the test terminal has started the text recognition service, so that the test server performs character recognition on the first screenshot by using a preset character recognition model, to obtain the multiple groups of characters and position information of each group of characters in the first interface; and receiving the multiple groups of characters returned by the test server and the position information of each group of characters in the first interface.
Optionally, the apparatus further includes:
And the second processing module is used for processing the screenshot of the first interface by adopting a preset character recognition model by the test terminal if the test server does not return the plurality of groups of characters and the position information of each group of characters in the first interface within the preset time length, so as to obtain the plurality of groups of characters and the position information of each group of characters in the first interface.
Optionally, the apparatus further includes:
the training module is used for training the initial character recognition model according to the plurality of sample images and the labeling information of the sample images to obtain the preset character recognition model;
the annotation information is used for representing sample characters in the sample image and positions of the sample characters in the sample image.
The foregoing apparatus is used for executing the method provided in the foregoing embodiment, and its implementation principle and technical effects are similar, and are not described herein again.
The above modules may be one or more integrated circuits configured to implement the above methods, for example: one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), or one or more microprocessors (digital singnal processor, abbreviated as DSP), or one or more field programmable gate arrays (Field Programmable Gate Array, abbreviated as FPGA), or the like. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
Fig. 9 is a schematic structural diagram of a processing device according to an embodiment of the present invention, as shown in fig. 9, the processing device may include: processor 901, memory 902.
The memory 902 is used for storing a program, and the processor 901 calls the program stored in the memory 902 to execute the above method embodiment. The specific implementation manner and the technical effect are similar, and are not repeated here.
For example, the method may include:
acquiring a test script containing a first character and a screenshot of a first interface of an application to be tested;
character recognition is carried out on the screenshot of the first interface, so that a plurality of groups of characters and the position information of each group of characters in the first interface are obtained;
determining a target set of characters from the plurality of sets of characters that match the first character;
and performing test operation on the first interface according to the test script and the position information corresponding to the target group character.
Optionally, the character recognition is performed on the screenshot of the first interface to obtain a plurality of groups of characters and position information of each group of characters in the first interface, including:
determining a target sentence in the test script according to the first character, a preset prefix name and an operation instruction;
And executing the target sentence, and processing the screenshot of the first interface to obtain the plurality of groups of characters and the position information of each group of characters in the first interface.
Optionally, the determining, according to the first character, the preset prefix name and the operation instruction, the target sentence in the test script includes:
replacing an initial picture in an initial sentence with the first character to obtain a replaced initial sentence, wherein the replaced initial sentence further comprises the operation instruction;
and adding the preset prefix name to the preset position of the replaced initial sentence to obtain the target sentence.
Optionally, determining a target set of characters matching the first character from the multiple sets of characters includes:
respectively matching each group of characters with the first character to obtain a matching degree corresponding to each group of characters;
and determining target group characters matched with the first character from the plurality of groups of characters according to the matching degree corresponding to each group of characters.
Optionally, the method further comprises:
acquiring attribute information of a plurality of controls in a second interface of the application to be tested by adopting a target test tool;
And executing test operation on the target control in the second interface by adopting the target test tool according to the attribute information of the plurality of controls and the attribute information of the target control.
Optionally, the method further comprises:
if the target testing tool fails to test the second interface, acquiring the second character and a screenshot of the second interface, wherein the attribute information of the target control comprises the second character;
processing the screenshot of the second interface to obtain a plurality of character sets and position information of each character set in the second interface;
determining a target character set matched with the second character from the plurality of character sets;
and adopting the target testing tool to perform testing operation on the target control in the second interface according to the position information of the target character set.
Optionally, the character recognition is performed on the first screenshot to obtain a plurality of groups of characters and position information of each group of characters in the first interface, including:
and carrying out character recognition on the first interface screenshot by the test terminal through a preset character recognition model to obtain a plurality of groups of characters and position information of each group of characters in the first interface.
Optionally, the character recognition is performed on the first screenshot to obtain a plurality of groups of characters and position information of each group of characters in the first interface, including:
if the test terminal starts the character recognition service, the test terminal sends the first interface screenshot to a test server so that the test server adopts a preset character recognition model to perform character recognition on the first screen screenshot to obtain the plurality of groups of characters and the position information of each group of characters in the first interface;
and receiving the multiple groups of characters returned by the test server and the position information of each group of characters in the first interface.
Optionally, the method further comprises:
if the test server side does not return the plurality of groups of characters and the position information of each group of characters in the first interface within the preset time length, the test terminal adopts a preset character recognition model to process the screenshot of the first interface, so that the plurality of groups of characters and the position information of each group of characters in the first interface are obtained.
Optionally, the method further comprises:
training an initial character recognition model according to a plurality of sample images and labeling information of the sample images to obtain the preset character recognition model;
The annotation information is used for representing sample characters in the sample image and positions of the sample characters in the sample image.
In summary, the screenshot of the first interface is subjected to character recognition, the target group character matched with the first character is determined from the plurality of groups of recognized characters, the first interface is subjected to test operation according to the position information corresponding to the recognized target group character, and the operation position of the first interface can be accurately determined by the screenshot character recognition of the first interface and the manner that the plurality of groups of recognized characters are matched with the first character, so that the test accuracy can be improved.
Optionally, the present invention also provides a program product, such as a computer readable storage medium, comprising a program for performing the above-described method embodiments when being executed by a processor.
For example, the method may include:
acquiring a test script containing a first character and a screenshot of a first interface of an application to be tested;
character recognition is carried out on the screenshot of the first interface, so that a plurality of groups of characters and the position information of each group of characters in the first interface are obtained;
determining a target set of characters from the plurality of sets of characters that match the first character;
And performing test operation on the first interface according to the test script and the position information corresponding to the target group character.
Optionally, the character recognition is performed on the screenshot of the first interface to obtain a plurality of groups of characters and position information of each group of characters in the first interface, including:
determining a target sentence in the test script according to the first character, a preset prefix name and an operation instruction;
and executing the target sentence, and processing the screenshot of the first interface to obtain the plurality of groups of characters and the position information of each group of characters in the first interface.
Optionally, the determining, according to the first character, the preset prefix name and the operation instruction, the target sentence in the test script includes:
replacing an initial picture in an initial sentence with the first character to obtain a replaced initial sentence, wherein the replaced initial sentence further comprises the operation instruction;
and adding the preset prefix name to the preset position of the replaced initial sentence to obtain the target sentence.
Optionally, determining a target set of characters matching the first character from the multiple sets of characters includes:
Respectively matching each group of characters with the first character to obtain a matching degree corresponding to each group of characters;
determining a target group character matched with the first character from the plurality of groups of characters according to the matching degree corresponding to each group of characters
Optionally, the method further comprises:
acquiring attribute information of a plurality of controls in a second interface of the application to be tested by adopting a target test tool;
and executing test operation on the target control in the second interface by adopting the target test tool according to the attribute information of the plurality of controls and the attribute information of the target control.
Optionally, the method further comprises:
if the target testing tool fails to test the second interface, acquiring the second character and a screenshot of the second interface, wherein the attribute information of the target control comprises the second character;
processing the screenshot of the second interface to obtain a plurality of character sets and position information of each character set in the second interface;
determining a target character set matched with the second character from the plurality of character sets;
and adopting the target testing tool to perform testing operation on the target control in the second interface according to the position information of the target character set.
Optionally, the character recognition is performed on the first screenshot to obtain a plurality of groups of characters and position information of each group of characters in the first interface, including:
and carrying out character recognition on the first interface screenshot by the test terminal through a preset character recognition model to obtain a plurality of groups of characters and position information of each group of characters in the first interface.
Optionally, the character recognition is performed on the first screenshot to obtain a plurality of groups of characters and position information of each group of characters in the first interface, including:
if the test terminal starts the character recognition service, the test terminal sends the first interface screenshot to a test server so that the test server adopts a preset character recognition model to perform character recognition on the first screen screenshot to obtain the plurality of groups of characters and the position information of each group of characters in the first interface;
and receiving the multiple groups of characters returned by the test server and the position information of each group of characters in the first interface.
Optionally, the method further comprises:
if the test server side does not return the plurality of groups of characters and the position information of each group of characters in the first interface within the preset time length, the test terminal adopts a preset character recognition model to process the screenshot of the first interface, so that the plurality of groups of characters and the position information of each group of characters in the first interface are obtained.
Optionally, the method further comprises:
training an initial character recognition model according to a plurality of sample images and labeling information of the sample images to obtain the preset character recognition model;
the annotation information is used for representing sample characters in the sample image and positions of the sample characters in the sample image.
In summary, the screenshot of the first interface is subjected to character recognition, the target group character matched with the first character is determined from the plurality of groups of recognized characters, the first interface is subjected to test operation according to the position information corresponding to the recognized target group character, and the operation position of the first interface can be accurately determined by the screenshot character recognition of the first interface and the manner that the plurality of groups of recognized characters are matched with the first character, so that the test accuracy can be improved.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in hardware plus software functional units.
The integrated units implemented in the form of software functional units described above may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (english: processor) to perform some of the steps of the methods according to the embodiments of the invention. And the aforementioned storage medium includes: u disk, mobile hard disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (13)
1. An application testing method, comprising:
acquiring a test script containing a first character and a screenshot of a first interface of an application to be tested;
character recognition is carried out on the screenshot of the first interface, so that a plurality of groups of characters and the position information of each group of characters in the first interface are obtained;
determining a target set of characters from the plurality of sets of characters that match the first character;
and performing test operation on the first interface according to the test script and the position information corresponding to the target group character.
2. The method of claim 1, wherein the character recognition of the screenshot of the first interface to obtain a plurality of sets of characters and location information of each set of characters in the first interface comprises:
determining a target sentence in the test script according to the first character, a preset prefix name and an operation instruction;
And executing the target sentence, and processing the screenshot of the first interface to obtain the plurality of groups of characters and the position information of each group of characters in the first interface.
3. The method according to claim 2, wherein the determining the target sentence in the test script according to the first character, the preset prefix name, and the operation instruction includes:
replacing an initial picture in an initial sentence with the first character to obtain a replaced initial sentence, wherein the replaced initial sentence further comprises the operation instruction;
and adding the preset prefix name to the preset position of the replaced initial sentence to obtain the target sentence.
4. The method of claim 1, wherein determining a target set of characters from the plurality of sets of characters that match the first character comprises:
respectively matching each group of characters with the first character to obtain a matching degree corresponding to each group of characters;
and determining target group characters matched with the first character from the plurality of groups of characters according to the matching degree corresponding to each group of characters.
5. The method according to claim 1, wherein the method further comprises:
Acquiring attribute information of a plurality of controls in a second interface of the application to be tested by adopting a target test tool;
and executing test operation on the target control in the second interface by adopting the target test tool according to the attribute information of the plurality of controls and the attribute information of the target control.
6. The method of claim 5, wherein the method further comprises:
if the target testing tool fails to test the second interface, acquiring the second character and a screenshot of the second interface, wherein the attribute information of the target control comprises the second character;
processing the screenshot of the second interface to obtain a plurality of character sets and position information of each character set in the second interface;
determining a target character set matched with the second character from the plurality of character sets;
and adopting the target testing tool to perform testing operation on the target control in the second interface according to the position information of the target character set.
7. The method of claim 1, wherein the performing character recognition on the first screenshot to obtain a plurality of sets of characters and position information of each set of characters in the first interface includes:
And carrying out character recognition on the first interface screenshot by the test terminal through a preset character recognition model to obtain a plurality of groups of characters and position information of each group of characters in the first interface.
8. The method of claim 1, wherein the performing character recognition on the first screenshot to obtain a plurality of sets of characters and position information of each set of characters in the first interface includes:
if the test terminal starts the character recognition service, the test terminal sends the first interface screenshot to a test server so that the test server adopts a preset character recognition model to perform character recognition on the first screen screenshot to obtain the plurality of groups of characters and the position information of each group of characters in the first interface;
and receiving the multiple groups of characters returned by the test server and the position information of each group of characters in the first interface.
9. The method of claim 8, wherein the method further comprises:
if the test server side does not return the plurality of groups of characters and the position information of each group of characters in the first interface within the preset time length, the test terminal adopts a preset character recognition model to process the screenshot of the first interface, so that the plurality of groups of characters and the position information of each group of characters in the first interface are obtained.
10. The method according to claim 7 or 8, characterized in that the method further comprises:
training an initial character recognition model according to a plurality of sample images and labeling information of the sample images to obtain the preset character recognition model;
the annotation information is used for representing sample characters in the sample image and positions of the sample characters in the sample image.
11. An application testing apparatus, comprising:
the acquisition module is used for acquiring a test script containing a first character and a screenshot of a first interface of an application to be tested;
the recognition module is used for carrying out character recognition on the screenshot of the first interface to obtain a plurality of groups of characters and the position information of each group of characters in the first interface;
a determining module, configured to determine a target set of characters that matches the first character from the multiple sets of characters;
and the operation module is used for carrying out test operation on the first interface according to the test script and the position information corresponding to the target group character.
12. A processing apparatus, comprising: a memory storing a computer program executable by the processor, and a processor implementing the test method of any one of the preceding claims 1-10 when the processor executes the computer program.
13. A computer readable storage medium, characterized in that the storage medium has stored thereon a computer program which, when read and executed, implements the test method according to any of the preceding claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310135419.7A CN116431468A (en) | 2023-02-17 | 2023-02-17 | Test method, test device, test processing equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310135419.7A CN116431468A (en) | 2023-02-17 | 2023-02-17 | Test method, test device, test processing equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116431468A true CN116431468A (en) | 2023-07-14 |
Family
ID=87082161
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310135419.7A Pending CN116431468A (en) | 2023-02-17 | 2023-02-17 | Test method, test device, test processing equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116431468A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116662211A (en) * | 2023-07-31 | 2023-08-29 | 四川弘和数智集团有限公司 | Display interface testing method, device, equipment and medium |
-
2023
- 2023-02-17 CN CN202310135419.7A patent/CN116431468A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116662211A (en) * | 2023-07-31 | 2023-08-29 | 四川弘和数智集团有限公司 | Display interface testing method, device, equipment and medium |
| CN116662211B (en) * | 2023-07-31 | 2023-11-03 | 四川弘和数智集团有限公司 | Display interface testing method, device, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109408384B (en) | Software application testing method and device, processor and electronic device | |
| CN107783898B (en) | Test method and test equipment for mobile application | |
| CN106485261B (en) | Image recognition method and device | |
| CN109194689B (en) | Abnormal behavior recognition method, device, server and storage medium | |
| CN109740040B (en) | Verification code identification method, device, storage medium and computer equipment | |
| CN109685805B (en) | Image segmentation method and device | |
| CN110826646A (en) | Robot vision testing method and device, storage medium and terminal equipment | |
| CN116431468A (en) | Test method, test device, test processing equipment and storage medium | |
| CN111198815B (en) | Compatibility testing method and device for user interface | |
| CN108845955B (en) | Chinese input method test method and device and electronic equipment | |
| CN113190444B (en) | Test method, test device and storage medium | |
| CN110688319A (en) | Application keep-alive capability test method and related device | |
| CN107783867B (en) | Gesture testing method and device, computer equipment and storage medium | |
| CN105302715B (en) | The acquisition methods and device of application program user interface | |
| CN115687146A (en) | BIOS (basic input output System) test method and device, computer equipment and storage medium | |
| CN110221958B (en) | Application testing method, device, computing equipment and computer readable storage medium | |
| CN111949510B (en) | Test processing method, device, electronic equipment and readable storage medium | |
| CN110085237B (en) | Recovery method, device and equipment of interactive process | |
| CN113360414A (en) | Software testing method, device, readable storage medium and computer program product | |
| CN111722886A (en) | Tool starting method and device, computer equipment and computer readable storage medium | |
| CN110968519A (en) | Game testing method, device, server and storage medium | |
| CN110909739A (en) | Picture identification and operation method and device, computer equipment and storage medium | |
| CN110297777B (en) | Input method evaluation method and device | |
| CN117389664B (en) | Unique control region division method and device, electronic equipment and storage medium | |
| CN116431461A (en) | Desktop application testing method, device, electronic equipment and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |