CN114328235A

CN114328235A - Test process verification method and medium for test case

Info

Publication number: CN114328235A
Application number: CN202111643191.XA
Authority: CN
Inventors: 王杰; 黄正
Original assignee: Zhejiang Lianyun Zhihui Technology Co ltd
Current assignee: Zhejiang Lianyun Zhihui Technology Co ltd
Priority date: 2021-12-29
Filing date: 2021-12-29
Publication date: 2022-04-12

Abstract

The application relates to the technical field of automatic testing, in particular to a test process verification method and a medium of a test case, which comprises the following steps: when a test case is executed on a user interface, determining a control on the uppermost layer of the user interface, and acquiring a screenshot of the control; identifying a color category set contained in the screenshot; comparing the color category set with a preset color category set; if the color type set is consistent with the preset color type set, determining that the test case is abnormal in execution; and if at least one color type in the preset color type set is not in the color type set, determining that the test case is normally executed. The method and the device have the effects of improving the testing efficiency during the automatic UI testing and reducing the workload.

Description

Test process verification method and medium for test case

Technical Field

The present application relates to the field of automated testing technologies, and in particular, to a method and medium for verifying a test procedure of a test case.

Background

The automatic test is a process for converting the test behavior driven by human into machine execution, and the latent defect in the software is discovered by setting a test case script, evaluating and executing the test on the software test.

In the automated testing, automated testing of a User Interface (UI) related to user interaction is also an important link in software testing. Generally, in the image-contrast-based UI automation test in the related art, a correct page screenshot is manually and periodically captured as an expected result, then the screenshot obtained in the UI automation execution process is compared with the expected result, if the screenshot is the same as the expected result, the test case passes, and otherwise, the test case does not pass.

However, in the UI automation test in the related art, the image of the screenshot is compared with the expected result, the UI image has many subtle differences, and the factors such as the font, the shape, and the characters on the UI image have a large influence on the image comparison, so that a large number of correct screenshots are often required to be captured as comparison objects for image comparison.

Disclosure of Invention

In order to improve the test efficiency during UI automatic test and reduce the workload, the application provides a test process verification method and medium of a test case.

In a first aspect, the test procedure verification method for the test case provided by the application adopts the following technical scheme:

the test process verification method of the test case is characterized in that: the method comprises the following steps:

when a test case is executed on a user interface, determining a control on the uppermost layer of the user interface, and acquiring a screenshot of the control;

identifying a set of color categories contained in the screenshot;

comparing the color category set with a preset color category set;

if the color type set is consistent with the preset color type set, determining that the test case is abnormal in execution;

and if at least one color type in the preset color type set is not in the color type set, determining that the test case is normally executed.

In some embodiments, after determining that the test case is abnormal in execution, the method further comprises: when the test case is executed to obtain a test result, marking the test result as a first state, wherein the first state represents that the test result of the test case fails to be executed;

after determining that the test case is executed normally, the method further includes: when the test case is executed to obtain a test result, the test result is marked to be in a second state, and the second state represents that the test result of the test case is successfully executed.

In some embodiments, after determining that the test case is abnormal in execution, the method further comprises: storing the screenshot into a first picture library, wherein the first picture library is used for storing pictures with abnormal test case execution;

after determining that the test case is executed normally, the method further includes: and storing the screenshot into a second picture library, wherein the second picture library is used for storing pictures with normal test case execution.

In some embodiments, if the color category set includes the preset color category set and at least one color category in the color category set is not in the preset color category set, comparing the screenshot with pictures in the first picture library, and comparing the screenshot with pictures in the second picture library, where the first picture library is used to store pictures with abnormal test case execution and the second picture library is used to store pictures with normal test case execution;

if the similarity between the screenshot and any picture in the first picture library exceeds a first threshold and the similarity between the screenshot and any picture in the second picture library is lower than a second threshold, determining that the test case is abnormal to execute, storing the screenshot into the first picture library, and marking a test result of the screenshot as a first state, wherein the first state represents that the test result of the test case fails to execute;

if the similarity between the screenshot and any picture in the second picture library is lower than a first threshold value and the similarity between the screenshot and any picture in the second picture library exceeds a second threshold value, determining that the test case is normally executed, storing the screenshot into the second picture library, and marking the test result of the screenshot as a second state, wherein the second state represents that the test result of the test case is successfully executed.

In some of these embodiments, the first and second thresholds are manually adjusted by a human.

In some of these embodiments, the method further comprises the steps of:

after the test case is completely executed, manually judging the execution result of the test case, the first picture library and the second picture library;

if the test case execution result contains the case with the execution error, manually correcting;

if the second picture library has the mistaken screenshot, manually moving the mistaken screenshot into the first picture library;

and if the misjudged screenshot exists in the first picture library, manually moving the misjudged screenshot into the second picture library.

In some of these embodiments, the first and second thresholds are also automatically adjusted according to a method of selecting values, the method of selecting values comprising the steps of:

packing the mistakenly judged screenshots which are manually judged and then moved from the second picture library to the first picture library into an abnormal set;

acquiring the similarity between each misjudged screenshot in the abnormal set and any screenshot in the first picture library;

selecting the similarity with the minimum value in the abnormal set, defining the similarity as a first reference threshold, and judging the size of the first reference threshold and the current first threshold;

if the numerical value of the first reference threshold is larger than or equal to the numerical value of the current first threshold, the numerical value of the first threshold is not changed, and if the numerical value of the first reference threshold is smaller than the numerical value of the current first threshold, the first reference threshold is used as a new first threshold;

packing the mistakenly judged screenshots which are manually judged and then moved from the first picture library to the second picture library into a normal set;

acquiring the similarity between each misjudged screenshot in the normal set and any screenshot in the second picture library;

selecting the similarity with the minimum value in the abnormal set, defining the similarity as a second reference threshold, and judging the size of the second reference threshold and the current second threshold;

if the numerical value of the second reference threshold is equal to or greater than the current numerical value of the second threshold, the numerical value of the second threshold is not changed, and if the numerical value of the second reference threshold is smaller than the current numerical value of the second threshold, the second reference threshold is set as a new second threshold.

In some embodiments, when the number of the screenshots in the first picture library and the second picture library is greater than a preset number threshold, the repeated screenshots in the first picture library and the second picture library are rejected according to a deduplication method, where the deduplication method includes:

selecting any screenshot in a first picture library;

comparing the screenshots with the remaining screenshots in the first picture library one by one,

if the remaining screenshots have screenshots with the similarity larger than or equal to a preset repeated threshold value with the selected screenshot, removing the selected screenshot from a first picture library,

if the rest screenshots do not have screenshots with the similarity larger than or equal to the preset repeated threshold value with the selected screenshot, storing the selected screenshot into a first picture library and selecting another screenshot for comparison until the number of the screenshots in the first picture library is smaller than the preset number threshold value;

selecting any screenshot in the second picture library;

comparing the screenshots with the remaining screenshots in the second picture library one by one,

if the remaining screenshots have screenshots with the similarity larger than or equal to a preset repeated threshold value with the selected screenshot, removing the selected screenshot from a second picture library,

if the rest screenshots do not have screenshots with the similarity larger than or equal to the preset repeated threshold value with the selected screenshot, storing the selected screenshot into the second picture library and selecting another screenshot for comparison until the number of the screenshots in the second picture library is smaller than the preset number threshold value.

In a second aspect, the present application provides a computer-readable storage medium, which adopts the following technical solutions:

a computer readable storage medium storing a computer program capable of being loaded by a processor with any one of the methods described above.

By the test case test process verification method and medium provided by the embodiment of the application, the image of the uppermost layer of the user interface is obtained, and the screenshot is performed according to the length and width range, so that the size of the screenshot can be reduced, the influence of invalid information in the screenshot on the image comparison judgment result is reduced, the judgment is performed only for the color in the image, a large number of screenshots of correct pages are not required to be manually intercepted as expected results to be compared, the test flow is simplified to a certain extent, and the complexity of image similarity comparison is also simplified.

Drawings

FIG. 1 is a flowchart of a test procedure verification method for test cases in an embodiment of the present application;

FIG. 2 is a flowchart after a test case is determined in the embodiment of the present application;

FIG. 3 is a flow chart of a method of selecting values in an embodiment of the present application;

fig. 4 is a flowchart of a method of flushing in an embodiment of the present application.

Detailed Description

For a clearer understanding of the objects, aspects and advantages of the present application, reference is made to the following description and accompanying drawings. However, it will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In some instances, well known methods, procedures, systems, components, and/or circuits have been described at a higher level without undue detail in order to avoid obscuring aspects of the application with unnecessary detail. It will be apparent to those of ordinary skill in the art that various changes can be made to the embodiments disclosed herein, and that the general principles defined herein may be applied to other embodiments and applications without departing from the principles and scope of the present application. Thus, the present application is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the scope of the present application as claimed.

Unless defined otherwise, technical or scientific terms used herein shall have the same general meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application, the terms "a," "an," "the," and the like do not denote a limitation of quantity, but rather are used in the singular or the plural. The terms "comprises," "comprising," "has," "having," and any variations thereof, as referred to in this application, are intended to cover non-exclusive inclusions; for example, a process, method, and system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or modules, but may include other steps or modules (elements) not listed or inherent to such process, method, article, or apparatus.

Reference to "a plurality" in this application means two or more. In general, the character "/" indicates a relationship in which the objects associated before and after are an "or". The terms "first," "second," "third," and the like in this application are used for distinguishing between similar items and not necessarily for describing a particular sequential or chronological order.

The terms "system," "engine," "unit," "module," and/or "block" referred to herein is a method for distinguishing, by level, different components, elements, parts, components, assemblies, or functions of different levels. These terms may be replaced with other expressions capable of achieving the same purpose. In general, reference herein to a "module," "unit," or "block" refers to a collection of logic or software instructions embodied in hardware or firmware. The "modules," "units," or "blocks" described herein may be implemented as software and/or hardware, and in the case of implementation as software, they may be stored in any type of non-volatile computer-readable storage medium or storage device.

In some embodiments, software modules/units/blocks may be compiled and linked into an executable program. It will be appreciated that software modules may be invokable from other modules/units/blocks or from themselves, and/or may be invoked in response to detected events or interrupts. Software modules/units/blocks configured for execution on a computing device may be provided on a computer-readable storage medium, such as a compact disc, digital video disc, flash drive, magnetic disk, or any other tangible medium, or downloaded as digital (and may be initially stored in a compressed or installable format that requires installation, decompression, or decryption prior to execution). Such software code may be stored partially or wholly on a storage device of the executing computing device and applied in the operation of the computing device. The software instructions may be embedded in firmware, such as an EPROM. It will also be appreciated that the hardware modules/units/blocks may be included in connected logic components, such as gates and flip-flops, and/or may be included in programmable units, such as programmable gate arrays or processors. The modules/units/blocks or computing device functions described herein may be implemented as software modules/units/blocks, and may also be represented in hardware or firmware. Generally, the modules/units/blocks described herein may be combined with other modules/units/blocks or, although they are physically organized or stored, may be divided into sub-modules/sub-units/sub-blocks. The description may apply to the system, the engine, or a portion thereof.

It will be understood that when an element, engine, module or block is referred to as being "on," "connected to" or "coupled to" another element, engine, module or block, it can be directly on, connected or coupled to or in communication with the other element, engine, module or block, or intervening elements, engines, modules or blocks may be present, unless the context clearly dictates otherwise. In this application, the term "and/or" may include any one or more of the associated listed items or combinations thereof.

The embodiment of the application discloses a test process verification method of a test case.

As shown in fig. 1, the test procedure verification method for the test case includes the following steps:

s10, when the test case is executed on the user interface, determining the control on the uppermost layer of the user interface, and acquiring the screenshot of the control.

S20, identifying the color category set contained in the screenshot.

And S30, comparing the color category set with a preset color category set.

S40, if the color type set is consistent with the preset color type set, determining that the test case is abnormal in execution.

S50, if at least one color type in the preset color type set is not in the color type set, determining that the test case is executed normally.

The Test Case (Test Case) is a description of a Test task performed on a specific software product, and embodies a Test scheme, a method and a strategy. The contents of the test object, the test environment, the input data, the test steps, the expected results, the test scripts and the like are included, and finally, a document is formed. Simply considered, a test case is a set of test inputs, execution conditions, and expected results compiled with a particular goal for verifying that a particular software requirement is met.

Test cases are executed by writing a test script, which generally refers to a series of instructions for a particular test that can be executed by an automated test tool. To improve the maintainability and reusability of test scripts, they must be built before they are executed. It may be found that the pre-existing operation will occur during several tests. Therefore, the destination should determine the targets of these operations so that their implementations can be multiplexed. A test script is computer readable instructions that automatically perform a test procedure (or a portion of a test procedure). The test script can be created (recorded) or automatically generated using a test automation tool, or done in a programming language, or it can be done by combining the first three methods.

A User Interface (UI) is a medium for interaction and exchange between a system and a User, and it realizes conversion between an internal form of information and a form that can be received by a human being. The crowded interface is related software designed for interaction between users and hardware, so that users can conveniently and effectively operate the hardware to achieve bidirectional interaction and complete desired work.

Determining project requirements, determining that the pages need to be traversed, and writing test case scripts, wherein in the tests, a correct page screenshot does not need to be acquired as an expected result of picture comparison, and only common operation steps of button clicking, text inputting and the like need to be written according to a service scene.

When the execution case with one user interface is executed, a plurality of layers of controls often exist on the user interface. The control is used for packaging data and methods, and the user interface control is used for developing and constructing a user interface and helping to complete development of interface elements such as windows, text boxes, buttons, pull-down menus and the like in software development. The concept of layers refers to DIV, which refers to positioning techniques in a cascading style, which may sometimes be referred to as layers. In this embodiment, when the test case is executed, what is intercepted is the DIV of the top control of the user interface.

And after the DIV of the uppermost control is obtained, screenshot is carried out on the DIV where the uppermost control is located according to the length and width range of the DIV.

And carrying out image recognition on the captured screenshot, and recognizing a color type set contained in the screenshot. The color category set includes at least one color.

As shown in FIG. 1, the set of preset color types is the color type on the abnormal interface when the abnormal condition occurs during the execution of the script. In order to reduce the erroneous judgment, a convention needs to be made in the code writing specification, for example, when a system exception error occurs, all popup windows should be marked with red fonts. At this time, the preset color type set can be preset to be red, so that when a red font error occurs, if red exists in the color type set, the execution exception of the test case is described.

In step S500, at least one color type in the preset color type set is not in the color type set, specifically: when the preset color type set only has one color, such as red, and only one color exists in the color type set, such as green, the red is not in the color type set under the condition that the red is not in the color type set, and the condition that at least one color in the preset color type set is not in the color type set is met, the test case is determined to be normally executed; when the preset color type is only one, such as red, a plurality of colors exist in the color type set, such as green and blue, but no red exists, the red is not in the color type set under the condition that the red is not in the color type set, and the condition that at least one color in the preset color type set is not in the color type set is met, the test case is determined to be normally executed; when the preset color type set has multiple colors, such as red and green, only one color exists in the color type set, such as black, but any one of the colors of the red and the green does not exist, and the condition that at least one color in the preset color type set is not in the color type set is met, the test case is determined to be normally executed; if the preset color category set has multiple colors, such as red and green, multiple colors exist in the color category set, such as blue and yellow, but any one of the colors of red and green does not exist, and the condition that at least one color in the preset color category set is not in the color category set is met, the test case is determined to be normally executed.

It should be noted that when there are multiple preset color category sets, such as red and green, the execution exception of the test case is defined when any one of the red or green exists in the color category set, rather than when there are two colors of red and green.

Further, since the bottom color of most of the controls of the user interface is white, in general, white may not be used as a contrast color between the color type set and the preset color type set, for example, when the recognized color of one control includes red and white, only red exists in the preset color type set, and at this time, white does not enter the color type set, so that the screenshot of red and white still exists and is determined as the execution exception of the test case.

The method only obtains the image of the uppermost layer of the user interface, and carries out screenshot according to the length and width range, so that the size of the screenshot can be reduced, the influence of invalid information in the screenshot on a picture comparison judgment result is reduced, the judgment is only carried out aiming at the color in the image, and a large number of correct page screenshots are not required to be manually intercepted as expected results to be compared, so that the test flow is simplified to a certain extent, and the complexity of picture similarity comparison is also simplified.

As shown in fig. 2, after determining that the execution of the execution use case is abnormal, the method further includes:

s60, when the test case is executed to obtain the test result, the test result is marked as a first state, and the first state represents that the test result of the test case fails to be executed.

After determining the execution exception of the test case, the method further comprises:

and S61, storing the screenshot into a first picture library, wherein the first picture library is used for storing pictures with abnormal test case execution.

After determining that the test case is executed normally, the method further includes:

and S70, when the test case is executed to obtain the test result, marking the test result as a second state, wherein the second state represents that the test result of the test case is successfully executed.

and S71, storing the screenshot into a second picture library, wherein the second picture library is used for storing pictures which are normal for test case execution.

As shown in fig. 2, after the test result of the test case fails to be executed, the screenshot is saved to the first picture library for storing the picture in which the test case is abnormally executed. And when the test result of the execution case is successfully executed, saving the screenshot to a second picture library for storing pictures of normal execution of the test case.

And the number of the pictures in the first picture library and the second picture library is expanded according to multiple tests of the test case. In the initial state, the operator can also obtain some pictures of normal test case execution and abnormal test case execution from other related channels, and respectively store the pictures in the first picture library and the second picture library as initial comparison examples.

As shown in fig. 2, the method further comprises the steps of:

s80, if the color type set includes a preset color type set and at least one color type in the color type set is not in the preset color type set, comparing the screenshot with the pictures in the first picture library, and comparing the screenshot with the pictures in the second picture library.

S90, if the similarity between the screenshot and any picture in the first picture library exceeds a first threshold value and the similarity between the screenshot and any picture in the second picture library is lower than a second threshold value, determining that the test case is executed abnormally, storing the screenshot into the first picture library, and marking the test result of the screenshot as a first state.

And S100, if the similarity between the screenshot and any picture in the second picture library is lower than a first threshold value and the similarity between the screenshot and any picture in the second picture library exceeds a second threshold value, determining that the test case is normally executed, storing the screenshot into the second picture library, and marking the test result of the screenshot as a second state.

The first picture library is used for storing pictures with abnormal test case execution, and the second picture library is used for storing pictures with normal test case execution. The first state represents that the test result of the test case is failed to be executed, and the second state represents that the test result of the test case is successfully executed.

The first threshold value: and on the premise of the same length and width of any picture in the screenshot and the first picture library, the similarity between the two pictures is determined, wherein the default is that in the same test case, the size of the control in the same level cannot change along with the running of the test. When the size of the control is increased under human interference, for example, when an operator increases the proportion of the whole user interface through the size of the computer, the screenshot and any picture in the first picture library need to be compared under the condition that the screenshot and any picture in the first picture library are converted into the same length and width.

The second threshold value: and on the premise of the same length and width of the screenshot and any picture in the second picture library, the similarity between the two pictures is determined, wherein the default is that in the same test case, the size of the control in the same level does not change along with the running of the test. When the size of the control is increased under human interference, for example, when an operator increases the proportion of the whole user interface through the size of the computer, the screenshot and any picture in the first picture library need to be compared under the condition that the screenshot and any picture in the first picture library are converted into the same length and width.

The initial values of the first threshold and the second threshold are manually adjusted by a human.

For example, the operator sets the first threshold to 70% and the second threshold to 60%, when the similarity between the screenshot and a certain picture in the first picture library is 75%, the similarity is greater than the first threshold, the screenshot is compared with the pictures in the second picture library, when the similarity between the screenshot and the certain picture in the second picture library is found to be 30% and the similarity is less than the second threshold, the execution exception of the test case is determined, the screenshot is stored in the first picture library, and the test result of the first screenshot is marked as the first state.

For another example, the operator sets the first threshold to 70% and the second threshold to 60%, when the similarity between the screenshot and a certain picture in the first picture library is 30%, the similarity is smaller than the first threshold, the screenshot is compared with the pictures in the second picture library, when the similarity between the screenshot and a certain picture in the second picture library is found to be 80%, the similarity is larger than the second threshold, it is determined that the test case is executed normally, the screenshot is stored in the second picture library, and the test result of the screenshot is marked as the second state.

Wherein, in a very small part of the situation, the similarity between the screenshot and any picture in the first picture library is larger than a first threshold, but the similarity between the screenshot and any picture in the second picture library is larger than a second threshold; or the similarity between the screenshot and any picture in the first picture is less than a first threshold, but the similarity between the screenshot and any picture in the second picture library is less than a second threshold.

In the above case, the two similarities obtained by comparing the screenshot with the first picture library and the second picture library respectively can be compared, and the similarity between the screenshot and which picture library is higher can be determined, and then the screenshot is pre-stored in which picture library. And if the similarity is consistent, storing the screenshot into any one of the picture libraries, and waiting for subsequent manual verification.

As shown in fig. 2, in order to reduce the misjudgment in some abnormal situations, a manual verification method is further included, which specifically includes the following steps:

and S110, after all the test cases are executed, manually judging the test case execution result, the first picture library and the second picture library, if the test case execution result contains a case with an execution error, manually correcting, if the second picture library contains a screenshot with a misjudgment, manually moving the screenshot with the misjudgment into the first picture library, and if the first picture library contains the screenshot with the misjudgment, manually moving the screenshot with the misjudgment into the second picture library.

During manual verification, if a misjudged screenshot exists in the first picture library, the misjudged screenshot is extracted from the first picture library and is placed in the second picture library, and the first state (test result execution failure of the test case) marked by the misjudged screenshot is changed into the second state (test result success of the test case); if the misjudged screenshots exist in the second picture library, the misjudged screenshots are extracted from the second picture library and are placed into the first picture library, and the second state (the test result of the test case is successfully executed) marked by the misjudged screenshots is changed into the first state (the test result of the test case is failed).

The algorithm cannot reach the perfection, the false judgment rate can be further reduced by a manual verification method, the lower the false alarm rate is, the longer the manual verification time interval can be, and the testing efficiency is improved.

In another embodiment, as shown in fig. 3, the values of the first threshold and the second threshold may be automatically adjusted according to a value selection method, which includes the following steps:

and S200, packing the mistakenly judged screenshots which are manually judged and then moved from the second picture library to the first picture library into an abnormal set.

S210, obtaining the similarity between each misjudged screenshot in the abnormal set and any screenshot in the first picture library.

S220, selecting the similarity with the minimum value in the abnormal set, defining the similarity as a first reference threshold, judging the size of the first reference threshold and the current first threshold, if the value of the first reference threshold is larger than or equal to the value of the current first threshold, not changing the value of the first threshold, and if the value of the first reference threshold is smaller than the value of the current first threshold, taking the first reference threshold as a new first threshold.

And S230, packing the mistakenly judged screenshots which are manually judged and then moved from the first picture library to the second picture library into a normal set.

S240, obtaining the similarity between each misjudged screenshot in the normal set and any screenshot in the second picture library.

S250, selecting the similarity with the minimum value in the abnormal set, defining the similarity as a second reference threshold, and judging the size of the second reference threshold and the current second threshold; if the numerical value of the second reference threshold is equal to or greater than the current numerical value of the second threshold, the numerical value of the second threshold is not changed, and if the numerical value of the second reference threshold is smaller than the current numerical value of the second threshold, the second reference threshold is set as a new second threshold.

As shown in fig. 3, the abnormal set includes all the misjudged screenshots that are manually moved from the second picture library to the first picture library, and the normal set includes all the misjudged screenshots that are manually moved from the first picture library to the second picture library.

The specific process is that all the screenshots in the abnormal set are compared with any screenshot in the first picture library, the smallest value of all the similarities in the abnormal set is selected as a first reference threshold, the magnitude of the first reference threshold and the current first threshold is judged, if the value of the first reference threshold is larger than or equal to the value of the first threshold, the value of the first threshold is not changed, and if the value of the first reference threshold is smaller than the current value of the first threshold, the first reference threshold is used as a new first threshold.

For example, the current first threshold is 95%, the similarity of three screenshots after comparison with the first picture library is 93%, 94% and 94.5%, and the screenshots are stored in the second picture library, after the operator performs manual verification, the screenshots are found to be abnormal screenshots, the screenshots are extracted from the second picture library and are placed back to the first picture library, at this time, the three screenshots are compared with any picture in the first picture library again, the compared similarities are respectively 93.7%, 94.4% and 96.4%, at this time, the smallest similarity is 94.7%, and the similarity is compared with the first threshold, it can be seen that the similarity is smaller than the first threshold, the value of the new first threshold is changed into 93%, 7%, and the numerical precision of the first threshold can be more accurate by using the method.

As shown in fig. 3, the method for selecting the second threshold is the same as the method for selecting the first threshold, and the specific process is as follows: comparing all the screenshots in the normal set with any screenshots in the second picture library, selecting the screenshot with the smallest value in all the similarities in the normal set as a second reference threshold, judging the value of the second reference threshold and the current second threshold, if the value of the second reference threshold is larger than or equal to the value of the second threshold, not changing the value of the second threshold, and if the value of the second reference threshold is smaller than the value of the current second threshold, taking the second reference threshold as a new second threshold.

The specific example is the same as the example of the value selecting method of the first threshold, and details are not described here.

As shown in fig. 4, in another embodiment, when the number of the screenshots in the first picture library and the second picture library is greater than a preset number threshold, the duplicate pictures in the first picture library and the second picture library are removed according to a duplicate removal method, where the duplicate removal method specifically includes the following steps:

s300, selecting any screenshot in the first picture library.

S310, comparing the screenshots with the remaining screenshots in the first picture library one by one.

And S311, if the rest screenshots have screenshots with the similarity more than or equal to the preset repeated threshold value with the selected screenshot, rejecting the selected screenshot from the first picture library.

And S312, if the rest screenshots do not have screenshots with the similarity degree with the selected screenshot being more than or equal to the preset repeated threshold, storing the selected screenshot into the first picture library and selecting another screenshot for comparison until the number of the screenshots in the first picture library is less than the preset number threshold.

And S320, selecting any screenshot in the second picture library.

And S330, comparing the screenshots with the remaining screenshots in the second picture library one by one.

And S331, if the rest screenshots have screenshots with the similarity more than or equal to the preset repeated threshold value with the selected screenshot, rejecting the selected screenshot from a second picture library.

And S332, if the rest screenshots do not have screenshots with the similarity degree with the selected screenshot being more than or equal to the preset repeated threshold, storing the selected screenshot into the second picture library and selecting another screenshot for comparison until the number of the screenshots in the second picture library is less than the preset number threshold.

As shown in fig. 4, the preset number threshold refers to the maximum number of pictures stored in the first picture library and the maximum number of pictures stored in the second picture library.

For example, when the preset number threshold of the first picture library is 500, after the test of the test case, 530 pictures are stored in the first picture library, the computer automatically runs the duplication elimination method, selects any one screenshot in the first picture library, compares the screenshot with the rest screenshots in the first picture library, when the similarity between the rest screenshots and the selected screenshot is more than or equal to the preset duplication threshold value, defines the screenshot as a duplicate screenshot, rejects the screenshot out of the first picture library, at this time, the number of pictures in the first picture library is changed to 529, continues to select the previous screenshot for comparison, if the similarity between the screenshot and all the remaining screenshots in the first picture library is less than the preset duplication threshold value, defines the screenshot as not a duplicate screenshot, saves the screenshot into the second picture library, selects another screenshot for comparison, until the number of screenshots in the second picture library is less than 500. The duplicate removal method of the second picture library is the same.

The embodiment of the application also discloses a computer readable storage medium which stores a computer program capable of being loaded by a processor and executing the method.

The above embodiments are preferred embodiments of the present application, and the protection scope of the present application is not limited by the above embodiments, so: all equivalent changes made according to the structure, shape and principle of the present application shall be covered by the protection scope of the present application.

Claims

1. The test process verification method of the test case is characterized in that: the method comprises the following steps:

identifying a set of color categories contained in the screenshot;

comparing the color category set with a preset color category set;

2. The test case test procedure verification method according to claim 1, characterized in that: after determining the execution exception of the test case, the method further comprises: when the test case is executed to obtain a test result, marking the test result as a first state, wherein the first state represents that the test result of the test case fails to be executed;

3. The test case test procedure verification method according to claim 2, wherein: after determining the execution exception of the test case, the method further comprises: storing the screenshot into a first picture library, wherein the first picture library is used for storing pictures with abnormal test case execution;

4. The test case test procedure verification method according to claim 1, characterized in that: the method further comprises the following steps:

if the color kind set comprises the preset color kind set and at least one color kind in the color kind set is not in the preset color kind set, comparing the screenshot with pictures in a first picture library, and simultaneously comparing the screenshot with pictures in a second picture library, wherein the first picture library is used for storing pictures with abnormal test case execution, and the second picture library is used for storing pictures with normal test case execution;

5. The test case test procedure verification method according to claim 4, wherein: the first and second thresholds are manually adjusted by a human.

6. The test case test procedure verification method according to claim 5, wherein: further comprising the steps of:

7. The test case test procedure verification method according to claim 6, wherein: the first threshold and the second threshold are also automatically adjusted according to a value selection method, wherein the value selection method comprises the following steps:

8. The test case test procedure verification method according to claim 7, wherein: when the number of the screenshots in the first picture library and the second picture library is larger than a preset number threshold, removing the repeated screenshots in the first picture library and the second picture library according to a duplication removing method, wherein the duplication removing method comprises the following steps:

selecting any screenshot in a first picture library;

selecting any screenshot in the second picture library;

9. A computer-readable storage medium characterized by: a computer program which can be loaded by a processor and which executes the method according to any of claims 1 to 8.