CN111198815B - Compatibility testing method and device for user interface - Google Patents
Compatibility testing method and device for user interface Download PDFInfo
- Publication number
- CN111198815B CN111198815B CN201911345387.3A CN201911345387A CN111198815B CN 111198815 B CN111198815 B CN 111198815B CN 201911345387 A CN201911345387 A CN 201911345387A CN 111198815 B CN111198815 B CN 111198815B
- Authority
- CN
- China
- Prior art keywords
- abnormal
- picture
- compatibility
- user interface
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 78
- 230000002159 abnormal effect Effects 0.000 claims abstract description 238
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 92
- 238000012549 training Methods 0.000 claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 33
- 230000002194 synthesizing effect Effects 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 6
- 230000008859 change Effects 0.000 claims description 3
- 230000005856 abnormality Effects 0.000 abstract description 18
- 230000008569 process Effects 0.000 description 6
- 230000015572 biosynthetic process Effects 0.000 description 5
- 238000003786 synthesis reaction Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011056 performance test Methods 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- User Interface Of Digital Computer (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application relates to the technical field of data service, and discloses a method and a device for testing compatibility of a user interface. The method comprises the steps of obtaining a user interface as a picture to be tested; inputting the test piece to be tested into a convolutional neural network model; the training set of the convolutional neural network model at least comprises a plurality of second abnormal pictures, the second abnormal pictures are synthesized by at least two first abnormal pictures, and the first abnormal pictures are user interfaces with compatibility abnormality acquired in advance; and determining whether the compatibility of the user interface is abnormal according to the output result of the convolutional neural network model. The problem of abnormal compatibility of a user interface which is unforeseen is identified through the trained convolutional neural network model, the identification accuracy is improved, and under the condition that the compatibility problem can not be predicted under the complex application scenes of different terminal models, operating systems and network conditions, the problem of abnormal compatibility can be accurately identified, and the complex compatibility test is realized.
Description
Technical Field
The embodiment of the application relates to the technical field of data service, in particular to a method and a device for testing the compatibility of a user interface.
Background
Before the application program of the computer or the mobile phone is released, a tester needs to test compatibility aiming at compatibility problem scenes of different machine types, different system platforms, different networks or different manufacturers and the like, and common compatibility problems comprise: application installation failure, application startup failure, application running failure, or user interface anomalies of the application, etc. When testing is performed for the compatibility problem of the user interface abnormality, the abnormal picture of the user interface is mainly identified in the following two ways. The first mode is to manually identify the abnormality of the downloaded picture to be tested; in the second mode, the picture to be tested is compared with the abnormal picture serving as the reference through a picture comparison algorithm, so that whether the picture to be tested is the abnormal picture or not is judged.
The inventors found that there are at least the following problems in the related art: firstly, the pictures are identified by using manpower, heavy and repeated work is brought to the testers, a large amount of manpower is wasted, and meanwhile, based on subjective feelings of the testers, the identification results are also quite different, so that the identification results are inaccurate. Secondly, the reference picture is required to be generated in advance by utilizing a picture comparison algorithm for identification, and the abnormal condition of which the reference picture is not prepared cannot be identified, so that the range of compatibility test is limited to a certain extent, and the test result is inaccurate.
Disclosure of Invention
The embodiment of the application aims to provide a method and a device for testing the compatibility of a user interface, which can identify some problems of unexpected user interface compatibility by a trained convolutional neural network model and improve the identification accuracy.
In order to solve the above technical problems, an embodiment of the present application provides a method for testing compatibility of a user interface, including: acquiring a user interface as a picture to be tested; inputting the test piece to be tested into a convolutional neural network model; the training set of the convolutional neural network model at least comprises a plurality of second abnormal pictures, the second abnormal pictures are synthesized by at least two first abnormal pictures, and the first abnormal pictures are user interfaces with compatibility abnormality acquired in advance; and determining whether the compatibility of the user interface is abnormal according to the output result of the convolutional neural network model.
The embodiment of the application also provides a compatibility testing device of the user interface, which comprises: the device comprises an acquisition module, an input module and a determination module; the acquisition module is used for acquiring a user interface as a picture to be tested; the input module is used for inputting the to-be-tested try sheet into the convolutional neural network model; the training set of the convolutional neural network model at least comprises a plurality of second abnormal pictures, the second abnormal pictures are synthesized by at least two first abnormal pictures, and the first abnormal pictures are user interfaces with compatibility abnormality acquired in advance; the determining module is used for determining whether the compatibility of the user interface is abnormal according to the output result of the convolutional neural network model.
The embodiment of the application also provides a server, which comprises: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of testing the compatibility of the user interface.
The embodiment of the application also provides a storage medium which stores a computer program, and the computer program realizes the compatibility testing method of the user interface when being executed by a processor.
Compared with the prior art, the embodiment of the application obtains the user interface as the picture to be tested when the compatibility test is carried out on the user interface. Inputting the acquired to-be-tested test sheet into a pre-trained convolutional neural network model, identifying a picture to be tested through the convolutional neural network model, and judging whether the picture to be tested has a compatibility problem according to a result output by the convolutional neural network model. The training set of the convolutional neural network model at least comprises a second abnormal picture, the second abnormal picture is not directly obtained from the compatible abnormal picture which is actually existed, and is not obtained by performing simple operations such as zooming, rotating and the like on the first abnormal picture which is directly obtained, but is synthesized through at least two first abnormal pictures which are actually obtained, the synthesized second abnormal picture has abnormal problems, but the characteristics contained in the second abnormal picture are not easily and directly thought of, so the problem of the compatibility abnormality of the user interface which is not foreseeable by the convolutional neural network model trained by the second abnormal picture can be identified, and the identification range and the identification accuracy are improved. Therefore, under the condition that the compatibility problem can not be predicted under the complex application scenes of different terminal models, operating systems and network conditions, the problem of abnormal compatibility can be accurately identified, and complex compatibility test is realized.
In addition, the second abnormal picture is synthesized as follows: determining the compatibility anomaly type of the second anomaly picture to be synthesized; acquiring first abnormal pictures of at least two compatibility abnormal types; inputting the acquired first abnormal picture into a pre-trained depth countermeasure convolutional neural network model, and synthesizing the input first abnormal picture through the depth countermeasure convolutional neural network model to generate a second abnormal picture. The second abnormal picture is synthesized through the depth countermeasure convolutional neural network model, so that the efficiency of synthesizing the second abnormal picture is improved, and the synthesized second abnormal picture can contain the characteristics related to the problem of compatibility abnormality through the trained depth countermeasure convolutional neural network model, so that the utilization rate of the generated second abnormal picture is improved.
In addition, after the second abnormal picture is generated, further comprising: matching the characteristics of the second abnormal picture with preset characteristics to obtain a matching value; and if the matching value reaches a preset threshold value, adding the second abnormal picture to the training set of the convolutional neural network model. By the aid of the method, the abnormal pictures used for training the model in the training set can carry the required characteristics, and the influence of useless abnormal pictures on the efficiency and accuracy of model training is avoided.
In addition, after the second abnormal picture is generated, further comprising: performing picture preprocessing on the second abnormal picture to generate a third abnormal picture; the picture preprocessing at least comprises any one of the following processing or a combination thereof: overturning treatment, scale change treatment or random buckling treatment; and adding the generated third abnormal picture to a training set of the convolutional neural network model. In this way, the number of the abnormal pictures in the training set can be further increased through a simple processing mode, and the convolutional neural network model can be further improved in recognition accuracy through training a large number of different abnormal pictures, so that the recognition result is more accurate.
In addition, after acquiring the user interface as the trial piece to be tested, the method further comprises: acquiring a terminal model and an operating system version of a terminal where a user interface is located; after determining whether the compatibility of the user interface is abnormal, further comprising: if the compatibility of the user interface is abnormal, the terminal model and the operating system version are recorded. By doing so, the terminal model and the operating system version number which are most likely to generate the compatibility problem of the user interface can be predicted according to the terminal model and the operating system version recorded after each compatibility test of the user interface, so that the terminal model and the operating system version number can be tested again, the compatibility problem of the user interface can be accurately determined by a tester, and the compatibility problem can be solved.
In addition, after determining whether the compatibility of the user interface is abnormal, further comprising: if the compatibility of the user interface is abnormal, adding the to-be-tested try sheet to the training set of the convolutional neural network model. In this way, the number of the abnormal pictures for training the convolutional neural network model can be increased through the abnormal pictures obtained in the compatibility test, and the recognition accuracy of the convolutional neural network model is improved, so that the accuracy of the compatibility test of the user interface is improved.
In addition, obtaining the user interface as the picture to be tested includes: screenshot processing is carried out on the user interface; taking the intercepted picture as a picture to be tested. In the compatibility test process of the user interface, the picture to be tested is automatically acquired, manual interception of the picture to be tested is not needed, and labor consumption is reduced.
Drawings
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.
FIG. 1 is a flowchart of a method for testing compatibility of a user interface in a first embodiment of the present application;
FIG. 2 is a flowchart of a method for compatibility testing of a user interface in a second embodiment of the present application;
FIG. 3 is a schematic diagram of a compatibility testing apparatus of a user interface in a third embodiment of the present application;
fig. 4 is a schematic diagram of a structure of a server according to a fourth embodiment of the present application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those of ordinary skill in the art that in various embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, the claimed technical solution of the present application can be realized without these technical details and various changes and modifications based on the following embodiments.
The following embodiments are divided for convenience of description, and should not be construed as limiting the specific implementation of the present application, and the embodiments can be mutually combined and referred to without contradiction.
A first embodiment of the present application relates to a compatibility testing method for a user interface, including: acquiring a user interface as a picture to be tested; inputting the test piece to be tested into a convolutional neural network model; the training set of the convolutional neural network model at least comprises a plurality of second abnormal pictures, the second abnormal pictures are synthesized by at least two first abnormal pictures, and the first abnormal pictures are user interfaces with compatibility abnormality acquired in advance; and determining whether the compatibility of the user interface to be tested is abnormal or not according to the output result of the convolutional neural network model, identifying some problems of human unforeseen user interface compatibility abnormality through the trained convolutional neural network model, and improving the identification accuracy. Implementation details of the compatibility testing method of the user interface of the present embodiment are specifically described below, and the following details are provided only for facilitating understanding, and are not necessary to implement the present embodiment.
The specific flow is shown in fig. 1, and the first embodiment relates to a method for testing compatibility of a user interface, which includes:
step 101, acquiring a user interface as a picture to be tested.
Specifically, when the compatibility of the application is tested, whether the compatibility of the application is abnormal or not can be judged through whether the user interface of the application is abnormal or not, and if the information displayed by the user interface is abnormal, the compatibility problem of the application is indicated. The abnormal condition of the user interface at least comprises any condition that, for example, network connection is abnormal in a popup window mode on the user interface, or characters or pictures on the user interface are not arranged according to a preset rule, the displayed characters or pictures are different from a specified size, or double images or blurring phenomena occur on the displayed characters or pictures, and the like.
When the compatibility test is carried out according to the user interface, the user interface is firstly required to be obtained as a picture to be tested, so that whether the picture has the abnormal condition or not can be judged according to the to-be-tested test piece. When the picture to be tested is obtained, the user interface can be intercepted in the process of testing the application performance, the intercepted user interface is used as the picture to be tested, and the picture to be tested which is intercepted in advance can be obtained for testing when the picture to be tested is obtained. The number of the obtained to-be-tested trying pieces can be one or a plurality, and when the number of the obtained to-be-tested trying pieces is a plurality of the obtained to-be-tested trying pieces, the obtained to-be-tested trying pieces are respectively different user interfaces, so that the test result is more comprehensive, and the test accuracy is improved.
In the following, a description will be given of acquiring an attempted chip to be tested by intercepting a user interface, and a series of preset operations, such as a clicking operation, a sliding operation, a selecting operation, a moving operation, etc., need to be performed when performing performance test on an application. After a preset operation is executed, intercepting the user interface after the operation, and taking the intercepted user interface as a picture to be tested. Or intercepting the current user interface in the test at certain time intervals according to a preset period to obtain the picture to be tested. When the pictures to be tested are intercepted in the mode, a series of preset operations set in the above mode are normally executed by the application, so that the intercepted pictures to be tested are ensured to be different user interfaces, and the test result is more comprehensive.
And 102, inputting the test piece to be tested into a convolutional neural network model.
Specifically, after the picture to be tested is acquired, the acquired trial slice to be tested is input into a pre-trained convolutional neural network model. The convolutional neural network model is trained by a plurality of second abnormal pictures, the second abnormal pictures are synthesized by at least two user interfaces (first abnormal pictures) with abnormal compatibility, and are not obtained through manually thinkable simple enlarging or shrinking operation of the actually obtained first abnormal pictures, so that the synthesized second abnormal pictures have obvious differences compared with the first abnormal pictures, and the differences are not easily thinkable, and therefore, the convolutional neural network model trained by the second abnormal pictures can identify the abnormal conditions of the user interfaces, which occur due to the unexpected compatibility abnormality. Under the complex current environment with different terminal models, operating systems and network conditions, the condition that the user interface is abnormal in the testing process is not limited to the abnormal condition, and a plurality of manually unforeseen user interface abnormal conditions exist.
When the first abnormal pictures are synthesized to obtain second abnormal pictures, at least two first abnormal pictures with the same type of abnormal problems can be subjected to characteristic value extraction, and the at least two first abnormal pictures are synthesized according to the extracted characteristic values, so that the synthesized second abnormal pictures also have the same type of abnormal problems as the first abnormal pictures.
And step 103, determining whether the compatibility of the user interface to be tested is abnormal according to the output result of the convolutional neural network model.
Specifically, after the to-be-tested test piece is input into the trained convolutional neural network model, the trained convolutional neural network model identifies and calculates the image content in the to-be-tested picture, and outputs an identification result, so that whether the to-be-tested test piece has an abnormal problem can be judged according to the identification result, and the type of the abnormal problem can be determined. For example, the trained convolutional neural network model can identify network anomaly problems, text overlapping problems, text shielding problems and the like, after the to-be-tested test piece is input into the trained convolutional neural network model, the trained convolutional neural network model identifies the characteristics of the to-be-tested picture, the identified characteristics of the to-be-tested picture are matched with the characteristics corresponding to the anomaly problems which can be identified by the convolutional identification network model, the probability of various anomaly problems in the to-be-tested picture is obtained according to the matching result, and the identified result is output, for example, the output result is: 1-3%,2-30% and 3-60%. The output result shows that the picture to be tested has 3% of probability of network abnormality, 30% of probability of text overlapping and 60% of probability of text shielding, and the picture to be tested is estimated to have larger probability of text shielding, so that the user interface to be tested is judged to have abnormality, and the compatibility of the tested application is abnormal.
In addition, when the abnormal problem exists in the to-be-tested trying sheet, the abnormal to-be-tested trying sheet is stored in the training set of the convolutional neural network model, so that the number of the training sets of the convolutional neural network model is increased, when the number of the training sets reaches the preset number or the preset training period is reached, the convolutional neural network model is trained again by using the expanded training sets, the recognition error of the convolutional neural network model is reduced, and the accuracy of the convolutional neural network model recognition is improved.
In practical application, a user or a tester can test the compatibility of a user interface of the terminal by using the cloud test platform, before testing, the user or the tester needs to select the terminal model and the operating system version of the compatibility test, select the tested application software, and finally trigger the automatic test execution. The cloud test platform automatically executes the test, namely automatically intercepts a user interface as a picture to be tested, inputs the test piece to be tested into the convolutional neural network model, recognizes the picture to be tested by the convolutional neural network model, and outputs a test result. And after the automatic test is finished, displaying the test result output by the convolutional neural network model to a user or a tester. The cloud test platform can record the terminal model and the operating system version with the compatibility abnormal problem after the test is finished, predicts the terminal model and the operating system version with the most possibility of the compatibility problem of the user interface according to the recorded terminal model and operating system version, and takes the predicted terminal model and operating system version as the recommended selection of the next compatibility automatic test, so that testers can test the terminal model and operating system version with the most possibility of the problem.
Compared with the prior art, the embodiment of the application obtains the user interface as the picture to be tested when the compatibility test is carried out on the user interface. Inputting the acquired to-be-tested test sheet into a pre-trained convolutional neural network model, identifying a picture to be tested through the convolutional neural network model, and judging whether the picture to be tested has a compatibility problem according to a result output by the convolutional neural network model. The training set of the convolutional neural network model at least comprises a second abnormal picture, the second abnormal picture is not directly obtained from the compatible abnormal picture which is actually existed, and is not obtained by performing simple operations such as zooming, rotating and the like on the first abnormal picture which is directly obtained, but is synthesized through at least two first abnormal pictures which are actually obtained, the synthesized second abnormal picture has abnormal problems, but the characteristics contained in the second abnormal picture are not easily and directly thought of, so the problem of the compatibility abnormality of the user interface which is not foreseeable by the convolutional neural network model trained by the second abnormal picture can be identified, and the identification range and the identification accuracy are improved. Therefore, under the condition that the compatibility problem can not be predicted under the complex application scenes of different terminal models, operating systems and network conditions, the problem of abnormal compatibility can be accurately identified, and complex compatibility test is realized.
In the second embodiment of the present application, a specific description is given of an acquisition process of a training set of a convolutional neural network model, a convolutional neural network is trained through an abnormal picture in the training set, a picture to be tested is identified through a trained convolutional neural network model, so that a compatibility test is implemented, the process of identifying the picture to be tested through the trained convolutional neural network model is the same as that described in the first embodiment, and steps of the part are not described again.
The process of obtaining the training set of the convolutional neural network model is shown in fig. 2, and includes:
step 201, determining a compatibility anomaly type of the second anomaly picture to be synthesized. Specifically, the compatibility anomaly type of the second anomaly picture corresponds to an anomaly problem occurring in the identification user interface, for example, the compatibility anomaly type of the network anomaly corresponding to an anomaly problem occurring in the user interface in which the network connection anomaly is displayed in a popup window; the abnormal problem of character ghost appears in the user interface corresponds to the compatibility abnormal type of character overlapping; and the compatibility abnormal type that the abnormal problem corresponding to the abnormal problem of the text which is not displayed at the preset position is shifted appears on the user interface. And synthesizing the first abnormal pictures of the same type, wherein the second abnormal pictures of the type are obtained with high probability, so that before the second abnormal pictures are synthesized, the types of the second abnormal pictures need to be confirmed preferentially, and the efficiency of synthesizing the second abnormal pictures can be improved.
Step 202, obtaining at least two first abnormal pictures of the compatibility abnormal type.
Specifically, there are multiple ways of obtaining the first abnormal picture, for example, the user may upload the first abnormal picture to the cloud test platform, and when uploading the first abnormal picture to the cloud test platform, the administrator may further need to label the abnormal problem existing in the uploaded abnormal picture, so as to classify the first abnormal picture uploaded by the user. And when the second abnormal picture is required to be synthesized, acquiring a first abnormal picture of a corresponding type uploaded by a user from a database of the cloud test platform, and synthesizing the first abnormal picture in the database. For another example, a user interface with an abnormal problem in the tested user interface may be stored as a first abnormal picture, and the type of the abnormality of the abnormal problem in the stored first abnormal picture may be marked correspondingly. And when the second abnormal picture is required to be synthesized, acquiring the stored first abnormal picture with the corresponding abnormal type for synthesis.
Step 203, inputting the obtained first abnormal picture into a pre-trained depth countermeasure convolutional neural network model, and synthesizing the input first abnormal picture through the depth countermeasure convolutional neural network model to generate a second abnormal picture.
Specifically, after the first abnormal pictures are acquired, inputting all the acquired first abnormal pictures into a pre-trained depth countermeasure convolutional neural network model, wherein the depth countermeasure convolutional neural network model comprises two models, one is a generating model and the other is a judging model, the generating model is used for learning the characteristics of the abnormal types according to the input first abnormal pictures of the same abnormal type and generating a new second abnormal picture according to the learned characteristics, the generated second abnormal picture possibly has certain noise influence on the identification of the second abnormal picture, and if the noise is large, the generated second abnormal picture does not have the characteristics of the abnormal problems of the types. The judging model is used for distinguishing the generated second abnormal picture, judging whether the generated second abnormal picture has the characteristics of the abnormal problem of the type or not by identifying the characteristic value of the generated second abnormal picture, if the generated second abnormal picture is judged not to have the characteristics of the abnormal problem of the type, informing the generating model to optimize, reducing noise existing in the generated second abnormal picture by the generating model until the percentage of the generated second abnormal picture which is judged to be normal by the judging model reaches a preset threshold value, and describing that the depth countermeasure convolutional neural network model comprising the generating model and the judging model can accurately synthesize the second abnormal picture meeting the requirement.
And 204, matching the characteristics of the second abnormal picture with the preset characteristics to obtain a matching value.
And 205, if the matching value reaches the preset threshold, adding the second abnormal picture to the training set of the convolutional neural network model.
Specifically, if there is a certain type of abnormal problem in the user interface, there is a certain specific feature in the user interface, for example, there may be a technical feature of text overlapping in the user interface of text overlapping problem, there may be a popup technical feature in the user interface of network abnormality, etc. And matching the characteristics of the second abnormal picture with the preset characteristics corresponding to the abnormal problems of each type, if the matching degree is high, indicating that the characteristics of the abnormal problems exist in the second abnormal picture, and adding the corresponding abnormal problems in the second abnormal picture as training pictures of the convolutional neural network model into a training set. By the aid of the method, under the condition that the accuracy of the depth countermeasure convolutional neural network model is not high, the generated second abnormal picture can be verified, and the fact that the accuracy of the convolutional neural network model is influenced by the invalid second abnormal picture is avoided. After the accuracy of the depth countermeasure convolutional neural network model is improved, this verification step may be omitted, thereby reducing the synthesis time of the second abnormal picture and improving the synthesis efficiency of the second abnormal picture.
In addition, in order to further expand the number of abnormal pictures in the training set of the convolutional neural network model, the first abnormal picture or the second abnormal picture meeting the requirement obtained above may be processed, for example, the image is processed by horizontal or vertical overturn, the preset size is randomly buckled, the size of the image is changed, and the image is rotated. And adding the images with abnormal problems in the processed images to a training set of the convolutional neural network model. The number of outliers to expand the training set may also be expanded by color dithering, or using Principal Component Analysis (PCA) and a supervised data model.
After the training set of a sufficient number of convolutional neural network models is obtained in the above manner, the convolutional neural network models are trained by using the abnormal pictures in the training set, after the accuracy of the trained convolutional neural network models reaches the standard, the trained convolutional neural network models are applied to the line, and the judgment of the compatibility abnormality of the user interface is performed in the manner mentioned in the first embodiment, so that the recognition efficiency and recognition accuracy of the compatibility abnormality problem are improved.
The above steps of the methods are divided, for clarity of description, and may be combined into one step or split into multiple steps when implemented, so long as they include the same logic relationship, and they are all within the protection scope of this patent; it is within the scope of this patent to add insignificant modifications to the algorithm or flow or introduce insignificant designs, but not to alter the core design of its algorithm and flow.
A third embodiment of the present application relates to a compatibility testing apparatus for a user interface, as shown in fig. 3, including: an acquisition module 31, an input module 32, a determination module 33; the acquisition module 31 is used for acquiring a user interface as a picture to be tested; the input module 32 is used for inputting the test piece to be tested into the convolutional neural network model; the training set of the convolutional neural network model at least comprises a plurality of second abnormal pictures, the second abnormal pictures are synthesized by at least two first abnormal pictures, and the first abnormal pictures are user interfaces with compatibility abnormality acquired in advance; the determining module 33 is configured to determine whether the compatibility of the user interface is abnormal according to the output result of the convolutional neural network model.
It is to be noted that this embodiment is an embodiment of the apparatus corresponding to the first embodiment, and this embodiment can be implemented in cooperation with the first embodiment. The related technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, they are not described here again. Accordingly, the related-art details mentioned in the present embodiment can also be applied to the first embodiment.
In addition, the compatibility testing device of the user interface further comprises: a synthesis module; the synthesis module is used for determining the compatibility abnormal type of the second abnormal picture to be synthesized; acquiring first abnormal pictures of at least two compatibility abnormal types; inputting the acquired first abnormal picture into a pre-trained depth countermeasure convolutional neural network model, and synthesizing the input first abnormal picture through the depth countermeasure convolutional neural network model to generate a second abnormal picture.
In addition, the device also comprises a matching module; the matching module is used for matching the characteristics of the second abnormal picture with preset characteristics to obtain a matching value; and if the matching value reaches a preset threshold value, adding the second abnormal picture to the training set of the convolutional neural network model.
In addition, the device also comprises a preprocessing module; the preprocessing module is used for preprocessing the second abnormal picture to generate a third abnormal picture; the picture preprocessing at least comprises any one of the following processing or a combination thereof: overturning treatment, scale change treatment or random buckling treatment; and adding the generated third abnormal picture to a training set of the convolutional neural network model.
In addition, the device also comprises a recording module; the recording module is used for recording the terminal model and the operating system version corresponding to the user interface when the compatibility of the user interface is abnormal.
It should be noted that each module in this embodiment is a logic module, and in practical application, one logic unit may be one physical unit, or may be a part of one physical unit, or may be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present application, units that are not so close to solving the technical problem presented by the present application are not introduced in the present embodiment, but this does not indicate that other units are not present in the present embodiment.
A fourth embodiment of the application relates to a server, as shown in fig. 4, comprising at least one processor 401; and a memory 402 communicatively coupled to the at least one processor 401; the memory 402 stores instructions executable by the at least one processor 401, and the instructions are executed by the at least one processor 401, so that the at least one processor 401 can execute the compatibility testing method of the user interface.
Where the memory 402 and the processor 401 are connected by a bus, the bus may comprise any number of interconnected buses and bridges, the buses connecting the various circuits of the one or more processors 401 and the memory 402 together. The bus may also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or may be a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor 401.
The processor 401 is responsible for managing the bus and general processing and may also provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And memory 402 may be used to store data used by processor 401 in performing operations.
A fifth embodiment of the present application relates to a computer-readable storage medium storing a computer program. The computer program implements the above-described method embodiments when executed by a processor.
That is, it will be understood by those skilled in the art that all or part of the steps in implementing the methods of the embodiments described above may be implemented by a program stored in a storage medium, where the program includes several instructions for causing a device (which may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps in the methods of the embodiments of the application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples of carrying out the application and that various changes in form and details may be made therein without departing from the spirit and scope of the application.
Claims (10)
1. A method for testing compatibility of a user interface, comprising:
acquiring a user interface as a picture to be tested;
inputting the to-be-tested test piece into a convolutional neural network model;
the training set of the convolutional neural network model at least comprises a plurality of second abnormal pictures, wherein the second abnormal pictures are synthesized by at least two first abnormal pictures, and the first abnormal pictures are user interfaces with abnormal compatibility acquired in advance;
determining whether the compatibility of the user interface is abnormal according to the output result of the convolutional neural network model;
when the first abnormal picture is synthesized to obtain the second abnormal picture, the method comprises the following steps: extracting characteristic values of at least two first abnormal pictures with the same type of abnormal problems, and synthesizing the at least two first abnormal pictures according to the extracted characteristic values, so that the synthesized second abnormal picture also has the same type of abnormal problems as the first abnormal picture.
2. The method for testing the compatibility of the user interface according to claim 1, wherein the second abnormal picture is synthesized as follows:
determining the compatibility anomaly type of the second anomaly picture to be synthesized;
acquiring the first abnormal pictures of at least two compatibility abnormal types;
inputting the acquired first abnormal picture into a pre-trained depth countermeasure convolutional neural network model, and synthesizing the input first abnormal picture through the depth countermeasure convolutional neural network model to generate a second abnormal picture.
3. The method for testing the compatibility of the user interface according to claim 2, further comprising, after the generating the second abnormal picture:
matching the characteristics of the second abnormal picture with preset characteristics to obtain a matching value;
and if the matching value reaches a preset threshold value, adding the second abnormal picture to the training set of the convolutional neural network model.
4. The method for testing the compatibility of the user interface according to claim 2, further comprising, after the generating the second abnormal picture:
performing picture preprocessing on the second abnormal picture to generate a third abnormal picture;
wherein, the picture preprocessing at least comprises any one of the following processing or the combination thereof: overturning treatment, scale change treatment or random buckling treatment;
and adding the generated third abnormal picture to a training set of the convolutional neural network model.
5. The method for testing the compatibility of a user interface according to claim 1, further comprising, after said acquiring the user interface as an attempted chip to be tested:
acquiring a terminal model and an operating system version of a terminal where the user interface is located;
after said determining whether the compatibility of the user interface is abnormal, further comprising:
and if the compatibility of the user interface is abnormal, recording the terminal model and the operating system version.
6. The method of claim 1, further comprising, after said determining whether the compatibility of the user interface is abnormal:
and if the compatibility of the user interface is abnormal, adding the to-be-tested try sheet to a training set of the convolutional neural network model.
7. The method for testing the compatibility of the user interface according to claim 1, wherein the step of acquiring the user interface as the picture to be tested comprises the steps of:
screenshot processing is carried out on the user interface;
taking the intercepted picture as a picture to be tested.
8. A compatibility testing apparatus for a user interface, comprising: the device comprises an acquisition module, an input module and a determination module;
the acquisition module is used for acquiring a user interface as a picture to be tested;
the input module is used for inputting the to-be-tested attempt sheet into a convolutional neural network model; the training set of the convolutional neural network model at least comprises a plurality of second abnormal pictures, wherein the second abnormal pictures are synthesized by at least two first abnormal pictures, and the first abnormal pictures are user interfaces with abnormal compatibility acquired in advance; when the first abnormal picture is synthesized to obtain the second abnormal picture, the method comprises the following steps: extracting characteristic values of at least two first abnormal pictures with the same type of abnormal problems, and synthesizing the at least two first abnormal pictures according to the extracted characteristic values, so that the synthesized second abnormal picture also has the same type of abnormal problems as the first abnormal picture;
and the determining module is used for determining whether the compatibility of the user interface is abnormal according to the output result of the convolutional neural network model.
9. A server, comprising:
at least one processor; the method comprises the steps of,
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of compatibility testing of a user interface as claimed in any one of claims 1 to 7.
10. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the method of compatibility testing of a user interface according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911345387.3A CN111198815B (en) | 2019-12-24 | 2019-12-24 | Compatibility testing method and device for user interface |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911345387.3A CN111198815B (en) | 2019-12-24 | 2019-12-24 | Compatibility testing method and device for user interface |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111198815A CN111198815A (en) | 2020-05-26 |
| CN111198815B true CN111198815B (en) | 2023-11-03 |
Family
ID=70747078
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911345387.3A Active CN111198815B (en) | 2019-12-24 | 2019-12-24 | Compatibility testing method and device for user interface |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111198815B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111915601B (en) * | 2020-08-12 | 2024-07-02 | 中国工商银行股份有限公司 | Abnormality test method, device and system for intelligent terminal |
| CN113448868B (en) * | 2021-07-16 | 2022-11-01 | 网易（杭州）网络有限公司 | Game software compatibility testing method, device, equipment and medium |
| CN113506291B (en) * | 2021-07-29 | 2024-03-26 | 上海幻电信息科技有限公司 | Compatibility testing method and device |
| CN113900865B (en) * | 2021-08-16 | 2023-07-11 | 广东电力通信科技有限公司 | Intelligent power grid equipment automatic test method, system and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025174A (en) * | 2017-04-06 | 2017-08-08 | 网易（杭州）网络有限公司 | For the method for the user interface abnormality test of equipment, device and readable storage media |
| CN108132887A (en) * | 2018-01-10 | 2018-06-08 | 百度在线网络技术（北京）有限公司 | User interface method of calibration, device, software testing system, terminal and medium |
| CN108229485A (en) * | 2018-02-08 | 2018-06-29 | 百度在线网络技术（北京）有限公司 | For testing the method and apparatus of user interface |
| CN108304318A (en) * | 2018-01-02 | 2018-07-20 | 深圳壹账通智能科技有限公司 | The test method and terminal device of equipment compatibility |
| CN110413529A (en) * | 2019-07-31 | 2019-11-05 | 中国工商银行股份有限公司 | Applied to the test method of electronic equipment, device, calculate equipment and medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190188559A1 (en) * | 2017-12-15 | 2019-06-20 | International Business Machines Corporation | System, method and recording medium for applying deep learning to mobile application testing |
| US10599951B2 (en) * | 2018-03-28 | 2020-03-24 | Kla-Tencor Corp. | Training a neural network for defect detection in low resolution images |
-
2019
- 2019-12-24 CN CN201911345387.3A patent/CN111198815B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025174A (en) * | 2017-04-06 | 2017-08-08 | 网易（杭州）网络有限公司 | For the method for the user interface abnormality test of equipment, device and readable storage media |
| CN108304318A (en) * | 2018-01-02 | 2018-07-20 | 深圳壹账通智能科技有限公司 | The test method and terminal device of equipment compatibility |
| CN108132887A (en) * | 2018-01-10 | 2018-06-08 | 百度在线网络技术（北京）有限公司 | User interface method of calibration, device, software testing system, terminal and medium |
| CN108229485A (en) * | 2018-02-08 | 2018-06-29 | 百度在线网络技术（北京）有限公司 | For testing the method and apparatus of user interface |
| CN110413529A (en) * | 2019-07-31 | 2019-11-05 | 中国工商银行股份有限公司 | Applied to the test method of electronic equipment, device, calculate equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111198815A (en) | 2020-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111198815B (en) | Compatibility testing method and device for user interface | |
| CN110443274B (en) | Abnormality detection method, abnormality detection device, computer device, and storage medium | |
| CN108021806B (en) | Malicious installation package identification method and device | |
| EP2806374A1 (en) | Method and system for automatic selection of one or more image processing algorithm | |
| CN111783812B (en) | Forbidden image recognition method, forbidden image recognition device and computer readable storage medium | |
| CN109685805B (en) | Image segmentation method and device | |
| CN112860676B (en) | Data cleaning method applied to big data mining and business analysis and cloud server | |
| CN111225299A (en) | ONU (optical network Unit) fault identification and repair method and device | |
| CN110427998A (en) | Model training, object detection method and device, electronic equipment, storage medium | |
| CN109905292A (en) | A kind of terminal device recognition methods, system and storage medium | |
| CN113254248B (en) | Fault diagnosis method, system and computing device | |
| CN112966687B (en) | Image segmentation model training method and device and communication equipment | |
| CN114297735A (en) | Data processing method and related device | |
| CN109324967A (en) | The method, apparatus and terminal device of application program pop-up components testing | |
| CN110716778A (en) | Application compatibility testing method, device and system | |
| CN114153730A (en) | Method for generating buried point configuration page and related equipment | |
| CN115730205A (en) | Method and device for configuring decision device and related equipment | |
| CN112286785A (en) | User interface abnormity detection method and device | |
| CN114816876B (en) | Automatic test system for server Redfish interface specifications | |
| CN117349189B (en) | APP new version testing method, equipment and medium | |
| CN113886745B (en) | Page picture testing method and device and electronic equipment | |
| CN111144973B (en) | Question ranking method and computer-readable storage medium | |
| CN117808816B (en) | Image anomaly detection method and device and electronic equipment | |
| CN116030501B (en) | Method and device for extracting bird detection data | |
| CN112329854B (en) | Method and device for checking picture cutting of advertisement material |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |