CN111124903A - Element coordinate automatic testing method based on visual system and storage medium - Google Patents
Element coordinate automatic testing method based on visual system and storage medium Download PDFInfo
- Publication number
- CN111124903A CN111124903A CN201911294618.2A CN201911294618A CN111124903A CN 111124903 A CN111124903 A CN 111124903A CN 201911294618 A CN201911294618 A CN 201911294618A CN 111124903 A CN111124903 A CN 111124903A
- Authority
- CN
- China
- Prior art keywords
- tested
- coordinates
- predefined
- interface picture
- elements
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3692—Test management for test results analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- General Engineering & Computer Science (AREA)
- Geometry (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to the technical field of image recognition, and provides an element coordinate automatic testing method based on a visual system and a storage medium. The invention automatically acquires element information by using a vision system, realizes the comprehensive coverage of element test and the automatic acquisition of the actual coordinates of the elements, quickly and intuitively obtains the test data of the interface to be tested and the element coordinate deviation by using material coverage, realizes the full-automatic test of the interface elements, greatly improves the image acquisition efficiency, reduces the waste of manpower and material resources and improves the interface test efficiency.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to an element coordinate automatic testing method based on a visual system and a storage medium.
Background
The interface test (UI test for short) is used for testing whether the layout of the functional modules of the user interface is reasonable, whether the overall style is consistent, whether the placement positions of all the controls accord with the use habits of customers, and also testing the convenience of interface operation, simplicity and comprehensiveness of navigation, the usability of page elements, whether characters in the interface are correct, whether names are unified, whether pages are attractive, whether combinations of the characters and pictures are perfect, and the like.
With the development and progress of modern science and technology, the requirements of users on interface display are continuously improved, even elements (including word sizes, pictures, backgrounds, keys and the like) required to be used in each area in the interface display and the positions, the heights and the widths of pixel points of the elements are defined, and partial interface materials (pictures and background pictures) are customized. The corresponding testing work must also be developed for the materials and pixel information of these elements. However, since the naked eye cannot accurately identify the pixel points, the test can be performed only by using an external tool for manual test. In addition, although some tools in the prior art can identify the position of the picture in the interface, the coordinate position of a certain element in the interface can still be obtained only manually. However, the number of elements included in an interface is often dozens to dozens of elements, the number of system interfaces to be tested is hundreds, if all the elements are manually tested, a large amount of manpower is inevitably needed, and the test points covered by the tool are not comprehensive, so that the high-efficiency interface test cannot be accurately and effectively realized.
Disclosure of Invention
The invention provides an element coordinate automatic testing method based on a vision system and a storage medium, and solves the technical problems of time consumption, labor consumption, low automation degree, high cost and poor testing efficiency of the conventional interface element testing method.
In order to solve the technical problems, the invention provides an element coordinate automatic testing method based on a vision system, which comprises the following steps:
s1, loading an interface picture to be tested;
s2, judging whether the interface picture to be tested is predefined, if so, entering the next step, and if not, customizing the interface picture to be tested;
s3, reading elements contained in the interface picture to be tested, and covering corresponding predefined materials in the interface picture to be tested;
and S4, reading the interface picture to be tested after covering the material, and outputting a test report according to the reading result.
Before the step S1, the method further includes the steps of:
s01, manufacturing an element starting frame and an element stopping frame according to the design standard of the interface picture to be tested;
s02, according to the UI visual system standard, elements in the interface picture to be tested are made into different types of element materials;
s03, making a Text material according to the Text type of the elements on the interface picture to be tested and the predefined coordinates, width and height;
s04, storing the starting border and the ending border, the element materials and the Text materials into a predefined database.
In step S3, the step of determining whether the interface picture to be tested of the current type has been predefined includes:
matching the name of the interface picture to be tested in the predefined database, wherein if the matching is successful, the predefinition is completed, and if the matching is failed, the predefinition is not completed;
the self-defining of the interface picture to be tested is as follows: the process proceeds to step S01.
The step S3 further includes: and reading the coordinates of all elements on the interface picture to be tested, and calculating the position deviation by combining the corresponding predefined coordinates.
Preferably, the reading of the coordinates of all elements on the interface picture to be tested is completed through a vision system.
The element types comprise texts, keys, icon pictures and background pictures; the element materials comprise a title material, a list material and a bullet frame material; the Text type comprises a word size; the predefined materials comprise a starting border and a stopping border, a title material, a list material, a popup material and a Text material.
In the step S3, the step of overlaying the corresponding predefined material into the interface picture to be tested specifically includes: and according to the predefined coordinates, covering the starting border and the ending border, the title material, the list material, the popup material and the Text material which correspond to the predefined coordinates onto the interface picture to be tested.
Preferably, in the step S4, the reading of the picture of the interface to be tested after covering the predefined material is completed through a vision system.
The test result comprises the contact ratio of the predefined materials and corresponding elements on the interface picture to be tested; the test report also comprises the position deviation of all elements in the interface picture to be tested and a material coverage effect picture.
The invention also provides a computer storage medium, which stores a computer program, wherein the computer program is executed by a processor to realize the automatic testing method for the element coordinate based on the visual system.
The invention provides an automatic element coordinate testing method based on a visual system, which is characterized in that element predefining is carried out according to the setting requirement of an interface or a client template, a predefined database is established, when testing is carried out, according to predefined coordinates, a starting frame and a stopping frame, a title material, a list material, a popup frame material and a Text material which correspond to the predefined coordinates are covered on an interface picture to be tested, the interface picture is automatically acquired by the visual system, a material covering effect picture and the actual coordinates of all elements are obtained, and then the actual coordinates and the predefined coordinates are calculated to obtain position deviation. The invention automatically acquires element information by using a vision system, realizes the comprehensive coverage of element test and the automatic acquisition of the actual coordinates of the elements, quickly and intuitively obtains the test data of the interface to be tested and the element coordinate deviation by using material coverage, realizes the full-automatic test of the interface elements, greatly improves the image acquisition efficiency, reduces the waste of manpower and material resources and improves the interface test efficiency.
Drawings
FIG. 1 is a system flow chart of a method for automated testing of coordinates of elements based on a vision system according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating operation of the method of FIG. 1 prior to step S1 according to an embodiment of the present invention;
FIG. 3 is a flowchart of the operation of FIG. 1 provided by an embodiment of the present invention;
Detailed Description
The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, which are given solely for the purpose of illustration and are not to be construed as limitations of the invention, including the drawings which are incorporated herein by reference and for illustration only and are not to be construed as limitations of the invention, since many variations thereof are possible without departing from the spirit and scope of the invention.
As shown in fig. 1, the method for automatically testing element coordinates based on a vision system according to an embodiment of the present invention includes the following steps:
s1, loading an interface picture to be tested;
s2, judging whether the interface picture to be tested is predefined, if so, entering the next step, and if not, customizing the interface picture to be tested;
s3, reading elements contained in the interface picture to be tested, and covering corresponding predefined materials in the interface picture to be tested;
and S4, reading the interface picture to be tested after covering the material, and outputting a test report according to the reading result.
Referring to fig. 2, before the step S1, the method further includes the steps of:
s01, manufacturing an element starting frame and an element stopping frame according to the design standard of the interface picture to be tested;
s02, according to the UI visual system standard, elements in the interface picture to be tested are made into different types of element materials;
s03, making a Text material according to the Text type of the elements on the interface picture to be tested and the predefined coordinates, width and height;
s04, storing the starting border and the ending border, the element materials and the Text materials into a predefined database.
In step S2, the step of determining whether the interface picture to be tested of the current type has been predefined includes: matching the name of the interface picture to be tested in the predefined database, wherein if the matching is successful, the predefinition is completed, and if the matching is failed, the predefinition is not completed; the predefined database includes all data of the predefined picture including, but not limited to, name, type, elements, materials, and coordinates of the interface.
The self-defining of the interface picture to be tested is as follows: the process proceeds to step S01.
The step S3 further includes: and reading the coordinates of all elements on the interface picture to be tested, and calculating the position deviation by combining the corresponding predefined coordinates.
Preferably, the reading of the coordinates of all elements on the interface picture to be tested is completed through a vision system.
The element types comprise texts, keys, icon pictures and background pictures; the element materials comprise a title material, a list material and a bullet frame material; the Text type comprises a word size; the predefined materials comprise a starting border and a stopping border, a title material, a list material, a popup material and a Text material.
In the step S3, the step of overlaying the corresponding predefined material into the interface picture to be tested specifically includes: and according to the predefined coordinates, covering the starting border and the ending border, the title material, the list material, the popup material and the Text material which correspond to the predefined coordinates onto the interface picture to be tested.
In step S4, the reading of the to-be-tested interface picture after the material is covered is completed through a vision system.
The test result comprises the contact ratio of the predefined materials and corresponding elements on the interface picture to be tested; the test report also comprises the position deviation of all elements in the interface picture to be tested and a material coverage effect picture.
The vision system includes, but is not limited to, laser scanners, linear and area CCD cameras, TV cameras, and digital cameras.
Referring to fig. 3, the specific workflow of the element coordinate automated testing method provided by the present invention is as follows:
loading an interface picture to be tested, matching the name of the interface picture to be tested with a predefined database, and if the matching is successful, representing that the interface type of the interface picture to be tested is predefined; if the matching fails, the predefinition is not completed, and a user-defined program is entered. The completed predefinition comprises directly input test standards and later established test standards according to a custom program.
The self-defining program comprises the following steps: manufacturing an element starting frame and an element stopping frame according to the design standard of an interface picture to be tested, manufacturing the elements in the interface picture to be tested into different types of titles, lists, pop frames and other materials according to the design standard of a UI visual system, and manufacturing a Text material according to the Text type of the elements and predefined coordinates, width and height; and finally, storing the starting border and the ending border, the element materials and the Text materials into a predefined database, and uniformly classifying the starting border and the ending border, the element materials and the Text materials into predefined materials.
And acquiring an image of the interface picture to be tested through a visual system, acquiring element icons and coordinates in the interface picture to be tested by utilizing image processing, acquiring predefined materials and predefined coordinates corresponding to the elements, and covering the starting frame and the ending frame, the title material, the list material, the pop frame material and the Text material which correspond to the predefined coordinates onto the interface picture to be tested according to the predefined coordinates.
And reading the coverage effect picture of the interface picture to be tested after covering the material by using a visual system again, calculating position deviation according to the element coordinate and the predefined coordinate, further calculating the contact ratio of the element and the predefined material, integrating the coverage effect picture, the element position deviation and the element contact ratio into a test report, and outputting the test report.
Corresponding to the above method for automatically testing element coordinates based on a vision system, an embodiment of the present invention further provides a computer storage medium, where a computer program is stored on the computer storage medium, and when the computer program is executed by a processor, the method for automatically testing element coordinates based on a vision system is implemented.
The computer-readable storage medium includes, but is not limited to, any type of disk including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks, ROMs (Read-Only memories), RAMs (Random access memories), EPROMs (EraSable Programmable Read-Only memories), EEPROMs (Electrically EraSable Programmable Read-Only memories), flash memories, magnetic cards, or optical cards. That is, a storage device includes any medium that stores or transmits information in a form readable by a device (e.g., a computer, a cellular phone), and may be a read-only memory, a magnetic or optical disk, or the like.
The embodiment of the invention provides an automatic element coordinate testing method based on a visual system, which comprises the steps of predefining elements according to the set requirements of an interface or a client template, establishing a predefined database, covering a starting frame and a stopping frame, a title material, a list material, a popup frame material and a Text material which correspond to the predefined coordinates onto an interface picture to be tested according to the predefined coordinates during testing, automatically acquiring the interface picture by using the visual system, obtaining a material covering effect picture and the actual coordinates of all the elements, and then calculating the material covering effect picture and the predefined coordinates to obtain position deviation. The invention automatically acquires element information by using a vision system, realizes the comprehensive coverage of element test and the automatic acquisition of the actual coordinates of the elements, quickly and intuitively obtains the test data of the interface to be tested and the element coordinate deviation by using material coverage, realizes the full-automatic test of the interface elements, greatly improves the image acquisition efficiency, reduces the waste of manpower and material resources and improves the interface test efficiency.
The embodiment of the invention also provides a computer readable storage medium, which realizes the test process and result, the data storage of the computer application program and the predefined database and ensures the accuracy of the interface element test.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (10)
1. An element coordinate automatic testing method based on a visual system is characterized by comprising the following steps:
s1, loading an interface picture to be tested;
s2, judging whether the interface picture to be tested is predefined, if so, entering the next step, and if not, customizing the interface picture to be tested;
s3, reading elements contained in the interface picture to be tested, and covering corresponding predefined materials in the interface picture to be tested;
and S4, reading the interface picture to be tested after covering the material, and outputting a test report according to the reading result.
2. The automated test method for coordinates of elements based on visual system as claimed in claim 1, wherein before said step S1, further comprising the steps of:
s01, manufacturing an element starting frame and an element stopping frame according to the design standard of the interface picture to be tested;
s02, according to the UI visual system standard, elements in the interface picture to be tested are made into different types of element materials;
s03, making a Text material according to the Text type of the elements on the interface picture to be tested and the predefined coordinates, width and height;
s04, storing the starting border and the ending border, the element materials and the Text materials into a predefined database.
3. The automated test method for coordinates of elements based on visual system as claimed in claim 2, wherein, in said step S2,
the method for judging whether the interface picture to be tested of the current type is predefined is specifically as follows: matching the name of the interface picture to be tested in the predefined database, wherein if the matching is successful, the predefinition is completed, and if the matching is failed, the predefinition is not completed;
the self-defining of the interface picture to be tested is as follows: the process proceeds to step S01.
4. The automated test method for coordinates of elements based on visual system as claimed in claim 2, wherein said step S3 further comprises: and reading the coordinates of all elements on the interface picture to be tested, and calculating the position deviation by combining the corresponding predefined coordinates.
5. The automated test method for coordinates of elements based on visual system as claimed in claim 4, wherein in said step S3: and finishing reading the coordinates of all elements on the interface picture to be tested through a vision system.
6. The automated vision system-based test method for coordinates of elements as claimed in claim 2, wherein: the element types comprise texts, keys, icon pictures and background pictures; the element materials comprise a title material, a list material and a bullet frame material; the Text type comprises a word size; the predefined materials comprise a starting border and a stopping border, a title material, a list material, a popup material and a Text material.
7. The method for automated testing of element coordinates based on visual system of claim 4, wherein in the step S3, the overlaying of the corresponding predefined material into the interface picture to be tested is specifically: and according to the predefined coordinates, covering the starting border and the ending border, the title material, the list material, the popup material and the Text material which correspond to the predefined coordinates onto the interface picture to be tested.
8. The automated vision system-based test method for coordinates of elements as claimed in claim 1, wherein: in step S4, the reading of the interface picture to be tested after covering the predefined material is completed by a vision system.
9. A method for automated testing of coordinates of elements based on a vision system as claimed in claim 1, characterized by: the test result comprises the contact ratio of the predefined material and the corresponding element on the interface picture to be tested; the test report also comprises position deviation of all elements in the interface picture to be tested and a predefined material coverage effect picture.
10. A computer storage medium, characterized in that: the computer storage medium has stored thereon a computer program which, when executed by a processor, implements the automated test method for coordinates of an element based on a vision system of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911294618.2A CN111124903B (en) | 2019-12-16 | 2019-12-16 | Element coordinate automatic test method based on vision system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911294618.2A CN111124903B (en) | 2019-12-16 | 2019-12-16 | Element coordinate automatic test method based on vision system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111124903A true CN111124903A (en) | 2020-05-08 |
| CN111124903B CN111124903B (en) | 2023-04-28 |
Family
ID=70499270
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911294618.2A Active CN111124903B (en) | 2019-12-16 | 2019-12-16 | Element coordinate automatic test method based on vision system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111124903B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112732559A (en) * | 2020-12-30 | 2021-04-30 | 广东粤云工业互联网创新科技有限公司 | Picture comparison method, automatic dial testing method and system and storage medium |
| CN116401178A (en) * | 2023-06-09 | 2023-07-07 | 成都融见软件科技有限公司 | UI component problem point positioning method, electronic device and medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050257203A1 (en) * | 2004-05-11 | 2005-11-17 | National Instruments Corporation | Visually indicating problems found during programmatic analysis of a graphical program |
| CN104516633A (en) * | 2014-11-19 | 2015-04-15 | 微梦创科网络科技（中国）有限公司 | User interface element management method and device |
| CN109408367A (en) * | 2018-09-03 | 2019-03-01 | 深圳壹账通智能科技有限公司 | A kind of method and terminal of the control element identifying interactive interface |
| CN109815156A (en) * | 2019-02-28 | 2019-05-28 | 北京百度网讯科技有限公司 | Displaying test method, device, equipment and the storage medium of visual element in the page |
| CN109992518A (en) * | 2019-04-10 | 2019-07-09 | 禄鹏 | Detection method, device, electronic equipment and the storage medium at the interface UI |
| US20190303269A1 (en) * | 2018-03-28 | 2019-10-03 | Layout.io Ltd | Methods and systems for testing visual aspects of a web page |
| CN110554957A (en) * | 2019-07-31 | 2019-12-10 | 北京三快在线科技有限公司 | method and device for testing user interface, electronic equipment and readable storage medium |
-
2019
- 2019-12-16 CN CN201911294618.2A patent/CN111124903B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050257203A1 (en) * | 2004-05-11 | 2005-11-17 | National Instruments Corporation | Visually indicating problems found during programmatic analysis of a graphical program |
| CN104516633A (en) * | 2014-11-19 | 2015-04-15 | 微梦创科网络科技（中国）有限公司 | User interface element management method and device |
| US20190303269A1 (en) * | 2018-03-28 | 2019-10-03 | Layout.io Ltd | Methods and systems for testing visual aspects of a web page |
| CN109408367A (en) * | 2018-09-03 | 2019-03-01 | 深圳壹账通智能科技有限公司 | A kind of method and terminal of the control element identifying interactive interface |
| CN109815156A (en) * | 2019-02-28 | 2019-05-28 | 北京百度网讯科技有限公司 | Displaying test method, device, equipment and the storage medium of visual element in the page |
| CN109992518A (en) * | 2019-04-10 | 2019-07-09 | 禄鹏 | Detection method, device, electronic equipment and the storage medium at the interface UI |
| CN110554957A (en) * | 2019-07-31 | 2019-12-10 | 北京三快在线科技有限公司 | method and device for testing user interface, electronic equipment and readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 韦庆杰: "一种Web应用跨浏览器兼容性测试方法" * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112732559A (en) * | 2020-12-30 | 2021-04-30 | 广东粤云工业互联网创新科技有限公司 | Picture comparison method, automatic dial testing method and system and storage medium |
| CN116401178A (en) * | 2023-06-09 | 2023-07-07 | 成都融见软件科技有限公司 | UI component problem point positioning method, electronic device and medium |
| CN116401178B (en) * | 2023-06-09 | 2023-08-11 | 成都融见软件科技有限公司 | UI component problem point positioning method, electronic device and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111124903B (en) | 2023-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6411731B1 (en) | Template-based image recognition and extraction | |
| CN111930381B (en) | Personalized page configuration method supporting SaaS multi-tenant system | |
| CN107832041B (en) | Method, device and equipment for making thematic page and readable storage medium | |
| JP6826293B2 (en) | Information information system and its processing method and program | |
| KR20140038381A (en) | Systems and methods for testing content of mobile communication devices | |
| CN108805519B (en) | Electronic generation method and device for paper schedule and electronic schedule generation method | |
| CN111124903B (en) | Element coordinate automatic test method based on vision system and storage medium | |
| CN110765015A (en) | Method for testing application to be tested and electronic equipment | |
| CN116402074A (en) | Code scanning equipment and working mode identification method, device and equipment thereof | |
| CN114049631A (en) | Data labeling method and device, computer equipment and storage medium | |
| CN117152302A (en) | Method, device, equipment and storage medium for generating display image of target object | |
| CN114627482A (en) | Method and system for realizing table digital processing based on image processing and character recognition | |
| CN107329791A (en) | Method, device and the computer-readable recording medium of application are installed in scanning | |
| CN111651368B (en) | User graphical interface testing method and computer readable storage medium | |
| CN110457219B (en) | Method and device for selecting time when test script is recorded | |
| CN105631412A (en) | Data recording method and terminal | |
| CN104407767A (en) | Method for regulating user interface | |
| CN111061414A (en) | Skin replacement method and device, electronic equipment and readable storage medium | |
| CN109783389A (en) | Element control automatic test approach, device and electronic equipment | |
| KR20040017376A (en) | Method and mobile phone for reading name card using camera for cellular phone | |
| JP3890096B2 (en) | Image editing system | |
| JPH09311944A (en) | Image editing device and its method and document editing system | |
| CN107885180B (en) | Test apparatus and test method | |
| CN111435313A (en) | Method and device for changing skin of software | |
| CN113836037B (en) | Interface interaction testing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |