CN112015634A - Page structure information generation method and device and electronic equipment - Google Patents
Page structure information generation method and device and electronic equipment Download PDFInfo
- Publication number
- CN112015634A CN112015634A CN201910468228.6A CN201910468228A CN112015634A CN 112015634 A CN112015634 A CN 112015634A CN 201910468228 A CN201910468228 A CN 201910468228A CN 112015634 A CN112015634 A CN 112015634A
- Authority
- CN
- China
- Prior art keywords
- information
- screenshot
- page
- coordinate information
- generating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012163 sequencing technique Methods 0.000 claims description 11
- 238000003860 storage Methods 0.000 claims description 8
- 238000000638 solvent extraction Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 abstract description 23
- 238000001545 Page's trend test Methods 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 9
- 238000012545 processing Methods 0.000 description 9
- 238000001514 detection method Methods 0.000 description 7
- 238000011161 development Methods 0.000 description 6
- 230000018109 developmental process Effects 0.000 description 6
- 238000012015 optical character recognition Methods 0.000 description 6
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005520 cutting process Methods 0.000 description 3
- 238000003709 image segmentation Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000000275 quality assurance Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000013522 software testing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention provides a page structure information generation method, a page structure information generation device and electronic equipment, wherein the method comprises the following steps: acquiring a screenshot of a target area of a page; acquiring type information of each element in the screenshot, and acquiring coordinate information of each element in the screenshot; and generating the structural information of the target area according to the type information and the coordinate information of each element in the screenshot. In the invention, the structure information of the page area corresponding to the screenshot can be generated by acquiring the screenshot of the page and acquiring the type information and the coordinate information of each element in the screenshot. The generation process of the page structure information is simple, easy to implement and universal. Therefore, the page structure information generated by the method can provide a test basis for page test, and the page test cost is reduced.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a page structure information generation method and device and electronic equipment.
Background
With the development of mobile internet technology, the internet information brings convenience to the life of people and improves the working efficiency of people. Pages (also called "web pages") are basic elements of the internet and are used to carry internet information. If a page has a defect (bug), the experience of a User is affected, and especially, the User Interface (UI) directly contacting with the User has a larger adverse effect due to the bug. At present, the page bug is mainly discovered through manual testing, but the manual testing is high in cost and low in efficiency. Or, in a page development link, in order to test whether a page has bugs, a tester often needs to add a debug (debug) backdoor in the page development process, and the method has the problems of high development cost and high code risk. Therefore, the problem that the cost of page testing is high exists in the prior art.
Disclosure of Invention
The embodiment of the invention provides a page structure information generation method and device and electronic equipment, which are used for providing structured information for page testing so as to solve the problem of higher page testing cost in the prior art.
In order to solve the technical problem, the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a method for generating page structure information, including:
acquiring a screenshot of a target area of a page;
acquiring type information of each element in the screenshot, and acquiring coordinate information of each element in the screenshot;
and generating the structural information of the target area according to the type information and the coordinate information of each element in the screenshot.
In a second aspect, an embodiment of the present invention provides a page structure information generating apparatus, including:
the page screenshot obtaining module is used for obtaining a screenshot of a target area of a page;
the element information acquisition module is used for acquiring the type information of each element in the screenshot and acquiring the coordinate information of each element in the screenshot;
and the structural information generating module is used for generating structural information of the target area according to the type information and the coordinate information of each element in the screenshot.
In a third aspect, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, the method for generating page structure information in the first aspect of the embodiment of the present invention is implemented.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps in the page structure information generating method in the first aspect of the embodiment of the present invention.
In the embodiment of the invention, the screenshot of the page is obtained, and the type information and the coordinate information of each element in the screenshot are obtained, so that the structure information of the page area corresponding to the screenshot can be generated. The process of acquiring the screenshot of the page is simple and easy to realize, and the acquisition of the element type information and the coordinate information can be realized by a mature image processing technology. Therefore, the generation process of the page structure information is simple, easy to implement and universal. Therefore, the page structure information generated by the method can provide a test basis for page test, and the page test cost is reduced.
Drawings
Fig. 1 is a schematic flowchart of a method for generating page structure information according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of layout of elements in a target area of a page according to an embodiment of the present invention;
FIG. 3 is a diagram illustrating the layout of elements in a target area of another page according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating the layout of elements in a target area of another page according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating the layout of elements in a target area of another page according to an embodiment of the present invention;
FIG. 6 is a structural class diagram of the page corresponding to FIG. 5;
fig. 7 is a schematic structural diagram of a page structure information generating apparatus according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of another page structure information generating apparatus according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, a page structure information generating method, which is applicable to a page structure information generating apparatus, includes the steps of:
step 101: and acquiring a screenshot of a target area of the page.
The page may be a user interface of the software product, or may be other common pages. The target area may be an entire page area, that is, an entire area of the page, or a partial page area, that is, a partial area of the page. The target area can be determined based on test requirements, if page testing needs to be carried out on the whole page area, screenshot can be carried out on the whole page area, and if page testing needs to be carried out on only part of the page area, screenshot can be carried out on the page area to be tested.
In this step, the page structure information generating apparatus may obtain the screenshot of the target area of the page in two ways. Firstly, a user manually captures a picture and inputs the picture into a page structure information generating device so that the page structure information generating device can obtain the picture; and secondly, automatically carrying out screenshot on the target area of the page by the page structure information generating device so that the screenshot is obtained by the page structure information generating device.
No matter which mode is adopted, the process of obtaining the page screenshot is simple and easy to realize.
Step 102: and acquiring the type information of each element in the screenshot, and acquiring the coordinate information of each element in the screenshot.
In this step, after the screenshot is obtained, the page structure information generation device may obtain the type information and the coordinate information of each element in the screenshot, respectively. Specifically, the acquisition of the type information and the coordinate information of each element in the screenshot can be realized by a mature image processing technology.
The elements generally include types such as pictures, texts, controls and the like, and the types of the elements in the screenshot can include at least one of the pictures, the texts and the controls. That is, at least one of a picture class element, a text class element, and a control class element may be included in the screenshot.
The type information of the above-mentioned element refers to information for characterizing the type of the element, for example, the element may be characterized as a picture class element by "# picture #", the element may be characterized as a text class element by "# text #", and the element may be characterized as a control class element by "# control #".
The coordinate information of the element refers to information for representing coordinates of the element, and the coordinate information of the element may include at least one of start coordinate information of an area where the element is located, end coordinate information of the area where the element is located, height information of the area where the element is located, and width information of the area where the element is located. Further, the start coordinate information of the area where the element is located includes the X-direction start coordinate information and the Y-direction start coordinate information of the area where the element is located.
Generally, the coordinate information of the element includes start coordinate information of an area where the element is located, height information of the area where the element is located, and width information of the area where the element is located, or the coordinate information of the element includes end coordinate information of the area where the element is located, height information of the area where the element is located, and width information of the area where the element is located.
It should be noted that, in order to determine the start coordinate information or the end coordinate information of the element, a coordinate reference point or a coordinate origin needs to be set, and generally, a point at the top left corner of the target area may be taken as the coordinate reference point or the coordinate origin, a point at the top left corner of the target area may be taken as a starting point, a line extending rightward is taken as a positive X-axis, and a line extending downward is taken as a positive Y-axis.
Step 103: and generating the structural information of the target area according to the type information and the coordinate information of each element in the screenshot.
In this step, the page structure information generating device may generate the structure information of the target area according to the type information and the coordinate information of each element in the screenshot. The structure information of the target area includes type information and coordinate information of each element.
It should be noted that the structure information of the page may be referred to as page structure tree information or page tree structure information. A page structure tree or page tree is understood to be a tree structure consisting of several nodes in a page. Any element including the screenshot itself can be regarded as a node of a page, and each node is inherited from a node object containing element coordinate information and type information. The screenshot can be used as a root node of the page, each parallel element in the screenshot can be used as a primary child node of the page, and elements contained in each element in the screenshot can be used as secondary child nodes of the page.
After the structure information of the target area of the page is generated, the structure information can be used for testing whether the bug exists in the target area of the page. For example, bugs that are more common to pages are layout errors and element deletions, e.g., element relative distance errors, element occlusions, and the like. After the structure information of the target area is generated, whether the target area has layout errors or element missing can be detected according to the type information and the coordinate information of each element contained in the structure information.
As shown in fig. 2, in order to verify whether the layout of the target area 21 in the page 20 is correct, a screenshot of the target area 21 may be obtained first. Secondly, the type information and the coordinate information of each element in the screenshot of the target area 21 are obtained, wherein the target area 21 comprises a character element 2a, a picture element 2b, a picture element 2c, a picture element 2d and a control element 2 e. And thirdly, generating the structure information of the target area 21 according to the type information and the coordinate information of each element in the screenshot, wherein the structure information comprises the type information and the coordinate information of each element in the target area 21. From the structural information of the target area 21, it is understood that the layout of the elements is reasonable, and there is no bug of layout in the target area 21 on the page 20.
As shown in fig. 3, in order to verify whether the layout of the target area 31 in the page 30 is correct, a screenshot of the target area 31 may be obtained first. Secondly, the type information and the coordinate information of each element in the screenshot of the target area 31 are obtained, wherein the target area 31 comprises a text element 3a, a picture element 3b, a picture element 3c and a control element 3 d. And thirdly, generating the structure information of the target area 31 according to the type information and the coordinate information of each element in the screenshot, wherein the structure information comprises the type information and the coordinate information of each element in the target area 31. From the structure information of the target area 31, it is known that there is an error in the relative distance between the picture class element 3b and the picture class element 3c, there is a lack of the picture class element between the picture class element 3b and the picture class element 3c, and there is a bug of a layout in the target area 21 in the page 20.
In the embodiment of the invention, the structural information of the page can be generated by acquiring the screenshot of the page and acquiring the type information and the coordinate information of each element in the screenshot by adopting a related image processing technology, and the whole process is simple, easy to implement and universal. The page structure information generated by the method can provide a test basis for page testing, and the page testing cost is reduced.
In the prior art, a new page is generally created by constructing page structure information by related designers, namely, a process from the page structure information to the page. The page structure information constructed by the relevant designers can not necessarily be acquired for page testing, and even if the page structure information can be acquired, the page structure information can not necessarily meet the requirements of page testing.
The embodiment of the invention realizes the reverse process from the page to the page structure information, and when the page needs to be tested, relevant testers can execute the page structure information generation method through the page structure information generation device at any time to generate the structure information of any page without being limited by whether the original page structure information for creating the page can be acquired or not. Moreover, the information included in the page structure information generated from the page can also focus on the information requirement of the tester, and the original page structure information used for creating the page is not needed, so that the page structure information generated from the page better meets the test requirement.
The following describes an application of the embodiment of the present invention in conjunction with a scenario H5(Hyper Text Markup Language 5).
More and more front-end automation is applied to software project quality assurance work, and a known verification method is to perform element existence verification and element feature (value) verification through a DOM (Document Object Model) tree provided by an automation framework. However, as the mobile App H5 is popularized in a componentization manner, for example, the H5 applet attached to the native App, the known automation framework cannot provide a DOM tree conveniently, so that automation work cannot be performed in the scene, and the software testing efficiency is further improved.
In the existing H5 scene, the front-end automation development needs to rely on a host native to open a debug interface and transmit an H5 page tree, and the scheme has high testing and development cost. Moreover, the image segmentation technology comprises a clustering method, a histogram method, an edge detection method, a region growing method and the like, and the image segmentation technology is used for identifying the control of the image.
In the embodiment of the invention, the reverse process from the page to the page structure information is realized, and the testability requirement of the automatic test under specific scenes such as H5 and the like can be met.
The following describes a process of acquiring the type information and coordinate information of each element in the screenshot based on image processing techniques such as image segmentation, noise detection, optical character recognition, and object detection.
Optionally, after obtaining the screenshot of the target area of the page, the method further includes:
dividing the screenshot into a plurality of regions, wherein the plurality of regions comprise each element of the screenshot;
obtaining type information of each element in the screenshot, including:
and identifying the areas to acquire the type information of the elements contained in each area.
The screenshot is divided into a plurality of regions, which may specifically include:
firstly, carrying out graying processing on a screenshot; secondly, the screen shot after the graying processing is respectively scanned line by line and column by column, pixel rows which are inconsistent with the background color are identified as transverse cutting lines, and pixel columns which are inconsistent with the background color are identified as longitudinal cutting lines. To this end, each transverse cut line and longitudinal cut line divides the screenshot into a number of independent regions, each independent region corresponding to an independent element, as shown in fig. 2 to 3.
After the screenshot is divided into a plurality of areas, semantic recognition technologies such as noise detection, optical character recognition and object detection can be performed on each area to acquire the type information of elements contained in each area.
As mentioned above, the elements generally include types such as pictures, texts, controls, and the like, and the following describes the identification manners of these three elements one by one.
First, picture element identification. The method comprises the steps of carrying out noise detection on each region of a screenshot based on an image technology, wherein compared with characters and general icons, the image region is wide in color distribution and large in noise number, so that the image region can be used as a judgment standard of an image, and the region with the noise number larger than a threshold value is judged as a region where an image element is located.
And secondly, recognizing the character elements. Based on an OCR (Optical Character Recognition) technology, all Character areas in the screenshot can be obtained, the image cutting area and the OCR recognized Character area are intersected, and the area with the occupation ratio larger than a threshold value is determined as the area where the Character element is located. Meanwhile, in order to further enhance the accuracy of character element identification, the characteristics of the adjacent region are added, and if the characteristics of the adjacent region of the character region are similar, the region where the character element is located can be determined. For adjacent text areas, if the sizes and colors of the texts in the two areas are the same, and the distance is smaller than a certain threshold value compared with the size of the texts, the two areas are aggregated into one area, so that the integrity of text elements is improved. For example, title recognition and paragraph recognition, since the previous screenshot segmentation and OCR only obtain a line of text regions independent of each other, aggregation needs to be performed according to features to obtain a complete title region or paragraph region.
And thirdly, identifying control elements. For some common rich semantic controls in the page, such as buttons of returning, closing, canceling, downloading and the like, the area proportion is smaller than that of an input picture, so that non-text and picture areas with the area proportion smaller than a threshold value are selected, and then the types of the controls are identified by using an object detection technology; besides these types of relatively independent controls, there are controls that partially float above the picture, and controls that contain text inside. For such controls, full-image recognition needs to be performed by using an object detection technology to detect the control types contained in the elements.
After the screenshot is divided into several regions, the coordinate information of each region may be calculated to obtain the coordinate information of each region.
The process realizes the acquisition of the type information and the coordinate information of each element in the screenshot, so that the structural information of the target area of the page can be generated through the type information and the coordinate information of each element.
Optionally, after obtaining coordinate information of each element in the screenshot, the method further includes:
determining the position relation information of each element in the screenshot according to the coordinate information of each element in the screenshot;
generating the structural information of the target area according to the type information and the coordinate information of each element in the screenshot, wherein the generating comprises the following steps:
and generating the structural information of the target area according to the type information, the coordinate information and the position relation information of each element in the screenshot.
In the screenshot, a mutual position relationship exists between the elements, and the position relationship between the elements may include at least one of a parallel relationship, a cross relationship and an inclusion relationship. For example, in the screenshot shown in fig. 4, the intersection exists in the region where the element 4a and the element 4b are located, but the element 4a is not completely within the region of the element 4b, and the positional relationship between the element 4a and the element 4b is a cross relationship. For example, in the screenshot shown in fig. 5, there is no intersection in the areas where the element 5a, the element 5b, the element 5c, the element 5d, and the element 5e are located, and the positional relationship among the element 5a, the element 5b, the element 5c, the element 5d, and the element 5e is a parallel relationship; the element 5g is completely contained by the element 5f, and the positional relationship between the element 5f and the element 5g is a containing relationship.
The position relation among the elements in the screenshot can also comprise a neighbor relation, the position relation is calculated according to the area where the current element is located, and the elements which are nearest to the area where the current element is located in the vertical and horizontal directions are respectively an upper neighbor, a lower neighbor, a left neighbor and a right neighbor of the current element. When a plurality of elements with the same distance exist, the elements are all neighbors of the current element. The positional relationship between the elements may include at least one of an upper neighbor relationship, a lower neighbor relationship, a left neighbor relationship, a right neighbor relationship, a cross relationship, and an inclusion relationship. For example, in the screenshot shown in fig. 5, the upper neighbor of element 5c is element 5b, the lower neighbor of element 5c is element 5d, the right neighbor of element 5c is element 5f, and there is no left neighbor of element 5 c.
After the position relation information of each element in the screenshot is determined, the structure information of the target area can be generated according to the type information, the coordinate information and the position relation information of each element in the screenshot. The structure information of the target area may include type information, coordinate information, and position relationship information of each element.
In the embodiment, the page structure information comprises the position relation information of each element, so that the page structure information can more intuitively display the position relation of each element, and the page testing efficiency can be improved.
Optionally, after obtaining coordinate information of each element in the screenshot, the method further includes:
determining the ordering information of each element in the screenshot according to the coordinate information of each element in the screenshot;
generating the structural information of the target area according to the type information and the coordinate information of each element in the screenshot, wherein the generating comprises the following steps:
and generating the structural information of the target area according to the type information, the coordinate information and the sequencing information of each element in the screenshot.
After obtaining the coordinate information of each element in the screenshot, the elements may be sorted according to the coordinate information of each element from top to bottom and from left to right, for example, the elements may be numbered by a number. For example, as shown in fig. 5, element 5a may be numbered "1", element 5b may be numbered "2", element 5c may be numbered "3", element 5f may be numbered "4", element 5g may be numbered "5", element 5d may be numbered "6", and element 5e may be numbered "7".
After determining the sorting information of each element in the screenshot, the structure information of the target area can be generated according to the type information, the coordinate information and the sorting information of each element in the screenshot. The structure information of the target area may include type information, coordinate information, and sorting information of each element.
In this embodiment, the page structure information includes the sorting information of each element, so that each element has a corresponding number or sequence number, and each element can be better identified or distinguished, and therefore, the page structure information can more intuitively display the relevant information of each element, and the efficiency of page testing can be improved.
Optionally, after obtaining coordinate information of each element in the screenshot, the method further includes:
determining position relation information and sequencing information of each element in the screenshot according to the coordinate information of each element in the screenshot;
generating the structural information of the target area according to the type information and the coordinate information of each element in the screenshot, wherein the generating comprises the following steps:
and generating the structural information of the target area according to the type information, the coordinate information, the position relation information and the sequencing information of each element in the screenshot.
In this embodiment, the structure information of the target area may include type information, coordinate information, positional relationship information, and sorting information of each element.
In this embodiment, when generating the structural information of the target area, the coordinate information and the type information of all elements in the screenshot may be sequentially obtained, the position relationship of each element and the sequence number of each element may be determined according to the coordinate information of all elements in the screenshot, and then the structural information of the page may be generated according to the type information, the coordinate information, the position relationship information, and the sorting information of all elements in the screenshot. Fig. 6 shows a structure class diagram of a page containing the above information.
For the page shown in fig. 5, the page structure information thereof may be as follows.
In summary, in the embodiment of the present invention, by obtaining the screenshot of the page and obtaining the type information and the coordinate information of each element in the screenshot, the structure information of the page area corresponding to the screenshot may be generated. The process of acquiring the screenshot of the page is simple and easy to realize, and the acquisition of the element type information and the coordinate information can be realized by a mature image processing technology. Therefore, the generation process of the page structure information is simple, easy to implement and universal. Therefore, the page structure information generated by the method can provide a test basis for page test, and the page test cost is reduced.
As shown in fig. 7, an embodiment of the present invention provides a page structure information generating apparatus 800, including:
a page screenshot obtaining module 801, configured to obtain a screenshot of a target area of a page;
an element information obtaining module 802, configured to obtain type information of each element in the screenshot, and obtain coordinate information of each element in the screenshot;
a structure information generating module 803, configured to generate structure information of the target area according to the type information and the coordinate information of each element in the screenshot.
Optionally, as shown in fig. 8, the page structure information generating apparatus 800 further includes:
a graph partitioning module 804, configured to partition the screenshot into a plurality of regions, where the plurality of regions include elements of the screenshot;
the element information obtaining module 802 is specifically configured to:
identifying the areas to acquire the type information of the elements contained in each area;
coordinate information of each region is acquired.
Optionally, the page structure information generating apparatus 800 further includes:
the position relation determining module is used for determining the position relation information of each element in the screenshot according to the coordinate information of each element in the screenshot;
the structure information generating module 803 is specifically configured to:
and generating the structural information of the target area according to the type information, the coordinate information and the position relation information of each element in the screenshot.
Optionally, the page structure information generating apparatus 800 further includes:
the sequencing information determining module is used for determining the sequencing information of each element in the screenshot according to the coordinate information of each element in the screenshot;
the structure information generating module 803 is specifically configured to:
and generating the structural information of the target area according to the type information, the coordinate information and the sequencing information of each element in the screenshot.
Optionally, the type of each element in the screenshot includes at least one of a picture, a text, and a control.
Optionally, the coordinate information includes at least one of start coordinate information of an area where the element is located, end coordinate information of the area where the element is located, height information of the area where the element is located, and width information of the area where the element is located.
Optionally, the position relationship includes at least one of a parallel relationship, a cross relationship, and an inclusion relationship; or,
the position relation comprises at least one of an upper neighbor relation, a lower neighbor relation, a left neighbor relation, a right neighbor relation, an inclusion relation and a cross relation.
It should be noted that any implementation manner in the method embodiment may be implemented by the page structure information generating apparatus 800 in this embodiment, and the same beneficial effects are achieved, and for avoiding repetition, details are not described here again.
As shown in fig. 9, an electronic device 900 according to an embodiment of the present invention is also provided, where the electronic device 900 includes a memory 901, a processor 902, and a computer program stored in the memory 901 and executable on the processor 902; when the processor 902 executes the computer program, the following steps are implemented:
in fig. 9, the bus architecture may include any number of interconnected buses and bridges, with one or more processors, represented by processor 902, and various circuits, represented by memory 901, linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. The bus interface provides an interface. The processor 902 is responsible for managing the bus architecture and general processing, and the memory 901 may store data used by the processor 902 in executing instructions. In the embodiment of the present invention, the electronic device includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted mobile terminal, a wearable device, and the like.
Optionally, when the processor 902 executes the computer program, the following is also implemented:
acquiring a screenshot of a target area of a page;
acquiring type information of each element in the screenshot, and acquiring coordinate information of each element in the screenshot;
and generating the structural information of the target area according to the type information and the coordinate information of each element in the screenshot.
Optionally, when the processor 902 executes the computer program, the following is also implemented:
dividing the screenshot into a plurality of regions, wherein the plurality of regions comprise each element of the screenshot;
identifying the areas to acquire the type information of the elements contained in each area;
coordinate information of each region is acquired.
Optionally, when the processor 902 executes the computer program, the following is also implemented:
determining the position relation information of each element in the screenshot according to the coordinate information of each element in the screenshot;
and generating the structural information of the target area according to the type information, the coordinate information and the position relation information of each element in the screenshot.
Optionally, when the processor 902 executes the computer program, the following is also implemented:
determining the ordering information of each element in the screenshot according to the coordinate information of each element in the screenshot;
and generating the structural information of the target area according to the type information, the coordinate information and the sequencing information of each element in the screenshot.
Optionally, the type of each element in the screenshot includes at least one of a picture, a text, and a control.
Optionally, the coordinate information includes at least one of start coordinate information of an area where the element is located, end coordinate information of the area where the element is located, height information of the area where the element is located, and width information of the area where the element is located.
Optionally, the position relationship includes at least one of a parallel relationship, a cross relationship, and an inclusion relationship; or,
the position relation comprises at least one of an upper neighbor relation, a lower neighbor relation, a left neighbor relation, a right neighbor relation, an inclusion relation and a cross relation.
It should be noted that any implementation manner in the method embodiment may be implemented by the electronic device 900 in this embodiment, and the same beneficial effects are achieved, and are not described herein again.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements the processes of the transmission method embodiment, and can achieve the same technical effects, and in order to avoid repetition, details are not repeated here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only one type of logical function division, and other division manners may be available in actual implementation, for example, a plurality of units or components may be combined or integrated into another device, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be physically included alone, or two or more units may be integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) to execute some steps of the transceiving method according to various embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (16)
1. A method for generating page structure information is characterized by comprising the following steps:
acquiring a screenshot of a target area of a page;
acquiring type information of each element in the screenshot, and acquiring coordinate information of each element in the screenshot;
and generating the structural information of the target area according to the type information and the coordinate information of each element in the screenshot.
2. The method of claim 1, after obtaining a screenshot of a target area of a page, the method further comprising:
dividing the screenshot into a plurality of regions, wherein the plurality of regions comprise each element of the screenshot;
obtaining type information of each element in the screenshot, including:
identifying the areas to acquire the type information of the elements contained in each area;
acquiring coordinate information of each element in the screenshot, wherein the coordinate information comprises the following steps:
coordinate information of each region is acquired.
3. The method of claim 1 or 2, wherein after obtaining coordinate information of each element in the screenshot, the method further comprises:
determining the position relation information of each element in the screenshot according to the coordinate information of each element in the screenshot;
generating the structural information of the target area according to the type information and the coordinate information of each element in the screenshot, wherein the generating comprises the following steps:
and generating the structural information of the target area according to the type information, the coordinate information and the position relation information of each element in the screenshot.
4. The method of claim 1 or 2, wherein after obtaining coordinate information of each element in the screenshot, the method further comprises:
determining the ordering information of each element in the screenshot according to the coordinate information of each element in the screenshot;
generating the structural information of the target area according to the type information and the coordinate information of each element in the screenshot, wherein the generating comprises the following steps:
and generating the structural information of the target area according to the type information, the coordinate information and the sequencing information of each element in the screenshot.
5. The method of claim 1 or 2, wherein the type of each element in the screenshot comprises at least one of a picture, a text, and a control.
6. The method according to claim 1 or 2, wherein the coordinate information includes at least one of start coordinate information of an area where the element is located, end coordinate information of an area where the element is located, height information of an area where the element is located, and width information of an area where the element is located.
7. The method of claim 3, wherein the positional relationship comprises at least one of a juxtaposition relationship, a crossing relationship, and an inclusion relationship; or,
the position relation comprises at least one of an upper neighbor relation, a lower neighbor relation, a left neighbor relation, a right neighbor relation, an inclusion relation and a cross relation.
8. A page structure information generating apparatus, characterized by comprising:
the page screenshot obtaining module is used for obtaining a screenshot of a target area of a page;
the element information acquisition module is used for acquiring the type information of each element in the screenshot and acquiring the coordinate information of each element in the screenshot;
and the structural information generating module is used for generating structural information of the target area according to the type information and the coordinate information of each element in the screenshot.
9. The apparatus of claim 8, further comprising:
a graph partitioning module for partitioning the screenshot into a plurality of regions, the plurality of regions including elements of the screenshot;
the element information acquisition module is specifically configured to:
identifying the areas to acquire the type information of the elements contained in each area;
coordinate information of each region is acquired.
10. The apparatus of claim 8 or 9, further comprising:
the position relation determining module is used for determining the position relation information of each element in the screenshot according to the coordinate information of each element in the screenshot;
the structural information generation module is specifically configured to:
and generating the structural information of the target area according to the type information, the coordinate information and the position relation information of each element in the screenshot.
11. The apparatus of claim 8 or 9, further comprising:
the sequencing information determining module is used for determining the sequencing information of each element in the screenshot according to the coordinate information of each element in the screenshot;
the structural information generation module is specifically configured to:
and generating the structural information of the target area according to the type information, the coordinate information and the sequencing information of each element in the screenshot.
12. The apparatus of claim 8 or 9, wherein the type of each element in the screenshot comprises at least one of a picture, a text, and a control.
13. The apparatus according to claim 8 or 9, wherein the coordinate information includes at least one of start coordinate information of an area where the element is located, end coordinate information of an area where the element is located, height information of an area where the element is located, and width information of an area where the element is located.
14. The apparatus of claim 10, wherein the positional relationship comprises at least one of a juxtaposition relationship, a crossing relationship, and an inclusion relationship; or,
the position relation comprises at least one of an upper neighbor relation, a lower neighbor relation, a left neighbor relation, a right neighbor relation, an inclusion relation and a cross relation.
15. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor; characterized in that the processor, when executing the computer program, implements the page structure information generating method according to any one of claims 1 to 7.
16. A computer-readable storage medium on which a computer program is stored, the program, when executed by a processor, implementing the steps in the page structure information generating method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910468228.6A CN112015634A (en) | 2019-05-31 | 2019-05-31 | Page structure information generation method and device and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910468228.6A CN112015634A (en) | 2019-05-31 | 2019-05-31 | Page structure information generation method and device and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112015634A true CN112015634A (en) | 2020-12-01 |
Family
ID=73500587
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910468228.6A Pending CN112015634A (en) | 2019-05-31 | 2019-05-31 | Page structure information generation method and device and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112015634A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112633270A (en) * | 2020-12-29 | 2021-04-09 | 平安银行股份有限公司 | Skeleton screen generation method, device and equipment based on picture recognition and storage medium |
| CN112650682A (en) * | 2020-12-28 | 2021-04-13 | 广州品唯软件有限公司 | Page test processing method and device, electronic equipment and readable storage medium |
| CN113342684A (en) * | 2021-06-29 | 2021-09-03 | 北京搜狗科技发展有限公司 | Webpage testing method, device and equipment |
| CN114116441A (en) * | 2020-12-30 | 2022-03-01 | 京东数科海益信息科技有限公司 | UI (user interface) testing method and device, electronic equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104850415A (en) * | 2014-02-13 | 2015-08-19 | 腾讯科技（深圳）有限公司 | Method and apparatus for loading pages |
| CN104932889A (en) * | 2015-06-15 | 2015-09-23 | 北京奇虎科技有限公司 | Page visual generation method and device |
| CN108496150A (en) * | 2016-10-18 | 2018-09-04 | 华为技术有限公司 | A kind of method and terminal of screenshot capture and reading |
| CN109086203A (en) * | 2018-07-20 | 2018-12-25 | 百度在线网络技术（北京）有限公司 | The detection method and device of the page |
| CN109145231A (en) * | 2017-06-27 | 2019-01-04 | 阿里巴巴集团控股有限公司 | A kind of generation method, device and the electronic equipment of interface element identifier |
| CN109189682A (en) * | 2018-08-27 | 2019-01-11 | 广州云测信息技术有限公司 | A kind of script method for recording and device |
| CN109299415A (en) * | 2018-09-30 | 2019-02-01 | 百度在线网络技术（北京）有限公司 | H5 page method of calibration and device |
| CN109815156A (en) * | 2019-02-28 | 2019-05-28 | 北京百度网讯科技有限公司 | Displaying test method, device, equipment and the storage medium of visual element in the page |
-
2019
- 2019-05-31 CN CN201910468228.6A patent/CN112015634A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104850415A (en) * | 2014-02-13 | 2015-08-19 | 腾讯科技（深圳）有限公司 | Method and apparatus for loading pages |
| CN104932889A (en) * | 2015-06-15 | 2015-09-23 | 北京奇虎科技有限公司 | Page visual generation method and device |
| CN108496150A (en) * | 2016-10-18 | 2018-09-04 | 华为技术有限公司 | A kind of method and terminal of screenshot capture and reading |
| CN109145231A (en) * | 2017-06-27 | 2019-01-04 | 阿里巴巴集团控股有限公司 | A kind of generation method, device and the electronic equipment of interface element identifier |
| CN109086203A (en) * | 2018-07-20 | 2018-12-25 | 百度在线网络技术（北京）有限公司 | The detection method and device of the page |
| CN109189682A (en) * | 2018-08-27 | 2019-01-11 | 广州云测信息技术有限公司 | A kind of script method for recording and device |
| CN109299415A (en) * | 2018-09-30 | 2019-02-01 | 百度在线网络技术（北京）有限公司 | H5 page method of calibration and device |
| CN109815156A (en) * | 2019-02-28 | 2019-05-28 | 北京百度网讯科技有限公司 | Displaying test method, device, equipment and the storage medium of visual element in the page |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112650682A (en) * | 2020-12-28 | 2021-04-13 | 广州品唯软件有限公司 | Page test processing method and device, electronic equipment and readable storage medium |
| CN112650682B (en) * | 2020-12-28 | 2024-06-11 | 广州品唯软件有限公司 | Page test processing method and device, electronic equipment and readable storage medium |
| CN112633270A (en) * | 2020-12-29 | 2021-04-09 | 平安银行股份有限公司 | Skeleton screen generation method, device and equipment based on picture recognition and storage medium |
| CN112633270B (en) * | 2020-12-29 | 2024-07-05 | 平安银行股份有限公司 | Skeleton screen generation method, device and equipment based on picture identification and storage medium |
| CN114116441A (en) * | 2020-12-30 | 2022-03-01 | 京东数科海益信息科技有限公司 | UI (user interface) testing method and device, electronic equipment and storage medium |
| CN113342684A (en) * | 2021-06-29 | 2021-09-03 | 北京搜狗科技发展有限公司 | Webpage testing method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9846634B2 (en) | Visual graphical user interface verification | |
| CN112015634A (en) | Page structure information generation method and device and electronic equipment | |
| CN110502985B (en) | Form identification method and device and form identification equipment | |
| CN111309618B (en) | Page element positioning method, page testing method and related devices | |
| US11783610B2 (en) | Document structure identification using post-processing error correction | |
| US20140147051A1 (en) | Method and system for webpage regression testing | |
| Yang et al. | Don’t do that! hunting down visual design smells in complex uis against design guidelines | |
| CN113391871A (en) | RPA element intelligent fusion picking method and system | |
| US11221833B1 (en) | Automated object detection for user interface generation | |
| CN105631393A (en) | Information recognition method and device | |
| CN102567565B (en) | Cable parameter processing method and system utilizing same | |
| US11556610B2 (en) | Content alignment | |
| CN110490237B (en) | Data processing method and device, storage medium and electronic equipment | |
| CN109656652A (en) | Webpage graph making method, apparatus, computer equipment and storage medium | |
| CN113657361A (en) | Page abnormity detection method and device and electronic equipment | |
| CN112988557A (en) | Search box positioning method, data acquisition device and medium | |
| CN116682130A (en) | Method, device and equipment for extracting icon information and readable storage medium | |
| CN112667517A (en) | Method, device, equipment and storage medium for acquiring automatic test script | |
| CN113936187A (en) | Text image synthesis method and device, storage medium and electronic equipment | |
| CN117574851A (en) | Method, device and storage medium for reconstructing circuit schematic diagram in EDA tool | |
| CN109871205A (en) | GUI code method of adjustment, device, computer installation and storage medium | |
| CN113722203A (en) | Program testing method and device, electronic device and computer readable storage medium | |
| CN110378566B (en) | Information checking method, device, storage medium and apparatus | |
| CN115905016A (en) | BIOS Setup search function test method and device, electronic equipment and storage medium | |
| CN112286785B (en) | Abnormality detection method and device for user interface |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |