CN114579420A - A kind of page difference detection method and system - Google Patents

Abstract

The application provides a page difference detection method and a page difference detection system. The page difference detection system can acquire the page to be detected, the standard page and a target detection mode determined by a user, then extracts page structure information and/or page element semantic information in the page to be detected and the standard page, and realizes difference detection of the page to be detected based on the target detection mode and the page structure information and/or page element semantic information in the page to be detected and the standard page. In the method, a user can independently select a difference detection mode, the diversified requirements of the user on page difference detection are effectively met, difference detection is carried out on the page to be detected based on page structure information and/or page element semantic information in the page to be detected and the standard page, background information is not needed, the accuracy of page difference detection is effectively improved, and the granularity range of page difference detection is effectively expanded.

Description

A kind of page difference detection method and system

technical field

The present application relates to the field of computer technology, and in particular, to a method and system for detecting page differences.

Background technique

After the software product is developed, it is usually necessary to perform page difference detection on the graphical user interface (GUI) of the software product, that is, to detect the software product in different environments (for example, environments in which different operating systems, browsers, and devices are combined) Whether the displayed page is different from the standard page of the software product.

At present, there are mainly two methods for page difference detection. Method 1: Determine the difference between the page to be detected and the standard page by detecting the difference between the background information (for example, code information) displayed by the same software product in the standard environment and the environment to be tested. ; Method 2: Determine the difference between the to-be-detected page and the standard page by detecting the pixel difference between the to-be-detected page and the standard page of the same software product. Method 1 relies on background information, which is difficult to obtain, and the difference in background code information cannot effectively represent page differences, so it will lead to low accuracy of difference detection results; Method 2 has a single detection method (only pixel differences are detected), which cannot effectively meet user needs. .

SUMMARY OF THE INVENTION

The present application provides a page difference detection method to improve the accuracy of page difference detection. In this method, a user can independently select a difference detection mode, which effectively meets the user's diverse needs for page difference detection. The present application also provides systems, devices, computer-readable storage media, and computer program products corresponding to the above methods.

In a first aspect, a page difference detection method is provided, and the method can be executed by a page difference detection system. In this method, the page difference detection system acquires the page to be detected and the standard page, and processes the page to be detected and the standard page to obtain the first page structure information and/or the semantic information of the first page element of the page to be detected, and the standard page the second page structure information and/or the second page element semantic information; then, the page difference detection system can also obtain the target detection mode determined by the user; according to the target detection mode, the first page structure information and/or the first page element semantics information, and the second page structure information and/or the second page element semantic information, and perform difference detection on the page to be detected to obtain a first difference detection result.

In the technical solutions of the embodiments of the present application, the page difference detection system can be based on the target detection mode determined by the user, the first page structure information and/or the first page element semantic information determined in the page to be detected, and the first page element semantic information in the standard page. The determined second page structure information and/or the second page element semantic information is used for difference detection on the page to be detected. In this way, the user's diverse needs for page difference detection are effectively met, and the accuracy of page difference detection is effectively improved.

In a possible design, the object detection mode includes one or more of the following modes: pixel alignment mode, page element semantic alignment mode, page layout alignment mode, and custom alignment mode.

In this design, a variety of comparison modes for page difference detection are provided, which effectively meet the user's diverse page difference detection needs.

In a possible design, when the target detection mode determined by the user includes the page element semantic comparison mode, the page difference detection system performs difference detection on the page to be detected, and obtains the first difference detection result. The specific implementation may be: The first difference detection result is determined by comparing the semantic information of the first page element with the semantic information of the second page element.

In this design, the page difference detection system effectively expands the detection granularity of page difference detection by comparing the page element semantic information of the page to be detected and the standard page, thereby effectively improving the accuracy of page difference detection.

In a possible design, the first page structure information includes at least one of the style, position or type of page elements in the page to be detected, and the second page structure information includes the style, position or type of page elements in standard pages At least one of; when the target detection mode determined by the user includes the page layout comparison mode, the page difference detection system performs difference detection on the page to be detected, and obtains the first difference detection result, which may be: comparing the first page structure information with the second difference detection result. The page structure information is used to determine the difference detection result, and the first difference detection result is determined.

There are various specific implementations for the page difference detection system to determine the first difference detection result by comparing the first page structure information and the second page structure information, including but not limited to the following three:

Manner 1: Count the degree of positional difference between the first page element in the page to be detected and the first page element and the second page element that have the same style and type in at least one of the standard pages.

Method 2: Count the degree of style difference between the first page element and the second page element that have at least one position and type in the to-be-detected page and the standard page.

Manner 3: Count the degree of difference between the types of the first page element and the second page element that have the same position and style in at least one of the page to be detected and the standard page.

It should be understood that the first page element may be any page element in the page to be detected, and the second page element is the page element corresponding to the any page element in the standard page. The positions of the first page element and the second page element in Modes 1 and 2 may be identical or partially the same (for example, 80% of the positions are the same), which is not specifically limited in this embodiment of the present application.

In this design, the page difference detection system determines the difference detection result by comparing the page structure information of the page to be detected and the standard page, which effectively expands the detection granularity of page difference detection, thereby effectively improving the accuracy of page difference detection.

In a possible design, when the target detection mode determined by the user includes a custom comparison mode, the page difference detection system performs difference detection on the page to be detected, obtains a first difference detection result, and can also receive a first command or a first operation, according to the first command or the first operation to determine the difference type concerned by the user; wherein, the difference type includes one or more of pixels, page element semantics, and page structure; and then the page difference detection system determines the difference type in the page to be detected. The corresponding first information, and the second information corresponding to the difference type in the standard page; the first difference detection result is determined according to the first information and the second information.

In this design, the user can independently select the type of difference to focus on, and the page difference detection system determines the difference detection result by comparing the first information and the second information corresponding to the difference type in the page to be detected and the standard page to determine the difference detection result, that is, effective It meets the diverse needs of users for difference detection, and effectively improves the accuracy of page difference detection.

In a possible design, the page to be tested is a screenshot of the GUI of the first graphical user interface of the web page to be tested, and the standard page is a screenshot of the standard GUI of the web page to be tested.

It should be understood that the first GUI screenshot of the page to be detected may be a graphical user interface (graphical user interface, GUI) screenshot of the webpage to be tested in different environments, that is, the first GUI screenshot may be a screenshot or a collection of multiple GUI screenshots, here , without any specific limitation. Wherein, the environment may be understood as an environment combined with one or more of an operating system, a hardware device, or a web browser and a version of the corresponding web browser. The standard GUI screenshot may specifically be a pre-designed GUI screenshot, or may be a GUI screenshot displayed by the web page to be tested under a user-specified environment (eg, Google Chrome of a Windows operating system).

In this design, the page difference detection system can detect the GUI screenshots of the webpage under test in different environments, which effectively improves the efficiency of page difference detection.

It should be noted that the page difference detection system obtains the page to be detected and the standard page, and there are multiple implementations, including but not limited to the following two:

Mode 1: Receive the first GUI screenshot (ie, the GUI screenshot to be tested) and the standard GUI screenshot uploaded by the user.

Method 2: Obtain the web page link input by the user and/or the hyperlink in the web page corresponding to the web page link; generate the first GUI screenshot and standard GUI screenshot based on the web page link and/or the hyperlink.

In a possible design, the page difference detection system performs difference detection on the page to be detected, and after obtaining the first difference detection result, it can also output the first difference detection result. The first difference detection result may include at least one item of pixel difference information, structural difference information, and semantic difference information of the page to be tested.

In this design, the page difference detection system outputs the first difference detection result, so that the user can intuitively determine the difference between the page to be detected and the standard page.

In a possible design, the page difference detection system can process the pages to be detected and standard pages through a deep learning model. Specifically, the page difference detection system can train GUI screenshots of massive web pages in different environments to obtain deep learning model; wherein, the input of the deep learning model is any GUI screenshot, and the output is the page structure information and/or page element semantic information in any GUI screenshot; and after obtaining the deep learning model, the page difference detection system can be detected. The page is input into the deep learning model, and based on the deep learning model, the first page structure information and/or the first page element semantic information is output; and, the standard page is input into the deep learning model, and the second page structure information and/or the first page structure information and/or the Two page element semantic information.

In this design, the page difference detection system determines the first page structure information and/or the first page element semantic information in the page to be detected through the deep learning model, and determines the second page structure information and/or the standard page in the standard page. The semantic information of the second page element can effectively improve the granularity range and accuracy of page feature recognition.

In a possible design, the page difference detection system performs difference detection on the page to be detected, and after obtaining the first difference detection result, it can further: receive a second operation; an area image, and determine the second area image corresponding to the first area image in the standard page; determine the second difference detection result between the first area image and the second area image; then based on the first difference detection result and The second difference detection result is to determine the third difference detection result between the page to be detected and the standard page, and output the third difference detection result.

The second difference detection result includes at least one item of pixel difference information, structural difference information, and semantic difference information between the first area image and the second area image.

In this design, on the basis that the page difference detection system performs the difference detection between the page to be detected and the standard page based on the target detection mode determined by the user, the user can choose to perform secondary detection on the partial images in the page to be detected. In this way, it is possible to meet the needs of users who want to focus on differences in some areas, thereby effectively improving user experience.

In a second aspect, a system for detecting page differences is provided. Exemplarily, the system for detecting page differences may include:

A processing unit, configured to acquire the page to be detected and the standard page, process the page to be detected and the standard page, and obtain the first page structure information and/or the semantic information of the first page element of the page to be detected, and the second page structure of the standard page information and/or second page element semantic information;

an interface unit for acquiring the target detection mode determined by the user;

The processing unit is further configured to perform difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information, and obtain The first difference detection result.

In a possible design, the object detection mode includes one or more of the following modes: pixel alignment mode, page element semantic alignment mode, page layout alignment mode, and custom alignment mode.

In a possible design, when the target detection mode determined by the user includes the page element semantic comparison mode, when the processing unit is used to perform difference detection on the page to be detected and obtain the first difference detection result, the processing unit is specifically used for:

The first difference detection result is determined by comparing the semantic information of the first page element with the semantic information of the second page element.

In a possible design, the first page structure information includes at least one of the style, position or type of page elements in the page to be detected, and the second page structure information includes the style, position or type of page elements in standard pages At least one of; when the target detection mode determined by the user includes the page layout comparison mode, the processing unit is used to perform difference detection on the page to be detected and obtain the first difference detection result, and is specifically used for:

The first difference detection result is determined by comparing the first page structure information with the second page structure information.

In a possible design, when the target detection mode determined by the user includes a user-defined comparison mode, before the processing unit is used to perform difference detection on the page to be detected, the interface unit is further configured to: receive a first command or a first operation The processing unit is also used to: determine the difference type that the user pays attention to according to the first command or the first operation; wherein, the difference type includes one or more of pixels, page element semantics, and page structure; the processing unit is used to detect The page performs difference detection, and when the first difference detection result is obtained, it is specifically used to: determine the first information corresponding to the difference type in the page to be detected, and the second information corresponding to the difference type in the standard page; according to the first information and the second information , and determine the first difference detection result.

In a possible design, the page to be tested is a screenshot of the GUI of the first graphical user interface of the web page to be tested, and the standard page is a screenshot of the standard GUI of the web page to be tested.

In a possible design, when the processing unit is used for the to-be-detected page and the standard page, it is specifically configured to: obtain the web page link input by the user and/or the hyperlink in the web page corresponding to the web page link; / or hyperlinks to generate first GUI screenshots and standard GUI screenshots.

In a possible design, the processing unit is configured to perform difference detection on the page to be detected, and after obtaining the first difference detection result, output the first difference detection result, where the first difference detection result includes pixel difference information and structure of the page to be tested. At least one item of difference information and semantic difference information.

In a possible design, before the processing unit is used to process the page to be detected and the standard page, the processing unit is also used to train GUI screenshots of massive web pages in different environments to obtain a deep learning model; The input of the model is any GUI screenshot, and the output is page structure information and/or page element semantic information in any GUI screenshot; when the processing unit is used to process the page to be detected and the standard page, it is specifically used for: The page is input into the deep learning model, and based on the deep learning model, the first page structure information and/or the first page element semantic information is output; and, the standard page is input into the deep learning model, and the second page structure information and/or the first page structure information and/or the Two page element semantic information.

In a possible design, the interface unit is further configured to: after the processing unit performs difference detection on the page to be detected, and obtains the first difference detection result, receiving the second operation; the processing unit is further configured to: perform the difference detection on the page to be detected according to the second operation. Determine the first area image that the user pays attention to in the page, and determine the second area image corresponding to the first area image in the standard page; determine the second difference detection result between the first area image and the second area image; The second difference detection result includes at least one item of pixel difference information, structural difference information, and semantic difference information between the first area image and the second area image; based on the first difference detection result and the second difference detection result, determine the page to be detected The third difference detection result with the standard page; the interface unit is also used for outputting the third difference detection result.

In a third aspect, a computing device is provided. Exemplarily, the computing device includes a processor and a memory;

The processor is configured to execute instructions stored in the memory to cause the apparatus to perform the method as described in the first aspect or any possible design of the first aspect.

In a fourth aspect, a computer-readable storage medium is provided, comprising instructions that, when executed on a computing device, cause the computing device to perform the method described in the first aspect or any possible design of the first aspect.

For the technical effects that can be achieved in any of the above-mentioned second to fourth aspects, the specific technical effects that can be achieved in any of the above-mentioned first aspects can be referred to in any of the above-mentioned second to fourth aspects. The description of the technical effects brought about by any of the possible designs in the first aspect above will not be repeated here.

Description of drawings

FIG. 1 is a schematic diagram of a system architecture to which an embodiment of the present application is applicable;

2 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

3A is a schematic diagram of a command user interface provided by an embodiment of the present application;

3B is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

4 is a schematic flowchart of a method for detecting page differences provided by an embodiment of the present application;

5 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

6 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a model training scenario provided by an embodiment of the present application;

8 is a schematic diagram of a scenario for acquiring feature information of a page to be detected and a standard page according to an embodiment of the present application;

9 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

10 is a schematic diagram of a page difference detection process provided by an embodiment of the present application;

11 is a schematic diagram of a page difference detection process provided by an embodiment of the present application;

12 is a schematic diagram of a page difference detection process provided by an embodiment of the present application;

13 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

14 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

15 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

16 is a schematic diagram of a process for determining a compatibility test result provided by an embodiment of the present application;

17 is a schematic diagram of a page difference detection process provided by an embodiment of the present application;

FIG. 18 is a schematic structural diagram of a computing device according to an embodiment of the present application.

Detailed ways

In the embodiments of this application, the term "at least one" refers to one or more, and "multiple" refers to two or more. "And/or" describes the association relationship of the associated objects, indicating that there can be three kinds of relationships, For example, A and/or B can mean that A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character "/" generally indicates that the associated object is a kind of " "or" relationship. "At least one of the following items" or similar expressions, refers to any combination of these items, including any combination of single item (item) or plural item (item). For example, a, b, Or at least one (one) of c, which can represent: a, b, c, a-b, a-c, b-c, or a-b-c, where any of a, b, c, a-b, a-c, b-c, or a-b-c can be It includes a single a, a single b, and a single c, and can also include multiple a, multiple b, and multiple c.

And, unless stated to the contrary, the ordinal numbers such as “first” and “second” mentioned in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the order, sequence, priority or priority of multiple objects. Importance. For example, the first priority criterion and the second priority criterion are only for distinguishing different criteria, and do not indicate the difference in content, priority, or importance of the two criteria.

In addition, the terms "comprising" and "having" in the embodiments and claims of the present application and the drawings are not exclusive. For example, a process, method, system, product or device that includes a series of steps or modules/units is not limited to the listed steps or modules, and may also include unlisted steps or modules/units.

With the continuous development of computer technology, the types of software products are becoming more and more abundant. In order to ensure user experience, the consistency of GUI display of software products in different environments is particularly important. Therefore, there is an urgent need and broad prospect to detect differences in page display of software products in different environments.

The embodiment of the present application provides a method for detecting page differences, and the method can be executed by a system for detecting page differences. The page difference detection system obtains the page to be detected, the standard page and the target detection mode determined by the user through a user interface such as a graphical user interface (GUI) or a command user interface (CUI), etc., and then extracts the page to be detected. and the page structure information and/or page element semantic information in the standard page, based on the target detection mode and the page structure information and/or page element semantic information in the page to be detected and the standard page, the difference detection of the page to be detected is realized, and then Get the difference detection result.

In this method, the user can independently select the difference detection mode, which effectively meets the user's diverse needs for page difference detection, and the page to be detected is based on the page structure information and/or page element semantic information in the page to be detected and the standard page. Difference detection does not need to rely on background code information, effectively improves the accuracy of page difference detection, and effectively expands the granularity of page difference detection.

Referring to FIG. 1 , FIG. 1 shows a schematic diagram of a system architecture to which the embodiments of the present application are applied. In FIG. 1 , the page difference detection system 100 may interact with a user to implement a page difference detection function.

It should be noted that, the page difference detection system 100 may be provided to the user in the form of an application program interface, and may also be provided to the user in the form of a web interface.

As shown in FIG. 1 , the page difference detection system 100 includes an interface unit 101 and a processing unit 102 .

The interface unit 101 is used to interact with the user.

In a possible embodiment, when the page difference detection system 100 is provided to the user in the form of an application program interface, the interface unit 101 may be a command user interface, and the page difference detection system 100 may receive user input through the command user interface (for example, receive program code containing object detection mode parameter information).

In another possible embodiment, when the page difference detection system 100 is provided to the user in the form of a web interface, the interface unit 101 may be a graphical user interface, and the page difference detection system 100 receives the user-executed information through the graphical user interface. Actions (for example, the action of uploading a picture, the action of selecting the object detection mode).

The processing unit 102 can be used to obtain the page to be detected and the standard page, process the page to be detected and the standard page, and obtain the first page structure information and/or the semantic information of the first page element of the page to be detected, and the second page of the standard page. Page structure information and/or second page element semantic information.

Further, the processing unit 102 can also be configured to perform the processing on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information Difference detection is performed to obtain a first difference detection result.

It should be understood that the page to be detected can be a screenshot of a graphical user interface (GUI) of a software product to be tested (eg, a web page, an application (application, App), etc.) in different environments, that is, the page to be detected can be one or more Multiple GUI screenshots are not specifically limited in this embodiment of the present application. Wherein, the environment may be understood as an environment combined with one or more of an operating system, a hardware device, or a web browser and a version of the corresponding web browser. The standard page is the standard GUI screenshot of the software product to be tested, which can be a pre-designed GUI screenshot, or can be displayed by the software product to be tested under the user-specified environment (for example, Google Chrome of a Windows operating system). screenshot of the GUI.

The processing unit 102 may include an acquisition module 1021 , a training module 1022 , a feature extraction module 1023 , a difference detection module 1024 , and a database 1025 .

The obtaining module 1021 can be used to obtain the page to be detected and the standard page, and obtain the target detection mode determined by the user.

In a possible embodiment, when the interface unit 101 receives a user operation or a calling command, the acquiring module 1021 may determine the target detection mode selected by the user according to the user operation or calling command.

In a possible embodiment, the interface unit 101 receives a uniform resource locator (uniform resource location, URL) (also referred to as a web page link) and/or a web page corresponding to the URL of the web page to be tested input by the user. When there is a hyperlink, the obtaining module 1021 may be configured to generate the to-be-detected page and the standard page corresponding to the to-be-tested webpage based on the webpage link, and save the to-be-detected page and the standard page in the database 1025 .

In another possible embodiment, when the interface unit 101 receives the at least two pictures uploaded by the user, the acquiring module 1021 may mark the at least two pictures uploaded by the user as a page to be detected and a standard page respectively.

The training module 1022 can be used to perform training based on the screenshots of the massive webpages stored in the database 1025, or the screenshots of the GUI of the massive webpages in different environments, to obtain a deep learning model; wherein, the input of the deep learning model is any of the screenshots of the massive webpages. A GUI screenshot, the output is page structure information and/or page element semantic information in any GUI screenshot.

The feature extraction module 1023 can be used to input the page to be detected into the deep learning model, and output the first page structure information and/or the semantic information of the first page element of the page to be detected based on the deep learning model; and, input the standard page The deep learning model outputs the second page structure information and/or the second page element semantic information of the standard page based on the deep learning model.

The difference detection module 1024 can be configured to perform difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information , to obtain the first difference detection result.

It should be noted that the above target detection mode may be any one of pixel comparison mode, page element semantic comparison mode, page layout comparison mode, and custom comparison mode, which is not specifically limited here. Therefore, the difference detection module 1024 is configured to realize the difference between the pages to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, the second page structure information and/or the second page element semantic information There are different implementations for detection.

In a possible embodiment, when the acquisition module 1021 determines that the target detection mode selected by the user includes the pixel comparison mode, the difference detection module 1024 can calculate the pixel difference at the same position between the page to be detected and the standard page to detect Determine the difference between the page to be inspected and the standard page.

In a possible embodiment, when the acquisition module 1021 determines that the target detection mode selected by the user includes the page element semantic comparison mode, the difference detection module 1024 can compare the semantic information of the first page element of the page to be detected with the semantic information of the standard page The second page element semantic information is used to determine the difference between the to-be-detected page and the standard page.

In a possible embodiment, when the acquisition module 1021 determines that the target detection mode selected by the user includes the page layout comparison mode, the difference detection module 1024 may compare the first page structure information and the second page structure information to determine the target detection mode to be detected. Differences between pages and standard pages.

Wherein, the first page structure information includes at least one of styles, positions or types of page elements in the page to be detected, and the second page structure information includes at least one of styles, positions or types of page elements in standard pages; differences; The detection module 1024 compares the first page structure information and the second page structure information to determine the specific implementation manner of the first difference detection result, including but not limited to the following three:

Manner 1: Count the degree of positional difference between the first page element in the page to be detected and the first page element and the second page element that have the same style and type in at least one of the standard pages.

Method 2: Count the degree of style difference between the first page element and the second page element that have at least one location and type in the page to be detected and the standard page.

Manner 3: Count the degree of difference between the types of the first page element and the second page element that have the same position and style in at least one of the page to be detected and the standard page.

It should be understood that the first page element may be any page element in the page to be detected, and the second page element is the page element corresponding to the any page element in the standard page. The positions of the first page element and the second page element in Modes 1 and 2 may be identical or partially the same (for example, 80% of the positions are the same), which is not specifically limited in this embodiment of the present application.

In this embodiment of the present application, each part of the page difference detection system 100 may be centrally deployed on a local computing device (a user device under the direct control of a user, such as a user terminal such as a notebook computer, a desktop computer, a smart phone, etc.) or In a cloud computing cluster (including at least one cloud computing device, such as a cloud server, etc.). Of course, various parts of the page difference detection system 100 can also be deployed in a cloud computing cluster in a distributed manner.

Taking the software product to be tested as a webpage as an example, the detailed process of the interaction between the page difference detection system 100 and the user when the page difference detection system 100 is provided to the user in different product forms is described with reference to the specific drawings.

It should be noted that when the page difference detection system 100 is provided to the user in the form of a web interface, the page difference detection system 100 provides the user with a graphical user interface, in which the user can input information (for example, pictures, URL, etc.), the page difference detection system 100 can then perform page difference detection based on the information received through this graphical user interface. In addition, in this graphical user interface provided by the page difference detection system 100, different user interfaces can also be displayed in different detection stages (before and after detection) of the page difference detection system 100 (the page difference detection shown in FIG. 2 ). interface, the web page setting interface shown in Figure 5, the web page setting interface shown in Figure 5). In this way, the page difference detection system 100 can have good interaction with the user in the process of page difference detection.

Correspondingly, when the page difference detection system 100 is provided to the user in the form of an application program interface, the page difference detection system 100 can interact with the user through a command user interface. It should be understood that the command user interface here may specifically be an interface for a user to use an integrated development environment (IDE) (for example, IPython, Anaconda, etc.); or a user interface for a third-party interface calling tool (for example, Swagger Codegen) .

In FIG. 3A, the interface of the IDE is exemplarily drawn, and the user can operate in the IDE through external input and output devices (such as a display, a keyboard, a mouse, etc.), for example, can write and upload GUI screenshots and standard GUI screenshots to the page The program code of the difference detection system 100, and the program code containing the URL and/or hyperlink of the web page to be tested to the page difference detection system 100.

In FIG. 3B, the interface of the third-party interface calling tool on the display is exemplarily drawn, and the interface provides a graphical user interface that can interact with the user. The user can use external input and output devices (such as a display, a keyboard and a mouse etc. device), input the URL of the webpage to be tested or the relevant parameter information of the target detection mode in the graphical user interface provided by the third-party interface calling tool, the third-party interface calling tool can be based on the URL of the webpage to be tested, the relevant parameter information of the target detection mode The parameter information (for example, the semantic comparison mode identifier and the semantic difference degree threshold) is detected, the difference detection is performed on the GUI screenshot corresponding to the web page to be tested, and the difference detection result is returned to the user through the graphical user interface.

For example, the user inputs "https://aims-wct.huawei.com/v1/comparison/strict" and related parameters of the target detection mode in the graphical user interface provided by the third-party interface calling tool shown in FIG. 3B, The third-party interface calling tool can call the page difference detection system 100 to implement difference detection for the GUI screenshot corresponding to "https://aims-wct.huawei.com/v1/comparison/strict", which is shown in FIG. 3B . The corresponding page difference detection results are displayed in the GUI of the .

This embodiment of the present application does not limit the deployment method of the IDE. For example, the IDE may be an application deployed in the cloud, and it may be a cloud IDE, that is, the IDE may be deployed in an edge computing device system or a cloud computing system. The external input and output device realizes the operation in this cloud IDE. The IDE may also be a client IDE installed on a local terminal computing device, that is, the IDE runs directly on the local computing device.

Please refer to FIG. 4. FIG. 4 shows a schematic flowchart of a page difference detection method provided by an embodiment of the present application. The method can be applied to the page difference detection system 100, and the method includes:

S401. The page difference detection system 100 acquires the page to be detected and the standard page.

It should be understood that the page to be tested may be one or more GUI screenshots of the webpage to be tested (also referred to as GUI screenshots to be tested), and the standard page may be a GUI screenshot of a standard page of the webpage to be tested (also referred to as standard GUI screenshots) , which may be a pre-designed GUI screenshot, or a GUI screenshot displayed by the web page to be tested under a user-specified environment (eg, Google Chrome of a Windows operating system).

In a possible design, the page difference detection system 100 is provided to the user in the form of a web interface, and the specific implementation manners that the page difference detection system 100 obtains the GUI screenshots to be tested and the standard GUI screenshots include but are not limited to the following two:

Manner 1, the page difference detection system 100 can receive the to-be-tested GUI screenshots and standard GUI screenshots of the to-be-tested webpage uploaded by the user through a graphical user interface (eg, the page difference detection interface shown in FIG. 2 ).

Mode 2, the page difference detection system 100 may receive the URL of the web page to be tested input by the user and/or the URL of the web page to be tested corresponding to the URL of the web page to be tested through a graphical user interface (for example, the page difference detection interface shown in FIG. 2). link, and then the page difference detection system 100 can generate a screenshot of the GUI to be tested and a screenshot of the standard GUI of the webpage to be tested based on the URL and/or the hyperlink.

The hyperlink may specifically be positioning information of a specific page element in the web page corresponding to the web page link. Exemplarily, assuming that the URL of the webpage to be tested is the website of Baidu, the hyperlink may be link information corresponding to "Baidu Academic" in the page displayed by the website of Baidu.

It should be noted that, before the page difference detection system 100 generates a screenshot of the GUI to be tested and a screenshot of a standard GUI of the web page to be tested based on the URL and/or hyperlink of the web page to be tested, the user can check the environment information and page information of the web page to be tested. set up.

The page difference detection system 100 can control the page difference detection interface after detecting that the user inputs the URL of the web page to be tested (for example, "https://baidu.com") in the page difference detection interface shown in FIG. 5(a). Jump to the web page setting interface as shown in Figure 5(b). The page difference detection system 100 can receive the user's environment setting information and page setting information of the webpage to be tested through the webpage setting interface.

Specifically, in FIG. 5(b), in the environment setting function provided by the page difference detection system 100 through the web page setting interface, the user can set the operating system, the setting of the browser and the browser version, and the setting of the screenshot application. .

Exemplarily, the operating system can be set to any one of windows, linux, macos, android (android), etc.; the screenshot application can be set to a browser or a screenshot application that comes with the operating system; the browser setting can be set to Google, Firefox (firefox) and other browser types, and you can also set the version information of these browsers (for example, the latest version, the historical version). Optionally, in the environment setting function provided by the page difference detection system 100 through the webpage page setting interface, in the environment setting function, the user may also set network proxy information.

Specifically, in FIG. 5(b), in the webpage page setting interface provided by the page difference detection system 100, the user can set the resolution of the page to be tested and the standard page (ie the width and height of the page), and whether to display a notification reminder etc.

In order to improve the efficiency of page difference detection, the web page setting interface shown in FIG. 5(b) also provides a default setting option. After the page difference detection system 100 detects the click operation selected for the default setting, it can The webpage configuration environment setting information, page setting information, etc., do not require manual input by the user. In this way, the user experience is effectively improved.

Optionally, the page difference detection system 100 may record the environment setting information and page setting information used by the user each time, and generate historical setting information in the default settings. After the default setting option is selected in the interface, the page difference detection system 100 can display the default setting interface as shown in FIG.

Further, after the page difference detection system 100 detects a click operation on any one of the original setting, the historical setting 1, and the historical setting 2, it can also receive the information input by the user, and update the specific parameter in the any one based on the information. Information (environment information, page setup information). For example, after determining that the user selects the historical setting 2, the page difference detection system 100 may also receive the environmental information input by the user to update the environmental setting information in the historical setting 2. In this way, the efficiency of page difference detection can be effectively improved.

In another possible design, the page difference detection system 100 is provided to the user in the form of an application program interface, and the specific implementations for the page difference detection system 100 to obtain the GUI screenshots to be tested and the standard GUI screenshots include but are not limited to the following two :

Mode 1, the page difference detection system 100 can receive the program code input by the user including the storage addresses of the GUI screenshots to be tested and the standard GUI screenshots through the command user interface (the IDE interface shown in FIG. 3A ), and obtain the program codes to be tested based on the program code. Screenshots of the GUI to be tested and standard GUI screenshots of the webpage.

Mode 2, the page difference detection system 100 can receive the URL of the web page to be tested and/or the hyperlink in the web page corresponding to the URL, and the program code of the environment setting information and the page setting information input by the user through the command user interface , and parse out this URL and/or this hyperlink, environment setting information, page setting information from this program code, based on this URL and/or this hyperlink, generate the environment setting information, page setting information of the webpage to be tested in this environment setting information Corresponding GUI screenshots to be tested and standard GUI screenshots.

Optionally, after the page difference detection system 100 generates the GUI screenshot to be tested and the standard GUI screenshot of the webpage to be tested, the page difference detection system 100 can use a certain encoding method (for example, in the form of a string) to screenshot the GUI to be tested. and standard GUI screenshots are stored in the data storage module of the page difference detection system 100 .

S402. The page difference detection system 100 processes the page to be detected and the standard page, and obtains first page structure information and/or first page element semantic information of the page to be detected, and second page structure information and/or second page structure information of the standard page Page element semantic information.

In a possible embodiment, the page difference detection system 100 may use a deep learning model to perform feature extraction on the page to be detected and the standard page, obtain first page structure information and/or first page element semantic information of the page to be detected, and The second page structure information and/or the second page element semantic information of the standard page.

Correspondingly, before processing the page to be detected and the standard page, the page difference detection system 100 needs to obtain the deep learning model first.

The following describes how to obtain the deep learning model.

Specifically, please refer to FIG. 7. In FIG. 7, the page difference detection system 100 can obtain a deep learning model by training a large number of webpage GUI screenshots; the input of the deep learning model can be any webpage GUI screenshot, and the output is the arbitrary webpage GUI screenshot. Page structure information and/or page element semantic information in a webpage GUI screenshot. The page structure information includes information such as the position, type, and style of page elements.

After the page difference detection system 100 obtains the deep learning model, feature extraction can be performed on the to-be-detected page and the standard page of the to-be-tested web page based on the deep learning model. As shown in FIG. 8 , the page difference detection system 100 inputs the page to be detected into the deep learning model, and can obtain the first page structure information and/or the first page element semantic information of the page to be detected; By inputting the deep learning model, the second page structure information and/or the second page element semantic information of the standard page can be obtained.

It should be understood that when the page to be detected is a screenshot of the GUI to be tested, the first page structure information and/or the semantic information of the first page element is the page structure information and/or the semantic information of the page element of a screenshot of the GUI to be tested; When the page is a plurality of GUI screenshots to be tested, the first page structure information and/or the first page element semantic information is a collection of page structure information and/or page element semantic information of the multiple GUI screenshots to be tested.

S403. The page difference detection system 100 acquires the target detection mode determined by the user.

In a possible design, the page difference detection system 100 is provided to the user in the form of a web interface, and the page difference detection system 100 can display a selection and comparison mode interface as shown in FIG. Comparison mode, page element semantic comparison mode, page layout comparison mode, and custom comparison mode; after the page difference detection system 100 detects a click operation for any comparison mode in the selection comparison mode interface, An alignment mode is determined as a target detection mode.

In another possible design, the page difference detection system 100 is provided to the user in the form of an application program interface, and the page difference detection system 100 can receive the inclusion target input by the user through a command user interface (the IDE interface shown in FIG. 3A ). The program code of the relevant parameters of the detection mode; based on the program code, the target detection mode is determined.

The relevant parameters may specifically be the difference type and difference degree corresponding to the target detection mode. For example, when the target detection mode is the pixel comparison mode, the relevant parameter is the identification of the pixel comparison mode, and the pixel difference ratio is 80%; for another example, when the target detection mode is the page layout comparison mode, the relevant parameter is the page layout ratio. The identification of the mode, the position offset threshold of the page elements; for another example, when the target detection mode is the page element semantic comparison mode, the relevant parameters are the page layout comparison mode identification and the similarity degree of the page element semantics; for another example, the target detection mode When the mode is a custom comparison mode, the relevant parameters are one or more of pixels, page element semantics, and page structure information. Optionally, the relevant parameters may further include location information of a specific area of the page to be detected and/or identification information of a specific page element.

S404, the page difference detection system 100 performs difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information, A first difference detection result is obtained.

It should be noted that the above target detection mode may be any one of a pixel comparison mode, a page element semantic comparison mode, a page layout comparison mode, and a custom comparison mode. Therefore, depending on the target detection mode, the specific implementation manner of the page difference detection system 100 to perform the difference detection on the page to be detected is also different.

The difference detection process in different alignment modes is described in detail below.

(1) Pixel comparison mode

In the pixel comparison mode, the page difference detection system 100 can extract the pixel value of each pixel in the screenshot of the GUI to be tested and the screenshot of the standard GUI, and compare the pixel size of each pixel at the same position in the screenshot of the GUI to be tested and the screenshot of the standard GUI in turn, Further, the degree of pixel difference between the GUI screenshot and the standard GUI screenshot is statistically detected. When the pixel difference degree satisfies the preset condition, it can be considered that there is no pixel difference between the GUI screenshot to be tested and the standard GUI screenshot.

Exemplarily, the preset condition is that the ratio of the same pixels in the screenshot of the GUI to be tested and the screenshot of the standard GUI is greater than the preset ratio (eg, 80%), and the page difference detection system 100 is at the same position in each screenshot of the GUI to be detected and the screenshot of the standard GUI. After comparing the pixel sizes on the screen, it is found that 90% of the pixels in the detected GUI screenshot and the standard GUI screenshot are the same, and it is considered that there is no pixel difference between the to-be-detected GUI screenshot and the standard GUI screenshot.

(2) page element semantic comparison mode

In the page element semantic comparison mode, the page difference detection system 100 can extract the semantic information of each page element in the GUI screenshot to be tested and the standard GUI screenshot, and count at least one first page element in the GUI screenshot to be tested and the standard GUI screenshot in turn The semantic difference degree of at least one of the second page elements, and the first difference detection result is determined according to the semantic difference degree. The position of each first page element in the at least one first page element in the GUI screenshot to be tested is in a one-to-one correspondence with the position of each second page element in the at least one second page element in the standard GUI screenshot. The first page element or the second page element may be text, pictures, hyperlinks, etc., which are not specifically limited in this embodiment of the present application.

Wherein, the page difference detection system 100 determines the first difference detection result according to the semantic difference degree. Specifically, it can be determined whether the semantic difference degree exceeds a preset threshold. If it exceeds the preset threshold, it is considered that the first page element and the second page element are There are page element semantic differences; if the preset threshold is not exceeded, it is considered that there is no page element semantic difference between the first page element and the second page element.

For example, assuming that the preset threshold is 2%, the first page element is text, the text is "I love China", and the text of the second page element corresponding to the first page element is "I love China", then the first page element is "I love China". The page element semantic difference between the first page element and the second page element is 25%, which exceeds the preset threshold, then it is considered that there is a page element semantic difference between the first page element and the second page element, that is, the GUI screenshot to be tested and the standard GUI screenshot There are semantic differences in page elements.

For another example, assuming that the preset threshold is 2%, there are 100 characters in the first page element, and 100 characters in the second page element, but there is a difference between the second page element and the first page element by 1 character, Then the page element semantic difference between the first page element and the second page element is 1%, and if it does not exceed the preset threshold, it is considered that there is no page element semantic difference between the first page element and the second page element, that is, the GUI screenshot and Page element semantic differences do not exist in standard GUI screenshots.

Optionally, the page difference detection system 100 determines the first difference detection result according to the semantic difference degree, and may also identify the first target object of the first page element and the second target object in the second page element, and compare the first target object of the first page element and the second target object of the second page element. Whether the target object and the second target object are consistent, if they are consistent, it is considered that there is no page element semantic difference between the first page element and the second page element; if they are inconsistent, it is considered that there is a page element semantic difference between the first page element and the second page element ;

For another example, assuming that the first page element and the second page element are pictures, the first target object in the first page element is "cat", and the second target object in the second page element is "dog", then the first If the target object and the second target object are inconsistent, it is considered that there is a semantic difference between the first page element and the second page element, that is, there is a semantic difference between the GUI screenshot to be tested and the standard GUI screenshot.

(3) Page layout comparison mode

In the page layout comparison mode, the page difference detection system 100 can extract the page structure information in the GUI screenshot to be detected and the standard GUI screenshot, wherein the page structure information includes the type, style, position, etc. of page elements, which is not limited in this application.

Here are three examples of possible alignments:

Manner 1, the page difference detection system 100 counts the position difference (eg, overall offset or partial offset) of the first page element in the GUI screenshot to be tested and the second page element in the standard GUI screenshot. The styles and types of the first page element and the second page element are the same.

Example 1, please refer to Fig. 10. The page element A in the standard GUI screenshot shown in Fig. 10(a) includes page elements A1, A2, and A3. Page element B has the same style and type as page element A. Page element B includes page elements B1, B2, and B3. The page difference detection system 100 compares the position of page element B in the GUI screenshot 1 to be tested with page element A. In the position in the standard GUI screenshot, it is found that the page element B has an overall offset relative to the page element A, and it is determined that there is a difference in the position of the page element between the GUI screenshot to be tested and the standard GUI screenshot.

Example 2, the page element C in the screenshot 2 of the GUI to be tested shown in FIG. 10(c) is of the same style and type as the page element A in the screenshot of the standard GUI shown in FIG. 10(a). The page element C includes Page elements C1, C2, C3, the page difference detection system 100 finds that the page element C exists relative to the page element A by comparing the position of the page element C in the GUI screenshot 2 to be tested with the position of the page element A in the standard GUI screenshot If some elements (ie, C3 in Fig. 10(c)) are offset, it is determined that there is a difference in the position of page elements between the screenshot of the GUI to be tested and the screenshot of the standard GUI.

Mode 2, the page difference detection system 100 counts the degree of style difference between the first page element in the GUI screenshot to be tested and the second page element in the standard GUI screenshot. The positions and types of the first page element and the second page element are the same. It should be understood that the positions of the first page element and the second page element may be the same or partially the same (for example, 80% of the positions are the same, depending on the threshold parameter set by the user), and the embodiments of the present application do not specific restrictions. The style of the page element includes one or more of the size, color, and shape of the page element.

Example 1, please refer to Fig. 11. Page element A in the screenshot of the standard GUI shown in Fig. 11(a) is in the same position and type as page element B in the screenshot 1 of the GUI under test shown in Fig. 11(b). The difference detection system 100 compares the style information of page element A and page element B and finds that there is a difference in shape between page element A and page element B, and determines that there is a style difference between the GUI screenshot 1 to be tested and the standard GUI screenshot.

Example 2, the position and type of page element C in the screenshot 2 of the GUI to be tested shown in FIG. 11(c) and the page element A in the screenshot of the standard GUI shown in FIG. 11(a) are the same, and the page difference detection system 100 By comparing the style information of page element A and page element C, it is found that there is a size difference between page element A and page element C, and it is determined that there is a style difference between the GUI screenshot 2 to be tested and the standard GUI screenshot.

Example 3, the page difference detection system 100 identifies that the color information of the first page element is blue, and the color information of the second page element is red, then there is a color difference between the first page element and the second page element; and then the page difference detection system 110 Determine that there is a style difference between the GUI screenshot to be tested and the standard GUI screenshot.

Mode 3, the page difference detection system 100 counts the degree of type difference between the first page element in the GUI screenshot to be tested and the second page element in the standard GUI screenshot.

Specifically, the page difference detection system 100 counts whether there are page elements missing, added, deleted, etc. in the screenshot of the GUI to be tested and the screenshot of the standard GUI.

Example 1, please refer to Fig. 12, the page element A that exists in the standard GUI screenshot shown in Fig. 12(a), the page element A includes page elements A1 and A2; while the GUI screenshot 1 to be tested shown in Fig. 12(b) If there is no page element with the same position and style as page element A, the page difference detection system 100 determines that the GUI screenshot 1 to be tested and the standard GUI screenshot have a page element type difference.

Example 2, please continue to refer to Figure 12, the position and style of page element C and page element A in screenshot 2 of the GUI to be tested shown in Figure 12(c) are the same, but there is still page element B in screenshot 2 of the GUI to be tested , the page difference detection system 100 determines that there is a page element type difference between the GUI screenshot 2 to be tested and the standard GUI screenshot.

Example 3, please continue to refer to Figure 12, the position and style of page element D in the screenshot 3 of the GUI to be tested shown in Figure 12(d) are the same as page element A, but only part of page element D1 in page element D, then It is considered that the page elements in the screenshot 3 of the GUI to be tested are partially missing, and the page difference detection system 100 determines that there is a difference in page element type between the screenshot 3 of the GUI to be tested and the standard GUI screenshot.

(4) Custom comparison mode

In the custom comparison mode, the page difference detection system 100 can obtain the difference type concerned by the user through the graphical user interface, and determine the first information corresponding to the difference type in the page to be detected, and the corresponding difference type in the standard page. second information; determining a first difference detection result according to the first information and the second information. The difference types concerned by the user may include one or more of pixels, page element semantics, and page structure.

Please refer to Fig. 13. Fig. 13 shows the difference type selection interface. In the difference type selection interface shown in Fig. 13, the user can select the difference type he is concerned about, such as pixels, page element semantics, page structure, etc., page difference detection When the system 100 detects that the user places the cursor on the page structure, it can further display the page element style under the page structure, the position of the page element, the type of the page element, and the difference type for the user to choose and pay attention to.

Exemplarily, when the page difference detection system 100 determines that the type of difference that the user focuses on is the style of the page element and the semantics of the page element, the first information extracted by the page difference detection system 100 in the screenshot of the GUI to be tested is the first page element. The second information is the style information and semantic information of the second page element in the standard GUI screenshot. The page difference detection system 100 compares the style information and semantic information of the first page element with the style of the second page element. information and semantic information to determine the difference between the GUI screenshot under test and the standard GUI screenshot. The type and location of the second page element and the first page element are the same. The first page element may be any page element in the pages to be detected, and the second page element may be any page in the pages to be detected. element.

Further, as shown in FIG. 14 , in the custom comparison mode, the page difference detection system 100 can also display an interface for selecting a specific area or selecting a specific element, in which the user can also select a specific page element (for example, Picture elements, text elements) to perform difference detection; or, the user can also select a specific area of the page to be detected for difference detection.

In the above embodiment, the user can independently select the difference detection mode, which effectively meets the user's diverse needs for page difference detection, and based on the page structure information and/or page element semantic information in the page to be detected and the standard page. Page difference detection can effectively improve the accuracy of page difference detection, and effectively expand the granularity range of page difference detection.

Optionally, after the page difference detection system 100 performs difference detection between the page to be detected and the standard page according to the target detection mode determined by the user to obtain the first difference detection result, the system 100 may further: receive a second operation; Determine the first area image that the user pays attention to, and determine the second area image corresponding to the first area image in the standard page; determine the second difference detection result between the first area image and the second area image; second The difference detection result includes at least one of pixel difference information, structural difference information, and semantic difference information between the first area image and the second area image; based on the first difference detection result and the second difference detection result, determine the page to be detected and the standard The third difference detection result of the page is output, and the third difference detection result is output. In this way, the user's need to focus on whether there is a difference in detail between some areas in the to-be-detected page and the corresponding area in the standard page can be effectively met, thereby effectively improving the user experience.

In a possible embodiment, when the page difference detection system 100 is provided to the user in the form of a web interface, the page difference detection system 100 can display the schematic diagram of the first difference detection result interface as shown in FIG. 15 . In the detection result interface, the user can choose to directly output the first difference detection result, or choose to re-detect the partial image of the page to be detected, and then the page difference detection system 100 can receive the user through the first difference detection result interface shown in FIG. 15 . The input position information of the partial image and the new target detection mode, based on the position information of the partial image, determine the first area image of the user's attention in the page to be detected, and determine the first area image in the standard page. The corresponding second area image; based on the new target detection mode, the first area image and the second area image are subjected to secondary detection to obtain a second difference detection result.

Exemplarily, after the page difference detection system 100 performs difference detection between the GUI screenshot to be tested and the standard GUI screenshot according to the semantic comparison mode of the page elements, and obtains the first difference detection result, the page difference detection system 100 detects that the user has input the content to be tested. The position information of the area A in the screenshot and the pixel comparison mode is selected, the page difference detection system 100 counts the pixel difference between the area A and the area B corresponding to the area A in the standard GUI screenshot, and obtains the second difference detection result, based on the first difference detection result. The difference detection result and the second difference detection result obtain the final difference detection result.

In another possible embodiment, when the page difference detection system 100 is provided to the user in the form of an application program interface, the page difference detection system 100 can receive the user input through a command user interface (the IDE interface shown in FIG. 3A ). The program code containing the local image position information and the new target detection mode, parse the program code to obtain the local image position information and the new target detection mode; based on the position information of the local image, determine the page to be detected that the user pays attention to. An area image, and a second area image corresponding to the first area image is determined in the standard page; and based on the new target detection mode, the first area image and the second area image are subjected to secondary detection to obtain a second difference detection As a result, a final difference detection result is obtained based on the first difference detection result and the second difference detection result.

Optionally, the page difference detection system 100 performs difference detection on the page to be detected, and after obtaining the first difference detection result, can also output the first difference detection result, where the first difference detection result includes the pixel difference of the webpage to be tested under different environments. At least one item of information, page structure difference information, and page element semantic difference information. Wherein, the page structure information includes types, styles, positions, etc. of page elements, which are not limited in this application.

Exemplarily, the first difference detection result may be one or more of semantic difference degree, pixel difference degree, position difference degree, type difference degree, and style difference degree of page elements. In this way, the page difference results are made concrete, and the user experience is effectively improved.

Optionally, referring to FIG. 16 , after the page difference detection system 100 obtains the difference detection result, it can determine the compatibility test result of the web page to be tested based on the basic information (for example, environmental information) of the GUI screenshot group to be tested and the difference detection result. .

Exemplarily, please refer to Table 1, in which Table 1 shows the compatibility of Google Chrome under the operating system environment corresponding to the GUI screenshot 1 to be tested, the screenshot 2 of the GUI to be tested, and the screenshot 3 of the GUI to be tested. Test Results.

Table 1

Screenshot of the GUI to be tested Compatibility Test Results Screenshot of GUI to be tested 1: Google Chrome 5.0 for Windows There are pixel differences Screenshot of GUI to be tested 2: Google Chrome 5.0 on Linux system There is a positional difference for page element A Screenshot 3 of GUI to be tested: Google Chrome 5.0 on Macos no difference

In a possible embodiment, when the page difference detection system 100 is provided to the user in the form of an application program interface, the page difference detection system 100 can display the compatibility test result through a command user interface (the IDE interface shown in FIG. 3A ). , or, a link to a document download showing compatibility test results.

In another possible embodiment, when the page difference detection system 100 is provided to the user in the form of a web interface, the page difference detection system 100 can display the compatibility test result to the user through a graphical user interface, for example, in the graphical user interface. The pages shown in Table 1; alternatively, the page difference detection system 100 may provide the user in the form of a document, and output a document download link on the graphical user interface.

Optionally, before the page difference detection system 100 extracts the page structure information and/or page semantic information in the page to be detected and the standard page based on the deep learning model, a GUI screenshot group (including a standard page GUI screenshot and a or multiple screenshots of the GUI to be tested) and perform basic image processing (eg, preprocessing such as grayscale, binarization, etc.). In this way, the image can be denoised, thereby effectively improving the recognition accuracy of page structure information and page semantic information.

Specifically, please refer to FIG. 17 . In FIG. 17 , after acquiring the page to be detected (screenshot 1 to be tested, screenshot 2 to be tested, etc.) and the standard page (screenshot of standard GUI), the page difference detection system 100 first takes a screenshot of the GUI After the group performs basic image processing, the page structure information and/or page semantic information of the pages to be detected and standard pages are extracted based on the deep learning model; further, based on the target detection mode selected by the user (ie any of the comparison modes), determine The first difference detection result between the standard GUI screenshot and the GUI screenshot to be tested.

It should be noted that the comparison mode shown in FIG. 17 can be divided into a preset mode and a custom mode. The preset mode may include the aforementioned pixel comparison mode, page element semantic comparison mode, and page layout comparison mode. The custom mode includes a user history template and a user-created template. The user history template is the historical comparison generated by the page difference detection system 100 according to the difference types that the user is concerned about when the user used the page difference detection system 100 to detect page differences last time. The pattern and the template newly set by the user are a new comparison pattern generated by the page difference detection system 100 according to the difference type concerned by the current user.

FIG. 18 provides a computing device. As shown in FIG. 18 , the computing device 1800 may specifically be used to implement the functions of the system 100 for detecting page differences in the embodiment shown in FIG. 1 above.

Computing device 1800 includes bus 1801 , processor 1802 , display 1803 , and memory 1804 . Communication between the processor 1802 , the memory 1804 and the display 1803 is through the bus 1801 . It should be understood that the computing device 1800 may not include the display 1803. For example, the computing device 1800 is a kind of background server, which is connected with other display devices through interfaces, so as to realize the function of displaying the interface through other display devices.

The bus 1801 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus or the like. The bus can be divided into address bus, data bus, control bus and so on. For ease of presentation, only one thick line is shown in FIG. 18, but it does not mean that there is only one bus or one type of bus.

The processor 1802 may be a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc. any one or more of the devices.

The display 1803 is an input/output (I/O) device. The device can display a graphical user interface on the screen for the user to perform corresponding operations. According to different manufacturing materials, the display 1803 can be classified into a liquid crystal display (LCD), an organic light emitting diode (OLED) display, and the like. Specifically, the display 1803 may display the code input by the user through the GUI.

Memory 1804 may include volatile memory, such as random access memory (RAM). The memory 1804 may also include non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, hard drive drive (HDD) or solid state drive (SSD) ).

Executable program codes are stored in the memory 1804, and the processor 1802 executes the executable program codes to execute the aforementioned page difference detection method.

In a possible embodiment, the processor 1802 executes the above program codes to control the display 1803 to receive user operations through a graphical user interface, and then controls the display 1803 to transmit the user operations to the processor 1802 through the bus 1801. operation, display the optional comparison mode through the graphical user interface, and determine the target detection mode according to the user operation, and perform page difference detection, and return the difference detection result to the user through the graphical user interface.

In another possible embodiment, the processor 1802 executes the above program code to control the display 1803 to receive program code input by the user through the command user interface (for example, a program including the URL of the webpage to be tested, the relevant parameters of the target detection mode) code), and then controls the display 1803 to transmit the program code input by the user to the processor 1802 through the bus 1801. The processor 1802 can determine to perform page difference detection according to the program code input by the user, and returns the difference detection result to the user through the command user interface.

In some possible implementations, the processor 1802 may also control other interfaces to receive user-input codes. Wherein, the other interface may be a microphone or the like. Specifically, the microphone can receive the code entered in the form of speech.

Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium may be any available medium that a computing device can store, or a data storage device such as a data center that contains one or more available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media (eg, solid state drives), and the like. The computer-readable storage medium includes instructions for instructing a computing device to execute the above-described page difference detection method applied to a page difference detection system.

The embodiments of the present application also provide a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computing device, all or part of the processes or functions described in the embodiments of the present application are generated.

The computer instructions may be stored in or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be transmitted over a wire from a website site, computer or data center. (eg coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg infrared, wireless, microwave, etc.) to another website site, computer or data center.

The computer program product may be a software installation package, and when any of the foregoing methods for detecting page differences is required, the computer program product may be downloaded and executed on a computing device.

The descriptions of the processes or structures corresponding to each of the above-mentioned drawings have their own emphasis, and for the parts that are not described in detail in a certain process or structure, reference may be made to the related descriptions of other processes or structures.

Claims (22)

1. a page difference detection method, is characterized in that, described method comprises: Get the page to be detected and the standard page; Process the to-be-detected page and the standard page to obtain first page structure information and/or first page element semantic information of the to-be-detected page, and second page structure information and/or of the standard page Semantic information of the second page element; Obtain the target detection mode determined by the user; According to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information, the The detection page performs difference detection to obtain a first difference detection result. 2. The method according to claim 1, wherein the target detection mode comprises one or more of the following modes: pixel comparison mode, page element semantic comparison mode, page layout comparison mode, automatic Define the alignment mode. 3 . The method according to claim 2 , wherein when the target detection mode determined by the user includes the page element semantic comparison mode, the difference detection is performed on the to-be-detected page, and the result is obtained: 4 . The first difference detection result, including: The first difference detection result is determined by comparing the semantic information of the first page element with the semantic information of the second page element. 4. The method according to claim 2, wherein the first page structure information comprises at least one of styles, positions or types of page elements in the to-be-detected page, and the second page structure information Including at least one of the style, position or type of page elements in the standard page; when the target detection mode determined by the user includes the page layout comparison mode, performing the detection on the page to be detected. Difference detection, obtain the first difference detection result, including: The first difference detection result is determined by comparing the first page structure information with the second page structure information. 5 . The method according to claim 2 , wherein when the target detection mode determined by the user includes the user-defined comparison mode, before the difference detection is performed on the pages to be detected, the method further includes: 6 . : Receive a first command or a first operation, and determine a difference type concerned by the user according to the first command or first operation; wherein the difference type includes one or more of pixels, page element semantics, and page structure ; The performing difference detection on the to-be-detected page to obtain a first difference detection result includes: Determine the first information corresponding to the difference type in the to-be-detected page, and the second information corresponding to the difference type in the standard page; determine the first information according to the first information and the second information A difference detection result. 6. The method according to any one of claims 1-5, wherein the page to be detected is a first graphical user interface GUI screenshot of the webpage to be tested, and the standard page is the standard of the webpage to be tested GUI screenshot. 7. The method according to claim 6, wherein the acquiring the page to be detected and the standard page comprises: Obtain the web page link input by the user and/or the hyperlink in the web page corresponding to the web page link; Based on the webpage link and/or the hyperlink, the first GUI screenshot and the standard GUI screenshot are generated. 8. The method according to any one of claims 1-7, characterized in that, after performing difference detection on the to-be-detected page and obtaining a first difference detection result, the method further comprises: The first difference detection result is output, where the first difference detection result includes at least one item of pixel difference information, structural difference information, and semantic difference information of the page to be tested. 9. The method according to any one of claims 1-8, wherein the processing of the page to be detected and the standard page is performed to obtain the first page structure information and/or the first page structure of the page to be detected. Or the semantic information of the first page element, and before the second page structure information and/or the semantic information of the second page element of the standard page, further include: Train GUI screenshots of massive web pages in different environments to obtain a deep learning model; Wherein, the input of the deep learning model is any GUI screenshot, and the output is the page structure information and/or page element semantic information in the any GUI screenshot; The processing of the to-be-detected page and the standard page is performed to obtain first page structure information and/or first page element semantic information of the to-be-detected page, and second page structure information and/or second page structure information of the standard page. / or second page element semantic information, including: inputting the page to be detected into the deep learning model, and outputting the first page structure information and/or the first page element semantic information based on the deep learning model; and, The standard page is input into the deep learning model, and the second page structure information and/or the second page element semantic information is output based on the deep learning model. 10. The method according to any one of claims 1-9, wherein after performing difference detection on the to-be-detected page and obtaining a first difference detection result, the method further comprises: receiving a second operation; determining a first area image of the user's attention in the to-be-detected page according to the second operation, and determining a second area image corresponding to the first area image in the standard page area image; Determine a second difference detection result between the first area image and the second area image; the second difference detection result includes pixel difference information and structure of the first area image and the second area image At least one of difference information and semantic difference information; Based on the first difference detection result and the second difference detection result, a third difference detection result between the page to be detected and the standard page is determined, and the third difference detection result is output. 11. A system for detecting page differences, comprising: a processing unit, configured to acquire a page to be detected and a standard page, process the page to be detected and the standard page, and obtain first page structure information and/or first page element semantic information of the page to be detected, and Second page structure information and/or second page element semantic information of the standard page; an interface unit for acquiring the target detection mode determined by the user; The processing unit is further configured to detect the mode according to the target, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information, and perform difference detection on the to-be-detected page to obtain a first difference detection result. 12. The system according to claim 11, wherein the target detection mode comprises one or more of the following modes: pixel comparison mode, page element semantic comparison mode, page layout comparison mode, automatic Define the alignment mode. 13 . The system according to claim 12 , wherein when the target detection mode determined by the user includes the page element semantic comparison mode, the processing unit is configured to detect the page to be detected. 14 . When difference detection is performed and the first difference detection result is obtained, it is specifically used for: The first difference detection result is determined by comparing the semantic information of the first page element with the semantic information of the second page element. 14. The system according to claim 12, wherein the first page structure information comprises at least one of styles, positions or types of page elements in the to-be-detected page, and the second page structure information including at least one of the style, position or type of page elements in the standard page; When the target detection mode determined by the user includes the page layout comparison mode, when the processing unit is configured to perform difference detection on the to-be-detected page and obtain a first difference detection result, the processing unit is specifically configured to: The first difference detection result is determined by comparing the first page structure information with the second page structure information. 15 . The system according to claim 12 , wherein when the target detection mode determined by the user includes the user-defined comparison mode, the processing unit is configured to perform a process on the to-be-detected page. 16 . Before the difference detection, the interface unit is further configured to: receive a first command or a first operation; The processing unit is further configured to: determine the difference type concerned by the user according to the first command or the first operation; wherein the difference type includes one or more of pixels, page element semantics, and page structure; When the processing unit is used to perform difference detection on the to-be-detected page and obtain a first difference detection result, it is specifically used for: Determine the first information corresponding to the difference type in the to-be-detected page, and the second information corresponding to the difference type in the standard page; determine the first information according to the first information and the second information A difference detection result. 16. The system according to any one of claims 11-15, wherein the page to be detected is a screenshot of a first graphical user interface (GUI) of the web page to be tested, and the standard page is a standard of the web page to be tested GUI screenshot. 17. The system according to claim 16, wherein when the processing unit is used for the page to be detected and the standard page, it is specifically used for: obtaining the web page link input by the user and/or the hyperlink in the web page corresponding to the web page link; Based on the webpage link and/or the hyperlink, the first GUI screenshot and the standard GUI screenshot are generated. 18. The system according to any one of claims 11-17, wherein after the processing unit is configured to perform difference detection on the to-be-detected page and obtain a first difference detection result, The first difference detection result is output, where the first difference detection result includes at least one item of pixel difference information, structural difference information, and semantic difference information of the page to be tested. 19. The system according to any one of claims 11-18, wherein the processing unit is further configured to: Before being used to process the to-be-detected page and the standard page, a deep learning model is obtained by training GUI screenshots of massive web pages in different environments; wherein the input of the deep learning model is any GUI screenshot , the output is the page structure information and/or page element semantic information in any of the GUI screenshots; When the processing unit is used to process the to-be-detected page and the standard page, it is specifically used for: inputting the page to be detected into the deep learning model, and outputting the first page structure information and/or the first page element semantic information based on the deep learning model; and, The standard page is input into the deep learning model, and the second page structure information and/or the second page element semantic information is output based on the deep learning model. 20. The system according to any one of claims 11-19, wherein the interface unit is further configured to: after the processing unit performs difference detection on the to-be-detected page and obtains a first difference detection result, receive a second difference detection result. operate; The processing unit is further configured to: determine, according to the second operation, a first area image that the user is concerned about in the to-be-detected page, and determine, in the standard page, a first area image corresponding to the first area image the second area image; determine the second difference detection result between the first area image and the second area image; the second difference detection result includes the first area image and the second area image at least one of pixel difference information, structural difference information, and semantic difference information; based on the first difference detection result and the second difference detection result, determine the third difference between the to-be-detected page and the standard page Test results; The interface unit is further configured to output the third difference detection result. 21. A computing device, wherein the device comprises a processor and a memory; The processor is adapted to execute instructions stored in the memory to cause the computing device to perform the method of any one of claims 1 to 10. 22. A computer-readable storage medium comprising instructions which, when executed on a computing device, perform the method of any one of claims 1 to 10.

Images