CN114579420A

CN114579420A - Page difference detection method and system

Info

Publication number: CN114579420A
Application number: CN202011373863.5A
Authority: CN
Inventors: 夏博; 张芮恺; 宋亚龙
Original assignee: Huawei Cloud Computing Technologies Co Ltd
Current assignee: Huawei Cloud Computing Technologies Co Ltd
Priority date: 2020-11-30
Filing date: 2020-11-30
Publication date: 2022-06-03

Abstract

The application provides a page difference detection method and a page difference detection system. The page difference detection system can acquire the page to be detected, the standard page and a target detection mode determined by a user, then extracts page structure information and/or page element semantic information in the page to be detected and the standard page, and realizes difference detection of the page to be detected based on the target detection mode and the page structure information and/or page element semantic information in the page to be detected and the standard page. In the method, a user can independently select a difference detection mode, the diversified requirements of the user on page difference detection are effectively met, difference detection is carried out on the page to be detected based on page structure information and/or page element semantic information in the page to be detected and the standard page, background information is not needed, the accuracy of page difference detection is effectively improved, and the granularity range of page difference detection is effectively expanded.

Description

Page difference detection method and system

Technical Field

The present application relates to the field of computer technologies, and in particular, to a method and a system for detecting a page difference.

Background

After the software product is developed, page difference detection is usually performed on a Graphical User Interface (GUI) of the software product, that is, whether a page displayed in a different environment (for example, an environment combined by different operating systems, browsers, and devices) of the software product is different from a standard page of the software product is detected.

At present, two main modes are adopted for detecting page difference, and the mode 1 is as follows: determining the difference between the page to be detected and the standard page by detecting the difference of background information (such as code information) displayed by the same software product in the standard environment and the environment to be detected; the mode 2 is as follows: and determining the difference between the page to be detected and the standard page by detecting the pixel difference between the page to be detected and the standard page of the same software product. The mode 1 depends on background information, the acquisition difficulty is high, and the difference of background code information cannot effectively represent page difference, so that the difference detection result has low precision; mode 2 the detection mode is single (only pixel difference is detected), and user requirements cannot be met effectively.

Disclosure of Invention

The application provides a page difference detection method for improving the precision of page difference detection. The application also provides a system, a device, a computer readable storage medium and a computer program product corresponding to the method.

In a first aspect, a page difference detection method is provided, which may be performed by a page difference detection system. In the method, a page difference detection system acquires a page to be detected and a standard page, processes the page to be detected and the standard page, and acquires first page structure information and/or first page element semantic information of the page to be detected and second page structure information and/or second page element semantic information of the standard page; then, the page difference detection system can also acquire a target detection mode determined by a user; and performing difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information to obtain a first difference detection result.

In the technical scheme of the embodiment of the application, the page difference detection system can perform difference detection on the page to be detected according to a target detection mode determined by a user, first page structure information and/or first page element semantic information determined in the page to be detected, and second page structure information and/or second page element semantic information determined in a standard page. Therefore, the diversified requirements of the user on page difference detection are effectively met, and the precision of the page difference detection is effectively improved.

In one possible design, the target detection mode includes one or more of the following modes: the method comprises a pixel comparison mode, a page element semantic comparison mode, a page layout comparison mode and a user-defined comparison mode.

In the design, various comparison modes for page difference detection are provided, and the diversified page difference detection requirements of users are effectively met.

In a possible design, when the target detection mode determined by the user includes a page element semantic comparison mode, the page difference detection system performs difference detection on a page to be detected to obtain a first difference detection result, and the specific implementation manner may be: and comparing the semantic information of the first page element with the semantic information of the second page element to determine a first difference detection result.

In the design, the page difference detection system effectively expands the detection granularity of page difference detection by comparing the semantic information of the page elements of the page to be detected and the standard page, and further effectively improves the accuracy of page difference detection.

In one possible design, the first page structure information includes at least one of a style, a position, or a type of a page element in the page to be detected, and the second page structure information includes at least one of a style, a position, or a type of a page element in the standard page; when the target detection mode determined by the user includes the page layout comparison mode, the page difference detection system performs difference detection on the page to be detected to obtain a first difference detection result, which may be: comparing the first page structure information and the second page structure information to determine a difference detection result, and determining a first difference detection result.

The page difference detection system compares the first page structure information with the second page structure information to determine a first difference detection result, which includes, but is not limited to, the following three specific embodiments:

in the mode 1, the position difference degree of the first page element and the second page element in the page to be detected, which have the same style and type with at least one of the standard pages, is counted.

And 2, counting the style difference degree of at least one first page element and one second page element with the same position and type in the page to be detected and the standard page.

And 3, counting the type difference degree of at least one first page element and one second page element which have the same position and the same style in the page to be detected and the standard page.

It should be understood that the first page element may be any page element in the page to be detected, and the second page element is a page element corresponding to the any page element in the standard page. The positions of the first page element and the second page element in the modes 1 and 2 are the same, and may be completely the same or partially the same (for example, 80% of the positions are the same), and the embodiment of the present application is not particularly limited.

In the design, the page difference detection system determines the difference detection result by comparing the page structure information of the page to be detected and the standard page, so that the detection granularity of page difference detection is effectively enlarged, and the accuracy of page difference detection is further effectively improved.

In one possible design, when the target detection mode determined by the user comprises a custom comparison mode, the page difference detection system performs difference detection on a page to be detected to obtain a first difference detection result, can also receive a first command or a first operation, and determines a difference type concerned by the user according to the first command or the first operation; wherein the difference type comprises one or more of pixel, page element semantic and page structure; then the page difference detection system determines first information corresponding to the difference type in the page to be detected and second information corresponding to the difference type in the standard page; and determining a first difference detection result according to the first information and the second information.

In the design, a user can independently select concerned difference types, and the page difference detection system determines difference detection results by comparing first information and second information corresponding to the difference types in the page to be detected and the standard page, so that diversified difference detection requirements of the user are effectively met, and the page difference detection accuracy is effectively improved.

In one possible design, the page to be detected is a first GUI screenshot of the web page to be detected, and the standard page is a standard GUI screenshot of the web page to be detected.

It should be understood that the page to be checked for the first GUI screenshot may be a Graphical User Interface (GUI) screenshot of the web page to be checked in different environments, that is, the first GUI screenshot may be one screenshot or a set of multiple GUI screenshots, and herein, the first GUI screenshot is not specifically limited. An environment may be understood to be an environment in which one or more of an operating system, a hardware device, or a web browser and a version of the corresponding web browser are combined. The standard GUI screenshot may be a pre-designed GUI screenshot, or a GUI screenshot displayed on the web page to be tested in a user-specified environment (e.g., a google browser of a windows (windows) system).

In the design, the page difference detection system can detect the GUI screenshots of the webpage to be detected in different environments, and the page difference detection efficiency is effectively improved.

It should be noted that, the page difference detection system obtains the page to be detected and the standard page, and there are various implementation manners, including but not limited to the following two:

in the mode 1, a first GUI screenshot (namely the GUI screenshot to be tested) and a standard GUI screenshot uploaded by a user are received.

In the mode 2, a webpage link input by a user and/or a hyperlink in a webpage corresponding to the webpage link are/is acquired; based on the web page link and/or hyperlink, a first GUI screenshot and a standard GUI screenshot are generated.

In a possible design, the page difference detection system performs difference detection on the page to be detected, and after obtaining a first difference detection result, may also output the first difference detection result. The first difference detection result may include at least one of pixel difference information, structure difference information, and semantic difference information of the page to be detected.

In the design, the page difference detection system outputs a first difference detection result, so that a user can intuitively determine the difference between the page to be detected and the standard page.

In one possible design, the page difference detection system can process a page to be detected and a standard page through a deep learning model, and specifically, the page difference detection system can train GUI screenshots of massive webpages in different environments to obtain the deep learning model; the input of the deep learning model is any GUI screenshot, and the output is page structure information and/or page element semantic information in any GUI screenshot; after the deep learning model is obtained, the page difference detection system can input the page to be detected into the deep learning model, and output first page structure information and/or first page element semantic information based on the deep learning model; and inputting the standard page into the deep learning model, and outputting second page structure information and/or second page element semantic information based on the deep learning model.

In the design, the page difference detection system determines first page structure information and/or first page element semantic information in a page to be detected through a deep learning model, and determines second page structure information and/or second page element semantic information in a standard page, so that the granularity range and accuracy of page feature recognition are effectively improved.

In a possible design, the page difference detection system performs difference detection on the page to be detected, and after obtaining the first difference detection result, the page difference detection system may further: receiving a second operation; determining a first area image concerned by the user in the page to be detected according to the second operation, and determining a second area image corresponding to the first area image in the standard page; determining a second difference detection result between the first area image and the second area image; and then determining a third difference detection result of the page to be detected and the standard page based on the first difference detection result and the second difference detection result, and outputting the third difference detection result.

The second difference detection result comprises at least one of pixel difference information, structure difference information and semantic difference information of the first area image and the second area image.

In the design, on the basis that the page difference detection system performs difference detection on the page to be detected and the standard page based on the target detection mode determined by the user, the user can select to perform secondary detection on the local image in the page to be detected. Therefore, the requirement that the user wants to focus on the difference condition of the partial area can be met, and the user experience is effectively improved.

In a second aspect, a page difference detection system is provided, which may include:

the processing unit is used for acquiring a page to be detected and a standard page, processing the page to be detected and the standard page, and acquiring first page structure information and/or first page element semantic information of the page to be detected and second page structure information and/or second page element semantic information of the standard page;

the interface unit is used for acquiring a target detection mode determined by a user;

and the processing unit is further used for performing difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information to obtain a first difference detection result.

In a possible design, when the target detection mode determined by the user includes a page element semantic comparison mode, the processing unit is specifically configured to, when performing difference detection on a page to be detected and obtaining a first difference detection result:

and comparing the semantic information of the first page element with the semantic information of the second page element to determine a first difference detection result.

In one possible design, the first page structure information includes at least one of a style, a position, or a type of a page element in the page to be detected, and the second page structure information includes at least one of a style, a position, or a type of a page element in the standard page; when the target detection mode determined by the user includes the page layout comparison mode, the processing unit is specifically configured to, when performing difference detection on the page to be detected to obtain a first difference detection result:

and comparing the first page structure information with the second page structure information to determine a first difference detection result.

In a possible design, when the target detection mode determined by the user includes the custom comparison mode, before the processing unit is configured to perform difference detection on the page to be detected, the interface unit is further configured to: receiving a first command or a first operation; the processing unit is further configured to: determining a difference type concerned by a user according to a first command or a first operation; wherein the difference type comprises one or more of pixel, page element semantic and page structure; the processing unit is specifically configured to, when the processing unit is configured to perform difference detection on a page to be detected to obtain a first difference detection result: determining first information corresponding to the difference type in the page to be detected and second information corresponding to the difference type in the standard page; and determining a first difference detection result according to the first information and the second information.

In one possible design, the processing unit, when being used for the pages to be detected and the standard pages, is specifically configured to: acquiring a webpage link input by a user and/or a hyperlink in a webpage corresponding to the webpage link; based on the web page link and/or hyperlink, a first GUI screenshot and a standard GUI screenshot are generated.

In one possible design, the processing unit is configured to perform difference detection on the page to be detected, and output a first difference detection result after obtaining the first difference detection result, where the first difference detection result includes at least one of pixel difference information, structural difference information, and semantic difference information of the page to be detected.

In one possible design, before the processing unit is used for processing the page to be detected and the standard page, the processing unit is also used for training the GUI screenshots of the massive webpages in different environments to obtain a deep learning model; the input of the deep learning model is any GUI screenshot, and the output is page structure information and/or page element semantic information in any GUI screenshot; when the processing unit is configured to process the page to be detected and the standard page, the processing unit is specifically configured to: inputting a page to be detected into a deep learning model, and outputting first page structure information and/or first page element semantic information based on the deep learning model; and inputting the standard page into the deep learning model, and outputting second page structure information and/or second page element semantic information based on the deep learning model.

In one possible design, the interface unit is further configured to: performing difference detection on the page to be detected by the processing unit to obtain a first difference detection result, and then receiving a second operation; the processing unit is further configured to: determining a first area image concerned by the user in the page to be detected according to the second operation, and determining a second area image corresponding to the first area image in the standard page; determining a second difference detection result between the first area image and the second area image; the second difference detection result comprises at least one of pixel difference information, structure difference information and semantic difference information of the first area image and the second area image; determining a third difference detection result of the page to be detected and the standard page based on the first difference detection result and the second difference detection result; and the interface unit is also used for outputting a third difference detection result.

In a third aspect, a computing device is provided, illustratively, comprising a processor, a memory;

the processor is configured to execute instructions stored in the memory to cause the apparatus to perform a method as described in the first aspect or any possible design of the first aspect.

In a fourth aspect, there is provided a computer-readable storage medium comprising instructions which, when executed on a computing device, cause the computing device to perform the method recited in the first aspect or any possible design of the first aspect.

The technical effects that can be achieved by any one of the second aspect to the fourth aspect may specifically refer to the technical effects that can be achieved by any one of the second aspect to the fourth aspect in the first aspect, and may specifically refer to the technical effect description brought by any one of the possible designs in the first aspect, which is not described herein again.

Drawings

FIG. 1 is a diagram illustrating a system architecture suitable for use in an embodiment of the present application;

FIG. 2 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

FIG. 3A is a schematic diagram of a command user interface provided by an embodiment of the present application;

FIG. 3B is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

fig. 4 is a schematic flowchart of a page difference detection method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

FIG. 7 is a schematic view of a scenario of model training provided in an embodiment of the present application;

fig. 8 is a scene schematic diagram for acquiring feature information of a to-be-detected page and a standard page according to an embodiment of the present application;

FIG. 9 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

fig. 10 is a schematic diagram of a page difference detection process provided in the embodiment of the present application;

fig. 11 is a schematic diagram of a page difference detection process provided in the embodiment of the present application;

fig. 12 is a schematic diagram of a page difference detection process provided in the embodiment of the present application;

FIG. 13 is a schematic diagram of a graphical user interface provided by an embodiment of the present application;

FIG. 14 is a schematic illustration of a graphical user interface provided by an embodiment of the present application;

FIG. 15 is a schematic illustration of a graphical user interface provided by an embodiment of the present application;

FIG. 16 is a diagram illustrating a process for determining a compatibility test result according to an embodiment of the present application;

fig. 17 is a schematic diagram of a page difference detection process provided in the embodiment of the present application;

fig. 18 is a schematic structural diagram of a computing device according to an embodiment of the present application.

Detailed Description

The term "at least one" in the embodiments of the present application means one or more, "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of the singular or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein any of a, b, c, a-b, a-c, b-c, or a-b-c may comprise a single a, a single b, a single c, or a plurality of a, a plurality of b, and a plurality of c.

And, unless stated to the contrary, the embodiments of the present application refer to the ordinal numbers "first", "second", etc., for distinguishing a plurality of objects, and do not limit the sequence, timing, priority, or importance of the plurality of objects. For example, the first priority criterion and the second priority criterion are only used for distinguishing different criteria, and do not represent the difference of the content, priority, importance, and the like of the two criteria.

Furthermore, the terms "comprising" and "having" in the description of the embodiments and claims of the present application and the accompanying drawings are not exclusive. For example, a process, method, system, article, or apparatus that comprises a list of steps or modules is not limited to only those steps or modules listed, but may include other steps or modules/elements not listed.

With the continuous development of computer technology, the types of software products are more and more abundant. In order to ensure user experience, the GUI display consistency of software products in different environments is particularly important, so that there are urgent needs and broad prospects for detecting page display differences of software products in different environments.

The embodiment of the application provides a page difference detection method, which can be executed by a page difference detection system. The page difference detection system acquires a page to be detected, a standard page and a target detection mode determined by a user through a user interface such as a Graphical User Interface (GUI) or a Command User Interface (CUI), extracts page structure information and/or page element semantic information in the page to be detected and the standard page, realizes difference detection of the page to be detected based on the target detection mode and the page structure information and/or page element semantic information in the page to be detected and the standard page, and further acquires a difference detection result.

In the method, a user can independently select a difference detection mode, the diversified requirements of the user on page difference detection are effectively met, difference detection is carried out on the page to be detected based on page structure information and/or page element semantic information in the page to be detected and the standard page, background code information is not required to be relied on, the accuracy of page difference detection is effectively improved, and the granularity range of page difference detection is effectively expanded.

Referring to fig. 1, fig. 1 is a schematic diagram illustrating a system architecture applicable to an embodiment of the present application, and in fig. 1, a page difference detection system 100 may interact with a user to implement a page difference detection function.

It should be noted that the page difference detection system 100 may be provided for the user in the form of an application program interface, and may also be provided for the user in the form of a web interface.

As shown in fig. 1, the page difference detection system 100 includes an interface unit 101 and a processing unit 102.

And an interface unit 101 for interacting with a user.

In one possible embodiment, when the page difference detection system 100 is provided to the user for use in the form of an application program interface, the interface unit 101 may be a command user interface through which the page difference detection system 100 may receive a call command input by the user (e.g., receive program code containing object detection mode parameter information).

In another possible embodiment, when the page difference detection system 100 is provided to the user in the form of a web interface, the interface unit 101 may be a graphical user interface, and the page difference detection system 100 receives an operation (e.g., an operation of uploading a picture, an operation of selecting a target detection mode) that can be performed by the user through the graphical user interface.

The processing unit 102 may be configured to acquire a page to be detected and a standard page, process the page to be detected and the standard page, and acquire first page structure information and/or first page element semantic information of the page to be detected, and second page structure information and/or second page element semantic information of the standard page.

Further, the processing unit 102 may be further configured to perform difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information, so as to obtain a first difference detection result.

It should be understood that the page to be detected may be a Graphical User Interface (GUI) screenshot of a software product to be detected (e.g., a web page, an application (App), etc.) in different environments, that is, the page to be detected may be one or more GUI screenshots, and the embodiment of the present application is not limited specifically. An environment may be understood to be an environment in which one or more of an operating system, a hardware device, or a web browser and a version of the corresponding web browser are combined. The standard page is a standard GUI screenshot of the software product to be tested, and may specifically be a pre-designed GUI screenshot, or a GUI screenshot displayed by the software product to be tested in an environment specified by a user (e.g., a google browser of a windows operation system).

The processing unit 102 may include an obtaining module 1021, a training module 1022, a feature extraction module 1023, a difference detection module 1024, and a database 1025.

The obtaining module 1021 may be configured to obtain a page to be detected and a standard page, and obtain a target detection mode determined by a user.

In a possible embodiment, when the interface unit 101 receives a user operation or a call command, the obtaining module 1021 may determine the target detection mode selected by the user according to the user operation or the call command.

In a possible embodiment, when the interface unit 101 receives a Uniform Resource Locator (URL) (also called a web link) of a web page to be detected input by a user and/or a hyperlink in the web page corresponding to the URL, the obtaining module 1021 may be configured to generate a page to be detected and a standard page corresponding to the web page to be detected based on the web link, and store the page to be detected and the standard page in the database 1025.

In another possible embodiment, when the interface unit 101 receives at least two pictures uploaded by the user, the obtaining module 1021 may mark the at least two pictures uploaded by the user as a to-be-detected page and a standard page, respectively.

The training module 1022 may be configured to train the massive web page screenshots stored in the database 1025, or the GUI screenshots of the massive web pages in different environments, to obtain a deep learning model; the input of the deep learning model is any GUI screenshot in the massive webpage screenshots, and the output is page structure information and/or page element semantic information in any GUI screenshot.

The feature extraction module 1023 may be configured to input the page to be detected into the deep learning model, and output first page structure information and/or first page element semantic information of the page to be detected based on the deep learning model; and inputting the standard page into the deep learning model, and outputting second page structure information and/or second page element semantic information of the standard page based on the deep learning model.

The difference detection module 1024 may be configured to perform difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information, so as to obtain a first difference detection result.

It should be noted that the target detection mode may be: any one of the pixel comparison mode, the page element semantic comparison mode, the page layout comparison mode, and the user-defined comparison mode, which is not specifically limited herein. Therefore, the difference detection module 1024 has different implementation manners when it is used to implement difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, the second page structure information and/or the second page element semantic information.

In a possible embodiment, when the obtaining module 1021 determines that the target detection mode selected by the user includes the pixel comparison mode, the difference detecting module 1024 may determine the difference between the page to be detected and the standard page by counting the pixel difference at each same position in the page to be detected and the standard page.

In one possible embodiment, when the obtaining module 1021 determines that the target detection mode selected by the user includes the page element semantic comparison mode, the difference detecting module 1024 may determine the difference between the page to be detected and the standard page by comparing the first page element semantic information of the page to be detected and the second page element semantic information of the standard page.

In a possible embodiment, when the obtaining module 1021 determines that the target detection mode selected by the user includes the page layout comparison mode, the difference detection module 1024 may determine the difference between the page to be detected and the standard page by comparing the first page structure information and the second page structure information.

The first page structure information comprises at least one of the style, position or type of a page element in the page to be detected, and the second page structure information comprises at least one of the style, position or type of the page element in the standard page; the difference detection module 1024 determines a first difference detection result by comparing the first page structure information with the second page structure information, and the first difference detection result may be implemented in various ways, including but not limited to the following three ways:

In the embodiment of the present application, the parts of the page difference detection system 100 may be centrally deployed in a local computing device (a user device under direct control of a user, such as a user terminal like a laptop, a desktop, a smartphone, etc.) or a cloud computing cluster (including at least one cloud computing device, such as a cloud server, etc.). Of course, various portions of the page difference detection system 100 may also be deployed in a distributed manner in a cloud computing cluster.

In the following, a detailed process of interaction between the page difference detection system 100 and a user when the page difference detection system 100 is provided in different product forms for the user is described by taking a software product to be detected as a web page as an example and combining specific figures.

It should be noted that, when the page difference detection system 100 is provided to the user in the form of a web interface, the page difference detection system 100 provides the user with a graphical user interface in which the user can input information (e.g., pictures, URLs, etc.), and then the page difference detection system 100 can perform page difference detection based on the information received through the graphical user interface. In addition, in the graphical user interface provided by the page difference detection system 100, different user interfaces (such as the page difference detection interface shown in fig. 2, the web page setting interface shown in fig. 5, and the web page setting interface shown in fig. 5) may be displayed at different detection stages (before and after detection) of the page difference detection system 100. In this way, the page difference detection system 100 can be enabled to interact well with the user in the process of performing page difference detection.

Accordingly, when the page difference detection system 100 is provided to the user in the form of an application program interface, the page difference detection system 100 may interact with the user by commanding the user interface. It should be understood that the command user interface herein may specifically be an interface for a user to use an Integrated Development Environment (IDE) (e.g., IPython, Anaconda, etc.); alternatively, the third party interface invokes the user interface of the tool (e.g., Swagger Codegen).

In fig. 3A, an exemplary interface of the IDE is drawn, and a user may operate in the IDE through an external input/output device (such as a display, a keyboard, a mouse, and the like), for example, program code for uploading a GUI screenshot and a standard GUI screenshot to the page difference detection system 100, and program code for including a URL of a web page to be detected and/or a hyperlink to the page difference detection system 100 may be written.

In fig. 3B, an interface of the third-party interface invoking tool in the display is exemplarily drawn, a graphical user interface capable of interacting with the user is provided in the interface, the user may input the URL of the web page to be detected or the related parameter information of the target detection mode in the graphical user interface provided by the third-party interface invoking tool through an external input/output device (such as a display, a keyboard, a mouse, and the like), and the third-party interface invoking tool may perform difference detection on the GUI screenshot corresponding to the web page to be detected based on the URL of the web page to be detected and the related parameter information of the target detection mode (for example, a semantic comparison mode identifier and a semantic difference degree threshold), and return a difference detection result to the user through the graphical user interface.

For example, the user inputs "https:// arms-wct. huawei. com/v 1/composition/string" and related parameters of the target detection mode in the graphical user interface provided by the third-party interface calling tool shown in fig. 3B, and the third-party interface calling tool may call the page difference detection system 100 to perform difference detection on the GUI screenshot corresponding to the "https:// arms-wct. huawei.com/v 1/composition/string" and display the corresponding page difference detection result in the graphical user interface shown in fig. 3B.

The embodiment of the application does not limit the deployment manner of the IDE, for example, the IDE may be an application deployed in a cloud and is a cloud IDE, that is, the IDE may be deployed in an edge computing device system or a cloud computing system, and a user may implement an operation in the cloud IDE through an external input/output device on a local device. The IDE may also be a client IDE installed on the local terminal computing device, i.e., running directly on the local computing device.

Referring to fig. 4, fig. 4 is a schematic flowchart illustrating a page difference detection method according to an embodiment of the present application, where the method can be applied to a page difference detection system 100, and the method includes:

s401, the page difference detection system 100 obtains the page to be detected and the standard page.

It should be understood that the page to be detected may be one or more GUI screenshots (also referred to as GUI screenshots) of the web page to be detected, the standard page may be a GUI screenshot (also referred to as a standard GUI screenshot) of a standard page of the web page to be detected, specifically, a pre-designed GUI screenshot, or a GUI screenshot displayed by the web page to be detected in a user-specified environment (for example, a google browser of a window operation (windows) system).

In one possible design, the page difference detection system 100 is provided to the user in the form of a web interface, and the page difference detection system 100 obtains specific embodiments of the GUI screenshot to be detected and the standard GUI screenshot, including but not limited to the following two types:

in mode 1, the page difference detection system 100 may receive, through a graphical user interface (e.g., the page difference detection interface shown in fig. 2), a to-be-detected GUI screenshot and a standard GUI screenshot of a to-be-detected web page uploaded by a user.

In the mode 2, the page difference detecting system 100 may receive, through a graphical user interface (for example, the page difference detecting interface shown in fig. 2), the URL of the webpage to be detected and/or the hyperlink in the webpage corresponding to the URL of the webpage to be detected, which are input by the user, and then the page difference detecting system 100 may generate the GUI screenshot to be detected and the standard GUI screenshot of the webpage to be detected based on the URL and/or the hyperlink.

The hyperlink may be positioning information of a specific page element in a web page corresponding to the web page link. For example, assuming that the URL of the web page to be tested is a hundred-degree website, the hyperlink may be link information corresponding to "Baidu academic" in a page displayed by the hundred-degree website.

It should be noted that before the page difference detection system 100 generates the GUI screenshot to be detected and the standard GUI screenshot of the web page to be detected based on the URL and/or the hyperlink of the web page to be detected, the user may set the environment information and the page information of the web page to be detected.

After the page difference detection system 100 detects that the user inputs the URL of the web page to be detected (e.g., "https:// basic. com") in the page difference detection interface shown in fig. 5(a), the page difference detection interface may be controlled to jump to the web page setting interface shown in fig. 5 (b). The page difference detection system 100 can receive the environment setting information and the page setting information of the webpage to be detected from the user through the webpage setting interface.

Specifically, in fig. 5(b), in the environment setting function provided by the page difference detection system 100 through the web page setting interface, the user may perform setting of the operating system, setting of the browser and the browser version, and setting of the screenshot application.

Illustratively, the operating system may be set to any of windows, linux, macos, android (android), and the like; the screenshot application can be set as a screenshot application carried by a browser or an operating system; the browser settings may be set to the browser type of google, firefox, etc., and additionally, version information (e.g., latest version, historical version) of these browsers may be set. Optionally, in an environment setting function provided by the page difference detection system 100 through the web page setting interface, the user may also set network agent information in the environment setting function.

Specifically, in fig. 5(b), in the web page setting interface provided by the page difference detection system 100, the user can set the resolutions of the page to be detected and the standard page (i.e. the width and height of the page), and whether to display the notification reminder, etc.

In order to improve the efficiency of page difference detection, a default setting option is further provided in the webpage setting interface shown in fig. 5(b), and after the page difference detection system 100 detects a click operation selected for the default setting, environment setting information, page setting information, and the like can be automatically configured for the webpage to be detected without manual input by the user. Therefore, user experience is effectively improved.

Optionally, the page difference detecting system 100 may record the environment setting information and the page setting information used by the user each time, and generate history setting information in default settings, and when the user uses the page difference detecting system 100 next time, after selecting a default setting option in the webpage setting interface, the page difference detecting system 100 may display a default setting interface as shown in fig. 6, and display selectable items such as the original setting, the history setting 1, the history setting 2, and the like in the default setting interface.

Further, after the page difference detection system 100 detects a click operation with respect to any one of the original setting, the history setting 1, and the history setting 2, information input by a user may be received, and specific parameter information (environment information, page setting information) in any one may be updated based on the information. For example, after the page difference detection system 100 determines that the user selects the history setting 2, the environment information input by the user may be received to update the environment setting information in the history setting 2. Therefore, the efficiency of page difference detection can be effectively improved.

In another possible design, the page difference detection system 100 is provided to the user in the form of an application program interface, and the page difference detection system 100 obtains specific embodiments of the GUI screenshot to be detected and the standard GUI screenshot, including but not limited to the following two types:

in mode 1, the page difference detecting system 100 may receive, through a command user interface (e.g., an IDE interface shown in fig. 3A), a program code input by a user and including a GUI screenshot to be detected and a standard GUI screenshot, and obtain the GUI screenshot to be detected and the standard GUI screenshot of the web page to be detected based on the program code.

In the mode 2, the page difference detection system 100 may receive, through the command user interface, a program code that includes the URL of the web page to be detected and/or the hyperlink in the web page corresponding to the URL, and the environment setting information and the page setting information, and parse the URL to be detected and/or the hyperlink, the environment setting information and the page setting information from the program code, and generate the GUI screenshot to be detected and the standard GUI screenshot of the web page to be detected corresponding to the environment setting information and the page setting information based on the URL and/or the hyperlink.

Optionally, after the page difference detection system 100 generates the GUI screenshot to be detected and the standard GUI screenshot of the web page to be detected, the page difference detection system 100 may store the GUI screenshot to be detected and the standard GUI screenshot in a data storage module of the page difference detection system 100 in a certain encoding manner (for example, in a form of a character string).

S402, the page difference detection system 100 processes the page to be detected and the standard page to obtain first page structure information and/or first page element semantic information of the page to be detected and second page structure information and/or second page element semantic information of the standard page.

In one possible embodiment, the page difference detection system 100 may perform feature extraction on the page to be detected and the standard page by using a deep learning model, and obtain first page structure information and/or first page element semantic information of the page to be detected, and second page structure information and/or second page element semantic information of the standard page.

Accordingly, the page difference detection system 100 needs to obtain the deep learning model before processing the page to be detected and the standard page.

The manner in which the deep learning model is obtained is described below.

Specifically, referring to fig. 7, in fig. 7, the page difference detection system 100 may obtain a deep learning model by training a large amount of web page GUI screenshots; the input of the deep learning model can be any webpage GUI screenshot, and the output is page structure information and/or page element semantic information in any webpage GUI screenshot. The page structure information includes information such as position, type, style and the like of the page elements.

After the page difference detection system 100 obtains the deep learning model, feature extraction can be performed on the to-be-detected page and the standard page of the to-be-detected web page based on the deep learning model. As shown in fig. 8, the page difference detection system 100 inputs the page to be detected into the deep learning model, and may obtain the first page structure information and/or the first page element semantic information of the page to be detected; the page difference detection system 100 inputs the standard page into the deep learning model, and can obtain second page structure information and/or second page element semantic information of the standard page.

It should be understood that when the page to be detected is a GUI screenshot to be detected, the first page structure information and/or the first page element semantic information is the page structure information and/or the page element semantic information of the GUI screenshot to be detected; when the page to be detected is the multiple to-be-detected GUI screenshots, the first page structure information and/or the first page element semantic information are/is a set of page structure information and/or page element semantic information of the multiple to-be-detected GUI screenshots.

S403, the page difference detection system 100 acquires the target detection mode determined by the user.

In one possible design, the page difference detection system 100 is provided for the user in a form of a web interface, and the page difference detection system 100 may display a selective contrast mode interface as shown in fig. 9, where the selective contrast mode interface shows a pixel contrast mode, a page element semantic contrast mode, a page layout contrast mode, and a custom contrast mode; after the page difference detection system 100 detects a click operation for any comparison mode in the selected comparison mode interface, it determines the any comparison mode as a target detection mode.

In another possible design, the page difference detection system 100 is provided to the user in the form of an application program interface, and the page difference detection system 100 may receive program code containing relevant parameters of the target detection mode input by the user through a command user interface (e.g., an IDE interface shown in fig. 3A); based on the program code, a target detection mode is determined.

The related parameters may specifically be a difference type and a difference degree corresponding to the target detection mode. For example, when the target detection mode is the pixel comparison mode, the related parameter is the identifier of the pixel comparison mode, and the pixel difference ratio is 80%; for another example, when the target detection mode is the page layout comparison mode, the related parameters are the identifier of the page layout comparison mode and the position offset threshold of the page element; for another example, when the target detection mode is a page element semantic comparison mode, the related parameters are page layout comparison mode identifiers and the similarity degree of page element semantics; for another example, when the target detection mode is the custom comparison mode, the related parameters are one or more of pixels, page element semantics, and page structure information. Optionally, the related parameters may further include position information of a specific area of the page to be detected and/or identification information of a specific page element.

S404, the page difference detection system 100 performs difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information, so as to obtain a first difference detection result.

It should be noted that the target detection mode may be: any one of a pixel comparison mode, a page element semantic comparison mode, a page layout comparison mode and a user-defined comparison mode. Therefore, the target detection modes are different, and the specific implementation manner of the page difference detection system 100 for performing difference detection on the page to be detected is also different.

The difference detection process in different alignment modes is described in detail below.

(1) Pixel comparison mode

In the pixel comparison mode, the page difference detection system 100 may extract a pixel value of each pixel in the to-be-detected GUI screenshot and the standard GUI screenshot, sequentially compare pixel sizes of the to-be-detected GUI screenshot and the standard GUI screenshot at each same position, and further count pixel difference degrees of the to-be-detected GUI screenshot and the standard GUI screenshot, and when the pixel difference degree satisfies a preset condition, it may be considered that there is no pixel difference between the to-be-detected GUI screenshot and the standard GUI screenshot.

Illustratively, the preset condition is that the same proportion of pixels in the to-be-detected GUI screenshot and the standard GUI screenshot is greater than a preset proportion (for example, 80%), and after the page difference detection system 100 compares the sizes of pixels at the same positions in the to-be-detected GUI screenshot and the standard GUI screenshot, it is found that 90% of the pixels in the to-be-detected GUI screenshot and 90% of the pixels in the standard GUI screenshot are the same, and it is determined that there is no pixel difference between the to-be-detected GUI screenshot and the standard GUI screenshot.

(2) Page element semantic comparison mode

In the page element semantic comparison mode, the page difference detection system 100 may extract semantic information of each page element in the GUI screenshot to be detected and the standard GUI screenshot, sequentially count semantic difference degrees of at least one first page element in the GUI screenshot to be detected and at least one second page element in the standard GUI screenshot, and determine a first difference detection result according to the semantic difference degrees. The positions of each first page element in the at least one first page element in the to-be-detected GUI screenshot are the same as the positions of each second page element in the at least one second page element in the standard GUI screenshot in a one-to-one correspondence mode. The first page element or the second page element may be a text, a picture, a hyperlink, or the like, and embodiments of the present application are not particularly limited.

The page difference detection system 100 determines a first difference detection result according to the semantic difference degree, specifically, whether the semantic difference degree exceeds a preset threshold is judged, and if the semantic difference degree exceeds the preset threshold, it is determined that the first page element and the second page element have a page element semantic difference; and if the page element semantic difference does not exist, determining that the first page element and the second page element have no page element semantic difference.

For example, if the preset threshold is 2%, the first page element is a character, the character is specifically "i love China", and the character of the second page element corresponding to the first page element is "i love China", the page element semantic difference degree between the first page element and the second page element is 25%, and if the preset threshold is exceeded, the page element semantic difference exists between the first page element and the second page element, that is, the page element semantic difference exists between the GUI screenshot to be detected and the standard GUI screenshot.

For another example, assuming that the preset threshold is 2%, there are 100 characters in the first page element, and there are also 100 characters in the second page element, but there is a difference between 1 character in the second page element and the first page element, the page element semantic difference degree between the first page element and the second page element is 1%, and if the preset threshold is not exceeded, it is determined that there is no page element semantic difference between the first page element and the second page element, that is, there is no page element semantic difference between the to-be-detected GUI screenshot and the standard GUI screenshot.

Optionally, the page difference detecting system 100 determines the first difference detection result according to the semantic difference degree, and may also identify a first target object of the first page element and a second target object of the second page element, compare whether the first target object and the second target object are consistent, and if so, determine that the first page element and the second page element have no semantic difference of the page elements; if the first page element and the second page element are not consistent, the semantic difference of the page elements is considered to exist between the first page element and the second page element;

for another example, if the first page element and the second page element are pictures, the first target object in the first page element is a "cat", and the second target object in the second page element is a "dog", the first target object and the second target object are inconsistent, and it is determined that the first page element and the second page element have a page element semantic difference, that is, the page element semantic difference exists in the to-be-detected GUI screenshot and the standard GUI screenshot.

(3) Page layout comparison mode

In the page layout comparison mode, the page difference detection system 100 may extract page structure information in the to-be-detected GUI screenshot and the standard GUI screenshot, where the page structure information includes a type, a style, a position, and the like of a page element, which is not limited in the present application.

Three of these possible alignments are exemplified below:

in manner 1, the page difference detection system 100 counts the degree of difference (e.g., a total shift or a partial shift) between the position of a first page element in the GUI screenshot to be detected and a second page element in the standard GUI screenshot. And the style and the type of the first page element and the second page element are the same.

Example 1, referring to fig. 10, as shown in fig. 10(a), the page element a in the standard GUI screenshot includes page elements a1, a2, A3, the style and type of the page element B in the GUI screenshot 1 to be tested shown in fig. 10(B) are the same as those of the page element a, the page element B includes page elements B1, B2, and B3, and the page difference detection system 100 determines that there is a page element position difference between the GUI screenshot to be tested and the standard GUI screenshot by comparing the position of the page element B in the GUI screenshot 1 to be tested with the position of the page element a in the standard GUI screenshot and finding that there is an overall deviation between the page element B and the page element a.

Example 2, a style and a type of a page element C in the GUI screenshot 2 to be tested shown in fig. 10(C) are the same as those of a page element a in the standard GUI screenshot shown in fig. 10(a), where the page element C includes page elements C1, C2, and C3, and the page difference detection system 100 determines that a page element position difference exists between the GUI screenshot to be tested and the standard GUI screenshot by comparing a position of the page element C in the GUI screenshot 2 to be tested with a position of the page element a in the standard GUI screenshot and finding that the page element C is offset from the page element a in a partial element (i.e., C3 in fig. 10 (C)).

In mode 2, the page difference detection system 100 counts the style difference degree between the first page element in the GUI screenshot to be detected and the second page element in the standard GUI screenshot. And the positions and types of the first page element and the second page element are the same. It should be understood that, here, the positions of the first page element and the second page element may be identical or partially identical (for example, 80% of the positions are identical, depending on the threshold parameter set by the user), and the embodiment of the present application is not limited specifically. Wherein the style of the page element comprises one or more of size, color and shape of the page element.

Example 1, referring to fig. 11, a page element a in the standard GUI screenshot shown in fig. 11(a) and a page element B in the GUI screenshot 1 to be tested shown in fig. 11(B) are the same in position and type, and the page difference detection system 100 determines that the GUI screenshot 1 to be tested and the standard GUI screenshot have a style difference by comparing the style information of the page element a and the page element B and finding that the page element a and the page element B have a shape difference.

Example 2, the position and the type of the page element C in the GUI screenshot 2 to be detected shown in fig. 11(C) are the same as those of the page element a in the standard GUI screenshot shown in fig. 11(a), and the page difference detection system 100 determines that the GUI screenshot 2 to be detected and the standard GUI screenshot have a style difference if the page element a and the page element C have a size difference by comparing the style information of the page element a and the page element C.

Example 3, the page difference detection system 100 identifies that the color information of the first page element is blue and the color information of the second page element is red, and then the first page element and the second page element have a color difference; and the page difference detection system 110 determines that a pattern difference exists between the GUI screenshot to be detected and the standard GUI screenshot.

In mode 3, the page difference detection system 100 counts the type difference degree between the first page element in the GUI screenshot to be detected and the second page element in the standard GUI screenshot.

Specifically, the page difference detection system 100 counts whether the GUI screenshot to be detected and the standard GUI screenshot have page element missing, adding, deleting, and the like.

Example 1, please refer to fig. 12, page element a present in the standard GUI screenshot shown in fig. 12(a), which includes page elements a1, a 2; however, if there is no page element with the same position and the same style as the page element a in the GUI screenshot 1 to be detected shown in fig. 12(b), the page difference detection system 100 determines that there is a page element type difference between the GUI screenshot 1 to be detected and the standard GUI screenshot.

Example 2, please continue to refer to fig. 12, where the position and style of the page element C in the GUI screenshot 2 to be detected shown in fig. 12(C) are the same as those of the page element a, but the page element B still exists in the GUI screenshot 2 to be detected, the page difference detection system 100 determines that the GUI screenshot 2 to be detected and the standard GUI screenshot have a page element type difference.

Example 3, please continue to refer to fig. 12, where the positions and styles of the page element D and the page element a in the GUI screenshot 3 to be detected shown in fig. 12(D) are the same, but only a part of the page element D1 in the page element D is present, the page element part in the GUI screenshot 3 to be detected is considered to be missing, and the page difference detection system 100 determines that the GUI screenshot 3 to be detected and the standard GUI screenshot have a page element type difference.

(4) Custom comparison mode

In the user-defined comparison mode, the page difference detection system 100 may obtain a difference type concerned by a user through a graphical user interface, and determine first information corresponding to the difference type in a page to be detected and second information corresponding to the difference type in a standard page; and determining a first difference detection result according to the first information and the second information. The difference type concerned by the user can comprise one or more of pixel, page element semantic and page structure.

Referring to fig. 13, fig. 13 shows a difference type selection interface, in the difference type selection interface shown in fig. 13, a user may select a difference type of interest, such as a pixel, a page element semantic, a page structure, and the like, and when the page difference detection system 100 detects that the user places a cursor on the page structure, a page element style under the page structure, a position of the page element, a type of the page element, and a difference type of interest for the user to autonomously select may be further displayed.

Illustratively, when the page difference detection system 100 determines that the difference type concerned by the user is the style of the page element and the semantic of the page element, the first information extracted by the page difference detection system 100 in the to-be-detected GUI screenshot is the style information and the semantic information of the first page element, the second information is the style information and the semantic information of the second page element in the standard GUI screenshot, and the page difference detection system 100 determines the difference between the to-be-detected GUI screenshot and the standard GUI screenshot by comparing the style information and the semantic information of the first page element with the style information and the semantic information of the second page element. The second page element and the first page element have the same type and position, the first page element may specifically be any page element in the page to be detected, and the second page element may specifically be any page element in the page to be detected.

Further, as shown in fig. 14, in the custom comparison mode, the page difference detection system 100 may further display an interface for selecting a specific area or a specific element, where the user may further select a specific page element (e.g., a picture element or a text element) for difference detection; alternatively, the user may select a specific region of the page to be detected for the difference detection.

In the embodiment, the user can independently select the difference detection mode, so that the diversified requirements of the user on the page difference detection are effectively met, the difference detection is performed on the page to be detected based on the page structure information and/or the page element semantic information in the page to be detected and the standard page, the accuracy of the page difference detection is effectively improved, and the granularity range of the page difference detection is effectively expanded.

Optionally, after the page difference detection system 100 performs difference detection on the page to be detected and the standard page according to the target detection mode determined by the user to obtain the first difference detection result, the method may further include: receiving a second operation; determining a first area image concerned by the user in the page to be detected according to the second operation, and determining a second area image corresponding to the first area image in the standard page; determining a second difference detection result between the first area image and the second area image; the second difference detection result comprises at least one of pixel difference information, structure difference information and semantic difference information of the first area image and the second area image; and determining a third difference detection result of the page to be detected and the standard page based on the first difference detection result and the second difference detection result, and outputting the third difference detection result. Therefore, the requirement that whether the partial area in the page to be detected is focused by the user and has detail difference with the corresponding area in the standard page can be effectively met, and the user experience is effectively improved.

In a possible embodiment, when the page difference detecting system 100 is provided for a user to use in a web interface, the page difference detecting system 100 may display a schematic diagram of a first difference detection result interface as shown in fig. 15, where the user may select to directly output a first difference detection result, or select to detect a local image of a page to be detected again, and then the page difference detecting system 100 may receive, through the first difference detection result interface shown in fig. 15, position information of the local image input by the user and a new target detection mode, determine, based on the position information of the local image, a first area image focused by the user in the page to be detected, and determine, in a standard page, a second area image corresponding to the first area image; and carrying out secondary detection on the first area image and the second area image based on the new target detection mode to obtain a second difference detection result.

Illustratively, after the page difference detection system 100 performs difference detection on the GUI screenshot to be detected and the standard GUI screenshot according to the semantic comparison mode of the page element to obtain a first difference detection result, the page difference detection system 100 detects that the user inputs the position information of the area a in the screenshot to be detected and selects the pixel comparison mode, the page difference detection system 100 counts the pixel difference between the area a and the area B corresponding to the area a in the standard GUI screenshot to obtain a second difference detection result, and a final difference detection result is obtained based on the first difference detection result and the second difference detection result.

In another possible embodiment, when the page difference detection system 100 is provided to the user in the form of an application program interface, the page difference detection system 100 may receive a program code containing the local image location information and the new object detection mode input by the user through a command user interface (e.g., an IDE interface shown in fig. 3A), and parse the program code to obtain the local image location information and the new object detection mode; determining a first area image concerned by a user in a page to be detected based on the position information of the local image, and determining a second area image corresponding to the first area image in a standard page; and carrying out secondary detection on the first area image and the second area image based on the new target detection mode to obtain a second difference detection result, and obtaining a final difference detection result based on the first difference detection result and the second difference detection result.

Optionally, the page difference detection system 100 performs difference detection on the page to be detected, and after obtaining a first difference detection result, may further output the first difference detection result, where the first difference detection result includes at least one of pixel difference information, page structure difference information, and page element semantic difference information of the web page to be detected in different environments. The page structure information includes types, styles, positions, and the like of page elements, which is not limited in the present application.

For example, the first difference detection result may be one or more of a semantic difference degree, a pixel difference degree, a position difference degree, a type difference degree, and a style difference degree of the page element. Therefore, the page difference result is materialized, and the user experience is effectively improved.

Optionally, referring to fig. 16, after the page difference detection system 100 obtains the difference detection result, the compatibility test result of the web page to be tested may be determined based on the basic information (e.g., environmental information) of the GUI screenshot group to be tested and the difference detection result.

For example, please refer to table 1, where table 1 shows a compatibility test result of the google browser of the web page to be tested in the operating system environment corresponding to the GUI screenshot 1 to be tested, the GUI screenshot 2 to be tested, and the GUI screenshot 3 to be tested.

TABLE 1

GUI screenshot to be tested	Results of compatibility testing
		Screenshot 1 of GUI to be tested: google browser 5.0 of Windows system	There is a pixel difference
And (3) screenshot 2 of the GUI to be tested: google browser 5.0 of linux system	There is a difference in the position of page element A
		Screenshot 3 of the GUI to be tested: google browser 5.0 of Macos system	There is no difference

In one possible embodiment, when the page difference detection system 100 is provided to the user in the form of an application program interface, the page difference detection system 100 may display the compatibility test result or a document download link displaying the compatibility test result by commanding the user interface (e.g., the IDE interface shown in FIG. 3A).

In another possible embodiment, when the page difference detection system 100 is provided to the user in the form of a web interface, the page difference detection system 100 may present the compatibility test result to the user through a graphical user interface, for example, a page shown in table 1 is displayed on the graphical user interface; alternatively, the page difference detection system 100 may be provided to the user in the form of a document, with a document download link being output at the graphical user interface.

Optionally, before the page difference detection system 100 extracts the page structure information and/or the page semantic information in the page to be detected and the standard page based on the deep learning model, basic image processing (for example, preprocessing such as graying, binarization and the like) may be performed on the GUI screenshot group of the webpage to be detected (including the standard page GUI screenshot and one or more GUI screenshots to be detected). Therefore, the noise reduction processing can be carried out on the image, and the identification precision of the page structure information and the page semantic information is effectively improved.

Specifically, please refer to fig. 17, in fig. 17, after acquiring a page to be detected (screenshot 1 to be detected, screenshot 2 to be detected, etc.) and a standard page (standard GUI screenshot), the page difference detection system 100 firstly performs basic image processing on a GUI screenshot group, and then extracts page structure information and/or page semantic information of the page to be detected and the standard page based on a deep learning model; and further determining a first difference detection result between the standard GUI screenshot and the GUI screenshot to be detected based on a target detection mode (namely any one of comparison modes) selected by the user.

It should be noted that the comparison mode shown in fig. 17 can be divided into a preset mode and a custom mode. The preset mode may include the pixel comparison mode, the page element semantic comparison mode, and the page layout comparison mode. The user-defined mode includes a user history template and a user newly-set template, the user history template is a history comparison mode generated by the page difference detection system 100 according to the difference type concerned by the user when the user uses the page difference detection system 100 to perform the page difference detection last time, and the user newly-set template is a new comparison mode generated by the page difference detection system 100 according to the difference type concerned by the current user.

FIG. 18 provides a computing device, such as that shown in FIG. 18, the computing device 1800 may be particularly useful for implementing the functionality of the page difference detection system 100 in the embodiment shown in FIG. 1 described above.

The computing device 1800 includes a bus 1801, a processor 1802, a display 1803, and memory 1804. Communication between the processor 1802, memory 1804, and display 1803 occurs via bus 1801. It is to be understood that computing device 1800 may not include display 1803 as well, for example: the computing device 1800 is a background server, which is connected to other display devices through interfaces to implement the function of displaying interfaces through other display devices.

The bus 1801 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 18, but this does not mean only one bus or one type of bus.

The processor 1802 may be any one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Micro Processor (MP), a Digital Signal Processor (DSP), and the like.

The display 1803 is an input/output (I/O) device. The device can display the graphical user interface on a screen for the user to perform corresponding operations. The display 1803 may be classified into a Liquid Crystal Display (LCD), an Organic Light Emitting Diode (OLED) display, and the like, according to a manufacturing material. Specifically, the display 1803 may display a code input by the user through the GUI.

The memory 1804 may include volatile memory (volatile memory), such as Random Access Memory (RAM). The memory 1804 may also include non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, a hard drive (HDD), or a Solid State Drive (SSD).

The memory 1804 stores executable program code that the processor 1802 executes to perform the aforementioned page difference detection method.

In a possible embodiment, the processor 1802 executes the program code to control the display 1803 to receive a user operation through the graphical user interface, then control the display 1803 to transmit the user operation to the processor 1802 through the bus 1801, and the processor 1802 may display an optional comparison mode through the graphical user interface according to the user operation, determine a target detection mode according to the user operation, perform page difference detection, and return a difference detection result to the user through the graphical user interface.

In another possible embodiment, the processor 1802 executes the program codes to control the display 1803 to receive the program codes input by the user (for example, the program codes including the URL of the web page to be detected and the relevant parameters of the target detection mode) through the command user interface, then control the display 1803 to transmit the program codes input by the user to the processor 1802 through the bus 1801, and the processor 1802 may determine to perform the page difference detection according to the program codes input by the user and return the difference detection result to the user through the command user interface.

In some possible implementations, the processor 1802 may also control other interfaces to receive user-entered code. Wherein the other interface may be a microphone or the like. In particular, the microphone may receive a code input in the form of voice.

The embodiment of the application also provides a computer readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store or a data storage device, such as a data center, that contains one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk), among others. The computer-readable storage medium includes instructions that direct a computing device to perform the page difference detection method described above as applied to a page difference detection system.

The embodiment of the application also provides a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computing device, cause the processes or functions described in accordance with embodiments of the application to occur, in whole or in part.

The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website site, computer, or data center to another website site, computer, or data center by wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wirelessly (e.g., infrared, wireless, microwave, etc.).

The computer program product may be a software installation package that can be downloaded and executed on a computing device in the event that any of the aforementioned page difference detection methods need to be used.

The descriptions of the flows or structures corresponding to the above-mentioned drawings have their respective emphasis, and a part that is not described in detail in a certain flow or structure may refer to the related descriptions of other flows or structures.

Claims

1. A page difference detection method is characterized by comprising the following steps:

acquiring a page to be detected and a standard page;

processing the page to be detected and the standard page to obtain first page structure information and/or first page element semantic information of the page to be detected and second page structure information and/or second page element semantic information of the standard page;

acquiring a target detection mode determined by a user;

and performing difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information to obtain a first difference detection result.

2. The method of claim 1, wherein the target detection mode comprises one or more of the following modes: the method comprises a pixel comparison mode, a page element semantic comparison mode, a page layout comparison mode and a user-defined comparison mode.

3. The method according to claim 2, wherein when the target detection mode determined by the user includes the page element semantic comparison mode, the performing difference detection on the page to be detected to obtain a first difference detection result includes:

and comparing the semantic information of the first page element with the semantic information of the second page element, and determining the first difference detection result.

4. The method according to claim 2, wherein the first page structure information includes at least one of a style, a position, or a type of a page element in the page to be detected, and the second page structure information includes at least one of a style, a position, or a type of a page element in the standard page; when the target detection mode determined by the user includes the page layout comparison mode, performing difference detection on the page to be detected to obtain a first difference detection result, including:

and comparing the first page structure information with the second page structure information to determine the first difference detection result.

5. The method according to claim 2, wherein when the target detection mode determined by the user includes the custom comparison mode, before performing difference detection on the page to be detected, the method further includes:

receiving a first command or a first operation, and determining a difference type concerned by the user according to the first command or the first operation; wherein the difference type comprises one or more of pixel, page element semantic, and page structure;

the performing difference detection on the page to be detected to obtain a first difference detection result includes:

determining first information corresponding to the difference type in the page to be detected and second information corresponding to the difference type in the standard page; and determining the first difference detection result according to the first information and the second information.

6. The method according to any one of claims 1 to 5, wherein the page to be detected is a first Graphical User Interface (GUI) screenshot of a web page to be detected, and the standard page is a standard GUI screenshot of the web page to be detected.

7. The method according to claim 6, wherein the acquiring the page to be detected and the standard page comprises:

acquiring a webpage link input by the user and/or a hyperlink in a webpage corresponding to the webpage link;

and generating the first GUI screenshot and the standard GUI screenshot based on the webpage link and/or the hyperlink.

8. The method according to any one of claims 1 to 7, wherein after the difference detection is performed on the page to be detected and a first difference detection result is obtained, the method further comprises:

and outputting the first difference detection result, wherein the first difference detection result comprises at least one of pixel difference information, structure difference information and semantic difference information of the page to be detected.

9. The method according to any one of claims 1 to 8, wherein before the processing the page to be detected and the standard page to obtain the first page structure information and/or the first page element semantic information of the page to be detected and the second page structure information and/or the second page element semantic information of the standard page, the method further comprises:

training the GUI screenshots of massive webpages in different environments to obtain a deep learning model;

the input of the deep learning model is any GUI screenshot, and the output is page structure information and/or page element semantic information in any GUI screenshot;

the processing the page to be detected and the standard page to obtain the first page structure information and/or the first page element semantic information of the page to be detected and the second page structure information and/or the second page element semantic information of the standard page includes:

inputting the page to be detected into the deep learning model, and outputting the first page structure information and/or the first page element semantic information based on the deep learning model; and the number of the first and second groups,

and inputting the standard page into the deep learning model, and outputting the second page structure information and/or the second page element semantic information based on the deep learning model.

10. The method according to any one of claims 1 to 9, wherein after performing difference detection on the page to be detected and obtaining a first difference detection result, the method further comprises:

receiving a second operation; determining a first area image concerned by the user in the page to be detected according to the second operation, and determining a second area image corresponding to the first area image in the standard page;

determining a second difference detection result between the first area image and the second area image; the second difference detection result comprises at least one of pixel difference information, structural difference information and semantic difference information of the first area image and the second area image;

and determining a third difference detection result of the page to be detected and the standard page based on the first difference detection result and the second difference detection result, and outputting the third difference detection result.

11. A page difference detection system, comprising:

the processing unit is further configured to perform difference detection on the page to be detected according to the target detection mode, the first page structure information and/or the first page element semantic information, and the second page structure information and/or the second page element semantic information, so as to obtain a first difference detection result.

12. The system of claim 11, wherein the target detection mode comprises one or more of the following modes: the method comprises a pixel comparison mode, a page element semantic comparison mode, a page layout comparison mode and a user-defined comparison mode.

13. The system according to claim 12, wherein when the target detection mode determined by the user includes the page element semantic comparison mode, the processing unit is specifically configured to, when performing difference detection on the page to be detected to obtain a first difference detection result:

14. The system according to claim 12, wherein the first page structure information includes at least one of a style, a position, or a type of a page element in the page to be detected, and the second page structure information includes at least one of a style, a position, or a type of a page element in the standard page;

when the target detection mode determined by the user includes the page layout comparison mode, the processing unit is specifically configured to, when performing difference detection on the page to be detected and obtaining a first difference detection result:

15. The system according to claim 12, wherein when the target detection mode determined by the user includes the custom comparison mode, before the processing unit is configured to perform difference detection on the page to be detected, the interface unit is further configured to: receiving a first command or a first operation;

the processing unit is further to: determining a difference type concerned by the user according to the first command or the first operation; wherein the difference type comprises one or more of pixel, page element semantic, and page structure;

the processing unit is specifically configured to, when the processing unit is configured to perform difference detection on the page to be detected and obtain a first difference detection result:

16. The system according to any one of claims 11 to 15, wherein the page to be detected is a first GUI screenshot of a web page to be detected, and the standard page is a standard GUI screenshot of the web page to be detected.

17. The system according to claim 16, wherein the processing unit, when being used for the pages to be detected and the standard pages, is specifically configured to:

18. The system according to any one of claims 11-17, wherein said processing unit, after being configured to perform difference detection on said page to be detected and obtain a first difference detection result,

19. The system of any one of claims 11-18, wherein the processing unit is further configured to:

before the page to be detected and the standard page are processed, training GUI screenshots of massive webpages in different environments to obtain a deep learning model; the input of the deep learning model is any GUI screenshot, and the output is page structure information and/or page element semantic information in any GUI screenshot;

when the processing unit is configured to process the page to be detected and the standard page, the processing unit is specifically configured to:

20. The system of any of claims 11-19, the interface unit further to: after the processing unit carries out difference detection on the page to be detected and a first difference detection result is obtained, receiving a second operation;

the processing unit is further to: determining a first area image concerned by the user in the page to be detected according to the second operation, and determining a second area image corresponding to the first area image in the standard page; determining a second difference detection result between the first area image and the second area image; the second difference detection result comprises at least one of pixel difference information, structural difference information and semantic difference information of the first area image and the second area image; determining a third difference detection result of the to-be-detected page and the standard page based on the first difference detection result and the second difference detection result;

the interface unit is further configured to output the third difference detection result.

21. A computing device, the device comprising a processor, a memory;

the processor is to execute instructions stored in the memory to cause the computing device to perform the method of any of claims 1 to 10.

22. A computer-readable storage medium comprising instructions that, when executed on a computing device, cause the computing device to perform the method of any of claims 1 to 10.