CN113344824A - Interface image comparison method and device, computer equipment and storage medium - Google Patents

Interface image comparison method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN113344824A
CN113344824A CN202110734898.5A CN202110734898A CN113344824A CN 113344824 A CN113344824 A CN 113344824A CN 202110734898 A CN202110734898 A CN 202110734898A CN 113344824 A CN113344824 A CN 113344824A
Authority
CN
China
Prior art keywords
image
images
interface
compared
operating environments
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110734898.5A
Other languages
Chinese (zh)
Inventor
赵启航
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Asset Management Co Ltd
Original Assignee
Ping An Asset Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Asset Management Co Ltd filed Critical Ping An Asset Management Co Ltd
Priority to CN202110734898.5A priority Critical patent/CN113344824A/en
Publication of CN113344824A publication Critical patent/CN113344824A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G06T5/70
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management
    • G06F11/3684Test management for test design, e.g. generating new test cases
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1407General aspects irrespective of display type, e.g. determination of decimal point position, display with fixed or driving decimal point, suppression of non-significant zeros
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/451Execution arrangements for user interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/40Filling a planar surface by adding surface attributes, e.g. colour or texture
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/40Image enhancement or restoration by the use of histogram techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20112Image segmentation details
    • G06T2207/20132Image cropping

Abstract

The application relates to an interface image comparison method, an interface image comparison device, computer equipment and a storage medium. The method comprises the following steps: acquiring at least two interface images in different operating environments and acquiring image display information corresponding to the interface images; determining a scroll information display area in the interface image according to the image display information; denoising the interface image according to the rolling information display area to obtain images to be compared under different operating environments; when the images to be compared under different operating environments do not have integral displacement, acquiring pixel statistical data corresponding to the images to be compared; and carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result. By adopting the method, the accurate comparison can be realized by utilizing the pixel statistical data to carry out the similarity comparison, and the interface image similarity comparison result with high image comparison accuracy is obtained.

Description

Interface image comparison method and device, computer equipment and storage medium
Technical Field
The present application relates to the technical field of UI (User Interface) design, and in particular, to an Interface image comparison method, apparatus, computer device, and storage medium.
Background
With the development of computer technology, UI automation test technology has emerged, which includes UI automation regression test and UI automation compatibility test. Regression testing refers to re-testing after old code has been modified to confirm that the modification did not introduce new errors or cause errors in other code. Compatibility testing refers to testing whether a device can operate properly in different network environments on a particular platform, between different application software, on different operating system platforms, and the like.
In the UI automation test, the same display picture is often tested in different operating environments in the regression test or the compatibility test by using a picture comparison method. Conventional image contrast is based primarily on simple vector similarity to measure image similarity.
However, the conventional image contrast method almost requires that the size and resolution of the image and the elements in the image are completely consistent to be successfully compared, the same display picture under different operating environments in the UI automation test may have differences such as different resolutions, slight displacement without affecting the picture, and the like, and the conventional image contrast method has a problem of low contrast accuracy in the UI automation test.
Disclosure of Invention
In view of the above, it is desirable to provide an interface image contrast method, an apparatus, a computer device and a storage medium capable of improving image contrast accuracy.
A method of interface image comparison, the method comprising:
acquiring at least two interface images in different operating environments and acquiring image display information corresponding to the interface images;
determining a scroll information display area in the interface image according to the image display information;
denoising the interface image according to the rolling information display area to obtain images to be compared under different operating environments;
when the images to be compared under different operating environments do not have integral displacement, acquiring pixel statistical data corresponding to the images to be compared;
and carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result.
In one embodiment, the obtaining at least two interface images in different operating environments and the image display information corresponding to the interface images includes:
receiving a processing request carrying processing parameters;
determining corresponding processing logic according to the processing parameters, and acquiring at least two operating environments and test cases corresponding to the processing logic;
running the test case in a running environment, and respectively intercepting interface images in the running environment in the running process;
and acquiring image display information corresponding to the interface image.
In one embodiment, denoising the interface image according to the scroll information display area to obtain images to be compared in different operating environments includes:
acquiring a relative position of an area corresponding to a scroll information display area;
and generating a filling image according to the relative position of the region, and filling the relative position of the region through the filling image to obtain images to be compared under different operating environments.
In one embodiment, denoising the interface image according to the scroll information display area to obtain images to be compared in different operating environments includes:
acquiring a relative position of an area corresponding to a scroll information display area;
and cutting the interface image according to the relative position of the area to obtain images to be compared in different operating environments.
In one embodiment, when there is no overall displacement in the images to be compared in different operating environments, before obtaining the pixel statistical data corresponding to the images to be compared, the method further includes:
acquiring integral displacement information corresponding to an operating environment;
and judging whether the images to be compared have integral displacement under different operating environments according to the integral displacement information.
In one embodiment, further comprising:
when the images to be compared in different operating environments have overall displacement, acquiring overall displacement offset;
carrying out displacement adjustment on the image to be compared according to the integral displacement offset, and updating the image to be compared;
and returning to the step of acquiring the pixel statistical data corresponding to the image to be compared.
In one embodiment, before performing similarity comparison according to the pixel statistical data and a preset similarity threshold to obtain an interface image similarity comparison result, the method further includes:
obtaining a sample image set, and denoising sample images in the sample image set, wherein the sample images comprise at least two sample interface images which respectively correspond to different operating environments;
obtaining sample pixel data of a sample interface image in a denoised sample image;
carrying out similarity contrast on the sample interface image according to the sample pixel data to obtain the image similarity of the sample image;
and obtaining a similarity threshold according to the image similarity.
An interfacial image contrast apparatus, the apparatus comprising:
the acquisition module is used for acquiring at least two interface images in different operating environments and acquiring image display information corresponding to the interface images;
the processing module is used for determining a scrolling information display area in the interface image according to the image display information;
the denoising module is used for denoising the interface image according to the rolling information display area to obtain images to be compared under different operating environments;
the statistical module is used for acquiring pixel statistical data corresponding to the images to be compared when the images to be compared under different operating environments do not have integral displacement;
and the comparison module is used for carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result.
A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring at least two interface images in different operating environments and acquiring image display information corresponding to the interface images;
determining a scroll information display area in the interface image according to the image display information;
denoising the interface image according to the rolling information display area to obtain images to be compared under different operating environments;
when the images to be compared under different operating environments do not have integral displacement, acquiring pixel statistical data corresponding to the images to be compared;
and carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring at least two interface images in different operating environments and acquiring image display information corresponding to the interface images;
determining a scroll information display area in the interface image according to the image display information;
denoising the interface image according to the rolling information display area to obtain images to be compared under different operating environments;
when the images to be compared under different operating environments do not have integral displacement, acquiring pixel statistical data corresponding to the images to be compared;
and carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result.
The interface image comparison method, the device, the computer equipment and the storage medium can obtain the images to be compared in different operating environments for interface image comparison by obtaining at least two interface images in different operating environments and obtaining the image display information corresponding to the interface images, determining the rolling information display area in the interface images according to the image display information, denoising the interface images according to the rolling information display area, removing interference, obtaining the images to be compared in different operating environments for interface image comparison, further obtaining the pixel statistical data corresponding to the images to be compared when the images to be compared in different operating environments do not have integral displacement, comparing the similarity by using the pixel statistical data and the preset similarity threshold value, obtaining the interface image similarity comparison result, and the whole process can denoise the interface images by using the rolling information display area and can not have integral displacement of the images to be compared, and the accurate comparison is realized by utilizing the pixel statistical data to carry out similarity comparison, so that an interface image similarity comparison result with high image comparison accuracy is obtained.
Drawings
FIG. 1 is a schematic flow chart diagram illustrating an exemplary method for interface image comparison;
FIG. 2 is a diagram illustrating an exemplary embodiment of an application scenario for the interface image contrast method;
FIG. 3 is a schematic flowchart of an interface image comparison method according to another embodiment;
FIG. 4 is a block diagram of an interface image contrast device according to an embodiment;
FIG. 5 is a diagram illustrating an internal structure of a computer device according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In an embodiment, as shown in fig. 1, an interface image comparison method is provided, and this embodiment is illustrated by applying the method to a terminal, it is to be understood that the method may also be applied to a server, may also be applied to a system including a terminal and a server, and is implemented by interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices, and the server can be implemented by an independent server or a server cluster formed by a plurality of servers. In this embodiment, the method includes the steps of:
and 102, acquiring at least two interface images in different operating environments and acquiring image display information corresponding to the interface images.
The interface image refers to a screenshot of the same page when the test case runs in different running environments. For example, during the compatibility test, the interface image may specifically refer to a screenshot of the same page in which the test case runs in browsers of different versions. For another example, in the regression test, the interface image may specifically refer to a screenshot of the same page in which the test case runs in different system environments. The image display information is used for explaining element attributes of each element in the interface image and relative image coordinates in the interface image, wherein the element attributes comprise element display modes. For example, the element attribute may specifically refer to a static display, i.e., no scrolling or change occurs during the display process. For another example, the element attribute may specifically refer to a dynamic display, i.e., a scroll or a change may occur during the display process. For example, the dynamic display may be specifically a timestamp, a ticker, a carousel map, and the like. The operating environment corresponds to the processing logic, and may be different according to different processing logic, for example, when the processing logic is characterized by a compatibility test, the operating environment may specifically refer to multiple different versions of browsers. As another example, when the processing logic is characterized as a regression test, the operating environment may specifically refer to a different system environment.
Specifically, when interface image comparison is required, the terminal receives a processing request carrying processing parameters of a user, determines corresponding processing logic according to the processing parameters after receiving the processing request, acquires a test case corresponding to the processing logic and at least two operating environments, acquires at least two interface images in different operating environments according to the test case and the at least two operating environments, and acquires image display information corresponding to the interface images.
And 104, determining a scroll information display area in the interface image according to the image display information.
The scroll information display area is an area for displaying scrollable information. For example, the scrollable information may specifically refer to a timestamp, a ticker, a carousel map, and the like.
Specifically, the terminal determines the element attribute of each element in the interface image according to the image display information, screens out the dynamically displayed element according to the element attribute, positions the scroll information display area according to the dynamically displayed element, and acquires the area relative position of the scroll information display area in the interface image.
And 106, denoising the interface image according to the rolling information display area to obtain images to be compared in different operating environments.
The denoising of the interface image refers to removing interference of a rolling information display area. Therefore, the information in the scroll information display area is scroll-displayed, and is not necessarily identical at different times, which may interfere with the image contrast, and therefore, it is necessary to reduce the interference by processing the scroll information display area. The images to be compared under different operating environments refer to interface images subjected to denoising.
Specifically, the terminal denoises the interface image according to the relative position of the scroll information display area in the interface image, and obtains images to be compared in different operating environments. The interface image may be denoised by covering a new static image on the scroll information display area in the interface image, directly cutting the scroll information display area in the interface image, and the like.
And 108, when the images to be compared under different operating environments do not have integral displacement, acquiring pixel statistical data corresponding to the images to be compared.
The overall displacement refers to the situation that all elements in the images to be compared have the same offset due to different operating environments. When there is the whole displacement, need handle whole displacement earlier, contrast the image of waiting to compare under the different operational environment again, only when not having the whole displacement, just directly carry out the contrast the image of waiting to compare under the different operational environment. The pixel statistical data is used for describing pixel information in the image to be compared. For example, the pixel statistical data may specifically refer to a pixel histogram, where the pixel histogram is a list of the number of pixels in the image to be compared, and each pixel value in the image to be compared corresponds to one histogram element. If the image to be compared has a plurality of image channels, the pixel histograms corresponding to the image to be compared are the synthesis of the pixel histograms of all the image channels, and the pixel histograms of all the image channels representing the pixel points at the same position are connected.
Specifically, when the images to be compared under different operating environments do not have integral displacement, the terminal performs brightness equalization processing on the images to be compared, so that the brightness of the images to be compared is close to that of the images to be compared, and then obtains pixel statistical data corresponding to the images to be compared after the brightness equalization, so that the pixel statistical data are used for carrying out similarity comparison. For example, when the pixel statistical data is a pixel histogram, the terminal may directly obtain the pixel histogram of the image to be compared through a function histogram (). The manner of performing the brightness equalization processing may be set as needed, and this embodiment is not limited in detail here. For example, the brightness equalization processing may be performed on the image to be compared by writing a brightness equalization code in advance.
And 110, carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result.
Specifically, the terminal determines pixel difference values between pixels at the same position on different interface images according to the pixel statistical data, calculates an arithmetic mean of the sum of squares of the pixel difference values, takes the square root of the arithmetic mean as similarity, and compares the similarity with a preset similarity threshold to obtain an interface image similarity comparison result. Further, since the basis for the similarity comparison is the pixel difference, the smaller the similarity is, the more similar the interface images are, and the larger the similarity is, the larger the difference of the interface images is.
The interface image comparison method comprises the steps of obtaining at least two interface images under different operation environments, obtaining image display information corresponding to the interface images, determining a rolling information display area in the interface images according to the image display information, denoising the interface images according to the rolling information display area, removing interference, obtaining images to be compared under different operation environments for interface image comparison, further obtaining pixel statistical data corresponding to the images to be compared when the images to be compared under different operation environments do not have integral displacement, carrying out similarity comparison by utilizing the pixel statistical data and a preset similarity threshold value, obtaining an interface image similarity comparison result, and carrying out the whole process on the basis that the rolling information display area is utilized to denoise the interface images and the images to be compared do not have integral displacement, and the accurate comparison is realized by utilizing the pixel statistical data to carry out similarity comparison, so that an interface image similarity comparison result with high image comparison accuracy is obtained.
In one embodiment, the obtaining at least two interface images in different operating environments and the image display information corresponding to the interface images includes:
receiving a processing request carrying processing parameters;
determining corresponding processing logic according to the processing parameters, and acquiring at least two operating environments and test cases corresponding to the processing logic;
running the test case in a running environment, and respectively intercepting interface images in the running environment in the running process;
and acquiring image display information corresponding to the interface image.
The processing parameter refers to a processing identifier corresponding to the processing logic. For example, the processing parameter may specifically be a number 1, 2, 3, etc. for distinguishing different processing logics. The processing logic refers to a specific processing manner, for example, the processing logic may specifically refer to compatibility testing, regression testing, performance monitoring, and the like, where the performance monitoring mainly refers to WEB (World Wide WEB) performance monitoring, and specifically may perform WEB performance monitoring analysis by introducing httpwatch (WEB page data analysis tool) capture packets. The test case is a description of a test task performed on a specific software product, and embodies a test scheme, a method, a technology and a strategy. The contents of the test object, the test environment, the input data, the test steps, the expected results, the test scripts and the like are included, and finally, a document is formed.
Specifically, when interface image comparison is required, the terminal receives a processing request carrying processing parameters, after the processing request is received, the terminal determines corresponding processing logic according to the processing parameters, obtains at least two operating environments and test cases corresponding to the processing logic, operates the test cases in the operating environments, respectively intercepts the same page displayed in the operating environments in the operating process to obtain an interface image, and obtains image display information corresponding to the interface image after the interface image is obtained. The test cases are pre-stored in the database and can be directly acquired.
Specifically, according to different processing logics, the terminal acquires the operating environment in different ways. For example, when the processing logic is a compatibility test, the terminal may obtain different operating environments by starting different versions of browsers. When the processing logic is regression testing, the terminal can obtain different operating environments by starting different system environments.
It should be noted that, when the processing logic is performance monitoring, the terminal directly introduces the httwatch packet to perform WEB performance monitoring analysis, and does not need to acquire an interface image and perform interface image comparison. The environment for starting the browsers of different versions and the system environments for starting different versions may be started at the terminal itself, or may be started by remotely operating other terminals for the terminal, and this embodiment is not specifically limited herein.
In this embodiment, after determining the corresponding processing logic according to the processing parameter, at least two operation environments and test cases corresponding to the processing logic are obtained, the test cases are operated in the operation environments, the interface images are respectively captured in the operation environments during the operation process, and the image display information corresponding to the interface images is obtained, so that the interface images and the image display information can be obtained.
In one embodiment, denoising the interface image according to the scroll information display area to obtain images to be compared in different operating environments includes:
acquiring a relative position of an area corresponding to a scroll information display area;
and generating a filling image according to the relative position of the region, and filling the relative position of the region through the filling image to obtain images to be compared under different operating environments.
Wherein the relative position of the area is used for representing the relative position of the scroll information display area in the interface image. For example, the relative position of the region may specifically refer to the relative coordinates of the scroll information display region in the interface image.
Specifically, after the scroll information display area is determined, the terminal obtains the area relative position corresponding to the scroll information display area, determines the size of the scroll information display area according to the area relative position, generates a filling image with the same size according to the size of the scroll information display area, and fills the area relative position through the filling image to eliminate the interference of the scroll information display area on the interface image contrast, so as to obtain images to be compared under different operating environments. The filling image may be set as needed, as long as the interface image contrast is not disturbed, for example, the filling image may specifically be a solid image.
In this embodiment, denoising can be achieved by acquiring the relative position of the region corresponding to the scroll information display region, generating a filling image according to the relative position of the region, and filling the relative position of the region through the filling image.
In one embodiment, denoising the interface image according to the scroll information display area to obtain images to be compared in different operating environments includes:
acquiring a relative position of an area corresponding to a scroll information display area;
and cutting the interface image according to the relative position of the area to obtain images to be compared in different operating environments.
Specifically, in addition to the filling manner adopted in the above embodiment, after the scroll information display area is determined, the terminal may also perform denoising in a manner of obtaining the area relative position corresponding to the scroll information display area and cutting the interface image according to the area relative position, so as to eliminate interference of the scroll information display area on the contrast of the interface image, and obtain images to be contrasted in different operating environments.
In this embodiment, by obtaining the relative position of the region corresponding to the scroll information display region, the interface image is cut according to the relative position of the region, and denoising can be performed by direct cutting.
In one embodiment, when there is no overall displacement in the images to be compared in different operating environments, before obtaining the pixel statistical data corresponding to the images to be compared, the method further includes:
acquiring integral displacement information corresponding to an operating environment;
and judging whether the images to be compared have integral displacement under different operating environments according to the integral displacement information.
The overall displacement information is used for describing whether overall displacement exists between two different operation environments of all elements in the same display page, the overall displacement information can be obtained by operating the same sample case under different operation environments, the operation conditions of the same sample case are analyzed and compared by operating the same sample case under every two different operation environments in a large number, whether overall displacement exists between the different operation environments can be determined, and the overall displacement information is stored in a database in advance.
Specifically, on the basis of determining the operation environments, the terminal can directly acquire integral displacement information corresponding to the operation environments from the database, and can determine whether integral displacement exists between the operation environments or not through the integral displacement information.
In this embodiment, by acquiring the overall displacement information corresponding to the operating environment, whether the image to be compared has overall displacement in different operating environments can be determined according to the overall displacement information.
In one embodiment, further comprising:
when the images to be compared in different operating environments have overall displacement, acquiring overall displacement offset;
carrying out displacement adjustment on the image to be compared according to the integral displacement offset, and updating the image to be compared;
and returning to the step of acquiring the pixel statistical data corresponding to the image to be compared.
The overall displacement offset refers to displacement offsets of all elements in the same display page in different operating environments, and for example, the overall displacement offset may be specifically represented by a pixel distance.
Specifically, when the images to be compared under different operating environments have overall displacement, in order to eliminate interference caused by the overall displacement, the terminal extracts the overall displacement offset from the overall displacement information, performs displacement adjustment on all elements in the images to be compared according to the overall displacement offset, updates the images to be compared, and returns to the step of obtaining the pixel statistical data corresponding to the images to be compared to continue interface image comparison after the updating is completed.
In this embodiment, by obtaining the overall displacement offset, when the image to be compared under different operating environments has overall displacement, the image to be compared is subjected to displacement adjustment according to the overall displacement offset, so as to update the image to be compared, and then the step of obtaining the pixel statistical data corresponding to the image to be compared is returned to continue the interface image comparison.
In one embodiment, before performing similarity comparison according to the pixel statistical data and a preset similarity threshold to obtain an interface image similarity comparison result, the method further includes:
obtaining a sample image set, and denoising sample images in the sample image set, wherein the sample images comprise at least two sample interface images which respectively correspond to different operating environments;
obtaining sample pixel data of a sample interface image in a denoised sample image;
carrying out similarity contrast on the sample interface image according to the sample pixel data to obtain the image similarity of the sample image;
and obtaining a similarity threshold according to the image similarity.
The sample interface image refers to a screenshot of the same page when the sample test case runs in different running environments. For example, in the compatibility test, the sample interface image may specifically refer to a screenshot of the same page in which the test case runs in browsers of different versions. For another example, in the regression test, the sample interface image may specifically refer to a screenshot of the same page in which the test case runs in different system environments. The sample pixel data is used to describe pixel information in the sample interface image. For example, the sample pixel data may specifically refer to a pixel histogram.
Specifically, the terminal obtains a sample image set, denoises the sample images in the sample image set to remove noise interference, obtains sample pixel data of sample interface images in the denoised sample images, performs similarity contrast on the sample interface images by using the sample pixel data to obtain image similarity of the sample images, sorts the image similarity of the sample images, and selects the maximum image similarity as a similarity threshold value.
Specifically, denoising the sample image in the sample image set refers to denoising a sample interface image included in the sample image, where denoising mainly refers to removing interference of a scroll information display area. The terminal can obtain sample image display information corresponding to the sample interface image, position a corresponding rolling information display area according to the sample image display information, obtain an area relative position corresponding to the corresponding rolling information display area, and perform denoising in a mode of image filling or direct cutting according to the area relative position.
Specifically, the step of comparing the similarity of the sample interface images by using the sample pixel data to obtain the image similarity of the sample images includes determining the pixel difference between the pixels at the same position on different sample interface images by using the sample pixel data, calculating the arithmetic mean of the square sum of the pixel differences, and taking the square root of the arithmetic mean as the image similarity of the sample images.
In the embodiment, the sample image set is obtained, the sample images in the sample image set are denoised, the sample pixel data of the sample interface image in the denoised sample image is obtained, the sample interface image is subjected to similarity comparison according to the sample pixel data to obtain the image similarity of the sample image, and the similarity threshold can be determined by using the image similarity of the sample image.
In one embodiment, the interface image comparison method of the present application is illustrated by an application scene graph, as shown in fig. 2.
The terminal uses Jenkins (continuous integration tool for monitoring continuous and repeated work) continuous integration service, Job pulls the latest case (namely test case) at regular time, the main program is operated, when interface image comparison or monitoring is needed, the terminal receives a processing request carrying processing parameters, and corresponding processing logic (at least one of UI automatic compatibility test, UI automatic regression test and WEB performance monitoring) is determined according to the processing parameters.
When the processing logic is WEB performance monitoring, the terminal directly introduces an httpwatch packet to carry out WEB performance monitoring analysis.
When the processing logic is a UI automated compatibility test or a UI automated regression test, acquiring at least two operating environments (as shown in the figure, browser 1 and browser 2 in the UI automated compatibility test, environment 1 and environment 2 when an analytic system is tested in the UI automated regression test) corresponding to the processing logic and a test case (specifically, a latest case that may be pulled by regular Job), operating the test case in the operating environment, respectively intercepting an interface image in the operating environment during the operating process, acquiring image display information (using selenium (a tool for Web application test)) corresponding to the interface image, determining a scroll information display area in the interface image according to the image display information, denoising the interface image according to the scroll information display area, and acquiring images to be compared in different operating environments, the method comprises the steps of obtaining integral displacement information corresponding to operation environments, judging whether integral displacement exists in images to be compared in different operation environments or not according to the integral displacement information, obtaining integral displacement offset when integral displacement exists in the images to be compared in different operation environments, carrying out displacement adjustment on the images to be compared according to the integral displacement offset, updating the images to be compared, obtaining pixel statistical data corresponding to the images to be compared, directly obtaining pixel statistical data corresponding to the images to be compared when integral displacement does not exist in the images to be compared in different operation environments, carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value, obtaining an interface image similarity comparison result (image contrast), and generating a report according to the interface image similarity comparison result.
In one embodiment, as shown in fig. 3, the interface image comparison method of the present application is illustrated by a schematic flow chart, and specifically comprises the following steps:
step 302, receiving a processing request carrying processing parameters;
step 304, determining corresponding processing logic according to the processing parameters, and acquiring at least two operating environments and test cases corresponding to the processing logic;
step 306, running the test case in a running environment, and respectively intercepting interface images in the running environment in the running process;
308, acquiring image display information corresponding to the interface image;
step 310, determining a scroll information display area in the interface image according to the image display information;
step 312, denoising the interface image according to the rolling information display area to obtain images to be compared in different operating environments;
step 314, acquiring integral displacement information corresponding to the operating environment;
step 316, judging whether the images to be compared under different operation environments have integral displacement according to the integral displacement information, skipping to step 318 when the images to be compared under different operation environments have integral displacement, and skipping to step 322 when the images to be compared under different operation environments do not have integral displacement;
step 318, obtaining the integral displacement offset;
step 320, performing displacement adjustment on the image to be compared according to the integral displacement offset, and updating the image to be compared;
322, acquiring pixel statistical data corresponding to the image to be compared;
324, acquiring a sample image set, and denoising sample images in the sample image set, wherein the sample images comprise at least two sample interface images which respectively correspond to different operating environments;
step 326, obtaining sample pixel data of a sample interface image in the denoised sample image;
step 328, comparing the similarity of the sample interface image according to the sample pixel data to obtain the image similarity of the sample image;
and step 330, obtaining a similarity threshold according to the image similarity.
And 332, performing similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result.
It should be understood that, although the steps in the flowcharts related to the above embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in each flowchart related to the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.
In one embodiment, as shown in fig. 4, there is provided an interface image contrast apparatus including: an obtaining module 402, a processing module 404, a denoising module 406, a statistics module 408, and a comparison module 410, wherein:
an obtaining module 402, configured to obtain at least two interface images in different operating environments, and obtain image display information corresponding to the interface images;
a processing module 404, configured to determine a scroll information display area in the interface image according to the image display information;
the denoising module 406 is configured to denoise the interface image according to the rolling information display area to obtain images to be compared in different operating environments;
the statistical module 408 is configured to obtain pixel statistical data corresponding to the image to be compared when the image to be compared under different operating environments does not have overall displacement;
and a comparison module 410, configured to perform similarity comparison according to the pixel statistical data and a preset similarity threshold, so as to obtain an interface image similarity comparison result.
The interface image contrast device obtains at least two interface images under different operating environments, obtains image display information corresponding to the interface images, determines a rolling information display area in the interface images according to the image display information, can denoise the interface images according to the rolling information display area, removes interference, obtains images to be contrasted under different operating environments for interface image contrast, further can obtain pixel statistical data corresponding to the images to be contrasted when the images to be contrasted under different operating environments do not have integral displacement, performs similarity contrast by using the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity contrast result, and can denoise the interface images by using the rolling information display area on the basis that the images to be contrasted do not have integral displacement, and the accurate comparison is realized by utilizing the pixel statistical data to carry out similarity comparison, so that an interface image similarity comparison result with high image comparison accuracy is obtained.
In one embodiment, the obtaining module is further configured to receive a processing request carrying processing parameters, determine corresponding processing logic according to the processing parameters, obtain at least two operating environments and test cases corresponding to the processing logic, operate the test cases in the operating environments, respectively intercept the interface images in the operating environments during the operation, and obtain image display information corresponding to the interface images.
In one embodiment, the denoising module is further configured to obtain a relative position of the region corresponding to the scroll information display region, generate a filling image according to the relative position of the region, and fill the relative position of the region through the filling image to obtain images to be compared in different operating environments.
In one embodiment, the denoising module is further configured to obtain a relative position of an area corresponding to the scroll information display area, and crop the interface image according to the relative position of the area to obtain images to be compared in different operating environments.
In one embodiment, the statistical module is further configured to obtain overall displacement information corresponding to the operating environment, and determine whether the image to be compared has overall displacement in different operating environments according to the overall displacement information.
In an embodiment, the statistical module is further configured to, when the image to be compared in different operating environments has an overall displacement, obtain an overall displacement offset, perform displacement adjustment on the image to be compared according to the overall displacement offset, update the image to be compared, and return to the step of obtaining the pixel statistical data corresponding to the image to be compared.
In one embodiment, the comparison module is further configured to obtain a sample image set, denoise sample images in the sample image set, where the sample image includes at least two sample interface images, the at least two sample interface images correspond to different operating environments, obtain sample pixel data of the sample interface image in the denoised sample image, perform similarity comparison on the sample interface images according to the sample pixel data to obtain image similarity of the sample images, and obtain a similarity threshold according to the image similarity.
For specific embodiments of the interface image comparison apparatus, reference may be made to the above embodiments of the interface image comparison method, which are not described herein again. The modules in the interface image contrast device can be wholly or partially implemented by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 5. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement an interface image comparison method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
Those skilled in the art will appreciate that the architecture shown in fig. 5 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:
acquiring at least two interface images in different operating environments and acquiring image display information corresponding to the interface images;
determining a scroll information display area in the interface image according to the image display information;
denoising the interface image according to the rolling information display area to obtain images to be compared under different operating environments;
when the images to be compared under different operating environments do not have integral displacement, acquiring pixel statistical data corresponding to the images to be compared;
and carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result.
In one embodiment, the processor, when executing the computer program, further performs the steps of: receiving a processing request carrying processing parameters, determining corresponding processing logic according to the processing parameters, acquiring at least two running environments and test cases corresponding to the processing logic, running the test cases in the running environments, respectively intercepting interface images in the running environments in the running process, and acquiring image display information corresponding to the interface images.
In one embodiment, the processor, when executing the computer program, further performs the steps of: and acquiring a relative position of the area corresponding to the scroll information display area, generating a filling image according to the relative position of the area, and filling the relative position of the area through the filling image to obtain images to be compared under different operating environments.
In one embodiment, the processor, when executing the computer program, further performs the steps of: and acquiring the relative position of the area corresponding to the scroll information display area, and cutting the interface image according to the relative position of the area to obtain images to be compared under different operating environments.
In one embodiment, the processor, when executing the computer program, further performs the steps of: and acquiring integral displacement information corresponding to the operating environment, and judging whether integral displacement exists in the images to be compared in different operating environments according to the integral displacement information.
In one embodiment, the processor, when executing the computer program, further performs the steps of: and when the images to be compared in different operating environments have overall displacement, acquiring overall displacement offset, performing displacement adjustment on the images to be compared according to the overall displacement offset, updating the images to be compared, and returning to the step of acquiring pixel statistical data corresponding to the images to be compared.
In one embodiment, the processor, when executing the computer program, further performs the steps of: the method comprises the steps of obtaining a sample image set, denoising sample images in the sample image set, wherein the sample images comprise at least two sample interface images, the at least two sample interface images correspond to different operating environments respectively, obtaining sample pixel data of the sample interface images in the denoised sample images, comparing the similarity of the sample interface images according to the sample pixel data to obtain the image similarity of the sample images, and obtaining a similarity threshold value according to the image similarity.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring at least two interface images in different operating environments and acquiring image display information corresponding to the interface images;
determining a scroll information display area in the interface image according to the image display information;
denoising the interface image according to the rolling information display area to obtain images to be compared under different operating environments;
when the images to be compared under different operating environments do not have integral displacement, acquiring pixel statistical data corresponding to the images to be compared;
and carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result.
In one embodiment, the computer program when executed by the processor further performs the steps of: receiving a processing request carrying processing parameters, determining corresponding processing logic according to the processing parameters, acquiring at least two running environments and test cases corresponding to the processing logic, running the test cases in the running environments, respectively intercepting interface images in the running environments in the running process, and acquiring image display information corresponding to the interface images.
In one embodiment, the computer program when executed by the processor further performs the steps of: and acquiring a relative position of the area corresponding to the scroll information display area, generating a filling image according to the relative position of the area, and filling the relative position of the area through the filling image to obtain images to be compared under different operating environments.
In one embodiment, the computer program when executed by the processor further performs the steps of: and acquiring the relative position of the area corresponding to the scroll information display area, and cutting the interface image according to the relative position of the area to obtain images to be compared under different operating environments.
In one embodiment, the computer program when executed by the processor further performs the steps of: and acquiring integral displacement information corresponding to the operating environment, and judging whether integral displacement exists in the images to be compared in different operating environments according to the integral displacement information.
In one embodiment, the computer program when executed by the processor further performs the steps of: and when the images to be compared in different operating environments have overall displacement, acquiring overall displacement offset, performing displacement adjustment on the images to be compared according to the overall displacement offset, updating the images to be compared, and returning to the step of acquiring pixel statistical data corresponding to the images to be compared.
In one embodiment, the computer program when executed by the processor further performs the steps of: the method comprises the steps of obtaining a sample image set, denoising sample images in the sample image set, wherein the sample images comprise at least two sample interface images, the at least two sample interface images correspond to different operating environments respectively, obtaining sample pixel data of the sample interface images in the denoised sample images, comparing the similarity of the sample interface images according to the sample pixel data to obtain the image similarity of the sample images, and obtaining a similarity threshold value according to the image similarity.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. An interface image comparison method, characterized in that the method comprises:
acquiring at least two interface images in different operating environments and acquiring image display information corresponding to the interface images;
determining a scrolling information display area in the interface image according to the image display information;
denoising the interface image according to the rolling information display area to obtain images to be compared in different operating environments;
when the images to be compared under different operating environments do not have integral displacement, acquiring pixel statistical data corresponding to the images to be compared;
and carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result.
2. The method according to claim 1, wherein the obtaining at least two interface images in different operating environments and obtaining image display information corresponding to the interface images comprises:
receiving a processing request carrying processing parameters;
determining corresponding processing logic according to the processing parameters, and acquiring at least two operating environments and test cases corresponding to the processing logic;
running the test case in the running environment, and respectively intercepting interface images in the running environment in the running process;
and acquiring image display information corresponding to the interface image.
3. The method according to claim 1, wherein the denoising the interface image according to the scroll information display area to obtain images to be compared in different operating environments comprises:
acquiring a relative position of an area corresponding to the scroll information display area;
and generating a filling image according to the relative position of the region, and filling the relative position of the region through the filling image to obtain images to be compared under different operating environments.
4. The method according to claim 1, wherein the denoising the interface image according to the scroll information display area to obtain images to be compared in different operating environments comprises:
acquiring a relative position of an area corresponding to the scroll information display area;
and cutting the interface image according to the relative position of the area to obtain images to be compared in different operating environments.
5. The method according to claim 1, wherein before acquiring the pixel statistical data corresponding to the image to be compared when there is no overall displacement in the image to be compared in the different operating environments, the method further comprises:
acquiring integral displacement information corresponding to an operating environment;
and judging whether the images to be compared have integral displacement under different operating environments according to the integral displacement information.
6. The method of claim 1, further comprising:
when the images to be compared in different operating environments have overall displacement, acquiring overall displacement offset;
carrying out displacement adjustment on the image to be compared according to the integral displacement offset, and updating the image to be compared;
and returning to the step of acquiring the pixel statistical data corresponding to the image to be compared.
7. The method according to claim 1, wherein before performing similarity comparison according to the pixel statistical data and a preset similarity threshold to obtain an interface image similarity comparison result, the method further comprises:
obtaining a sample image set, and denoising sample images in the sample image set, wherein the sample images comprise at least two sample interface images which respectively correspond to different operating environments;
obtaining sample pixel data of a sample interface image in a denoised sample image;
carrying out similarity comparison on the sample interface image according to the sample pixel data to obtain the image similarity of the sample image;
and obtaining a similarity threshold according to the image similarity.
8. An interfacial image contrast apparatus, comprising:
the acquisition module is used for acquiring at least two interface images in different operating environments and acquiring image display information corresponding to the interface images;
the processing module is used for determining a rolling information display area in the interface image according to the image display information;
the denoising module is used for denoising the interface image according to the rolling information display area to obtain images to be compared under different operating environments;
the statistical module is used for acquiring pixel statistical data corresponding to the images to be compared when the images to be compared under different operating environments do not have integral displacement;
and the comparison module is used for carrying out similarity comparison according to the pixel statistical data and a preset similarity threshold value to obtain an interface image similarity comparison result.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.
CN202110734898.5A 2021-06-30 2021-06-30 Interface image comparison method and device, computer equipment and storage medium Pending CN113344824A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110734898.5A CN113344824A (en) 2021-06-30 2021-06-30 Interface image comparison method and device, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110734898.5A CN113344824A (en) 2021-06-30 2021-06-30 Interface image comparison method and device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
CN113344824A true CN113344824A (en) 2021-09-03

Family

ID=77481684

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110734898.5A Pending CN113344824A (en) 2021-06-30 2021-06-30 Interface image comparison method and device, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113344824A (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7917618B1 (en) * 2009-03-24 2011-03-29 Amazon Technologies, Inc. Monitoring web site content
JP2012103786A (en) * 2010-11-08 2012-05-31 Canon Marketing Japan Inc Test support device, control method, and program
WO2016075552A1 (en) * 2014-11-14 2016-05-19 Yandex Europe Ag Method of testing webpage layout
US20170031799A1 (en) * 2015-07-28 2017-02-02 Testplant Europe Ltd Method and apparatus for creating reference images for an automated test of software with a graphical user interface
CN106502891A (en) * 2016-10-19 2017-03-15 广州视源电子科技股份有限公司 The automatic testing method and device of user interface
US9898444B1 (en) * 2016-03-18 2018-02-20 Amazon Technologies, Inc. Image comparison for user interface testing
CN112711526A (en) * 2019-10-25 2021-04-27 腾讯科技(深圳)有限公司 UI test method, device, equipment and storage medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7917618B1 (en) * 2009-03-24 2011-03-29 Amazon Technologies, Inc. Monitoring web site content
JP2012103786A (en) * 2010-11-08 2012-05-31 Canon Marketing Japan Inc Test support device, control method, and program
WO2016075552A1 (en) * 2014-11-14 2016-05-19 Yandex Europe Ag Method of testing webpage layout
US20170031799A1 (en) * 2015-07-28 2017-02-02 Testplant Europe Ltd Method and apparatus for creating reference images for an automated test of software with a graphical user interface
US9898444B1 (en) * 2016-03-18 2018-02-20 Amazon Technologies, Inc. Image comparison for user interface testing
CN106502891A (en) * 2016-10-19 2017-03-15 广州视源电子科技股份有限公司 The automatic testing method and device of user interface
CN112711526A (en) * 2019-10-25 2021-04-27 腾讯科技(深圳)有限公司 UI test method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN109766256B (en) Method and device for testing H5 page performance in application program and computer equipment
CN107025174B (en) Method, device and readable storage medium for user interface anomaly test of equipment
CN111061526B (en) Automatic test method, device, computer equipment and storage medium
CN111309618B (en) Page element positioning method, page testing method and related devices
CN110889457B (en) Sample image classification training method and device, computer equipment and storage medium
CN103777939A (en) Method for measuring webpage first screen time
CN108304243B (en) Interface generation method and device, computer equipment and storage medium
CN103324521A (en) Method for measuring initial screen time of webpage
CN107807841B (en) Server simulation method, device, equipment and readable storage medium
CN110209978B (en) Data processing method and related device
CN110659092A (en) Webpage screenshot method and device, computer equipment and storage medium
CN110248235B (en) Software teaching method, device, terminal equipment and medium
CN114003160A (en) Data visualization display method and device, computer equipment and storage medium
CN110968375A (en) Interface control method and device, intelligent terminal and computer readable storage medium
US9075494B2 (en) Systems and methods for performing object selection
CN113344824A (en) Interface image comparison method and device, computer equipment and storage medium
CN115797267A (en) Image quality evaluation method, system, electronic device, and storage medium
CN112084444B (en) Page loading time detection method, device and computer readable storage medium
CN108519962B (en) Font display method and apparatus applied to android system, and terminal device
CN112052150A (en) Page loading time detection method, equipment, storage medium and device
CN112015378A (en) Skeleton screen page generation method and device, computer equipment and storage medium
CN112015650B (en) Event testing method and device based on computer vision
CN116797707B (en) Interface graphic rendering method and system based on operating system adaptation
CN111190574A (en) Method, device, equipment and storage medium for selecting options of multi-stage linkage assembly
CN116600137B (en) Video image compression storage or decompression method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination