CN113032071B - Page element positioning method, page testing method, device, equipment and medium - Google Patents
Page element positioning method, page testing method, device, equipment and medium Download PDFInfo
- Publication number
- CN113032071B CN113032071B CN202110320330.9A CN202110320330A CN113032071B CN 113032071 B CN113032071 B CN 113032071B CN 202110320330 A CN202110320330 A CN 202110320330A CN 113032071 B CN113032071 B CN 113032071B
- Authority
- CN
- China
- Prior art keywords
- page
- area
- image
- target
- initial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000012360 testing method Methods 0.000 title claims abstract description 46
- 238000012545 processing Methods 0.000 claims abstract description 14
- 238000010586 diagram Methods 0.000 claims description 30
- 238000000605 extraction Methods 0.000 claims description 9
- 238000009499 grossing Methods 0.000 claims description 6
- 230000008447 perception Effects 0.000 claims description 6
- 238000001545 Page's trend test Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 2
- 238000013473 artificial intelligence Methods 0.000 abstract description 7
- 238000013135 deep learning Methods 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000004590 computer program Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000009792 diffusion process Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000003631 expected effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000004927 fusion Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000002730 additional effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a page element positioning method, a page testing method, a page element positioning device, page testing equipment and a page element testing medium, which relate to the field of artificial intelligence, in particular to deep learning, image processing and testing technologies. The specific implementation scheme is as follows: determining an initial positioning area matched with an element template in a page image to be tested; in the page image, taking the central point of the initial positioning area as an initial point to perform area growth to the periphery so as to obtain a target positioning area matched with the element template; wherein the termination condition of the region growing is determined based on the difference between points on the boundary of the growing region and the initial point. According to the method and the device, the initial positioning area is determined through template matching in the page image to be tested, then the target positioning area is determined through area growth based on the center point of the initial positioning area, true machines with different resolutions can be adapted, and positioning efficiency and accuracy are improved.
Description
Technical Field
The application relates to the field of artificial intelligence, in particular to a deep learning, image processing and testing technology, and specifically relates to a page element positioning method, a page testing method, a device, equipment and a medium.
Background
The visual recall technique for the front-end automatic test refers to: by utilizing information carried on a visually relevant carrier (e.g., an image), it is achieved that whether the content of the static resource of the page block exists or accords with the expected effect is judged, and the use case which does not accord with the expected effect is recalled, for example: missing or cluttered key control element patterns, etc.
Currently, in the prior art, a region similar to a template is located from an image to be tested by a template matching method so as to verify whether the content of the region meets the expectations. However, the existing template matching method cannot accurately and efficiently locate the template and the image to be tested under the condition of different resolutions.
Disclosure of Invention
The application provides a page element positioning method, a page testing method, a device, equipment and a medium, so as to improve the efficiency and accuracy of page element positioning and page testing.
In a first aspect, the present application provides a method for positioning a page element, including:
determining an initial positioning area matched with an element template in a page image to be tested;
in the page image, taking the central point of the initial positioning area as an initial point to perform area growth to the periphery so as to obtain a target positioning area matched with the element template;
wherein the termination condition of the region growing is determined based on the difference between points on the boundary of the growing region and the initial point.
In a second aspect, the present application further provides a page testing method, including:
positioning a target area matched with the element template in a page image to be tested;
calculating the image similarity of the element template and the target area, and determining the test result of the page image according to the value of the image similarity;
the target area is located according to the page element locating method according to any embodiment of the application.
In a third aspect, the present application further provides a page element positioning device, including:
the initial positioning area determining module is used for determining an initial positioning area matched with the element template in the page image to be tested;
the target positioning area determining module is used for carrying out area growth to the periphery by taking the central point of the initial positioning area as an initial point in the page image to obtain a target positioning area matched with the element template;
wherein the termination condition of the region growing is determined based on the difference between points on the boundary of the growing region and the initial point.
In a fourth aspect, the present application further provides a page testing apparatus, including:
the target area positioning module is used for positioning a target area matched with the element template in the page image to be tested;
the test result acquisition module is used for calculating the image similarity between the element template and the target area and determining the test result of the page image according to the value of the image similarity;
the target area is located by the page element locating device according to any embodiment of the application.
In a fifth aspect, the present application further provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the page element locating method of any embodiment of the present application.
In a sixth aspect, the present application also provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the page element positioning method according to any embodiment of the present application.
In a seventh aspect, the present application further provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the page test method described in any of the embodiments of the present application.
In an eighth aspect, the present application also provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the page testing method according to any embodiment of the present application.
It should be understood that the description of this section is not intended to identify key or critical features of the application or to delineate the scope of the application. Other features of the present application will become apparent from the following description, and additional effects of the alternative described above will be apparent from the following description of the specific embodiments.
Drawings
The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
FIG. 1 is a flow diagram of a page element locating method according to an embodiment of the present application;
FIG. 2 is a flow diagram of a page element locating method according to an embodiment of the present application;
FIG. 3 is a flow diagram of a page test method according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a page element positioning device according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a page testing apparatus according to an embodiment of the present application;
fig. 6 is a block diagram of an electronic device for implementing a page element positioning method according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a flow chart of a page element positioning method according to an embodiment of the present application, where the embodiment may be applied to a case of positioning on a page image to be tested according to an element template when performing a page test, and relates to the field of artificial intelligence, and in particular, to deep learning, image processing and testing technologies. The method may be performed by a page element positioning device implemented in software and/or hardware, preferably configured in an electronic device, such as a computer device or a server. As shown in fig. 1, the method specifically includes the following steps:
s101, determining an initial positioning area matched with an element template in a page image to be tested.
The page image may be a page image of a mobile terminal to be tested, whether an element on an element template exists on the page image needs to be tested, whether the display of the element on the page image accords with an expected effect, and the like. Thus, the element is located on the page image. In the embodiment of the application, any template matching method in the prior art can be used for determining the initial positioning area matched with the element template in the page image.
S102, in the page image, taking the central point of the initial positioning area as an initial point to perform area growth to the periphery so as to obtain a target positioning area matched with the element template; wherein the termination condition of the region growing is determined according to the difference between each point on the boundary of the growing region and the initial point.
In the front-end automatic testing process, a large number of terminal devices of different models need to be tested, and the screen sizes and resolutions of the devices are different. Therefore, the initial positioning area determined according to the existing template matching method is not accurate.
In the embodiment of the application, on the basis of an initial positioning area, a center point is taken as an initial point, and a more accurate target positioning area matched with an element template is obtained in an area growing mode. The center point of the initial positioning area is the point with the highest matching quality, so that the area can be increased by taking the center point as an initial point. Specifically, the target positioning area can be determined according to the boundary of diffusion until the termination condition is met by diffusing around the initial point. The termination condition is determined according to the difference performance of each point on the boundary of the increased region and the initial point, that is, the difference performance of each point on the boundary of the region of the finally obtained target positioning region and the initial point needs to meet the set condition, wherein the difference performance of each point and the initial point can be determined according to the difference between the values of each point and each point on the X axis and the Y axis.
It should be noted that, because the existing template matching method is only suitable for the situation that the resolution of the element template image and the resolution of the page image to be detected are the same, if the resolutions are different, the target area cannot be accurately located. In the embodiment of the application, the center point of the initial positioning area is used as an initial point to perform area growth to the periphery, and a growing termination condition is set, so that the target positioning area can be obtained after the condition is reached. Therefore, the size of the final positioning area is not limited, and the growth is ended by setting the growth termination condition, so that the positioning area matched with the element template diagram can be accurately positioned no matter whether the page image to be detected is large or small in resolution.
According to the technical scheme, the initial positioning area is determined through template matching in the page image to be tested, then the target positioning area is determined through area growth based on the center point of the initial positioning area, true machines with different resolutions can be adapted, and positioning efficiency and accuracy are improved.
Fig. 2 is a flow chart of a page element positioning method according to an embodiment of the present application, where the embodiment is further optimized based on the foregoing embodiment. As shown in fig. 2, the method specifically includes the following steps:
s201, respectively extracting image features of the page image and the element template by utilizing a pre-trained feature extraction model.
For example, image feature extraction may be performed using a strategy of transfer learning, using vgg19 network structure trained on ImageNet dataset as feature extraction model. And the extracted features can be richer by adopting a multi-layer fusion strategy, so that the bottom texture features are more beneficial to the improvement of the performance of a template matching algorithm, for example, the features of the first layer and the third layer in the feature extraction model are selected for fusion.
S202, calculating the image similarity of the element template and each region in the page image according to the extracted image features.
Specifically, the size of the sliding window may be determined according to the size of the element template, and then the sliding window is moved on the page image, and for each movement, the image similarity between the element template image and the area corresponding to the sliding window is calculated, for example, the image similarity is determined by calculating the cosine distance between the image features.
S203, calculating a target area which maximizes the overall matching quality in the page image according to the image similarity by using a quality perception template matching method, and taking the target area as an initial positioning area.
The target area is the best matching area calculated by using a quality perception template matching method.
S204, determining a quality thermodynamic diagram of the page image according to the image similarity of the element template and each region in the page image.
Namely, the difference between the image similarity of different areas and the element template can be intuitively seen through the quality thermodynamic diagram.
S205, performing convolution smoothing operation on the mass thermodynamic diagram by using an average filter with the same size as the element template.
To filter out some of the noise and give a more intuitive feel, the mass thermodynamic diagram may be convolved with an averaging filter of the same size as the element template.
S206, in the page image, taking the central point of the initial positioning area as an initial point, diffusing the initial point to the periphery of the initial point by rectangles until the average value of the pixel values of each point on each side of the obtained target rectangle is smaller than the product of the pixel value of the central point and a preset threshold value, and taking the area corresponding to the target rectangle as the target positioning area.
The region growing operation can be performed according to a thermodynamic diagram after convolution smoothing. Specifically, the diffusion may be performed in a rectangle, and the end condition of the growth is set as: the average value of the pixel values of each point on each side of the target rectangle is smaller than the product of the pixel value of the center point and a preset threshold value. That is, the center point of the initial positioning area is taken as an initial point, the rectangle is diffused to the periphery of the initial positioning area, the diffused rectangle can be initialized in advance, then the diffusion is performed in a certain step length, the diffusion is stopped until the diffused rectangle meets the above condition, and the area corresponding to the obtained rectangle is the target positioning area. By means of constraint of the end conditions of region growing, page images of terminals to be detected with different resolutions can be matched with accurate positioning regions. And the test personnel are not required to test the equipment to be tested with different resolutions respectively, so that the test efficiency is improved.
S207, displaying the target positioning area in the mass thermodynamic diagram.
In the embodiment of the application, the final target positioning area is determined by adopting an area growing mode, and is not limited by the size of the element template and the positioning area, so that the element template in the embodiment of the application can not only comprise the template of the specific page element, but also comprise the template of the custom page area. The template of the specific page element refers to a template taking the page element as a main body, and only one page element is usually arranged on the template. The template of the custom page area refers to a test template custom-defined according to test requirements, the template can contain a plurality of elements, a region is formed by the plurality of elements, and positioning is performed on the page image to be tested aiming at the region.
According to the technical scheme, the initial positioning area matched with the element template is determined in the page image to be tested by using the quality perception template matching method, and the method can be applied even when the background of the page image is changed greatly. And then, according to the image similarity of the element template and each region in the page image, determining the quality thermodynamic diagram of the page image, and carrying out convolution smoothing operation on the quality thermodynamic diagram, so that partial noise is filtered out, and more visual feeling is given to people. Finally, based on the center point of the initial positioning area, the target positioning area is determined through area growth, so that the positioning method can be adapted to true machines with different resolutions, the positioning efficiency and accuracy are improved, and the positioning method is further suitable for positioning the self-defined page area and has a wider application range.
In addition, for the large graph at the PC end, a multi-scale space pyramid normalization correlation coefficient matching algorithm can be adopted for positioning, and the method specifically comprises the following steps:
1. constructing a multi-scale space according to the template map to obtain a plurality of template maps with different scales;
specifically, the template map can be scaled in equal proportion, or a K-means clustering algorithm is adopted to construct a multi-scale space;
2. extracting image features of the image to be detected and each template image with different scales;
3. and carrying out template matching on each template image and the image to be detected according to the extracted image characteristics by adopting a normalized correlation coefficient matching algorithm, and determining the region which is most similar to a certain template image in the image to be detected as an interested target region.
Fig. 3 is a flow chart of a page testing method according to an embodiment of the present application, where the embodiment may be applied to a case where positioning is performed on a page image to be tested according to an element template and then the page image is tested according to a positioning area, and relates to the field of artificial intelligence, and in particular, to deep learning, image processing and testing technologies. The method may be performed by a page testing apparatus implemented in software and/or hardware, preferably configured in an electronic device, such as a computer device or server. As shown in fig. 3, the method specifically includes the following steps:
s301, positioning a target area matched with the element template in the page image to be tested.
The target area is located according to the page element locating method according to any embodiment of the application. And will not be described in detail herein.
S302, calculating the image similarity of the element template and the target area, and determining the test result of the page image according to the value of the image similarity.
The larger the value of the image similarity is, the more similar the display of the element on the page image is to the template, the more the display of the element in the page image accords with the expectation, and the better the test effect is.
According to the technical scheme, the initial positioning area is determined through template matching in the page image to be tested, then the target positioning area is determined through area growth based on the center point of the initial positioning area, true machines with different resolutions can be adapted, positioning efficiency and accuracy are improved, and page testing efficiency and accuracy are improved.
Fig. 4 is a schematic structural diagram of a page element positioning device according to an embodiment of the present application, where the embodiment may be applied to a case of positioning on a page image to be tested according to an element template when performing a page test, and relates to the field of artificial intelligence, and in particular, to deep learning, image processing and testing technologies. The device can realize the page element positioning method in any embodiment of the application. As shown in fig. 4, the apparatus 400 specifically includes:
an initial positioning area determining module 401, configured to determine an initial positioning area in the page image to be tested, which is matched with the element template;
the target positioning area determining module 402 is configured to perform area growth around the page image with a center point of the initial positioning area as an initial point, so as to obtain a target positioning area matched with the element template;
wherein the termination condition of the region growing is determined based on the difference between points on the boundary of the growing region and the initial point.
Optionally, the target positioning area determining module is specifically configured to:
and in the page image, taking the central point of the initial positioning area as an initial point, diffusing the initial point to the periphery of the initial point by using a rectangle until the average value of the pixel values of each point on each side of the obtained target rectangle is smaller than the product of the pixel value of the central point and a preset threshold value, and taking the area corresponding to the target rectangle as the target positioning area.
Optionally, the initial positioning area determining module includes:
the image feature extraction unit is used for extracting image features of the page image and the element template respectively by utilizing a pre-trained feature extraction model;
the image similarity calculation unit is used for calculating the image similarity of each region in the element template and the page image according to the extracted image features;
the initial positioning area determining unit is used for calculating a target area which maximizes the overall matching quality in the page image according to the image similarity by using a quality perception template matching method, and taking the target area as the initial positioning area.
Optionally, the apparatus further comprises a thermodynamic diagram processing module, specifically configured to:
before the target positioning area determining module performs area growth, determining a quality thermodynamic diagram of the page image according to the image similarity between the element template and each area in the page image;
and performing convolution smoothing operation on the quality thermodynamic diagram by using an average filter with the same size as the element template.
Optionally, the apparatus further comprises:
and the target positioning area display module is used for displaying the target positioning area in the quality thermodynamic diagram.
Optionally, the element templates include templates of specific page elements and templates of custom page areas.
The page element positioning device 400 provided in the embodiment of the present application may execute the page element positioning method provided in any embodiment of the present application, and has the corresponding functional module and beneficial effects of the execution method. Reference may be made to the description of any method embodiment herein for details not described in this embodiment.
Fig. 5 is a schematic structural diagram of a page testing device according to an embodiment of the present application, where the embodiment may be applied to a case where positioning is performed on a page image to be tested according to an element template and then the page image is tested according to a positioning area, and relates to the field of artificial intelligence, and in particular, to deep learning, image processing and testing technologies. The device can realize the page testing method according to any embodiment of the application. As shown in fig. 5, the apparatus 500 specifically includes:
the target area positioning module 501 is configured to position a target area matched with the element template in the page image to be tested;
the test result obtaining module 502 is configured to calculate an image similarity between the element template and the target area, and determine a test result of the page image according to a value of the image similarity;
the target area is located by the page element locating device according to any embodiment of the application.
The page testing device 500 provided in the embodiment of the present application may execute the page testing method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. Reference may be made to the description of any method embodiment herein for details not described in this embodiment.
According to embodiments of the present application, there is also provided an electronic device, a readable storage medium and a computer program product.
Fig. 6 illustrates a schematic block diagram of an example electronic device 600 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 6, the apparatus 600 includes a computing unit 601 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 may also be stored. The computing unit 601, ROM 602, and RAM 603 are connected to each other by a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
Various components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, mouse, etc.; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the respective methods and processes described above, such as the page element positioning method. For example, in some embodiments, the page element locating method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM 602 and/or the communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the page element positioning method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the page element locating method in any other suitable way (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome. The server may also be a server of a distributed system or a server that incorporates a blockchain.
Furthermore, according to embodiments of the present application, there is provided another electronic device, another readable storage medium and another computer program product for performing one or more steps of the page testing method described in any of the embodiments of the present application. The specific structure and program code can be described in the context of the embodiment shown in fig. 6, and will not be described in detail herein.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.
Claims (16)
1. A method for locating page elements, comprising:
determining an initial positioning area matched with an element template in a page image to be tested;
in the page image, taking the central point of the initial positioning area as an initial point to perform area growth to the periphery so as to obtain a target positioning area matched with the element template;
wherein the termination condition of the region growing is determined according to the difference performance of each point on the boundary of the growing region and the initial point;
in the page image, the area growing is performed around by taking the central point of the initial positioning area as an initial point, so as to obtain a target positioning area matched with the element template, and the method comprises the following steps:
and in the page image, taking the central point of the initial positioning area as an initial point, diffusing the initial point to the periphery of the initial point by using a rectangle until the average value of the pixel values of each point on each side of the obtained target rectangle is smaller than the product of the pixel value of the central point and a preset threshold value, and taking the area corresponding to the target rectangle as the target positioning area.
2. The method of claim 1, wherein the determining an initial positioning area in the page image to be tested that matches the element template comprises:
respectively extracting image features of the page image and the element template by utilizing a pre-trained feature extraction model;
calculating the image similarity of each region in the element template and the page image according to the extracted image features;
and calculating a target area which maximizes the overall matching quality in the page image according to the image similarity by using a quality perception template matching method, and taking the target area as the initial positioning area.
3. The method of claim 2, wherein prior to performing the region growing, the method further comprises:
determining a quality thermodynamic diagram of the page image according to the image similarity of the element template and each region in the page image;
and performing convolution smoothing operation on the quality thermodynamic diagram by using an average filter with the same size as the element template.
4. A method according to claim 3, further comprising: and displaying the target positioning area in the mass thermodynamic diagram.
5. The method of claim 1, wherein the element templates include templates of specific page elements and templates of custom page regions.
6. A method of page testing, comprising:
positioning a target area matched with the element template in a page image to be tested;
calculating the image similarity of the element template and the target area, and determining the test result of the page image according to the value of the image similarity;
the target area is located according to the page element locating method as claimed in any one of claims 1 to 5.
7. A page element locating device, comprising:
the initial positioning area determining module is used for determining an initial positioning area matched with the element template in the page image to be tested;
the target positioning area determining module is used for carrying out area growth to the periphery by taking the central point of the initial positioning area as an initial point in the page image to obtain a target positioning area matched with the element template;
wherein the termination condition of the region growing is determined according to the difference performance of each point on the boundary of the growing region and the initial point;
the target positioning area determining module is specifically configured to:
and in the page image, taking the central point of the initial positioning area as an initial point, diffusing the initial point to the periphery of the initial point by using a rectangle until the average value of the pixel values of each point on each side of the obtained target rectangle is smaller than the product of the pixel value of the central point and a preset threshold value, and taking the area corresponding to the target rectangle as the target positioning area.
8. The apparatus of claim 7, wherein the initial positioning region determination module comprises:
the image feature extraction unit is used for extracting image features of the page image and the element template respectively by utilizing a pre-trained feature extraction model;
the image similarity calculation unit is used for calculating the image similarity of each region in the element template and the page image according to the extracted image features;
the initial positioning area determining unit is used for calculating a target area which maximizes the overall matching quality in the page image according to the image similarity by using a quality perception template matching method, and taking the target area as the initial positioning area.
9. The apparatus of claim 8, further comprising a thermodynamic diagram processing module, in particular for:
before the target positioning area determining module performs area growth, determining a quality thermodynamic diagram of the page image according to the image similarity between the element template and each area in the page image;
and performing convolution smoothing operation on the quality thermodynamic diagram by using an average filter with the same size as the element template.
10. The apparatus of claim 9, further comprising:
and the target positioning area display module is used for displaying the target positioning area in the quality thermodynamic diagram.
11. The apparatus of claim 7, wherein the element templates comprise templates of specific page elements and templates of custom page regions.
12. A page testing apparatus, comprising:
the target area positioning module is used for positioning a target area matched with the element template in the page image to be tested;
the test result acquisition module is used for calculating the image similarity between the element template and the target area and determining the test result of the page image according to the value of the image similarity;
wherein the target area is located by a page element locating device according to any of claims 7-11.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the page element locating method of any one of claims 1-5.
14. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the page element locating method of any one of claims 1-5.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the page test method of claim 6.
16. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the page testing method of claim 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110320330.9A CN113032071B (en) | 2021-03-25 | 2021-03-25 | Page element positioning method, page testing method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110320330.9A CN113032071B (en) | 2021-03-25 | 2021-03-25 | Page element positioning method, page testing method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113032071A CN113032071A (en) | 2021-06-25 |
| CN113032071B true CN113032071B (en) | 2024-02-06 |
Family
ID=76473778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110320330.9A Active CN113032071B (en) | 2021-03-25 | 2021-03-25 | Page element positioning method, page testing method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113032071B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113609028A (en) * | 2021-10-09 | 2021-11-05 | 统信软件技术有限公司 | Page testing method, computing device and readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111309618A (en) * | 2020-02-24 | 2020-06-19 | 广州市百果园信息技术有限公司 | Page element positioning method, page testing method and related device |
| WO2020233053A1 (en) * | 2019-05-22 | 2020-11-26 | 深圳壹账通智能科技有限公司 | Method, apparatus, device, and storage medium for making test page script |
-
2021
- 2021-03-25 CN CN202110320330.9A patent/CN113032071B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020233053A1 (en) * | 2019-05-22 | 2020-11-26 | 深圳壹账通智能科技有限公司 | Method, apparatus, device, and storage medium for making test page script |
| CN111309618A (en) * | 2020-02-24 | 2020-06-19 | 广州市百果园信息技术有限公司 | Page element positioning method, page testing method and related device |
Non-Patent Citations (2)
| Title |
|---|
| On Web Page extraction based on position of DIV;Xunhua Liu;IEEE;全文 * |
| 微细对接焊缝初始点定位方法;孙农亮;崔红星;梁少华;;光学精密工程(第03期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113032071A (en) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112800915B (en) | Building change detection method, device, electronic equipment and storage medium | |
| CN112560862B (en) | Text recognition method and device and electronic equipment | |
| CN112488060B (en) | Target detection method, device, equipment and medium | |
| CN112528858A (en) | Training method, device, equipment, medium and product of human body posture estimation model | |
| CN113205041A (en) | Structured information extraction method, device, equipment and storage medium | |
| CN113032071B (en) | Page element positioning method, page testing method, device, equipment and medium | |
| CN115359308A (en) | Model training method, apparatus, device, storage medium, and program for identifying difficult cases | |
| CN114445825A (en) | Character detection method and device, electronic equipment and storage medium | |
| CN112508005B (en) | Method, apparatus, device and storage medium for processing image | |
| CN113435257B (en) | Method, device, equipment and storage medium for identifying form image | |
| CN114972361A (en) | Blood flow segmentation method, device, equipment and storage medium | |
| CN114119990A (en) | Method, apparatus and computer program product for image feature point matching | |
| CN113610856A (en) | Method and device for training image segmentation model and image segmentation | |
| CN113362297B (en) | Image processing method, apparatus and storage medium for printed circuit board | |
| CN113361575B (en) | Model training method and device and electronic equipment | |
| CN114092739B (en) | Image processing method, apparatus, device, storage medium, and program product | |
| CN117746069B (en) | Graph searching model training method and graph searching method | |
| CN116258769B (en) | Positioning verification method and device, electronic equipment and storage medium | |
| CN114037865B (en) | Image processing method, apparatus, device, storage medium, and program product | |
| CN116128863B (en) | Medical image processing method, device and equipment | |
| CN113506368B (en) | Map data fusion method, map data fusion device, electronic device, map data fusion medium, and program product | |
| CN116664427A (en) | Image processing method, device, equipment and storage medium | |
| CN116596941A (en) | Image segmentation method, device, equipment and storage medium | |
| CN115841480A (en) | Image detection method and device, electronic equipment and storage medium | |
| CN116994038A (en) | Adjustment method and device of blood vessel classification model, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |