CN113032071A - Page element positioning method, page testing method, device, equipment and medium - Google Patents
Page element positioning method, page testing method, device, equipment and medium Download PDFInfo
- Publication number
- CN113032071A CN113032071A CN202110320330.9A CN202110320330A CN113032071A CN 113032071 A CN113032071 A CN 113032071A CN 202110320330 A CN202110320330 A CN 202110320330A CN 113032071 A CN113032071 A CN 113032071A
- Authority
- CN
- China
- Prior art keywords
- page
- area
- image
- positioning
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000012360 testing method Methods 0.000 title claims abstract description 47
- 238000012545 processing Methods 0.000 claims abstract description 14
- 238000010586 diagram Methods 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 17
- 238000000605 extraction Methods 0.000 claims description 10
- 238000009499 grossing Methods 0.000 claims description 7
- 230000008447 perception Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 239000000126 substance Substances 0.000 claims description 4
- 230000004807 localization Effects 0.000 claims 1
- 238000013473 artificial intelligence Methods 0.000 abstract description 7
- 238000013135 deep learning Methods 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 abstract description 6
- 238000004891 communication Methods 0.000 description 8
- 238000009792 diffusion process Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000001545 Page's trend test Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000003631 expected effect Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003064 k means clustering Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Computer Hardware Design (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a page element positioning method, a page testing device, a page element positioning medium and a page testing program product, relates to the field of artificial intelligence, and particularly relates to deep learning, image processing and testing technologies. The specific implementation scheme is as follows: determining an initial positioning area matched with the element template in the page image to be tested; in the page image, taking the central point of the initial positioning area as an initial point to perform area growth towards the periphery to obtain a target positioning area matched with the element template; wherein the termination condition of the region growing is determined according to the difference performance of each point on the growing region boundary and the initial point. According to the method and the device, the initial positioning area is determined in the page image to be tested through template matching, then the target positioning area is determined through area growth based on the central point of the initial positioning area, the real machine with different resolutions can be adapted, and the positioning efficiency and accuracy are improved.
Description
Technical Field
The present application relates to the field of artificial intelligence, and in particular, to a deep learning, image processing, and testing technique, and more particularly, to a page element positioning method, a page testing method, apparatus, device, medium, and program product.
Background
The visual recall technology of the front-end automatic test is as follows: by using the information carried on the visually relevant carrier (e.g. image), the effect of determining whether the content of the static resource of the page block exists or meets the expectation is achieved, and the use case which does not meet the expectation effect is recalled, for example: the missing of the key control or the disordered element style of the key control, etc.
At present, the prior art usually locates a region similar to the template from the image to be tested by a template matching method to verify whether the content of the region is in accordance with the expectation. However, the existing template matching method cannot accurately and efficiently position the template and the image to be tested under the condition of different resolutions.
Disclosure of Invention
The application provides a page element positioning method, a page testing method, a device, equipment, a medium and a program product, which are used for improving the efficiency and the accuracy of page element positioning and page testing.
In a first aspect, the present application provides a page element positioning method, including:
determining an initial positioning area matched with the element template in the page image to be tested;
in the page image, taking the central point of the initial positioning area as an initial point to perform area growth towards the periphery to obtain a target positioning area matched with the element template;
wherein the termination condition of the region growing is determined according to the difference performance of each point on the growing region boundary and the initial point.
In a second aspect, the present application further provides a page testing method, including:
positioning a target area matched with the element template in the page image to be tested;
calculating the image similarity of the element template and a target area, and determining the test result of the page image according to the value of the image similarity;
the target area is obtained by positioning according to the page element positioning method according to any embodiment of the application.
In a third aspect, the present application further provides a page element positioning apparatus, including:
the initial positioning area determining module is used for determining an initial positioning area matched with the element template in the page image to be tested;
a target positioning area determining module, configured to perform area growth around the central point of the initial positioning area as an initial point in the page image to obtain a target positioning area matched with the element template;
wherein the termination condition of the region growing is determined according to the difference performance of each point on the growing region boundary and the initial point.
In a fourth aspect, the present application further provides a page testing apparatus, including:
the target area positioning module is used for positioning a target area matched with the element template in the page image to be tested;
the test result acquisition module is used for calculating the image similarity of the element template and a target area and determining the test result of the page image according to the value of the image similarity;
the target area is obtained by positioning by the page element positioning device according to any embodiment of the present application.
In a fifth aspect, the present application further provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the page element positioning method of any embodiment of the present application.
In a sixth aspect, the present application further provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the page element positioning method according to any embodiment of the present application.
In a seventh aspect, the present application further provides a computer program product, including a computer program, where the computer program, when executed by a processor, implements the page element positioning method according to any embodiment of the present application.
In an eighth aspect, the present application further provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the page test method of any embodiment of the present application.
In a ninth aspect, the present application further provides a non-transitory computer readable storage medium having stored thereon computer instructions for causing a computer to execute the page testing method according to any of the embodiments of the present application.
In a tenth aspect, the present application further provides a computer program product comprising a computer program, which when executed by a processor, implements the page testing method according to any of the embodiments of the present application.
It should be understood that the statements herein do not intend to identify key or critical features of the present application, nor to limit the scope of the present application. Other features of the present application will become readily apparent from the following description, and other effects of the above alternatives will be described hereinafter in conjunction with specific embodiments.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is a schematic flowchart of a page element positioning method according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating a page element positioning method according to an embodiment of the present application;
FIG. 3 is a flowchart illustrating a page testing method according to an embodiment of the present application;
FIG. 4 is a schematic structural diagram of a page element positioning apparatus according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of a page testing apparatus according to an embodiment of the present application;
fig. 6 is a block diagram of an electronic device for implementing a page element positioning method according to an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a schematic flowchart of a page element positioning method according to an embodiment of the present application, which is applicable to a case where positioning is performed on a page image to be tested according to an element template when performing a page test, and relates to the field of artificial intelligence, and in particular to deep learning, image processing, and testing technologies. The method can be executed by a page element positioning apparatus, which is implemented by software and/or hardware, and is preferably configured in an electronic device, such as a computer device or a server. As shown in fig. 1, the method specifically includes the following steps:
s101, determining an initial positioning area matched with the element template in the page image to be tested.
The page image may be a page image of the mobile terminal to be tested, whether an element on the element template exists on the page image needs to be tested, whether the display of the element on the page image meets an expected effect, and the like. Therefore, the element is positioned on the page image. In the embodiment of the application, any template matching method in the prior art can be used for determining the initial positioning area matched with the element template in the page image.
S102, in the page image, taking the central point of the initial positioning area as an initial point to perform area growth towards the periphery to obtain a target positioning area matched with the element template; the termination condition of the region growing is determined according to the difference performance of each point on the growing region boundary and the initial point.
In the front-end automatic test process, a large number of terminal devices of different models need to be tested, and the screen sizes and resolutions of the devices are different. Therefore, the initial positioning region determined according to the existing template matching method is not accurate.
In the embodiment of the application, on the basis of the initial positioning area, the central point of the initial positioning area is used as an initial point, and a more accurate target positioning area matched with the element template is obtained in a region growing mode. The central point of the initial positioning region is the point with the highest matching quality, so that the region growth can be performed by taking the central point as the initial point. Specifically, diffusion can be performed around the initial point until a termination condition is met, and the target positioning area can be determined according to the diffusion boundary. The termination condition is determined according to the difference expression between each point on the growing area boundary and the initial point, that is, the difference expression between each point on the area boundary of the target positioning area and the initial point finally obtained needs to satisfy the setting condition, wherein the difference expression between each point and the initial point can be determined according to the difference between the point and the value on the X axis and the Y axis.
It should be noted that, because the existing template matching method is only suitable for the situation that the resolution of the element template map is the same as that of the page image to be detected, if the resolution is different, the target area cannot be accurately located. In the embodiment of the application, the central point of the initial positioning region is used as the initial point to perform region growth towards the periphery, the termination condition of the growth is set, and the target positioning region can be obtained after the condition is reached. Therefore, the size of the final positioning area does not need to be limited, and the growth is finished in a mode of setting a growth termination condition, so that the positioning area matched with the element template picture can be accurately positioned no matter whether the page image to be detected is large or small and the resolution is not good.
According to the technical scheme, the initial positioning area is determined in the page image to be tested through template matching, then the target positioning area is determined through area growth based on the central point of the initial positioning area, the real machine with different resolutions can be adapted, and the positioning efficiency and accuracy are improved.
Fig. 2 is a schematic flowchart of a page element positioning method according to an embodiment of the present application, and the embodiment further performs optimization based on the above embodiment. As shown in fig. 2, the method specifically includes the following steps:
s201, respectively extracting image features of the page image and the element template by using a pre-trained feature extraction model.
For example, the image feature extraction may be performed using vgg19 network structures trained on the ImageNet dataset as a feature extraction model using a strategy of migration learning. Moreover, a multi-layer fusion strategy can be adopted to enrich the extracted features, so that the bottom texture features are more favorable for improving the performance of the template matching algorithm, for example, the features of the first layer and the third layer in a feature extraction model are selected for fusion.
S202, according to the extracted image features, calculating the image similarity of the element template and each region in the page image.
Specifically, the size of the sliding window may be determined according to the size of the element template, and then the sliding window is moved on the page image, and each time the sliding window is moved, the image similarity of the area corresponding to the element template map and the sliding window is calculated, for example, the image similarity is determined by calculating the cosine distance between the image features.
S203, calculating a target area which maximizes the overall matching quality in the page image according to the image similarity by using a quality perception template matching method, and taking the target area as an initial positioning area.
The target area is the optimal matching area calculated by using a quality perception template matching method.
And S204, determining the quality thermodynamic diagram of the page image according to the image similarity of the element template and each region in the page image.
That is, the difference between the image similarity of different regions and element templates can be visually seen through the quality thermodynamic diagram.
And S205, performing convolution smoothing operation on the quality thermodynamic diagram by using an average filter with the same size as the element template.
In order to filter out partial noise and give a more intuitive feeling, a convolution smoothing operation can be performed on the quality thermodynamic diagram by adopting an average filter with the same size as the element template.
S206, in the page image, taking the central point of the initial positioning area as an initial point, diffusing the initial point to the periphery by using a rectangle until the obtained mean value of the pixel values of each point on each side of the target rectangle is smaller than the product of the pixel value of the central point and a preset threshold value, and taking the area corresponding to the target rectangle as a target positioning area.
The region growing operation can be performed according to the thermodynamic diagram after the convolution smoothing process. Specifically, the diffusion may be performed in a rectangular shape, and the end condition of the growth is set as: the average value of the pixel values of all points on each edge of the target rectangle is smaller than the product of the pixel value of the central point and a preset threshold value. That is, the central point of the initial positioning area is used as the initial point, and the diffusion is performed to the periphery thereof by using a rectangle, and the diffused rectangle can be initialized in advance, and then the diffusion is performed by a certain step length until the diffused rectangle satisfies the above condition, and the diffusion is terminated, and the area corresponding to the obtained rectangle is the target positioning area. Through the constraint of the end condition of the region growing, the accurate positioning region can be matched for the page images of the terminal to be detected with different resolutions. Moreover, testers do not need to respectively test the devices to be tested with different resolutions, and the test efficiency is improved.
And S207, displaying the target positioning area in the mass thermodynamic diagram.
In the embodiment of the present application, since the final target location area is determined in a region growing manner, and is not limited by the size of the element template and the location area, the element template in the embodiment of the present application may include not only a template of a specific page element, but also a template of a custom page area. The template of the specific page element is a template with the page element as a main body, and only one page element is usually arranged on the template. The template for customizing the page area is a test template customized according to test requirements, the template can comprise a plurality of elements, the plurality of elements form an area, and the area is positioned on the page image to be tested.
According to the technical scheme of the embodiment of the application, the initial positioning area matched with the element template is determined in the page image to be tested by using the quality perception template matching method, and the method can be applied even when the background of the page image changes greatly. And then, according to the image similarity of the element template and each region in the page image, determining a quality thermodynamic diagram of the page image, performing convolution smoothing operation on the quality thermodynamic diagram, filtering out partial noise and giving people a more visual feeling. Finally, the target positioning area is determined through area growth based on the central point of the initial positioning area, so that the positioning method can be adapted to real machines with different resolutions, the positioning efficiency and accuracy are improved, and the positioning method is also suitable for positioning the user-defined page area and has a wider application range.
In addition, for the large image at the PC end, a multi-scale spatial pyramid normalized correlation coefficient matching algorithm may also be used for positioning, which specifically includes:
1. constructing a multi-scale space according to the template drawing to obtain a plurality of template drawings with different scales;
specifically, the template graph can be scaled in equal proportion, or a K-means clustering algorithm is adopted to construct a multi-scale space;
2. carrying out image feature extraction on the image to be detected and each template picture with different scales;
3. and respectively carrying out template matching on each template picture and the image to be detected by adopting a normalized correlation coefficient matching algorithm according to the extracted image characteristics, and determining an area which is most similar to a certain template picture in the image to be detected as an interested target area.
Fig. 3 is a schematic flowchart of a page testing method according to an embodiment of the present application, which is applicable to a case where a page image to be tested is first positioned according to an element template, and then the page image is tested according to a positioning area, and relates to the field of artificial intelligence, in particular to deep learning, image processing, and testing technologies. The method can be executed by a page testing device, which is implemented by software and/or hardware, and is preferably configured in an electronic device, such as a computer device or a server. As shown in fig. 3, the method specifically includes the following steps:
s301, positioning a target area matched with the element template in the page image to be tested.
The target area is obtained by positioning according to the page element positioning method according to any embodiment of the application. And will not be described in detail herein.
S302, calculating the image similarity of the element template and the target area, and determining the test result of the page image according to the value of the image similarity.
The larger the value of the image similarity is, the more similar the display of the element on the page image is to the template, the more the display of the element in the page image is in accordance with the expectation, and the better the test effect is.
According to the technical scheme, the initial positioning area is determined in the page image to be tested through template matching, then the target positioning area is determined through area growth based on the central point of the initial positioning area, the target positioning area can be adapted to real machines with different resolutions, the positioning efficiency and accuracy are improved, and then the page testing efficiency and accuracy are improved.
Fig. 4 is a schematic structural diagram of a page element positioning apparatus according to an embodiment of the present application, which is applicable to a case where positioning is performed on a page image to be tested according to an element template when performing a page test, and relates to the field of artificial intelligence, and in particular to deep learning, image processing, and testing technologies. The device can realize the page element positioning method in any embodiment of the application. As shown in fig. 4, the apparatus 400 specifically includes:
an initial positioning region determining module 401, configured to determine an initial positioning region, which is matched with the element template, in the page image to be tested;
a target positioning region determining module 402, configured to perform region growing around the central point of the initial positioning region as an initial point in the page image, to obtain a target positioning region matched with the element template;
wherein the termination condition of the region growing is determined according to the difference performance of each point on the growing region boundary and the initial point.
Optionally, the target location area determining module is specifically configured to:
and in the page image, taking the central point of the initial positioning area as an initial point, diffusing the initial point to the periphery by using a rectangle until the obtained mean value of the pixel values of each point on each side of the target rectangle is smaller than the product of the pixel value of the central point and a preset threshold value, and taking the area corresponding to the target rectangle as the target positioning area.
Optionally, the initial positioning area determining module includes:
the image feature extraction unit is used for respectively extracting the image features of the page image and the element template by utilizing a pre-trained feature extraction model;
the image similarity calculation unit is used for calculating the image similarity of each area in the element template and the page image according to the extracted image characteristics;
and the initial positioning area determining unit is used for calculating a target area which maximizes the overall matching quality in the page image according to the image similarity by using a quality perception template matching method, and taking the target area as the initial positioning area.
Optionally, the apparatus further includes a thermodynamic diagram processing module, specifically configured to:
before the target positioning region determining module performs region growing, determining a quality thermodynamic diagram of the page image according to the image similarity of the element template and each region in the page image;
and performing convolution smoothing operation on the mass thermodynamic diagram by using an average filter with the same size as the element template.
Optionally, the apparatus further comprises:
and the target positioning area display module is used for displaying the target positioning area in the quality thermodynamic diagram.
Optionally, the element templates include a template of a specific page element and a template of a custom page area.
The page element positioning apparatus 400 provided in the embodiment of the present application can execute the page element positioning method provided in any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method. Reference may be made to the description of any method embodiment of the present application for details not explicitly described in this embodiment.
Fig. 5 is a schematic structural diagram of a page testing apparatus according to an embodiment of the present application, which is applicable to a case where a page image to be tested is first positioned according to an element template, and then the page image is tested according to a positioning area, and relates to the field of artificial intelligence, in particular to deep learning, image processing, and testing technologies. The device can realize the page testing method in any embodiment of the application. As shown in fig. 5, the apparatus 500 specifically includes:
a target area positioning module 501, configured to position a target area matching the element template in the page image to be tested;
a test result obtaining module 502, configured to calculate an image similarity between the element template and a target region, and determine a test result of the page image according to a value of the image similarity;
the target area is obtained by positioning by the page element positioning device according to any embodiment of the present application.
The page testing device 500 provided by the embodiment of the present application can execute the page testing method provided by any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method. Reference may be made to the description of any method embodiment of the present application for details not explicitly described in this embodiment.
There is also provided, in accordance with an embodiment of the present application, an electronic device, a readable storage medium, and a computer program product.
FIG. 6 illustrates a schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 6, the apparatus 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the device 600 can also be stored. The calculation unit 601, the ROM 602, and the RAM603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
A number of components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, a mouse, or the like; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The calculation unit 601 performs the respective methods and processes described above, such as the page element positioning method. For example, in some embodiments, the page element positioning method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM 602 and/or the communication unit 609. When the computer program is loaded into RAM603 and executed by the computing unit 601, one or more steps of the page element positioning method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the page element locating method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), blockchain networks, and the internet.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service are overcome. The server may also be a server of a distributed system, or a server incorporating a blockchain.
Furthermore, according to an embodiment of the present application, there is provided another electronic device, another readable storage medium, and another computer program product for performing one or more steps of the page testing method according to any embodiment of the present application. The specific structure and program code thereof can be referred to the content description of the embodiment shown in fig. 6, and are not described herein again.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel or sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.
The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.
Claims (20)
1. A page element positioning method comprises the following steps:
determining an initial positioning area matched with the element template in the page image to be tested;
in the page image, taking the central point of the initial positioning area as an initial point to perform area growth towards the periphery to obtain a target positioning area matched with the element template;
wherein the termination condition of the region growing is determined according to the difference performance of each point on the growing region boundary and the initial point.
2. The method of claim 1, wherein the performing region growing around the central point of the initial positioning region as an initial point in the page image to obtain a target positioning region matching the element template comprises:
and in the page image, taking the central point of the initial positioning area as an initial point, diffusing the initial point to the periphery by using a rectangle until the obtained mean value of the pixel values of each point on each side of the target rectangle is smaller than the product of the pixel value of the central point and a preset threshold value, and taking the area corresponding to the target rectangle as the target positioning area.
3. The method of claim 1, wherein the determining an initial positioning area in the page image to be tested that matches the element template comprises:
respectively extracting image features of the page image and the element template by using a pre-trained feature extraction model;
calculating the image similarity of the element template and each region in the page image according to the extracted image features;
and calculating a target area which maximizes the overall matching quality in the page image according to the image similarity by using a quality perception template matching method, and taking the target area as the initial positioning area.
4. The method of claim 3, wherein prior to performing the region growing, the method further comprises:
determining a quality thermodynamic diagram of the page image according to the image similarity of the element template and each region in the page image;
and performing convolution smoothing operation on the mass thermodynamic diagram by using an average filter with the same size as the element template.
5. The method of claim 4, further comprising: displaying the target localization area in the mass thermodynamic diagram.
6. The method of claim 1, wherein the element templates include templates of specific page elements and templates of custom page regions.
7. A page testing method comprises the following steps:
positioning a target area matched with the element template in the page image to be tested;
calculating the image similarity of the element template and a target area, and determining the test result of the page image according to the value of the image similarity;
wherein the target area is located according to the page element locating method of any one of claims 1 to 6.
8. A page element positioning apparatus, comprising:
the initial positioning area determining module is used for determining an initial positioning area matched with the element template in the page image to be tested;
a target positioning area determining module, configured to perform area growth around the central point of the initial positioning area as an initial point in the page image to obtain a target positioning area matched with the element template;
wherein the termination condition of the region growing is determined according to the difference performance of each point on the growing region boundary and the initial point.
9. The apparatus of claim 8, wherein the target location area determination module is specifically configured to:
and in the page image, taking the central point of the initial positioning area as an initial point, diffusing the initial point to the periphery by using a rectangle until the obtained mean value of the pixel values of each point on each side of the target rectangle is smaller than the product of the pixel value of the central point and a preset threshold value, and taking the area corresponding to the target rectangle as the target positioning area.
10. The apparatus of claim 8, wherein the initial positioning area determination module comprises:
the image feature extraction unit is used for respectively extracting the image features of the page image and the element template by utilizing a pre-trained feature extraction model;
the image similarity calculation unit is used for calculating the image similarity of each area in the element template and the page image according to the extracted image characteristics;
and the initial positioning area determining unit is used for calculating a target area which maximizes the overall matching quality in the page image according to the image similarity by using a quality perception template matching method, and taking the target area as the initial positioning area.
11. The apparatus according to claim 10, further comprising a thermodynamic diagram processing module, specifically configured to:
before the target positioning region determining module performs region growing, determining a quality thermodynamic diagram of the page image according to the image similarity of the element template and each region in the page image;
and performing convolution smoothing operation on the mass thermodynamic diagram by using an average filter with the same size as the element template.
12. The apparatus of claim 11, further comprising:
and the target positioning area display module is used for displaying the target positioning area in the quality thermodynamic diagram.
13. The apparatus of claim 8, wherein the element templates include a template of a specific page element and a template of a custom page area.
14. A page testing apparatus, comprising:
the target area positioning module is used for positioning a target area matched with the element template in the page image to be tested;
the test result acquisition module is used for calculating the image similarity of the element template and a target area and determining the test result of the page image according to the value of the image similarity;
wherein the target area is located by the page element locating device according to any one of claims 8 to 13.
15. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the page element positioning method of any one of claims 1-6.
16. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the page element positioning method of any one of claims 1 to 6.
17. A computer program product comprising a computer program which, when executed by a processor, implements a page element positioning method according to any one of claims 1-6.
18. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the page testing method of claim 7.
19. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the page testing method of claim 7.
20. A computer program product comprising a computer program which, when executed by a processor, implements the page testing method of claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110320330.9A CN113032071B (en) | 2021-03-25 | 2021-03-25 | Page element positioning method, page testing method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110320330.9A CN113032071B (en) | 2021-03-25 | 2021-03-25 | Page element positioning method, page testing method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113032071A true CN113032071A (en) | 2021-06-25 |
| CN113032071B CN113032071B (en) | 2024-02-06 |
Family
ID=76473778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110320330.9A Active CN113032071B (en) | 2021-03-25 | 2021-03-25 | Page element positioning method, page testing method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113032071B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113609028A (en) * | 2021-10-09 | 2021-11-05 | 统信软件技术有限公司 | Page testing method, computing device and readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111309618A (en) * | 2020-02-24 | 2020-06-19 | 广州市百果园信息技术有限公司 | Page element positioning method, page testing method and related device |
| WO2020233053A1 (en) * | 2019-05-22 | 2020-11-26 | 深圳壹账通智能科技有限公司 | Method, apparatus, device, and storage medium for making test page script |
-
2021
- 2021-03-25 CN CN202110320330.9A patent/CN113032071B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020233053A1 (en) * | 2019-05-22 | 2020-11-26 | 深圳壹账通智能科技有限公司 | Method, apparatus, device, and storage medium for making test page script |
| CN111309618A (en) * | 2020-02-24 | 2020-06-19 | 广州市百果园信息技术有限公司 | Page element positioning method, page testing method and related device |
Non-Patent Citations (2)
| Title |
|---|
| XUNHUA LIU: "On Web Page extraction based on position of DIV", IEEE * |
| 孙农亮;崔红星;梁少华;: "微细对接焊缝初始点定位方法", 光学精密工程, no. 03 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113609028A (en) * | 2021-10-09 | 2021-11-05 | 统信软件技术有限公司 | Page testing method, computing device and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113032071B (en) | 2024-02-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112560862B (en) | Text recognition method and device and electronic equipment | |
| CN111625668B (en) | Object detection and candidate filtering system | |
| CN112488060B (en) | Target detection method, device, equipment and medium | |
| CN113705628B (en) | Determination method and device of pre-training model, electronic equipment and storage medium | |
| CN112562069A (en) | Three-dimensional model construction method, device, equipment and storage medium | |
| CN113205041A (en) | Structured information extraction method, device, equipment and storage medium | |
| CN112861885A (en) | Image recognition method and device, electronic equipment and storage medium | |
| CN113344862A (en) | Defect detection method, defect detection device, electronic equipment and storage medium | |
| CN113643260A (en) | Method, apparatus, device, medium and product for detecting image quality | |
| US20230096921A1 (en) | Image recognition method and apparatus, electronic device and readable storage medium | |
| CN115359308A (en) | Model training method, apparatus, device, storage medium, and program for identifying difficult cases | |
| CN114596431A (en) | Information determination method and device and electronic equipment | |
| CN114120454A (en) | Training method and device of living body detection model, electronic equipment and storage medium | |
| CN113032071B (en) | Page element positioning method, page testing method, device, equipment and medium | |
| CN113627361A (en) | Training method and device for face recognition model and computer program product | |
| CN112862017A (en) | Point cloud data labeling method, device, equipment and medium | |
| CN112508005A (en) | Method, apparatus, device and storage medium for processing image | |
| CN113610856A (en) | Method and device for training image segmentation model and image segmentation | |
| CN114972361A (en) | Blood flow segmentation method, device, equipment and storage medium | |
| CN114119990A (en) | Method, apparatus and computer program product for image feature point matching | |
| CN113780297A (en) | Image processing method, device, equipment and storage medium | |
| CN113435257A (en) | Method, device and equipment for identifying form image and storage medium | |
| CN113361575B (en) | Model training method and device and electronic equipment | |
| CN114359561A (en) | Target detection method and training method and device of target detection model | |
| CN116664427A (en) | Image processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |