CN109901988A - A kind of page elements localization method and device for automatic test - Google Patents
A kind of page elements localization method and device for automatic test Download PDFInfo
- Publication number
- CN109901988A CN109901988A CN201711307879.4A CN201711307879A CN109901988A CN 109901988 A CN109901988 A CN 109901988A CN 201711307879 A CN201711307879 A CN 201711307879A CN 109901988 A CN109901988 A CN 109901988A
- Authority
- CN
- China
- Prior art keywords
- page
- point
- identified
- characteristic point
- characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000012360 testing method Methods 0.000 title claims abstract description 46
- 230000004807 localization Effects 0.000 title claims abstract description 21
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 53
- 238000012545 processing Methods 0.000 claims description 12
- 239000011159 matrix material Substances 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000001629 suppression Effects 0.000 claims description 5
- 238000010586 diagram Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 11
- HUTDUHSNJYTCAR-UHFFFAOYSA-N ancymidol Chemical compound C1=CC(OC)=CC=C1C(O)(C=1C=NC=NC=1)C1CC1 HUTDUHSNJYTCAR-UHFFFAOYSA-N 0.000 description 7
- 230000006854 communication Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 230000004044 response Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000005291 magnetic effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 210000003127 knee Anatomy 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a kind of page elements localization method and devices for automatic test, are related to field of computer technology.One specific embodiment of this method includes: that all characteristic points of the page to be identified are obtained by image recognition algorithm；By each characteristic point, standard picture corresponding with page elements is matched respectively, to obtain multiple match points；Position of the page elements in the page to be identified is determined according to each match point.The embodiment is accurately positioned the page elements of the page to be identified by image recognition algorithm come assisted automated test frame, improves identification to page elements, positioning accuracy, and then improve testing efficiency.
Description
Technical field
The present invention relates to computer field more particularly to a kind of page elements localization methods and dress for automatic test
It sets.
Background technique
In Android UI (User Interface, user interface) automatic test, after installing application to be tested, point
All page elements in the application are analysed to ensure that each page elements can be tested frame and get.It can lead in the prior art
UiAutomator is crossed to analyze acquisition page elements, wherein UiAutomator is the automated test frame for Android, base
All Android event actions are supported in sheet, can not check source code to grab the page elements on application interface.
In realizing process of the present invention, at least there are the following problems in the prior art for inventor's discovery: in automatic test,
It needs to identify page elements, position, however UiAutomator is identified, the accuracy rate of positioning webpage element is lower, often
Test case is caused to execute interruption since page elements identify mistake, if it is desired to continue to execute test and then need to rerun
Test script influences testing efficiency.
Summary of the invention
In view of this, the embodiment of the present invention provides a kind of page elements localization method and device for automatic test,
The page elements of the page to be identified are accurately positioned come assisted automated test frame by image recognition algorithm, which improves
Identification to page elements, positioning accuracy, and then improve testing efficiency.
To achieve the above object, according to an aspect of an embodiment of the present invention, it provides a kind of for automatic test
Page elements localization method.
A kind of page elements localization method for automatic test of the embodiment of the present invention, comprising: pass through image recognition
Algorithm obtains all characteristic points of the page to be identified；By each characteristic point respectively standard picture corresponding with page elements into
Row matching, to obtain multiple match points；Determine the page elements in the page to be identified according to each match point
Position.
Optionally, all characteristic points that the page to be identified is obtained by image recognition algorithm, comprising: building is to be identified
The Hessen matrix of each pixel in the page, to obtain the characteristic value of each pixel；To the characteristic value difference of each pixel
Non-maxima suppression processing is carried out, with all preliminary characteristic points of the determination page to be identified；Choose each preliminary spy
Levy the principal direction of point；It is each described preliminary according to the scale-value and the principal direction where each preliminary characteristic point
Characteristic point constructs a feature vector respectively, and described eigenvector is the characteristic point.
Optionally, described by each characteristic point, standard picture corresponding with page elements is matched respectively, to obtain
Take multiple match points, comprising: calculate separately all features of each characteristic point and standard picture in the page to be identified
The Euclidean distance of point；The ratio of the corresponding the smallest Euclidean distance of each characteristic point and time the smallest Euclidean distance is confirmed respectively
Value is less than the threshold value of setting, and the characteristic point constitutes matching double points with the characteristic point of corresponding minimum Eustachian distance；By taking out at random
Sample consistency algorithm removes the matching double points of the matching error in the matching double points, to obtain multiple match points.
It is optionally, described that position of the page elements in the page to be identified is determined according to each match point,
Include: that matching central point is determined according to each match point, using the matching central point screen coordinate as the page
Position of the element in the page to be identified.
Optionally, described image recognizer is SURF algorithm.
To achieve the above object, according to an aspect of an embodiment of the present invention, it provides a kind of for automatic test
Page elements positioning device.
A kind of page elements positioning device for automatic test of the embodiment of the present invention, comprising: obtain module, be used for
All characteristic points of the page to be identified are obtained by image recognition algorithm；Matching module, for distinguishing each characteristic point
Standard picture corresponding with the page elements is matched, to obtain multiple match points；Determining module, for according to each described
Match point determines position of the page elements in the page to be identified.
Optionally, the acquisition module, is also used to: the Hessen matrix of each pixel in the page to be identified is constructed, with
To the characteristic value of each pixel；Non-maxima suppression processing is carried out respectively to the characteristic value of each pixel, described in determination
All preliminary characteristic points of the page to be identified；Choose the principal direction of each preliminary characteristic point；And according to each described first
The scale-value and the principal direction where characteristic point are walked, constructs a feature vector respectively for each preliminary characteristic point,
Described eigenvector is the characteristic point.
Optionally, the matching module, is also used to: calculating separately each characteristic point and mark in the page to be identified
The Euclidean distance of all characteristic points of quasi- image；The corresponding the smallest Euclidean distance of each characteristic point and time most is confirmed respectively
The ratio of small Euclidean distance is less than the threshold value of setting, and the characteristic point is constituted with the characteristic point of corresponding minimum Eustachian distance and matched
Point pair；And the matching double points of the matching error in the matching double points are removed by RANSAC algorithm, to obtain
Multiple match points.
Optionally, the determining module, is also used to: matching central point is determined according to each match point, by the matching
Central point is in position of the coordinate as the page elements in the page to be identified of screen.
To achieve the above object, according to an embodiment of the present invention in another aspect, providing a kind of electronic equipment.
The a kind of electronic equipment of the embodiment of the present invention, comprising: one or more processors；Storage device, for storing one
A or multiple programs, when one or more of programs are executed by one or more of processors, so that one or more
A processor realizes a kind of page elements localization method for automatic test of the embodiment of the present invention.
To achieve the above object, according to an embodiment of the present invention in another aspect, providing a kind of computer-readable medium.
A kind of computer-readable medium of the embodiment of the present invention, is stored thereon with computer program, and described program is processed
A kind of page elements localization method for automatic test of the embodiment of the present invention is realized when device executes.
One embodiment in foregoing invention has the following advantages that or the utility model has the advantages that is assisted by image recognition algorithm certainly
For dynamicization test frame to be accurately positioned the page elements of the page to be identified, which improves identification, positioning to page elements
Precision improves testing efficiency；The page to be identified is obtained based on SURF (Speeded-Up Robust Features) algorithm
Characteristic point not only has to the robustness of scale and rotation, but also arithmetic speed is very fast, based on a small amount of characteristic point can position to
Identify the page elements of the page；By SURF algorithm and RANSAC algorithm, (RANdomSample Consensus, random sampling are consistent
Property algorithm) it combines, it is applied in automatic test, improves the accuracy of identification of page elements.
Further effect possessed by above-mentioned non-usual optional way adds hereinafter in conjunction with specific embodiment
With explanation.
Detailed description of the invention
Attached drawing for a better understanding of the present invention, does not constitute an undue limitation on the present invention.Wherein:
Fig. 1 is the schematic diagram of the key step of page elements localization method according to an embodiment of the present invention；
Fig. 2 is the main flow schematic diagram of page elements localization method according to an embodiment of the present invention；
Fig. 3 is the schematic diagram of the main modular of page elements positioning device according to an embodiment of the present invention；
Fig. 4 is that the embodiment of the present invention can be applied to exemplary system architecture figure therein；
Fig. 5 is the structural schematic diagram for being suitable for the computer installation of the electronic equipment to realize the embodiment of the present invention.
Specific embodiment
Below in conjunction with attached drawing, an exemplary embodiment of the present invention will be described, including the various of the embodiment of the present invention
Details should think them only exemplary to help understanding.Therefore, those of ordinary skill in the art should recognize
It arrives, it can be with various changes and modifications are made to the embodiments described herein, without departing from scope and spirit of the present invention.Together
Sample, for clarity and conciseness, descriptions of well-known functions and structures are omitted from the following description.
Fig. 1 is the schematic diagram of the key step of page elements localization method according to an embodiment of the present invention.As shown in Figure 1,
The page elements localization method of the embodiment of the present invention, mainly includes the following steps:
Step S101: all characteristic points of the page to be identified are obtained by image recognition algorithm.Wherein, the page to be identified
Face be pending test application program current page screen screenshot.Described image recognizer can be SURF
(Speeded-Up Robust Features accelerates robust feature) algorithm, SIFT (Scale-invariant feature
Transform, scale invariant feature conversion) algorithm etc..SURF algorithm is a steady image recognition and description algorithm, the calculation
Method can complete the matching of object in two images under temperate conditions, and realize real-time processing substantially.SIFT algorithm tool
There is scale invariability, can detect key point in the picture.
Step S102: by each characteristic point, standard picture corresponding with page elements is matched respectively, to obtain
Multiple match points.Wherein, to be in the application program of pending test include the corresponding standard picture of page elements in each page
Page elements corresponding to standard picture.The page elements include control (such as button, text box) and pictorial element.It is logical
Image matching algorithm, such as arest neighbors time neighbour ratio method, Feature Points Matching algorithm are crossed, by all characteristic points of the page to be identified
It is matched respectively with standard picture, obtains multiple match points.Wherein, the principle of arest neighbors time neighbour's ratio method are as follows: take in figure
Some key point, by traversal find away from two nearest key points, in the two key points, if arest neighbors with
The ratio of secondary neighbour is less than some threshold value, then is determined as a pair of of match point.
Step S103: position of the page elements in the page to be identified is determined according to each match point.Root
Determine matching central point according to each match point, using the matching central point screen coordinate as the page elements in institute
State the position in the page to be identified.
Fig. 2 is the main flow schematic diagram of page elements localization method according to an embodiment of the present invention.As shown in Fig. 2, with
For automated test frame is Uiautomator, the page elements localization method of the embodiment of the present invention mainly includes following step
It is rapid:
Step S201: it using the picture of page elements all in the page to be identified as standard picture library, and saves.Wherein,
It include multiple standard pictures in the standard picture library, the picture of a page elements is a standard picture.
Step S202:Uiautomator calls test script by run system command, to be passed through by test script
UiSelector () method carrys out positioning webpage element.The system command include Python (Python be a kind of object-oriented,
Explanation type computer programming language) or Perl (Practical Extraction and Reporting Language,
It is practical to extract and report language) execution order.
Step S203: judging whether positioning succeeds, if positioned successfully, executes step S208；If positioning failure, holds
Row step S204.By UiSelector () method come positioning webpage element in step S202, if it is possible to obtain UiObject
Object then illustrates to position successfully, if it is not, illustrating positioning failure.
Step S204:Uiautomator calls takeScreenshot () method to intercept current screen image, the current screen
Curtain image is the page to be identified.
Step S205: all characteristic points of the page to be identified are obtained by image recognition algorithm.It is with image recognition algorithm
For SURF algorithm, this step is illustrated.This step specifically includes:
(1) the Hessen matrix of each pixel in the page to be identified is constructed, to obtain the characteristic value of each pixel.Its
In, Hessen matrix (Hessian Matrix) is the square of the second-order partial differential coefficient composition for the real-valued function that an independent variable is vector
Matrix.The Hessian Matrix of some pixel in the page to be identified are as follows:
In formula, H (f (x, y)) is the characteristic value of the pixel；F (x, y) is in the page to be identified at pixel (x, y)
Pixel value.
(2) non-maxima suppression processing is carried out to the characteristic value of each pixel, respectively with the determination page to be identified
All preliminary characteristic points.SURF algorithm can be such that original image remains unchanged and only change filter size, save drop and adopt
Sample process, improves processing speed.In order to solve, since image is in different size, to cause when matching the image of different scale
The unmatched problem of characteristic point size, is added thereto scale factor when the step determines preliminary characteristic point.Wherein, ruler
Degree is fog-level, and the scale space of image is expression of the diagram picture under different resolutions.
Specific implementation are as follows: the characteristic value of each pixel and the characteristic value of 26 points of its field of three dimension are subjected to size ratio
Compared with remaining, as reserved spy if the characteristic value of some pixel is the maximum value or minimum value in this 26 points
Sign point.Then, sub-pixel characteristic point is obtained using 3 dimensional linear interpolation methods to all reservation feature points, while removes feature
Value is less than the point of predetermined threshold, and increasing extreme value reduces the pixel quantity detected, finally only several feature point of maximum intensity meetings
It is detected, the point being detected is preliminary characteristic point.
This step uses the filter of size corresponding with the image analytic degree of the scale layer, by taking 3 × 3 filters as an example,
By remaining 8 point in current pixel point and itself scale layer and above and under two scale layers (each scale layer 9
Point), totally 26 points are compared, if the characteristic value of current pixel point is greater than surrounding pixel point, can determine that the pixel is pre-
Stay characteristic point.
(3) principal direction of each preliminary characteristic point is chosen.In order to guarantee rotational invariance, in SURF algorithm, no
Count its histogram of gradients, but the Harr wavelet character in statistical nature point field.In being with some preliminary characteristic point
The heart, calculating radius are to count all first in 60 degree of sectors in the circle shaped neighborhood region of 6S (S is the scale-value where the preliminary characteristic point)
It walks the Harr small echo of characteristic point both horizontally and vertically and responds summation, and assign Gauss weight coefficient to these responses,
So that the response contribution close to preliminary characteristic point is big, and the contribution of the response far from preliminary characteristic point is small, then will be within the scope of 60 degree
Response be summed to form new vector, traverse entire round field, select the direction of longest vector as the preliminary characteristic point
Principal direction.All preliminary characteristic points are calculated one by one in the manner described above, obtain the main side of each preliminary characteristic point
To.Wherein, Harr small echo is the orthogonal wavelet function and simplest one with compact schemes used in wavelet analysis
A wavelet function, it is single rectangular wave of the supporting domain in t ∈ [0,1] range.
It (4) is each preliminary spy according to the scale-value and the principal direction where each preliminary characteristic point
Sign point constructs a feature vector respectively, and described eigenvector is the characteristic point.One is taken around preliminary characteristic point
Then the square-shaped frame is divided into 16 sub-regions, in each subregion by the square-shaped frame that side length is 20S, direction is principal direction
It is interior statistics 25 pixels Haar wavelet character horizontally and vertically, here be all horizontally and vertically
For opposite principal direction.The Haar wavelet character is the sum of the sum of horizontal direction value, horizontal direction absolute value, vertical direction value
The sum of, the sum of vertical direction absolute value this four dimensional feature, finally obtain the feature vector of 4 × 4 × 4=64 dimension to indicate this
Characteristic point.
Step S206: by each characteristic point, standard picture corresponding with the page elements is matched respectively, with
Obtain multiple match points.The step specifically includes:
(1) for each characteristic point of the page to be identified, each characteristic point and standard picture library Plays are calculated separately
The Euclidean distance of all characteristic points of image, the size of more calculated Euclidean distance is to obtain minimum euclidean distance and time most
Small Euclidean distance.Decision metric of the Euclidean distance as two characteristic point similitudes is used in the embodiment of the present invention.Wherein, standard
The characteristic point acquisition of image can be realized by the SURF algorithm of step S205.
(2) confirm that the corresponding the smallest Euclidean distance of each characteristic point and the ratio of time the smallest Euclidean distance are less than respectively
When the threshold value of setting, then this feature point constitutes matching double points with the characteristic point of corresponding minimum Eustachian distance.I.e. if page to be identified
In face certain characteristic point to standard picture arest neighbors characteristic point (i.e. the smallest characteristic point of Euclidean distance) distance DzjWith to from it
Secondary neighbour's characteristic point (i.e. the smallest characteristic point of Euclidean distance time) DcjDistance ratio be less than given threshold ε when, this feature point
Matching double points are constituted with the arest neighbors characteristic point of standard picture；When if it is larger than or equal to given threshold ε, then match point cannot be constituted
It is right.Matching double points are determined using arest neighbors time neighbour's ratio method in the embodiment of the present invention.
(3) matching double points of the matching error in the matching double points are removed, by RANSAC algorithm to obtain multiple
With point.The matching double points obtained in step (2) may have the matching double points of some matching errors, not be able to satisfy practical want
It asks, so the characteristic point of matching error can be removed by RANSAC algorithm, finally obtains accurate feature points as match point.
Step S207: determining position of the page elements in the page to be identified according to each match point, will
The position is encapsulated as UiObject object.Matching central point is determined according to each match point, and the matching central point is existed
Position of the coordinate of screen as the page elements in the page to be identified.
Step S208:Uiautomator calls the method for returning to UiObject object, continues to execute test script, tests
Test report is generated after script execution.
Can be seen that for page elements localization method through image recognition algorithm through the embodiment of the present invention assists
For automated test frame to be accurately positioned the page elements of the page to be identified, which improves identification to page elements, fixed
Position precision, improves testing efficiency；The characteristic point of the page to be identified is obtained based on SURF algorithm, is not only had to scale and rotation
Robustness, and arithmetic speed is very fast, and the page elements of the page to be identified can be positioned based on a small amount of characteristic point；SURF is calculated
Method and RANSAC algorithm combine, and are applied in automatic test, improve the accuracy of identification of page elements.
Fig. 3 is the schematic diagram of the main modular of page elements positioning device according to an embodiment of the present invention.As shown in figure 3,
The page elements positioning device 300 of the embodiment of the present invention, specifically includes that
Module 301 is obtained, for obtaining all characteristic points of the page to be identified by image recognition algorithm.Wherein, described
The page to be identified be pending test application program current page screen screenshot.Described image recognizer can be
SURF algorithm, SIFT algorithm etc..
Matching module 302, for standard picture corresponding with the page elements to carry out respectively by each characteristic point
Matching, to obtain multiple match points.Wherein, the corresponding standard picture of page elements is each in the application program of pending test
Standard picture corresponding to the page elements for including in the page.The page elements include control (such as button, text box) and
Pictorial element.By image matching algorithm, such as arest neighbors time neighbour ratio method, Feature Points Matching algorithm, by the page to be identified
All characteristic points matched respectively with standard picture, obtain multiple match points.
Determining module 303, for determining the page elements in the page to be identified according to each match point
Position.Determine matching central point according to each match point, using the matching central point screen coordinate as the page
Position of the element in the page to be identified.
From the above, it can be seen that by image recognition algorithm come assisted automated test frame to be accurately positioned wait know
The page elements of the other page, which improve identification to page elements, positioning accuracy, improve testing efficiency；It is based on
SURF algorithm obtains the characteristic point of the page to be identified, not only has the robustness to scale and rotation, but also arithmetic speed is very fast,
The page elements of the page to be identified can be positioned based on a small amount of characteristic point；SURF algorithm and RANSAC algorithm are combined, applied
Into automatic test, the accuracy of identification of page elements is improved.
Fig. 4, which is shown, can apply the page elements localization method of the embodiment of the present invention or showing for page elements positioning device
Example property system architecture 400.
As shown in figure 4, system architecture 400 may include terminal device 401,402,403, network 404 and server 405.
Network 404 between terminal device 401,402,403 and server 405 to provide the medium of communication link.Network 404 can be with
Including various connection types, such as wired, wireless communication link or fiber optic cables etc..
User can be used terminal device 401,402,403 and be interacted by network 404 with server 405, to receive or send out
Send message etc..Various telecommunication customer end applications, such as the application of shopping class, net can be installed on terminal device 401,402,403
(merely illustrative) such as the application of page browsing device, searching class application, instant messaging tools, mailbox client, social platform softwares.
Terminal device 401,402,403 can be the various electronic equipments with display screen and supported web page browsing, packet
Include but be not limited to smart phone, tablet computer, pocket computer on knee and desktop computer etc..
Server 405 can be to provide the server of various services, such as utilize terminal device 401,402,403 to user
Generated click event provides the back-stage management server (merely illustrative) supported.Back-stage management server can be to receiving
The data such as click data, content of text analyze etc. processing, and (such as target push information, product are believed by processing result
Breath -- merely illustrative) feed back to terminal device.
It should be noted that page elements localization method provided by the embodiment of the present application is generally executed by server 405,
Correspondingly, page elements positioning device is generally positioned in server 405.
It should be understood that the number of terminal device, network and server in Fig. 4 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.
According to an embodiment of the invention, the present invention also provides a kind of electronic equipment and a kind of computer-readable medium.
Electronic equipment of the invention includes: one or more processors；Storage device, for storing one or more journeys
Sequence, when one or more of programs are executed by one or more of processors, so that one or more of processors are real
A kind of page elements localization method of the existing embodiment of the present invention.
Computer-readable medium of the invention is stored thereon with computer program, real when described program is executed by processor
A kind of page elements localization method of the existing embodiment of the present invention.
Below with reference to Fig. 5, it illustrates the computer systems 500 being suitable for realize the electronic equipment of the embodiment of the present invention
Structural schematic diagram.Electronic equipment shown in Fig. 5 is only an example, function to the embodiment of the present invention and should not use model
Shroud carrys out any restrictions.
As shown in figure 5, computer system 500 includes central processing unit (CPU) 501, it can be read-only according to being stored in
Program in memory (ROM) 502 or be loaded into the program in random access storage device (RAM) 503 from storage section 508 and
Execute various movements appropriate and processing.In RAM 503, also it is stored with computer system 500 and operates required various programs
And data.CPU 501, ROM 502 and RAM 503 are connected with each other by bus 504.Input/output (I/O) interface 505
It is connected to bus 504.
I/O interface 505 is connected to lower component: the importation 506 including keyboard, mouse etc.；It is penetrated including such as cathode
The output par, c 507 of spool (CRT), liquid crystal display (LCD) etc. and loudspeaker etc.；Storage section 508 including hard disk etc.；
And the communications portion 509 of the network interface card including LAN card, modem etc..Communications portion 509 via such as because
The network of spy's net executes communication process.Driver 510 is also connected to I/O interface 505 as needed.Detachable media 511, such as
Disk, CD, magneto-optic disk, semiconductor memory etc. are mounted on as needed on driver 510, in order to read from thereon
Computer program be mounted into storage section 508 as needed.
Particularly, disclosed embodiment, the process of key step figure description above may be implemented as counting according to the present invention
Calculation machine software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable
Computer program on medium, the computer program include the program code for executing method shown in key step figure.?
In such embodiment, which can be downloaded and installed from network by communications portion 509, and/or from can
Medium 511 is dismantled to be mounted.When the computer program is executed by central processing unit (CPU) 501, system of the invention is executed
The above-mentioned function of middle restriction.
It should be noted that computer-readable medium shown in the present invention can be computer-readable signal media or meter
Calculation machine readable storage medium storing program for executing either the two any combination.Computer readable storage medium for example can be --- but not
Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter
The more specific example of calculation machine readable storage medium storing program for executing can include but is not limited to: have the electrical connection, just of one or more conducting wires
Taking formula computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed read-only storage
Device (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device,
Or above-mentioned any appropriate combination.In the present invention, computer readable storage medium can be it is any include or storage journey
The tangible medium of sequence, the program can be commanded execution system, device or device use or in connection.And at this
In invention, computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.Include on computer-readable medium
Program code can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF etc. are above-mentioned
Any appropriate combination.
Flow chart and block diagram in attached drawing are illustrated according to the system of various embodiments of the invention, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
Being described in module involved in the embodiment of the present invention can be realized by way of software, can also be by hard
The mode of part is realized.Described module also can be set in the processor, for example, can be described as: a kind of processor packet
It includes and obtains module, matching module and determining module.Wherein, the title of these modules is not constituted under certain conditions to the module
The restriction of itself is also described as " obtaining all spies of the page to be identified by image recognition algorithm for example, obtaining module
Levy the module of point ".
As on the other hand, the present invention also provides a kind of computer-readable medium, which be can be
Included in equipment described in above-described embodiment；It is also possible to individualism, and without in the supplying equipment.Above-mentioned calculating
Machine readable medium carries one or more program, when said one or multiple programs are executed by the equipment, makes
Obtaining the equipment includes: that all characteristic points of the page to be identified are obtained by image recognition algorithm；Each characteristic point is distinguished
Standard picture corresponding with page elements is matched, to obtain multiple match points；The page is determined according to each match point
Position of the surface element in the page to be identified.
From the above, it can be seen that by image recognition algorithm come assisted automated test frame to be accurately positioned wait know
The page elements of the other page, which improve identification to page elements, positioning accuracy, improve testing efficiency；It is based on
SURF algorithm obtains the characteristic point of the page to be identified, not only has the robustness to scale and rotation, but also arithmetic speed is very fast,
The page elements of the page to be identified can be positioned based on a small amount of characteristic point；SURF algorithm and RANSAC algorithm are combined, applied
Into automatic test, the accuracy of identification of page elements is improved.
Method provided by the embodiment of the present invention can be performed in the said goods, has the corresponding functional module of execution method and has
Beneficial effect.The not technical detail of detailed description in the present embodiment, reference can be made to method provided by the embodiment of the present invention.
Above-mentioned specific embodiment, does not constitute a limitation on the scope of protection of the present invention.Those skilled in the art should be bright
It is white, design requirement and other factors are depended on, various modifications, combination, sub-portfolio and substitution can occur.It is any
Made modifications, equivalent substitutions and improvements etc. within the spirit and principles in the present invention, should be included in the scope of the present invention
Within.
Claims (11)
1. a kind of page elements localization method for automatic test characterized by comprising
All characteristic points of the page to be identified are obtained by image recognition algorithm；
By each characteristic point, standard picture corresponding with page elements is matched respectively, to obtain multiple match points；
Position of the page elements in the page to be identified is determined according to each match point.
2. the method according to claim 1, wherein described obtain the page to be identified by image recognition algorithm
All characteristic points, comprising:
The Hessen matrix of each pixel in the page to be identified is constructed, to obtain the characteristic value of each pixel；
Non-maxima suppression processing is carried out respectively to the characteristic value of each pixel, with all first of the determination page to be identified
Walk characteristic point；
Choose the principal direction of each preliminary characteristic point；
According to the scale-value and the principal direction where each preliminary characteristic point, for each preliminary characteristic point difference
A feature vector is constructed, described eigenvector is the characteristic point.
3. the method according to claim 1, wherein it is described by each characteristic point respectively with page elements pair
The standard picture answered is matched, to obtain multiple match points, comprising:
Calculate separately the Euclidean distance of each characteristic point and all characteristic points of standard picture in the page to be identified；
Confirm that the ratio of the corresponding the smallest Euclidean distance of each characteristic point and time the smallest Euclidean distance is less than respectively to set
Fixed threshold value, the characteristic point constitute matching double points with the characteristic point of corresponding minimum Eustachian distance；
The matching double points of the matching error in the matching double points are removed, by RANSAC algorithm to obtain multiple
With point.
4. the method according to claim 1, wherein described determine the page elements according to each match point
Position in the page to be identified, comprising: matching central point is determined according to each match point, by the matching central point
In position of the coordinate as the page elements in the page to be identified of screen.
5. described in any item methods according to claims 1 to 4, which is characterized in that described image recognizer is SURF calculation
Method.
6. a kind of page elements positioning device for automatic test characterized by comprising
Module is obtained, for obtaining all characteristic points of the page to be identified by image recognition algorithm；
Matching module, for by each characteristic point, standard picture corresponding with the page elements to be matched respectively, with
Obtain multiple match points；
Determining module, for determining position of the page elements in the page to be identified according to each match point.
7. device according to claim 6, which is characterized in that the acquisition module is also used to:
The Hessen matrix of each pixel in the page to be identified is constructed, to obtain the characteristic value of each pixel；
Non-maxima suppression processing is carried out respectively to the characteristic value of each pixel, with all first of the determination page to be identified
Walk characteristic point；
Choose the principal direction of each preliminary characteristic point；And
According to the scale-value and the principal direction where each preliminary characteristic point, for each preliminary characteristic point difference
A feature vector is constructed, described eigenvector is the characteristic point.
8. device according to claim 6, which is characterized in that the matching module is also used to:
Calculate separately the Euclidean distance of each characteristic point and all characteristic points of standard picture in the page to be identified；
Confirm that the ratio of the corresponding the smallest Euclidean distance of each characteristic point and time the smallest Euclidean distance is less than respectively to set
Fixed threshold value, the characteristic point constitute matching double points with the characteristic point of corresponding minimum Eustachian distance；And
The matching double points of the matching error in the matching double points are removed, by RANSAC algorithm to obtain multiple
With point.
9. device according to claim 6, which is characterized in that the determining module is also used to: according to each match point
Determine matching central point, using it is described matching central point screen coordinate as the page elements in the page to be identified
Position.
10. a kind of electronic equipment characterized by comprising
One or more processors；
Storage device, for storing one or more programs,
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
Now such as method as claimed in any one of claims 1 to 5.
11. a kind of computer-readable medium, is stored thereon with computer program, which is characterized in that described program is held by processor
Such as method as claimed in any one of claims 1 to 5 is realized when row.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711307879.4A CN109901988A (en) | 2017-12-11 | 2017-12-11 | A kind of page elements localization method and device for automatic test |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711307879.4A CN109901988A (en) | 2017-12-11 | 2017-12-11 | A kind of page elements localization method and device for automatic test |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109901988A true CN109901988A (en) | 2019-06-18 |
Family
ID=66942304
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711307879.4A Pending CN109901988A (en) | 2017-12-11 | 2017-12-11 | A kind of page elements localization method and device for automatic test |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109901988A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110399291A (en) * | 2019-06-20 | 2019-11-01 | 平安普惠企业管理有限公司 | User Page test method and relevant device based on image recognition |
| CN110888810A (en) * | 2019-11-19 | 2020-03-17 | 广东润联信息技术有限公司 | Method and device for automatic identification and marking, computer equipment and storage medium |
| CN112286780A (en) * | 2019-07-23 | 2021-01-29 | 浙江宇视科技有限公司 | Method, device and equipment for testing recognition algorithm and storage medium |
| CN112988557A (en) * | 2019-12-17 | 2021-06-18 | 腾讯科技（深圳）有限公司 | Search box positioning method, data acquisition device and medium |
| CN113360414A (en) * | 2021-07-20 | 2021-09-07 | 深圳前海微众银行股份有限公司 | Software testing method, device, readable storage medium and computer program product |
| CN112988557B (en) * | 2019-12-17 | 2024-06-11 | 腾讯科技（深圳）有限公司 | Search box positioning method, data acquisition method, device and medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102722731A (en) * | 2012-05-28 | 2012-10-10 | 南京航空航天大学 | Efficient image matching method based on improved scale invariant feature transform (SIFT) algorithm |
| US20130004087A1 (en) * | 2011-06-30 | 2013-01-03 | American Express Travel Related Services Company, Inc. | Method and system for webpage regression testing |
| CN103217438A (en) * | 2013-04-02 | 2013-07-24 | 天津大学 | Accurate circuit board element location and detection method based on image feature |
| CN105513038A (en) * | 2014-10-20 | 2016-04-20 | 网易（杭州）网络有限公司 | Image matching method and mobile phone application test platform |
| CN107067415A (en) * | 2017-03-21 | 2017-08-18 | 南京航空航天大学 | A kind of quick accurate positioning method of target based on images match |
| CN107274442A (en) * | 2017-07-04 | 2017-10-20 | 北京云测信息技术有限公司 | A kind of image-recognizing method and device |
-
2017
- 2017-12-11 CN CN201711307879.4A patent/CN109901988A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130004087A1 (en) * | 2011-06-30 | 2013-01-03 | American Express Travel Related Services Company, Inc. | Method and system for webpage regression testing |
| CN102722731A (en) * | 2012-05-28 | 2012-10-10 | 南京航空航天大学 | Efficient image matching method based on improved scale invariant feature transform (SIFT) algorithm |
| CN103217438A (en) * | 2013-04-02 | 2013-07-24 | 天津大学 | Accurate circuit board element location and detection method based on image feature |
| CN105513038A (en) * | 2014-10-20 | 2016-04-20 | 网易（杭州）网络有限公司 | Image matching method and mobile phone application test platform |
| CN107067415A (en) * | 2017-03-21 | 2017-08-18 | 南京航空航天大学 | A kind of quick accurate positioning method of target based on images match |
| CN107274442A (en) * | 2017-07-04 | 2017-10-20 | 北京云测信息技术有限公司 | A kind of image-recognizing method and device |
Non-Patent Citations (8)
| Title |
|---|
| STRCCC: "Surf算法解析", 《HTTPS://BLOG.CSDN.NET/STREAMCHUANXI/ARTICLE/DETAILS/50924549?IVK_SA=1024320U》 * |
| TOSTQ: "SURF算法详解及同SIFT算法的比较", 《HTTPS://BLOG.CSDN.NET/TOSTQ/ARTICLE/DETAILS/49472709》 * |
| 张永宏等: "基于SURF的抗重复特征干扰图像配准方法", 《计算机工程与设计》 * |
| 曾峦: "《侦察图像获取与融合技术》", 31 May 2015, 国防工业出版社 * |
| 李昕宇: "基于图像匹配的移动应用自动化测试方法研究", 《计算机工程与应用》 * |
| 李杭: "《伪造数字图像盲检测技术研究》", 31 January 2016, 吉林大学出版社 * |
| 松子茶: "SURF特征提取分析", 《HTTPS://BLOG.CSDN.NET/SONGZITEA/ARTICLE/DETAILS/16986423》 * |
| 赵毅力等: "基于SURF特征点的图像配准系统", 《计算机应用》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110399291A (en) * | 2019-06-20 | 2019-11-01 | 平安普惠企业管理有限公司 | User Page test method and relevant device based on image recognition |
| CN112286780A (en) * | 2019-07-23 | 2021-01-29 | 浙江宇视科技有限公司 | Method, device and equipment for testing recognition algorithm and storage medium |
| CN112286780B (en) * | 2019-07-23 | 2024-03-12 | 浙江宇视科技有限公司 | Method, device, equipment and storage medium for testing recognition algorithm |
| CN110888810A (en) * | 2019-11-19 | 2020-03-17 | 广东润联信息技术有限公司 | Method and device for automatic identification and marking, computer equipment and storage medium |
| CN112988557A (en) * | 2019-12-17 | 2021-06-18 | 腾讯科技（深圳）有限公司 | Search box positioning method, data acquisition device and medium |
| CN112988557B (en) * | 2019-12-17 | 2024-06-11 | 腾讯科技（深圳）有限公司 | Search box positioning method, data acquisition method, device and medium |
| CN113360414A (en) * | 2021-07-20 | 2021-09-07 | 深圳前海微众银行股份有限公司 | Software testing method, device, readable storage medium and computer program product |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108898186B (en) | Method and device for extracting image | |
| US10762387B2 (en) | Method and apparatus for processing image | |
| CN109117358A (en) | test method and test device for electronic equipment | |
| CN109117831A (en) | The training method and device of object detection network | |
| CN109901988A (en) | A kind of page elements localization method and device for automatic test | |
| US11699283B2 (en) | System and method for finding and classifying lines in an image with a vision system | |
| CN108229418B (en) | Human body key point detection method and apparatus, electronic device, storage medium, and program | |
| CN109934181A (en) | Text recognition method, device, equipment and computer-readable medium | |
| CN110619807B (en) | Method and device for generating global thermodynamic diagram | |
| CN108280455A (en) | Human body critical point detection method and apparatus, electronic equipment, program and medium | |
| CN109472524A (en) | Information processing method and device | |
| CN109242801A (en) | Image processing method and device | |
| CN109389072A (en) | Data processing method and device | |
| CN113239227B (en) | Image data structuring method, device, electronic equipment and computer readable medium | |
| CN109272050B (en) | Image processing method and device | |
| CN108595448A (en) | Information-pushing method and device | |
| CN109118456A (en) | Image processing method and device | |
| CN110110666A (en) | Object detection method and device | |
| CN110070076A (en) | Method and apparatus for choosing trained sample | |
| CN113705362A (en) | Training method and device of image detection model, electronic equipment and storage medium | |
| CN109614327A (en) | Method and apparatus for output information | |
| CN108492284A (en) | Method and apparatus for the perspective shape for determining image | |
| CN108182457A (en) | For generating the method and apparatus of information | |
| CN115311469A (en) | Image labeling method, training method, image processing method and electronic equipment | |
| CN109064464A (en) | Method and apparatus for detecting battery pole piece burr |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190618 |
|
| RJ01 | Rejection of invention patent application after publication |