CN104657377B - A kind of multichannel webpage control localization method and device - Google Patents

A kind of multichannel webpage control localization method and device Download PDF

Info

Publication number
CN104657377B
CN104657377B CN201310590812.1A CN201310590812A CN104657377B CN 104657377 B CN104657377 B CN 104657377B CN 201310590812 A CN201310590812 A CN 201310590812A CN 104657377 B CN104657377 B CN 104657377B
Authority
CN
China
Prior art keywords
control
target widget
location data
thread
attribute
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201310590812.1A
Other languages
Chinese (zh)
Other versions
CN104657377A (en
Inventor
甘崇志
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Singapore Holdings Pte Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201310590812.1A priority Critical patent/CN104657377B/en
Publication of CN104657377A publication Critical patent/CN104657377A/en
Application granted granted Critical
Publication of CN104657377B publication Critical patent/CN104657377B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3664Environments for testing or debugging software
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • G06F11/3672Test management

Abstract

The invention discloses a kind of multichannel webpage control localization method and device, is related to webpage control location technology.The inventive method includes:Effective attribute of target widget is extracted to set A;Judge whether there is label and attribute set and set A identical controls in current DOM structure, if do not have, set A is the location data of target widget, if, and with target widget it is fraternal control, using position of the index information instruction target widget in fraternal control, the index information and the location data that set A is target widget;If so, but be non-fraternal control with target widget, the location data of the parent control of target widget is extracted, the location data and set A of the parent control are the location data of target widget;Target widget is positioned according to the location data of target widget.The invention also discloses multichannel webpage control positioner.Technical scheme has taken into account location efficiency while ensureing in more stably orient control.

Description

A kind of multichannel webpage control localization method and device
Technical field
The present invention relates to webpage control location technology, more particularly to a kind of multichannel webpage control localization method and device.
Background technology
With the rise of internet, web2.0 popularization, all kinds of web are applied and relevant art is quickly grown, based on web's Page test activity also increasingly abundantization, starts entirely especially for the automatic test of the analog subscriber function of web page Face is applied and promoted, and as the driver frameworks such as the selenium that industry is famous, watir also arise at the historic moment, these frameworks can drive Browser simultaneously provides user's api interface to operate browser and its web page element.Wherein most basic is exactly determining for webpage control Position, the positioning method that API is provided is also tight format match, for example, searching the control element button that an ID is AA, that is abstract XPATH grammers be find_element (' XPATH ', ' //A [@id=AA] ').Or the mode of canonical matching searches one The element that class attributes are started with BB, abstract grammer are find_element (' class ' ,/^BB/).Meanwhile in single category Property when can not uniquely position, also support the mode of Multiple Attribute Combinatorial to be accurately positioned element, such as find_element (‘XPATH’,‘//A[@id=AAand@name=BB]’).So this basic atomic formula, which searches mode, can meet own Element positioning, also simultaneously, it is accurate formula.The implication of accurate formula refers to that control has multiple attributes and an XPATH road Footpath, we may by any attribute or multiple attributes or certain XPATH path can position matching to this control, but appoint A kind of what positioning method is all accurate after having chosen, for example is positioned with ID, then the ID of the element can not be changed.
Meanwhile also having technology sikuli of the one kind based on image recognition analog automatization, it is a kind of graphic programming side Formula, to the control on the page, as long as providing the administrative division map of control element, with regard to the control can be navigated to, meanwhile, to specified control Carry out a series of analog mouse keyboard operations.Its positioning method has a similarity parameter, and similarity is higher, to matching area Pixel request it is also higher.Lookup and operation to web page element can also be realized by it, because being to be based on image recognition, It is supportive just poor on cross-platform or different resolution machine.Generally, its web page element control identification method Accurate formula, once it is determined that the picture region of a control, if the cosmetic variation of that control, may result in identification less than Or identification mistake.
With the popularization of these driver frameworks, in testing field, UI automatic test part also Rapid Accumulation, to saving hand Work is tested, and is returned using quick, there is provided efficient quality assurance brings huge benefit.For the release quickly of the Internet model Provide the guarantee of high availability.
But at the same time, because quickly issue brings the quick renewal of webpage, while dynamic web page element is in itself It is also likely to be change, thus just brings the variation of web page element attribute and XPATH paths.This based on control element just to being determined The web automatic tests case of position brings huge challenge, and the change of control likely results in the positioning failure of control, so as to lead Case operation failure is caused, is then the artificial reparation problem again.Because this frequently change, make these UI automatic tests The problem of case becomes heavyweight, and the difficulty of maintenance also turns into headache, such as an element for having CLASS, NAME, ID etc., rise Test code just based on NAME positioning, if NAME changes, even if other elements attribute is not changed, the element is in positioning It can fail, written in code person must just find this change point, and using new effective NAME values by hand again or using other instead has The property value of effect is positioned.Such issues that the number that occurs it is more when, the maintenance cost brought just is incremented by, and also allows Our automation CASE searched based on control or other instruments become heavyweight.
Meanwhile when web automatic tests are carried out, functional checking is relatively easily realized, but the page Appearance style the problems such as monitoring be but difficult to accomplish, but but need verify.The especially control of some core business streams Part, displaying style issues not only bring bad Consumer's Experience, in some instances it may even be possible to the normal use of user is influenceed, but outside the page The checking of sight above, or obtains the duplication of labour of repetitive manual, and this people is reduced without a replacement set or aid For work.
Therefore, just having following technological challenge and demand, can change fault-tolerant such control to a certain extent so that i.e. Local attribute is had change or after the relative position of html document changes, if attribute that other of the element do not change or Still effective or outward appearance is still effective in path, it becomes possible to the control is quickly correctly still navigated to, meanwhile, for the control Element, there can be a mechanism to monitor the change of its outward appearance so that in WEB automatic tests, if outward appearance occurs in the control Pattern is compared with normal appearance style originally and changed, it is meant that the problem of potential is present, can be correctly reflected to use Family.
And current control positioning using regular coding accurate formula matching way, i.e., by control position API come Search the element for meeting some attributes match or XPATH matchings.But the problem of which, is once the matching condition chosen is lost Effect, can not just position the element.Necessary manual intervention, is replaced with new right value.Result in stability it is not high and safeguard Cost is too big.
Assuming that there is control element E, its effective attribute set is S { a, b, c... }, its XPATH in current page document Path is the subset that X.S1 is S, is defined as uniquely indicating element E. because S differs surely only by S1 in current document One indicates the element, so S1 may be not present.(Because have no effect on the validity of derivation, it is assumed here that S1 is to exist 's).
Element E is required to look up now, that usable exhaustive method of accurate formula matched and searched institute is as follows:
Find_element(XPATH):Element E. is accurately searched by XPATH
Find_element(S1):By effective attribute to location element E.
It can be learnt from above:
When any one node changes in XPATH paths, element E is accurately searched by XPATH to fail.
Any one attribute is changed in S1 gathers, and location element E can also be failed by effective attribute.
But actually element E is still effective, can be properly positioned by other attribute S2, S3... in S.Its In, S2, S3... are S subsets.
It can be seen from the above description that if current accurate formula matching way is when choosing any one change of attribute, Will Wrong localization, stability is poor.
The content of the invention
The technical problem to be solved by the invention is to provide a kind of multichannel webpage control localization method and device, to improve The stability of webpage control positioning.
In order to solve the above-mentioned technical problem, the invention discloses a kind of multichannel webpage control localization method, this method to include:
Effective attribute of target widget is extracted to set A;
In current file object model(DOM)In structure, label and attribute set are searched with the effectively attribute to set A Identical control;
If find, and the control found and target widget are fraternal control, then indicate target using index information Position of the control in fraternal control, and using the index information and the effectively positioning of the attribute to set A as target widget Data;
If find, but the parent control of the control found and the non-fraternal control of target widget, then extraction target widget The location data of part, and the location data of the parent control for the target widget extracted and the effectively attribute are made to set A For the location data of target widget;
If do not found, using the effectively location data of the attribute to set A as target widget;
Target widget is positioned according to the location data of the target widget.
Alternatively, in the above method, the effectively attribute is to comprising at least tag types, the label of control, category in set A Property value is to sequence.
Alternatively, the above method also includes:
Following one or more of information are extracted also from the target widget, and the information of extraction is added to the target In the location data of control:
Cascading Style Sheet(CSS)Selector information, active path information, page area figure information.
Alternatively, in the above method, the process that is positioned according to the location data of the target widget to target widget Including:
When judging to include the index information in the location data, establish multiple lookup threads while search the mesh Control is marked, the lookup thread established comprises at least the attribute to successively decrease and searches thread.
Wherein, when including the index information in the location data, the lookup thread of foundation also includes following one kind Or several threads:
CSS searches thread, figure script searches thread, active path information attribute searches thread.
Alternatively, in the above method, the process that is positioned according to the location data of the target widget to target widget Including:
Judge not including when having the index information in the location data, establish described in multiple lookup threads lookups simultaneously Target widget, the thread of searching established comprise at least partial circulating lookup thread and incremental attribute lookup thread.
Wherein, not comprising when having the index information in the location data, the lookup thread of foundation is also included as next Kind or several threads:
CSS searches thread, figure script searches thread, active path information attribute searches thread.
Alternatively, the above method also includes:
The control outward appearance of the target widget is monitored, when the control outward appearance of the target widget changes, to renewal Control outward appearance analyzed and feed back to user.
The invention also discloses a kind of multichannel webpage control positioner, the device includes:
Extraction module, effective attribute of target widget is extracted to set A, in current DOM structure, searches label and attribute Set and the effectively attribute are to the identical controls of set A, if found, and the control found would be with target widget Fraternal control, then position of the target widget in fraternal control is indicated using index information, and by the index information and described had Attribute is imitated to location datas of the set A as target widget;If find, but the control found and the non-brother of target widget Control, then the location data of the parent control of target widget is extracted, and by the positioning of the parent control for the target widget extracted Data and the effectively attribute are to location datas of the set A as target widget;If do not found, by effective attribute To location datas of the set A as target widget;
Locating module, target widget is positioned according to the location data of the target widget.
Alternatively, in said apparatus, the effectively attribute is to comprising at least tag types, the label of control, category in set A Property value is to sequence.
Alternatively, in said apparatus, the extraction module, following one or more of letters are extracted also from the target widget Breath, and the information of extraction is added in the location data of the target widget:
Cascading Style Sheet(CSS)Selector information, active path information, page area figure information.
Alternatively, in said apparatus, the locating module includes:
Judge module, judge whether include the index information in the location data;
First processing units, when having the index information in the location data, establish multiple lookup threads while look into The target widget is looked for, the lookup thread established comprises at least the attribute to successively decrease and searches thread.
Alternatively, in said apparatus, the lookup thread that the first processing units are established also includes following one or more Thread:
CSS searches thread, figure script searches thread, active path information attribute searches thread.
Alternatively, in said apparatus, the locating module also includes:
Second processing unit, not comprising when having the index information in the location data, establish multiple lookup threads The target widget is searched simultaneously, the thread of searching established comprises at least partial circulating lookup thread and incremental attribute lookup line Journey.
Alternatively, in said apparatus, the lookup thread that the second processing unit is established also includes following one or more Thread:
CSS searches thread, figure script searches thread, active path information attribute searches thread.
Alternatively, said apparatus also includes, monitoring module, the control outward appearance of the target widget is monitored, when the target When the control outward appearance of control changes, the control outward appearance of renewal is analyzed and feeds back to user.
Technical scheme provide it is a kind of from abstract data extraction be defined into location algorithm and visual analysis feedback it is whole Solution is covered, the core of location algorithm is to ensure to take into account location efficiency while more stably orient control, and makes It is lower that CASE maintenance costs must be tested.
Brief description of the drawings
Fig. 1 is the flow chart of control localization method in the present embodiment 1;
Fig. 2 is the flow chart of control localization method in the present embodiment 2;
Fig. 3 is location algorithm flow chart in control localization method shown in Fig. 2;
Fig. 4 is the algorithm flow chart of incremental attribute finger;
Fig. 5 is the algorithm flow chart of attribute finger of successively decreasing;
Fig. 6 is that outward appearance monitors schematic flow sheet in the present embodiment;
Fig. 7 is multichannel webpage control positioning device structure schematic diagram in the present embodiment.
Embodiment
For the object, technical solutions and advantages of the present invention are more clearly understood, below in conjunction with accompanying drawing to skill of the present invention Art scheme is described in further detail.It should be noted that in the case where not conflicting, in embodiments herein and embodiment Feature can arbitrarily be mutually combined.
In order to make up the deficiency of existing control lookup technology, strengthen stability and fault-tolerance in control positioning so that Even if local attribute has change or after the relative position of html document changes, as long as the attribute that other of the element do not change Or still effective or outward appearance is still effective in path, it becomes possible to the control is quickly correctly still navigated to, therefore it is automatic in UI When changing test application, inventor proposes a kind of written in code pattern different from legacy controls positioning, but first extracts target control A series of abstract datas of part, these data equivalent to target widget metadata, then use more containment type location algorithm And strategy, based on these metadata, the multithreading of self-align type is carried out simultaneously automatically according to the attribute set of target widget Row positioning.
Embodiment 1
The location data extracted in the present embodiment is the set of properties set of target widget.In view of in some cases, in net Multiple controls be present in page element, its property value, tag types(Tag name)It is identical, only the difference is that its parent control Part is different.In addition, there is also the control under same parent control with identical attribute tags type, difference only exists It is different relative to the skew of parent control between each control.Therefore, in order to be accurately positioned control, in extraction target widget Location data when need to meet following 3 principles:1st, effective attribute comprising the control is to set A(, can be with practical application Filter out some and be not suitable for attribute pair as positioning, such as null value, style attributes etc., principle is to remove the category being easily varied Property)
If the 2nd, such control be present, its label and attribute set and the valid genus of target widget in current DOM structure Property it is identical to set A, and be the control of brother with target widget, i.e., can not determine which be only by label and attribute Individual control, then newly-increased index is needed in the location data of the target widget(index)Represent target widget in fraternal control Position.
If the 3rd, such control be present, its label and attribute set and the valid genus of target widget in current DOM structure Property, but not its fraternal control identical to set A;Newly-increased parent control is then needed in the location data of the target widget Location data, to limit which parent control is the target widget be under.Wherein, the parent control of target widget is determined Position data, are equally extracted using above-mentioned 1,2 and 3 rule.
Based on above-mentioned thought, a kind of multichannel webpage control positioning and monitoring method of the present embodiment offer, as shown in figure 1, extremely It is few to include following operation:
Step 101, effective attribute of target widget is extracted to set A;
In the present embodiment, effective attribute is to set A including at least the label of control, tag types, property value to sequence (Attribute_pairs).
Step 102, in current DOM structure, label and attribute set is searched and effective attribute is identical to set A Control;
Step 103a, when finding, and when the control found and target widget are fraternal control, referred to using index information Show position of the target widget in fraternal control, and the index information and effective attribute are determined as target widget set A Position data;
Step 103b, when finding, but the control found and the non-fraternal control of target widget, then extract target widget The location data of parent control, and by the location data of the parent control for the target widget extracted and effective attribute to set A Location data as target widget;
Wherein, can be using parent control as new target widget when extracting the location data of parent control of target widget The operations of above-mentioned steps 101 to 103 is repeated to extract location data.
And when target widget does not have parent control, the location data of the parent control in location data is configured to sky, i.e., The positioning for illustrating target widget is whole html documents scope.
Step 103c, if do not found, using location data of effective attribute to set A as target widget;
Step 104, target widget is positioned according to the location data of target widget.
Wherein, when carrying out positioning action to target widget according to location data, any-mode can be used, the present embodiment is herein It is not restricted.
It should be noted that in order to ensure the stability of control positioning, can also be by CSS selector and/or active path Information(xpath)It is added in location data, using alternative as one when self-adjusting location algorithm can not find target widget Method.
In addition, for target widget, it is necessary to which a mechanism monitors the change of its outward appearance, because in WEB automatic tests In, changed if the appearance style of target widget is compared with initial pattern, it is possible to which existing influences the potential of control positioning Problem.Therefore, the page area figure information of target widget can be added in location data, so that monitoring objective control is outer See, and when the outward appearance of target widget changes, this change is analyzed and feeds back to user in time.
In summary, based on the above method, preferred scheme can also be extracted as next when extracting location data Kind or several information, and the information of extraction is added in the location data of the target widget:
CSS selector information, xpath, page area figure information.
Embodiment 2
The present embodiment is based on the location data that target widget is extracted in the embodiment 13 points of rules to be met, there is provided a kind of Preferable multichannel webpage control positioning and monitoring method, as shown in Fig. 2 including at least following operation:
Step 201, effective attribute of target widget is extracted to set A;
The step is exactly to filter out some to be not suitable for attribute pair as positioning, such as null value, style attributes etc., so as to Effective attribute of target widget is obtained to set.
In the present embodiment, effective attribute is to set A including at least the label of control, tag types, property value to sequence (Attribute_pairs).
Step 202, in current DOM structure, judge whether to find label and attribute set and target widget Effective attribute is to the identical controls of set A, if it is possible to finds, then into step 203, if searched less than general Effective attribute of target widget is defined as the location data of target widget to set A, into step 206;
Step 203, whether the control that judges to find and target widget are fraternal control, if it is, into step 204, otherwise into step 105;
Step 204, using an index(index)Information indicates position of the target widget in fraternal control, and by target Effective attribute of control is defined as the location data of target widget to set A and index information, into step 206.
Step 205, the location data of the parent control of the target widget is extracted, and by the positioning of the parent control extracted Effective attribute of data and target widget is defined as the location data of target widget to set A;
, can be using parent control as new target when extracting the location data of parent control of target widget in the step Control extracts location data using the operation of above-mentioned steps 101 to 105.
From aforesaid operations as can be seen that for the high control of some multiplicities, its location data may form a number According to chain, it is contemplated that search performance and validity, chain length is more short better, i.e., as far as possible satisfaction with the mode of the minimum chain of length come table Show target widget.Too long of chain, any of which node is wrong may to influence whole chain, but does not Qiang Zhiyaoqiu, i.e., preferably Ground, select shorter control chain.The structure of the location data as follows for being a target widget:
As can be seen that this above-mentioned structure is a Data-Link.Chain head is the control that parent is null, for example, using json Data represent a link control, and its possible location data is:
In addition, CSS selector or xpath are also frequently applied in the lookup of target widget.Therefore in order to ensure control The stability of positioning, this 2 extra control informations can be added in preferential treatment scheme into location data, as in self-adjusting Location algorithm can not find alternative approach during target widget.
Further, if the appearance style of a certain control changes, it is understood that there may be potential the problem of influenceing positioning.Cause The page area figure information of this target widget can also be added in location data, and this pictorial information, which can be not only used for control, to be determined Position, moreover it can be used to which control outward appearance monitors.But control positioning is nor affected on to a certain extent without it, so and optional additional Data.When including page area figure information in location data, according to the outward appearance of the page area figure INFORMATION DISCOVERY target widget When changing, the control outward appearance of renewal is analyzed and feeds back to user.
Step 206, positioning action is carried out to target widget according to identified location data.
Because the location data of target widget is bar chain, that is to say, that the positioning of any one control is all based on his father Offspring's control of class control.The father of top control is linked as sky.
So the position fixing process of the target widget in the present embodiment is the process of an iterative recursive, since top control, Iteration eventually finds the leaf control of bottom.And the input of this iterative recursive operation is the location data of target widget(Parent The location data of control may be sky, be whole html documents scope for empty i.e. explanation current lookup), export and then navigate to Target widget.Parent control by the use of the result as input next time again, the process is circulated together with next target widget as son. Untill iterating to leaf elements.In the present embodiment, there is provided a kind of location algorithm of optimization, as shown in figure 3, including following operation:
Step 301, the location data of target widget is inputted.
Step 302, judge whether there is index in location data(The instruction target widget that i.e. above method is added is in brother Positional information in younger brother's control), if it is not, into step 303, if so, into step 307;
In the step, corresponding lookup algorithm is selected according to judged result, is because if it is target widget to have index Parent control under have multiple child controls, although this parent control is not necessarily direct father, have multiple controls that there is phase Same set of properties.The result that i.e. explanation is found by the set of properties of target widget is also a plurality of, then location algorithm is different.
Step 303, main thread group of distributors establishes multiple lookup threads while searches target widget, the lookup line established Journey comprises at least the one or more in following 5 kinds:
A, partial circulating(Short-circuit):Multiple threads may be corresponding with, because an effective attribute of searching is exactly One lookup factor(An information i.e. in location data), therefore establish a thread for each factor of searching.Than if any ID, NAME, then there are 2 lookup threads, search the element of ID and NAME matchings respectively, any one finds and only equal to one Result, then return to the control, otherwise return empty.
Short-circuit finger sets of threads:It is to quickly navigate to purpose element, its location algorithm is as follows:
Assuming that the location data set S { a, b, c } of target widget
Then each information is a lookup factor, it is assumed that has 3 factors.Each factor pair answers a thread, respectively simultaneously Row searches the element of the matching attribute.
For the single factor searches thread, if finding object element as sky, return empty.If search multiple members Element, because it is which that cannot determine, also return to sky.If only an element is found, then can release by incremental category The result that property finger is found is also this, because it is unique.
This finger has the change of any one information in certain fault-tolerance, such as set S, may pass through it He finds target widget at unchanged information.
In the case where there is index, illustrate by its orient come control must be multiple, so there is index In the case of, without using Short-circuit fingers.
B, incremental attribute finger:One thread, take and 2 information are taken in the multiple information included from location data The mode that all information go matching is progressively increased to, seeking scope and positioning result is gradually reduced.
It is the flow chart for incremental attribute finger shown in Fig. 4, by Fig. 4 it is known that algorithm in iterative process Time overhead is N (N is the number of element property)
Initial sets S { A, B ... } and initial empty set S2 { };
An element is taken out in circulation from S, if sky, then returns and searches control failure;
The lookup for the element being added S2 is concentrated(Actually S2 lookup collection credit union is since 2 combinations of attributes, because 1 Individual attribute in Short-circuit search, here in order to describe unification, it is assumed that since 1).
By S2 set search control, the sequence of sets in S2 is AND operation.A, B are suppose there is, that searches successful Attribute meets A&&B.
If find unique single control, then it is assumed that search successfully, return to the control.If finding sky control, recognize For the information for thinking newly to add is to fail, then removes the information.If find multiple controls, then it is assumed that current attribute Combination can't be positioned uniquely, continued to attempt to a newly-increased attribute and reduced the scope.
C, XPATH attributes finger(This is optional lookup thread):A corresponding thread, when providing the redundant field Use.Searched for providing a redundancy when other fingers are come to nothing.But because XPATH is easily varied, this result Reliability is low.So it is only used as segmented result.
D, CSS fingers(This is optional lookup thread):A corresponding thread, is just used when providing the redundant field. CSS fingers use the grammer of css selectors, and speed is good compared with attribute increase and decrease finger, and stability is better than xpath, can conduct Extra matching way.
E, figure script(SIKULI)Finger(This is optional lookup thread):A corresponding thread, should providing Just used during redundant field.Control matching is carried out using sikuli picture recognition engine, the result and other selections that it is matched The result type that device returns is different, so the operation to the result is also to be operated with mouse-keyboard related sikuli thereafter.This The result of finger is also the important foundation of the result of control outward appearance monitoring.
Step 304, judge whether any of thread A, B and D thread finds control, if found, find Control is target widget, if do not found, into step 305;
Step 305, judge whether thread C finds control, if found, the control found is target widget, If do not found, into step 306;
Step 306, using the picture recognition in thread E result as the result of positioning action, and terminate whole positioning Flow.
Each bar thread parallel for being established is searched, and main thread simultaneously withouts waiting for all threads and terminates just to terminate.As long as remove The result that any one thread outside XPATH attribute fingers has non-NULL returns, then it is assumed that have found the control.In general Short-circuit is attempted to once, is returned if hit, its is fastest.Incremental attribute finger can attempt attribute Combination, it is relatively slow.
If Short-circuit, incremental attribute finger and CSS fingers fail to find control, return XPATH result.The result of sikuli selectors is referred to if XPATH also returns to sky, if the success of sikuli selectors Match somebody with somebody, then return to the result of picture recognition type, otherwise, all fingers all fail coupling element, then search control failure.
Step 307, main thread group of distributors, multiple lookup threads is established while search element, the lookup thread established Including at least the one or more in following 4 kinds:
F, the attribute finger to successively decrease:A corresponding thread, takes in the multiple information included from location data successively Choose information to be added in lookup set, untill all information that location data includes are successively decreased and taken.
The attribute finger to successively decrease is similar with being incremented by finger, is not both uniquely, and it can choose attribute from S complete or collected works always Add and search collection and stop until S attributes are decremented to sky.Because it is to have index situation using the attribute finger to successively decrease, then It can be multiple controls that the combination illustrated in the currently active attribute, which is searched, determine which is by index.So matching must be found Attribute maximum set, otherwise may index to the scope element that should not have.Successively decrease attribute finger algorithm it is as shown in Figure 5:
Take an information to add from S and search collection S2, searched by S2.
If the result set found is the scope that index can be indexed, this interim findings is deposited.Otherwise, removed from S2 This information.
Whether S is decremented to empty set, if not empty set, is recycled to and starts to operate, then takes an information to add from S and search Collection S2 is continued to search for, if empty set, selection result.
If interim findings set is present, the index elements of interim findings set are returned.Otherwise, empty, lookup is returned Failure.
C, XPATH attributes finger(This is optional lookup thread):A corresponding thread, when providing the redundant field Use.Searched for providing a redundancy when other fingers are come to nothing.But because XPATH is easily varied, this result Reliability is low.So it is only used as segmented result.
D, CSS fingers(This is optional lookup thread):A corresponding thread, is just used when providing the redundant field. Control is searched using the mode of css selectors, because css selectors generally more stable and uniqueness is preferable, providing should Redundancy finger.
E, SIKULI fingers(This is optional lookup thread):A corresponding thread, just makes when providing the redundant field With.Control matching is carried out using sikuli picture recognition engine, the result class that the result and other selectors that it is matched return Type is different, so the operation to the result thereafter and the mouse-keyboard with sikuli correlations operate.The result of this finger It is the important foundation of the result of control outward appearance monitoring.
Step 308, judge whether thread F or D find control, if found, the control found is target control Part, if do not found, into step 309;
Step 309, judge whether thread C finds control, if found, the control found is target widget, If do not found, into step 310;
Step 310, using the picture recognition in thread E result as the result of positioning action, and terminate whole positioning Flow.
Each bar thread parallel for being established is searched, and main thread simultaneously withouts waiting for all threads and terminates just to terminate.As long as The result that any one thread in addition to XPATH attribute fingers has non-NULL returns, then it is assumed that have found the control.If add Sikuli fingers, then the result of sikuli fingers can be waited, because the result can be used as control cosmetic variation to monitor Foundation.Quickly, the result for not interfering with main thread is corresponding for this finger speed.
If the attribute finger and CSS fingers that incrementally subtract fail to find element, XPATH result is returned.
The result of sikuli selectors is referred to if XPATH also returns to sky, if sikuli selector successful match, The result of picture recognition type is returned to, otherwise, all fingers all fail coupling element, then search element failure.
Wherein, when the web page area figure of target widget is contained in location data, the outward appearance prison of control can be provided simultaneously Control.The logic of monitoring is as shown in Figure 6:
SIKULI picture engines position to control, if positioned successfully, then it represents that the outward appearance of control is simultaneously unchanged.
If positioning is unsuccessful, then judges other positioning methods of the non-SIKULI based on web(Incremental attribute finger, CSS, XPATH etc.)Whether succeed, if it fails, representing that the outward appearance of control and built-in attribute all change there occurs nonlocalizable Become.Nonlocalizable change information is returned into user.
If positioned successfully, take other modes and position successful result, represent control outward appearance change, but inside still may be used The control is navigated in a manner of being searched by attribute.The cosmetic variation of control is probably potential bug risks.
By obtained control, coordinate+size information of control is obtained.
Sectional drawing is carried out to the specified size area of the coordinate position.(It can generally be cut according to certain exaggerated scale Figure).
The change of prompting control simultaneously returns for the control information of sectional drawing in displaying user.
In this manner, can in automation process, in addition to verifying the functional availability of webpage, is also carried out simultaneously The outward appearance monitoring of critical function control.
Furthermore, it is contemplated that the DOM for being mainly used in browser is searched or operation, in order to support multiple different viewings well Device product.In a particular application, using javascript programming language and similar selenium actuation techniques, can reach most Good implementation result.Simultaneously sikuli calling is needed to encapsulate the calling of java class libraries, ruby, python, java can be provided Support well, while it is good implementation selection there are corresponding selenium api. again.
Embodiment 3
The present embodiment provides a kind of multichannel webpage control positioner, and the localization method of above-described embodiment 1 and 2 can be achieved, The device as shown in fig. 7, comprises:
Extraction module, effective attribute of target widget is extracted to set A, in current DOM structure, searches label and attribute Set and the effectively attribute are to the identical controls of set A, if found, and the control found would be with target widget Fraternal control, then position of the target widget in fraternal control is indicated using index information, and by the index information and described had Attribute is imitated to location datas of the set A as target widget;If find, but the control found and the non-brother of target widget Control, then the location data of the parent control of target widget is extracted, and by the positioning of the parent control for the target widget extracted Data and the effectively attribute are to location datas of the set A as target widget;If do not found, by effective attribute To location datas of the set A as target widget;
Wherein, effective attribute is to comprising at least tag types, the label of control, property value to sequence in set A.
Preferably, said extracted module, following one or more of information can also be extracted from target widget, and will extraction Information be added in the location data of target widget:
Cascading Style Sheet(CSS)Selector information, active path information, page area figure information.
Wherein, CSS selector, xpath are often applied in the lookup of target widget.Therefore, the positioning number of target widget According to containing CSS selector information, active path information, it is possible to provided when self-adjusting location algorithm can not find target widget One alternative approach, ensure the stability of control positioning.
And the page area figure information of target widget can also be added in location data, this pictorial information can not only be used Positioned in control, moreover it can be used to which control outward appearance monitors.But control positioning is nor affected on to a certain extent without it, so and can The additional data of choosing.
Locating module, the location data for the target widget extracted according to extraction module position to target widget.
In the present embodiment, above-mentioned locating module includes again:
Judge module, judge whether include the index information in the location data;
First processing units, when having the index information in location data, establish multiple lookup threads while search institute Target widget is stated, the lookup thread established comprises at least the attribute to successively decrease and searches thread.
Alternatively, first processing units can also establish following one or more of threads:
CSS searches thread, figure script searches thread, active path information attribute searches thread.
Second processing unit, do not include in location data when having the index information, establish multiple lookup threads simultaneously The target widget is searched, the thread of searching established comprises at least partial circulating lookup thread and incremental attribute lookup thread.
Alternatively, second processing unit can also establish following one or more of threads:
CSS searches thread, figure script searches thread, active path information attribute searches thread.
Changed in view of the appearance style of control, it is understood that there may be potential the problem of influenceing positioning.Therefore, it is above-mentioned Device can also increase a monitoring module, be monitored for control outward appearance, when including page area figure information in location data, When being changed according to the outward appearance of the page area figure INFORMATION DISCOVERY target widget, the control outward appearance of renewal analyze and anti- Feed user.
From above-described embodiment as can be seen that technical scheme has the following advantages that:
Stability is high:According to above-mentioned lookup algorithm, it can be seen that the attribute even in target widget occurs to a certain degree Increase and decrease, renewal or delete.As long as other attributes of the target widget are still effective, then the control just can be successfully positioned. Meanwhile even if the current attribute of target widget fails or influenceed by other elements, as long as its outward appearance does not change, area can also be passed through Domain figure recognizes the control.
Efficiency high:Search procedure in preferred scheme is because having multi-channel approach while carrying out, and centre returns to knot with most fast The path of fruit is as response, therefore location efficiency is high.Wherein, the fastest response time is 1(Short-circuit, CSS), most slowly Speed be N(N is the number that the location data of target widget includes information), when the exhaustive complete all information of the finger that incrementally successively decreases In the case of be N.After taking parallel multithread, positioning cost arrives N for 1, identical with accurate formula attributes match efficiency.
One of ordinary skill in the art will appreciate that all or part of step in the above method can be instructed by program Related hardware is completed, and described program can be stored in computer-readable recording medium, such as read-only storage, disk or CD Deng.Alternatively, all or part of step of above-described embodiment can also be realized using one or more integrated circuits.Accordingly Ground, each module/unit in above-described embodiment can be realized in the form of hardware, can also use the shape of software function module Formula is realized.The application is not restricted to the combination of the hardware and software of any particular form.
It is described above, it is only the preferred embodiments of the present invention, is not intended to limit the scope of the present invention.It is all this Within the spirit and principle of invention, any modification, equivalent substitution and improvements done etc., the protection model of the present invention should be included in Within enclosing.

Claims (16)

1. a kind of multichannel webpage control localization method, it is characterised in that this method includes:
Effective attribute of target widget is extracted to set A;
In current file object model (DOM) structure, search label and attribute set and the effectively attribute is complete to set A Identical control;
If find, and the control found and target widget are fraternal control, then indicate target widget using index information Position in fraternal control, and using the index information and the effectively location data of the attribute to set A as target widget;
If find, but the control found and the non-fraternal control of target widget, then extract the parent control of target widget Location data, the location data for the parent control that the target widget does not have to configure target widget during parent control is sky, and will The location data of the parent control of the target widget and the effectively attribute are to location datas of the set A as target widget;
If do not found, using the effectively location data of the attribute to set A as target widget;
Target widget is positioned according to the location data of the target widget.
2. the method as described in claim 1, it is characterised in that it is described effectively attribute in set A including at least tag types, The label of control, property value are to sequence.
3. method as claimed in claim 1 or 2, it is characterised in that this method also includes:
Following one or more of information are extracted also from the target widget, and the information of extraction is added to the target widget Location data in:
Cascading Style Sheet (CSS) selector information, active path information, page area figure information.
4. method as claimed in claim 3, it is characterised in that entered according to the location data of the target widget to target widget The process of row positioning includes:
When judging to include the index information in the location data, establish multiple lookup threads while search the target control Part, the lookup thread established comprise at least the attribute to successively decrease and search thread;
The attribute that successively decreases is searched thread and taken to be chosen information in the multiple information included from location data and is added to successively Search in set, untill all information that location data includes are successively decreased and taken.
5. method as claimed in claim 4, it is characterised in that when including the index information in the location data, build Vertical lookup thread also includes following one or more of threads:
CSS searches thread, figure script searches thread, active path information attribute searches thread.
6. method as claimed in claim 3, it is characterised in that entered according to the location data of the target widget to target widget The process of row positioning includes:
Judge not including when having the index information in the location data, establish multiple lookup threads while search the target Control, the thread of searching established comprise at least partial circulating lookup thread and incremental attribute lookup thread;
The partial circulating searches thread and establishes a thread for each factor of searching, and the lookup factor is the location data In an information;
The incremental attribute lookup thread, which is taken in the multiple information included from location data, takes 2 information to progressively increase to All information go the mode of matching, and seeking scope and positioning result is gradually reduced.
7. method as claimed in claim 6, it is characterised in that do not include in the location data when having the index information, The lookup thread of foundation also includes following one or more of threads:
CSS searches thread, figure script searches thread, active path information attribute searches thread.
8. method as claimed in claim 3, it is characterised in that this method also includes:
The control outward appearance of the target widget is monitored, when the control outward appearance of the target widget changes, the control to renewal Part outward appearance is analyzed and feeds back to user.
9. a kind of multichannel webpage control positioner, it is characterised in that the device includes:
Extraction module, effective attribute of target widget is extracted to set A, in current file object model (DOM) structure, is searched Label and attribute set with the effectively attribute to the identical controls of set A, if find, and the control found and Target widget is fraternal control, then indicates position of the target widget in fraternal control using index information, and the index is believed Breath and the effectively attribute are to location datas of the set A as target widget;If find, but the control and target found The non-fraternal control of control, then the location data of the parent control of target widget is extracted, when the target widget does not have parent control The location data for configuring the parent control of target widget is sky, and by the location data of the parent control of the target widget and institute Effective attribute is stated to location datas of the set A as target widget;If do not found, by the effectively attribute to set A Location data as target widget;
Locating module, target widget is positioned according to the location data of the target widget.
10. device as claimed in claim 9, it is characterised in that the effectively attribute is to comprising at least tag class in set A Type, the label of control, property value are to sequence.
11. the device as described in claim 9 or 10, it is characterised in that
The extraction module, following one or more of information are extracted also from the target widget, and the information of extraction is added Into the location data of the target widget:
Cascading Style Sheet (CSS) selector information, active path information, page area figure information.
12. device as claimed in claim 11, it is characterised in that the locating module includes:
Judge module, judge whether include the index information in the location data;
First processing units, when having the index information in the location data, establish multiple lookup threads while search institute Target widget is stated, the lookup thread established comprises at least the attribute to successively decrease and searches thread;
The attribute that successively decreases is searched thread and taken to be chosen information in the multiple information included from location data and is added to successively Search in set, untill all information that location data includes are successively decreased and taken.
13. device as claimed in claim 12, it is characterised in that the lookup thread that the first processing units are established also includes Following one or more of threads:
CSS searches thread, figure script searches thread, active path information attribute searches thread.
14. device as claimed in claim 12, it is characterised in that the locating module also includes:
Second processing unit, do not include in the location data when having the index information, establish multiple lookup threads simultaneously The target widget is searched, the thread of searching established comprises at least partial circulating lookup thread and incremental attribute lookup thread;
The partial circulating searches thread and establishes a thread for each factor of searching, and the lookup factor is the location data In an information;
The incremental attribute lookup thread, which is taken in the multiple information included from location data, takes 2 information to progressively increase to All information go the mode of matching, and seeking scope and positioning result is gradually reduced.
15. device as claimed in claim 14, it is characterised in that the lookup thread that the second processing unit is established also includes Following one or more of threads:
CSS searches thread, figure script searches thread, active path information attribute searches thread.
16. device as claimed in claim 11, it is characterised in that the device also includes:
Monitoring module, the control outward appearance of the target widget is monitored, it is right when the control outward appearance of the target widget changes The control outward appearance of renewal is analyzed and feeds back to user.
CN201310590812.1A 2013-11-20 2013-11-20 A kind of multichannel webpage control localization method and device Active CN104657377B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310590812.1A CN104657377B (en) 2013-11-20 2013-11-20 A kind of multichannel webpage control localization method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310590812.1A CN104657377B (en) 2013-11-20 2013-11-20 A kind of multichannel webpage control localization method and device

Publications (2)

Publication Number Publication Date
CN104657377A CN104657377A (en) 2015-05-27
CN104657377B true CN104657377B (en) 2018-04-03

Family

ID=53248523

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310590812.1A Active CN104657377B (en) 2013-11-20 2013-11-20 A kind of multichannel webpage control localization method and device

Country Status (1)

Country Link
CN (1) CN104657377B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105320603B (en) * 2015-12-04 2018-04-06 上海斐讯数据通信技术有限公司 The alignment system and method and its automated testing method of browser page element
CN107515821B (en) * 2016-06-17 2021-08-31 百度在线网络技术(北京)有限公司 Control testing method and device
CN106503124B (en) * 2016-10-19 2019-08-30 北京小米移动软件有限公司 The method and apparatus that multimedia file positions in the page
CN107015907A (en) * 2017-03-15 2017-08-04 中国地质大学(武汉) A kind of system and method for automatic accurate positioning webpage element
CN107220066B (en) * 2017-06-28 2020-08-28 北京金山安全软件有限公司 Method and device for releasing application software in application store
CN107402756B (en) * 2017-07-07 2020-07-31 北京小米移动软件有限公司 Method, device and terminal for drawing page
CN108804181B (en) * 2018-05-28 2021-09-28 北京小米移动软件有限公司 Control content obtaining method and device and storage medium
CN109977267B (en) * 2019-02-02 2021-09-03 北京云测信息技术有限公司 Method and device for determining XPath path
CN110727603A (en) * 2019-10-25 2020-01-24 重庆誉存大数据科技有限公司 Improved method for automatically testing positioning elements based on UI (user interface)
CN111221612B (en) * 2020-01-03 2023-08-29 北京小米移动软件有限公司 Method, device and medium for positioning user interface element
CN111368241B (en) * 2020-03-05 2023-05-30 苏州数字力量教育科技有限公司 Webpage element identification method based on XPath

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6571253B1 (en) * 2000-04-28 2003-05-27 International Business Machines Corporation Hierarchical view of data binding between display elements that are organized in a hierarchical structure to a data store that is also organized in a hierarchical structure
CN102622395A (en) * 2011-12-20 2012-08-01 深信服网络科技(深圳)有限公司 Method, server and client for assisting in accessing to webpage by application virtualization
CN102637128A (en) * 2012-03-14 2012-08-15 北京神州数码思特奇信息技术股份有限公司 Method for generating selection controls of general levels
CN103177115A (en) * 2013-04-03 2013-06-26 北京奇虎科技有限公司 Method and device of extracting page link of webpage

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6571253B1 (en) * 2000-04-28 2003-05-27 International Business Machines Corporation Hierarchical view of data binding between display elements that are organized in a hierarchical structure to a data store that is also organized in a hierarchical structure
CN102622395A (en) * 2011-12-20 2012-08-01 深信服网络科技(深圳)有限公司 Method, server and client for assisting in accessing to webpage by application virtualization
CN102637128A (en) * 2012-03-14 2012-08-15 北京神州数码思特奇信息技术股份有限公司 Method for generating selection controls of general levels
CN103177115A (en) * 2013-04-03 2013-06-26 北京奇虎科技有限公司 Method and device of extracting page link of webpage

Also Published As

Publication number Publication date
CN104657377A (en) 2015-05-27

Similar Documents

Publication Publication Date Title
CN104657377B (en) A kind of multichannel webpage control localization method and device
US10956786B2 (en) Machine learning based extraction of partition objects from electronic documents
US11550856B2 (en) Artificial intelligence for product data extraction
US8725775B2 (en) Identifying and abstracting a visualization point from an arbitrary two-dimensional dataset into a unified metadata for further consumption
CN101957816B (en) Webpage metadata automatic extraction method and system based on multi-page comparison
US20150339213A1 (en) Automated testing of an application system
US8756571B2 (en) Natural language text instructions
KR20210100600A (en) software testing
US20130315480A1 (en) Matching text to images
CN111079043B (en) Key content positioning method
CN108496190B (en) Annotation system for extracting attributes from electronic data structures
US11256912B2 (en) Electronic form identification using spatial information
US9817875B2 (en) Methods and systems for automated data characterization and extraction
CN109344355B (en) Automatic regression detection and block matching self-adaption method and device for webpage change
CN110119423A (en) A kind of data analysis method and computer readable storage medium of configurableization
Hong et al. Information extraction for search engines using fast heuristic techniques
CN111324797B (en) Method and device for precisely acquiring data at high speed
US20140301644A1 (en) Extracting Reading Order Text and Semantic Entities
CN110119441A (en) Text based on Hanzi structure clicks identifying code identification and filling method
CN106960058A (en) A kind of structure of web page alteration detection method and system
Xie et al. Psychologically-inspired, unsupervised inference of perceptual groups of GUI widgets from GUI images
CN111813443A (en) Method and tool for automatically filling code sample by JavaFX
CN111158973B (en) Web application dynamic evolution monitoring method
Trieschnigg et al. Ranking XPaths for extracting search result records
JP4172388B2 (en) Link diagnostic device, link diagnostic method, and link diagnostic program.

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20240208

Address after: # 01-21, Lai Zan Da Building 1, 51 Belarusian Road, Singapore

Patentee after: Alibaba Singapore Holdings Ltd.

Country or region after: Singapore

Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands

Patentee before: ALIBABA GROUP HOLDING Ltd.

Country or region before: United Kingdom