CN108363599B - User interface shows recognition methods and terminal device - Google Patents
User interface shows recognition methods and terminal device Download PDFInfo
- Publication number
- CN108363599B CN108363599B CN201810029553.8A CN201810029553A CN108363599B CN 108363599 B CN108363599 B CN 108363599B CN 201810029553 A CN201810029553 A CN 201810029553A CN 108363599 B CN108363599 B CN 108363599B
- Authority
- CN
- China
- Prior art keywords
- screenshot
- feature vector
- target
- interface
- user interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The present invention is suitable for technical field of mobile terminals, provides a kind of user interface and shows recognition methods and terminal device.This method comprises: obtaining the target interface screenshot of user interface to be detected, the feature vector of target interface screenshot is extracted；The normal interface screenshot of the normal user interface prestored is obtained, the feature vector of normal interface screenshot is extracted；According to the feature vector of the feature vector of target interface screenshot and normal interface screenshot, the COS distance of target interface screenshot Yu normal interface screenshot is determined；Judge whether COS distance is more than preset threshold；If it is determined that being more than preset threshold, then level and the position of each element control of user interface to be detected are obtained；Whether level and the position for detecting each element control are abnormal；If detecting, the level of each element control and position are normal, determine that user interface to be detected is shown normally, solve the problems, such as that existing user interface shows that script maintenance efficiency existing for detection method is low.
Description
Technical field
The invention belongs to technical field of mobile terminals more particularly to a kind of user interface to show that recognition methods and terminal are set
It is standby.
Background technique
Currently, several operation systems, such as Android operation system can be applied on the mobile terminals such as mobile phone, due to not
The mobile phone resolution ratio of same type machine is different, and several operation systems are applied to the mobile phone of different resolution, it is necessary to different resolution
Mobile phone is adapted, and can just user interface be made normally to show.Existing detection user interface show is detected by test script,
Entire test script is executed, is compared test result with the fixed value of setting after the completion of test, if the two is consistent, is surveyed
Family interface display on probation is normal, and otherwise, test user interface shows abnormal.But there are script maintenances to imitate for existing detection method
The low problem of rate needs to rewrite entire test case, while being also required to reset fixed value if user interface has adjustment,
Testing cost is high, is not suitable for practical application.
Summary of the invention
The embodiment of the present invention provides a kind of user interface and shows recognition methods and terminal device, to solve existing user interface
Show the low problem of script maintenance efficiency existing for detection method.
The first aspect of the embodiment of the present invention provides a kind of user interface display recognition methods, comprising:
The target interface screenshot for obtaining user interface to be detected extracts the feature vector of the target interface screenshot；
The normal interface screenshot for obtaining the normal user interface prestored, extracts the feature vector of the normal interface screenshot；
According to the feature vector of the feature vector of the target interface screenshot and the normal interface screenshot, the mesh is determined
It demarcates the COS distance of face screenshot and the normal interface screenshot；
Judge whether the COS distance is more than preset threshold；
If it is determined that the COS distance is more than the preset threshold, then each element of the user interface to be detected is obtained
The level of control and position；
Whether level and the position for detecting each element control are abnormal；
If detecting, the level of each element control and position are normal, determine that the user interface to be detected is aobvious
Show normal.
Optionally, the feature vector for extracting the target interface screenshot includes:
The first preset number sample, each the second preset number of sample extraction are chosen from the target interface screenshot
Feature；
According to the second preset number feature of the first preset number sample and each sample extraction, first is constituted
Matrix；
The second matrix is obtained according to the transposed matrix of first matrix and first matrix；
The feature vector for calculating second matrix determines the target interface according to the feature vector of second matrix
The feature vector of screenshot.
Optionally, the feature vector according to the target interface screenshot and the feature of the normal interface screenshot to
Amount, determines that the COS distance of the target interface screenshot and the normal interface screenshot includes:
Calculate separately the length of the feature vector of the target interface screenshot and the feature vector of the normal interface screenshot；
Obtain the inner product of the feature vector of the target interface screenshot and the feature vector of the normal interface screenshot；
According to the length of the feature vector of the target interface screenshot, the length of the feature vector of the normal interface screenshot
With the inner product, the COS distance of the target interface screenshot Yu the normal interface screenshot is determined.
Optionally, the level of each element control for obtaining the user interface to be detected and position include:
Operating status, which is generated, by Software Development Kit (Software Development Kit, SDK) obtains order；
The layer that order obtains each element control of the user interface to be detected in real time is obtained according to the operation state
Grade and position.
Optionally, the level of each element control of the detection and position whether include: extremely
The child control and child control pair in each element control are determined according to the level of each element control
The female control answered；
Whether the position of detection target child control is arranged in the front end of the corresponding target mother control of the target child control, and
Detect whether range shared by the target child control exceeds range shared by the target mother control, the target child control is to determine
Each element control in child control in any one child control.
The second aspect of the embodiment of the present invention provides a kind of user interface and shows identification terminal equipment, including memory,
Processor and storage in the memory and the computer program that can run on the processor, the processor execution
Following steps are realized when the computer program:
The target interface screenshot for obtaining user interface to be detected extracts the feature vector of the target interface screenshot；
The normal interface screenshot for obtaining the normal user interface prestored, extracts the feature vector of the normal interface screenshot；
According to the feature vector of the feature vector of the target interface screenshot and the normal interface screenshot, the mesh is determined
It demarcates the COS distance of face screenshot and the normal interface screenshot；
Judge whether the COS distance is more than preset threshold；
If it is determined that the COS distance is more than the preset threshold, then each element of the user interface to be detected is obtained
The level of control and position；
Whether level and the position for detecting each element control are abnormal；
If detecting, the level of each element control and position are normal, determine that the user interface to be detected is aobvious
Show normal.
Optionally, the feature vector for extracting the target interface screenshot includes:
The first preset number sample, each the second preset number of sample extraction are chosen from the target interface screenshot
Feature；
According to the second preset number feature of the first preset number sample and each sample extraction, first is constituted
Matrix；
The second matrix is obtained according to the transposed matrix of first matrix and first matrix；
The feature vector for calculating second matrix determines the target interface according to the feature vector of second matrix
The feature vector of screenshot.
Optionally, the feature vector according to the target interface screenshot and the feature of the normal interface screenshot to
Amount, determines that the COS distance of the target interface screenshot and the normal interface screenshot includes:
Calculate separately the length of the feature vector of the target interface screenshot and the feature vector of the normal interface screenshot；
Obtain the inner product of the feature vector of the target interface screenshot and the feature vector of the normal interface screenshot；
According to the length of the feature vector of the target interface screenshot, the length of the feature vector of the normal interface screenshot
With the inner product, the COS distance of the target interface screenshot Yu the normal interface screenshot is determined.
Optionally, the level of each element control of the detection and position whether include: extremely
The child control and child control pair in each element control are determined according to the level of each element control
The female control answered；
Whether the position of detection target child control is arranged in the front end of the corresponding target mother control of the target child control, and
Detect whether range shared by the target child control exceeds range shared by the target mother control, the target child control is to determine
Each element control in child control in any one child control.
The third aspect of the embodiment of the present invention provides a kind of computer readable storage medium, the computer-readable storage
Media storage has computer program, realizes that above-mentioned user interface such as shows identification side when the computer program is executed by processor
The step of method.
Existing beneficial effect is the embodiment of the present invention compared with prior art: the embodiment of the present invention is to be detected by obtaining
The target interface screenshot of user interface, determines the COS distance of target interface screenshot Yu normal interface screenshot, is determining the cosine
When distance is more than preset threshold, whether level and the position for detecting each element control of user interface to be detected are abnormal, are examining
When surveying all normal, determine that user interface to be detected is shown normally, solves existing user interface and show script existing for detection method
The low problem of maintenance efficiency, each element control of screenshot and user interface to be detected in combination with user interface to be detected
Level and position, detection user interface whether abnormal show, improve user interface show recognition accuracy, reduce testing cost,
It is suitble to practical application.
Detailed description of the invention
It to describe the technical solutions in the embodiments of the present invention more clearly, below will be to embodiment or description of the prior art
Needed in attached drawing be briefly described, it should be apparent that, the accompanying drawings in the following description is only of the invention some
Embodiment for those of ordinary skill in the art without any creative labor, can also be according to these
Attached drawing obtains other attached drawings.
Fig. 1 is the schematic flow diagram that a kind of user interface provided in an embodiment of the present invention shows recognition methods；
Fig. 2 be another embodiment of the present invention provides a kind of user interface show recognition methods schematic flow diagram；
Fig. 3 is the schematic flow diagram that a kind of user interface that yet another embodiment of the invention provides shows recognition methods；
Fig. 4 is the schematic flow diagram that a kind of user interface that further embodiment of this invention provides shows recognition methods；
Fig. 5 is the schematic flow diagram that a kind of user interface that further embodiment of this invention provides shows recognition methods；
Fig. 6 is the schematic block diagram that a kind of user interface provided in an embodiment of the present invention shows recognizer；
Fig. 7 is the schematic block diagram that a kind of user interface provided in an embodiment of the present invention shows identification terminal equipment.
Specific embodiment
In being described below, for illustration and not for limitation, the tool of such as particular system structure, technology etc is proposed
Body details, to understand thoroughly the embodiment of the present invention.However, it will be clear to one skilled in the art that there is no these specific
The present invention also may be implemented in the other embodiments of details.In other situations, it omits to well-known system, device, electricity
The detailed description of road and method, in case unnecessary details interferes description of the invention.
In order to illustrate technical solutions according to the invention, the following is a description of specific embodiments.
It is the schematic flow diagram that a kind of user interface provided in an embodiment of the present invention shows recognition methods referring to Fig. 1, Fig. 1,
In this embodiment, it is illustrated so that the angle of terminal is set out as an example, here, terminal can be smart phone, tablet computer etc.
Mobile terminal.As shown in Figure 1, in this embodiment, the treatment process of terminal may comprise steps of:
S101: obtaining the target interface screenshot of user interface to be detected, extracts the feature vector of the target interface screenshot.
Here, user interface to be detected is any one user interface for needing to detect, and user interface is referred to as making
User interface is the medium interacted between system and user with information exchange, it realizes internal form and the mankind of information
It can receive the conversion between form.
The mode for obtaining the target interface screenshot of user interface to be detected can be acquisition in real time, or when default
Between obtain, specific acquisition modes can be set according to actual needs.The target interface screenshot for obtaining user interface to be detected it
Afterwards, the target interface screenshot that can show acquisition, the prompt for whether reacquiring screenshot can also be generated, is obtained again if receiving
Take screenshot to request, the target interface screenshot of user interface to be detected reacquired according to the request, can by repeatedly obtain to
The target interface screenshot for detecting user interface selects satisfactory target circle from multiple target interface screenshots of acquisition
Face screenshot, such as the target interface screenshot that selection picture is clearest.
After the feature vector for extracting the target interface screenshot, the feature vector of extraction can be saved, is facilitated subsequent
Data query.
S102: obtaining the normal interface screenshot of the normal user interface prestored, extracts the feature of the normal interface screenshot
Vector.
Specifically, the mode for obtaining the normal user interface prestored include: can be stored in advance user interface type with just
The corresponding relationship of normal user interface, obtains the type of user interface to be detected, determines that the type obtained is corresponding according to above-mentioned relation
Normal user interface.
Interface when the above-mentioned normal user interface prestored can also normally be shown by obtaining user interface to be detected determines.
It obtains the mode of the normal interface screenshot of the normal user interface prestored and obtains the target of user interface to be detected
The mode of interface screenshot is identical, can be acquisition in real time, or obtain in preset time.It here, can also be by multiple
The normal interface screenshot for obtaining the normal user interface prestored selects one from multiple normal interface screenshots of acquisition and conforms to
The normal interface screenshot asked.
S103: it according to the feature vector of the feature vector of the target interface screenshot and the normal interface screenshot, determines
The COS distance of the target interface screenshot and the normal interface screenshot.
Here, COS distance, also referred to as cosine similarity, be use in vector space two vectorial angle cosine values as
Measure the measurement of the size of two inter-individual differences.
Vector is directive line segment in hyperspace, if the direction of two vectors is consistent, i.e. angle close to zero, that
The two vectors are with regard to close.
S104: judge whether the COS distance is more than preset threshold.
Specifically, preset threshold is arranged according to actual needs, if above-mentioned COS distance is more than the preset threshold, executes step
S105 can directly determine that above-mentioned user interface display to be detected is abnormal if above-mentioned COS distance is less than the preset threshold, into
One step can be reminded by alarm.
Determine that above-mentioned user interface display to be detected is abnormal above by COS distance, is directed to user interface to be detected
Show the larger situation of anomaly ratio, this is because it is above-mentioned after the target interface screenshot for obtaining user interface to be detected, it is extracted
The feature vector of target interface screenshot carries out the abnormal judgement of subsequent image display, feature vector according to the feature vector of image
It is the global feature of image, this judgment mode is determining in the case where user interface to be detected shows the lesser situation of anomaly ratio
When above-mentioned COS distance is more than preset threshold, it is easy to appear erroneous judgement, therefore when determining that above-mentioned COS distance is more than preset threshold
Execute following detection step.
S105: if it is determined that the COS distance is more than the preset threshold, then each of the user interface to be detected is obtained
The level of a element control and position.
Here, the level of each element control of above-mentioned user interface to be detected may include application in Mac, browsing
Device, iOS APP etc..
The level and position for obtaining each element control of the user interface to be detected can be in automatic test courses
In or obtain during multimachine compatibility test.
S106: whether the level of each element control of detection and position are abnormal.
Specifically, use to be detected can be obtained with the corresponding relationship of pre-set user interface type and abnormal control detected rule
The type at family interface determines the corresponding abnormal control detection rule of type of the user interface to be detected obtained according to above-mentioned relation
Then, according to the exception control detected rule detect above-mentioned each element control level and position it is whether abnormal.
S107: if detecting, the level of each element control and position are normal, determine the user to be detected
Interface display is normal.
Here, if detecting has the level of one or more element controls abnormal in the level of each element control,
Then determine that the user interface display to be detected is abnormal, the first exception information can be generated, it further can also be different according to first
Normal information generates the first warning message, is alarmed according to the first warning message.If detecting the position of each element control
There is the malposition of one or more element controls in setting, then determines that the user interface display to be detected is abnormal, can be generated
Second exception information, further can also according to the second exception information generate the second warning message, according to the second warning message into
Row alarm.
It is evidenced from the above discussion that the user interface of the embodiment of the present invention shows recognition methods, existing user circle can solve
Face shows the low problem of script maintenance efficiency existing for detection method, simultaneously because can be with by the screenshot of user interface to be detected
The bigger abnormal show of user interface to be detected is detected, level and position in conjunction with each element control of user interface to be detected
The smaller abnormal show of user interface can further be detected by setting, and improved user interface and shown recognition accuracy, reduce and survey
Cost is tried, practical application is suitble to.
Referring to Fig. 2, Fig. 2 be another embodiment of the present invention provides a kind of user interface show recognition methods signal stream
Cheng Tu.The difference of embodiment corresponding with Fig. 1 is: the target interface screenshot for obtaining user interface to be detected extracts institute
The feature vector for stating target interface screenshot may include S201.Wherein S102~S107 in S202~S207 and a upper embodiment
It is identical, referring specifically to the associated description of S102~S107 in above-described embodiment, do not repeat herein.Specifically, S201 can wrap
Include S2011~S2014:
S2011: obtaining the target interface screenshot of user interface to be detected, and it is pre- that first is chosen from the target interface screenshot
If number sample, each the second preset number of sample extraction feature.
Here, the first preset number and the second preset number can be set according to actual needs, such as can be from acquisition
M sample, n feature of each sample extraction are chosen in screenshot.
S2012: according to the second preset number feature of the first preset number sample and each sample extraction, structure
At the first matrix.
It specifically,, can general for n feature of each sample extraction to choose m sample in the above-mentioned screenshot from acquisition
Each sample constitutes the first matrix A of m*n as a line.
S2013: the second matrix is obtained according to the transposed matrix of first matrix and first matrix.
Here it is possible to which the matrix transposition of the first matrix is obtained the second Matrix C=A (t) * A multiplied by original matrix.
S2014: the feature vector of second matrix is calculated, the mesh is determined according to the feature vector of second matrix
It demarcates the feature vector of face screenshot.
Specifically, the feature vector of above-mentioned Matrix C is found out, this feature vector is the feature vector of screenshot.
Here, by choosing sample from target interface screenshot, from each sample extraction feature, target interface screenshot is obtained
Feature vector, method is simple, and the result of acquisition is accurate, be suitble to application.
Referring to Fig. 3, Fig. 3 is the signal stream that a kind of user interface that yet another embodiment of the invention provides shows recognition methods
Cheng Tu.The difference of embodiment corresponding with Fig. 1 is: the feature vector according to the target interface screenshot and described normal
The feature vector of interface screenshot determines that the COS distance of the target interface screenshot and the normal interface screenshot may include
S303.Wherein S301~S302 is identical as S101~S102 in a upper embodiment, in S304~S307 and a upper embodiment
S104~S107 is identical, referring specifically to the associated description of S101~S102, S104~S107 in above-described embodiment, does not go to live in the household of one's in-laws on getting married herein
It states.Specifically, S303 may include S3031~S3033:
S3031: the feature vector of the target interface screenshot and the feature vector of the normal interface screenshot are calculated separately
Length.
S3032: the interior of the feature vector of the target interface screenshot and the feature vector of the normal interface screenshot is obtained
Product.
S3033: according to the length of the feature vector of the target interface screenshot, the feature vector of the normal interface screenshot
Length and the inner product, determine the COS distance of the target interface screenshot Yu the normal interface screenshot.
Specifically, the cosine law describes the relationship of any one angle and three sides in triangle.Given triangle
The angle that the cosine law finds out each angle of triangle can be used in three sides.It is assumed that three sides of a triangle are a, b and c, correspond to
Three angles be A, B and C, then the cosine of angle A are as follows:
If regarding the both sides b and c of triangle as two vectors, above-mentioned formula is equivalent to:
Wherein denominator indicates the length of two vectors, and molecule indicates the inner product of two vectors.
For a specific example, if the feature vector of the feature vector of target interface screenshot X and normal interface screenshot Y
It is respectively: x1, x2 ..., x10 and y1, y2 ..., y10.
Then, the COS distance between them can be indicated with the cosine value of angle between them:
When the co sinus vector included angle of two screenshots is equal to 1, the two screenshots repeat completely, when the cosine value of angle is close
When 1, two screenshots are similar, and folder cosine of an angle is smaller, and two screenshots are more uncorrelated.
Referring to Fig. 4, Fig. 4 is the signal stream that a kind of user interface that further embodiment of this invention provides shows recognition methods
Cheng Tu.The difference of embodiment corresponding with Fig. 1 is: described if it is determined that the COS distance is then obtained more than the preset threshold
The level and position for taking each element control of the user interface to be detected may include S405.Wherein S401~S404 with it is upper
S101~S104 in one embodiment is identical, and S406~S407 is identical as S106~S107 in a upper embodiment, please specifically join
The associated description for readding S101~S104 in above-described embodiment, S106~S107, does not repeat herein.Specifically, S405 may include
S4051~S4052:
S4051: if it is determined that the COS distance is more than the preset threshold, then operating status is generated by SDK and obtains life
It enables.
S4052: each element control that order obtains the user interface to be detected in real time is obtained according to the operation state
The level of part and position.
Here, operating status is generated by SDK and obtains order, obtained in real time according to operating status acquisition order to be detected
The level and location information (rectangular coordinates for returning to two coordinate points in upper left and bottom right) of each element control of user interface, letter
Singly, conveniently, accelerate subsequent processing.
Referring to Fig. 5, Fig. 5 is the signal stream that a kind of user interface that further embodiment of this invention provides shows recognition methods
Cheng Tu.The difference of embodiment corresponding with Fig. 1 is: whether the level of each element control of detection and position are abnormal
It may include S506.Wherein S501~S505 is identical as S101~S105 in a upper embodiment, in S507 and a upper embodiment
S107 it is identical, referring specifically to the associated description of S101~S105, S107 in above-described embodiment, do not repeat herein.Specifically,
S506 may include S5061~S5062:
S5061: the child control in each element control, Yi Jizi are determined according to the level of each element control
The corresponding female control of control.
S5062: whether the position of detection target child control is arranged in the corresponding target mother control of the target child control
Front end, and detect whether range shared by the target child control exceeds range shared by the target mother control, the target control
Part is any one child control in the child control in determining each element control.
Specifically, by taking control one, control two as an example, control two is the child control of control one, and control one is the mother of control two
Control, for control two in the front end of control one, control one is rectangle frame, and position is (x1, y1) (x2, y2), represents a left side for rectangle frame
Inferior horn and the lower right corner, control two are also rectangle frame, and position is (x3, y3) (x4, y4).The corresponding inspection of above-mentioned user interface to be detected
Gauge then can be with are as follows: the front end of female control is arranged in child control, and range shared by child control cannot be beyond range shared by female control.I.e.
Above-mentioned control two is in one front end of control, and x3>x1, x4<x2, y1>y3, y2<y4 just meet rule, otherwise user interface to be detected
Display is just problematic.
Here, whether judge that user interface to be detected is aobvious extremely by the level and position that detect above-mentioned each element control
Whether problematic show, due to being judged according to control location, can find that smaller interface display is abnormal, and satisfaction is actually answered
With needs.
It should be understood that the size of the serial number of each step is not meant that the order of the execution order in above-described embodiment, each process
Execution sequence should be determined by its function and internal logic, the implementation process without coping with the embodiment of the present invention constitutes any limit
It is fixed.
Recognition methods is shown corresponding to user interface described in foregoing embodiments, and Fig. 6 shows offer of the embodiment of the present invention
User interface show recognizer running environment schematic diagram.For ease of description, it illustrates only related to the present embodiment
Part.
In the present embodiment, the user interface shows that recognizer 600 is installed and run in terminal device 60.It should
Terminal device 60 can be mobile terminal, palm PC, server etc..The terminal device 60 may include, but be not limited only to, storage
Device 601, processor 602 and display 603.Fig. 6 illustrates only the terminal device 60 with component 601-603, it should be understood that
Be, it is not required that implement all components shown, the implementation that can be substituted is more or less component.
The memory 601 can be the internal storage unit of the terminal device 60 in some embodiments, such as should
The hard disk or memory of terminal device 60.The memory 601 is also possible to the terminal device 60 in further embodiments
The plug-in type hard disk being equipped on External memory equipment, such as the terminal device 60, intelligent memory card (Smart Media
Card, SMC), secure digital (Secure Digital, SD) card, flash card (Flash Card) etc..Further, described to deposit
Reservoir 601 can also both including the terminal device 60 internal storage unit and also including External memory equipment.The memory
601 for storing the application software and Various types of data for being installed on the terminal device 60, such as user interface display identification
The program code etc. of program 600.The memory 601 can be also used for temporarily storing the number that has exported or will export
According to.
The processor 602 can be a central processing unit (Central Processing in some embodiments
Unit, CPU), microprocessor or other data processing chips, for run the program code stored in the memory 601 or
Data are handled, such as executes the user interface and shows recognizer 600 etc..
The display 603 can be light-emitting diode display, liquid crystal display, touch control type LCD in some embodiments and show
Device and Organic Light Emitting Diode (Organic Light-Emitting Diode, OLED) touch device etc..The display 603
For being shown in the information handled in the terminal device 60 and for showing visual user interface, such as application menu
Interface, application icon interface etc..The component 601-603 of the terminal device 60 is in communication with each other by system bus.
Referring to Fig. 7, being the functional block diagram that user interface provided in an embodiment of the present invention shows recognizer 600.?
In the present embodiment, the user interface shows that recognizer 600 can be divided into one or more modules, it is one or
The multiple modules of person are stored in the memory 601, and (the present embodiment is the processor by one or more processors
602) performed, to complete the present invention.For example, the user interface shows that recognizer 600 can be divided in Fig. 7
It is single at the first screenshot processing unit 701, the second screenshot processing unit 702, COS distance determination unit 703, COS distance judgement
Member 704, element control acquiring unit 705, element control detection unit 706 and user interface show judging unit 707.The present invention
So-called unit is the series of computation machine program instruction section for referring to complete specific function, than program more suitable for described in description
User interface shows implementation procedure of the recognizer 600 in the terminal device 60.The mould will specifically be introduced by being described below
The function of block 701-707.
Wherein, the first screenshot processing unit 701 extracts institute for obtaining the target interface screenshot of user interface to be detected
State the feature vector of target interface screenshot.Second screenshot processing unit 702, for obtaining the normal of the normal user interface prestored
Interface screenshot extracts the feature vector of the normal interface screenshot.COS distance determination unit 703, for according to the target
The feature vector of the feature vector of interface screenshot and the normal interface screenshot determines the target interface screenshot and described normal
The COS distance of interface screenshot.COS distance judging unit 704, for judging whether the COS distance is more than preset threshold.
Element control acquiring unit 705 is used for if it is determined that the COS distance then obtains the use to be detected more than the preset threshold
The level of each element control at family interface and position.Element control detection unit 706, for detecting each element control
Level and position it is whether abnormal.User interface shows judging unit 707, if for detecting the layer of each element control
Grade and position are normal, then determine that the user interface to be detected is shown normally.
Optionally, the first screenshot processing unit 701 can be divided into sample selection unit 7011, the first matrix structure
At unit 7012, the second matrix obtaining unit 7013 and feature vector determination unit 7014.
Wherein, sample selection unit 7011, for obtaining the target interface screenshot of user interface to be detected, from the target
The first preset number sample, each the second preset number of sample extraction feature are chosen in the screenshot of interface.First matrix is constituted
Unit 7012, for the second preset number feature according to the first preset number sample and each sample extraction, structure
At the first matrix.Second matrix obtaining unit 7013, for the transposed matrix according to first matrix and first matrix
Obtain the second matrix.Feature vector determination unit 7014, for calculating the feature vector of second matrix, according to described second
The feature vector of matrix determines the feature vector of the target interface screenshot.
Optionally, the COS distance determination unit 703 can be divided into vector length computing unit 7031, in vector
Product obtaining unit 7032 and COS distance processing unit 7033.
Wherein, vector length computing unit 7031, for calculating separately feature vector and the institute of the target interface screenshot
State the length of the feature vector of normal interface screenshot.Inner product of vectors obtaining unit 7032, for obtaining the target interface screenshot
Feature vector and the normal interface screenshot feature vector inner product.COS distance processing unit 7033, for according to institute
The length and the inner product of the length of the feature vector of target interface screenshot, the feature vector of the normal interface screenshot are stated, really
The COS distance of fixed the target interface screenshot and the normal interface screenshot.
Optionally, the element control acquiring unit 705 can be divided into order generation unit 7051 and element control
Processing unit 7052.
Wherein, order generation unit 7051 is used for if it is determined that the COS distance then passes through more than the preset threshold
SDK generates operating status and obtains order.Element control processing unit 7052, it is real-time for obtaining order according to the operation state
Obtain level and the position of each element control of the user interface to be detected.
Optionally, the element control detection unit 706 can also be divided into primary and secondary control determination unit 7061 and son
Female control detection unit 7062.
Wherein, primary and secondary control determination unit 7061, it is each described for being determined according to the level of each element control
The corresponding female control of child control and child control in element control.Primary and secondary control detection unit 7062, for detecting target
Whether the position of control is arranged in the front end of the corresponding target mother control of the target child control, and detects the target child control
Whether shared range exceeds range shared by the target mother control, and the target child control is determining each element control
In child control in any one child control.
It is apparent to those skilled in the art that for convenience of description and succinctly, only with above-mentioned each function
Can unit, module division progress for example, in practical application, can according to need and by above-mentioned function distribution by different
Functional unit, module are completed, i.e., the internal structure of described device is divided into different functional unit or module, more than completing
The all or part of function of description.Each functional unit in embodiment, module can integrate in one processing unit, can also
To be that each unit physically exists alone, can also be integrated in one unit with two or more units, it is above-mentioned integrated
Unit both can take the form of hardware realization, can also realize in the form of software functional units.In addition, each function list
Member, the specific name of module are also only for convenience of distinguishing each other, the protection scope being not intended to limit this application.Above system
The specific work process of middle unit, module, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In the above-described embodiments, it all emphasizes particularly on different fields to the description of each embodiment, is not described in detail or remembers in some embodiment
The part of load may refer to the associated description of other embodiments.
Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually
It is implemented in hardware or software, the specific application and design constraint depending on technical solution.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
The scope of the present invention.
In embodiment provided by the present invention, it should be understood that disclosed device/terminal device and method, it can be with
It realizes by another way.For example, device described above/terminal device embodiment is only schematical, for example, institute
The division of module or unit is stated, only a kind of logical function partition, there may be another division manner in actual implementation, such as
Multiple units or components can be combined or can be integrated into another system, or some features can be ignored or not executed.Separately
A bit, shown or discussed mutual coupling or direct-coupling or communication connection can be through some interfaces, device
Or the INDIRECT COUPLING or communication connection of unit, it can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
If the integrated module/unit be realized in the form of SFU software functional unit and as independent product sale or
In use, can store in a computer readable storage medium.Based on this understanding, the present invention realizes above-mentioned implementation
All or part of the process in example method, can also instruct relevant hardware to complete, the meter by computer program
Calculation machine program can be stored in a computer readable storage medium, the computer program when being executed by processor, it can be achieved that on
The step of stating each embodiment of the method.Wherein, the computer program includes computer program code, the computer program generation
Code can be source code form, object identification code form, executable file or certain intermediate forms etc..The computer-readable medium
It may include: any entity or device, recording medium, USB flash disk, mobile hard disk, magnetic that can carry the computer program code
Dish, CD, computer storage, read-only memory (Read-Only Memory, ROM), random access memory (Random
Access Memory, RAM), electric carrier signal, telecommunication signal and software distribution medium etc..It should be noted that the meter
The content that calculation machine readable medium includes can carry out increase and decrease appropriate according to the requirement made laws in jurisdiction with patent practice,
It such as does not include electric carrier signal and telecommunications according to legislation and patent practice, computer-readable medium in certain jurisdictions
Signal.
Embodiment described above is merely illustrative of the technical solution of the present invention, rather than its limitations；Although referring to aforementioned reality
Applying example, invention is explained in detail, those skilled in the art should understand that: it still can be to aforementioned each
Technical solution documented by embodiment is modified or equivalent replacement of some of the technical features；And these are modified
Or replacement, the spirit and scope for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution should all
It is included within protection scope of the present invention.
Claims (10)
1. a kind of user interface shows recognition methods characterized by comprising
The target interface screenshot for obtaining user interface to be detected extracts the feature vector of the target interface screenshot；
The normal interface screenshot for obtaining the normal user interface prestored, extracts the feature vector of the normal interface screenshot；
According to the feature vector of the feature vector of the target interface screenshot and the normal interface screenshot, target circle is determined
The COS distance of face screenshot and the normal interface screenshot；
Judge whether the COS distance is more than preset threshold；
If it is determined that the COS distance is more than the preset threshold, then each element control of the user interface to be detected is obtained
Level and position；
Whether level and the position for detecting each element control are abnormal；
If detecting, the level of each element control and position are normal, determine that the user interface to be detected is shown just
Often.
2. user interface as described in claim 1 shows recognition methods, which is characterized in that described to extract the target interface section
The feature vector of figure includes:
The first preset number sample is chosen from the target interface screenshot, each the second preset number of sample extraction is special
Sign；
According to the second preset number feature of the first preset number sample and each sample extraction, the first square is constituted
Battle array；
The second matrix is obtained according to the transposed matrix of first matrix and first matrix；
The feature vector for calculating second matrix determines the target interface screenshot according to the feature vector of second matrix
Feature vector.
3. user interface as described in claim 1 shows recognition methods, which is characterized in that described to be cut according to the target interface
The feature vector of the feature vector of figure and the normal interface screenshot determines that the target interface screenshot and the normal interface are cut
The COS distance of figure includes:
Calculate separately the length of the feature vector of the target interface screenshot and the feature vector of the normal interface screenshot；
Obtain the inner product of the feature vector of the target interface screenshot and the feature vector of the normal interface screenshot；
According to the length of the feature vector of the target interface screenshot, the length of the feature vector of the normal interface screenshot and institute
Inner product is stated, determines the COS distance of the target interface screenshot Yu the normal interface screenshot.
4. user interface as described in claim 1 shows recognition methods, which is characterized in that described to obtain the user to be detected
The level of each element control at interface and position include:
Operating status, which is generated, by Software Development Kit SDK obtains order；
According to the operation state obtain order obtain in real time the user interface to be detected each element control level and
Position.
5. user interface as described in claim 1 shows recognition methods, which is characterized in that each element control of detection
The level of part and position whether include: extremely
Determine that child control in each element control and child control are corresponding according to the level of each element control
Female control；
Whether the position of detection target child control is arranged in the front end of the corresponding target mother control of the target child control, and detects
Whether range shared by the target child control exceeds range shared by the target mother control, and the target child control is each of determination
Any one child control in child control in a element control.
6. a kind of user interface shows identification terminal equipment, which is characterized in that including memory, processor and be stored in described
In memory and the computer program that can run on the processor, the processor are realized when executing the computer program
Following steps:
The target interface screenshot for obtaining user interface to be detected extracts the feature vector of the target interface screenshot；
The normal interface screenshot for obtaining the normal user interface prestored, extracts the feature vector of the normal interface screenshot；
According to the feature vector of the feature vector of the target interface screenshot and the normal interface screenshot, target circle is determined
The COS distance of face screenshot and the normal interface screenshot；
Judge whether the COS distance is more than preset threshold；
If it is determined that the COS distance is more than the preset threshold, then each element control of the user interface to be detected is obtained
Level and position；
Whether level and the position for detecting each element control are abnormal；
If detecting, the level of each element control and position are normal, determine that the user interface to be detected is shown just
Often.
7. user interface as claimed in claim 6 shows identification terminal equipment, which is characterized in that described to extract target circle
The feature vector of face screenshot includes:
The first preset number sample is chosen from the target interface screenshot, each the second preset number of sample extraction is special
Sign；
According to the second preset number feature of the first preset number sample and each sample extraction, the first square is constituted
Battle array；
The second matrix is obtained according to the transposed matrix of first matrix and first matrix；
The feature vector for calculating second matrix determines the target interface screenshot according to the feature vector of second matrix
Feature vector.
8. user interface as claimed in claim 6 shows identification terminal equipment, which is characterized in that described according to target circle
The feature vector of the feature vector of face screenshot and the normal interface screenshot determines the target interface screenshot and the normal boundary
The COS distance of face screenshot includes:
Calculate separately the length of the feature vector of the target interface screenshot and the feature vector of the normal interface screenshot；
Obtain the inner product of the feature vector of the target interface screenshot and the feature vector of the normal interface screenshot；
According to the length of the feature vector of the target interface screenshot, the length of the feature vector of the normal interface screenshot and institute
Inner product is stated, determines the COS distance of the target interface screenshot Yu the normal interface screenshot.
9. user interface as claimed in claim 6 shows identification terminal equipment, which is characterized in that each member of detection
The level of plain control and position whether include: extremely
Determine that child control in each element control and child control are corresponding according to the level of each element control
Female control；
Whether the position of detection target child control is arranged in the front end of the corresponding target mother control of the target child control, and detects
Whether range shared by the target child control exceeds range shared by the target mother control, and the target child control is each of determination
Any one child control in child control in a element control.
10. a kind of computer readable storage medium, the computer-readable recording medium storage has computer program, and feature exists
In realization user interface as described in any one of claim 1 to 5 shows identification side when the computer program is executed by processor
The step of method.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810029553.8A CN108363599B (en) | 2018-01-12 | 2018-01-12 | User interface shows recognition methods and terminal device |
| PCT/CN2018/097518 WO2019136961A1 (en) | 2018-01-12 | 2018-07-27 | User interface display identification method, terminal device, storage medium and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810029553.8A CN108363599B (en) | 2018-01-12 | 2018-01-12 | User interface shows recognition methods and terminal device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108363599A CN108363599A (en) | 2018-08-03 |
| CN108363599B true CN108363599B (en) | 2019-07-19 |
Family
ID=63011346
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810029553.8A Active CN108363599B (en) | 2018-01-12 | 2018-01-12 | User interface shows recognition methods and terminal device |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN108363599B (en) |
| WO (1) | WO2019136961A1 (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109144868B (en) * | 2018-08-15 | 2022-02-01 | 无线生活（杭州）信息科技有限公司 | Page judgment method and device |
| CN109240923B (en) * | 2018-08-31 | 2021-06-04 | 福建天泉教育科技有限公司 | Interface test script generation method and computer readable storage medium |
| CN109446061B (en) * | 2018-09-17 | 2022-06-10 | 平安科技（深圳）有限公司 | Page detection method, computer readable storage medium and terminal device |
| CN111475396A (en) * | 2019-01-24 | 2020-07-31 | 北京嘀嘀无限科技发展有限公司 | Program detection method, device, electronic equipment and storage medium |
| CN109886431B (en) * | 2019-01-25 | 2021-05-07 | 北京首都国际机场股份有限公司 | Automatic inspection system for airport navigation display terminal |
| CN109857673B (en) * | 2019-02-25 | 2022-02-15 | 北京云测信息技术有限公司 | Control identification method and device |
| CN109976854B (en) * | 2019-03-22 | 2023-02-24 | 维沃移动通信有限公司 | Application processing method and terminal equipment |
| CN110309073B (en) * | 2019-06-28 | 2021-07-27 | 上海交通大学 | Method, system and terminal for automatically detecting user interface errors of mobile application program |
| CN112231034A (en) * | 2019-12-23 | 2021-01-15 | 北京来也网络科技有限公司 | Software interface element identification method and device combining RPA and AI |
| CN111600995A (en) * | 2020-05-15 | 2020-08-28 | 上海莉莉丝科技股份有限公司 | Mobile device and method for correcting deviation of game user interface of mobile device |
| CN111880984A (en) * | 2020-06-09 | 2020-11-03 | 上海容易网电子商务股份有限公司 | Monitoring system for android interactive screen application running state |
| CN112686338A (en) * | 2021-03-10 | 2021-04-20 | 卡斯柯信号(北京)有限公司 | Image information identification method, device, equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103927480A (en) * | 2013-01-14 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Method, device and system for identifying malicious web page |
| CN106874753A (en) * | 2016-12-30 | 2017-06-20 | 中国建设银行股份有限公司 | The method and device at the abnormal interface of identification |
| CN107025174A (en) * | 2017-04-06 | 2017-08-08 | 网易（杭州）网络有限公司 | For the method for the user interface abnormality test of equipment, device and readable storage media |
| CN107315682A (en) * | 2017-06-22 | 2017-11-03 | 北京凤凰理理它信息技术有限公司 | Test browser-safe method, device, storage medium and electronic equipment |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8861844B2 (en) * | 2010-03-29 | 2014-10-14 | Ebay Inc. | Pre-computing digests for image similarity searching of image-based listings in a network-based publication system |
-
2018
- 2018-01-12 CN CN201810029553.8A patent/CN108363599B/en active Active
- 2018-07-27 WO PCT/CN2018/097518 patent/WO2019136961A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103927480A (en) * | 2013-01-14 | 2014-07-16 | 腾讯科技（深圳）有限公司 | Method, device and system for identifying malicious web page |
| CN106874753A (en) * | 2016-12-30 | 2017-06-20 | 中国建设银行股份有限公司 | The method and device at the abnormal interface of identification |
| CN107025174A (en) * | 2017-04-06 | 2017-08-08 | 网易（杭州）网络有限公司 | For the method for the user interface abnormality test of equipment, device and readable storage media |
| CN107315682A (en) * | 2017-06-22 | 2017-11-03 | 北京凤凰理理它信息技术有限公司 | Test browser-safe method, device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108363599A (en) | 2018-08-03 |
| WO2019136961A1 (en) | 2019-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108363599B (en) | User interface shows recognition methods and terminal device | |
| CN109753991A (en) | Abnormal deviation data examination method and device | |
| CN108540218A (en) | Electric power optical cable safety failure processing method and processing device | |
| US20240110956A1 (en) | Current sensor, current measurement device, system and apparatus, and storage medium | |
| CN109740630A (en) | Method for processing abnormal data and device | |
| CN109408303A (en) | Test data analysis method and Related product | |
| CN105842559B (en) | Test result wiring method, device and test macro and mobile terminal | |
| CN106201856A (en) | A kind of multi version performance test methods and device | |
| CN108955901A (en) | A kind of infrared measurement of temperature method, system and terminal device | |
| CN108845747A (en) | A kind of Touch-control error prevention method, apparatus and terminal device | |
| CN107643948A (en) | Application program management-control method, device, medium and electronic equipment | |
| CN105190546B (en) | Cost computing device, the method for cost accounting and computer readable recording medium storing program for performing | |
| CN110460305A (en) | Fault detection method, device, readable medium and the electronic equipment of photovoltaic array | |
| CN109800152A (en) | A kind of automated testing method and terminal device | |
| CN108229494A (en) | network training method, processing method, device, storage medium and electronic equipment | |
| CN109443669A (en) | A kind of leakage detection method and device | |
| CN109003686A (en) | Localization method, device, computer equipment and the storage medium of nuclear power station loosening part | |
| CN106781659B (en) | A kind of parking space state a reference value scaling method and device | |
| CN106462266B (en) | Detect the method and detection device of input equipment | |
| CN116188962A (en) | Error identification method and device for electric energy metering device and computer equipment | |
| CN115878491A (en) | Interface abnormity detection method and device, electronic equipment, storage medium and chip | |
| CN105912417A (en) | Detection method of virtual system, related software running method and related device | |
| CN109324145A (en) | On-line monitoring equipment automatic calibration method and terminal device | |
| CN105630140A (en) | Interface rotating system and method | |
| CN109696614A (en) | Circuit test optimization method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1257521 Country of ref document: HK |