CN110245681A

CN110245681A - Model generating method, application interface method for detecting abnormality, device, terminal device and computer readable storage medium

Info

Publication number: CN110245681A
Application number: CN201910390787.XA
Authority: CN
Inventors: 李洋
Original assignee: Beijing QIYI Century Science and Technology Co Ltd
Current assignee: Beijing QIYI Century Science and Technology Co Ltd
Priority date: 2019-05-10
Filing date: 2019-05-10
Publication date: 2019-09-17

Abstract

The embodiment of the present invention provides a kind of model generating method, application interface method for detecting abnormality, device, terminal device and computer readable storage medium, is related to data processing field.The application interface method for detecting abnormality includes: by screenshot input area detection model in interface to be detected, so that the region detection model carries out region detection to the interface screenshot, the corresponding target category in target area and the target area that the interface screenshot is included is obtained；Wherein, the region detection model is to be generated using the model generating method；The target area includes at least one of following regions: showing incomplete region, the region with overlay text, the region with messy code character, pop-up region, message notifying frame region；According to the corresponding target category in the target area and the target area, the corresponding abnormal results of the interface screenshot are obtained.The present invention improves the accuracy and efficiency of interface display abnormality detection.

Description

Model generating method, application interface method for detecting abnormality, device, terminal device and meter Calculation machine readable storage medium storing program for executing

Technical field

The present invention relates to data processing fields, more particularly to a kind of model generating method, application interface abnormality detection side Method, device, terminal device and computer readable storage medium.

Background technique

Application interface is the medium interacted between system and user with information exchange, it realizes the inside shape of information Formula and user can receive the conversion between form.Application interface allows users to easily and effectively go operation hardware to reach Two-way interactive.

Application interface show it is whether correct, can largely influence user usage experience.In the prior art, pass through Human eye, document object model tree etc. detect the display of application interface.

But application interface is detected by human eye in the prior art, due to human eye fatigue etc., easily there is mistake, Application interface is detected using document object model tree etc., is typically only capable in detection application interface with the presence or absence of certain element, Whether normally show, then can not detect as the element, needs whether further detection elements normally show, detection efficiency It is low.

Summary of the invention

In view of the above problems, it proposes the embodiment of the present invention and overcomes the above problem or at least partly in order to provide one kind Application interface method for detecting abnormality, device, terminal device and the computer readable storage medium to solve the above problems.

According to the first aspect of the invention, a kind of model generating method is provided, which comprises

Obtain the screenshot sample data of application interface；The screenshot sample data include in following sample datas at least It is a kind of: to show incomplete interface screenshot, the interface screenshot with overlay text, the interface screenshot with messy code character, there is bullet The interface screenshot of window, the interface screenshot with message notifying frame；

The screenshot sample data is trained using multilayer convolutional neural networks, formation zone detection model.

Optionally, the multilayer convolutional neural networks include: that you only need to see a convolutional neural networks third edition YOLOv3。

Optionally, the method for obtaining the interface screenshot with overlay text includes: the word segment in normal interface screenshot Text is added, the interface screenshot with overlay text is obtained；

And/or

It includes: the setting messy code character in normal interface screenshot that obtaining, which has the method for the interface screenshot of messy code character, is obtained To the interface screenshot with messy code character.

According to the second aspect of the invention, a kind of application interface method for detecting abnormality is provided, which comprises

By screenshot input area detection model in interface to be detected, so that the region detection model cuts the interface Figure carries out region detection, obtains the corresponding target class in target area and the target area that the interface screenshot is included Not；Wherein, the region detection model is to be generated using described in any item model generating methods；The target area includes At least one of following regions: incomplete region, the region with overlay text, the region with messy code character, bullet are shown Window region, message notifying frame region；

According to the corresponding target category in the target area and the target area, it is corresponding to obtain the interface screenshot Abnormal results.

It optionally, include: pop-up region and/or message notifying frame region in the target area that the interface screenshot is included In the case where, it is described according to the corresponding target category in the target area and the target area, obtain the interface screenshot Corresponding abnormal results include:

Identification obtains the text information of the target area；

The text information is matched with default unusual character, if there are the default exceptions in the text information The target area is then determined as abnormal area, and the target category is determined as the target area and is corresponded to by character Abnormal class.

Optionally, the method also includes:

Export the corresponding abnormal results of the interface screenshot.

Optionally, the method also includes:

The corresponding abnormal results of the interface screenshot are stored in network data base.

Optionally, the method also includes:

Receive the abnormal results inquiry request for the network data base that terminal is sent；

In the network data base, the corresponding abnormal results of the abnormal results inquiry request are obtained, and to the end End returns to the abnormal results.

According to the third aspect of the invention we, a kind of model generating means are provided, described device includes:

Sample data obtains module, for obtaining the screenshot sample data of application interface；The screenshot sample data includes At least one of following sample data: it shows incomplete interface screenshot, the interface screenshot with overlay text, there is messy code The interface screenshot of character, the interface screenshot with pop-up, the interface screenshot with message notifying frame；

Training module, for being trained using multilayer convolutional neural networks to the screenshot sample data, formation zone Detection model.

Optionally, the sample data obtains module and is specifically used for: text is added in the word segment of normal interface screenshot, Obtain the interface screenshot with overlay text；

And/or

Messy code character is set in normal interface screenshot, obtains the interface screenshot with messy code character.

According to the fourth aspect of the invention, a kind of application interface abnormal detector is provided, described device includes:

Region detection module is used for screenshot input area detection model in interface to be detected, so that the region is examined It surveys model and region detection is carried out to the interface screenshot, obtain the target area and the target that the interface screenshot is included The corresponding target category in region；Wherein, the region detection model is to be generated using described in any item model generating methods； The target area includes at least one of following regions: showing incomplete region, the region with overlay text, has disorderly The region of code character, pop-up region, message notifying frame region；

Abnormal results obtain module, are used for according to the corresponding target category in the target area and the target area, Obtain the corresponding abnormal results of the interface screenshot.It optionally, include: pop-up in the target area that the interface screenshot is included In the case where region and/or message notifying frame region, the abnormal results obtain module and include:

Recognition unit obtains the text information of the target area for identification；

Abnormal results acquiring unit, for matching the text information with default unusual character, if the text There are the default unusual characters in information, then the target area are determined as abnormal area, and by the target category It is determined as the corresponding abnormal class in the target area.

Optionally, described device further include:

Abnormal results output module, for exporting the corresponding abnormal results of the interface screenshot.

Optionally, described device further include:

Memory module, for the corresponding abnormal results of the interface screenshot to be stored in network data base.

Optionally, described device further include:

Inquiry request receiving module, the abnormal results inquiry for the network data base for receiving terminal transmission are asked It asks；

Abnormal results return module, it is corresponding in the network data base, obtaining the abnormal results inquiry request Abnormal results, and return to the abnormal results to the terminal.

According to the fifth aspect of the invention, it provides a kind of terminal device, including processor, memory and is stored in described It is real when the computer program is executed by the processor on memory and the computer program that can run on the processor Any model generating method or described in any item application interface method for detecting abnormality now as above.

According to the sixth aspect of the invention, a kind of computer readable storage medium, the computer-readable storage are provided Computer program is stored on medium, and as above any model generation side is realized when the computer program is executed by processor Method or described in any item application interface method for detecting abnormality.

The embodiment of the present invention includes following advantages:

The embodiment of the present invention, in embodiments of the present invention, first with multilayer convolutional neural networks to screenshot sample data Be trained to obtain region detection model, by above-mentioned zone detection model, detect target area that interface screenshot is included and The corresponding target category in target area obtains interface screenshot pair further according to the corresponding target category in target area and target area On the one hand the abnormal results answered realize the automatic detection to interface display exception, avoid detection caused by human eye fatigue etc. The problem of mistake can promote the accuracy of interface display abnormality detection to a certain extent；On the other hand, incomplete according to display Target area, the target area with overlay text, the target area with messy code character, pop-up target area, message mentions Show the corresponding target category in frame target area and each target area, further obtains the corresponding abnormal results of interface screenshot, The detection whether normally shown through covering each element avoids asking of further detecting whether each element normally show Topic, improves the efficiency of interface display abnormality detection to a certain extent.

The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention, And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can It is clearer and more comprehensible, the followings are specific embodiments of the present invention.

Detailed description of the invention

By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:

Fig. 1 is a kind of step flow chart of model generating method provided in an embodiment of the present invention；

Fig. 2 is the structural schematic diagram of YOLOv3 provided in an embodiment of the present invention a kind of；

Fig. 3 is a kind of step flow chart of application interface method for detecting abnormality provided in an embodiment of the present invention；

Fig. 4 is a kind of schematic diagram of the interface screenshot of application provided in an embodiment of the present invention；

Fig. 5 is the corresponding target in the target area that interface screenshot provided in an embodiment of the present invention is included and target area The schematic diagram of classification；

Fig. 6 is the step flow chart of another application interface method for detecting abnormality provided in an embodiment of the present invention；

Fig. 7 is a kind of block diagram of model generating means provided in an embodiment of the present invention；

Fig. 8 is a kind of block diagram of application interface abnormal detector provided in an embodiment of the present invention；

Fig. 9 is the block diagram of another application interface abnormal detector provided in an embodiment of the present invention.

Specific embodiment

In order to make the foregoing objectives, features and advantages of the present invention clearer and more comprehensible, with reference to the accompanying drawing and specific real Applying mode, the present invention is described in further detail.

It should be appreciated that described herein, specific examples are only used to explain the present invention, and only present invention a part is real Example is applied, instead of all the embodiments, is not intended to limit the present invention.

It referring to Fig.1, is a kind of step flow chart of model generating method provided in an embodiment of the present invention, this method can answer For server or terminal, in embodiments of the present invention, this is not especially limited.

Step 101: obtaining the screenshot sample data of application interface；The screenshot sample data includes following sample data At least one of: show that incomplete interface screenshot, the interface screenshot with overlay text, the interface with messy code character are cut Figure, the interface screenshot with pop-up, the interface screenshot with message notifying frame.

In the embodiment of the present invention, above-mentioned application can be any application installed in terminal.The screenshot sample of application interface Data can be applied in operational process for this, it is understood that there may be show the corresponding screenshot image in abnormal interface or video image etc.. The screenshot sample data of the application interface may include: text and/or image etc..In the present invention is implemented, this is not made specific It limits.

In the embodiment of the present invention, show that incomplete interface screenshot can be with are as follows: due to no network or network difference etc., cause Without completely showing or the interface screenshot that does not show completely.Interface screenshot with overlay text can be with are as follows: word segment has Overlapping, so that the ambiguous interface screenshot of word segment.Interface screenshot with messy code character can be with are as follows: existing can not understand Messy code character interface screenshot.

In embodiments of the present invention, pop-up can be for other than informing user information, it is also necessary to which user replys, if user It does not reply, then can not carry out the interacting message window of other operations.Interface screenshot with pop-up can be with are as follows: includes pop-up Interface screenshot.Message notifying frame can be to inform user information, not need user's reply, and can be after the default display time certainly The dynamic message to disappear informs frame.Interface screenshot with message notifying frame can be with are as follows: includes the interface screenshot of message notifying frame.

In embodiments of the present invention, screenshot sample data may include at least one of above 5 kinds of sample datas.To cut Pattern notebook data specifically includes sample data and quantity of Different categories of samples data of which type etc. and is not especially limited.Such as, Screenshot sample data can be with are as follows: shows incomplete interface screenshot, the interface screenshot with overlay text, the boundary with messy code character Face screenshot.

It in embodiments of the present invention, can be in the case where eye recognition be to interface display exception, using automatic screenshot work Tool carries out automatic screenshot to the abnormal interface of display, to obtain the screenshot sample data of application interface.Alternatively, in eye recognition In the case where interface display exception, by screenshot key etc., screenshot manually is carried out to the abnormal interface of display, to obtain application The screenshot sample data at interface.In embodiments of the present invention, this is not especially limited.

In embodiments of the present invention, the problem inadequate for the screenshot sample data quantity for coping with interface, alternatively, in order to mention Rise region detection model training effectiveness etc., optionally, obtain have overlay text interface screenshot method may include: The word segment of normal interface screenshot adds text, obtains the interface screenshot with overlay text；And/or it obtains with messy code The method of the interface screenshot of character may include: the setting messy code character in normal interface screenshot, obtain having messy code character Interface screenshot.

Specifically, normal interface screenshot can be with are as follows: each element show it is complete and correct, there is no messy code character, no There are interface screenshots of overlay text etc..Text can be added, and then had in the word segment of above-mentioned normal interface screenshot There is the interface screenshot sample data of overlay text.Alternatively, the messy code word that can not be understood can also be arranged in normal interface screenshot Symbol, obtains having the interface screenshot sample data of messy code character to help to subtract with the quantity of the screenshot sample data of extended interface The time of the screenshot sample data of application interface is obtained, less with the efficiency of lift scheme training.

Step 102: the screenshot sample data being trained using multilayer convolutional neural networks, mould is detected in formation zone Type.

Specifically, first can the screenshot sample data to above-mentioned application interface manually marked, obtain each screenshot The corresponding label information of sample data.

Above-mentioned label information can be with are as follows: the target area and each target area in each screenshot sample data are corresponding Target category, optionally, above-mentioned label information can also include: seat of the above-mentioned each target area in screenshot sample data Cursor position etc..

Above-mentioned target area may include at least one of following regions: showing incomplete region, has overlay text Region, the region with messy code character, pop-up region, message notifying frame region.

Show that the incomplete corresponding target category in region can be with are as follows: without network or network difference classification.With overlay text The corresponding target category in region can be with are as follows: text overlays classification.The corresponding target category in region with messy code character can be with are as follows: Messy code character class.The corresponding target category in pop-up region can be with are as follows: pop-up classification.The corresponding target class of message notifying frame region It can be with are as follows: message notifying frame classification etc..

In embodiments of the present invention, optionally, above-mentioned label information can be stored as preset format, the preset format is fixed The justice corresponding target category in target area, coordinate position of the target area etc., meanwhile, which also defines target Byte number that the byte number and position in label information and coordinate position that classification occupies occupy and in label information Position, and then corresponding target category in target area etc. can rapidly be got by above-mentioned preset format, and then after convenience The comparison of continuous label information and convolutional neural networks output feature.

For example, the preset format can be text formatting, text format can be with are as follows: the corresponding target category in target area The coordinate position etc. of+target area.The corresponding target category in target area occupies preceding 8 bytes of the label information, target Rear 8 bytes of the occupancy such as the coordinate position in the region label information.The coordinate position of each target area may include: each The center point coordinate of target area, the height dimension of each target area, width dimensions of each target area etc..

In embodiments of the present invention, above-mentioned multilayer convolutional neural networks can be with are as follows: by multiple convolutional neural networks according to pre- If the convolutional neural networks of sequence setting.Above-mentioned each convolutional neural networks extract the convolution feature of screenshot sample data respectively, According to above-mentioned preset order, the convolution feature of extraction is successively transferred to subsequent convolutional neural networks, and then obtain above-mentioned section Corresponding target category in target area and target area that pattern notebook data is included etc..

In embodiments of the present invention, above-mentioned multilayer convolutional neural networks can carry out: convolution, batch regularization, non-linear The operation such as activation, operation diversification, can be avoided convolution Character losing, can promote the region detection of generation to a certain extent The accuracy of model.The loss function of above-mentioned multilayer convolutional neural networks can be by: error of coordinate, Duplication error, classification miss The composition such as difference, loss function Consideration is comprehensive, can promote the accuracy of the region detection model of generation to a certain extent. Each convolutional layer usually extracts specific feature respectively, above-mentioned multilayer convolutional neural networks since the convolution number of plies for including is more, And then the special characteristic extracted is more fully, so that the region detection model generated is more accurate to the detection in region.

In embodiments of the present invention, corresponding training environment can be configured to above-mentioned multilayer convolutional neural networks.It is above-mentioned more Layer convolutional neural networks usually have pre-training weight, after the completion of training environment configuration, by above-mentioned pre-training weight, utilize Multilayer convolutional neural networks carry out feature extraction to above-mentioned screenshot sample data.By the feature of output and above-mentioned screenshot sample data Corresponding label information is compared, and according to comparison result, above-mentioned pre-training weight is constantly adjusted, until the feature of above-mentioned output The similarity degree of label information corresponding with above-mentioned screenshot sample data reaches default similarity, then, in such cases, it is believed that raw The corresponding multilayer convolutional neural networks of pre-training weight in such cases are determined as region inspection at region detection model Survey model.

In embodiments of the present invention, configuring corresponding training environment to above-mentioned multilayer convolutional neural networks mainly can be with are as follows: Configuration classification number, configuration process number etc., in embodiments of the present invention, are not especially limited this.For example, the embodiment of the present invention In, above-mentioned zone detection model, which needs to identify or detect 5 different area classifications, can then configure above-mentioned classification number to 5.2 can be configured by process number, it, can be with the formation efficiency of lifting region model for one process.

In embodiments of the present invention, optionally, after the detection model of formation zone, on the more practical boundary of successive accumulated , can also be using above-mentioned practical interface screenshot as above-mentioned screenshot sample data in the case where the screenshot of face, further training or optimization The region detection model of above-mentioned generation, and then the further accuracy of lifting region detection model.In embodiments of the present invention, right This is not especially limited.

For example, above-mentioned multilayer convolutional neural networks may include: YOLOv3 (You only look once Version 3, you only need to see a convolutional neural networks third edition).Referring to shown in Fig. 2, Fig. 2 is that the embodiment of the present invention provides A kind of YOLOv3 structural schematic diagram.1,2,4, the 8 of the leftmost side can select the repetition of the network unit of part in Fig. 2 for frame Number.Convolutional layer is used to extract the convolution feature of screenshot sample data in the multilayer convolutional neural networks, and residual error layer is used for extraction Above-mentioned convolution feature and the screenshot sample data actual characteristic, be compared, calculate the above-mentioned convolution feature that extracts with Difference between the actual characteristic of the screenshot sample data.Global mean value pond layer is used for whole convolution feature calculations to extraction Mean value.The scale 3 that length and width dimensions are 32 × 32 is exported after 26th convolutional layer predicts convolution feature, the 43rd convolutional layer output length The wide scale 2 having a size of 16 × 16 predicts convolution feature, and the scale 1 that the 52nd convolutional layer output length and width dimensions are 8 × 8 predicts volume Product feature, full articulamentum are used to the corresponding convolution feature of above-mentioned mean value, scale 1 predicting that convolution feature, scale 2 predict that convolution is special Sign, scale 3 predict that convolution feature carries out convolution algorithm, form feature consistent and complete with screenshot sample data structure and combine, To obtain initial territorial classification.Normalization layer is used to that above-mentioned complete feature combination to be normalized, and obtains above-mentioned Region detection model.

In Fig. 2, corresponding number in convolution kernel quantity column characterizes the quantity of convolution kernel in the layer network, convolution kernel The corresponding number of size column, characterizes the length and width dimensions of convolution sum in the layer network, and output arranges corresponding number, characterizes the layer The length and width dimensions of the convolution feature of network output.For example, " convolutional layer 32 3 × 3 256 × 256 " are directed to, the meaning of expression are as follows: The convolutional layer is respectively 3 × 3 convolution kernel by 32 length and width dimensions, extracts convolution feature, the corresponding square of convolution feature of extraction The length and width dimensions of battle array are 256 × 256.

The YOLOv3 includes 52 convolutional layers, and each convolutional layer usually extracts specific special respectively in 52 convolutional layers Sign, YOLOv3 is since the convolution number of plies for including is more, and then the special characteristic extracted is more fully, so that the region detection generated Model is more accurate to the detection in region.

In embodiments of the present invention, using multilayer convolutional neural networks to the incomplete interface screenshot of display, with overlapping text The interface screenshot of word, the interface screenshot with messy code character, the interface screenshot with pop-up, the interface with message notifying frame are cut Figure is trained to obtain region detection model, and above-mentioned sample data is with strong points, and, each convolutional layer usually extracts specific respectively Feature, multilayer convolutional neural networks since the convolution number of plies for including is more, and then extract special characteristic more fully so that The region detection model of generation is more accurate to the detection in region.

It is a kind of step flow chart of application interface method for detecting abnormality provided in an embodiment of the present invention, the party referring to Fig. 3 Method can be applied in any terminal or server, in the present invention is implemented, be not especially limited to this.This method can wrap It includes:

Step 201: by screenshot input area detection model in interface to be detected, so that the region detection model is to institute It states interface screenshot and carries out region detection, obtain target area that the interface screenshot is included and the target area is corresponding Target category；Wherein, the region detection model is to be generated using described in any item model generating methods；The target area Domain includes at least one of following regions: showing incomplete region, the region with overlay text, the area with messy code character Domain, pop-up region, message notifying frame region.

In the embodiment of the present invention, interface screenshot to be detected can be for each application in the process of running, arbitrary interface pair Interface screenshot answered etc..The interface image may include applying in operational process for this, the video image etc. at interface.It is above-mentioned to answer With can be any application installed in any terminal.In the present invention is implemented, this is not especially limited.

Referring to shown in Fig. 4, Fig. 4 is a kind of schematic diagram of the interface screenshot of application provided in an embodiment of the present invention.The present invention In embodiment, which can be black white image or color image, which can also be a certain frame in video Or multiple image etc..In embodiments of the present invention, this is not especially limited.

In the embodiment of the present invention, the region detection model is is generated according to model generating method above-mentioned.By it is above-mentioned to The interface screenshot of detection inputs the region detection model, which carries out convolution feature extraction to above-mentioned interface screenshot Deng operation, to carry out region detection to the interface screenshot, target area and target area that the interface screenshot is included are exported Corresponding target type.In the embodiment of the present invention, the target area that above-mentioned interface screenshot is included may include in following regions At least one: show incomplete target area, the region with overlay text, the region with messy code character, pop-up region, At least one of message notifying frame region, normal region etc..

In the embodiment of the present invention, content is not shown completely, or the target area absolutely not shown can be residual to show Scarce region shows that the incomplete corresponding target category in region can be no network or vulnerable network classification.There are overlay texts Region can be the region with overlay text, and the corresponding target category in region with overlay text can be text overlays class Not.It can be the region with messy code character, the corresponding target class in region with messy code character there are the region of messy code character It can not be messy code character class.It can be pop-up region there are the region of pop-up, the corresponding target category in pop-up region can be with For pop-up classification.It can be message notifying frame region, the corresponding target of message notifying frame region there are the region of message notifying frame Classification can be message notifying frame classification.Region in the screenshot of interface in addition to above-mentioned five class region can be normal region, just The corresponding classification in normal region can be normal category.Alternatively, each element shows that complete, normal region can be normal area Domain, the corresponding classification in the normal region can be normal category.

For example, referring to shown in Fig. 5, Fig. 5 be the target area that interface screenshot provided in an embodiment of the present invention is included and The schematic diagram of the corresponding target category in each target area.Interface screenshot shown in Fig. 4 is inputted into above-mentioned zone detection model, to this Interface screenshot carries out region detection, and obtaining target area that the interface screenshot is included can be 5, respectively can be with are as follows: region 1 To region 3, region 5 to region 6, normal region is region 4.Region 1 illustrates only a part of content, and there are also one point of partial contents It does not show, region that can be incomplete for display, corresponding target category can be with are as follows: without network or vulnerable network classification, in region 2 It can be the region with messy code character with messy code character, corresponding target category can be with are as follows: messy code character class.Region 3 Text with overlapping can be the region with overlay text, and corresponding target category can be with are as follows: text overlays classification.Area Each element is shown normal, complete in domain 4, can be normal region, and corresponding classification can be with are as follows: normal category.Have in region 5 There is pop-up, can be pop-up region, corresponding target category can be with are as follows: pop-up classification.There is message notifying frame in region 6, it is right The target category answered can be with are as follows: message notifying frame classification.

It, optionally, can be with before by screenshot input area detection model in interface to be detected in the embodiment of the present invention It include: the interface screenshot for obtaining application.Specifically, can be intercepted manually by user, alternatively, can be according between preset time Every the interface screenshot of automatic interception application.In embodiments of the present invention, this is not especially limited.

In embodiments of the present invention, if the above method is applied in server, optionally, above-mentioned interface screenshot can be needle To the interface screenshot applied in terminal, terminal can be communicated with server, and above-mentioned interface screenshot can be sent to by terminal Above-mentioned server.The terminal may include mobile terminal etc..After server receives the interface screenshot of terminal transmission, it can will connect The interface screenshot received carries out region detection as screenshot input area detection model in interface to be detected.Implement in the present invention In example, this is not especially limited.

Step 202: according to the corresponding target category in the target area and the target area, obtaining the interface and cut Scheme corresponding abnormal results.

In the embodiment of the present invention, can according to the corresponding target category in above-mentioned target area and above-mentioned target area, into One step determines the corresponding abnormal results of above-mentioned interface screenshot, the abnormal results may include: above-mentioned interface screenshot included it is different Normal region and the corresponding abnormal class of each abnormal area.In embodiments of the present invention, this is not especially limited.

Specifically, if the corresponding target category in target area are as follows: without network vulnerable network classification, text overlays classification, messy code At least one of character class etc., or, if target area are as follows: show incomplete region, the region with overlay text, have At least one of the region of messy code character, it will usually influence the application of user's normal use, therefore, screenshot is included at interface Target area include: the incomplete region of display, the region with overlay text, at least one in the region with messy code character Kind, or, the corresponding target category in target area that each interface screenshot is included includes: no network or vulnerable network classification, text Overlapping classes in the case where at least one of messy code character class etc., target area above-mentioned in the screenshot of interface can be determined Above-mentioned target category is determined as the corresponding abnormal class of abnormal area for abnormal area.

It should be noted that in embodiments of the present invention, for target area are as follows: pop-up region and/or message notifying frame Region or target category are as follows: pop-up classification and/or message notifying frame classification, directly from Show Styles, possibly can not directly really It is fixed whether abnormal, the text information according to corresponding to the region is usually also needed, further determines whether be abnormal etc..

For example, the corresponding target class in target area and each target area for being included for interface screenshot shown in Fig. 4 Not, from Show Styles, it can show that region 1 is the incomplete target area of display, corresponding target category are as follows: without network or Vulnerable network classification, region 2 are the target area with messy code character, corresponding target category: text messy code classification, region 3 are as follows: Target area with overlay text, corresponding target category: text overlays classification, it will usually which influencing user's normal use, this is answered With then region 1 is abnormal area, and corresponding abnormal class can be with are as follows: without network or vulnerable network classification, region 2 is abnormal area Corresponding abnormal class can be with are as follows: messy code character class, region 3 can be abnormal area, and corresponding abnormal class can be with are as follows: text Word overlapping classes.By further judging, text information corresponding to region 5, region 6, the corresponding abnormal class of above-mentioned zone 5 It can be with are as follows: pop-up classification.The corresponding abnormal class in region 6 can be with are as follows: message notifying frame classification.

It, optionally, can be by the corresponding boundary in each interface of application if application includes multiple interfaces in the embodiment of the present invention Face screenshot, carries out abnormality detection, and finally obtains the corresponding abnormal results of the corresponding interface screenshot in each interface, and then can be with Each interface abnormal area that may be present and abnormal class are fully understanded, to facilitate developer more comprehensively to correct or tie up Protect above-mentioned application.For example, if application includes 20 interfaces, the corresponding interface screenshot in 20 interfaces that can include by the application It carries out abnormality detection, obtains 20 interfaces that the application includes, the corresponding exception of the corresponding interface screenshot in each interface As a result.

In conclusion in embodiments of the present invention, being carried out first with multilayer convolutional neural networks to screenshot sample data Training obtains region detection model, by above-mentioned zone detection model, detects the target area and target that interface screenshot is included It is corresponding to obtain interface screenshot further according to the corresponding target category in target area and target area for the corresponding target category in region On the one hand abnormal results realize the automatic detection to interface display exception, avoid and detect mistake caused by human eye fatigue etc. The problem of, the accuracy of interface display abnormality detection can be promoted to a certain extent；On the other hand, the mesh incomplete according to display Mark region, the target area with overlay text, the target area with messy code character, pop-up target area, message notifying frame The corresponding target category in target area and each target area further obtains the corresponding abnormal results of interface screenshot, has contained The detection whether each element normally shows has been covered, has avoided and the problem of whether each element normally shows further is detected, from The efficiency of interface display abnormality detection is improved to a certain extent.

Fig. 6 is the step flow chart of another application interface method for detecting abnormality provided in an embodiment of the present invention, this method May include:

Step 301: by screenshot input area detection model in interface to be detected, so that the region detection model is to institute It states interface screenshot and carries out region detection, obtain target area that the interface screenshot is included and the target area is corresponding Target category；Wherein, the region detection model is to be generated using described in any item model generating methods；The target area Domain includes at least one of following regions: showing incomplete region, the region with overlay text, the area with messy code character Domain, pop-up region, message notifying frame region.

In embodiments of the present invention, step 301 is referred to the related record of above-mentioned steps 201, in order to avoid repeating, this Place repeats no more.

Step 302: including: pop-up region and/or message notifying frame area in the target area that the interface screenshot is included In the case where domain, identification obtains the text information of the target area.

In embodiments of the present invention, for target area are as follows: pop-up regional aim classification is and/or message notifying frame region In the case where, alternatively, target category are as follows: in the case where pop-up classification and/or message notifying frame classification, directly from Show Styles In, can not also determine whether exception, usually also need the text information according to corresponding to the target area, further determine whether It is abnormal etc..

Specifically, can identify to obtain the text information of above-mentioned target area.Identification obtains text in above-mentioned target area Or all text informations in image.Such as, OCR (Optical CharacterRecognition, optical character knowledge can be passed through Not) Text region carries out Text region, obtains the text of above-mentioned target area to the above-mentioned target area in above-mentioned interface screenshot This information.

In the embodiment of the present invention, optionally, the target area that interface screenshot is included can be by above-mentioned in the screenshot of interface Coordinate information, dimension information of target area etc. are embodied.It can coordinate information, dimension information based on above-mentioned target area Deng using OPENCV (Open Source ComputerVision Library, computer vision library of increasing income) at above-mentioned interface Cut pop-up classification and/or the corresponding target area of message notifying frame classification in screenshot, then by Text region, to above-mentioned boundary Above-mentioned target area in the screenshot of face carries out Text region, obtains the text information of above-mentioned target area.In the embodiment of the present invention In, this is not especially limited.Coordinate information, dimension information based on above-mentioned target area etc., using OPENCV on above-mentioned boundary Cut pop-up classification and/or the corresponding target area of message notifying frame classification in the screenshot of face, it can be more accurate, easy from upper It states and cuts pop-up classification and/or the corresponding target area of message notifying frame classification in the screenshot of interface, be conducive to promote subsequent exception The accuracy of testing result.

For example, being directed to above-mentioned example, referring to Fig. 5, region 5 is pop-up region, corresponding target category are as follows: pop-up classification. Region 6 is message notifying frame region, corresponding target category are as follows: message notifying frame classification.Directly from Show Styles, usually also It can not determine whether exception, the text information that identification obtains region 5 can be with are as follows: " connection IQIYI-VPN Https: // Vpn.iqiyi.com connection connection error server is turned off connection and determines ".The text information that identification obtains region 6 can be with are as follows: " scenic spots and historical sites function is offline ".

Step 303: the text information being matched with default unusual character, if existing in the text information described Default unusual character, then be determined as abnormal area for the target area, and the target category is determined as the target The corresponding abnormal class in region.

In embodiments of the present invention, default unusual character can be set in advance, and above-mentioned default unusual character can be can Characterize the character of above-mentioned pop-up or message notifying frame display exception.Such as, refer to should not be above-mentioned for above-mentioned default unusual character The character etc. shown in pop-up or message notifying frame.The default unusual character can wait according to specific needs, be configured.At this In inventive embodiments, this is not especially limited.For example, above-mentioned default unusual character may include: failure, mistake, server It is turned off, is offline, error etc..

In embodiments of the present invention, above-mentioned text information can be matched with above-mentioned default unusual character, if above-mentioned In text information exist with the above-mentioned default matched unusual character of unusual character, then above-mentioned target area is determined as above-mentioned exception The corresponding target category in above-mentioned target area is determined as the corresponding abnormal class of above-mentioned abnormal area by region.

For example, be directed to above-mentioned example, if default unusual character include: failure, mistake, server be turned off, be offline, Error etc..Then exist in the text information in region 5 matched with above-mentioned unusual character: mistake, server are turned off, these are different Normal character.Then region 5 can be determined as abnormal area, if the corresponding target category in region 5 is pop-up classification, region 5 is right The abnormal class answered can be with are as follows: pop-up classification.Exist in the text information in region 6 matched with above-mentioned unusual character: it is offline, this Region 6 then can be determined as abnormal area by the unusual character of sample, if the corresponding target category in region 6 is message notifying frame class Not, then the corresponding abnormal class in region 6 can be with are as follows: message notifying frame classification.

In embodiments of the present invention, if being not present in above-mentioned text information and the matched abnormal word of above-mentioned default unusual character Symbol, then be determined as normal region for above-mentioned target area.In the case where above-mentioned target area is normal region, can export The testing result that target area is normal region is stated, so that the clearly above-mentioned target area of user is not necessarily to additional attention and maintenance, into And it can make user will be in the limited maintenance for focusing on abnormal area.

Step 304: exporting the corresponding abnormal results of the interface screenshot.

In embodiments of the present invention, the corresponding abnormal results of above-mentioned interface screenshot can be exported.Specifically, can show The corresponding abnormal results of interface screenshot are stated, or, corresponding abnormal results of the above-mentioned interface screenshot of voice broadcast etc..Implement in the present invention In example, this is not especially limited.

In embodiments of the present invention, it by exporting the corresponding abnormal results of above-mentioned interface screenshot, and then is obtained convenient for user The corresponding abnormal results of above-mentioned interface screenshot.

In the embodiment of the present invention, optionally, if this method is applied to server, if above-mentioned interface screenshot is to apply in terminal Interface screenshot, above-mentioned interface screenshot is sent to server by terminal, and optionally, this method can also include: that server can be with Abnormal results corresponding to above-mentioned interface screenshot are returned into above-mentioned terminal by corresponding interface or network etc..

For example, if this method is applied to server, server can be by corresponding interface or network etc., will be above-mentioned Abnormal results corresponding to the screenshot of interface, for referring to Fig. 5, the corresponding abnormal class in region 1 are as follows: without network or vulnerable network class Not, the corresponding abnormal class in region 2 are as follows: messy code character class, the corresponding abnormal class in region 3 are as follows: text overlays classification, region 5 corresponding abnormal class are as follows: pop-up classification, the corresponding abnormal class in region 6 are as follows: message notifying frame classification returns to above-mentioned end End.

In embodiments of the present invention, for terminal to server send interface screenshot, server to above-mentioned interface screenshot into Row abnormality detection, and to terminal return abnormality detection result scheme, above-mentioned detection there is no by terminal itself carry out, only by Terminal sends the interface screenshot of application, without the more process of occupied terminal, will not interfered with terminal normal operation.

Step 305: the corresponding abnormal results of the interface screenshot are stored in network data base.

In embodiments of the present invention, network data base can be preset, the network data base can for relevant database or Non-relational database etc..In embodiments of the present invention, this is not especially limited.For example, the network data base may include: MySQL database etc..Above-mentioned abnormal results can be stored in network data base.

In embodiments of the present invention, the corresponding abnormal results of above-mentioned interface screenshot are being stored in the mistake in network data base Journey can store the identification information of the interface screenshot simultaneously.The identification information of the interface screenshot is cut for distinguishing different interfaces Figure.In embodiments of the present invention, the particular content of the identification information of interface screenshot is not especially limited.

Step 306: receiving the abnormal results inquiry request for the network data base that terminal is sent.

Specifically, while forming network data base the available network data base network linking.Will be above-mentioned After abnormal results are stored in above-mentioned network data base, can be returned to terminal the corresponding network linking of the network data base and The corresponding identification information of interface screenshot.The network linking is used for after receiving trigger action, is jumped in the network data base. The identification information in network data base for distinguishing different interface screenshots.Can receive terminal transmission is directed to the network number According to the abnormal results inquiry request in library.The abnormal results request for information may include: the corresponding network linking of the network data base With the identification information of interface screenshot.

Such as, if this method is applied to server, which can store the corresponding abnormal results in interface to network In database, and the identification information of the corresponding network linking of the network data base and interface screenshot is sent to terminal, in terminal In the case where receiving the trigger action for the network linking, can be equivalent to be connected to terminal transmission for network data The abnormal results inquiry request of a part in library.

Step 307: in the network data base, the corresponding abnormal results of the abnormal results inquiry request are obtained, to The terminal returns to the corresponding abnormal results of the abnormal results inquiry request.

In embodiments of the present invention, in the case where terminal receives the trigger action for the network linking, Ke Yixiang When in the abnormal results inquiry request of a part for network data base for being connected to terminal transmission.Then, server jumps to The corresponding network data base of the network linking, then receive the identification information of the interface screenshot of terminal transmission, then, from the network data Abnormal results corresponding with the interface screenshot of above-mentioned identification information match are searched in library, are equivalent to and are obtained from the network data base It has arrived abnormal inquiry and has requested corresponding abnormal results, returned to above-mentioned abnormal results to terminal.

In embodiments of the present invention, above-mentioned abnormal results are stored in network data base, receive what above-mentioned terminal was sent For the abnormal results inquiry request of the network data base, in above-mentioned network data base, abnormal results inquiry request pair is obtained The abnormal results answered, Xiang Shangshu terminal return to above-mentioned abnormal results, can be convenient user at any time, repeatedly inquire above-mentioned exception everywhere As a result.

In the embodiment of the present invention, accurate detection goes out abnormal results corresponding to the interface screenshot applied, and can be convenient out Hair personnel more targetedly correct or safeguard above-mentioned application, can be from the working efficiency for largely promoting developer.

In conclusion in embodiments of the present invention, being carried out first with multilayer convolutional neural networks to screenshot sample data Training obtains region detection model, by above-mentioned zone detection model, detects the target area and target that interface screenshot is included It is corresponding to obtain interface screenshot further according to the corresponding target category in target area and target area for the corresponding target category in region On the one hand abnormal results realize the automatic detection to interface display exception, avoid and detect mistake caused by human eye fatigue etc. The problem of, the accuracy of interface display abnormality detection can be promoted to a certain extent；On the other hand, the mesh incomplete according to display Mark region, the target area with overlay text, the target area with messy code character, pop-up target area, message notifying frame The corresponding target category in target area and each target area further obtains the corresponding abnormal results of interface screenshot, has contained The detection whether each element normally shows has been covered, has avoided and the problem of whether each element normally shows further is detected, from The efficiency of interface display abnormality detection is improved to a certain extent.It should be noted that for embodiment of the method, in order to simply retouch It states, therefore, it is stated as a series of action combinations, but those skilled in the art should understand that, the embodiment of the present application is not It is limited by described sequence of movement, because certain steps can be sequentially or same using other according to the embodiment of the present application Shi Jinhang.Secondly, those skilled in the art should also know that, the embodiments described in the specification are all preferred embodiments, Related movement might not all be necessary to the embodiment of the present application.

Fig. 7 is a kind of model generating means provided in an embodiment of the present invention, and referring to shown in Fig. 7, described device 700 be can wrap It includes:

Sample data obtains module 701, for obtaining the screenshot sample data of application interface；The screenshot sample data packet It includes following at least one of sample datas: showing incomplete interface screenshot, the interface screenshot with overlay text, has disorderly The interface screenshot of code character, the interface screenshot with pop-up, the interface screenshot with message notifying frame；

Training module 702, for being trained using multilayer convolutional neural networks to the screenshot sample data, generation area Domain detection model.

Optionally, the sample data obtains module 701 and is specifically used for: in the word segment addition text of normal interface screenshot Word obtains the interface screenshot with overlay text；

And/or

Fig. 8 is a kind of block diagram of application interface abnormal detector provided in an embodiment of the present invention, as shown in figure 8, the dress Setting 800 may include:

Region detection module 801 is used for by screenshot input area detection model in interface to be detected, so that the region Detection model carries out region detection to the interface screenshot, obtains the target area and the mesh that the interface screenshot is included Mark the corresponding target category in region；Wherein, the region detection model is to be generated using described in any item model generating methods 's；The target area includes at least one of following regions: showing incomplete region, the region with overlay text, tool There are the region, pop-up region, message notifying frame region of messy code character；

Abnormal results obtain module 802, for according to the corresponding target class in the target area and the target area Not, the corresponding abnormal results of the interface screenshot are obtained.

Optionally, it on the basis of above-mentioned Fig. 8, referring to shown in Fig. 9, is wrapped in the target area that the interface screenshot is included Include: in the case where pop-up region and/or message notifying frame region, the abnormal results obtain module 802 and may include:

Recognition unit 8021 obtains the text information of the target area for identification；

Abnormal results acquiring unit 8022, for matching the text information with default unusual character, if described There are the default unusual characters in text information, then the target area are determined as abnormal area, and by the target Classification is determined as the corresponding abnormal class in the target area.

Optionally, described device can also include:

Abnormal results output module 803, for exporting the corresponding abnormal results of the interface screenshot.

Optionally, described device can also include:

Memory module 804, for the corresponding abnormal results of the interface screenshot to be stored in network data base.

Optionally, described device can also include:

Inquiry request receiving module 805, the abnormal results for the network data base for receiving terminal transmission are looked into Ask request；

Abnormal results return module 806, for obtaining the abnormal results inquiry request pair in the network data base The abnormal results answered, and the abnormal results are returned to the terminal.

For device embodiment, since it is basically similar to the method embodiment, related so being described relatively simple Place illustrates referring to the part of embodiment of the method.

All the embodiments in this specification are described in a progressive manner, the highlights of each of the examples are with The difference of other embodiments, the same or similar parts between the embodiments can be referred to each other.

It should be understood by those skilled in the art that, the embodiment of the embodiment of the present invention can provide as method, apparatus or calculate Machine program product.Therefore, the embodiment of the present invention can be used complete hardware embodiment, complete software embodiment or combine software and The form of the embodiment of hardware aspect.Moreover, the embodiment of the present invention can be used one or more wherein include computer can With in the computer-usable storage medium (including but not limited to magnetic disk storage, CD-ROM, optical memory etc.) of program code The form of the computer program product of implementation.

In a typical configuration, the computer equipment includes one or more processors (CPU), input/output Interface, network interface and memory.Memory may include the non-volatile memory in computer-readable medium, random access memory The forms such as device (RAM) and/or Nonvolatile memory, such as read-only memory (ROM) or flash memory (flash RAM).Memory is to calculate The example of machine readable medium.Computer-readable medium includes that permanent and non-permanent, removable and non-removable media can be with Realize that information is stored by any method or technique.Information can be computer readable instructions, data structure, the module of program or Other data.The example of the storage medium of computer includes, but are not limited to phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other kinds of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory techniques, CD-ROM are read-only Memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, tape magnetic disk storage or Other magnetic storage devices or any other non-transmission medium, can be used for storage can be accessed by a computing device information.According to Herein defines, and computer-readable medium does not include non-persistent computer readable media (transitory media), such as The data-signal and carrier wave of modulation.

The embodiment of the present invention be referring to according to the method for the embodiment of the present invention, terminal device (system) and computer program The flowchart and/or the block diagram of product describes.It should be understood that flowchart and/or the block diagram can be realized by computer program instructions In each flow and/or block and flowchart and/or the block diagram in process and/or box combination.It can provide these Computer program instructions are examined extremely to general purpose computer, special purpose computer, Embedded Processor or other programmable applications interfaces The processor of terminal device is surveyed to generate a machine, so that whole by computer or other programmable applications interface abnormality detections The instruction that the processor of end equipment executes generates for realizing in one or more flows of the flowchart and/or one, block diagram The device for the function of being specified in box or multiple boxes.

These computer program instructions, which may also be stored in, is able to guide computer or other programmable applications interface abnormality detections In terminal device computer-readable memory operate in a specific manner, so that finger stored in the computer readable memory It enables and generates the manufacture including command device, which realizes in one or more flows of the flowchart and/or box The function of being specified in figure one box or multiple boxes.

These computer program instructions can also be loaded into computer or other programmable applications interface abnormality detection terminals are set It is standby upper, so that series of operation steps are executed on computer or other programmable terminal equipments to generate computer implemented place Reason, thus the instruction that is executed on computer or other programmable terminal equipments provide for realizing in one process of flow chart or The step of function of being specified in multiple processes and/or one or more blocks of the block diagram.

Although the preferred embodiment of the embodiment of the present invention has been described, once a person skilled in the art knows bases This creative concept, then additional changes and modifications can be made to these embodiments.So the following claims are intended to be interpreted as Including preferred embodiment and fall into all change and modification of range of embodiment of the invention.

Finally, it is to be noted that, herein, relational terms such as first and second and the like be used merely to by One entity or operation are distinguished with another entity or operation, without necessarily requiring or implying these entities or operation Between there are any actual relationship or orders.Moreover, the terms "include", "comprise" or its any other variant meaning Covering non-exclusive inclusion, so that process, method, article or terminal device including a series of elements not only wrap Those elements are included, but also including other elements that are not explicitly listed, or further includes for this process, method, article Or the element that terminal device is intrinsic.In the absence of more restrictions, being wanted by what sentence "including a ..." limited Element, it is not excluded that there is also other identical elements in process, method, article or the terminal device for including the element.

It to a kind of application interface method for detecting abnormality provided by the present invention, device, equipment and computer-readable deposits above Storage media is described in detail, and used herein a specific example illustrates the principle and implementation of the invention, The above description of the embodiment is only used to help understand the method for the present invention and its core ideas；Meanwhile for the one of this field As technical staff, according to the thought of the present invention, there will be changes in the specific implementation manner and application range, to sum up institute It states, the contents of this specification are not to be construed as limiting the invention.

Claims

1. a kind of model generating method, which is characterized in that the described method includes:

Obtain the screenshot sample data of application interface；The screenshot sample data includes at least one in following sample datas Kind: it shows incomplete interface screenshot, the interface screenshot with overlay text, the interface screenshot with messy code character, there is pop-up Interface screenshot, the interface screenshot with message notifying frame；

2. the method according to claim 1, wherein the multilayer convolutional neural networks include: that you only need to see Convolutional neural networks third edition YOLOv3.

3. method according to claim 1 or 2, which is characterized in that

The method for obtaining the interface screenshot with overlay text includes: the word segment addition text in normal interface screenshot, is obtained To the interface screenshot with overlay text；

And/or

It includes: the setting messy code character in normal interface screenshot that obtaining, which has the method for the interface screenshot of messy code character, is had There is the interface screenshot of messy code character.

4. a kind of application interface method for detecting abnormality, which is characterized in that the described method includes:

By screenshot input area detection model in interface to be detected so that the region detection model to the interface screenshot into Row region detection obtains the corresponding target category in target area and the target area that the interface screenshot is included；Its In, the region detection model is to be generated using model generating method described in any one of any one of claims 1 to 33；The mesh Mark region includes at least one of following regions: showing incomplete region, the region with overlay text, has messy code character Region, pop-up region, message notifying frame region；

According to the corresponding target category in the target area and the target area, the corresponding exception of the interface screenshot is obtained As a result.

5. according to the method described in claim 4, it is characterized in that, including: in the target area that the interface screenshot is included It is described corresponding according to the target area and the target area in the case where pop-up region and/or message notifying frame region Target category, obtaining the corresponding abnormal results of the interface screenshot includes:

Identification obtains the text information of the target area；

The text information is matched with default unusual character, if there are the default abnormal words in the text information Symbol, then be determined as abnormal area for the target area, and that the target category is determined as the target area is corresponding Abnormal class.

6. according to the method described in claim 4, it is characterized by further comprising:

Export the corresponding abnormal results of the interface screenshot.

7. according to the method described in claim 4, it is characterized by further comprising:

8. the method according to the description of claim 7 is characterized in that further include:

In the network data base, the corresponding abnormal results of the abnormal results inquiry request are obtained, and return to the terminal Return the abnormal results.

9. a kind of model generating means, which is characterized in that described device includes:

Sample data obtains module, for obtaining the screenshot sample data of application interface；The screenshot sample data includes following At least one of sample data: show incomplete interface screenshot, the interface screenshot with overlay text, there is messy code character Interface screenshot, the interface screenshot with pop-up, the interface screenshot with message notifying frame；

10. device according to claim 9, which is characterized in that the multilayer convolutional neural networks include: that you only need to see Convolutional neural networks third edition YOLOv3.

11. device according to claim 9 or 10, which is characterized in that the sample data obtains module and is specifically used for:

Text is added in the word segment of normal interface screenshot, obtains the interface screenshot with overlay text；

And/or

12. a kind of application interface abnormal detector, which is characterized in that described device includes:

Region detection module is used for by screenshot input area detection model in interface to be detected, so that the region detection mould Type carries out region detection to the interface screenshot, obtains the target area and the target area that the interface screenshot is included Corresponding target category；Wherein, the region detection model is to be generated using model described in any one of any one of claims 1 to 33 What method generated；The target area includes at least one of following regions: showing incomplete region, with overlay text Region, the region with messy code character, pop-up region, message notifying frame region；

Abnormal results obtain module, for obtaining according to the corresponding target category in the target area and the target area The corresponding abnormal results of the interface screenshot.

13. device according to claim 12, which is characterized in that wrapped in the target area that the interface screenshot is included Include: in the case where pop-up region and/or message notifying frame region, the abnormal results obtain module and include:

Abnormal results acquiring unit, for matching the text information with default unusual character, if the text information In there are the default unusual characters, then the target area is determined as abnormal area, and the target category is determined For the corresponding abnormal class in the target area.

14. device according to claim 12, which is characterized in that described device further include:

15. device according to claim 12, which is characterized in that described device further include:

16. device according to claim 15, which is characterized in that described device further include:

Inquiry request receiving module, for receiving the abnormal results inquiry request for the network data base of terminal transmission；

Abnormal results return module, for it is corresponding different to obtain the abnormal results inquiry request in the network data base Often as a result, and returning to the abnormal results to the terminal.

17. a kind of terminal device, which is characterized in that including processor, memory and be stored on the memory and can be in institute The computer program run on processor is stated, such as claims 1 to 3 is realized when the computer program is executed by the processor Any one of described in model generating method or 4 to 8 in any application interface method for detecting abnormality.

18. a kind of computer readable storage medium, which is characterized in that store computer journey on the computer readable storage medium Sequence, the computer program realize model generating method as claimed any one in claims 1 to 3 when being executed by processor, Or any application interface method for detecting abnormality in 4 to 8.