CN106681917A - Method for automatically evaluating front ends on basis of neural networks - Google Patents
Method for automatically evaluating front ends on basis of neural networks Download PDFInfo
- Publication number
- CN106681917A CN106681917A CN201611190341.5A CN201611190341A CN106681917A CN 106681917 A CN106681917 A CN 106681917A CN 201611190341 A CN201611190341 A CN 201611190341A CN 106681917 A CN106681917 A CN 106681917A
- Authority
- CN
- China
- Prior art keywords
- data
- mouse
- developer
- interface
- page
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3604—Software analysis for verifying properties of programs
- G06F11/3608—Software analysis for verifying properties of programs using formal methods, e.g. model checking, abstract interpretation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Abstract
The invention discloses a method for automatically evaluating front ends on the basis of neural networks. The technical scheme includes that collected data are analyzed by the aid of neural network algorithms, locations possibly noticed by developers when the front ends are evaluated are computed and are portions which have design defects and need to be redesigned, evaluation experience of the developers are stored by the aid of the neural networks which are used as media, interface tests carried out by the developers are simulated by the aid of stored data, and accordingly the front ends can be automatically evaluated. The method has the advantages that neural network processes are applied to large-scale WEB user behavior simulation procedures, accordingly, the training data acquisition cost can be effectively reduced, the training efficiency can be improved, and the reusability of user behavior training results can be guaranteed.
Description
Technical field
The invention belongs to computer automation evaluation areas, it is related to front end to assess, artificial intelligence, the side such as automatic test
Face, developer can be used the assessment for carrying out front end of this method automation to judge that the inventive method is based primarily upon neutral net
Realized.
Background technology
1. artificial neural network
Artificial neural network (Artificial Neural Networks, ANN) provides one for machine learning developer
Kind universal and method of practicality from sample learning value be the function of real number, centrifugal pump or vector.It is substantially a kind of mould
Imitative biological neural network characterization carries out the Mathematical Modeling of information processing, and his powerful computing capability mostlys come from its parallel point
Cloth structure and his learning ability and generalization ability.By constantly research, artificial neural network technology is in multiple fields
Achieve good application.
Neural network model is made up of the association between substantial amounts of node and node, and node is also known as neuron.Each node
Represent specific a function, referred to as excitation function；Connection between each two node has corresponding weighted value, referred to as weight,
This constitutes the memory of neutral net.Using the data of training sample during neural metwork training, by corresponding algorithm, such as instead
To propagation algorithm, in the corrective network such as inverse propagation algorithm between each node weight value, realize the fitting to relationship by objective (RBO).
Neutral net is generally hierarchy, it is general for be divided into input layer (Input Layer), hidden layer (Hidden
Layer), three layers of output layer (Output Layer).Wherein input layer is used to receive training information；Hidden layer can include many
Layer, but typically can be using one layer, for simulating bio-networks analytic learning data correlation, the node layer number is more, and structure is got over
Complexity, more easily realizes the simulation of non-linear relation；The information that output layer feedback is exported after calculating.But with the development of technology,
Above structure may be no longer appropriate for new neural network.
By development for many years, nerual network technique is increasingly mature, and in the market has various neural network algorithm frameworks, such as
Joone, Neuroph etc., but for the consideration of each side such as copyright, it is desirable to have more choices of technology, the present invention does not make
Existing class libraries is used, but voluntarily realizes that convolutional neural networks are carried out at data using Java language according to neural network algorithm principle
Reason.
2. existing the related of WEB tests is constituted and problem
Existing WEB test assignments were roughly divided into such as the next stage：Development tests (interface testing, ui testing),
Phase of integration testing tests (operation flow test), and Pre-delivery Test of reaching the standard grade (compatibility test, safety test).
● interface testing：
Interface testing is that the data-interface (mostly AJAX is called) tested on WEB is returned for given |input paramete and result
Whether consistent with expected return, the unit testing use-case that is completed is covered when this class testing is generally rear end engineer coding,
It is the precondition for entering phase of integration testing.
Typically can write and run using this kind of unit test tools of JUnit, it is also possible to be tested using URL such as POSTMAN
Instrument is performed.
The subject matter of interface testing is that this class testing-case is that rear end engineer writes and directly uses, and
Under normal circumstances, because various economic or resource reasons cause research staff's construction cycle general all more nervous, institute was in the past
More basic test case is completed toward the only time to write, usually correct data and some boundary conditions, the project also having
Integration testing is completely dependent on, oneself does not do unit testing, the practical combinations in business are produced for ordinary circumstance and multiple interfaces
Raw situation will not typically write test case.
● ui testing：
Ui testing is used to test WEB application single interface display effect, user's operating function etc., this class testing one
As for front end engineer after the completion of the page is by art designing's figure transposition, access corresponding page data-interface after carry out, test enter
The precondition of phase of integration testing.
Substantial amounts of ui testing is carried out in itself based on browser, and front end engineer opens correspondence and is made WEB page simultaneously
Observation and original copy difference, then trigger action one by one again, verifies whether consistent with expectation.The problem of this test is to be difficult to automatically
Change, the component that the page is changed each time or the page is had influence on must all be re-executed once by hand, be lost unavoidably when quantity is big
Spill mistake.
Certainly also have similar to test frame as Selenium (PhantomJS or SlimerJS), other can be used
Programming language (Java, C#, Ruby, Python) hand-coding head end test use-case, so as to realize that test case is reusable, but
Here have individual problem in order to in the page can interactive object accurately interact, Test Engineer must specify HTML element selector
To choose target interactive object, such as after clicking certain button agreement contract engagement, could select to redirect lower one page, thus make
Test case must be write becomes increasingly complex.
In order to avoid the appearance of problem above, the test frame of the above can also be carried out the page for going to certain phase
Sectional drawing, then relatively judges rate of change based on picture again.So problem here is if accurately to judge page correctness
Can only be by judging that whether it is consistent with expection that HTLM elements are constituted, so some are too difficult.Or artificial identification before comparing
Correct sectional drawing and the difference of this sectional drawing judge whether that test passes through with this, but is again so not accurate enough.
● operation flow is tested：
Operation flow test is also the functional test on WEB, and whether the multiple business demands for testing complete WEB application can
Enough smooth completions, measurement can reach how many business objective, be the main goal in research of phase of integration testing.
Operation flow test general manner is consistent with ui testing, and simply test case can be across multiple pages.It is aobvious
Obtain more complicated.Therefore the testing tool more than can also be used herein, but write the cost of test case with business
Complexity linear multiplier.
● compatibility test：
Compatibility test is whether test can reach the system of operation flow can be normal under different browser environments
Offer service.
Compatibility test be presently mainly refer to business is performed on more browsers, then inspection result whether with before
Operation flow result is consistent, and whether interface display is normal.If this partial service automatic main is considered with Selenium so
Test frame, by WebDriver (can cause test case in IE, the operation of Firefox, Chrome platform), problem is
In order to realize automation, the generation of substantial amounts of selection HTML element as described in ui testing, must be just write
Code, realizes performing operation flow test in different browsers.
● safety test：
Safety test is that test meets business demand, and whether the system of compatibility requirement has security breaches, if Ke Nengzao
Into information leakage, the problems such as administrative power is captured.
Basic security test can be simulated abnormal request and be surveyed to by interface using browser and POSTMAN instruments
The interface of examination is completed.More professional safety test may consider to use the SQL injection of this kind of specialties of SqlMap to detect instrument
Carry out.The problem of this part similar to interface testing, but except the limited time will not do it is many in addition to, many research staff for
The problem understanding of safety is the real difficult point of safety test.
The content of the invention
Present invention mainly solves problem：Current Internet enterprises accordingly, have enough general lack of front end developer
The development person that experience can carry out front end assessment is more deficient, and a small number of developers assume responsibility for largely heavy and single commenting
Estimate work, this brings larger operating pressure to them；And working will certainly reduce the efficiency and matter of evaluation for a long time
The links such as amount, the product payment after influence.Yet with the reason such as testing requirement and scheme complexity, related realization automation is commented
The research for estimating field is less, and feasibility is not high, and most of Internet enterprises using traditional manual review mode still before being carried out
End assessment.Meanwhile, programmer/Test Engineer, the test mode of domestic consumer are different.Programmer/Test Engineer's
Test mode is mainly the functional module for being responsible for oneself, is triggered one by one according to design cycle, and auth response whether
It is consistent with expection.And the custom of user is then that content interested on the page is triggered and tested.Although, by programmer
Function and performance requirement of the test when can ensure used aloned website be met, but, because lack publishing
Preceding high-volume domestic consumer test, it will leave larger non-functional risk in the website of production status, and it is such
Testing requirement is cannot to be completed by existing automated test tool, such as, based on image recognition, keyboard and mouse operation record
Mode of picture etc., therefore, the present invention is with regard to a kind of new front end appraisal procedure based on neutral net of this proposition.
The technical scheme is that：A kind of front end appraisal procedure based on neutral net, uses neutral net
Algorithm is analyzed to collecting the data come, and calculates developer in the position for may note when front end is assessed, these
Position is the presence of the part that design defect needs to redesign, by the assessment experience of developer with neutral net as media storage
Get off, and the test at interface is carried out using the digital simulation developer for storing, realize automation assessment, including two stages：
1) data collections processing stage：
Collection front end developer carries out browse action data when front end is assessed, and browse action data are dynamic including mouse
Make, developer's notice concentrated position when going out gather data according to these data analyses, and by corresponding sectional drawing and result of calculation
Preserve, be that next step neural network learning is prepared；
Screen is wherein evenly divided into multiple regions from horizontally and vertically direction, developer's mouse in training is being felt
The region of interest stops, and the residence time by mouse in a certain position marks out developer's notice concentrated position；
2) the neural network learnings stage：
Carry out experience when front end is assessed using neural network learning developer to be operated with custom, afterwards using training
Neuron network simulation developer carry out interface estimation, the business demand of defective locations prediction is completed, with before corresponding during training
End sectional drawing and developer's notice concentrated position as training set, after network training is finished, using interface to be assessed as
Input, the possible notice concentrated position of developer is exported using neutral net.
The data collection of data collection processing stage is specially：
By monitoring track and the residence time information of the mouse of mouse situation of movement collection developer, using Chrome
Correlative code in the monitoring module content script of browser realizes that the monitor method provided using Javascript is supervised
Mouse situation of movement is listened, data record is just carried out when developer's mouse is moved, during gather data in units of screen, i.e.,
Mouse action of the content viewable and developer at record current browser interface in this panel region, when developer carries out mouse
Just mean that the operation at this interface is collected during the operations of influence screen display content such as mark is rolled, scaling to finish, monitor module just
Collection section background module background is given by data, information and corresponding interface sectional drawing that section background module will be collected is collected
The AJAX components for being used together JQuery offers carry out data upload, preserve for subsequent data analysis, and carry out data scrubbing
Prepare next round to monitor.
Further, the same time only collect current browser interface mouse mobile data, the data at non-present interface are neither
Collect, also do not retain, collect the Shipping Options Page id that section background module obtains current page in page furbishing or page creation, lead to
Crossing id orders current page monitoring module carries out data collection operation, and section background module is collected during page layout switch by current data
Pass, if data not enough if give up, it is ensured that the matching of peration data and page screenshot.
After server receives collection segment data, start to analyze notice concentrated position of the developer in respective interface：If
Mouse position is exactly the notice concentrated position of developer, first, all of mouse positional data is traveled through, according to mouse
Each pixel weight in stop place and residence Time Calculation interface, mouse is more long in certain point residence time, then the point and neighbouring
The weight of pixel is higher；The page is then divided into multiple regions, weight highest region is calculated, assert here it is surveying
The notice concentrated area of developer during examination.After calculating notice concentrated area, by the image storage after result and compression,
Prepare ensuing neural network learning.
Preferably, the present invention realizes neutral net using Java, using feedforward neural network type, including
Convolutional neural networks and BP neural network.
During neural net method is applied to extensive WEB customer behavior modelings by the present invention, effectively reduce
Gather the cost of training data, improve the efficiency of training, and ensure that the durability of user behavior training result：
1) the present invention provides the JavaScript plug-in units based on existing browser execution, can not change WEB websites original
Have on the basis of realizing and realize function.
2) the present invention by neural net method set up user operate and be tested between interface corresponding relation when use
Thicker capture region instead of pixel is effectively reduced training sample set quantity, improves training effectiveness；
4) end results of the invention can be by supporting to send the customizing browser execution that mouse/touch code is triggered, together
When support that multiple test assignments/multiple are tested user's training set and concurrently performed, by being then based on neural fusion, in new page
Adjustment need not be re-started when being applied on face just can directly be carried out, it is ensured that the durability of user behavior training result.
Brief description of the drawings
Fig. 1 is the modular structure schematic diagram of the inventive method.
Fig. 2 is data collection use-case timing diagram in the inventive method.
Fig. 3 is the data gathering method schematic diagram of the inventive method.
Fig. 4 is neuron schematic diagram.
Fig. 5 is the inventive method Architecture of Feed-forward Neural Network schematic diagram.
Fig. 6 realizes class figure for BP neural network.
Fig. 7 is SGD timing diagrams.
Fig. 8 realizes class figure for convolutional neural networks.
Specific embodiment
It is an object of the present invention to：Automatically carry out front end assessment, judge front-end interface there may be the position of defect with
And whether checking interface development meets exploitation and requires.Invention general realization be：Developer is collected using data collection instrument
Action data when front end browses is carried out, study is analyzed to the data collected using neural network algorithm afterwards, finally
The neuron network simulation developer finished using training is carried out automation front end and assessed.
The present invention proposes a kind of front end appraisal procedure based on neutral net, using neural network algorithm to searching
The data that collection comes are analyzed, assessment of the analog development personnel to front-end interface, realize automation assessment, including two stages：
1) data collections processing stage：
The daily browse action data carried out when front end is assessed of developer are collected, browse action data are dynamic including mouse
Make, developer's notice concentrated position when going out gather data according to these data analyses, and by corresponding sectional drawing and result of calculation
Preserve, be that next step neural network learning is prepared；
Screen is wherein evenly divided into multiple regions from horizontally and vertically direction, it is desirable to which developer uses mouse in training
Mark out their contents interested, and allow stop of the mouse in the region long period interested, by mouse a certain
The residence time of position marks out developer's notice concentrated position；
2) the neural network learnings stage：
Carry out experience when front end is assessed using neural network learning developer to be operated with custom, afterwards using training
Neuron network simulation developer carry out interface estimation, the business demand of defective locations prediction is completed, with before corresponding during training
End sectional drawing and developer's notice concentrated position as training set, after network training is finished, using interface to be assessed as
Input, the possible notice concentrated position of developer is exported using neutral net,
Wherein in the type of neutral net, convolutional neural networks are used.
Specific implementation of the invention addressed below, below explanation is only used for clearly describing technical side of the invention
Case, it is impossible to limited the scope of the invention with this.The realization of the inventive method has used pipeline-filtration device structure, is divided into number
According to module, data analysis module and neural network learning module is collected, data gathering module is corresponding with data analysis module described
Data collection processing stage work, and the task in neural network learning module then responsible nerve e-learning stage.
System architecture is as shown in figure 1, hereafter will successively introduce the Key Implementation of these three modules：
Data gathering module
The crucial use-case of data gathering module is data collection.The to the effect that collection exploit person of data collection use-case
Action message of the member when operation is estimated, mainly mouse action, that is, collect track and the stop of the mouse of developer
Temporal information.After information search is finished, the information collected will pass to data analysis module together with the sectional drawing of interface.This module
The main correlative code by monitoring module (content_script) in middle data collection part realizes that we utilize
The monitor method that Javascript is provided monitors mouse situation of movement, and data note is just carried out when developer's mouse is moved
Record.When developer just mean when mouse rollovers etc. are operated that the operation collection at this interface is finished, module is monitored just by number
Section background module (background) is collected according to giving, collect section background module carries out sectional drawing using ChromeAPI, and uses
The AJAX components that JQuery is provided carry out data upload.
In gather data in units of screen, i.e., we record the visual interior of current browser interface to data gathering module
Hold and mouse action of the developer in this panel region.When developer is after necessarily being operated, plug-in unit is according to definition
Rule judges to need to carry out data record, and notifies that collecting section background module carries out screen interception；When detecting rolling, scaling etc.
When influenceing the operation of screen display content, plug-in unit will upload data, and carry out data scrubbing preparation next round monitoring.
Except rolling operation, data upload can be also carried out when Shipping Options Page switching or refresh activity is detected.This be by
, by system maintenance, it is singleton in collect section background module to be.In order to prevent multiple data and the page for monitoring module collection
Sectional drawing is mismatched, and we take the same time only to collect the strategy of current browser interface mouse mobile data, non-present interface
Data are neither collected, and are not also retained.Collect the mark that section background module obtains current page in page furbishing or page creation
Page id is signed, monitoring module by id orders current page carries out data collection operation, and section background module meeting is collected during page layout switch
Current data is uploaded, if data not enough if give up, it is ensured that the matching of peration data and page screenshot.
The action sequence of data collection is briefly described below, timing diagram is as shown in Figure 2
Main flow：
1) developer opens new Shipping Options Page, or jumps to the new page by being input into network address or clickable hyperlinks；
2) .Chrome detects corresponding operating, and calls the monitor method tabChange for collecting and being registered in section background module
(monitoring Shipping Options Page change) or tabUpdate (monitoring page furbishing)；
3) collects section background module and opens monitoring to current label page transmission by the information transmitting methods that Chrome is provided
Signal；
4) is monitored after module receives coherent signal and is opened mouse event monitoring, when record mouse location information is with stopping
Between information；
5) was monitored after module collected enough data and is assert that this page data is meaningful, it is desirable to collect section background module
Carry out page screenshot；
6) monitors module and continues to monitor developer's mouse action, until detecting page scroll operation or page zoom-in and zoom-out
Operation；
7) data that monitorings module will be collected are sent to collection section background module；
8) integrated together with the screenshot capture that collection section background modules finish mouse data with coding and be sent to data analysis
Module.
Exception stream：
5a) developer carries out window scaling or page scroll in the case where plug-in unit does not collect enough data
Deng influence page display content operation；
6) stops this monitoring, clears up data, is monitored next time from 4) proceeding by.
Data analysis module
The crucial use-case of data analysis is the notice concentrated position for analyzing developer in respective interface.It is assumed that mouse
Mark position is exactly the notice concentrated position of developer, and we devise following algorithm and calculate notice concentrated position：
First, algorithm travels through all of mouse positional data, according to each pixel power in mouse stop place and residence Time Calculation interface
Weight, mouse is more long in certain point residence time, then the weight of the point and neighbouring pixel is higher；Then be divided into for the page by we
Multiple regions, calculate weight highest region, we assert here it is test when tester notice concentrated area.
After calculating notice concentrated area, the image storage after result and compression is prepared ensuing neural network learning by us.
In processing data, we are according to the big arrays such as the size of screenshot capture structure, each pixel on record screen
The weight of point, processes the mouse message for receiving successively afterwards.It is assumed that mouse position is developer at that time
Notice concentrated position, but this position is typically sub-fraction deviation, therefore mouse position record can increase mouse institute
In position and the weight of the pixel nearer apart from position.The algorithm that we design is increased apart from mouse position one
The weight of all pixels in set a distance, increased size is by depending on a distance from mouse exact position.Because calculating accurate distance
Need the larger computing resource of consumption, we using approximate substitution mode, as shown in figure 3, by pixel and mouse position
Level, vertical distance sum are considered as the distance with mouse position, and in addition in view of the influence of mouse residence time, therefore we will
Above-mentioned two variable is multiplied as final weight increment, is described in detail below：
1) defines radiation scope l；
2) processing datas：It is assumed that receiving a following data：Mouse (x, y) point on screen has stopped time t；
3) settings weight more new range：More new range be abscissa at [x-l, x+l], ordinate is in [y-l, y+l]
All pixels point；
4) weight put in the range of renewals：For the point (m, n) in the data radiation scope, the weight of the point
Increment is (| m-x |+| n-y |) * t.
Analysis of neural network module
We mainly realize convolutional neural networks to this part with Java, and are built based on this neural network model suitable
When neutral net.Then the training set for being collected using us is trained to the parameter in neutral net, final to use instruction
The parameter perfected completes the automation assessment to interface.
Mistake already mentioned above, neutral net is even formed by the mutual structure of a large amount of neurons.As shown in figure 4, neuron
Receive one group of input, it is defeated after activation primitive is activated with being eventually passed plus a bias term b after corresponding weight w weighted sum
Go out.Wherein the weight w of neuron and bias b are the parameter that can be trained, and equivalent to the memory of neuron, and activation primitive is then
It is the defect that linear model ability to express difference is made up to introduce non-linear factor.
Neutral net can be generally divided into 3 kinds, respectively feedforward neural network, feedback god according to the mutual contact mode of neuron
Through network and self-organizing network, our this two kinds of neural network model realized belong to the classification of feedforward neural network.
As shown in figure 5, feedforward neural network includes input layer, an output layer and some hidden layers, wherein except except input layer,
Each neuron of remaining each layer receives the output of all neurons of preceding layer as input, after being activated through row weighted sum
Export to next layer, so circulation is until output end product.The characteristics of feed-forward type neutral net is that data are transmitted in a network
During can only forward, until reaching output layer, interlayer is without feedback signal backward.
Shown in Fig. 6 is that BP neural network realizes class figure, and it is right based on " one layer " in neutral net that we select
As building our BP neural network.Abstract class Layer is 3 kinds of layer models in BP neural network：Input layer
(InputLayer), the parent of hidden layer (HiddenLayer) and output layer (OutputLayer), Layer classes define one group
Two parameters of parameter, wherein weights and biases be in all of neuron in Layer parameter it is abstract, use matrix
(DMatrix:Double type matrix) connection weight and the biasing of all neurons are represented, this causes all of neutral net
Operation is all based on matrix operation, including to the matrixing of input and output regulation.In addition, abstract class Layer also specify
Four abstract methods are respectively init (parameter initialization for the layer network is operated), feedforward (by preceding to biography
The activation value a) that passs to settle accounts every layer, backpropagation (error calculated per layer network is inversely propagated by error),
Update (updates the weights and bias in network by the error of every layer network).In view of activation primitive
(activation function) and cost function (cost function) might have various realizations, and we have taken out phase
The interface answered：ActivationFunction and CostFunction, and various realizations will be given.And class BPNetwork is right
The higher level of abstraction of BP neural network, is responsible for receiving training set and test set and one group of hyper parameter (super parameter) is come
Training network, wherein accuracy methods are to calculate the recognition accuracy for specifying sample set, and recognize is that identification is specifically tested
Card code input, and SGD is most important method, by stochastic gradient descent (stochastic gradient descent) come
Training BP neural network, specific training process is as shown in Figure 7.
Shown in Fig. 8 is that convolutional neural networks realize class figure, the design and BP nerve nets above of convolutional neural networks
Network is consistent in overall architecture.But compared to BP neural network, convolutional neural networks are with characteristic pattern (feature map)
As the object for the treatment of, and because the presence of multi-kernel convolution may export multiple characteristic patterns.If each characteristic pattern square
If matrix representation, convolutional neural networks may need to process multiple matrixes, so two have been had more in abstract class Layer with matrix
Array is the interface of input.In addition, convolutional neural networks have had more convolutional layer (ConvLayer), sample level
(MaxPoolLayer) and full articulamentum (FullConLayer) definition.On the abstract of convolutional neural networks
(ConvNetwork) and training process design and BP neural network it is basically identical, simply implement distinct.
We can carry out the automation to front end page after neural network model has been realized with using neutral net
Assessment.When being trained to training set using neutral net, have it is following some should be noted：
1) number of hidden layer is not The more the better in .BP neutral nets, and general list hidden layer BP neural network is just
With good learning ability.
2) we the adjustment to parameter in neutral net is realized using BP algorithm (Back Propagation Algorithm), using
During BP algorithm, it would be desirable to the speed that arrange parameter is adjusted every time, i.e. pace of learning lr (learning rate).Lr typically sets
It is set between 0.01 to 0.1, and needs are progressively adjusted according to training process.
3) convolutional neural networks have bigger learning ability compared to BP neural network, but convolutional neural networks are general
The number of plies is more and convolution process is more complicated, it is necessary to bigger computing cost.
The above is only the preferred embodiments of the invention, the improvement carried out on the premise of the technology of the present invention principle is not departed from
Protection scope of the present invention should be also considered as with mutation.
Claims (6)
1. a kind of front end appraisal procedure based on neutral net, it is characterized in that using neural network algorithm to collecting what is come
Data are analyzed, and calculate developer in the position for may note when front end is assessed, and these positions are in the presence of design
Defect needs the part for redesigning, and the assessment experience of developer is got off by media storage of neutral net, and utilization is deposited
The digital simulation developer of storage carries out the test at interface, realizes automation assessment, including two stages：
1) data collections processing stage：
Collection front end developer carries out browse action data when front end is assessed, and browse action data include mouse action,
Developer's notice concentrated position when going out gather data according to these data analyses, and corresponding sectional drawing and result of calculation are protected
Deposit, be that next step neural network learning is prepared；
Screen is wherein evenly divided into multiple regions from horizontally and vertically direction, developer in training mouse interested
Region stop, the residence time by mouse in a certain position marks out developer's notice concentrated position；
2) the neural network learnings stage：
Experience when front end is assessed is carried out using neural network learning developer to be operated with custom, and the god for training is used afterwards
Interface estimation is carried out through network analog developer, the business demand of defective locations prediction is completed, is cut with respective front ends during training
Scheme and developer's notice concentrated position is used as training set, after network training is finished, using interface to be assessed as input,
The possible notice concentrated position of developer is exported using neutral net.
2. a kind of front end appraisal procedure based on neutral net according to claim 1, it is characterized in that data are searched
The data collection for collecting processing stage is specially：
By monitoring track and the residence time information of the mouse of mouse situation of movement collection developer, browsed using Chrome
Correlative code in the monitoring module content script of device realizes that the monitor method provided using Javascript monitors mouse
Mark situation of movement, data record is just carried out when developer's mouse is moved, during gather data in units of screen, that is, is recorded
Mouse action of the content viewable and developer at current browser interface in this panel region, when developer carries out mouse rolling
Just mean that the operation at this interface is collected during the operation of the influence screen display content such as dynamic, scaling to finish, monitor module just by number
Section background module background is collected according to giving, the information that collection section background module will be collected is together with corresponding interface sectional drawing
The AJAX components provided using JQuery carry out data upload, preserve for subsequent data analysis, and carry out data scrubbing preparation
Next round is monitored.
3. a kind of front end appraisal procedure based on neutral net according to claim 2, it is characterized in that with for the moment
Between only collect current browser interface mouse mobile data, the data at non-present interface are neither collected, also not retained, and collect section backstage
Module obtains the Shipping Options Page id of current page in page furbishing or page creation, and module is monitored by id orders current page
Carry out data collection operation, during page layout switch collect section background module current data is uploaded, if data not enough if give up, protect
The matching of card peration data and page screenshot.
4. a kind of front end appraisal procedure based on neutral net according to Claims 2 or 3, it is characterized in that data
Collect in processing stage, analysis developer is in the notice concentrated position of respective interface：If mouse position is exactly out
The notice concentrated position of hair personnel, first, travels through all of mouse positional data, according to mouse stop place and residence time
Each pixel weight in interface is calculated, mouse is more long in certain point residence time, then the weight of the point and neighbouring pixel is higher；With
The page is divided into multiple regions afterwards, weight highest region is calculated, assert here it is during test developer attention
Power concentrated area, after calculating notice concentrated area, by the image storage after result and compression, prepares ensuing nerve net
Network learns.
5. a kind of front end appraisal procedure based on neutral net according to claim 4, it is characterized in that according to screen
The big arrays such as the size structure of curtain sectional drawing, the weight of each pixel, processes the mouse for receiving successively afterwards on record screen
Mark information, increases the weight of all pixels in the certain distance of mouse position, and increased size is by from the accurate position of mouse
Depending on the distance put, specially the distance and the two variables of mouse residence time is multiplied and increase as final weight
Amount, wherein calculating mouse exact position using the mode of approximate substitution：
1) defines radiation scope l；
2) processing datas：It is assumed that receiving a following data：Mouse (x, y) point on screen has stopped t；
3) settings weight more new range：More new range be abscissa at [x-l, x+l], ordinate is all in [y-l, y+l]
Pixel；
4) weight put in the range of renewals：For the point (m, n) in the data radiation scope, the increment of the weight of the point
It is (| m-x |+| n-y |) * t.
6. a kind of front end appraisal procedure based on neutral net according to claim 1, it is characterized in that using
Java realizes neutral net, using feedforward neural network type, including convolutional neural networks and BP neural network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611190341.5A CN106681917B (en) | 2016-12-21 | 2016-12-21 | A kind of front end appraisal procedure neural network based |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611190341.5A CN106681917B (en) | 2016-12-21 | 2016-12-21 | A kind of front end appraisal procedure neural network based |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106681917A true CN106681917A (en) | 2017-05-17 |
| CN106681917B CN106681917B (en) | 2019-06-18 |
Family
ID=58870799
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611190341.5A Active CN106681917B (en) | 2016-12-21 | 2016-12-21 | A kind of front end appraisal procedure neural network based |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106681917B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107678817A (en) * | 2017-09-21 | 2018-02-09 | 北京三快在线科技有限公司 | View switching method and apparatus and computing device |
| CN107766852A (en) * | 2017-12-06 | 2018-03-06 | 电子科技大学 | A kind of man-machine mouse track detection method based on convolutional neural networks |
| CN109388737A (en) * | 2017-08-03 | 2019-02-26 | 腾讯科技（北京）有限公司 | A kind of sending method, device and the storage medium of the exposure data of content item |
| CN110275820A (en) * | 2018-03-15 | 2019-09-24 | 阿里巴巴集团控股有限公司 | Page compatibility test method, system and equipment |
| CN110955369A (en) * | 2019-11-19 | 2020-04-03 | 广东智媒云图科技股份有限公司 | Focus judgment method, device and equipment based on click position and storage medium |
| CN111400628A (en) * | 2020-03-12 | 2020-07-10 | 腾讯科技（深圳）有限公司 | Information propagation method, device, equipment and medium |
| CN111752870A (en) * | 2020-05-28 | 2020-10-09 | 苏州浪潮智能科技有限公司 | Compatible method and system of server interface based on neural network |
| CN111797007A (en) * | 2020-06-18 | 2020-10-20 | 中国科学院软件研究所 | Automatic detection and positioning method for application program user interface defects and electronic device |
| CN111930634A (en) * | 2020-09-09 | 2020-11-13 | 北京瑞莱智慧科技有限公司 | Model processing method, device, medium and computing equipment |
| CN112257075A (en) * | 2020-11-11 | 2021-01-22 | 福建有度网络安全技术有限公司 | System vulnerability detection method, device, equipment and storage medium under intranet environment |
| CN108108622B (en) * | 2017-12-13 | 2021-03-16 | 上海交通大学 | Vulnerability detection system based on deep convolutional network and control flow graph |
| CN113791781A (en) * | 2021-09-09 | 2021-12-14 | 南京大学 | Interface function scene division method and device based on graph neural network |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102946412A (en) * | 2012-10-17 | 2013-02-27 | 沈阳创达技术交易市场有限公司 | Internet-based friend-making platform |
| US20150350174A1 (en) * | 2014-05-30 | 2015-12-03 | Ca, Inc. | Controlling application programming interface transactions based on content of earlier transactions |
| CN105786999A (en) * | 2016-02-17 | 2016-07-20 | 扬州大学 | Software-developer visualization recommendation method based on complex network relationship |
-
2016
- 2016-12-21 CN CN201611190341.5A patent/CN106681917B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102946412A (en) * | 2012-10-17 | 2013-02-27 | 沈阳创达技术交易市场有限公司 | Internet-based friend-making platform |
| US20150350174A1 (en) * | 2014-05-30 | 2015-12-03 | Ca, Inc. | Controlling application programming interface transactions based on content of earlier transactions |
| CN105786999A (en) * | 2016-02-17 | 2016-07-20 | 扬州大学 | Software-developer visualization recommendation method based on complex network relationship |
Non-Patent Citations (2)
| Title |
|---|
| 孙会君 等: "应用人工神经网络确定评价指标的权重", 《山东科技大学学报(自然科学版)》 * |
| 李玉蓉: "基于神经网络和线性像素置换的颜色量化", 《光电工程》 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109388737A (en) * | 2017-08-03 | 2019-02-26 | 腾讯科技（北京）有限公司 | A kind of sending method, device and the storage medium of the exposure data of content item |
| CN109388737B (en) * | 2017-08-03 | 2023-03-31 | 腾讯科技（北京）有限公司 | Method and device for sending exposure data of content item and storage medium |
| CN107678817A (en) * | 2017-09-21 | 2018-02-09 | 北京三快在线科技有限公司 | View switching method and apparatus and computing device |
| CN107766852A (en) * | 2017-12-06 | 2018-03-06 | 电子科技大学 | A kind of man-machine mouse track detection method based on convolutional neural networks |
| CN108108622B (en) * | 2017-12-13 | 2021-03-16 | 上海交通大学 | Vulnerability detection system based on deep convolutional network and control flow graph |
| CN110275820A (en) * | 2018-03-15 | 2019-09-24 | 阿里巴巴集团控股有限公司 | Page compatibility test method, system and equipment |
| CN110275820B (en) * | 2018-03-15 | 2023-11-21 | 阿里巴巴集团控股有限公司 | Page compatibility testing method, system and equipment |
| CN110955369B (en) * | 2019-11-19 | 2022-04-01 | 广东智媒云图科技股份有限公司 | Focus judgment method, device and equipment based on click position and storage medium |
| CN110955369A (en) * | 2019-11-19 | 2020-04-03 | 广东智媒云图科技股份有限公司 | Focus judgment method, device and equipment based on click position and storage medium |
| CN111400628A (en) * | 2020-03-12 | 2020-07-10 | 腾讯科技（深圳）有限公司 | Information propagation method, device, equipment and medium |
| CN111752870A (en) * | 2020-05-28 | 2020-10-09 | 苏州浪潮智能科技有限公司 | Compatible method and system of server interface based on neural network |
| CN111797007A (en) * | 2020-06-18 | 2020-10-20 | 中国科学院软件研究所 | Automatic detection and positioning method for application program user interface defects and electronic device |
| CN111930634A (en) * | 2020-09-09 | 2020-11-13 | 北京瑞莱智慧科技有限公司 | Model processing method, device, medium and computing equipment |
| CN112257075A (en) * | 2020-11-11 | 2021-01-22 | 福建有度网络安全技术有限公司 | System vulnerability detection method, device, equipment and storage medium under intranet environment |
| CN113791781A (en) * | 2021-09-09 | 2021-12-14 | 南京大学 | Interface function scene division method and device based on graph neural network |
| CN113791781B (en) * | 2021-09-09 | 2023-07-18 | 南京大学 | Interface function scene division method and device based on graph neural network |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106681917B (en) | 2019-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106681917B (en) | A kind of front end appraisal procedure neural network based | |
| CN107862331A (en) | It is a kind of based on time series and CNN unsafe acts recognition methods and system | |
| Mounce et al. | Burst detection using hydraulic data from water distribution systems with artificial neural networks | |
| CN112101431A (en) | Electronic equipment fault diagnosis system | |
| Linn et al. | Desktop activity mining-a new level of detail in mining business processes | |
| CN110245285A (en) | A kind of personalized recommendation method based on Heterogeneous Information network | |
| Hon et al. | Applications of Bayesian approaches in construction management research: a systematic review | |
| CN111709393B (en) | Structural damage identification method combining convolution and cyclic neural network | |
| Tomaszewski et al. | Statistical models vs. expert estimation for fault prediction in modified code–an industrial case study | |
| Huang et al. | Physics-guided deep neural network for structural damage identification | |
| Hu et al. | A new quantitative digital twin maturity model for high-end equipment | |
| Liu et al. | Quantifying the reliability of defects located by bridge inspectors through human observation behavioral analysis | |
| Altaweel et al. | A framework to structure agent-based modeling data for social-ecological systems | |
| Kenett et al. | Analytics as an enabler of advanced manufacturing | |
| Schlegel et al. | Modelspex: Model specification using explainable artificial intelligence methods | |
| Chaudhari et al. | Supporting designer learning and performance in design space exploration: a goal-setting approach | |
| CN113033845B (en) | Construction method and device for power transmission resource co-construction and sharing | |
| Li et al. | Structural health monitoring response reconstruction based on UAGAN under structural condition variations with few-shot learning [J] | |
| Waghmode et al. | A collective study of PCA and neural network based on COCOMO for software cost estimation | |
| CN110780660A (en) | Tobacco production industry control system fault diagnosis method based on production state | |
| Guo et al. | MHT: A multiscale hourglass-transformer for remaining useful life prediction of aircraft engine | |
| Kumar et al. | Data Management and Visual Information Processing using Machine Learning | |
| Deibert et al. | Mobile technology in the construction industry–the impact on business processes in job production | |
| Yuan et al. | A Risk Evaluation Method Before Using a Black-box Model Considering Sensor Failure | |
| Goel et al. | Investigating of high and low impact faults in object-oriented projects |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |