CN108197030A - Software interface based on deep learning tests cloud platform device and test method automatically - Google Patents
Software interface based on deep learning tests cloud platform device and test method automatically Download PDFInfo
- Publication number
- CN108197030A CN108197030A CN201810034716.1A CN201810034716A CN108197030A CN 108197030 A CN108197030 A CN 108197030A CN 201810034716 A CN201810034716 A CN 201810034716A CN 108197030 A CN108197030 A CN 108197030A
- Authority
- CN
- China
- Prior art keywords
- sectional drawing
- image
- module
- software interface
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of software interfaces based on deep learning to test cloud platform device automatically, including several user terminals and Cloud Server, user terminal includes Web browser and automatic screen capture module, Cloud Server includes Web server, database, deep neural network model, sectional drawing receiving module, webcrawler module, Web browser and Web server communication connection, automatic screen capture module is connect with sectional drawing receiving module, sectional drawing receiving module is connected respectively to database with webcrawler module, database is connect respectively with Web server and deep neural network model.Test method includes the following steps：(1) automatic sectional drawing, and the label of aesthetic measure is carried out to sectional drawing；(2) training of deep neural network model is carried out；(3) make to evaluate and test the software interface image newly inputted, evaluate its aesthetic measure.The present invention can be realized is classified different software interfaces according to aesthetic measure automatically, time saving and energy saving, high efficiency.
Description
Technical field
The present invention relates to a kind of cloud platform device, specifically a kind of software interface based on deep learning is tested cloud and is put down automatically
Table apparatus and test method.
Background technology
Interface detection (abbreviation UI tests), test user interface function module layout whether rationally, whole style be
Whether the placement location of no consistent, each control meets client's use habit, additionally wants test interface simple operation, navigation
Being easily understood property, the availability of page elements, whether word is correct in interface, and whether name unifies, and whether the page is beautiful, text
Word, picture combination whether perfection etc..
Traditional software interface is tested mainly based on artificial, time-consuming and laborious, and there are the problems such as retest.
Invention content
The purpose of the present invention is to provide a kind of software interfaces based on deep learning to test cloud platform device and survey automatically
Method for testing, to solve the problems mentioned in the above background technology.
To achieve the above object, the present invention provides following technical solution：
A kind of software interface based on deep learning tests cloud platform device automatically, is taken including several user terminals and cloud
Business device, every user terminal include Web browser and automatic screen capture module, and the Cloud Server includes Web server, number
According to library, deep neural network model, sectional drawing receiving module, webcrawler module, the Web browser and Web server communication
Connection, the automatic screen capture module are connect with sectional drawing receiving module, and the sectional drawing receiving module connects respectively with webcrawler module
Be connected to database, the database is connect respectively with Web server and deep neural network model, the webcrawler module with
Image search engine connects.
As further embodiment of the present invention：The automatic screen capture module is deployed in user terminal, and interception user is using
Different interfaces during software, and upload to Cloud Server.
As further scheme of the invention：The sectional drawing receives communication module and is deployed in Cloud Server, collect by with
The image that terminal automatic screen capture module in family uploads, and image is stored in file system, image information is stored in database.
As further scheme of the invention：The Web server is deployed in Cloud Server, for showing boundary to user
Face image, and receive evaluation of the user to interface.
As further scheme of the invention：The webcrawler module is deployed in Cloud Server, utilizes Internet
On image search engine, carry out software interface image download, and by the image crawled be stored in file system, by image information
It is stored in database.
As further scheme of the invention：The database is used to store the information of software interface sectional drawing image, packet
Include its number, store path, corresponding dbase.
As further scheme of the invention：The deep neural network module completes the beautiful interface journey based on image
Spend automating sorting function.
As further scheme of the invention：The software interface based on deep learning tests cloud platform dress automatically
The test method put, includes the following steps：
(1) user is using carrying out automatic sectional drawing, and the label of aesthetic measure is carried out to sectional drawing during software；
(2) training of deep neural network model is carried out using the interface image of tape label；
(3) deep neural network model obtained using training, evaluates and tests the software interface image newly inputted, evaluates
Its aesthetic measure.
Compared with prior art, the beneficial effects of the invention are as follows：The present invention can realize by different software interfaces according to
Aesthetic measure is classified automatically.The present invention is based on depth learning technologies, and interface detection, time saving and energy saving, efficiency are carried out to software
Efficiently.
Description of the drawings
Fig. 1 is the structure diagram that the software interface based on deep learning tests cloud platform device automatically.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other without making creative work
Embodiment shall fall within the protection scope of the present invention.
Referring to Fig. 1, in the embodiment of the present invention, a kind of software interface based on deep learning tests cloud platform dress automatically
It puts, including several user terminals and Cloud Server, every user terminal includes Web browser and automatic screen capture module,
The Cloud Server includes Web server, database, deep neural network (DNN) model, sectional drawing receiving module, web crawlers
Module, the Web browser and Web server communication connection, Web server is deployed in Cloud Server, for being shown to user
Interface image, and evaluation of the user to interface is received, the automatic screen capture module is connect with sectional drawing receiving module, automatic screenshotss mould
Block portion is deployed on user terminal, different interfaces of the interception user when using software, and uploads to Cloud Server, and sectional drawing receives communication
Module is deployed in Cloud Server, collects the image uploaded by the automatic screen capture module of user terminal, and image is stored in file system,
Image information is stored in database；The sectional drawing receiving module is connected respectively to database, web crawlers with webcrawler module
Module is deployed in Cloud Server, using the image search engine on Internet, carries out the download of software interface image, and will climb
The image deposit file system taken, database is stored in by image information；Database is used to store the letter of software interface sectional drawing image
Breath, including its number, store path, corresponding dbase；The database respectively with Web server and deep neural network
(DNN) model connects, and deep neural network (DNN) module completes the beautiful interface degree based on image and classifies automatically (evaluation) work(
Can, the webcrawler module is connect with image search engine.
Several user terminals include following functions：The automatic screenshotss upload of software interface, browsing and marker software circle
Face sectional drawing image；The Cloud Server includes following functions：Image data is received, is stored, deep neural network (DNN) model instruction
Practice and store and dispose Web server.
First, deep neural network (DNN) model
1. TensorFlow is selected to realize the DNN models of classification of images.Its feature is：
(1) its stabilization API (such as Keras and SkFlow) with highly compatible, and its side of being applied in combination with Numpy
Just.
(2) it is with significant platform flexibility.TensorFlow can be in the various machines from HPC to embedded system
Be applicable in.
(3) along with the support of Google and its community, more and more technologies and model are added into
In TensorFlow.
The present invention realizes two kinds of convolutional neural networks (CNN) algorithms using TensorFlow.The characteristics of CNN, is that its is right
All image classification data collection have good nicety of grading.
2. specific CNN network models type selecting, selects AlexNet and ResNet, feature is deeper structure and is scheming
As the good classification performance in classification application.
(1) one of CNN models that Alexnet will be realized as us in this project, feature is：Alexnet
Different from traditional LeNet CNN, it is more accurate and effective than LeNet, because of its algorithm and network structure.Alexnet can be
It is trained on multiple GPU.
(2) with the continuous intensification of neural network level, the classification results of flat network topology do not have significant change.ResNet
(residual error neural network) be intended to handle deeper into network problem, and influence classification results.Residual error neural network is short by introducing
Path avoids disappearance gradient problem, can transmit gradient in very deep network range.
2nd, Web technologies
Ajax (asynchronous JavaScript and XML)
In the embodiment of the present invention, the class label of the synchronous browsing picture in webpage is needed, and end of uploading onto the server
Database.In addition to using including the basic language including HTML5, CSS and JavaScript and being connected in server end
PHP documents except, the embodiment of the present invention comes the part of more new web page using Ajax, and feature is entire without reloading
The page.
For the present invention, when the screenshot capture of tested application program is uploaded on Cloud Server, Web page should
This shows them immediately, and content is reloaded without refresh page.Therefore, Ajax (asynchronous JavaScript and XML) can
To meet the needs of server is with server exchange low volume data as a kind of suitable technology.It more intuitively allows tester
Member and client's browsing pictures and label.
The Web technologies that the present invention uses have following features：
1. there are one MVC (model-view-controller) optimizations based on Ajax, more visualization data are provided and are represented
With better user experience.
2. its course of work is：First, Ajax sends from browser and asks.Then server receives to ask and return one
JSON data to browser through.Finally can there are one using front end tool interface render process.
3. the MVC of optimization is reduced to front-end and back-end Fractionation regimen using REST technologies.REST and Ajax has opposite
Rigidity, structure one and web front end and the connection framework of rear end can be selected.
3rd, automatic sectional drawing and uploading file
In the present invention, application module completion and automatic recognition software user interface during sectional drawing, upload to Cloud Server.
For the embodiment of the present invention using C# as development language, feature is there is the grammer system more more advanced than Java, and needed for C#
Operation platform build in internal Windows, and ensure the compatibility of its software.In order to reach required sectional drawing function, this
Invention is carried out using the method for calling Windows API, and feature is more easily obtain the handle of active window, obtain
The size and location of window is obtained, then realizes sectional drawing function.
After intercepting and capturing sectional drawing data, image data will be uploaded to Cloud Server, so the present invention is in automatic screen capture module
Upload function is provided, high in the clouds MySQL database server and locality connection are got up.It needs to remotely access permission, can make
It is accessed with any remote terminal.
By configuration server bind address and corresponding portal is added, the embodiment of the present invention can pass through ADO.net skills
Art is in client Connection Service device, so as to fulfill uploading pictures data.It, can be by DNN models after image data is uploaded to database
It is trained, can also be shown by Web server.
In the embodiment of the present invention, the software interface based on deep learning tests the test method of cloud platform device automatically, step
It is rapid as follows：
(1) user is using carrying out automatic sectional drawing, and the label of aesthetic measure (whether beautiful) is carried out to sectional drawing during software.
(2) training of deep neural network model is carried out using the interface image of tape label.
(3) deep neural network model obtained using training, evaluates and tests the software interface image newly inputted, evaluates
Its aesthetic measure.
The deep learning platform for recommending the maturation such as TensorFlow, Caffe carries out the structure of model.
It is obvious to a person skilled in the art that the present invention is not limited to the details of above-mentioned exemplary embodiment, Er Qie
In the case of without departing substantially from spirit or essential attributes of the invention, the present invention can be realized in other specific forms.Therefore, no matter
From the point of view of which point, the present embodiments are to be considered as illustrative and not restrictive, and the scope of the present invention is by appended power
Profit requirement rather than above description limit, it is intended that all by what is fallen within the meaning and scope of the equivalent requirements of the claims
Variation is included within the present invention.Any reference numeral in claim should not be considered as to the involved claim of limitation.
In addition, it should be understood that although this specification is described in terms of embodiments, but not each embodiment is only wrapped
Containing an independent technical solution, this description of the specification is merely for the sake of clarity, and those skilled in the art should
It considers the specification as a whole, the technical solutions in each embodiment can also be properly combined, forms those skilled in the art
The other embodiment being appreciated that.
Claims (8)
1. a kind of software interface based on deep learning tests cloud platform device automatically, including several user terminals and cloud service
Device, which is characterized in that every user terminal includes Web browser and automatic screen capture module, and the Cloud Server includes Web
Server, database, deep neural network model, sectional drawing receiving module, webcrawler module, the Web browser and Web clothes
Business device communication connection, the automatic screen capture module are connect with sectional drawing receiving module, the sectional drawing receiving module and web crawlers mould
Block is connected respectively to database, and the database is connect respectively with Web server and deep neural network model, and the network is climbed
Erpoglyph block is connect with image search engine.
2. the software interface according to claim 1 based on deep learning tests cloud platform device automatically, which is characterized in that
The automatic screen capture module is deployed in user terminal, different interfaces of the interception user when using software, and uploads to cloud service
Device.
3. the software interface according to claim 1 based on deep learning tests cloud platform device automatically, which is characterized in that
The sectional drawing receives communication module and is deployed in Cloud Server, collects the image uploaded by the automatic screen capture module of user terminal, and will
Image is stored in file system, and image information is stored in database.
4. the software interface according to claim 1 based on deep learning tests cloud platform device automatically, which is characterized in that
The Web server is deployed in Cloud Server, for showing interface image to user, and receives evaluation of the user to interface.
5. the software interface according to claim 1 based on deep learning tests cloud platform device automatically, which is characterized in that
The webcrawler module is deployed in Cloud Server, using the image search engine on Internet, carries out software interface image
Download, and by the image crawled be stored in file system, image information is stored in database.
6. the software interface according to claim 1 based on deep learning tests cloud platform device automatically, which is characterized in that
The database is used to store the information of software interface sectional drawing image, including its number, store path, corresponding dbase.
7. the software interface according to claim 1 based on deep learning tests cloud platform device automatically, which is characterized in that
The deep neural network module completes the beautiful interface degree automating sorting function based on image.
8. a kind of software interface based on deep learning as described in claim 1-2 is any tests the survey of cloud platform device automatically
Method for testing, which is characterized in that include the following steps：
(1) user is using carrying out automatic sectional drawing, and the label of aesthetic measure is carried out to sectional drawing during software；
(2) training of deep neural network model is carried out using the interface image of tape label；
(3) deep neural network model obtained using training, evaluates and tests the software interface image newly inputted, evaluates its U.S.
Sight degree.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810034716.1A CN108197030B (en) | 2018-01-15 | 2018-01-15 | Software interface automatic test cloud platform device based on deep learning and test method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810034716.1A CN108197030B (en) | 2018-01-15 | 2018-01-15 | Software interface automatic test cloud platform device based on deep learning and test method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108197030A true CN108197030A (en) | 2018-06-22 |
| CN108197030B CN108197030B (en) | 2021-03-30 |
Family
ID=62589393
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810034716.1A Active CN108197030B (en) | 2018-01-15 | 2018-01-15 | Software interface automatic test cloud platform device based on deep learning and test method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108197030B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109086201A (en) * | 2018-07-16 | 2018-12-25 | 曙光信息产业(北京)有限公司 | Automatic software test method and system |
| CN109324844A (en) * | 2018-09-30 | 2019-02-12 | 武汉斗鱼网络科技有限公司 | A kind of method, apparatus and computer equipment at detection window interface |
| CN110008993A (en) * | 2019-03-01 | 2019-07-12 | 华东师范大学 | A kind of end-to-end image-recognizing method based on deep neural network |
| CN110647940A (en) * | 2019-09-25 | 2020-01-03 | 捻果科技(深圳)有限公司 | Airport apron foreign person monitoring method based on video analysis and deep learning |
| CN112306846A (en) * | 2019-07-31 | 2021-02-02 | 北京大学 | Mobile application black box testing method based on deep learning |
| CN113837627A (en) * | 2021-09-28 | 2021-12-24 | 卡斯柯信号有限公司 | Job title review platform and method based on text processing |
| WO2023193417A1 (en) * | 2022-04-08 | 2023-10-12 | 郑州大学第一附属医院 | Deep learning-based gui testing method and system, and storage device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101295371A (en) * | 2008-04-18 | 2008-10-29 | 浙江大学 | Computer estimation method of Chinese character writing morphology beauty |
| CN103544101A (en) * | 2012-07-16 | 2014-01-29 | 深圳市世纪光速信息技术有限公司 | Wireless platform development automatic test system and method |
| US20160140424A1 (en) * | 2014-11-13 | 2016-05-19 | Nec Laboratories America, Inc. | Object-centric Fine-grained Image Classification |
| CN105894025A (en) * | 2016-03-30 | 2016-08-24 | 中国科学院自动化研究所 | Natural image aesthetic feeling quality assessment method based on multitask deep learning |
| CN107301425A (en) * | 2017-06-09 | 2017-10-27 | 浙江工业大学 | A kind of allowing child daubing methods of marking based on deep learning |
-
2018
- 2018-01-15 CN CN201810034716.1A patent/CN108197030B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101295371A (en) * | 2008-04-18 | 2008-10-29 | 浙江大学 | Computer estimation method of Chinese character writing morphology beauty |
| CN103544101A (en) * | 2012-07-16 | 2014-01-29 | 深圳市世纪光速信息技术有限公司 | Wireless platform development automatic test system and method |
| US20160140424A1 (en) * | 2014-11-13 | 2016-05-19 | Nec Laboratories America, Inc. | Object-centric Fine-grained Image Classification |
| CN105894025A (en) * | 2016-03-30 | 2016-08-24 | 中国科学院自动化研究所 | Natural image aesthetic feeling quality assessment method based on multitask deep learning |
| CN107301425A (en) * | 2017-06-09 | 2017-10-27 | 浙江工业大学 | A kind of allowing child daubing methods of marking based on deep learning |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109086201A (en) * | 2018-07-16 | 2018-12-25 | 曙光信息产业(北京)有限公司 | Automatic software test method and system |
| CN109324844A (en) * | 2018-09-30 | 2019-02-12 | 武汉斗鱼网络科技有限公司 | A kind of method, apparatus and computer equipment at detection window interface |
| CN110008993A (en) * | 2019-03-01 | 2019-07-12 | 华东师范大学 | A kind of end-to-end image-recognizing method based on deep neural network |
| CN112306846A (en) * | 2019-07-31 | 2021-02-02 | 北京大学 | Mobile application black box testing method based on deep learning |
| CN112306846B (en) * | 2019-07-31 | 2022-02-11 | 北京大学 | Mobile application black box testing method based on deep learning |
| CN110647940A (en) * | 2019-09-25 | 2020-01-03 | 捻果科技(深圳)有限公司 | Airport apron foreign person monitoring method based on video analysis and deep learning |
| CN113837627A (en) * | 2021-09-28 | 2021-12-24 | 卡斯柯信号有限公司 | Job title review platform and method based on text processing |
| CN113837627B (en) * | 2021-09-28 | 2024-03-15 | 卡斯柯信号有限公司 | Title review platform and method based on text processing |
| WO2023193417A1 (en) * | 2022-04-08 | 2023-10-12 | 郑州大学第一附属医院 | Deep learning-based gui testing method and system, and storage device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108197030B (en) | 2021-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108197030A (en) | Software interface based on deep learning tests cloud platform device and test method automatically | |
| US10360257B2 (en) | System and method for image annotation | |
| CN108874919B (en) | Automatic checking method for planned land, electronic equipment, storage medium and system | |
| CN109784381A (en) | Markup information processing method, device and electronic equipment | |
| CN108733764B (en) | Advertisement filtering rule generation method based on machine learning and advertisement filtering system | |
| US20170032189A1 (en) | Method, apparatus and computer-readable medium for image scene determination | |
| CN104750737B (en) | A kind of photograph album management method and device | |
| CN107911719B (en) | Video Dynamic recommendation device | |
| KR102135477B1 (en) | Method and system for image automatic classification | |
| CN110659581B (en) | Image processing method, device, equipment and storage medium | |
| CN104156356B (en) | Personalized Navigation page generation method and device | |
| CN110110075A (en) | Web page classification method, device and computer readable storage medium | |
| CN102799890A (en) | Image clustering method | |
| CN103098051A (en) | Search engine optmization assistant | |
| CN107644100A (en) | Information processing method, device and system and computer-readable recording medium | |
| CN106326242A (en) | Application pushing method and apparatus | |
| CN111191133B (en) | Service search processing method, device and equipment | |
| CN109902251A (en) | Scheme Choice method, apparatus, terminal and readable storage medium storing program for executing based on decision tree | |
| CN107179983A (en) | The method of calibration and device of ui testing result | |
| CN106603690A (en) | Data analysis device, data analysis processing system and data analysis method | |
| CN106126593B (en) | Earth's surface outdoor scene information generating method and device based on network opening photo resources | |
| CN108074033A (en) | Processing method, system, electronic equipment and the storage medium of achievement data | |
| CN109871770A (en) | Property ownership certificate recognition methods, device, equipment and storage medium | |
| CN110377218A (en) | Data processing method, device, computer equipment and storage medium | |
| CN106802797A (en) | The generation method and device of a kind of application programming interface |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |