CN112099773A - Method for realizing desktop application development - Google Patents
Method for realizing desktop application development Download PDFInfo
- Publication number
- CN112099773A CN112099773A CN202010958965.7A CN202010958965A CN112099773A CN 112099773 A CN112099773 A CN 112099773A CN 202010958965 A CN202010958965 A CN 202010958965A CN 112099773 A CN112099773 A CN 112099773A
- Authority
- CN
- China
- Prior art keywords
- desktop application
- application development
- areas
- design drawing
- electron
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 238000011161 development Methods 0.000 title claims abstract description 25
- 238000013461 design Methods 0.000 claims abstract description 24
- 238000013527 convolutional neural network Methods 0.000 claims description 12
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 238000003062 neural network model Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 4
- 102100032202 Cornulin Human genes 0.000 claims description 3
- 101000920981 Homo sapiens Cornulin Proteins 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 abstract description 3
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 230000015572 biosynthetic process Effects 0.000 abstract description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/38—Creation or generation of source code for implementing user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/31—Programming languages or programming paradigms
- G06F8/315—Object-oriented languages
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/30—Creation or generation of source code
- G06F8/35—Creation or generation of source code model driven
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the field of artificial intelligence deep learning, and particularly provides a method for realizing desktop application development. Compared with the prior art, the method and the system can reduce code writing work and a large amount of time and energy for project development in the process from the interface design drawing to the formation of a proper desktop application, and have good popularization value.
Description
Technical Field
The invention relates to the field of artificial intelligence deep learning, and particularly provides a method for realizing desktop application development.
Background
The convolutional neural network is a feedforward neural network which comprises convolutional calculation and has a deep structure, and is one of representative algorithms of deep learning. Convolutional neural networks have a characteristic learning ability, and can perform translation invariant classification on input information according to a hierarchical structure thereof, and are also called "translation invariant artificial neural networks".
Electron is a cross-platform desktop application development tool published by GitHub, supports Web technology to develop desktop applications, is developed based on C + +, the GUI core comes from Chrome, and the JavaScript engine uses v 8.
In actual desktop application development, a large amount of code writing work is needed from interface design drawing to appropriate desktop application formation, and a large amount of time and energy of project development are consumed.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a method for realizing desktop application development with strong practicability.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a method for realizing desktop application development comprises the steps of firstly inputting an interface design drawing, identifying and segmenting an image through a convolutional neural network, then generating a program by utilizing a code written by an Electron development tool and a JavaScript language, packaging the program into an exe file, and finally generating a desktop application program.
Further, the format of the input interface design drawing is jpg,. png,. bmp,. jpeg.
Further, the areas of the interface design drawing are classified through a first neural network model, the interface design drawing is divided into a plurality of areas, and the position information of the areas is determined.
Further, after the position information of a plurality of areas is obtained through the first neural network, the areas are respectively identified through the second neural network model, the text information of the areas is extracted, the format of the text information is json format, and the components of the interface design drawing are determined.
Preferably, the first neural network and the second neural network use an OpenCV technique, use vgg16 network to detect text direction, use CTPN (CNN + RNN) network to detect text regions, and use CRNN (CNN + LSTM + CTC) to perform text recognition.
Further, generating character string information of the components and the position information of the plurality of areas, and generating Electron codes of the functions of the interface design drawing based on the character string information.
Preferably, when generating the Electron code, node. js is installed first, and the Electron is installed globally using the version of v10 or more.
Furthermore, reading the Json format file by using the Javas language, and analyzing the content in the Json format file.
Further, by utilizing an Electron tool, according to the data of the data structure, generating a corresponding element on the position information in each text message, and circulating until all the messages are generated.
Compared with the prior art, the method for realizing desktop application development has the following outstanding beneficial effects:
the invention can reduce the code writing work and a great deal of time and energy for project development from the interface design drawing to the process of forming a proper desktop application. By the method, the continuous interface design drawing can be automatically generated into the desktop application, and the method is convenient and quick.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flow chart diagram illustrating a method for implementing desktop application development.
Detailed Description
The present invention will be described in further detail with reference to specific embodiments in order to better understand the technical solutions of the present invention. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
A preferred embodiment is given below:
referring to fig. 1, in the embodiment, a desktop application development method is implemented,
firstly, inputting an interface design drawing, and identifying and segmenting an image through a convolutional neural network:
the CNN model in the implementation is generated by training a large number of pictures with interface design, so that not only can the text content be extracted, but also the basic information of each element in the pictures can be extracted. For example, font size, font style, font color, background color, component name, component design style, specific location of the page, and the page size ratio of the image.
Inputting an interface design drawing or a picture group in a JPG/PN/BMP/JPEG format, classifying the area of the interface design drawing through a first neural network model, dividing the interface design drawing into a plurality of areas, and determining the position information of the plurality of areas.
For example, a layout analysis is performed to analyze each element in the picture, such as a title, text, picture, icon, component, etc.
And (3) cutting characters (pictures), namely cutting each element analyzed out of the layout into a single module.
Respectively identifying a plurality of areas by the information of each module through a second neural network model, and identifying information such as layout, color, icons, character content, font size, font style, coordinates of the position point and the like corresponding to the module for marking; if the module is a picture, only the position information of the module is recorded; if the module is a component, the same component, e.g., radio box, multiple box, slider, switch, etc., needs to be found in the VUE component library.
The module information is recorded in json format and the components of the interface plan are determined.
Then, a program is generated by utilizing a code written by an Electron development tool and a JavaScript language, and the program is packaged into an exe file:
and reading the Json file by using the Javas language, and analyzing the content in the Json file.
For automatic generation of desktop application code, we need to install node.js first, preferably using the version above v10, and then install Electron globally.
And generating corresponding elements on the position information in each module information according to the data of the data structure by using an Electron tool, and circulating until all information is generated.
Js, reading an entry file in package, here exemplified as main; creating a rendering process in a main process; reading the layout and style information of the application page; IPC is used to perform tasks and obtain information in the host process.
Creating a main process file as a main process file:
introducing an electron module, creating an electron reference, creating a window reference, declaring a main window to be opened, setting the size of the opened window, loading the page, monitoring a closing event, and setting the main window to null.
And reading the json file transmitted from the first stage, and converting the json file into a corresponding interface of the desktop application. After an interface is provided, after a render folder is independently created, an index.js file is created in the file, then the index.html page is introduced first, then codes in the index.js are written, and the codes are as follows:
}var fs＝require('fs')；
window.onload＝function(){
var btn＝this.document.querySelector('#btn')
var mybaby＝this.document.querySelector('#mybaby')
btn.onclick＝function(){
fs.readFile('xiaojiejie.txt',(err,data)＝>{
mybaby.innerHTML＝data
})
}
}
creating a demo2.html file, then quickly generating a basic structure of html, writing a button, and introducing a rendered js page. When a menu is written in an Electron, a template needs to be established first. And opening the main process main.js file, and directly adding codes in the ready life cycle to realize the self-defined menu.
After the application is developed by using the Electron, the application is packaged by using the Electron-package. And generating a final desktop application, exe file.
In order to generate a model specific to an interface design drawing in the system, an OpenCV technology is utilized, vgg16 networks are used for detecting text directions, CTPN (CNN + RNN) networks are used for detecting text regions, and CRNN (CNN + LSTM + CTC) is utilized for text recognition. Firstly, training each network mentioned above through a large number of pictures containing PPT pages and labeled information, and finally generating an identification model of an interface design drawing specially aiming at the system.
The code recording the module information in json format is:
a new main.js file, the code of the main process file is as follows:
var electron＝require('electron')
var app＝electron.app
var BrowserWindow＝electron.BrowserWindow；
var mainWindow＝null；
app.on('ready',()＝>{
mainWindow＝new BrowserWindow({width:400,height:400})
mainWindow.loadFile('index.html')
mainWindow.on('closed',()＝>{
mainWindow＝null
})
})
the above embodiments are only specific cases of the present invention, and the protection scope of the present invention includes but is not limited to the above embodiments, and any suitable changes or substitutions that are consistent with the claims of the desktop application development method of the present invention and are made by those skilled in the art should fall within the protection scope of the present invention.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (9)
1. A method for realizing desktop application development is characterized in that an interface design drawing is input, images are recognized and segmented through a convolutional neural network, then a program is generated by utilizing a code written by an Electron development tool and a JavaScript language, the program is packaged into an exe file, and finally a desktop application program is generated.
2. The method for realizing desktop application development according to claim 1, wherein the format of the input interface design drawing is jpg,. png,. bmp,. jpeg.
3. The method of claim 2, wherein the areas of the interface design drawing are classified by the first neural network model, the interface design drawing is divided into a plurality of areas, and the position information of the areas is determined.
4. The method for realizing desktop application development according to claim 3, wherein after the position information of a plurality of areas is obtained through the first neural network, the areas are respectively identified through the second neural network model, the text information of the areas is extracted, the format of the text information is json format, and the components of the interface design drawing are determined.
5. The method for realizing desktop application development according to claim 4,
the first neural network and the second neural network utilize OpenCV technology, vgg16 networks are used for detecting text directions, CTPN (CNN + RNN) networks are used for detecting text regions, and CRNN (CNN + LSTM + CTC) is used for text recognition.
6. The method for realizing desktop application development according to claim 3 or 4, wherein string characters are generated from the components and the position information of the plurality of areas, and Electron codes of the functions of the interface design drawing are generated based on the string characters.
7. The method for realizing desktop application development according to claim 6, wherein when generating the Electron code, node.js is installed first, and the Electron is installed globally by using the version above v 10.
8. The method of claim 7, wherein the Json-formatted file is read by Java language, and the content of the Json-formatted file is parsed.
9. The method for realizing desktop application development according to claim 8, wherein an Electron tool is used to generate a corresponding element with position information in each text message according to data of the data structure, and the process is repeated until all messages are generated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010958965.7A CN112099773A (en) | 2020-09-14 | 2020-09-14 | Method for realizing desktop application development |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010958965.7A CN112099773A (en) | 2020-09-14 | 2020-09-14 | Method for realizing desktop application development |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112099773A true CN112099773A (en) | 2020-12-18 |
Family
ID=73751634
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010958965.7A Pending CN112099773A (en) | 2020-09-14 | 2020-09-14 | Method for realizing desktop application development |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112099773A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116361279A (en) * | 2023-06-02 | 2023-06-30 | 北京麟卓信息科技有限公司 | Cross-platform artificial intelligent data labeling method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109783094A (en) * | 2018-12-15 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Front end page generation method, device, computer equipment and storage medium |
| US20190347059A1 (en) * | 2018-05-14 | 2019-11-14 | Schneider Electric Industries Sas | Computer-implemented method and system for generating a mobile application from a desktop application |
| CN110990010A (en) * | 2019-12-03 | 2020-04-10 | 锐捷网络股份有限公司 | Software interface code generation method and device |
| CN111414165A (en) * | 2019-01-06 | 2020-07-14 | 阿里巴巴集团控股有限公司 | Interface code generation method and equipment |
-
2020
- 2020-09-14 CN CN202010958965.7A patent/CN112099773A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190347059A1 (en) * | 2018-05-14 | 2019-11-14 | Schneider Electric Industries Sas | Computer-implemented method and system for generating a mobile application from a desktop application |
| CN109783094A (en) * | 2018-12-15 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Front end page generation method, device, computer equipment and storage medium |
| CN111414165A (en) * | 2019-01-06 | 2020-07-14 | 阿里巴巴集团控股有限公司 | Interface code generation method and equipment |
| CN110990010A (en) * | 2019-12-03 | 2020-04-10 | 锐捷网络股份有限公司 | Software interface code generation method and device |
Non-Patent Citations (3)
| Title |
|---|
| VINH-TIEP NGUYEN; MINH-TRIET TRAN; ANH-DUC DUONG;: "Picture-Driven User Interface Development for Applications on Multi-platforms", 16TH INTERNATIONAL CONFERENCE ON HUMAN-COMPUTER INTERACTION (HCI), 27 June 2014 (2014-06-27) * |
| 宋小鹏; 王斌武;: "JavaScript编程在新工科能动类专业课程体系中的应用", vol. 978, 30 June 2019, 电子科技大学出版社, pages: 29 - 32 * |
| 王江峰: "基于FPGA的CNN自动代码生成设计与实现", 中国优秀硕士学位论文全文数据库 (信息科技辑), no. 12, 15 December 2018 (2018-12-15) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116361279A (en) * | 2023-06-02 | 2023-06-30 | 北京麟卓信息科技有限公司 | Cross-platform artificial intelligent data labeling method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11954139B2 (en) | Deep document processing with self-supervised learning | |
| CN112149399B (en) | Table information extraction method, device, equipment and medium based on RPA and AI | |
| JP6838209B1 (en) | Document image analyzer, document image analysis method and program | |
| CN113221181B (en) | Table type information extraction system and method with privacy protection function | |
| CN111832382A (en) | Optical character recognition error correction based on visual and textual content | |
| US11494588B2 (en) | Ground truth generation for image segmentation | |
| CN115210747B (en) | Method and system for digital image processing | |
| CN114596566B (en) | Text recognition method and related device | |
| Den Hartog et al. | Knowledge-based interpretation of utility maps | |
| Akinbade et al. | An adaptive thresholding algorithm-based optical character recognition system for information extraction in complex images | |
| Altinbas et al. | GUI element detection from mobile UI images using YOLOv5 | |
| US20220076109A1 (en) | System for contextual and positional parameterized record building | |
| CN112099773A (en) | Method for realizing desktop application development | |
| CN112114803B (en) | Method, equipment and medium for generating front end code of UI (user interface) based on deep learning | |
| Sen et al. | Design of novel feature vector for recognition of online handwritten Bangla basic characters | |
| Vu et al. | Revising FUNSD dataset for key-value detection in document images | |
| CN117725458A (en) | Method and device for obtaining threat information sample data generation model | |
| WO2023093850A1 (en) | Component identification method and apparatus, electronic device, and storage medium | |
| Tran et al. | A deep learning-based system for document layout analysis | |
| Ferreira et al. | Automatically Generating Websites from Hand-drawn Mockups | |
| CN115373658A (en) | Method and device for automatically generating front-end code based on Web picture | |
| Kaur et al. | Performance evaluation of various feature selection techniques for offline handwritten Gurumukhi place name recognition | |
| Batomalaque et al. | Image to text conversion technique for anti-plagiarism system | |
| Madan et al. | Parsing and summarizing infographics with synthetically trained icon detection | |
| WO2022085021A1 (en) | Systems and methods for cognitive information mining |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |