CN115981622A - Visual programming device and method - Google Patents
Visual programming device and method Download PDFInfo
- Publication number
- CN115981622A CN115981622A CN202211632379.9A CN202211632379A CN115981622A CN 115981622 A CN115981622 A CN 115981622A CN 202211632379 A CN202211632379 A CN 202211632379A CN 115981622 A CN115981622 A CN 115981622A
- Authority
- CN
- China
- Prior art keywords
- library
- visual programming
- algorithm
- visual
- operator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000007 visual effect Effects 0.000 title claims abstract description 58
- 238000000034 method Methods 0.000 title claims abstract description 19
- 230000006870 function Effects 0.000 claims abstract description 34
- 238000011161 development Methods 0.000 claims abstract description 20
- 238000013461 design Methods 0.000 claims abstract description 13
- 238000010801 machine learning Methods 0.000 claims abstract description 9
- 230000018109 developmental process Effects 0.000 claims description 18
- 238000012545 processing Methods 0.000 claims description 5
- 230000000694 effects Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 abstract description 7
- ODKSFYDXXFIFQN-UHFFFAOYSA-M argininate Chemical compound [O-]C(=O)C(N)CCCNC(N)=N ODKSFYDXXFIFQN-UHFFFAOYSA-M 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000003607 modifier Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Landscapes
- Stored Programmes (AREA)
Abstract
The invention discloses a visual programming device and a method, wherein the visual programming device is an integrated development environment and comprises an interpreter, a standard library, an operator library, a graph editor library, an image display library and an auxiliary design tool, the standard library, the operator library, the graph editor library and a registered image display library are loaded through the interpreter, and an execution library function is explained; the standard library provides a sequence control function of a visual programming language and basic mathematical operation; the operator library provides operators of the machine learning visual algorithm; the diagram editor library provides a visual programming function of a language, and visual programming is carried out by utilizing a representation form of a diagram and a user dragging mode; the image display library is used for displaying the intermediate result of the algorithm; the aided design tool provides a tool for a user to develop visual algorithms. The invention designs an integrated development environment of the machine learning algorithm, finishes the development task of the machine vision algorithm by editing the form of the flow chart, does not need to depend on professional technology, and is convenient to use.
Description
Technical Field
The invention relates to the field of machine vision programming, in particular to a visual programming device and a visual programming method.
Background
The conventional executable program is a specialized programming work performed by a software development engineer using a certain programming language by means of a compiler. Such a program generation method often requires a very specialized technician, and a large amount of specialized work is also required, which is inefficient and inconvenient, and has a high dependency on the specialized technician.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a visual programming device and a visual programming method.
The technical scheme of the invention is as follows:
a visual programming device is an integrated development environment and comprises an interpreter, a standard library, an operator library, a graph editor library, an image display library and an auxiliary design tool, wherein the interpreter is used for loading the standard library, the operator library, the graph editor library and a registered image display library and interpreting and executing library functions;
the standard library provides a sequence control function of a visual programming language and basic mathematical operation;
the operator library provides operators of a machine learning vision algorithm;
the graph editor library provides a visual programming function of a language and carries out visual programming by utilizing a representation form of a graph and a user dragging mode;
the image display library is used for displaying the intermediate result of the algorithm;
the aided design tool provides a tool for a user to develop a visual algorithm.
The operators provided by the operator library comprise operations of image clipping, color transformation, filtering, threshold segmentation and machine learning related operators.
The auxiliary design tool provides tools including color picking, point position selection, rectangular area selection, rotating rectangle selection, circle selection, ellipse selection and polygon selection.
The visual programming method of the visual programming device comprises the following steps:
carrying out operations of project management, project creation, library file loading and visual algorithm creation in a development environment;
providing a preset standard library function;
providing a preset operator library function;
and providing a graph editor library, and loading the graph editor through the interpreter to create the visual programming device.
According to the standard library function, a user can create a control flow of a program and basic numerical operation by dragging a control function in the standard library function into a graph editing area.
And the operator library function can be connected with the control flow nodes by dragging the operator into the graph editing area by a user, so that the control of the operator execution sequence is completed.
The user can adjust the input and output parameters of the operator and run the program in the development environment to show the effect of the algorithm.
An interpreter built in the development environment can interpret and execute the algorithm, and the intermediate image processing result of the algorithm is displayed in real time through an image display library.
The user can freely combine operators in a line connection mode to develop the algorithm.
The visual programming device can be released to a third party for use, and the algorithm file and the interpreter can be released together so as to deploy the algorithm to the third party for use.
Compared with the prior art, the invention has the beneficial effects that: the invention designs an integrated development environment of the machine learning algorithm, finishes the development task of the machine vision algorithm by editing the form of the flow chart, does not need to depend on professional technology, and is convenient to use.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic structural diagram of a visual programming apparatus provided in the present invention;
FIG. 2 is a schematic diagram of a visual programming method provided by the present invention;
fig. 3 is a data structure diagram of a visual programming language provided by the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
Examples
Referring to fig. 1, the embodiment provides a visual programming apparatus, which is an integrated development environment and includes an interpreter, a standard library, an operator library, a graph editor library, an image display library and an auxiliary design tool, where the interpreter loads the standard library, the operator library, the graph editor library and a registered image display library, and interprets and executes library functions.
The standard library provides sequence control function of visual programming language and basic mathematics.
The operator library provides operators of the machine learning vision algorithm, including operations of image cropping, color transformation, filtering, threshold segmentation and machine learning related operators. The library utilizes the image processing functionality of OpenCV, as well as the deep learning engine functionality of onnx.
The diagram editor library provides a visual programming function of a language, and visual programming is carried out by utilizing a representation form of a diagram and a user dragging mode. The library utilizes the UI interaction functionality of Qt.
The image display library is used for displaying the intermediate result of the algorithm, so that a user can conveniently observe the execution process and debug the visual algorithm. The library utilizes the 2D image processing and interaction functions of vtk.
The aided design tool provides a tool for a user to develop visual algorithms, and comprises the steps of picking up colors, selecting point positions, selecting rectangular areas, selecting rotating rectangles, selecting circles, selecting ellipses, selecting polygons and the like.
The visual programming method of the visual programming device, as shown in fig. 2, includes:
performing operations such as project management, project creation, library file loading, visual algorithm creation and the like in a development environment;
providing a preset standard library function, and creating a control flow of a program and basic numerical operation by dragging a control function in the standard library function into a graph editing area by a user;
providing a preset operator library function, and connecting a user with a control flow node by dragging an operator into a graph editing area to complete control of operator execution sequence;
the graph editor library is provided, the graph editor is loaded through the interpreter, namely a visual programming device is created and can be issued to a third party for use, the algorithm file and the interpreter can be issued together so as to deploy the algorithm to the third party for use, and meanwhile, the self-compiling performance of the high-level programming language is achieved.
The user can adjust the input and output parameters of the operator and run the program in the development environment to show the effect of the algorithm. An interpreter built in the development environment can interpret and execute the algorithm, and the intermediate image processing result of the algorithm is displayed in real time through an image display library.
The user can freely combine operators in a line connection mode to develop the algorithm.
The invention designs a new visual programming language, called UL language, for the integrated development environment of the machine vision algorithm. The UL language includes a suite of kits: interpreter, designer, standard library. The integrated development environment utilizes UL language toolkits: an interpreter, a designer are integrated into the development environment, and a visual calculator library is added.
UL language architecture:
UL editor, which can create class libraries. Each class library is a file, and a class library may depend on other class libraries, which contain many classes.
Standard library files are class libraries that have been implemented, including system. Intet 32, system. Boolean, system. String, etc. Standard library functions are implemented in the C + + interpreter and also include control flows such as branches, loops, etc.
The custom library is a library created by the user himself with the UL editor, and generally the custom library needs to rely on a standard library, since a set of class libraries and functions of the control flow have been implemented in the standard library. Unless the user wants to implement the C + + interpreter himself, it must rely on a standard library.
The C + + interpreter is responsible for loading class library files and interpreting and executing functions in the class library.
The visual programming language provides a plurality of library files which are definition files of classes and functions of the visual programming language. Stored in a simple document format for easy access. Json format storage is currently used. As shown in fig. 3, is represented by the following structure:
bags (Package)
A package is a file that contains a number of classes. The packages have dependencies. Bag with bag interior declaring dependency of the bag
| Field(s) | Type (B) | Description of the invention |
| Name | string | Bag name |
| Dependences | string[] | Names of other packages on which the package depends |
| Types | Type{} | Classes contained within a package |
Class (Type)
Each class has a unique ID, represented by a GUID. GUID immutable
| Field(s) | Type (B) | Description of the invention |
| ID | string | guid |
| Name | string | Class name |
| Namespace | string | Namespace of classes |
| Parent | string | Parent ID |
| Modifier | string | Modifier symbol |
| Members | Member[] | Member list contained in class |
Member (Member)
Each member having a unique ID
Graph (Graph) Graph is an implementation of a function
| Field(s) | Types of | Description of the invention |
| Args | Arg[] | Parameter list |
| Outputs | Arg[] | Output list |
| Nodes | Node[] | Including a node |
Node (Node)
| Field(s) | Type (B) | Description of the invention |
| NodeID | string | Node ID, which is unique in a graph |
| MethodID | string | Invoked method ID |
| Name | string | Node display name |
| Inputs | Pin[] | Input port list |
| Outputs | Pin[] | Output port list |
| Comment | string | Note that |
Port (Pin)
| Field(s) | Types of | Description of the invention |
| TypeID | string | Port type ID |
| Name | string | Port name |
| Value | string | Port input value (constant) |
| Links | string[] | A connection list of ports. One port can have N links |
Parameter (Arg)
| Field(s) | Type (B) | Description of the preferred embodiment |
| TypeID | string | Parameter type ID |
| Name | string | Parameter name |
| DefaultValue | string | Default value |
The present invention is not limited to the above preferred embodiments, and any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A visual programming apparatus, characterized by: the method is an integrated development environment, which comprises an interpreter, a standard library, an operator library, a graph editor library, an image display library and an auxiliary design tool, wherein the standard library, the operator library, the graph editor library and a registered image display library are loaded through the interpreter, and the function of an execution library is interpreted;
the standard library provides a sequence control function of a visual programming language and basic mathematical operation;
the operator library provides operators of a machine learning vision algorithm;
the graph editor library provides a visual programming function of a language, and visual programming is performed by using a representation form of a graph and a user dragging mode;
the image display library is used for displaying intermediate results of the algorithm;
the aided design tool provides a tool for a user to develop a visual algorithm.
2. A visual programming apparatus according to claim 1, wherein: the operators provided by the operator library comprise operations of image clipping, color transformation, filtering, threshold segmentation and machine learning related operators.
3. A visual programming apparatus according to claim 1, wherein: the tools provided by the aided design tool comprise color picking, point position selection, rectangular area selection, rotating rectangle selection, circle selection, ellipse selection and polygon selection.
4. A visual programming method of the visual programming device according to any one of claims 1 to 3, comprising:
carrying out operations of project management, project creation, library file loading and visual algorithm creation in a development environment;
providing a preset standard library function;
providing a preset operator library function;
and providing a graph editor library, and loading the graph editor through an interpreter to create the visual programming device.
5. A visual programming method according to claim 4, wherein: according to the standard library function, a user can create a control flow of a program and basic numerical operation by dragging a control function in the standard library function into a graph editing area.
6. A visual programming method according to claim 4, wherein: according to the operator library function, a user can drag operators into a graph editing area to be connected with control flow nodes, and control of operator execution sequence is completed.
7. A visual programming method according to claim 4, wherein: the user can adjust the input and output parameters of the operator and run the program in the development environment to show the effect of the algorithm.
8. A visual programming method according to claim 7, wherein: the interpreter built in the development environment can interpret and execute the algorithm, and the intermediate image processing result of the algorithm is displayed in real time through the image display library.
9. A visual programming method according to claim 4, wherein: the user can freely combine operators in a line connection mode to develop the algorithm.
10. A visual programming method according to claim 4, wherein: the visual programming device can be released to a third party for use, and the algorithm file and the interpreter can be released together so as to deploy the algorithm to the third party for use.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211632379.9A CN115981622A (en) | 2022-12-19 | 2022-12-19 | Visual programming device and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211632379.9A CN115981622A (en) | 2022-12-19 | 2022-12-19 | Visual programming device and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115981622A true CN115981622A (en) | 2023-04-18 |
Family
ID=85962047
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211632379.9A Pending CN115981622A (en) | 2022-12-19 | 2022-12-19 | Visual programming device and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115981622A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106293748A (en) * | 2016-08-15 | 2017-01-04 | 苏州博众精工科技有限公司 | A kind of graphic interactive Vision Builder for Automated Inspection and method of work thereof |
| CN106708510A (en) * | 2016-11-28 | 2017-05-24 | 华南理工大学 | Graphical configuration-based industrial intelligent camera software system structure and method |
| CN108279890A (en) * | 2017-01-06 | 2018-07-13 | 阿里巴巴集团控股有限公司 | Component is issued and the constructing method of component based on graphical machine learning algorithm platform, graphical machine learning algorithm platform |
| CN109634596A (en) * | 2018-12-11 | 2019-04-16 | 清华大学 | Visual programming tools based on flow chart |
| WO2020077326A1 (en) * | 2018-10-12 | 2020-04-16 | Code Walker L.L.C. | Annotated method for computer code mapping and visualization |
| CN111240662A (en) * | 2020-01-16 | 2020-06-05 | 同方知网(北京)技术有限公司 | Spark machine learning system and learning method based on task visual dragging |
-
2022
- 2022-12-19 CN CN202211632379.9A patent/CN115981622A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106293748A (en) * | 2016-08-15 | 2017-01-04 | 苏州博众精工科技有限公司 | A kind of graphic interactive Vision Builder for Automated Inspection and method of work thereof |
| CN106708510A (en) * | 2016-11-28 | 2017-05-24 | 华南理工大学 | Graphical configuration-based industrial intelligent camera software system structure and method |
| CN108279890A (en) * | 2017-01-06 | 2018-07-13 | 阿里巴巴集团控股有限公司 | Component is issued and the constructing method of component based on graphical machine learning algorithm platform, graphical machine learning algorithm platform |
| WO2020077326A1 (en) * | 2018-10-12 | 2020-04-16 | Code Walker L.L.C. | Annotated method for computer code mapping and visualization |
| CN109634596A (en) * | 2018-12-11 | 2019-04-16 | 清华大学 | Visual programming tools based on flow chart |
| CN111240662A (en) * | 2020-01-16 | 2020-06-05 | 同方知网(北京)技术有限公司 | Spark machine learning system and learning method based on task visual dragging |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101169702B1 (en) | Method of programming a processing system | |
| US7409679B2 (en) | System and method for computer-aided graph-based dependency analysis | |
| Kreuseler et al. | A history mechanism for visual data mining | |
| North et al. | Applications of graph visualization | |
| EP1849064A1 (en) | Applying coding standards in graphical programming environments | |
| CA2544392C (en) | Method and system for reversible design tree transformations | |
| CN112506500A (en) | Visual control creating method and device | |
| CN110673842A (en) | Visual programming method and system, storage medium and equipment | |
| Kasyanov | Methods and tools for structural information visualization | |
| US20160018952A1 (en) | Multi-Format Editors | |
| CN115981622A (en) | Visual programming device and method | |
| CN115934069A (en) | Visual program compiling method and device, electronic equipment and storage medium | |
| US7835895B1 (en) | Signal inheritance in a graphical model environment | |
| Koskinen et al. | Model processing tools in UML | |
| Bäsken et al. | GeoWin a generic tool for interactive visualization of geometric algorithms | |
| Eckstein et al. | Interactive data inspection and program development for computer vision | |
| Barzdins et al. | An MDE-based graphical tool building framework | |
| Almeida et al. | A tool for programming with interaction nets | |
| Eckstein et al. | Architecture for computer vision application development within the HORUS system | |
| Grundy et al. | A visual programming environment for object-oriented languages | |
| Backlund et al. | Generation of graphic language-oriented design environments | |
| Jäger et al. | UPGRADE: A Framework for Building Graph-Based Software Engineering Tools | |
| Kasyanov | Hierarchical graph models and information visualizaion | |
| Jäger | UPGRADE—A Framework for Graph-Based Visual Applications | |
| Korhonen et al. | Design pattern for algorithm animation and simulation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |