WO2021035552A1 - 支持基于神经元块图形编程的系统、方法及存储介质 - Google Patents

支持基于神经元块图形编程的系统、方法及存储介质 Download PDF

Info

Publication number
WO2021035552A1
WO2021035552A1 PCT/CN2019/102903 CN2019102903W WO2021035552A1 WO 2021035552 A1 WO2021035552 A1 WO 2021035552A1 CN 2019102903 W CN2019102903 W CN 2019102903W WO 2021035552 A1 WO2021035552 A1 WO 2021035552A1
Authority
WO
WIPO (PCT)
Prior art keywords
programming
neuron
graphics
neuron block
block
Prior art date
Application number
PCT/CN2019/102903
Other languages
English (en)
French (fr)
Inventor
唐立三
胡飞凰
唐秉超
Original Assignee
西门子股份公司
西门子（中国）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 西门子股份公司, 西门子（中国）有限公司 filed Critical 西门子股份公司
Priority to US17/637,541 priority Critical patent/US20220283787A1/en
Priority to EP19943717.9A priority patent/EP4006662A4/en
Priority to PCT/CN2019/102903 priority patent/WO2021035552A1/zh
Priority to CN201980097442.3A priority patent/CN113966494A/zh
Publication of WO2021035552A1 publication Critical patent/WO2021035552A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
    • G05B19/05—Programmable logic controllers, e.g. simulating logic interconnections of signals according to ladder diagrams or function charts
    • G05B19/056—Programming the PLC
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/30—Creation or generation of source code
    • G06F8/34—Graphical or visual programming
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/10—Interfaces, programming languages or software development kits, e.g. for simulating neural networks

Definitions

  • the present invention relates to the industrial field, in particular to a system, a method and a storage medium supporting graphical programming based on neuron blocks.
  • a system supporting graphical programming based on neuron blocks is proposed, and on the other hand, a method and computer-readable storage medium supporting graphical programming based on neuron blocks are proposed to facilitate It enables users who are accustomed to coding in industrial fields such as PLC to realize the coding of AI applications without barriers.
  • a system for supporting graphical programming based on neuron blocks proposed in the embodiment of the present invention includes: a neuron block code library that stores program codes and corresponding description files of each neuron block corresponding to each functional element in artificial intelligence; programming;
  • the element library which stores the program codes and corresponding description files of each programming element used to implement various judgment logic and control logic; a graphical management module, which is used to extract the description of each neuron block from the neuron block code library File, generate a corresponding neuron block graphic according to the description file of each neuron block, extract the description file of each programming element from the programming element library, and generate a corresponding programming element graphic according to the description file of each programming element
  • the neuron block graphics and the programming element graphics are in compliance with a set industry standard;
  • a graphical modeling interface is used to present the neuron block graphics and the programming generated by the graphical management module Element graphics, receiving the artificial intelligence graphical model established by the user based on the industrial field standards by selecting the corresponding neuron block graphics and/or programming element graphics; the code
  • the industrial field standard is the IEC61131-3 standard.
  • the program code of each neuron block includes: a plurality of program codes corresponding to different programming languages and different artificial intelligence frameworks for each neuron block;
  • the program code of each programming element includes: A plurality of program codes corresponding to different programming languages and different artificial intelligence frameworks for each programming element;
  • the code converter is based on the neuron block graphics and/or programming element graphics selected in the artificial intelligence graphical model, and the obtained The target programming language and target artificial intelligence framework corresponding to the target system configured by the user are extracted from the neuron block code library and the programming element library.
  • the different programming languages and different artificial intelligence frameworks include: TensorFlow framework and PyTorch framework in Python language, and Caffe framework in C++ language.
  • the neuron block includes: one or more of a two-dimensional convolution function block, a two-dimensional maximum pool block, a flattening block from two-dimensional to one-dimensional, and an integrated conversion block from more to less.
  • the programming element includes: a programming element for performing conditional judgment logic, a programming element for performing loop control logic, a programming element for performing AND or non-control logic, and a programming element for enabling and disabling logic.
  • a programming element for performing conditional judgment logic a programming element for performing loop control logic
  • a programming element for performing AND or non-control logic a programming element for performing AND or non-control logic
  • a programming element for enabling and disabling logic One or more of the programming elements that enable the control logic.
  • the data types used in the program code of each neuron block include: basic data types that conform to the industrial field standards; and corresponding artificial intelligence frameworks including tuples, tensors, and nulls.
  • the new data type includes: basic data types that conform to the industrial field standards; and corresponding artificial intelligence frameworks including tuples, tensors, and nulls. The new data type.
  • the transcoder is further used to receive the artificial intelligence model coding program input by the user, parse the coding program line by line, and obtain keywords that can represent neuron blocks or programming elements. Keywords determine the corresponding neuron block or programming element, and determine the corresponding neuron block graphic or programming element graphic and its interface data and neighbors according to the connection relationship and interface data between adjacent neuron blocks and/or programming elements The connection relationship between the graphics generates a corresponding artificial intelligence graphical model, and the artificial intelligence graphical model is provided to the graphical modeling interface for display.
  • a method for supporting graphical programming based on neuron blocks proposed in the embodiment of the present invention includes: pre-stored in a neuron block code library the program codes and corresponding descriptions of each neuron block corresponding to each functional element in artificial intelligence File, a programming element library stores the program codes and corresponding description files of each programming element used to implement various judgment logic and control logic; when the system is started, each neuron block is extracted from the neuron block code library According to the description file of each neuron block, the corresponding neuron block graphic is generated, the description file of each programming element is extracted from the programming element library, and the corresponding programming is generated according to the description file of each programming element Element graphics; both the neuron block graphics and the programming element graphics conform to a set industry standard; the neuron block graphics and the programming element graphics are presented in a graphical modeling interface, and the user is received An artificial intelligence graphical model established by selecting corresponding neuron block graphics and/or programming element graphics based on the industrial field standards; according to the neuron block graphics and/or programming element graphics selected in the artificial intelligence graphical
  • the program code of each neuron block includes: a plurality of program codes corresponding to different programming languages and different artificial intelligence frameworks for each neuron block;
  • the program code of each programming element includes: A plurality of program codes corresponding to different programming languages and different artificial intelligence frameworks for each programming element;
  • the neuron block graphics and/or programming element graphics selected according to the artificial intelligence graphical model are selected from the neuron block code library
  • the corresponding program code extracted from the programming element library is: according to the neuron block graphics and/or programming element graphics selected in the artificial intelligence graphical model, and the obtained target corresponding to the target system configured by the user
  • the programming language and the target artificial intelligence framework extract corresponding program codes from the neuron block code library and/or the programming element library.
  • the programming element includes: a programming element for performing conditional judgment logic, a programming element for performing loop control logic, a programming element for performing AND or non-control logic, and a programming element for enabling and disabling logic.
  • a programming element for performing conditional judgment logic a programming element for performing loop control logic
  • a programming element for performing AND or non-control logic a programming element for performing AND or non-control logic
  • a programming element for enabling and disabling logic One or more of the programming elements that enable the control logic.
  • the data types used in the program code of each neuron block include: basic data types that conform to the industrial field standards; and corresponding artificial intelligence frameworks including tuples, tensors, and nulls.
  • the new data type includes: basic data types that conform to the industrial field standards; and corresponding artificial intelligence frameworks including tuples, tensors, and nulls. The new data type.
  • the method further includes: receiving an artificial intelligence model coding program input by a user; parsing the coding program line by line; acquiring keywords that can represent neuron blocks or programming elements; determining according to the keywords Corresponding neuron block or programming element; according to the connection relationship and interface data between adjacent neuron blocks and/or programming elements, determine the corresponding neuron block graphics or programming element graphics and their interface data and the adjacent graphics And generate the corresponding artificial intelligence graphical model; display the artificial intelligence graphical model through the graphical modeling interface.
  • Another system for supporting neuron block-based graphics programming proposed in the embodiment of the present invention includes: at least one memory, at least one processor, at least one database, and at least one display, wherein: the at least one memory is used to store a computer Program; the at least one database is used to provide a neuron block code library and a programming element library; the neuron block code library is used to store the program codes and corresponding neuron blocks corresponding to the functional elements in the artificial intelligence Description file; the programming element library is used to store the program codes and corresponding description files of each programming element used to implement various judgment logic and control logic; the display is used to provide a graphical modeling interface; the at least A processor is used to call the computer program stored in the at least one memory to execute the method for supporting neuron block-based graphic programming in any of the above-mentioned embodiments.
  • a computer-readable storage medium proposed in an embodiment of the present invention has a computer program stored thereon; the computer program can be executed by a processor and realize the support for neuron block-based graphics programming in any of the above-mentioned embodiments Methods.
  • each neuron block is set for each functional element in AI, and three realizations of program code, description file and graphical representation are completed for each neuron block.
  • its graphical representation is made to conform to a set industrial field standard such as the IEC61131-3 standard, thereby providing users with a programming mode that follows the same programming mode as the PLC development software of the IEC61131-3 standard, thereby making them accustomed to industrial fields such as PLC
  • the coding user can implement the model building of the AI application without barriers, and then the system performs code conversion based on the model building, and generates a coding program that meets the operating environment of the target system, thereby completing barrier-free AI programming.
  • the final coding program can be converted into a coding program that meets the programming language and AI framework of the target system, which improves the flexibility and flexibility of programming. Versatility.
  • the compatibility with the user's existing coding program can be realized.
  • Fig. 1 is an exemplary structure diagram of a system supporting graphical programming based on neuron blocks in an embodiment of the present invention.
  • Fig. 2 is an example of a typical neuron block graph in an example of the present invention.
  • Fig. 3 is an example of a neuron block graph of a two-dimensional convolution function in an embodiment of the present invention.
  • Fig. 4 is a simplified display example of Fig. 3 in an example of the present invention.
  • Fig. 5A is an example of a neuron block graph of a two-dimensional largest pool block in an example of the present invention.
  • Fig. 5B is a simplified display example of Fig. 5A in an example of the present invention.
  • Fig. 6A is an artificial intelligence model built by a user in an example of the present invention.
  • Fig. 6B is a simplified display example of Fig. 6A in an example.
  • Fig. 7 is a very famous convolutional neural network (CNN) model AlexNet built by a user in an example of the present invention.
  • CNN convolutional neural network
  • Fig. 8 is an exemplary flowchart of a method for supporting graphical programming based on neuron blocks in an embodiment of the present invention.
  • Fig. 9 is an exemplary structure diagram of another system supporting graphical programming based on neuron blocks in an embodiment of the present invention.
  • a neuron block can be set corresponding to each functional element in AI.
  • the neuron block can include: two-dimensional convolution function block, two-dimensional maximum pool block, flattened block from two-dimensional to one-dimensional, and from many to Few fully connected blocks, etc.
  • the program code, description file and graphical representation of three objects can be realized.
  • the neuron block can be a high-level abstraction that follows the function block specification in IEC61131-3, that is, it can be defined as a function in IEC61131-3.
  • the expansion of the block to maintain the same programming mode as the PLC. It can be built in a separate development environment or integrated into the PLC development environment. It can be implemented according to the actual situation, and there is no restriction on it here. As long as the automation system users can form artificial intelligence models without learning other programming languages and being familiar with other integrated development environments (IDE). After that, the system can convert the AI model built by the user based on the graphically represented neural block into executable target program code.
  • IDE integrated development environments
  • Fig. 1 is an exemplary structure diagram of a system supporting graphical programming based on neuron blocks in an embodiment of the present invention.
  • the system may include: a neuron block code library 10, a programming element library 20, a graphical management module 30, a graphical modeling interface 40 and a code converter 50.
  • the neuron block code library 10 is used to store and manage the program codes and corresponding description files of each neuron block corresponding to each functional element in the AI.
  • the data types involved in each neuron block may include: basic data types that comply with the industrial field standards; and new data types including tuples, tensors, and voids corresponding to the artificial intelligence framework.
  • table 1 shows the data types involved in each neuron block in the embodiment of the present invention.
  • the "a” in the “remarks” column represents the basic data type compatible with the IEC61131-3 standard
  • the "b” in the “remarks” column represents the new data type corresponding to the artificial intelligence framework.
  • Tuples are collections of objects.
  • the object is another data type defined in Table 1.
  • the format of the tuple can be defined as (object[, object]).
  • Tensor is a keyword defined in the field of artificial intelligence, which represents the generalization of vectors and matrices. In fact, it is a special multidimensional array.
  • the grammatical definition format of the description file of each neuron block can follow the Bacchus Normal Form (BNF), which is often used in program language definitions including the IEC61131-3 specification.
  • BNF Bacchus Normal Form
  • the neuron block is identified by the keyword "NEURAL_BLOCK", and the keyword "'END_NEURAL_BLOCK'” is used to indicate the end of a neuron block.
  • Each neuron block has its own name, and its name can be an identifier. No matter its program code, description file, or its graphical representation, its name is consistent, so the corresponding search can be carried out based on the name as a keyword.
  • the above two-dimensional convolution function block can be named Conv2d
  • the two-dimensional maximum pooling block can be named Maxpooling2d
  • the flattened block from two-dimensional to one-dimensional can be named Flatten
  • the fully connected block from more to less can be named Dense.
  • a neuron block includes three types of variables: input type, output type, and input-output type.
  • VAR_INPUT used to identify all input type variables displayed on the left side of the neuron block graph.
  • the declaration statement of the input type variable can be composed of several fields: variable name, variable type, whether it is visible to the user, whether it is necessary, default value, etc.
  • the supported variable types can include all the variable types listed in Table 1.
  • B.VAR_OUTPUT Used to identify all output type variables displayed on the right side of the neuron block graph.
  • the output type variable declaration statement consists of several fields: output variable name, output variable data type, whether it is visible to the user, etc.
  • RETAIN and NON_RETAINb used to identify whether to retain data when the power is off.
  • C.VAR_IN_OUT Used to identify input and output variables. These variables are displayed on the left and right sides of the neuron block graph.
  • the input and output type variable declaration statement consists of several fields: input and output variable name, input and output variable data type, whether it is visible to the user, whether it is necessary, default value, etc.
  • the input and output variable name is an identifier, and the identifier is composed of characters, numbers, and underscores, and the first character must be a letter or underscore. Supports 26 letters that are case-sensitive, and supports 10 numbers from 0 to 9.
  • the keyword VISIBLE defines whether the variable modified by this keyword is displayed in the folding mode to simplify the visual part of the graph neuron block.
  • the keyword REQUIRED defines whether the user is required to give the input value of the variable. For variables that are not modified with this keyword, if the user does not provide input, the built-in default value is used as the input value.
  • the description file of the two-dimensional convolution function used in the TensorFlow AI framework of Python may include: using NEURAL_BLOCK conv2d to indicate that the neuron block is named conv2d; using VAR_INPUT to identify the beginning of the input type variable, and the input variable Including: input enable, input data, number of convolution kernels, convolution kernel size, step size, filling method, input data format, expansion ratio, activation function, use bias, weight initialization method, bias initialization method, application
  • the regular term on the weight, the regular term imposed on the bias vector, the regular term imposed on the output, the constraint term imposed on the weight and the constraint term imposed on the bias use VAR_OUTPUT to identify the beginning of the output type variable
  • the output variables include: execution completion, execution status and output data.
  • the programming element library 20 is used to store program codes and corresponding description files of each programming element used to implement various judgment logic and control logic.
  • programming elements may include but are not limited to:
  • the programming rules of the program codes and description files of the programming elements can be consistent with the programming rules of the neuron block.
  • the program code for each programming element may also include a plurality of program codes corresponding to different programming languages and different artificial intelligence frameworks.
  • the names of the program code, description file, and graphical representation of each programming element are also the same.
  • the programming element library 20 may be a programming element library compliant with industry standards such as the IEC61131-3 standard.
  • the graphical management module 30 is used to extract the description file of each neuron block from the neuron block code library 10, generate corresponding neuron block graphics according to the description file of each neuron block, and extract each neuron block from the programming element library 20.
  • the description file of the programming element generates a corresponding programming element graphic according to the description file of each programming element.
  • both the neuron block graphics and the programming element graphics conform to an industry standard, such as the IEC61131-3 standard.
  • the description file extracted from the neuron block code library 10 or the programming element library 20 can be loaded into the memory, and then the description file loaded into the memory is parsed to generate the corresponding graphical representation component.
  • Figure 2 shows an example of a graphical representation of a typical neuron block in an example. As shown in Figure 2, it has five input variables and three output variables. It can be seen that the layout of the graphical representation of the neuron block refers to the function block specification defined by the IEC61131-3 standard. Among them, EN means input enable, ENO means output enable. IN1 is a tensor input, IN2 is an integer input, IN3 is a real number input, IN4 is a tuple input, Status is an integer output, and Output is a tensor output.
  • EN input enable
  • ENO means output enable
  • IN1 is a tensor input
  • IN2 is an integer input
  • IN3 is a real number input
  • IN4 is a tuple input
  • Status is an integer output
  • Output is a tensor output.
  • Fig. 3 is an example of a graphical representation of a neuron block of a two-dimensional convolution function in an example. As shown in Figure 3, it includes 17 inputs and 3 outputs. Among them, EN represents Boolean input enable, ENO represents Boolean output enable, Input represents tensor input, filters (convolution kernel number) represents the output dimension of data, kernel_size (convolution kernel size) represents integer or A list/tuple composed of a single integer, the spatial or time domain window length of the convolution kernel, strides (step length) represents an integer or a list/tuple composed of a single integer, which is the step length of the convolution, and padding (padding) represents complement 0 or complement 1 strategy, data_format (data format) indicates the input format of the data, dilation_rate (dilation ratio) indicates the expansion ratio of tuple input, activation (activation function) indicates processing using the specified activation function, and use_bias (using bias) indicates Whether to use bias, kernel_initial
  • FIG. 4 shows a simplified display example of FIG. 3 in one example.
  • the user can hide or show part of the data by clicking the symbol at the bottom. For example, click the " ⁇ " symbol at the bottom to hide a part of the data, and click the " ⁇ " symbol at the bottom to show the hidden data.
  • Fig. 5A is an example of a graphical representation of the neuron block of the two-dimensional largest pool block in an example. As shown in Figure 5, it includes 7 inputs and 3 outputs. Among them, EN represents Boolean input enable, ENO represents Boolean output enable, Input represents tensor input, pool_size (sampling factor) represents the downsampling factor in two directions (vertical, horizontal), strides (step Long) represents the step size of sampling, padding represents the filling method of data input, data_format represents the position of the channel dimension of the representative image, and name represents the name of the two-dimensional maximum pooling layer. Status is the execution status, and Output is the tensor output.
  • Fig. 5B is a simplified display example of Fig. 5A in an example of the present invention.
  • the graphical modeling interface 40 is used to present the neuron block graphics and the programming element graphics generated by the graphics management module 30, and receive the user to select the corresponding neuron block graphics and/or programming element graphics based on the industry standard AI graphical model established.
  • the neuron block graphics and programming element graphics can be presented in various ways in the graphical modeling interface 40.
  • it can be a small icon + large graphic presentation mode similar to an industry standard such as the IEC61131-3 standard. That is, firstly, the icons corresponding to the neuron block graphics and programming element graphics are presented through the menu window.
  • the icon corresponding to the icon can be displayed in the modeling window according to the user's mouse position. Large graphics, and then the user can instantiate and initialize the large graphics by dragging, connecting, inputting and outputting.
  • the value of the input and output variables of the neuron block Conv2d of the two-dimensional convolution function comes from the first convolution layer of a CNN model VGG16 model.
  • the description file after user instantiation and initialization can include: NEURAL_BLOCK conv2d indicates that the neuron block is named conv2d; VAR_INPUT is used to identify the beginning of the input type variable, and the input variables include: the input enable is the BOOL type, the input image size is: length and width are 224 pixels RGB 3 channels, 64 A convolution kernel, the size of the convolution kernel is 3 times 3, the step size in the X and Y directions are both 1, the convolution result at the boundary is preserved, the TensorFlow format, that is, the input data format is (224,224,3), and the expansion ratio is 1.
  • a complete AI graphical model can include multiple neuron blocks and/or programming elements.
  • Figure 6A shows an AI model constructed by a user in an example consisting of two Conv2d neuron blocks and one maxpooling2d neuron block.
  • Fig. 6B is a simplified display example of Fig. 6A in an example.
  • the corresponding instantiation description file may include: For the first Conv2D neuron block, 64 convolution kernels are used, the size of the convolution kernel is 3 times 3, and relu activation is used Function, keep the convolution result at the boundary, set the name of the two-dimensional convolution neuron block to block1_conv1; for the second Conv2D neuron block, use 64 convolution kernels, the size of the convolution kernel is 3 times 3, use relu Activate the function, keep the convolution result at the boundary, set the name of the two-dimensional convolution neuron block to block1_conv2; for the MaxPooling2d neuron block, set the sampling factor to 2 times 2, set the step size to 2 times 2, and set the two-dimensional maximum The name of the pooled neuron block is block1_pool.
  • Figure 7 shows a very famous CNN model AlexNet built by a user in an example. It consists of 5 Conv2d neuron blocks, 3 Maxpooling2d neuron blocks, one Flatten neuron block and 3 Dense neuron blocks. It can be seen that its modeling environment is very similar to the PLC modeling environment of the IEC61131-3 standard.
  • the code converter 50 is used for extracting corresponding program codes from the neuron block code library and/or the programming element library according to the neuron block graphics and/or programming element graphics selected in the artificial intelligence graphical model , And generate a complete coding program (that is, the final AI source code) according to the connection relationship and interface data between adjacent graphics in the selected neuron block graphics and/or programming element graphics.
  • the code converter 50 can scan the neuron block graphics and/or programming element graphics selected in the artificial intelligence graphical model in the order from left to right and top to bottom one by one, according to the current scan The name corresponding to the graph, extract the corresponding program code from the neuron block code library or the programming element library, and write the program code into the file cache; after the scan is completed, according to the selected neuron block graph and The connection relationship and interface data (such as input and output data) between adjacent graphics in the programming element graphics are generated, and a complete coding program is generated and written into an AI program file with an AI file name.
  • the code converter 50 can be based on the neural network selected in the AI graphical model. Metablock graphics and/or programming element graphics, and the acquired target programming language and target artificial intelligence framework corresponding to the target system configured by the user, and the corresponding program codes are extracted from the neuron block code library 10 and the programming element library 20.
  • the code converter 50 may further receive the AI model coding program input by the user, analyze the coding program line by line, and obtain keywords that can represent neuron blocks or programming elements. , Determine the corresponding neuron block or programming element according to the keyword, and determine the corresponding neuron block graphic or programming element graphic and its interface according to the connection relationship and interface data between adjacent neuron blocks and/or programming elements The connection relationship between the data and the adjacent graphics is generated, and the corresponding AI graphical model is generated, and the AI graphical model is provided to the graphical modeling interface 40 for display.
  • the transcoder 50 can complete the forward transformation from the graphical model to the encoding program, and can also complete the inverse transformation from the encoding program to the graphical model.
  • the system for supporting neuron block graphic programming in the embodiment of the present invention has been described in detail above, and the method for supporting neuron block graphic programming in the embodiment of the present invention will be described below.
  • the support in the embodiment of the present invention is based on neuron block graphics.
  • the method of metablock graphic programming can be implemented by the system shown in FIG. 1 that supports neuron block graphic programming, and for details that are not disclosed in the method embodiment of the present invention, please refer to the corresponding description in the system embodiment. Let me repeat them one by one.
  • Fig. 8 is an exemplary flowchart of a method for supporting graphical programming based on neuron blocks in an embodiment of the present invention. As shown in Figure 8, the method may include the following steps:
  • Step S82 Prestore the program code and corresponding description file of each neuron block corresponding to each functional element in the artificial intelligence in a neuron block code library, and store it in a programming element library for realizing various judgment logic and control
  • the program code and corresponding description file of each programming element of logic may include: programming elements for conditional judgment logic, programming elements for loop control logic, programming elements for AND or non-control logic, and programming elements for enabling and disabling control logic.
  • step S84 when the system is started, the description file of each neuron block is extracted from the neuron block code library, and the corresponding neuron block graphic is generated according to the description file of each neuron block.
  • the description file of each programming element is extracted, and the corresponding programming element graphic is generated according to the description file of each programming element; both the neuron block graphic and the programming element graphic conform to an industry standard.
  • Step S86 presenting the neuron block graphics and the programming element graphics in a graphical modeling interface, and receiving the artificial creation created by the user based on the industry standards by selecting the corresponding neuron block graphics and/or programming element graphics. Intelligent graphical model.
  • Step S88 According to the neuron block graphics and/or programming element graphics selected in the artificial intelligence graphical model, extract the corresponding program code from the neuron block code library and/or the programming element library, and according to The connection relationship and interface data between adjacent graphics in the selected neuron block graphics and/or programming element graphics are used to generate a complete coding program.
  • the program code of each neuron block may include: a plurality of program codes corresponding to different programming languages and different artificial intelligence frameworks for each neuron block;
  • the program code of each programming element may include : A plurality of program codes corresponding to different programming languages and different artificial intelligence frameworks for each programming element; correspondingly, the neuron block graphics and/or programming element graphics selected in the artificial intelligence graphical model can be selected in step S88 , And the acquired target programming language and target artificial intelligence framework corresponding to the target system configured by the user, and the corresponding program code is extracted from the neuron block code library and/or the programming element library.
  • the above method may further include: receiving an artificial intelligence model coding program input by a user; parsing the coding program line by line; obtaining keywords that can represent neuron blocks or programming elements; and determining corresponding keywords based on the keywords Neuron block or programming element; according to the connection relationship and interface data between adjacent neuron blocks and/or programming elements, determine the corresponding neuron block graphics or programming element graphics and the connection between its interface data and adjacent graphics And generate a corresponding artificial intelligence graphical model; and display the artificial intelligence graphical model through the graphical modeling interface.
  • Fig. 9 is an exemplary structure diagram of another system supporting graphical programming based on neuron blocks in an embodiment of the present invention.
  • the system may include: at least one memory 91, at least one processor 92, at least one database 93 and at least one display 94.
  • some other components may also be included, such as communication ports. These components communicate via the bus 95.
  • At least one memory 91 is used to store computer programs.
  • the computer program can be understood as including the various modules of the system shown in FIG. 1 supporting graphical programming based on neuron blocks.
  • at least one memory 91 may also store an operating system and the like.
  • Operating systems include but are not limited to: Android operating system, Symbian operating system, Windows operating system, Linux operating system and so on.
  • At least one database 93 is used to provide a neuron block code library and a programming element library; the neuron block code library is used to store program codes and corresponding description files of each neuron block corresponding to each functional element in artificial intelligence; The programming element library is used to store the program codes and corresponding description files of each programming element used to implement various judgment logic and control logic.
  • the display 94 is used to provide a graphical modeling interface.
  • At least one processor 92 is configured to call a computer program stored in at least one memory 91 to execute the method for supporting neuron block-based graphic programming described in the embodiment of the present invention.
  • the processor 92 may be a CPU, a processing unit/module, an ASIC, a logic module, or a programmable gate array. It can receive and send data through the communication port.
  • a hardware module may include specially designed permanent circuits or logic devices (such as dedicated processors, such as FPGAs or ASICs) to complete specific operations.
  • the hardware module may also include programmable logic devices or circuits temporarily configured by software (for example, including general-purpose processors or other programmable processors) for performing specific operations.
  • software for example, including general-purpose processors or other programmable processors
  • the embodiment of the present invention also provides a computer software that can be executed on a server or a server cluster or a cloud platform.
  • the computer software can be executed by a processor and implement the neuron-based support described in the embodiment of the present invention. Block graphics programming method.
  • the embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and the computer program can be executed by a processor and realize the support based on neuron block graphics in the embodiment of the present invention.
  • Programming method Specifically, a system or device equipped with a storage medium may be provided, and the software program code for realizing the function of any one of the above-mentioned embodiments is stored on the storage medium, and the computer (or CPU or MPU of the system or device) ) Read and execute the program code stored in the storage medium.
  • an operating system or the like operating on the computer can also be used to complete part or all of the actual operations through instructions based on the program code.
  • Implementations of storage media used to provide program codes include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), Magnetic tape, non-volatile memory card and ROM.
  • the program code can be downloaded from the server computer via a communication network.
  • the industrial field standard IEC61131-3 standard is mainly used as an example for description. In practical applications, other industrial field standards can also be used as needed, as long as it is convenient for users in the industrial field to realize barrier-free AI Just program.
  • the user can first configure the target system of the coding program, including programming language, AI framework, etc., when using it. After that, the user can build an AI graphical model based on the neuron block graphics and the programming element graphics presented in the graphical modeling interface 40, and after the AI graphical model is built, click the code conversion, and the system Complete the conversion of the graphical model to the coding program (ie source code). After that, users can use powerful PC, cluster, cloud and other training resources for AI model training based on the completed AI coding program.
  • the target system of the coding program including programming language, AI framework, etc.
  • each neuron block is set for each functional element in AI, and three realizations of program code, description file and graphical representation are completed for each neuron block.
  • its graphical representation is made to conform to a set industrial field standard such as the IEC61131-3 standard, thereby providing users with a programming mode that follows the same programming mode as the PLC development software of the IEC61131-3 standard, thereby making them accustomed to industrial fields such as PLC
  • the coding user can implement the model building of the AI application without barriers, and then the system performs code conversion based on the model building, and generates a coding program that meets the operating environment of the target system, thereby completing barrier-free AI programming.
  • the final coding program can be converted into a coding program that meets the programming language and AI framework of the target system, which improves the flexibility and flexibility of programming. Versatility.
  • the compatibility with the user's existing coding program can be realized.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Automation & Control Theory (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Stored Programmes (AREA)

Abstract

一种支持基于神经元块图形编程的系统、方法及存储介质。其中，编码系统包括：存储各神经元块的程序代码和对应的描述文件的神经元块代码库(10)；存储各编程元素的程序代码和对应的描述文件的编程元素库(20)；用于从神经元块代码库和/或编程元素库中提取各描述文件并生成对应的图形的图形化管理模块(30)；所述图形符合一工业领域标准；用于呈现图形化管理模块生成的图形，并接收用户基于所述图形建立的人工智能图形化模型的图形化建模界面(40)；和用于将所述人工智能图形化模型转换为对应的编码程序的代码转换器(50)。该系统能够使习惯于PLC等工业领域编码的用户能够无障碍实现对AI应用的编码。

Description

支持基于神经元块图形编程的系统、方法及存储介质 技术领域
本发明涉及工业领域，特别是一种支持基于神经元块图形编程的系统、方法及存储介质。
背景技术
人工智能(Artificial Intelligence，AI)技术已在人脸检测、自然语言处理等领域得到应用发展，目前自动化系统的用户(如工程师)也正试图将AI应用于工业领域，以提高自动化系统的性能和能力。然而，自动化系统用户大多擅长于开发由IEC61131标准化的可编程逻辑控制器(PLC)，而不是AI领域常用的Python或C++(CPP)等语言编程。这使得他们在学习其他编程语言、人工智能框架和库，使用另一种不同的集成开发环境(IDE)进行AI应用的开发时，会面临很多新的挑战和困难。
虽然目前也存在几种人工智能开发环境软件，如微软的机器学习工作室、Knime的Knime分析平台、根特大学的Dianne、RapidMiner的RapidMiner工作室。但是，所有这些解决方案都没有遵循或扩展到标准。因此，自动化系统用户仍然需要学习和使用一个新的开发环境。
发明内容
有鉴于此，本发明实施例中一方面提出了一种支持基于神经元块图形编程的系统，另一方面提出了一种支持基于神经元块图形编程的方法和计算机可读存储介质，以便于使习惯于PLC等工业领域编码的用户能够无障碍实现对AI应用的编码。
本发明实施例中提出的一种支持基于神经元块图形编程的系统，包括：神经元块代码库，存储对应人工智能中各功能元素的各神经元块的程序代码和对应的描述文件；编程元素库，存储用于实现各种判断逻辑和控制逻辑的各编程元素的程序代码和对应的描述文件；图形化管理模块，用于从所述神经元块代码库中提取各神经元块的描述文件，根据所述各神经元块的描述文件生成对应的神经元块图形，从所述编程元素库中提取各编程元素的描述文件，根据所述各编程元素的描述文件生成对应的编程元素图形；所述 神经元块图形和所述编程元素图形均符合一设定的工业领域标准；图形化建模界面，用于呈现所述图形化管理模块生成的所述神经元块图形和所述编程元素图形，接收用户基于所述工业领域标准选用对应的神经元块图形和/或编程元素图形建立的人工智能图形化模型；代码转换器，用于根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码，并根据所选用的神经元块图形和/或编程元素图形中各相邻图形之间的连接关系和接口数据，生成完整的编码程序。
在一个实施方式中，所述工业领域标准为IEC61131-3标准。
在一个实施方式中，所述各神经元块的程序代码包括：针对每个神经元块的对应不同编程语言和不同人工智能框架的复数个程序代码；所述各编程元素的程序代码包括：针对每个编程元素的对应不同编程语言和不同人工智能框架的复数个程序代码；所述代码转换器根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，以及获取的用户配置的目标系统所对应的目标编程语言和目标人工智能框架，从所述神经元块代码库和所述编程元素库中提取对应的程序代码。
在一个实施方式中，所述不同编程语言和不同人工智能框架包括：Python语言的TensorFlow框架和PyTorch框架，以及C++语言的Caffe框架。
在一个实施方式中，所述神经元块包括：二维卷积函数块、二维最大池块、由二维到一维的平整化块、由多到少的整合转换块中的一个或多个。
在一个实施方式中，所述编程元素包括：用于进行条件判断逻辑的编程元素、用于进行循环控制逻辑的编程元素、用于进行与或非控制逻辑的编程元素和用于使能和非使能控制逻辑的编程元素中的一个或多个。
在一个实施方式中，所述各神经元块的程序代码中所采用的数据类型包括：符合所述工业领域标准的基本数据类型；和对应人工智能框架的包括元组、张量和空在内的新增数据类型。
在一个实施方式中，所述代码转换器进一步用于接收用户输入的人工智能模型编码程序，对所述编码程序逐行进行解析，获取能够代表神经元块或编程元素的关键词，根据所述关键词确定对应的神经元块或编程元素，根据相邻神经元块和/或编程元素之间的连接关系以及接口数据，确定对应的神经元块图形或编程元素图形及其接口数据和相邻图形之间的连接关系，生成对应的人工智能图形化模型，将所述人工智能图形化模型提供给图形化建模界面进行显示。
本发明实施例中提出的一种支持基于神经元块图形编程的方法，包括：预先在一神 经元块代码库中存储对应人工智能中各功能元素的各神经元块的程序代码和对应的描述文件，在一编程元素库中存储用于实现各种判断逻辑和控制逻辑的各编程元素的程序代码和对应的描述文件；系统启动时，从所述神经元块代码库中提取各神经元块的描述文件，根据所述各神经元块的描述文件生成对应的神经元块图形，从所述编程元素库中提取各编程元素的描述文件，根据所述各编程元素的描述文件生成对应的编程元素图形；所述神经元块图形和所述编程元素图形均符合一设定的工业领域标准；在一图形化建模界面中呈现所述神经元块图形和所述编程元素图形，并接收用户基于所述工业领域标准选用对应的神经元块图形和/或编程元素图形建立的人工智能图形化模型；根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码，并根据所选用的神经元块图形和/或编程元素图形中各相邻图形之间的连接关系和接口数据，生成完整的编码程序。
在一个实施方式中，所述各神经元块的程序代码包括：针对每个神经元块的对应不同编程语言和不同人工智能框架的复数个程序代码；所述各编程元素的程序代码包括：针对每个编程元素的对应不同编程语言和不同人工智能框架的复数个程序代码；所述根据人工智能图形化模型中选用的神经元块图形和/或编程元素图形，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码为：根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，以及获取的用户配置的目标系统所对应的目标编程语言和目标人工智能框架，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码。
在一个实施方式中，所述编程元素包括：用于进行条件判断逻辑的编程元素、用于进行循环控制逻辑的编程元素、用于进行与或非控制逻辑的编程元素和用于使能和非使能控制逻辑的编程元素中的一个或多个。
在一个实施方式中，所述各神经元块的程序代码中所采用的数据类型包括：符合所述工业领域标准的基本数据类型；和对应人工智能框架的包括元组、张量和空在内的新增数据类型。
在一个实施方式中，该方法进一步包括：接收用户输入的人工智能模型编码程序；对所述编码程序逐行进行解析；获取能够代表神经元块或编程元素的关键词；根据所述关键词确定对应的神经元块或编程元素；根据相邻神经元块和/或编程元素之间的连接关系以及接口数据，确定对应的神经元块图形或编程元素图形及其接口数据和相邻图形之间的连接关系，并生成对应的人工智能图形化模型；将所述人工智能图形化模型通过所述图形化建模界面进行显示。
本发明实施例中提出的又一种支持基于神经元块图形编程的系统，包括：至少一个存储器、至少一个处理器、至少一个数据库和至少一个显示器，其中：所述至少一个存储器用于存储计算机程序；所述至少一个数据库用于提供一神经元块代码库和一编程元素库；所述神经元块代码库用于存储对应人工智能中各功能元素的各神经元块的程序代码和对应的描述文件；所述编程元素库用于存储用于实现各种判断逻辑和控制逻辑的各编程元素的程序代码和对应的描述文件；所述显示器用于提供一图形化建模界面；所述至少一个处理器用于调用所述至少一个存储器中存储的计算机程序，执行如上所述任一实施方式中的支持基于神经元块图形编程的方法。
本发明实施例中提出的一种计算机可读存储介质，其上存储有计算机程序；所述计算机程序能够被一处理器执行并实现如上所述任一实施方式中的支持基于神经元块图形编程的方法。
从上述方案中可以看出，由于本发明实施例中针对AI中的各功能元素设置了各个神经元块，并为每个神经元块完成了程序代码、描述文件和图形化表示三种实现，特别是使得其图形化表示符合一设定的工业领域标准如IEC61131-3标准，从而为用户提供了一种遵循IEC61131-3标准的PLC开发软件相同的编程模式，进而使得习惯于PLC等工业领域编码的用户能够无障碍实现对AI应用的模型搭建，之后系统基于该模型搭建进行代码转换，生成符合目标系统运行环境的编码程序，从而完成了无障碍的AI编程。
此外，通过为每个神经元块构建不同的编程语言和不同AI框架的程序代码，使得最终的编码程序可以转换为满足目标系统的编程语言和AI框架的编码程序，提高了编程的灵活性和通用性。
进一步地，通过将用户的已有编码程序转换为对应的图形化模块，可以实现对用户已有编码程序的兼容。
附图说明
下面将通过参照附图详细描述本发明的优选实施例，使本领域的普通技术人员更清楚本发明的上述及其它特征和优点，附图中：
图1为本发明实施例中一种支持基于神经元块图形编程的系统的示例性结构图。
图2为本发明一个例子中典型的神经元块图形的示例。
图3为本发明实施例中二维卷积函数的神经元块图形的示例。
图4为本发明一个例子中图3的简化显示示例。
图5A为本发明一个例子中二维最大池块的神经元块图形的示例。
图5B为本发明一个例子中图5A的简化显示示例。
图6A为本发明一个例子中用户搭建的人工智能模型。
图6B为一个例子中图6A的简化显示示例。
图7为本发明一个例子中用户搭建的一个非常著名的卷积神经网络(CNN)模型AlexNet。
图8为本发明实施例中支持基于神经元块图形编程的方法的示例性流程图。
图9为本发明实施例中又一种支持基于神经元块图形编程的系统的示例性结构图。
其中，附图标记如下：
标号 含义
10 神经元块代码库
20 编程元素库
30 图形化管理模块
40 图形化建模界面
50 代码转换器
S82、S84、S86、S88 步骤
91 存储器
92 处理器
93 数据库
94 显示器
95 总线
具体实施方式
为了描述上的简洁和直观，下文通过描述若干代表性的实施方式来对本发明的方案进行阐述。实施方式中大量的细节仅用于帮助理解本发明的方案。但是很明显，本发明的技术方案实现时可以不局限于这些细节。为了避免不必要地模糊了本发明的方案，一些实施方式没有进行细致地描述，而是仅给出了框架。下文中，“包括”是指“包括但不限于”，“根据……”是指“至少根据……，但不限于仅根据……”。由于汉语的语言习惯，下文中没有特别指出一个成分的数量时，意味着该成分可以是一个也可以是多个，或可理解为至少一个。
本发明实施例中，为了使习惯于PLC等工业领域编码的用户如自动化系统用户能够 无障碍实现对AI应用的编码，考虑提供一种遵循设定的工业领域标准如IEC61131-3标准的类似PLC等工业开发软件的编码系统。
对应AI中的每个功能元素可设置一个神经元块，例如，神经元块可包括：二维卷积函数块、二维最大池块、由二维到一维的扁平化块以及由多到少的全连接块等。并且针对每个神经元块可实现其程序代码、描述文件和图形化表示三个对象，神经元块可以是遵循IEC61131-3中功能块规范的高级抽象，即可以被定义为IEC61131-3中功能块的扩展，以保持与PLC相同的编程模式。其既可以在一个单独的开发环境中架构，也可以集成到PLC的开发环境中。具体可根据实际情况实现，此处不对其进行限制。只要能够使得自动化系统用户可以在不学习其他编程语言和熟悉其他集成开发环境(Integrated Development Environment，IDE)的情况下构成人工智能模型即可。之后，系统可根据用户基于图形化表示的神经块构建的AI模型，将其转换为可执行的目标程序代码。
为了使本发明的技术方案及优点更加清楚明白，以下结合附图及实施方式，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施方式仅仅用以阐述性说明本发明，并不用于限定本发明的保护范围。
图1为本发明实施例中一种支持基于神经元块图形编程的系统的示例性结构图。如图1所示，该系统中可包括：神经元块代码库10、编程元素库20、图形化管理模块30、图形化建模界面40和代码转换器50。
其中，神经元块代码库10用于存储和管理对应AI中各功能元素的各神经元块的程序代码和对应的描述文件。
具体实现时，考虑到实际应用中可能存在不同编程语言的不同AI框架，例如，Python语言的TensorFlow框架和PyTorch框架，以及C++语言的Caffe框架等，为了使得该支持基于神经元块图形编程的系统更加通用和灵活，本发明实施例中，针对每个神经元块，可分别实现一种编程语言的一种AI框架的程序代码。也就是说，对应上述提到的Python语言的TensorFlow框架和PyTorch框架，以及C++语言的Caffe框架，针对每个神经元块可实现三个程序代码，即针对每个神经元模块，可实现一种Python语言的TensorFlow框架的程序代码，一种Python语言的PyTorch框架的程序代码，和一种C++语言的Caffe框架的程序代码。当然，在其他实施方式中，针对每个神经元块，也可仅采用一种编程语言的一种AI框架进行实现。或者，仅采用主流的几种编程语言的AI框架进行实现。具体可根据实际情况确定，此处不对其进行限定。
具体实现时，各神经元块所涉及的数据类型可包括：符合所述工业领域标准的基本数据类型；和对应人工智能框架的包括元组、张量和空在内的新增数据类型。例如，下 表1中示出了本发明实施例中各神经元块所涉及的各数据类型。
Figure PCTCN2019102903-appb-000001
Figure PCTCN2019102903-appb-000002
表1
其中，“备注”栏里的“a”表示与IEC61131-3标准兼容的基本数据类型，“备注”栏里的“b”表示对应人工智能框架的新增数据类型。
下面对上述表1中的三个新增数据类型进行简单介绍：
1)TUPLE(元组)
元组是对象的集合。在本发明实施例中，所述对象是表1中定义的其他数据类型。元组的格式可定义为(object[，object])。
2)TENSOR(张量)
张量是人工智能领域中定义的一个关键字，它表示向量和矩阵的泛化。实际上，它是一个特殊的多维数组。
3)None(空)
表示空的概念。对于Python类型的接口，None直接用作输入，但是对于C++类型 的接口，None将根据输入参数的数据类型映射到一个特殊值。
各神经元块的描述文件的语法定义格式可遵循巴克斯范式(BNF)，其在包括IEC61131-3规范的程序语言定义中使用较多。
例如，一个神经元块的描述文件的语法规范中的关键字段可描述如下：
1)神经元块由关键词“NEURAL_BLOCK”来识别，由关键词“'END_NEURAL_BLOCK'”来表示一个神经元块的结束。每个神经元块都有自己的名字，其名字可以为标识符。无论是其程序代码、描述文件，还是其图形化表示其名字都是一致的，因此可根据名字作为关键字进行对应的检索。例如，上述二维卷积函数块可以命名未Conv2d，二维最大池块可以命名为Maxpooling2d，由二维到一维的扁平化块可以命名为Flatten，由多到少的全连接块可以命名为Dense。
2)一个神经元块包括三种类型的变量：输入类型、输出类型和输入输出类型。
A.VAR_INPUT：用于识别显示在神经元块图形左侧的所有输入类型变量。输入类型变量的声明语句可由若干字段组成：变量名、变量类型、是否对用户可见、是否必需、默认值等，其支持的变量类型可包括表1中列出的所有变量类型。
B.VAR_OUTPUT：用于识别显示在神经元块图形右侧的所有输出类型变量。输出类型变量声明语句由若干字段组成：输出变量名，输出变量数据类型，是否对用户可见等。此外，用RETAIN和NON_RETAINb标识断电时是否保持数据。
C.VAR_IN_OUT：用于识别输入输出类型变量，这些变量显示在神经元块图形的左右两侧。输入输出类型变量声明语句由若干字段组成：输入输出变量名，输入输出变量数据类型，是否对用户可见，是否必需，默认值等。输入输出变量名为标识符，且标识符由字符、数字和下划线组成，且首字符必须为字母或下划线。支持大小写区分的26个字母，支持0到9的10个数字。
3)关键字VISIBLE定义该关键字所修饰的变量是否显示在折叠模式下，以简化图形神经元块的可视部分。
4)关键字REQUIRED定义是否需要用户给出该变量的输入值。对于不使用此关键字修饰的变量，如果用户不提供输入，则将内置默认值用作输入值。
例如，一个例子中在Python的TensorFlow AI框架中使用的二维卷积函数的描述文件可包括：利用NEURAL_BLOCK conv2d来表示神经元块名为conv2d；利用VAR_INPUT来识别输入类型变量的开始，并且输入变量包括：输入使能、输入数据、卷积核数目、卷积核大小、步长、填充方式、输入数据格式、膨胀比例、激活函数、使用偏置、权值初始化方法、偏置初始化方法、施加在权重上的正则项、施加在偏置向量上的正则、施 加在输出上的正则项、施加在权重上的约束项和施加在偏置上的约束项；利用VAR_OUTPUT来识别输出类型变量的开始，并且输出变量包括：执行完成、执行状态和输出数据。
编程元素库20用于存储用于实现各种判断逻辑和控制逻辑的各编程元素的程序代码和对应的描述文件。具体实现时，编程元素可包括但不限于：
1)用于进行条件判断逻辑的编程元素，例如：IF…THEN…ELSIF…ELSIF…END_IF。
2)用于进行循环控制逻辑的编程元素，例如，WHILE…DO…END_WHILE。
3)用于进行与或非控制逻辑的编程元素，例如，NOT,AND,OR。
4)用于使能和非使能控制逻辑的编程元素，例如，常开触点(Normally Open Contact)、常闭触点(Normally Closed Contact)、线圈(Coils)。
具体实现时，编程元素的程序代码和描述文件的编制规则可同神经元块的编制规则一致。例如，针对每个编程元素的程序代码也可包括对应不同编程语言和不同人工智能框架的复数个程序代码。又如，每个编程元素的程序代码、描述文件和图形化表示三者的名字也都一致。编程元素库20可以是符合工业领域标准例如IEC61131-3标准的编程元素库。
图形化管理模块30用于从神经元块代码库10中提取各神经元块的描述文件，根据所述各神经元块的描述文件生成对应的神经元块图形，从编程元素库20中提取各编程元素的描述文件，根据所述各编程元素的描述文件生成对应的编程元素图形。其中，所述神经元块图形和所述编程元素图形均符合一工业领域标准，例如IEC61131-3标准。其中，从神经元块代码库10或编程元素库20中提取的描述文件可加载至内存中，然后对加载至内存中的描述文件进行解析生成对应的图形化表示部件。
例如，图2示出了一个例子中典型的神经元块的图形化表示的示例。如图2所示，其有五个输入变量和三个输出变量。可见，该神经元块的图形化表示的布局参照了IEC61131-3标准定义的功能块规范。其中，EN表示输入使能，ENO表示输出使能。IN1为张量输入，IN2为整数输入，IN3为实数输入，IN4为元组输入，Status为整数输出，Output为张量输出。
图3为一个例子中二维卷积函数的神经元块的图形化表示的示例。如图3所示，其包括17个输入和3个输出。其中，EN表示布尔型的输入使能，ENO表示布尔型的输出使能，Input表示张量输入，filters(卷积核数码)表示数据的输出维度，kernel_size(卷积核大小)表示整数或由单个整数构成的list/tuple，卷积核的空域或时域窗长度，strides(步长)表示整数或由单个整数构成的list/tuple，为卷积的步长，padding(填充方式) 表示补0或补1策略，data_format(数据格式)表示数据的输入格式，dilation_rate(膨胀比例)表示元组输入的膨胀比例，activation(激活函数)表示使用指定的激活函数处理，use_bias(使用偏置)表示是否使用偏置，kernel_initializer(权值初始化方法)表示为预定义初始化方法名的字符串，或用于初始化权重的初始化器，bias_initializer(权值初始化方法)表示为预定义初始化方法名的字符串，或用于初始化权重的初始化器，kernel_regulatizer(施加在权重上的正则项)表示为权重施加指定的正则项，bias_regulatizer(施加在偏置向量上的正则项)表示为偏置施加指定的正则项，activaty_regulatizer(施加在输出上的正则项)表示为输出施加正则项，kernel_constraint(施加在权重上的约束项)表示为权重施加约束项，bias_constraint(施加在偏置上的约束项)表示为偏置施加约束项。Status为执行状态，Output为张量输出。
此外，考虑到图形化表示的神经元块图形有时输入输出数据较多，如图3中输入数据较多，因此为了节约空间，便于全局显示整个AI模型，可针对输入输出数据较多的神经元块的图形化表示进行简化显示，例如，针对图3所示示例，可将不常用的输入数据进行隐藏显示。例如，图4中示出了一个例子中图3的简化显示示例。如图4所示，用户可以通过点击底部的符号来隐藏或显示部分数据。例如，单击底部的“︽”符号来隐藏一部分数据，并通过单击底部的“︾”符号来显示隐藏的数据。
图5A为一个例子中二维最大池块的神经元块的图形化表示的示例。如图5所示，其包括7个输入和3个输出。其中，EN表示布尔型的输入使能，ENO表示布尔型的输出使能，Input表示张量输入，pool_size(采样因子)表示两个方向(竖直，水平)上的下采样因子，strides(步长)表示采样的步长，padding(填充方式)表示数据输入的填充方式，data_format(数据格式)表示代表图像的通道维的位置，name(名字)表示二维最大池化层的名称。Status为执行状态，Output为张量输出。图5B为本发明一个例子中图5A的简化显示示例。
图形化建模界面40用于呈现图形化管理模块30生成的所述神经元块图形和所述编程元素图形，接收用户基于所述工业领域标准选用对应的神经元块图形和/或编程元素图形建立的AI图形化模型。
具体实现时，神经元块图形和编程元素图形在图形化建模界面40中的呈现方式可有多种。例如，可以是类似一工业领域标准如IEC61131-3标准的小图标+大图形的呈现方式。即首先通过菜单窗口呈现各神经元块图形和编程元素图形所对应的各个图标，当用户进行AI图形化模型搭建点击某个图标时，可根据用户的鼠标位置在建模窗口呈现该图标对应的大图形，进而用户可对该大图形进行拖拽、连线和输入输出等实例化和初始化 编辑。例如，一个例子中二维卷积函数的神经元块Conv2d的输入输出变量的值来自于一CNN模型VGG16模型的第一卷积层，其经用户实例化和初始化后的描述文件可包括：利用NEURAL_BLOCK conv2d来表示神经元块名为conv2d；利用VAR_INPUT来识别输入类型变量的开始，并且输入变量包括：输入使能为BOOL类型、输入图片大小为：长宽分别为224个像素RGB 3通道、64个卷积核、卷积核大小为3乘3、X和Y方向步长均为1、保留边界处的卷积结果、TensorFlow格式，即输入数据格式为(224,224,3)、膨胀比例为1、不使用激活函数、使用偏置、使用'GLOROT_UNIFORM’类型的权值初始化方法、使用‘ZERO’类型的偏置初始化方法、不对权重施加正则项、不对偏置施加正则项、不对输出施加正则项、不对权重施加约束项、不对偏置施加约束项；利用VAR_OUTPUT来识别输出类型变量的开始，并且输出变量包括：神经元模块执行完成、执行状态和输出张量。
一个完整的AI图形化模型可包括多个神经元块和/或编程元素。图6A示出了一个例子中用户搭建的由两个Conv2d神经元块和一个maxpooling2d神经元块组成的AI模型。图6B为一个例子中图6A的简化显示示例。
对应图6A和图6B所示的AI模型，其对应的实例化描述文件可包括：对于第一个Conv2D神经元块，使用64个卷积核，卷积核大小为3乘3、使用relu激活函数、保留边界处的卷积结果、设置二维卷积神经元块的名称为block1_conv1；对于第二个Conv2D神经元块，使用64个卷积核，卷积核大小为3乘3、使用relu激活函数、保留边界处的卷积结果、设置二维卷积神经元块的名称为block1_conv2；对于MaxPooling2d神经元块，设置采样因子为2乘2，设置步长为2乘2，设置二维最大池化神经元块的名称为block1_pool。
图7示出了一个例子中用户搭建的一个非常著名的CNN模型AlexNet。其由5个Conv2d神经元块、3个Maxpooling2d神经元块、一个Flatten神经元块和3个Dense神经元块组成。可见，其建模环境和IEC61131-3标准的PLC建模环境很像。
代码转换器50用于根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码，并根据所选用的神经元块图形和/或编程元素图形中各相邻图形之间的连接关系和接口数据，生成完整的编码程序(即最终的AI源代码)。
具体实现时，代码转换器50可对所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形按照从左到右、从上到下的顺序进行逐一扫描，根据当前扫描的图形对应的名称，从所述神经元块代码库或所述编程元素库中提取对应的程序代码，并将程序 代码写入文件缓存中；在扫描完成后，根据所选用的神经元块图形和编程元素图形中各相邻图形之间的连接关系和接口数据(如输入输出数据)，生成完整的编码程序并写入具有AI文件名的AI程序文件中。
考虑到在存在不同编程语言的不同AI框架的情况下，同一神经元块或编程元素可能对应多个程序代码的实现，因此此时代码转换器50可根据所述AI图形化模型中选用的神经元块图形和/或编程元素图形，以及获取的用户配置的目标系统所对应的目标编程语言和目标人工智能框架，从神经元块代码库10和编程元素库20中提取对应的程序代码。
进一步地，为了兼容用户的已有AI编码程序，代码转换器50可进一步接收用户输入的AI模型编码程序，对所述编码程序逐行进行解析，获取能够代表神经元块或编程元素的关键词，根据所述关键词确定对应的神经元块或编程元素，根据相邻神经元块和/或编程元素之间的连接关系以及接口数据，确定对应的神经元块图形或编程元素图形及其接口数据和相邻图形之间的连接关系，并生成对应的AI图形化模型，将所述AI图形化模型提供给图形化建模界面40进行显示。
也就是说，代码转换器50既可完成由图形化模型到编码程序的正变换，也可完成从编码程序到图形化模型的逆变换。
以上对本发明实施例中的支持基于神经元块图形编程的系统进行了详细描述，下面再对本发明实施例中的支持基于神经元块图形编程的方法进行描述，本发明实施例中的支持基于神经元块图形编程的方法可通过图1所示的支持基于神经元块图形编程的系统进行实现，并且对于本发明方法实施例中未披露的细节可参见系统实施例中的对应描述，此处不再一一赘述。
图8为本发明实施例中支持基于神经元块图形编程的方法的示例性流程图。如图8所示，该方法可包括如下步骤：
步骤S82，预先在一神经元块代码库中存储对应人工智能中各功能元素的各神经元块的程序代码和对应的描述文件，在一编程元素库中存储用于实现各种判断逻辑和控制逻辑的各编程元素的程序代码和对应的描述文件。其中，编程元素可包括：用于进行条件判断逻辑的编程元素、用于进行循环控制逻辑的编程元素、用于进行与或非控制逻辑的编程元素和用于使能和非使能控制逻辑的编程元素中的一个或多个。
步骤S84，系统启动时，从所述神经元块代码库中提取各神经元块的描述文件，根据所述各神经元块的描述文件生成对应的神经元块图形，从所述编程元素库中提取各编程元素的描述文件，根据所述各编程元素的描述文件生成对应的编程元素图形；所述神经元块图形和所述编程元素图形均符合一工业领域标准。
步骤S86，在一图形化建模界面中呈现所述神经元块图形和所述编程元素图形，并接收用户基于所述工业领域标准选用对应的神经元块图形和/或编程元素图形建立的人工智能图形化模型。
步骤S88，根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码，并根据所选用的神经元块图形和/或编程元素图形中各相邻图形之间的连接关系和接口数据，生成完整的编码程序。
在一个实施方式中，所述各神经元块的程序代码可包括：针对每个神经元块的对应不同编程语言和不同人工智能框架的复数个程序代码；所述各编程元素的程序代码可包括：针对每个编程元素的对应不同编程语言和不同人工智能框架的复数个程序代码；相应地，步骤S88中可根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，以及获取的用户配置的目标系统所对应的目标编程语言和目标人工智能框架，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码。
进一步地，上述方法可进一步包括：接收用户输入的人工智能模型编码程序；对所述编码程序逐行进行解析；获取能够代表神经元块或编程元素的关键词；根据所述关键词确定对应的神经元块或编程元素；根据相邻神经元块和/或编程元素之间的连接关系以及接口数据，确定对应的神经元块图形或编程元素图形及其接口数据和相邻图形之间的连接关系，并生成对应的人工智能图形化模型；将所述人工智能图形化模型通过所述图形化建模界面进行显示。
图9为本发明实施例中又一种支持基于神经元块图形编程的系统的示例性结构图。如图9所示，该系统可包括：至少一个存储器91、至少一个处理器92、至少一个数据库93和至少一个显示器94。此外，还可以包括一些其它组件，例如通信端口等。这些组件通过总线95进行通信。
其中：至少一个存储器91用于存储计算机程序。在一个实施方式中，该计算机程序可以理解为包括图1所示的支持基于神经元块图形编程的系统的各个模块。此外，至少一个存储器91还可存储操作系统等。操作系统包括但不限于：Android操作系统、Symbian操作系统、Windows操作系统、Linux操作系统等等。
至少一个数据库93用于提供一神经元块代码库和一编程元素库；所述神经元块代码库用于存储对应人工智能中各功能元素的各神经元块的程序代码和对应的描述文件；所述编程元素库用于存储用于实现各种判断逻辑和控制逻辑的各编程元素的程序代码和对应的描述文件。
显示器94用于提供一图形化建模界面。
至少一个处理器92用于调用至少一个存储器91中存储的计算机程序，执行本发明实施例中所述的支持基于神经元块图形编程的方法。处理器92可以为CPU，处理单元/模块，ASIC，逻辑模块或可编程门阵列等。其可通过所述通信端口进行数据的接收和发送。
需要说明的是，上述各流程和各结构图中不是所有的步骤和模块都是必须的，可以根据实际的需要忽略某些步骤或模块。各步骤的执行顺序不是固定的，可以根据需要进行调整。各模块的划分仅仅是为了便于描述采用的功能上的划分，实际实现时，一个模块可以分由多个模块实现，多个模块的功能也可以由同一个模块实现，这些模块可以位于同一个设备中，也可以位于不同的设备中。
可以理解，上述各实施方式中的硬件模块可以以机械方式或电子方式实现。例如，一个硬件模块可以包括专门设计的永久性电路或逻辑器件(如专用处理器，如FPGA或ASIC)用于完成特定的操作。硬件模块也可以包括由软件临时配置的可编程逻辑器件或电路(如包括通用处理器或其它可编程处理器)用于执行特定操作。至于具体采用机械方式，或是采用专用的永久性电路，或是采用临时配置的电路(如由软件进行配置)来实现硬件模块，可以根据成本和时间上的考虑来决定。
另外，本发明实施例中还提供一种能够在服务器或服务器集群或云平台上执行的计算机软件，所述计算机软件能够被一处理器执行并实现本发明实施例中所述的支持基于神经元块图形编程的方法。
此外，本发明实施例中还提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序能够被一处理器执行并实现本发明实施例中所述的支持基于神经元块图形编程的方法。具体地，可以提供配有存储介质的系统或者装置，在该存储介质上存储着实现上述实施例中任一实施方式的功能的软件程序代码，且使该系统或者装置的计算机(或CPU或MPU)读出并执行存储在存储介质中的程序代码。此外，还可以通过基于程序代码的指令使计算机上操作的操作系统等来完成部分或者全部的实际操作。还可以将从存储介质读出的程序代码写到插入计算机内的扩展板中所设置的存储器中或者写到与计算机相连接的扩展单元中设置的存储器中，随后基于程序代码的指令使安装在扩展板或者扩展单元上的CPU等来执行部分和全部实际操作，从而实现上述实施方式中任一实施方式的功能。用于提供程序代码的存储介质实施方式包括软盘、硬盘、磁光盘、光盘(如CD-ROM、CD-R、CD-RW、DVD-ROM、DVD-RAM、DVD-RW、DVD+RW)、磁带、非易失性存储卡和ROM。可选择地，可以由通信网络从服务器计算机上下载程序 代码。
本发明实施例中主要是以工业领域标准IEC61131-3标准为例进行说明的，实际应用中，也可根据需要采用其它的工业领域标准，只要是方便本工业领域内的用户实现无障碍的AI编程即可。
基于本发明实施例中的支持基于神经元块图形编程的系统、装置或方法，用户在使用时，可首先配置编码程序的目标系统，包括编程语言、AI框架等。之后，用户可基于图形化建模界面40中呈现的所述神经元块图形和所述编程元素图形进行AI图形化模型搭建，并在AI图形化模型搭建完成后，通过点击代码转换，由系统完成图形化模型到编码程序(即源代码)的转换。之后，用户可基于完成的AI编码程序利用强大的PC机、集群、云等培训资源进行AI模型训练。
从上述方案中可以看出，由于本发明实施例中针对AI中的各功能元素设置了各个神经元块，并为每个神经元块完成了程序代码、描述文件和图形化表示三种实现，特别是使得其图形化表示符合一设定的工业领域标准如IEC61131-3标准，从而为用户提供了一种遵循IEC61131-3标准的PLC开发软件相同的编程模式，进而使得习惯于PLC等工业领域编码的用户能够无障碍实现对AI应用的模型搭建，之后系统基于该模型搭建进行代码转换，生成符合目标系统运行环境的编码程序，从而完成了无障碍的AI编程。
此外，通过为每个神经元块构建不同的编程语言和不同AI框架的程序代码，使得最终的编码程序可以转换为满足目标系统的编程语言和AI框架的编码程序，提高了编程的灵活性和通用性。
进一步地，通过将用户的已有编码程序转换为对应的图形化模块，可以实现对用户已有编码程序的兼容。
以上所述仅为本发明的较佳实施例而已，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (15)

  1. 支持基于神经元块图形编程的系统，其特征在于，包括：
    神经元块代码库(10)，存储对应人工智能中各功能元素的各神经元块的程序代码和对应的描述文件；
    编程元素库(20)，存储用于实现各种判断逻辑和控制逻辑的各编程元素的程序代码和对应的描述文件；
    图形化管理模块(30)，用于从所述神经元块代码库中提取各神经元块的描述文件，根据所述各神经元块的描述文件生成对应的神经元块图形，从所述编程元素库中提取各编程元素的描述文件，根据所述各编程元素的描述文件生成对应的编程元素图形；所述神经元块图形和所述编程元素图形均符合一设定的工业领域标准；
    图形化建模界面(40)，用于呈现所述图形化管理模块(30)生成的所述神经元块图形和所述编程元素图形，接收用户基于所述工业领域标准选用对应的神经元块图形和/或编程元素图形建立的人工智能图形化模型；
    代码转换器(50)，用于根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码，并根据所选用的神经元块图形和/或编程元素图形中各相邻图形之间的连接关系和接口数据，生成完整的编码程序。
  2. 根据权利要求1所述的支持基于神经元块图形编程的系统，其特征在于，所述工业领域标准为IEC61131-3标准。
  3. 根据权利要求1所述的支持基于神经元块图形编程的系统，其特征在于，
    所述各神经元块的程序代码包括：针对每个神经元块的对应不同编程语言和不同人工智能框架的复数个程序代码；
    所述各编程元素的程序代码包括：针对每个编程元素的对应不同编程语言和不同人工智能框架的复数个程序代码；
    所述代码转换器(50)根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，以及获取的用户配置的目标系统所对应的目标编程语言和目标人工智能框架，从所述神经元块代码库和所述编程元素库中提取对应的程序代码。
  4. 根据权利要求3所述的支持基于神经元块图形编程的系统，其特征在于，所述不同编程语言和不同人工智能框架包括：Python语言的TensorFlow框架和PyTorch框架，以及C++语言的Caffe框架。
  5. 根据权利要求1所述的支持基于神经元块图形编程的系统，其特征在于，所述神 经元块包括：二维卷积函数块、二维最大池块、由二维到一维的平整化块、由多到少的整合转换块中的一个或多个。
  6. 根据权利要求1所述的支持基于神经元块图形编程的系统，其特征在于，所述编程元素包括：用于进行条件判断逻辑的编程元素、用于进行循环控制逻辑的编程元素、用于进行与或非控制逻辑的编程元素和用于使能和非使能控制逻辑的编程元素中的一个或多个。
  7. 根据权利要求1所述的支持基于神经元块图形编程的系统，其特征在于，所述各神经元块的程序代码中所采用的数据类型包括：
    符合所述工业领域标准的基本数据类型；和
    对应人工智能框架的包括元组、张量和空在内的新增数据类型。
  8. 根据权利要求1至7中任一项所述的支持基于神经元块图形编程的系统，其特征在于，所述代码转换器(50)进一步用于接收用户输入的人工智能模型编码程序，对所述编码程序逐行进行解析，获取能够代表神经元块或编程元素的关键词，根据所述关键词确定对应的神经元块或编程元素，根据相邻神经元块和/或编程元素之间的连接关系以及接口数据，确定对应的神经元块图形或编程元素图形及其接口数据和相邻图形之间的连接关系，生成对应的人工智能图形化模型，将所述人工智能图形化模型提供给图形化建模界面(40)进行显示。
  9. 支持基于神经元块图形编程的方法，其特征在于，包括：
    预先在一神经元块代码库中存储对应人工智能中各功能元素的各神经元块的程序代码和对应的描述文件，在一编程元素库中存储用于实现各种判断逻辑和控制逻辑的各编程元素的程序代码和对应的描述文件(S82)；
    系统启动时，从所述神经元块代码库中提取各神经元块的描述文件，根据所述各神经元块的描述文件生成对应的神经元块图形，从所述编程元素库中提取各编程元素的描述文件，根据所述各编程元素的描述文件生成对应的编程元素图形(S84)；所述神经元块图形和所述编程元素图形均符合一设定的工业领域标准；
    在一图形化建模界面中呈现所述神经元块图形和所述编程元素图形，并接收用户基于所述工业领域标准选用对应的神经元块图形和/或编程元素图形建立的人工智能图形化模型(S86)；
    根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码，并根据所选用的神经元块图形和/或编程元素图形中各相邻图形之间的连接关系和接口数据，生成完整的编码程序 (S88)。
  10. 根据权利要求9所述的支持基于神经元块图形编程的方法，其特征在于，
    所述各神经元块的程序代码包括：针对每个神经元块的对应不同编程语言和不同人工智能框架的复数个程序代码；
    所述各编程元素的程序代码包括：针对每个编程元素的对应不同编程语言和不同人工智能框架的复数个程序代码；
    所述根据人工智能图形化模型中选用的神经元块图形和/或编程元素图形，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码为：根据所述人工智能图形化模型中选用的神经元块图形和/或编程元素图形，以及获取的用户配置的目标系统所对应的目标编程语言和目标人工智能框架，从所述神经元块代码库和/或所述编程元素库中提取对应的程序代码。
  11. 根据权利要求9所述的支持基于神经元块图形编程的方法，其特征在于，所述编程元素包括：用于进行条件判断逻辑的编程元素、用于进行循环控制逻辑的编程元素、用于进行与或非控制逻辑的编程元素和用于使能和非使能控制逻辑的编程元素中的一个或多个。
  12. 根据权利要求9所述的支持基于神经元块图形编程的方法，其特征在于，所述各神经元块的程序代码中所采用的数据类型包括：
    符合所述工业领域标准的基本数据类型；和
    对应人工智能框架的包括元组、张量和空在内的新增数据类型。
  13. 根据权利要求9至12中任一项所述的支持基于神经元块图形编程的方法，其特征在于，进一步包括：
    接收用户输入的人工智能模型编码程序；
    对所述编码程序逐行进行解析；
    获取能够代表神经元块或编程元素的关键词；
    根据所述关键词确定对应的神经元块或编程元素；
    根据相邻神经元块和/或编程元素之间的连接关系以及接口数据，确定对应的神经元块图形或编程元素图形及其接口数据和相邻图形之间的连接关系，并生成对应的人工智能图形化模型；
    将所述人工智能图形化模型通过所述图形化建模界面进行显示。
  14. 支持基于神经元块图形编程的系统，其特征在于，包括：至少一个存储器(91)、至少一个处理器(92)、至少一个数据库(93)和至少一个显示器(94)，其中：
    所述至少一个存储器(91)用于存储计算机程序；
    所述至少一个数据库(93)用于提供一神经元块代码库和一编程元素库；所述神经元块代码库用于存储对应人工智能中各功能元素的各神经元块的程序代码和对应的描述文件；所述编程元素库用于存储用于实现各种判断逻辑和控制逻辑的各编程元素的程序代码和对应的描述文件；
    所述显示器(94)用于提供一图形化建模界面；
    所述至少一个处理器(92)用于调用所述至少一个存储器(91)中存储的计算机程序，执行如权利要求9至13中任一项所述的支持基于神经元块图形编程的方法。
  15. 计算机可读存储介质，其上存储有计算机程序；其特征在于，所述计算机程序能够被一处理器执行并实现如权利要求9至13中任一项所述的支持基于神经元块图形编程的方法。
PCT/CN2019/102903 2019-08-27 2019-08-27 支持基于神经元块图形编程的系统、方法及存储介质 WO2021035552A1 (zh)

Priority Applications (4)

Application Number Priority Date Filing Date Title
US17/637,541 US20220283787A1 (en) 2019-08-27 2019-08-27 System and method supporting graphical programming based on neuron blocks, and storage medium
EP19943717.9A EP4006662A4 (en) 2019-08-27 2019-08-27 SYSTEM AND METHODS TO SUPPORT GRAPHIC PROGRAMMING BASED ON NEURON BLOCKS AND STORAGE MEDIA
PCT/CN2019/102903 WO2021035552A1 (zh) 2019-08-27 2019-08-27 支持基于神经元块图形编程的系统、方法及存储介质
CN201980097442.3A CN113966494A (zh) 2019-08-27 2019-08-27 支持基于神经元块图形编程的系统、方法及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/102903 WO2021035552A1 (zh) 2019-08-27 2019-08-27 支持基于神经元块图形编程的系统、方法及存储介质

Publications (1)

Publication Number Publication Date
WO2021035552A1 true WO2021035552A1 (zh) 2021-03-04

Family

ID=74683306

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/102903 WO2021035552A1 (zh) 2019-08-27 2019-08-27 支持基于神经元块图形编程的系统、方法及存储介质

Country Status (4)

Country Link
US (1) US20220283787A1 (zh)
EP (1) EP4006662A4 (zh)
CN (1) CN113966494A (zh)
WO (1) WO2021035552A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023284312A1 (zh) * 2021-07-15 2023-01-19 上海商汤智能科技有限公司 工作流程的构建方法及装置、设备、计算机存储介质及计算机程序产品

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115454414B (zh) * 2022-11-10 2023-03-24 凯云联创（北京）科技有限公司 一种代码处理方法、装置及计算机可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020007483A1 (en) * 1997-01-29 2002-01-17 Lopez Luis R. Interactive flow visualization, graphical editing and analysis of textual languages
CN1945526A (zh) * 2005-10-09 2007-04-11 光宝科技股份有限公司 应用程序的图形用户界面呈现方法及系统
CN105139071A (zh) * 2015-07-27 2015-12-09 清华大学 一种以现场可编程门阵列的逻辑片为基本单元模拟生物神经元网络的方法
CN108345265A (zh) * 2018-01-24 2018-07-31 珠海优特电力科技股份有限公司 硬件设备的编程方法、可视化编程平台、存储器和处理器
CN109978000A (zh) * 2019-02-19 2019-07-05 深圳点猫科技有限公司 一种基于图形化编程工具的分类ai实现方法及电子设备

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3441309B2 (ja) * 1996-09-13 2003-09-02 株式会社山武 図的プログラミングにおける機能オブジェクトの表示方法
US6282699B1 (en) * 1999-02-23 2001-08-28 National Instruments Corporation Code node for a graphical programming system which invokes execution of textual code
US8347264B2 (en) * 2008-09-30 2013-01-01 Ics Triplex Isagraf Inc. Method and system for an automation collaborative framework
US8255875B2 (en) * 2008-09-30 2012-08-28 Rockwell Automation Technologies, Inc. Application builder for industrial automation
US8229578B2 (en) * 2009-09-21 2012-07-24 Fisher-Rosemount Systems, Inc. Methods and apparatus to manage module run sequences in a process control environment
US8983632B2 (en) * 2011-03-29 2015-03-17 Honeywell International Inc. Function block execution framework
US8689184B2 (en) * 2011-03-30 2014-04-01 The Procter & Gamble Company Apparatus, system, and method for managing industrial software configurations
US9098164B2 (en) * 2012-08-03 2015-08-04 National Instruments Corporation Physics based diagram editor
EP2827240A1 (en) * 2013-07-17 2015-01-21 ABB Technology AG Method for generating control-code by a control-code-diagram
EP3416105A4 (en) * 2016-02-12 2019-02-20 Sony Corporation INFORMATION PROCESSING METHOD AND INFORMATION PROCESSING DEVICE
US10048826B2 (en) * 2016-10-04 2018-08-14 Sas Institute Inc. Interactive visualizations of a convolutional neural network
CN106959851A (zh) * 2017-03-03 2017-07-18 同济大学 一种面向人工智能研究的模块化可编程分布式交互系统
CN111886602A (zh) * 2018-03-28 2020-11-03 索尼公司 信息处理方法、信息处理设备和程序
JP7015207B2 (ja) * 2018-04-27 2022-02-02 株式会社日立製作所 ビジュアルプログラミングツールを用いてフローを作成することを支援する装置および方法
US20200050935A1 (en) * 2018-08-10 2020-02-13 Nvidia Corporation Deep learning model execution using tagged data
US10896034B2 (en) * 2018-11-14 2021-01-19 Babu Vinod Methods and systems for automated screen display generation and configuration
US11314488B2 (en) * 2018-11-14 2022-04-26 Babu Vinod Methods and systems for automated screen display generation and configuration
US20200184261A1 (en) * 2018-12-05 2020-06-11 International Business Machines Corporation Collaborative deep learning model authoring tool
US20200285449A1 (en) * 2019-03-06 2020-09-10 Veritone, Inc. Visual programming environment
CN110007922B (zh) * 2019-03-21 2022-10-21 深圳点猫科技有限公司 基于人工智能的图形化源代码的编译方法、装置和设备
US10831456B1 (en) * 2019-05-31 2020-11-10 The Mathworks, Inc. External code integrations within a computing environment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020007483A1 (en) * 1997-01-29 2002-01-17 Lopez Luis R. Interactive flow visualization, graphical editing and analysis of textual languages
CN1945526A (zh) * 2005-10-09 2007-04-11 光宝科技股份有限公司 应用程序的图形用户界面呈现方法及系统
CN105139071A (zh) * 2015-07-27 2015-12-09 清华大学 一种以现场可编程门阵列的逻辑片为基本单元模拟生物神经元网络的方法
CN108345265A (zh) * 2018-01-24 2018-07-31 珠海优特电力科技股份有限公司 硬件设备的编程方法、可视化编程平台、存储器和处理器
CN109978000A (zh) * 2019-02-19 2019-07-05 深圳点猫科技有限公司 一种基于图形化编程工具的分类ai实现方法及电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4006662A4 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023284312A1 (zh) * 2021-07-15 2023-01-19 上海商汤智能科技有限公司 工作流程的构建方法及装置、设备、计算机存储介质及计算机程序产品

Also Published As

Publication number Publication date
CN113966494A (zh) 2022-01-21
US20220283787A1 (en) 2022-09-08
EP4006662A4 (en) 2023-04-12
EP4006662A1 (en) 2022-06-01

Similar Documents

Publication Publication Date Title
Bressert SciPy and NumPy: an overview for developers
JP6563381B2 (ja) 再計算ユーザインタフェースにおける変換の編さん
EP3745256A1 (en) External code integrations within a computing environment
Ono et al. CyREST: turbocharging Cytoscape access for external tools via a RESTful API
US8810595B2 (en) Declarative approach for visualization
US8527943B1 (en) System and method of application development
KR20150143473A (ko) 재계산 사용자 인터페이스 내의 신호 캡처 컨트롤
US10114619B2 (en) Integrated development environment with multiple editors
US10755001B2 (en) Port management for graphical modeling
KR20150143472A (ko) 컨트롤 이벤트의 생성 기법
US10810018B2 (en) Device with extensibility
Bernaschina et al. Formal semantics of OMG’s Interaction Flow Modeling Language (IFML) for mobile and rich-client application model driven development
US20160140261A1 (en) Lean product modeling systems and methods
WO2021035552A1 (zh) 支持基于神经元块图形编程的系统、方法及存储介质
Westra Building mapping applications with QGIS
US20230011461A1 (en) Method and system for generating engineering diagrams in an engineering system
Van Tendeloo et al. Concrete syntax: A multi-paradigm modelling approach
Ledur et al. A high-level dsl for geospatial visualizations with multi-core parallelism support
US10846061B2 (en) Development environment for real-time application development
US20230030039A1 (en) Modular Software Architecture for Machine Learning Pipelines
Design et al. MIT Architecture
US20140007045A1 (en) Systems and methods for enabling customization of visual elements based on a specified class
WO2023036400A1 (en) Managing an app, especially developing an app comprising a trained function, method and system
US11501062B1 (en) Systems and methods for displaying data including hierarchical headers
de Lange Basic Concepts of Computer Science

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19943717

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019943717

Country of ref document: EP

Effective date: 20220224