CN111267097B - Industrial robot auxiliary programming method based on natural language - Google Patents

Industrial robot auxiliary programming method based on natural language Download PDF

Info

Publication number
CN111267097B
CN111267097B CN202010066644.6A CN202010066644A CN111267097B CN 111267097 B CN111267097 B CN 111267097B CN 202010066644 A CN202010066644 A CN 202010066644A CN 111267097 B CN111267097 B CN 111267097B
Authority
CN
China
Prior art keywords
attention
instruction
language
environment
programming
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010066644.6A
Other languages
Chinese (zh)
Other versions
CN111267097A (en
Inventor
胡海洋
刘翰文
陈洁
李忠金
黄彬彬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN202010066644.6A priority Critical patent/CN111267097B/en
Publication of CN111267097A publication Critical patent/CN111267097A/en
Application granted granted Critical
Publication of CN111267097B publication Critical patent/CN111267097B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1658—Programme controls characterised by programming, planning systems for manipulators characterised by programming language

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Numerical Control (AREA)

Abstract

The invention provides an industrial robot auxiliary programming method based on natural language, which generates corresponding robot execution codes according to language instructions and environment images. The invention is divided into three parts: 1) the language instruction and the characteristics of the factory environment are extracted by using a bidirectional recurrent neural network (Bi-RNN) with long-short time memory (LSTM) and a fast regional convolutional neural network (F-RCNN) respectively. 2) An alignment algorithm of a 'multi-attention machine' model and machine translation is provided to correctly match an object in an environment with an instruction, so as to identify a specified object and output a coordinate point for placing the object. 3) And generating robot codes of the operation by using the result output by the model and matching with a CoBlox modular programming mode. The multi-attention machine model adopted by the invention improves the identification precision and solves the problem that the object cannot be accurately identified in the industrial environment by the current method. The modularized programming technical scheme simplifies the programming complexity of engineers and effectively improves the development efficiency.

Description

Industrial robot auxiliary programming method based on natural language
Technical Field
The application belongs to the technical field of robot programming, and particularly relates to a robot programming technology based on natural language and machine vision.
Background
With the rapid development of robot technology in recent decades, the concept of intelligent manufacturing is deeply thought. The mechanical arm technology is widely applied to industrial production environments, the cooperative robot integrates the advantages of human beings and mechanical equipment, and the production efficiency can be remarkably improved by the close cooperation of the cooperative robot and workers on a production line.
All current mechanical tasks are carefully designed and coded by engineers to assist and replace workers in performing a single mechanical task. Engineers often write robot code in an online or offline programming manner, which is time consuming and time consuming, and is far from meeting the change of product requirements. For example, eight months are needed for writing a large-scale vehicle body arc welding robot program, and a half-month debugging program is needed for changing each welding node, so that the high programming overhead forces small and medium-sized enterprises to be unable to benefit from intelligent manufacturing.
In recent years, researchers have been exploring the field of robot programming. The manual programming is a robot programming tool which has the widest application range and the highest use frequency in the market. Official programming interfaces are commonly used with programming languages with early programming language features such as ABB RAPID, KUKA KRL, etc. In existing production environments, however, engineers are required to spend a significant amount of time writing code in the platform for each manufacturing task, even if it contains a large amount of duplicate or similar code. Moreover, the encoding mode strictly follows the fixed language specification, which is not beneficial to the quick learning and use of novices.
CoBlox modular programming (David Weintrop et al, Block-based programming for induced roots, Published in 2017 IEEE Blocks and Beyond works)
In addition, with the development of natural language and artificial intelligence in linguistics, scholars have gained popular progress in the field of automatic programming, such as a robot analyzing human language or actions through a neural network so as to correctly understand human instructions and perform tasks. But such methods only output the behavior and state of the robot and do not provide the source code needed by the industrial engineer. This programming approach does not allow for off-line code level modification when a recipe adjustment is required in an industrial setting. And because the code text is not generated, the reutilization of similar codes in other projects is not facilitated.
In this situation, the existing programming techniques cannot meet the needs of industrial smart manufacturing due to their inherent drawbacks.
Disclosure of Invention
To overcome the above-mentioned deficiencies of the prior art, it is desirable to provide a fast programming method supporting abstract input to meet the current requirements of industrial smart manufacturing. The invention provides an industrial robot auxiliary programming method based on natural language, which generates corresponding robot execution codes according to language instructions and environment images. It is mainly divided into three parts: 1) the language instruction and the characteristics of the factory environment are extracted by using a bidirectional recurrent neural network (Bi-RNN) with long-short time memory (LSTM) and a fast regional convolutional neural network (F-RCNN) respectively. 2) The invention provides a multi-attention machine model and an alignment algorithm of machine translation, which are used for correctly matching an object in an environment with an instruction, so that a specified object is identified and a coordinate point for placing the object is output. 3) And generating robot codes of the operation by using the result output by the model and matching with a CoBlox modular programming mode.
An industrial robot auxiliary programming method based on natural language comprises the following steps:
and (1) preprocessing input data. The input data are a language instruction and an environment image, the characteristics of the language instruction are extracted by using a Bi-RNN with an LSTM, and the environment image is processed by an F-RCNN to obtain a target candidate region.
And (2) analyzing an object determined by the language instruction in the environment, namely a target object, by adopting an alignment algorithm of machine translation. The alignment algorithm is accomplished through a multi-attention machine model that includes word-object attention, command-object attention, and object-object attention mechanisms.
And (3) training a multi-attention mechanism model, and identifying the position of the target object in the environment and the reference characteristics of the object placement point.
And (4) predicting the position of the reference object through a multi-attention machine mechanism, predicting the position where the target object is to be placed by using a Monte Carlo algorithm (MCMC) in combination with language instruction characteristics, and outputting coordinates.
And (5) constructing a programmable logic controller constraint database (PLC constraint library) and CoBlox modular programming.
And (6) analyzing the language instruction to enable the analysis result to be matched with the PLC constraint library and the modularized programming code, and generating a final robot auxiliary code by combining the coordinate output in the step (4).
The invention has the following beneficial effects:
compared with the prior art, the invention solves the problems of low visual identification rate of the existing machine and repeated programming of engineers. The industrial robot programming method based on natural language provided by the invention mainly has the following innovation points: 1) using natural language to assist in the generation of industrial robot code; 2) a multi-step attention mechanism model is used for improving the machine vision precision; 3) and the robot code is obtained by adopting a CoBlox modular programming mode, similar codes do not need to be repeatedly written, and the development burden of developers is simplified.
The invention can generate the source code without strictly following the language specification but the natural language of human, thereby improving the abstraction, reducing the programming threshold of the robot and providing a good programming method for novice. The multi-attention machine model adopted by the invention improves the identification precision and solves the problem that the object cannot be accurately identified in the industrial environment by the current method. The modularized programming technical scheme simplifies the programming complexity of engineers and effectively improves the development efficiency.
Drawings
FIG. 1 is a schematic diagram of the overall model structure of the present invention;
fig. 2 is a PLC signal table of the ABB palletizing robot;
FIG. 3 is a flow chart illustrating task 3 execution of the present invention;
FIG. 4 is a task flow diagram of the present invention.
Detailed Description
The invention comprises three sub-tasks, which are further described below with reference to the figures and examples.
FIG. 1 is a schematic diagram of the overall model structure of the present invention.
The method is divided into three connected subtasks, and comprises the following specific steps:
task 1. identification of target object
And (1) preprocessing the input language instruction and the environment image. The preprocessing comprises the steps of extracting language features of language instructions by using Bi-RNN with LSTM and preprocessing an environment image by using F-RCNN, thereby obtaining target candidate region features. The method comprises the following specific steps:
1.1 instruction encoding: instruction I consisting of I wordsi＝{x1，x2，x３，…，xiInput RNN network. Through Bi-RNN language instruction with LSTMCoding is carried out, and a hidden state sequence I is generated recursivelyiThen through the learning function psixxiThe instructions are mapped to fixed dimensions.
Ii＝Bi_LSTM(ψx(xi)，Ii-1) (1)
Figure GDA0002453181390000041
Figure GDA0002453181390000042
Represents the mean value of the instructions, IiIs an instruction
Figure GDA0002453181390000043
The word embedded representation.
1.2, environment coding, namely preprocessing an environment image by using F-RCNN to obtain the characteristics of a full-connection layer acquired image candidate region:
Figure GDA0002453181390000044
Figure GDA0002453181390000045
a d-dimensional visual embedded representation representing the first m detection boxes.
Figure GDA0002453181390000046
Is a spatial and behavioral characterization of each object, WBAnd a are the weight and bias value parameters of the object, respectively.
Mapping V and I to the same dimension through one fully connected layer and one-dimensional convolutional layer, respectively:
V＝relu(CONV1d(Vm)) (4)
I＝relu(LINEAR(Ii)) (5)
wherein V ∈ Rm×dIs m objects V ═ V1，V2，…，VmThe set of (c). I is as large as RdIs an instructionThe characteristics of (1).
And (2) identifying the object specified by the language instruction. How to correctly match the instructions with the environmental objects and how to select certain objects from similar environmental objects is critical to improve the accuracy of the recognition. The invention adopts an alignment matching algorithm to solve the problem of matching the language and the object, and additionally uses a multi-attention mechanism to improve the machine vision precision. The specific process is as follows:
a new multi-attention mechanism processing process is proposed: the three attention associations are Word-Object, Object-Instruction and Object-Object. The multi-attention mechanism is used to measure the matching probability of the language instruction and the object in each environment, namely, the possibility of predicting each object in the environment. Attention-passing module
Figure GDA0002453181390000051
Matching the object with the language instruction, then normalizing by using a Softmax function to obtain a conditional distribution P
Figure GDA0002453181390000052
And instructing the determined target object distribution probability.
Figure GDA0002453181390000053
Figure GDA0002453181390000054
Is a discrete representation of the source target object.
Predicting the loss function of a source target object as a conditional distribution probability
Figure GDA0002453181390000055
And the cross entropy of the real position G (E) of the object in the environment, the present invention uses an Adam optimizer to tune the loss function.
Figure GDA0002453181390000056
The multi-step attention mechanism flow is as follows:
2.1 Object-Object: firstly, calculating a difference value of the image candidate region features extracted by the F-RCNN in the step (1) to generate an O-O relation attention mechanism matrix:
AW p＝Wf×p(Vi-Vj) (8)
Vi，Vjas defined above, (V)i-Vj) Is a matrix of m x m and represents the difference of the image characteristic representation of the ith target object and the jth object. WfIs a trained attention matrix representing the relational attention matrix after n executions.
2.2Word-Object:
Computing each word x in a language instruction using an alignment algorithmiHidden unit output h for each temporal statetThereby representing xiMatching score with each object m in the environment score:
Figure GDA0002453181390000057
global vector
Figure GDA0002453181390000058
Is a target object bmSum of weights of (a):
Figure GDA0002453181390000061
2.3 Object-analysis: relating all target objects to O-O attention matrix AW pMultiplication of vectors in global natural language
Figure GDA0002453181390000062
Under the guidance of (3), computing the global vector and the embedded feature matrix of the target object.
Figure GDA0002453181390000063
And (3): and training a multi-attention machine model, normalizing the result to obtain the probability distribution of the target object, and determining the position (Source) of the target object.
Task 2: predicting location of target placement
And (4) predicting the position of the reference object by using the multi-attention mechanism model again and combining language instruction characteristics. The position where the target object is to be placed is predicted by a monte carlo algorithm (MCMC), and coordinates are output.
The prediction process is decomposed into two sub-processes: reference identification (Reference) and Offset (Offset) are carried out by the following specific processes:
4.1reference (R) the method of predicting the position of the reference is the same as the method of predicting the position of the target object in the step (3), so the probability of the position of the reference is calculated using the same method in the step (3):
Figure GDA0002453181390000064
bmfor m objects in the environment,
Figure GDA0002453181390000065
for attention moment array in current environment
And 4.2offset (O), modeling the offset O (the difference value between the real target position and the predicted position) according to the language instruction characteristics, assuming that the language instruction characteristics obey Gaussian distribution, and fitting the instruction by adopting multidimensional Gaussian distribution with fixed covariance so as to predict the offset O.
P(O＝o|I)∝N(μo，∑o) (13)
μo＝W1σ(W2hfc6+b1)+b2 (14)
hfchIs the penultimate fully connected layer, μ, of F-RCNNoIs the center of the gaussian distribution (object coordinates (x, y, z)) generated by the fully connected layers and the command features. b1，b2Is a bias parameter, W1，W2Respectively, a weight matrix for the instructions and the objects.
4.3 predicting target position: and defining the target position as T ═ Offset + Reference, and determining the coordinates of the object placement point by adopting a Monte Carlo sampling (MCMC) method.
Distributed sampling of references and offsets, using a set of samples
Figure GDA0002453181390000071
Of (2) a
Figure GDA0002453181390000072
Representing a set of sampled samples, tnIs formed by
Figure GDA0002453181390000073
The predicted location.
Figure GDA0002453181390000074
Figure GDA0002453181390000075
Figure GDA0002453181390000076
Will tGi-tnThe distance between the real position and the predicted position is used as a negative reward, a Reinforcement Learning idea is adopted, and N random variables are fitted through a Monte Carlo method
Figure GDA0002453181390000077
Sample sequence samples of (1). The specific method comprises the following steps:
Figure GDA0002453181390000078
the placement point of the object specified by the confirmation instruction in the environment is fitted by the monte carlo method, and the placement point coordinates (x, y, z) are output.
Task 3. generating robot code
And (5) constructing a programmable controller constraint database (PLC constraint library) and CoBlox modular programming.
5.1PLC constraint library: in an actual production environment, the robot task is constrained by the PLC signals. And selecting proper PLC constraint as a PLC constraint library according to actual conditions. A PLC is a programmable controller that can interact with a robot and constrain the robot behavior, written and defined by a PLC engineer. At present, each manufacturer of mechanical arms does not have uniform PLC signal regulation, and the problem that the same signal of a machine has various expressions in a programming platform exists. Aiming at the problem that the regulations of various manufacturers are not uniform at present, the embodiment adopts a PLC signal table of an ABB palletizing robot (see part of the attached figure 2).
5.2CoBlox Modular Programming: and packaging the function body with default parameters by adopting a CoBlox modular programming mode. For example, Move < speed > to < somewhere > indicates that the robot arm is commanded to Move to a fixed position, the neural network predicts the target and placement position (< somewhere >) that the robot arm grabs in place of the original manually entered position parameters, < speed > will assign default parameters and allow the programmer to manually modify; SET < what > is then mapped to the corresponding signal by the segmentation tool intercepting the PLC field.
And (6) analyzing the language instruction by adopting a StanfordNLP tool, matching the participle and the PLC signal in a PLC constraint library, and then combining and matching CoBlox modular programming by adopting a BM25 algorithm to automatically generate a robot program framework. And finally, filling the target position coordinates, namely the coordinates of the placing points, which are predicted in the step (4) into a robot program to generate a complete robot code. (the task flow is shown in the attached figure 3)
The invention is controlled by engineers, allows voice and text input of robot task instructions, and outputs task execution codes directly at the terminal, with intermediate processes invisible. The code generation method not only supports a flexible input mode, but also can restrict the programming behavior of an engineer and improve the programming normative by adopting a modular programming method while simplifying the programming task. (see FIG. 4 for task flow).

Claims (7)

1. An industrial robot auxiliary programming method based on natural language is characterized by comprising the following steps:
step (1), input data is preprocessed; the input data are a language instruction and an environment image, the characteristics of the language instruction are extracted by using a Bi-RNN with an LSTM, and the environment image is processed by an F-RCNN to obtain a target candidate region;
step (2), resolving an object determined by a language instruction in an environment, namely a target object, by adopting an alignment algorithm of machine translation; the alignment algorithm is completed through a multi-attention machine model, wherein the multi-attention machine model comprises a word-object attention mechanism, an instruction-object attention mechanism and an object-object attention mechanism;
step (3), training a multi-attention mechanism model, and identifying the position of a target object in the environment and reference characteristics of an object placement point;
predicting the position of a reference object through a multi-attention machine system, predicting the position where a target object is to be placed by using a Monte Carlo algorithm (MCMC) in combination with language instruction characteristics, and outputting coordinates;
step (5), constructing a programmable logic controller constraint database (PLC constraint library) and CoBlox modular programming;
and (6) analyzing the language instruction to enable the analysis result to be matched with the PLC constraint library and the modularized programming code, and generating a final robot auxiliary code by combining the coordinate output in the step (4).
2. The natural language based industrial robot aided programming method according to claim 1, wherein the steps of (1) preprocessing the inputted language instruction and the environment image; the preprocessing comprises the steps of extracting language features of language instructions by using a Bi-RNN with LSTM and preprocessing an environment image by using F-RCNN so as to obtain target candidate region features; the method comprises the following specific steps:
1.1, instruction encoding: instruction I consisting of I wordsi＝{x1，x2，x3，...，xiInputting RNN network; by means of Bi-RNN pairs with LSTMEncoding language instructions and recursively generating a sequence of hidden states IiThen through the learning function psix(xi) Mapping the instructions to fixed dimensions;
Ii＝Bi_LSTM(ψx(xi)，Ii-1) (1)
Figure FDA0002896006520000021
Figure FDA0002896006520000022
represents the mean value of the instructions, IiIs an instruction
Figure FDA0002896006520000023
The word embedding representation of (a);
1.2, environment coding: preprocessing an environment image by using F-RCNN to obtain the characteristics of a full-connection layer acquired image candidate region:
Figure FDA0002896006520000024
Figure FDA0002896006520000025
a d-dimensional visually embedded representation representing the first m detection boxes; f. ofbmIs a spatial and behavioral characterization of each object, WBAnd a are the weight and deviation value parameters of the object, respectively;
mapping V and I to the same dimension through one fully connected layer and one-dimensional convolutional layer, respectively:
V＝relu(CONV1d(Vm)) (4)
I＝relu(LINEAR(Ii)) (5)
wherein V ∈ Rn×dIs m objects V ═ V1，V2，…，VmA set of { fraction }; i is as large as RdIs a characteristic of the instruction.
3. The natural language based industrial robot aided programming method according to claim 2, wherein the step (2) of recognizing an object specified by a language instruction; the problem of matching language and objects is solved by adopting an alignment matching algorithm, and in addition, the machine vision precision is improved by using a multi-attention mechanism; the specific process is as follows:
a new multi-attention mechanism processing process is proposed: combining three attention of Word-Object attention, Instruction-Object attention and Object-Object attention, namely Word-Object attention, Object-Instruction and Object-Object attention; measuring the matching probability of the language instruction and the object in each environment by using a multi-attention machine mechanism, namely predicting the possibility of each object in the environment; attention-passing module
Figure FDA0002896006520000026
Matching the object with the language instruction, then normalizing by using a Softmax function to obtain a conditional distribution P
Figure FDA0002896006520000027
Instructing the determined target object distribution probability;
Figure FDA0002896006520000028
Figure FDA0002896006520000031
is a discrete representation of the source target object;
predicting the loss function of a source target object as a conditional distribution probability
Figure FDA0002896006520000032
And the cross entropy of the real position G (E) of the object in the environment, the invention uses an Adam optimizer to carry out tuning on the loss function;
Figure FDA0002896006520000033
the multi-step attention mechanism flow is as follows:
2.1, Object-Object: firstly, calculating a difference value of the image candidate region features extracted by the F-RCNN in the step (1) to generate an O-O relation attention mechanism matrix:
AW p＝Wf×p(Vi-Vj) (8)
(Vi-Vj) The matrix is m multiplied by m and represents the difference of the image characteristic representation of the ith target object and the jth object; wfIs a trained attention matrix, representing a relational attention matrix after n times of execution;
2.2、Word-Object：
computing each word x in a language instruction using an alignment algorithmiHidden unit output h for each temporal statetThereby representing xiMatching score with each object m in the environment score:
Figure FDA0002896006520000034
global vector
Figure FDA0002896006520000035
Is a target object bmSum of weights of (a):
Figure FDA0002896006520000036
2.3, Object-instruction: relating all target objects to O-O attention matrix AW pMultiplication of vectors in global natural language
Figure FDA0002896006520000037
Under the guidance of (3), calculating an embedded characteristic matrix of the global vector and the target object;
Figure FDA0002896006520000038
4. a natural language based industrial robot aided programming method according to claim 3, wherein the step (3): and training a multi-attention machine model, normalizing the result to obtain the probability distribution of the target object, and determining the position (Source) of the target object.
5. The natural language based industrial robot aided programming method according to claim 4, characterized in that, in the step (4), the multi-attention mechanism model is used again to predict the position of the reference object and combine the language instruction characteristics; predicting a position where the target object is to be placed through a Monte Carlo algorithm (MCMC), and outputting coordinates;
the prediction process is decomposed into two sub-processes: reference identification (Reference) and Offset (Offset) are carried out by the following specific processes:
4.1reference (R): the method of predicting the position of the reference is the same as the method of predicting the position of the target object in step (3), so the probability of the position of the reference is calculated using the same method in step (3):
Figure FDA0002896006520000041
bmfor m objects in the environment,
Figure FDA0002896006520000042
for attention moment array in current environment
4.2offset (O): modeling an offset O (a difference value between a real target position and a predicted position) according to the language instruction characteristics, assuming that the language instruction characteristics obey Gaussian distribution, and fitting an instruction by adopting multidimensional Gaussian distribution with fixed covariance to predict the offset O;
P(O＝o|I)∝N(μo，∑o) (13)
μo＝W1σ(W2hfc6+b1)+b2 (14)
hfc6is the penultimate fully connected layer, μ, of F-RCNNoIs the center of the gaussian distribution (object coordinates (x, y, z)) generated by the fully connected layers and the command features; b1，b2Is a bias parameter, W1，W2Weight matrices for the instructions and objects, respectively;
4.3 predicting target position: defining the target position as T ═ Offset + Reference, and determining the coordinates of the object placement point by adopting a Monte Carlo sampling (MCMC) method;
distributed sampling of references and offsets, using a set of samples
Figure FDA0002896006520000043
Of (2) a
Figure FDA0002896006520000044
Representing a set of sampled samples, tnIs formed by
Figure FDA0002896006520000045
A predicted location;
Figure FDA0002896006520000046
Figure FDA0002896006520000047
Figure FDA0002896006520000051
will tGT-tnThe distance between the real position and the predicted position is used as a negative reward, a Reinforcement Learning idea is adopted, and N random variables are fitted through a Monte Carlo method
Figure FDA0002896006520000052
The sample sequence samples of (a); the specific method comprises the following steps:
Figure FDA0002896006520000053
6. a natural language based industrial robot aided programming method according to claim 5, characterized in that, the step (5) is to construct a programmable controller constraint database (PLC constraint library) and CoBlox modular programming;
5.1PLC constraint library: in an actual production environment, a robot task is restricted by a PLC signal; selecting a proper PLC constraint as a PLC constraint library according to actual conditions;
5.2CoBlox Modular Programming: and packaging the function body with default parameters by adopting a CoBlox modular programming mode.
7. The industrial robot aided programming method based on natural language as claimed in claim 6, wherein in step (6), language instructions are analyzed by adopting a StanfordLP tool, participles and PLC signals are matched in a PLC constraint library, and then CoBlox modular programming is combined and matched by adopting a BM25 algorithm, so that a robot program frame is automatically generated; and finally, filling the target position coordinates, namely the coordinates of the placing points, which are predicted in the step (4) into a robot program to generate a complete robot code.
CN202010066644.6A 2020-01-20 2020-01-20 Industrial robot auxiliary programming method based on natural language Active CN111267097B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010066644.6A CN111267097B (en) 2020-01-20 2020-01-20 Industrial robot auxiliary programming method based on natural language

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010066644.6A CN111267097B (en) 2020-01-20 2020-01-20 Industrial robot auxiliary programming method based on natural language

Publications (2)

Publication Number Publication Date
CN111267097A CN111267097A (en) 2020-06-12
CN111267097B true CN111267097B (en) 2021-03-02

Family

ID=70994098

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010066644.6A Active CN111267097B (en) 2020-01-20 2020-01-20 Industrial robot auxiliary programming method based on natural language

Country Status (1)

Country Link
CN (1) CN111267097B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111948622A (en) * 2020-08-07 2020-11-17 哈尔滨工程大学 Linear frequency modulation radar signal TOA estimation algorithm based on parallel CNN-LSTM
CN112255962A (en) * 2020-10-30 2021-01-22 浙江佳乐科仪股份有限公司 PLC programming system based on artificial intelligence

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108073392A (en) * 2017-12-29 2018-05-25 上海宽全智能科技有限公司 Intelligence programming method, equipment and storage medium based on natural language
CN108345583A (en) * 2017-12-28 2018-07-31 中国科学院自动化研究所 Event recognition and sorting technique based on multi-lingual attention mechanism and device
CN108447477A (en) * 2018-01-30 2018-08-24 华南理工大学 A kind of robot control method based on natural language understanding

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11231826B2 (en) * 2015-03-08 2022-01-25 Google Llc Annotations in software applications for invoking dialog system functions

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108345583A (en) * 2017-12-28 2018-07-31 中国科学院自动化研究所 Event recognition and sorting technique based on multi-lingual attention mechanism and device
CN108073392A (en) * 2017-12-29 2018-05-25 上海宽全智能科技有限公司 Intelligence programming method, equipment and storage medium based on natural language
CN108447477A (en) * 2018-01-30 2018-08-24 华南理工大学 A kind of robot control method based on natural language understanding

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Blockly Goes to Work:Block-based Programming for Industrial Robots;David Weintrop;《2017 IEEE Blocks and Beyond Workshop》;IEEE;20171201;29-36 *

Also Published As

Publication number Publication date
CN111267097A (en) 2020-06-12

Similar Documents

Publication Publication Date Title
CN107150347B (en) Robot perception and understanding method based on man-machine cooperation
Yang et al. Hidden Markov model approach to skill learning and its application to telerobotics
US6424956B1 (en) Stochastic encoder/decoder/predictor
CN111267097B (en) Industrial robot auxiliary programming method based on natural language
CN105171745A (en) Robot off-line programming system
EP3704550B1 (en) Generation of a control system for a target system
Tahriri et al. Optimizing the robot arm movement time using virtual reality robotic teaching system
Fishwick FUZZY SIMULATION: SPECIFYING AND IDENTIFYING QUALITATIVE MODELS∗
Zhao et al. Model accelerated reinforcement learning for high precision robotic assembly
Xia et al. Virtual comissioning of manufacturing system intelligent control
US20230311309A1 (en) Method and system for determining optimized program parameters for a robot program
Hu et al. Natural language-based automatic programming for industrial robots
Liu et al. Knowledge-guided robot learning on compliance control for robotic assembly task with predictive model
TW202001871A (en) Voice actuated industrial machine control system
JPH06332506A (en) Nonlinear controller
CN114779661B (en) Chemical synthesis robot system based on multi-classification generation confrontation imitation learning algorithm
CN115512214A (en) Indoor visual navigation method based on causal attention
CN115796001A (en) Data missing value prediction method and system based on seq2seq
CN114692615A (en) Small sample semantic graph recognition method for small languages
Kacalak et al. Intelligent assessment of structure correctness using antipatterns
Цзяньцзюнь et al. Software analysis for mobile robots control programs
Nasrallah et al. Elevating Mobile Robotics: Pioneering Applications of Artificial Intelligence and Machine Learning.
Chau et al. SMCGen: Generating reconfigurable design for sequential Monte Carlo applications
CN113298697B (en) Method for converting two-dimensional graphic elements into vector graphic elements based on artificial neural network
KR102665220B1 (en) Apparatus and method for performing predictive simulation based on digital twin

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant